# OS Patch Management

## Overview

This document describes the end-to-end Patch management process, patch installations, and insights on the patching.

**Optimized and Effective Patch Management of OpsRamp**

* **Visibility**\
  &#x20;OpsRamp provides Partner and Customer level visibility across the customer Infrastructure making it easy and simple to manage all patch management activities. Simple clicks running Patch scan, Patch configuration, the configuration of whitelisted for partner’s infrastructure along with scheduling enables global and site level visibility for users.
* **Manageability**\
  &#x20;OpsRamp empowers users to manage the infrastructure from a single central location providing easy enablement, approvals for patching, monitoring, access etc. Custom Jobs can be scheduled for running various automation scripts for 3rd party patches along with standard patches running for Windows and Linux proactively.
* **Customization**\
  &#x20;OpsRamp gives users the ability to create custom configurations minimizing manual intervention. Users can run customized scripts using Run Book Automation feature for Windows and Linux. RBA scripts can be made available at Global or Customer level providing optimization and control.
* **Reports**\
  &#x20;OpsRamp provides detailed reports outlining the patch metrics along with the ability to schedule the report in the preferred format for the recurring/specific time period.

## Scenarios

### Assign missing patch job and schedule the job

**Scenario**: A user wants to assign missing patch job to all client devices and schedule the job at a specific time. Example: Every Wednesday at 10 AM.\
&#x20;**Solution**: See [here](#Create_Job_for_Missing_Patches) for instructions for assigning missing patch and schedule the job.

### Assign and schedule patch installation job

**Scenario**: A user wants to assign and schedule patch installation job to install all the approved patches among the missing patches list for the devices.\
&#x20;**Solution**: See [here](#Installation_of_Patches) for instructions for assigning and scheduling patch installation job.

### Assign application patches and run the schedules

**Scenario**: A user wants to assign application scan job to all client devices and schedule the job at a specific time. Example: Every Wednesday at 10 AM.\
&#x20;**Solution**: See [here](#Installing_Applications) for instructions for assigning patches and run the scheduled patch installation job.

### Update application patches and run the schedules

**Scenario**: A user wants to update application update job to all client devices and schedule the job at a specific time. Example: Every Wednesday at 10 AM.\
&#x20;**Solution**: See [here](#Installing_Applications) for instructions for assigning patches and run the scheduled patch installation job.

## Patch Management – Windows and Linux

### Create Job for Missing Patches

To manage the patches, User needs to create a job for the missing patches:

1. Login to OpsRamp.
2. Click **All Clients** and select the client.
3. On the drop-down menu, click **Setup.**
4. On **Automation** tab, click **Jobs**.\
   &#x20;![](https://docs.opsramp.com/wp-content/uploads/2017/06/New-UI-Automation-400x237.png)
5. Select the client and then click **Create**.
6. Select the **Client**, **Job Type – Missing Patches Request**, and provide a **Job Name**.
7. In Job Schedule section, select **Start Date**, **Recurrence pattern**.
8. Click **+Add Devices** to add the devices.
9. Select **Any New Desktop** to apply to any newly discovered desktop. Or Select **Any New Server** to apply to any newly discovered server and then click **Save**.

### View Missing Patches

#### By Patch

User can view the list of missing patches applicable for the onboarded devices. User can also view the patch statuses – Missing, Approved, Installed and Failed.

To view the list of missing patches

1. On **Automation** tab, click **Patch Management** and then click **By Patch**. List of patches appears.
2. Click **Missing** patch numbers to view the devices for which the patches are recommended.

![](https://docs.opsramp.com/wp-content/uploads/2017/06/View-Patches-By-Patch-800x178.png)

#### By Device

User can view the list of devices which has missing patches.

To view the list of devices with missing patches

1. On **Automation** tab, click **Patch Management** and then click **By Device**. List of devices appears.
2. Click **Missing** patch numbers to view the patches recommended for the selected device.

![](https://docs.opsramp.com/wp-content/uploads/2017/06/View-Patches-by-device-1-800x214.png)

### Patch Scan

Patch Scan job is performed automatically on the devices when OpsRamp Agent is installed. It scans the devices and updates the Windows patches as per the Schedule. Once the missing patches are listed in patch management list, the user can approve the desired patches for installation. The approval can be based on patch rating or on specific patches.

### Patch Configuration

User can configure the patches depending on the patch details, recurrence details, and client details and reboot details.

1. On the **Automation** tab, click **Patch Management** and then click **Patch Configuration.**
2. Select the client.
3. Provide the details of Patch Configuration.
   1. Assign Devices
   2. Approval Type
   3. Reboot Options
   4. Patching Schedule
4. Click **Finish**.

&#x20;**Note**:

* The patch configuration is displayed in the configured list and click **Run Now** to install the approved patches as per requirement.
* The approved patches will be installed only when a patch configuration is added.
* Click [here](https://opsramp.gitbook.io/feature-guide/notifications-1#Patch) to read about how to configure email notifications.

### Patch Notification

### Patch Approvals

The approved ID list will be sent to the agent for installation. The patch install schedule will help configure the installation time and the source for the patches to be installed.

Patches can be approved after listing the missing patches:

#### By Patch

To approve the list of missing patches for the devices

1. On **Automation** tab, click **Patch Management** and then click **By Patch**. List of patches appears.
2. Click on **Missing** patch numbers to view the devices for which the patches are recommended.
3. Select the device and click **Approve**.![](https://docs.opsramp.com/wp-content/uploads/2017/06/Patch-Approval-ByPatch-800x280.png)

#### By Device

To approve the missing patches from devices

1. On **Automation** tab, Click **Patch Management** and then click **By Device**. List of devices appears.
2. Click on **Missing** patch numbers to view the patches recommended for the selected device.
3. Select the required patches and click **Approve**.\
   &#x20;![](https://docs.opsramp.com/wp-content/uploads/2017/06/Patch-Approval-ByDevice-2-800x279.png)

#### Bulk Patch Approval

To approve patches in bulk

1. On **Automation** tab, click **Patch Management** and then click **Patch Approvals**.
   1. Select clients and devices
   2. Select type of patches.
2. List of unapproved patches appears.
3. Select patches and click **Submit**.

**Note**: For Linux, few patches are dependent on other patches, for such patches you need to approve by clicking **OK** on the confirmation window. You will get the confirmation window when you select the patch that is dependent.

## Patch Management – Windows

### Configure WSUS Settings

1. Click **Infrastructure** tab, and select the device on which WSUS settings need to be applied.
2. Click **Settings** icon and then click **WSUS Settings**.
3. Click **Enable WSUS** and confirm the settings.

## Patch Feed

A feed is a source of published information. Patch feed refers to a source of patch information with all the available attributes of the patch.

The patch feed can be sourced from an OS vendor, a software package provider or from a partner who provides additional qualifications or insights into the published packages and manages it.

A typical patch feed provides the following information:

* The patch details such as patch name, patch ID, severity, OS Version, and release date
* Patch rating, if rated by the feed provider
* CVE ID, if given by the feed provider

### Scope

**Partner controls**

The feed created by Partner is available to all the clients as the default feed.

**Client Controls**

Client has the following scope:

* A client has default feed pre-integrated as a plugin if the feed is provided by Partner
  * Client views the partner feed as default feed if partner has defined a feed
  * Client does not get default feed if partner has not defined any feeds
* Any user who defines a feed can also choose to qualify the packages in the feed with respect to the pre-defined properties – Patch rating and CVE ID

### Patch Feed Integration

OpsRamp’s Patch Feed integration enables user to associate custom ratings with Windows and Linux patches published by the OS vendor. Instantiate your own Patch Feed, apply custom ratings to patches via APIs and patch devices against custom rated patches.

Qualify patches by providing the following attributes via API.

* Patch rating (Whitelisted, Blacklisted)
* CVE ID (Common Vulnerabilities and Exposures identifier)

OpsRamp supports two types of patch feeds:

* **Windows patch feed** – This feed is generated from Microsoft authoritative source to contain all the patch data released by Microsoft for different operating system versions.
* **Linux patch feed** – This feed is generated to contain patch data from different distributions of Linux from all its authoritative sources.

#### Install Patch Integration

To install patch integration:

1. On the left-hand side panel, in **Integrations** menu, click **Integrations**.
2. Click **Available Integrations**, click **Patch**, select the required OS patch feed and click **Install**.
3. Provide a name for the patch feed integration and click **Install**.\
   &#x20;![](https://docs.opsramp.com/wp-content/uploads/2017/06/Patch-Feed-800x421.png)
4. Click **Save** in **Authentication** section. Default authentication type is **OAUTH2**.
5. Use the **Key** and **Secret** to generate the authentication token required for API.
6. Copy the patch feed **UID**. This will be used while rating the patches for the respective patch feed through API.\
   &#x20;![](https://docs.opsramp.com/wp-content/uploads/2017/06/Patch-Feed-Integration-800x288.png)

   **Note**:

   * When client user unselects the option **Use Partner Feed**, the ratings of the patches changes to **Not Rated** and auto-approval of patches reverts to unapproved and new feed for client is created.\\

   * Patch integration appears in **My Integrations** once installed.

Click [here](https://docs.opsramp.com/patch-feed-apis/) to see how to create a Patch feed using API

The rated patches are listed as **Whitelisted.**

![](https://docs.opsramp.com/wp-content/uploads/2018/08/Patch-whitelisted-1.png)

### Auto-Approval of Patches

A **Whitelisted** patch is automatically approved on all user resources if **Auto-Approve** option is selected in[ **Patch Configuration**](https://docs.opsramp.com/os-patch-management/#Patch_Configuration).\
&#x20;![](https://docs.opsramp.com/wp-content/uploads/2018/08/Auto-Approve-Patch-1024x270-1-800x211.png)

### Uninstall Feed

A user cannot uninstall a managed feed if any user in the organization is using the feed.

Example: A partner cannot uninstall a managed feed if any of the clients of the partner are using the feed.

## Patch Baseline

A patch baseline is a collection of patches that are approved for installation on your instances. From a given feed, the user selects a subset of the packages that address the key vulnerabilities. The chosen set of packages forms the patch baseline.

Specification for Baseline name:

* Baseline name provided by the user cannot have blank spaces and the name needs to be case insensitive.
* Baseline name can have alphabets, numbers from 0-9 and one special character ‘\_’, but cannot have any other special characters like % \* # @ etc. Example: win16\_sec\_Sep18

#### Scope of Baseline

* A baseline created by partner is available to partner users and their client users depending on the scope chosen.
* A baseline created by the client is only available to the client users.

### Static Baseline

Static baselines consist of a list of patches selected by the user. This list of patches in the baseline does not change unless updated by the user.

To configure static baseline:

1. On **Automation** tab, click **Patch Management** and then click **Patch Baselines**.
2. Select the client.
3. Click **Add** to create a new baseline.
4. Select the **Include selected patches** option.
5. Provide the details of the patch baseline.
6. Click **Save.**\
   &#x20;![](https://docs.opsramp.com/wp-content/uploads/2017/06/Static-Baseline-1-800x469.png)\\

### Dynamic Baseline

Dynamic baselines consist of a set of patches that meet the selected filter criteria. The list of patches in a dynamic baseline varies as the available patches change when the filters are applied dynamically with the change.

**Note:**

User can define a baseline by selecting only the packages from the feed which satisfy the conditions defined on the properties on the feed such as severity, rating, and CVE IDs.

To configure dynamic baseline:

1. On **Automation** tab, click **Patch Management** and then click **Patch Baselines**.
2. Select the client.
3. Click **Add** to create a new baseline.
4. Select the **Include patches** **that satisfies below rules** option.
5. Provide the details of the patch baseline.
6. Click **Save.**\
   &#x20;![](https://docs.opsramp.com/wp-content/uploads/2017/06/Dynamic-baseline-800x611.png)\\

## Patch Compliance Configuration

User can configure patch compliance check jobs to track the compliance of selected devices or device groups against the configured baselines. Patch compliance check job is automatically computed after every run of the patch scan on the device.

Compliance is a metric that shows the number of patches in the baseline which are not installed on the device.

* Zero → compliant
* Greater than zero → non-compliant

To configure patch compliance configuration:

1. On **Automation** tab, click **Patch Management** and then click **Patch Compliance Configuration**.
2. Select the client.
3. Click **Add** to create a new compliance configuration.
4. Provide the details.
5. Click **Save.**\
   &#x20;![](https://docs.opsramp.com/wp-content/uploads/2017/06/Patch-compliance-check-713x800.png)\\

**Note**:

* Once created, Patch Compliance check job appears under **Jobs** in **Automation** tab.
* Only one instance of job is allowed per client.

### Patch Compliance History Widget <a href="#nextgenpatching-patchcompliancehistorywidget" id="nextgenpatching-patchcompliancehistorywidget"></a>

OpsRamp tracks the patch compliance of a device for each assigned baseline as a compliance metric. User can see the patch compliance history of each such device against each of the baselines that are assigned to the device in the form of a graph.

Graphs provide a time series representation of compliance values.

To view patch compliance graph:

1. On **Infrastructure** tab, click **Resources** and select the device for which you need to view the graph.
2. On the left-hand side panel, click **Metrics.**
3. View OS **Patch Compliance Graph**.\
   &#x20;![](https://docs.opsramp.com/wp-content/uploads/2017/06/Graph-1-400x208.png)

Not&#x65;**:** Compliance history for baselines of a different OS version appears as **Not Applicable**.

### Patch Compliance Snapshot Widget <a href="#nextgenpatching-patchcompliancesnapshotwidget" id="nextgenpatching-patchcompliancesnapshotwidget"></a>

Patch Compliance Snapshot widget allows the user to view the compliance status of a set of devices for a specific baseline.

The compliance data for each device is stored as a time series metric. The compliance snapshot widget gives the compliance status of a group of devices using the latest compliance data available from the last successfully run compliance job.

Patch Compliance Snapshot widget is a pie chart with 2 possible data.

1. No of devices non-compliant
2. No of devices compliant

Example: In a device group **Application Server** of 25 devices

If user creates a baseline **Windows\_2008\_R2** using Windows feed and configures Patch Compliance job check against the baseline. OpsRamp starts tracking patch compliance of devices in the device group **Application Server** against the baseline **Windows\_2008\_R2.**

When the patch compliance widget on the dashboard is selected with parameters:

Device group: Application Server

Baseline: Windows\_2008\_R2

The widget shows that one among the 25 devices is non-compliant for the selected baseline **Windows\_2008\_R2.**

![](https://docs.opsramp.com/wp-content/uploads/2017/06/patch-compliance-snapshot-widget-1-400x216.png)

## Installation of Patches

### Windows

1. Agent uses WUA API to fetch the missing patches and install the patches in the device
2. After the installation of patches is completed by Agent, agent sends the overall installation result of patches installed with Error codes
3. Alerts has five stages
   1. **Initiate**: Patch scan is initiated
   2. **Search**: Missing Patches are found and approved
   3. **Download initiated**: Patches are downloaded from the Microsoft server and WSUS
   4. **Install complete**: Installation is completed successfully
   5. Patch Management Completed

### Linux

1. Agent uses following APIs to fetch the missing patches and install the patches in the device
   * **Ubuntu** – Ubuntu core python-apt API
   * **RHEL** – Yum core API
   * **SLES** – zypper commands to get the patches
2. The agent uses locally configured system repository to download and install package upgrades.
3. After the installation of patches is completed by Agent, agent sends the overall installation result of patches installed with errors, if any.
4. Alerts has four stages
   1. Initiate: Patch scan is initiated
   2. Search: Missing Patches are found and approved
   3. Install complete: Installation is completed successfully
   4. Patch Management Completed

## Manage Patch Views

To manage patch view

1. Click **Alerts** tab.
2. Click **Manage Views**.
3. Click **Add**.
4. Select Client, Sites, Device Groups, Device Type, Device Status and Host Name.
5. Select Alert Type, Sub Alert Type, Priority, Status and Actions.
6. Click **Apply** to get the list of alerts alert browser.

## Types of Alerts

### Windows

The different types of alerts generated during the patching as follows:

1. Open the alert browser, select the ID, a new pop-up window appears with five different stages as
   * Initiate
   * Search
   * Download initiated
   * Install complete
   * Patch Management Completed
2. After reboot **OK** alert will be generated.

**Note**: If any patch installation requires a reboot and when the system is rebooted during any course of time, the agent sends an alert to alert browser displays Pending reboot after patching completed.

### Linux

The different types of alerts generated during the patching as follows:

1. Open the alert browser, select the ID, a new pop-up window appears with four different stages as
   * Initiate
   * Search
   * Install complete
   * Patch Management Completed.
   * After reboot OK alert will be generated.

**Note**: If any patch installation requires a reboot and when the system is rebooted during any course of time, the agent sends an alert to alert browser displays Pending reboot after patching completed.

## Third Party Patch Management

Application patching is used to install and update third-party applications in selected devices.

## Installing Applications

For installing applications silently, follow the steps:

1. On the **Automation** tab, click **Install Applications**. List of supported applications for Application patching is displayed.
2. Select the applications and click **Install Now** and then click **Next**.

Click **Confirm** to confirm to confirm the application patch.

## Updating Applications

Configure Application Scan Job to scan applications installed in the device. For configuring Application Scan job.

1. On the **Automation** tab, click **Jobs**.
2. Click **Create** to create a new job.
3. Select Job Type as **Application Scan**.
4. Schedule the job and click **Save**.
5. After configuring application scan job, run the job by clicking **Run Now** in the job list.
6. Applications which are installed and supported by OpsRamp are visible to the user in device level and client level.

For device level visibility, On the **Infrastructure** tab, click on the device and then click **Device Details**. You can view the application patch summary.

For client visibility

1. On the **Automation** tab, click **Application Patch Scan**.
2. Select the applications for updating to latest version. After selecting the applications, you can configure Application Update job by following the instructions below.
3. On the Maintenance tab, click **Jobs**.
4. Click **Create** to create a new job.
5. Select Job Type as **Application Update**.

We can scan and schedule a job for Application Update also. After configuring Application Update job, run the job by clicking **Run Now**.

Now the selected applications are updated to latest versions after a certain period of time and updated versions are reflected portal.

## Troubleshooting

| Common Issues                                              | Steps to Resolve                                                                                                                                                                                                                                                                                                                                            |
| ---------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Patches not matching with the Microsoft Server and Devices | <ul><li>Check for the LAN connectivity.</li><li>Check for Microsoft Update server.</li><li>Check for WSUS local Server settings.</li><li>Enable the WSUS Settings on the selected devices.</li><li>Add a job for the Missing Patches.</li><li>Schedule the Job for the devices.</li><li>Reboot if required for completed installation of patches.</li></ul> |

## Frequently Asked Questions

1. **What are the pre-requisites for the Patch Management**?\
   &#x20;WUA service should be running on the device.
2. **How are Missing Patches obtained for the devices**?\
   &#x20;User should create a job for the missing patches.
3. **What is Patch Scan**?\
   &#x20;Devices are automatically scanned when the OpsRamp Agent is installed on the devices.
4. **What is Patch Configuration**?\
   &#x20;User can configure the patches depending on the patch details, Recurrence details, and client details and reboot details.
5. **What is Patch Approvals**?\
   &#x20;Patches should be approved for installation. Process Below\.Desktops All “Security and Critical” patches with patch rating of “Whitelisted” would be auto approved for installation Servers By default, Patches must be approved for Servers from the Patch Approval Page.
6. **What are the different types of Approvals**?\
   &#x20;There are two types of Approvals\
   &#x20;Auto Approve: Automatically gets approved.\
   &#x20;Manual Approve: Needs approval for installation manually.\
   &#x20;**Note**: Auto-approval is applicable only for Windows patching. You need to manually approve patches for Linux devices.
7. **What are the different phases of Patch in Alert Browser**?\
   &#x20;There are 5 stages of alert for patch completion: Initiate Search, Download initiated, Install complete, and Patch Management Completed.
8. **Are there Role Based Access Alerts during patching**?\
   &#x20;After reboot OK alert will send, if the user configured maintenance during patching RBA cannot be run at that time.
9. **What happens when the device goes down in the patch window**?\
   &#x20;Nothing can be done if the device went to offline it is required that the device should be up and running.
10. **What happens if the patch fails**?\
    &#x20;Raise a critical alert as patch installation failed with a reason.
11. **When rebooting done for the devices after patching what happens to the missing reboots**?\
    &#x20;If the user configured to reboot after patching for servers it will reboot desktops which will show a popup as “patching completed and need to reboot”. If the user wants to reboot at that time he will reboot else ignore when user is not enabled automatic reboot.
12. **How to check for the WSUS setting**?\
    &#x20;Click Infrastructure tab->settings->WSUS settings.
13. **What is the purpose of creating a Job**?\
    &#x20;Once a job is created missing patches are collected from the servers configured at the device.
14. **What is the reason for the mismatches in the patches of local machine with the Servers Patches**?\
    &#x20;Check the server settings could be Microsoft server or WSUS server \[local].
15. **How do we update the application patching**?\
    &#x20;Application patching is performed in two cases:
    1. The device should be available during the update activity performed at the scheduled time.
    2. Installed application version should be less than the supported version.
16. **Can incidents be created during patching**?\
    &#x20;No, Incidents are not created during patching.
