ServiceNow

Overview

ServiceNow is an IT service management platform for recording, tracking, and managing a company’s enterprise-level IT processes in a single location.

The Datadog ServiceNow integration is a two-way integration that allows you to:

Push Datadog-generated events to ServiceNow tickets, as well as manage the resolution workflow from within Datadog through IT service management (ITSM) and IT operations management (ITOM)
Use Datadog as a discovery mechanism for ServiceNow Configuration Management Database (CMDB) Configuration Items (CIs) with the service graph connector
Enrich business-specific information stored as CIs in ServiceNow CMDB with your hosts, services, and devices information from Datadog, enabling you to better understand your infrastructure usage, accelerate troubleshooting, and maximize resource utilization

Datadog integrates with the following ServiceNow tools:

ITOM
ITSM
CMDB

Note: The Datadog ServiceNow integration supports ServiceNow releases that are not listed as end of life.

Configure the ServiceNow tile in Datadog

Navigate in Datadog to the ServiceNow integration tile on the Integrations page.
Click on Add New Instance.
Add the instance name, which is the subdomain of your ServiceNow domain: <INSTANCE_NAME>.service-now.com.
Add the username and password for your ServiceNow instance.

Note: You can create a limited user in ServiceNow just for Datadog.

CMDB setup

Service Graph Connector for Datadog

Service Graph Connector for Observability - Datadog can automatically populate server and database configuration items (CIs) in the CMDB for new resources discovered by Datadog. The Service Graph connector is available through the ServiceNow store.

For configuration, follow the Service Graph Connector’s guided setup instructions.

Supported CI Types:

Server
Amazon RDS

The notes below only apply if you have already configured the integration for ServiceNow ITOM/ITSM:

Service Graph Connector does not use the Target table and Custom table values from the configuration tile. You can save the integration with the Target table default values.
The same ITOM/ITSM user can be used for Service Graph Connector by granting this user the role of cmdb_import_api_admin as described in the Service Graph Connector’s guided setup instructions.

Host tagging

Enrich your Datadog hosts with ServiceNow CMDB metadata through host tagging.

To enable ingestion of host tags:

Configure a Query Builder query in your ServiceNow instance that returns all of the hosts you wish to tag in Datadog.
Schedule the query to execute at your desired refresh interval.
Once the query has been saved in ServiceNow, go to Datadog’s ServiceNow integration tile. Select the Host Tagging in the CMDB Enrichment tab.
Under Query Configuration, click the Add New Query button.
Select the ServiceNow Instance and the Query from the dropdown menus.
Select a value for the Hostname Column that maps your query’s root CI hostname field to Datadog’s hostname field.
Select any optional field name remapping with Column Name Maps.
Click Save.

Expect host tags to populate in Datadog shortly after your queries’ scheduled executions.

Screenshot of Host info tab showing ServiceNow host tags

Monitor the ingestion process in the Datadog Events Explorer by scoping your search query on source:servicenow.

Screenshot showing one running ingestion

Troubleshooting

For host tagging to work correctly, ensure that the following are true in your system:

The user who created and executes the Query Builder query matches a username in your Datadog configuration. The user in ServiceNow must have the role cmdb_query_builder_read.
The number of results returned by your query must be less than or equal to the glide.cmdb.query.max_results_limit setting in ServiceNow. By default, the maximum number of results is 10000. To change the setting, go to Configuration -> CMDB Properties -> Query Builder Properties.
All CIs configured in your Query Builder query must have a 1 label. This ensures you have not created any duplicate CIs, which the parser does not support.

Limitations

Ingestion is limited to 100k hosts per execution.
Updates to hosts are throttled to a few thousand per hour. Take this limit into consideration when choosing your schedule interval.
Tagging does not work for Linux machines with lower case host names, because host aliases in Datadog are case sensitive.

Service tagging

Enrich your Datadog Service Catalog with ServiceNow CMDB metadata through service tagging.

With service tagging, you can populate your Datadog Service Catalog with services from your ServiceNow CMDB.

Setup

To enable ingestion of service data:

Configure a Query Builder query in your ServiceNow instance that returns all of the services with which you wish to enrich the Service Catalog.
Schedule the query to execute at your desired refresh interval.
Once the query is saved in ServiceNow, go to Datadog’s ServiceNow integration tile. Select Service Tagging in the CMDB Enrichment tab.
Under Query Configuration, click the Add New Query button.
Select the ServiceNow Instance and the Query from the dropdown menus.
Select a value from the Service Name Column dropdown menu. The value matches the column name on your query’s root service CI, and populates the service name in the service catalog.
Configure schema mappings to pull additional metadata about your service into the service catalog. See Service definitions for details. For Datadog to accept the ingestion, each field in the mapping needs to be of the correct type to map to the service catalog service definition schema.
Click Save.

Expect to see service data populated in Datadog a few minutes after your queries’ scheduled executions. To view ingestion errors, go to the Events Explorer and search for events with source:servicenow.

Screenshot of the Service Configuration panel showing metadata populated from ServiceNow

Troubleshooting

For service ingestion to work correctly, ensure that the following are true in your system:

The user who created and executes the Query Builder query matches a username in your Datadog configuration. The user in ServiceNow must have the role cmdb_query_builder_read.
The number of results returned by your query must be less than or equal to the glide.cmdb.query.max_results_limit setting in ServiceNow. By default, the maximum number of results is 10000. To change the setting, go to Configuration -> CMDB Properties -> Query Builder Properties.
All CIs configured in your Query Builder query must have a 1 label. This ensures you have not created any duplicate CIs, which the parser does not support.

Network device tagging

Add tags to your network devices in Datadog populated with data from your ServiceNow CMDB.

With device tagging, you can dynamically enrich network devices monitored by Datadog Network Device Monitoring with device metadata from your ServiceNow CMDB.

To enable ingestion of device tags:

Configure a Query Builder query in your ServiceNow instance. Make sure it is returning the device IP Address.
Schedule the query to execute at your desired refresh interval.
If you are using a custom IP namespace in Datadog, you need to add it to ServiceNow. Create a column on the Network device CI called u_dd_device_namespace, populated by the corresponding namespace for each device. If this column is not present, the default namespace is used.
Once the query is saved in ServiceNow, go to Datadog’s ServiceNow integration tile. Select Device Tagging in the CMDB Enrichment tab.
Under Query Configuration, click the Add New Query button.
Select the ServiceNow Instance and the Query from the dropdown menus.
Select the IP Address column that maps your query’s IP Address field to Datadog’s IP Address field.
Select any optional field name remappings.
Click Save.

You can expect to see network device tags populated in Datadog within a few minutes after your queries’ scheduled executions. Any ingestion errors are reported through events viewable in your events explorer.

Monitor the ingestion process in the Datadog Events Explorer by scoping your search query on source:servicenow.

Troubleshooting

Verify that the user who created or is executing the querybuilder query is the same user in your Datadog configuration and has the role cmdb_query_builder_read.
Check that your query isn’t returning more results than your glide.cmdb.query.max_results_limit setting in Servicenow is configured to allow. Make sure all CIs configured in your querybuilder query have a ‘1’ label. Ensure you did not create any duplicate CIs, as the parser doesn’t support them.

Limitations

Ingestion is limited to 100k hosts per execution.
Network device tagging is limited to SNMP devices.
Updates to devices are throttled to a few thousand per hour. Take this into consideration when choosing your schedule interval.

Reference Tables

Use Reference Tables to automatically enrich logs and events with additional fields from your ServiceNow CIs. With Reference Tables, you can map sets of value fields to a primary key, such as a hostname, and automatically append these fields to all logs or events that contain the specified key.

To enable ingestion of Reference Tables:

Configure a Query Builder query in your ServiceNow instance.
Schedule the query to run at your desired refresh interval.
Save the query.
Select, Add New Query, and choose your query from the dropdown menu.
In the primary key dropdown, select the column name you want to use as your primary key.
1. Optionally, create a Processing Pipeline with this primary key to enrich and correlate logs and events.
Enter a name for your reference table.
Click Save.

The Reference Table will be populated with the data from the query shortly after saving.

Caveats and restrictions

Reference Table name must be unique.
Deletions and schema updates of existing tables are not supported.

ITOM and ITSM setup

To use the Datadog integration for Monitors, Case Management, and Incident Management, follow these steps:

Install the latest Datadog Update Set
Create a ServiceNow Account with correct permissions for Datadog
Configure Datadog applications for use with ITOM and ITSM

Install the Datadog update set

Download the latest Update Set here: Datadog-Snow_Update_Set_v2.5.4.xml

Note: Integration with Case Management is only supported with v2.4.0 and later. Integration with Incident Management and bidirectional syncing with Case Management is only supported with v2.5.0 and later.

In ServiceNow:

Manually import the Update Set XML file that you downloaded.
Once you import the XML file, the Update Set should show a state of Loaded. Click on the name of the Update Set to preview the changes.
Once you preview the Update Set to ensure there are no errors, select Commit Update Set to merge the application into your system.

After setup is complete, search Datadog in the ServiceNow navigation menu to access the five tables and Configuration Page for bidirectional syncing setup.

Configuration
Datadog Incidents ITSM
Datadog Cases ITOM, formerly known as Datadog Case Event
Datadog Cases ITSM, formerly known as Datadog Case Incident
Datadog Monitors ITOM, formerly known as Datadog Event
Datadog Monitors ITSM, formerly known as Datadog Incident

Create a ServiceNow Account with correct permissions for Datadog

To use the integration, create a ServiceNow user (for example, with username “datadog”) and assign it all of the following roles:

x_datad_datadog.user and
import_set_loader and
import_transformer

Send Monitor Notifications direct to ITOM and ITSM modules

If you’d like to send notifications directly to the ITOM module Event table or the ITSM module Incident table, the ServiceNow user needs both of the following roles:

ITIL and
evt_mgmt_integration

Sync bidirectionally with Datadog Case Management

If you’d like to sync incident state for resolution, the ServiceNow user needs one of the following roles:

ITIL or
list_updater or
sn_incident_write

If you’d like to sync incident state for closure, the ServiceNow user needs the following role:

ITIL_admin

Note: Manual updates made to a ticket in ServiceNow by this ServiceNow user will not be synced to Datadog.

Configure Datadog applications for use with ITOM and ITSM modules

Note: You must set up the ServiceNow tile in the Datadog integrations page before completing the following steps:

For Monitor Notifications using @servicenow-<INSTANCE_NAME> in Datadog, select the interim table to send notifications to in the ServiceNow tile.

From the Monitor Notifications dropdown in the ServiceNow integration tile, select the table that monitor notifications will write to.
To validate the integration is set up correctly, add @servicenow-<INSTANCE_NAME> in a monitor or event notification. The raw data populates rows in the interim table and is forwarded to the ServiceNow table specified in the mappings and transformations you created.
Use transform maps in ServiceNow to customize the formatting for data sent to the tables.
Customize the notification payload with available Datadog variables or custom strings.

If you use Datadog’s Case Management, you can select an interim table to sync cases in the ServiceNow tile (disabled by default).

From the Case Management dropdown, select the table for cases.
To validate the integration is set up correctly, navigate to Case Management in Datadog and select Create ServiceNow Incident. Choose the instance and optional assignment group, and then click Create.
Use transform maps in ServiceNow to customize the formatting for data sent to the tables.

Note: The Datadog Cases ITOM table is not set up for updates. The Datadog Cases ITSM table is recommended for this version.

An instance in the ServiceNow integration tile with monitor notifications, custom notification payloads, and case management

Sync state and comments bidirectionally with Case Management

To allow edits in ServiceNow to update their associated cases in Datadog, a ServiceNow user with the x_datad_datadog.user role and admin role must configure the installation settings for the ITOM/ITSM Integration for Datadog app in ServiceNow:

Navigate to the configuration settings page for the ITOM/ITSM Integration for Datadog app by clicking All in the upper-left hand corner, typing ITOM/ITSM Integration for Datadog into the filter, and clicking the Configuration link that appears in the filtered list.
Choose your Datadog Data Center site.
Paste a Datadog API Key, which can be found in your Organization Settings, in the API Key field.
Paste a Datadog Service Account Application Key, which can be found in your Organization Settings, in the Application Key field.
Check the Enabled checkbox and save your configuration changes.

After configuring the installation settings in ServiceNow, return to Datadog Case Management to configure the integration.

Note: It is important to use a Service Account Application Key for this setup rather than a user’s Application Key. A user’s Application Key is tied to the user’s account permissions. If the user’s permissions are reduced or if the user is deactivated, bidirectional syncing between ServiceNow and Datadog will stop. A Service Account Application Key is not tied to an individual user, so bidirectional sync will not be impacted by user account changes.

Configuration Settings in ServiceNow to sync ServiceNow changes in Datadog

Customize data for Monitor Notifications with transform maps

The Datadog Monitors ITSM and Datadog Cases ITSM tables use a transform map to transform Datadog records into ServiceNow incidents. Similarly, the Datadog Monitors ITOM and Datadog Cases ITOM transform Datadog records into ServiceNow events.

The Datadog Monitors ITOM and Datadog Monitors ITSM tables use transform maps to transform Datadog records into ServiceNow events and incidents respectively. You can customize the ServiceNow events and incidents information in these tables by customizing the notification payload in the Datadog Configuration Tile and extending the transform maps in ServiceNow.

Note: The Datadog Cases ITOM and Datadog Cases ITSM tables similarly use transform maps; however, transform map customization is not recommended for use with Case Management given the payload for Datadog cases is not customizable.

Troubleshooting

If you’re not seeing events in your ServiceNow tables and instead have

An error message in your Datadog integration tile or an Error while trying to post to your ServiceNow instance notification:
- Verify only the subdomain was used when entering your instance name.
- Verify the user you created has the required permissions.
- Verify the username and password are correct.
The integration is configured, an alert triggered, and no ticket is created:
- Confirm that the interim table is populated. If so, the issue is with mappings and transformations. You can debug your mappings and scripts further by navigating to Transform Errors in ServiceNow.
- Confirm that you’re working with the interim table you specified in the tile.
The ServiceNow user needs rest_service and x_datad_datadog.user roles so that it can access the import tables. If you’re using the legacy way of sending notifications directly to either the Incident table or Event table, you need the permissions itil and evt_mgmt_integration.

Need additional help? Contact Datadog support.

Knowledge base

Datadog import host auto-flush rule

To prevent the import set table x_datad_datadog_import_host from accumulating too many rows, an auto-flush rule has been added to the Table Cleaner tool to keep only the last 24 hours of data. This configuration setting can be changed as needed by navigating to sys_auto_flush_list.do in the filter navigator and going into the rule for the x_datad_datadog_import_host table. The Age in seconds field can be updated accordingly.

Auto-generate support tickets from Datadog alerts

After ServiceNow is connected to your Datadog account, alerts received can automatically create support tickets and send them to the ServiceNow ticketing queue. From there, your support team is notified of issues using the communication workflows that you have already established inside ServiceNow. Mention @servicenow in the alert message or add @servicenow to the notification list for that monitor.

Use variables in ticket payload and field mappings

Variables can be used in the body of your alerts or in field mappings to ensure details from the event are included in ServiceNow. For example, you can include the title and severity in the appropriate ServiceNow field or you can include a link back to the specific incident in Datadog right from the ServiceNow ticket.

Incident Priority field mapping

The priority field in ServiceNow incidents is read only and can only be updated using priority lookup rules.

Define Impact and Urgency in monitors to calculate the ServiceNow incident priority.

Automate support resolution workflow

Once the monitor state returns to normal, the associated support ticket is automatically marked as “resolved”.

Defining custom mappings

Click one of the tables, for example Datadog Monitors ITSM Tables, and scroll to the bottom of the record to see the link for the associated transform map.

Understanding the mapping

Click on the name of the transform map to view the record:

At the top are two important fields on the Transform record - Source table and Target table:

Notes:

The source is the import set table you selected (Datadog Monitors ITSM Tables) and the target is your actual incident table (or event table) where events are stored.
The field mappings are at the bottom of the record. Some basic mappings are included. This is where you select the fields to include, define the format, and select the target fields in your ServiceNow instance.

Add a new field mapping

Click New:

Select the source and target fields for one to one mappings:

Or, check the Use source script box and define transformations:

Note: For mapping any custom fields in the Integration Tile, you can use the following mapping script for either the Datadog Monitors ITOM and Datadog Monitors ITSM Transform maps. In this example, the field my_field was defined as a custom field in the integration tile:

answer = (function transformEntry(source)
{
    var additional_info = JSON.parse(source.additional_info);
    return additional_info.custom_my_field;
})(source);

Define multiple mappings

Use Mapping Assist (under Related Links) to map several source and target fields:

Validation

To validate the integration is set up correctly, add @servicenow in a monitor or event notification. The raw data populates rows in the interim table and is forwarded to the ServiceNow table specified in the mappings and transformations you created.