Use Observability Pipelines’ Microsoft Sentinel destination to send logs to Microsoft Sentinel.

Setup

Set up the Microsoft Sentinel destination and its environment variables when you set up a pipeline. The information below is configured in the pipelines UI, except for Prerequisites which provides instructions on how to find the information you need in Microsoft Azure.

Set up the destination

Prerequisites

To set up the Microsoft Sentinel destination, you need the following information:

Do the following to get that information:

1. Create or identify a Data Collection Rule (DCR).
   1. In the Azure Portal, navigate to Azure Monitor → Data Collection Rules.
   2. Create a DCR or use an existing one for custom logs. See Data collection rules (DCRs) in Azure Monitor to learn more about creating or viewing DCRs.
   3. Take note of the DCR Immutable ID and, if you are using private links, the DCR’s Data Collection Endpoint (DCE). You need this information when you set up the Microsoft Sentinel destination.
   4. Define a custom table (for example, Custom-MyLogs_CL) in the DCR, which is where Observability Pipelines sends logs to.
2. Get the ingestion URL.
   1. In the DCR, locate the Logs Ingestion API endpoint. The endpoint has the format: https://<DCE-ID>.ingest.monitor.azure.com/dataCollectionRules/<DCR-Immutable-ID>/streams/<Stream-Name>?api-version=2023-01-01, where the <Stream-Name> typically matches your custom table (for example, Custom-MyLogs_CL).
   2. The ingestion URL is needed when you set up you Microsoft Sentinel destination’s environment variable.
3. To authenticate the Observability Pipelines Worker with Microsoft Sentinel:
   1. In the Azure Portal, navigate to Azure AD > App Registrations and register an Azure Active Directory (AD) application. See Register an application in Microsoft Entra ID for information on creating a new application.
   2. Generate a Client Secret.
   3. Assign it the Monitoring Metrics Publisher role on the Log Analytics workspace
   4. Take note of the Tenant ID, Client ID, and Client Secret. You need this information when you set up the Microsoft Sentinel destination.

Set up destination in Observability Pipelines

To set up the Microsoft Sentinel destination in Observability Pipelines:

1. Enter the client ID for your application, such as 550e8400-e29b-41d4-a716-446655440000.
2. Enter the directory ID for your tenant, such as 72f988bf-86f1-41af-91ab-2d7cd011db47. This is the Azure AD tenant ID.
3. Enter the name of the table, such as Custom-MyLogs, to which you are sending logs.
4. Enter the Data Collection Rule (DCR) immutable ID, such as dcr-000a00a000a00000a000000aa000a0aa.

Set the environment variables

• Data collection endpoint (DCE)
  • The DCE endpoint URL is shown as the Logs Ingestion Endpoint or Data Collection Endpoint on the DCR Overview page. An example URL: https://<DCE-ID>.ingest.monitor.azure.com/dataCollectionRules/<DCR-Immutable-ID>/streams/<Stream-Name>?api-version=2023-01-01.
  • Stored as the environment variable DD_OP_DESTINATION_MICROSOFT_SENTINEL_DCE_URI
• Client secret
  • This is the Azure AD application’s client secret, such as 550e8400-e29b-41d4-a716-446655440000.
  • Stored as the environment variable DD_OP_DESTINATION_MICROSOFT_SENTINEL_CLIENT_SECRET

How the destination works

Event batching

A batch of events is flushed when one of these parameters is met. See event batching for more information.