Microsoft Azure Data Explorer

Overview

Azure Data Explorer is a highly scalable and secure analytics service that enables you to do rich exploration of structured and unstructured data for instant insights. Optimized for ad-hoc queries, Azure Data Explorer enables data exploration over raw, structured, and semi-structured data delivering fast time to insight. Use Datadog to monitor your Azure Data Explorer performance and utilization in context with the rest of your applications and infrastructure.

Get metrics from Azure Data Explorer to:

  • Track ingestion, processing, and latency performance of your Data Explorer instances.
  • Monitor the utilization of your Data Explorer compute, memory, and network resources.

Setup

Installation

If you haven’t already, set up the Microsoft Azure integration first. There are no other installation steps.

Data Collected

Metrics

azure.kusto_clusters.batch_blob_count
(gauge)
Number of data sources in an aggregated batch for ingestion.
azure.kusto_clusters.batch_duration
(gauge)
Duration of the aggregation phase in the ingestion flow.
Shown as second
azure.kusto_clusters.batches_processed
(count)
Number of batches aggregated for ingestion. Batching Type: whether the batch reached batching time; data size or number of files limit set by batching policy.
azure.kusto_clusters.batch_size
(gauge)
Uncompressed expected data size in an aggregated batch for ingestion.
Shown as byte
azure.kusto_clusters.blobs_dropped
(count)
Number of blobs permanently rejected by a component.
azure.kusto_clusters.blobs_processed
(count)
Number of blobs processed by a component.
azure.kusto_clusters.blobs_received
(count)
Number of blobs received from input stream by a component.
azure.kusto_clusters.cache_utilization
(gauge)
Utilization level in the cluster scope.
Shown as percent
azure.kusto_clusters.cache_utilization_factor
(gauge)
Percentage difference between the current number of instances and the optimal number of instances (per cache utilization).
Shown as percent
azure.kusto_clusters.continuous_export_max_lateness_minutes
(gauge)
Lateness (in minutes) reported by the continuous export jobs in the cluster.
azure.kusto_clusters.continuous_export_num_of_records_exported
(count)
Number of records exported; fired for every storage artifact written during the export operation.
azure.kusto_clusters.continuous_export_pending_count
(gauge)
Number of pending continuous export jobs ready for execution.
azure.kusto_clusters.continuous_export_result
(count)
Indicates whether Continuous Export succeeded or failed.
azure.kusto_clusters.cpu
(gauge)
CPU utilization level.
Shown as percent
azure.kusto_clusters.discovery_latency
(gauge)
Reported by data connections (if exist). Time in seconds from when a message is enqueued or event is created until it is discovered by data connection. This time is not included in the Azure Data Explorer total ingestion duration.
Shown as second
azure.kusto_clusters.events_dropped
(count)
Number of events dropped permanently by data connection. An Ingestion result metric with a failure reason will be sent.
azure.kusto_clusters.events_processed
(count)
Number of events processed by the cluster.
azure.kusto_clusters.events_processed_for_event_hubs
(count)
Number of events processed by the cluster when ingesting from Event/IoT Hub.
azure.kusto_clusters.events_received
(count)
Number of events received by data connection.
azure.kusto_clusters.export_utilization
(gauge)
Export utilization.
Shown as percent
azure.kusto_clusters.ingestion_latency_in_seconds
(gauge)
Latency of data ingested; from the time the data was received in the cluster until it's ready for query. The ingestion latency period depends on the ingestion scenario.
Shown as second
azure.kusto_clusters.ingestion_result
(count)
Total number of sources that either failed or succeeded to be ingested. Splitting the metric by status; you can get detailed information about the status of the ingestion operations.
azure.kusto_clusters.ingestion_utilization
(gauge)
Ratio of used ingestion slots in the cluster.
Shown as percent
azure.kusto_clusters.ingestion_volume_in_mb
(count)
Overall volume of ingested data to the cluster.
Shown as byte
azure.kusto_clusters.instance_count
(gauge)
Total instance count.
azure.kusto_clusters.keep_alive
(gauge)
Sanity check indicates the cluster responds to queries.
azure.kusto_clusters.materialized_view_age_minutes
(gauge)
Materialized view age in minutes.
azure.kusto_clusters.materialized_view_age_seconds
(gauge)
Materialized view age in seconds.
Shown as second
azure.kusto_clusters.materialized_view_data_loss
(gauge)
Indicates potential data loss in materialized view.
azure.kusto_clusters.materialized_view_extents_rebuild
(gauge)
Number of extents rebuild.
azure.kusto_clusters.materialized_view_health
(gauge)
Health of the materialized view (1 for healthy; 0 for non-healthy).
azure.kusto_clusters.materialized_view_records_in_delta
(gauge)
Number of records in the non-materialized part of the view.
azure.kusto_clusters.materialized_view_result
(gauge)
Result of the materialization process.
azure.kusto_clusters.query_duration
(gauge)
Queriesduration in seconds.
Shown as millisecond
azure.kusto_clusters.query_result
(count)
Total number of queries.
azure.kusto_clusters.queue_length
(gauge)
Number of pending messages in a component's queue.
azure.kusto_clusters.queue_oldest_message
(gauge)
Time in seconds from when the oldest message in queue was inserted.
azure.kusto_clusters.received_data_size_bytes
(gauge)
Size of data received by data connection. This is the size of the data stream; or of raw data size if provided.
Shown as byte
azure.kusto_clusters.stage_latency
(gauge)
Cumulative time from when a message is discovered until it is received by the reporting component for processing (discovery time is set when message is enqueued for ingestion queue; or when discovered by data connection).
Shown as second
azure.kusto_clusters.streaming_ingest_data_rate
(gauge)
Streaming ingest data rate (MB per second).
azure.kusto_clusters.streaming_ingest_duration
(gauge)
Streaming ingest duration in milliseconds.
Shown as millisecond
azure.kusto_clusters.streaming_ingest_results
(count)
Streaming ingest result.
azure.kusto_clusters.total_number_of_concurrent_queries
(gauge)
Total number of concurrent queries.
azure.kusto_clusters.total_number_of_extents
(gauge)
Total number of data extents.
azure.kusto_clusters.total_number_of_throttled_commands
(count)
Total number of throttled commands.
azure.kusto_clusters.total_number_of_throttled_queries
(gauge)
Total number of throttled queries.
azure.kusto_clusters.weak_consistency_latency
(gauge)
Max latency between the previous metadata sync and the next one (in db/node scope).
Shown as second
azure.kusto_clusters.count
(gauge)
Count of Kusto clusters.
azure.kusto_clusters.status
(gauge)
Status of Kusto clusters(deprecated).

Events

The Azure Data Explorer integration does not include any events.

Service Checks

The Azure Data Explorer integration does not include any service checks.

Troubleshooting

Need help? Contact Datadog support.

PREVIEWING: safchain/fix-custom-agent