Microsoft Azure Data Factory

概要

Azure Data Factory は、データの保管・移動・処理サービスを自動化されたデータパイプラインとして構築するクラウドデータ統合サービスです。

Datadog Azure インテグレーションを使用して、Data Factory からメトリクスを収集できます。

セットアップ

インストール

Microsoft Azure インテグレーションをまだセットアップしていない場合は、最初にセットアップします。それ以上のインストール手順はありません。

収集データ

メトリクス

azure.datafactory_factories.activity_cancelled_runs
(count)
Cancelled activity runs metrics.
azure.datafactory_factories.activity_failed_runs
(count)
Failed activity runs metrics.
azure.datafactory_factories.activity_succeeded_runs
(count)
Succeeded activity runs metrics.
azure.datafactory_factories.airflow_integration_runtime_celery_task_timeout_error
(count)
Airflow integration runtime celery task timeout error.
azure.datafactory_factories.airflow_integration_runtime_collect_db_dags
(gauge)
Airflow integration runtime collect DB dags.
Shown as millisecond
azure.datafactory_factories.airflow_integration_runtime_cpu_percentage
(gauge)
Airflow integration runtime cpu percentage.
Shown as percent
azure.datafactory_factories.airflow_integration_runtime_cpu_usage
(gauge)
Airflow integration runtime memory usage.
azure.datafactory_factories.airflow_integration_runtime_dag_bag_size
(count)
Airflow integration runtime DAG bag size.
azure.datafactory_factories.airflow_integration_runtime_dag_callback_exceptions
(count)
Airflow integration runtime DAG callback exceptions.
azure.datafactory_factories.airflow_integration_runtime_dag_file_refresh_error
(count)
Airflow integration runtime DAG file refresh error.
azure.datafactory_factories.airflow_integration_runtime_dag_processing_import_errors
(count)
Airflow integration runtime DAG processing import errors.
azure.datafactory_factories.airflow_integration_runtime_dag_processing_last_duration
(gauge)
Airflow integration runtime DAG processing last duration.
Shown as millisecond
azure.datafactory_factories.airflow_integration_runtime_dag_processing_last_run_seconds_ago
(gauge)
Airflow integration runtime DAG processing last run seconds ago.
Shown as second
azure.datafactory_factories.airflow_integration_runtime_dag_processing_manager_stalls
(count)
Airflow integration runtime DAG processingmanager stalls.
azure.datafactory_factories.airflow_integration_runtime_dag_processing_processes
(count)
Airflow integration runtime DAG processing processes.
azure.datafactory_factories.airflow_integration_runtime_dag_processing_processor_timeouts
(gauge)
Airflow integration runtime DAG processing processor timeouts.
Shown as second
azure.datafactory_factories.airflow_integration_runtime_dag_processing_total_parse_time
(gauge)
Airflow integration runtime DAG processing total parse time.
Shown as second
azure.datafactory_factories.airflow_integration_runtime_dag_run_dependency_check
(gauge)
Airflow integration runtime DAG run dependency check.
Shown as millisecond
azure.datafactory_factories.airflow_integration_runtime_dag_run_duration_failed
(gauge)
Airflow integration runtime DAG run duration failed.
Shown as millisecond
azure.datafactory_factories.airflow_integration_runtime_dag_run_duration_success
(gauge)
Airflow integration runtime DAG run duration success.
Shown as millisecond
azure.datafactory_factories.airflow_integration_runtime_dag_run_first_task_scheduling_delay
(gauge)
Airflow integration runtime DAG run first task scheduling delay.
Shown as millisecond
azure.datafactory_factories.airflow_integration_runtime_dag_run_schedule_delay
(gauge)
Airflow integration runtime DAG run schedule delay.
Shown as millisecond
azure.datafactory_factories.airflow_integration_runtime_executor_open_slots
(count)
Airflow integration runtime executor open slots.
azure.datafactory_factories.airflow_integration_runtime_executor_queued_tasks
(count)
Airflow integration runtime executor queued tasks.
azure.datafactory_factories.airflow_integration_runtime_executor_running_tasks
(count)
Airflow integration runtime executor running tasks.
azure.datafactory_factories.airflow_integration_runtime_job_end
(count)
Airflow integration runtime job end.
azure.datafactory_factories.airflow_integration_runtime_job_heartbeat_failure
(count)
Airflow integration runtime heartbeat failure.
azure.datafactory_factories.airflow_integration_runtime_job_start
(count)
Airflow integration runtime job start.
azure.datafactory_factories.airflow_integration_runtime_memory_percentage
(gauge)
Airflow integration runtime memory percentage.
Shown as percent
azure.datafactory_factories.airflow_integration_runtime_operator_failures
(count)
Airflow integration runtime operator failures.
azure.datafactory_factories.airflow_integration_runtime_operator_successes
(count)
Airflow integration runtime operator successes.
azure.datafactory_factories.airflow_integration_runtime_pool_open_slots
(count)
Airflow integration runtime pool open slots.
azure.datafactory_factories.airflow_integration_runtime_pool_queued_slots
(count)
Airflow integration runtime pool queued slots.
azure.datafactory_factories.airflow_integration_runtime_pool_running_slots
(count)
Airflow integration runtime pool running slots.
azure.datafactory_factories.airflow_integration_runtime_pool_starving_tasks
(count)
Airflow integration runtime pool starving tasks.
azure.datafactory_factories.airflow_integration_runtime_scheduler_critical_section_busy
(count)
Airflow integration runtime scheduler critical section busy.
azure.datafactory_factories.airflow_integration_runtime_scheduler_critical_section_duration
(gauge)
Airflow integration runtime scheduler critical section duration.
Shown as millisecond
azure.datafactory_factories.airflow_integration_runtime_scheduler_failed_sla_email_attempts
(count)
Airflow integration runtime scheduler failed SLA email attempts.
azure.datafactory_factories.airflow_integration_runtime_scheduler_heartbeat
(count)
Airflow integration runtime scheduler heartbeats.
azure.datafactory_factories.airflow_integration_runtime_scheduler_orphaned_tasks_adopted
(count)
Airflow integration runtime scheduler orphaned tasks adopted.
azure.datafactory_factories.airflow_integration_runtime_scheduler_orphaned_tasks_cleared
(count)
Airflow integration runtime scheduler orphaned tasks cleared.
azure.datafactory_factories.airflow_integration_runtime_scheduler_tasks_executable
(count)
Airflow integration runtime scheduler tasks executable.
azure.datafactory_factories.airflow_integration_runtime_scheduler_tasks_killed_externally
(count)
Airflow integration runtime scheduler tasks killed externally.
azure.datafactory_factories.airflow_integration_runtime_scheduler_tasks_running
(count)
Airflow integration runtime scheduler tasks running.
azure.datafactory_factories.airflow_integration_runtime_scheduler_tasks_starving
(count)
Airflow integration runtime scheduler tasks starving.
azure.datafactory_factories.airflow_integration_runtime_started_task_instances
(count)
Airflow integration runtime started task instances.
azure.datafactory_factories.airflow_integration_runtime_task_instance_created_using_operator
(count)
Airflow integration runtime task instance created using operator.
azure.datafactory_factories.airflow_integration_runtime_task_instance_duration
(gauge)
Airflow integration runtime task instance duration.
Shown as millisecond
azure.datafactory_factories.airflow_integration_runtime_task_instance_failures
(count)
Airflow integration runtime task instance failures.
azure.datafactory_factories.airflow_integration_runtime_task_instance_finished
(count)
Airflow integration runtime task instance finished.
azure.datafactory_factories.airflow_integration_runtime_task_instance_previously_succeeded
(count)
Airflow integration runtime task instance previously succeeded.
azure.datafactory_factories.airflow_integration_runtime_task_instance_successes
(count)
Airflow integration runtime task instance successes.
azure.datafactory_factories.airflow_integration_runtime_task_removed_from_dag
(count)
Airflow integration runtime task removed from DAG.
azure.datafactory_factories.airflow_integration_runtime_task_restored_to_dag
(count)
Airflow integration runtime task restored to DAG.
azure.datafactory_factories.airflow_integration_runtime_triggers_blocked_main_thread
(count)
Airflow integration runtime triggers blocked main thread.
azure.datafactory_factories.airflow_integration_runtime_triggers_failed
(count)
Airflow integration runtime triggers failed.
azure.datafactory_factories.airflow_integration_runtime_triggers_running
(count)
Airflow integration runtime triggers running.
azure.datafactory_factories.airflow_integration_runtime_triggers_succeeded
(count)
Airflow integration runtime triggers succeeded.
azure.datafactory_factories.airflow_integration_runtime_zombies_killed
(count)
Airflow integration runtime zombie tasks killed.
azure.datafactory_factories.factory_size_in_gb_units
(gauge)
Total factory size (GB unit).
azure.datafactory_factories.integration_runtime_available_memory
(gauge)
Integration runtime available memory.
Shown as byte
azure.datafactory_factories.integration_runtime_available_node_number
(gauge)
Integration runtime available node count.
azure.datafactory_factories.integration_runtime_average_task_pickup_delay
(gauge)
Integration runtime queue duration.
Shown as second
azure.datafactory_factories.integration_runtime_cpu_percentage
(gauge)
Integration runtime CPU utilization.
Shown as percent
azure.datafactory_factories.integration_runtime_queue_length
(gauge)
Integration runtime queue length.
azure.datafactory_factories.max_allowed_factory_size_in_gb_units
(gauge)
Maximum allowed factory size (GB unit).
azure.datafactory_factories.max_allowed_resource_count
(gauge)
Maximum allowed entities count.
azure.datafactory_factories.pipeline_cancelled_runs
(count)
Cancelled pipeline runs metrics.
azure.datafactory_factories.pipeline_elapsed_time_runs
(count)
Elapsed time pipeline runs metrics.
azure.datafactory_factories.pipeline_failed_runs
(count)
Failed pipeline runs metrics.
azure.datafactory_factories.pipeline_succeeded_runs
(count)
Succeeded pipeline runs metrics.
azure.datafactory_factories.resource_count
(gauge)
Total entities count.
azure.datafactory_factories.ssis_integration_runtime_start_cancel
(count)
Cancelled SSIS integration runtime start metrics.
azure.datafactory_factories.ssis_integration_runtime_start_failed
(count)
Failed SSIS integration runtime start metrics.
azure.datafactory_factories.ssis_integration_runtime_start_succeeded
(count)
Succeeded SSIS integration runtime start metrics.
azure.datafactory_factories.ssis_integration_runtime_stop_stuck
(count)
Stuck SSIS integration runtime stop metrics.
azure.datafactory_factories.ssis_integration_runtime_stop_succeeded
(count)
Succeeded SSIS integration runtime stop metrics.
azure.datafactory_factories.ssis_package_execution_cancel
(count)
Cancelled SSIS package execution metrics.
azure.datafactory_factories.ssis_package_execution_failed
(count)
Failed SSIS package execution metrics.
azure.datafactory_factories.ssis_package_execution_succeeded
(count)
Succeeded SSIS package execution metrics.
azure.datafactory_factories.trigger_cancelled_runs
(count)
Cancelled trigger runs metrics.
azure.datafactory_factories.trigger_failed_runs
(count)
Failed trigger runs metrics.
azure.datafactory_factories.trigger_succeeded_runs
(count)
Succeeded trigger runs metrics.
azure.datafactory_factories.count
(gauge)
Count of DataFactory factories.
azure.datafactory_factories.status
(gauge)
Status of DataFactory factories(deprecated).

イベント

Azure Data Factory インテグレーションには、イベントは含まれません。

サービスチェック

Azure Data Factory インテグレーションには、サービスのチェック機能は含まれません。

トラブルシューティング

ご不明な点は、Datadog のサポートチームまでお問合せください。

PREVIEWING: alai97/reorganize-some-sections-in-dora-metrics