- 필수 기능
- 시작하기
- Glossary
- 표준 속성
- Guides
- Agent
- 통합
- 개방형텔레메트리
- 개발자
- API
- Datadog Mobile App
- CoScreen
- Cloudcraft
- 앱 내
- 서비스 관리
- 인프라스트럭처
- 애플리케이션 성능
- APM
- Continuous Profiler
- 스팬 시각화
- 데이터 스트림 모니터링
- 데이터 작업 모니터링
- 디지털 경험
- 소프트웨어 제공
- 보안
- AI Observability
- 로그 관리
- 관리
Supported OS
This check monitors two parts of Hive: Hive Metastore and HiveServer2.
The Hive check is included in the Datadog Agent package. No additional installation is needed on your server.
Edit the Hive configuration file in HIVE_HOME/conf/hive-site.xml
to enable the Hive Metastore and HiveServer2 metrics by adding these properties:
<property>
<name>hive.metastore.metrics.enabled</name>
<value>true</value>
</property>
<property>
<name>hive.server2.metrics.enabled</name>
<value>true</value>
</property>
Enable a JMX remote connection for the HiveServer2 and/or for the Hive Metastore. For example, set the HADOOP_CLIENT_OPTS
environment variable:
export HADOOP_CLIENT_OPTS="$HADOOP_CLIENT_OPTS -Dcom.sun.management.jmxremote \
-Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false \
-Dcom.sun.management.jmxremote.port=8808"
Then restart the HiveServer2 or the Hive Metastore. Hive Metastore and HiveServer2 cannot share the same JMX connection.
To configure this check for an Agent running on a host:
Follow the instructions below to configure this check for an Agent running on a host. For containerized environments, see the Containerized section.
Edit the hive.d/conf.yaml
file, in the conf.d/
folder at the root of your Agent’s configuration directory to start collecting your hive performance data. See the sample hive.d/conf.yaml for all available configuration options.
This check has a limit of 350 metrics per instance. The number of returned metrics is indicated in the status page. You can specify the metrics you are interested in by editing the configuration below. To learn how to customize the metrics to collect, see the JMX Checks documentation for more detailed instructions. If you need to monitor more metrics, contact Datadog support.
Available for Agent versions >6.0
Collecting logs is disabled by default in the Datadog Agent, you need to enable it in datadog.yaml
:
logs_enabled: true
Add this configuration block to your hive.d/conf.yaml
file to start collecting your Hive logs:
logs:
- type: file
path: /tmp/<USER>/hive.log
source: hive
service: '<SERVICE_NAME>'
log_processing_rules:
- type: multi_line
name: new_log_start_with_date
pattern: \d{4}\-\d{2}\-\d{2}
Change the path
and service
parameter values and configure them for your environment. See the sample hive.d/conf.yaml for all available configuration options.
For containerized environments, see the Autodiscovery Integration Templates for guidance on applying the parameters below.
To collect metrics with the Datadog-Hive integration, see the Autodiscovery with JMX guide.
Available for Agent versions >6.0
Collecting logs is disabled by default in the Datadog Agent. To enable it, see Kubernetes Log Collection.
Parameter | Value |
---|---|
<LOG_CONFIG> | {"source": "hive", "service": "<SERVICE_NAME>", "log_processing_rules":{"type":"multi_line","name":"new_log_start_with_date", "pattern":"\d{4}\-\d{2}\-\d{2}"}} |
Run the Agent’s status subcommand and look for Hive
under the Checks section.
hive.metastore.api.alter_table (rate) | API call to alter table. Shown as task |
hive.metastore.api.alter_table.active_call (gauge) | Active API call to alter table. Shown as unit |
hive.metastore.api.create_table (rate) | API call to create a table. Shown as task |
hive.metastore.api.create_table.active_call (gauge) | Active API call to create a table. Shown as unit |
hive.metastore.api.drop_table (rate) | API call to drop a table. Shown as task |
hive.metastore.api.drop_table.active_call (gauge) | Active API call to drop a table. Shown as unit |
hive.metastore.api.flushcache (rate) | API flushcache. Shown as task |
hive.metastore.api.flushcache.active_call (gauge) | Active API flushcache. Shown as unit |
hive.metastore.api.get_all_databases (rate) | API call to get all databases. Shown as task |
hive.metastore.api.get_all_databases.active_call (gauge) | Active API call to get all databases. Shown as unit |
hive.metastore.api.get_all_functions (rate) | API call to get all functions. Shown as task |
hive.metastore.api.get_all_functions.active_call (gauge) | Active API call to get all functions. Shown as unit |
hive.metastore.api.get_all_tables (rate) | API call to get all tables. Shown as task |
hive.metastore.api.get_all_tables.active_call (gauge) | Active API call to get all tables. Shown as unit |
hive.metastore.api.get_database (rate) | API call to get a database. Shown as task |
hive.metastore.api.get_database.active_call (gauge) | Active API call to get a database. Shown as unit |
hive.metastore.api.get_table (rate) | API call to get a table. Shown as task |
hive.metastore.api.get_table.active_call (gauge) | Active API call to get a table. Shown as unit |
hive.metastore.api.get_table_req (rate) | Shown as task |
hive.metastore.api.get_table_req.active_call (gauge) | Shown as unit |
hive.metastore.api.init (rate) | API initialization. Shown as task |
hive.metastore.api.init.active_call (gauge) | Active API initialization. Shown as unit |
hive.metastore.api.shutdown (rate) | API shutdown. Shown as task |
hive.metastore.api.shutdown.active_call (gauge) | Active API shutdown. Shown as unit |
hive.metastore.db.created (rate) | Total number of created database. Shown as item |
hive.metastore.db.deleted (rate) | Total number of deleted database. Shown as item |
hive.metastore.db.init (rate) | Number of initialized database. Shown as item |
hive.metastore.directsql_errors (gauge) | Number of SQL error. Shown as unit |
hive.metastore.open_connections (gauge) | Number of connection opened. Shown as connection |
hive.metastore.partition.created (rate) | Total number of created partition. Shown as item |
hive.metastore.partition.deleted (rate) | Total number of deleted partition. Shown as item |
hive.metastore.partition.init (rate) | Number of initialized partition. Shown as item |
hive.metastore.table.created (rate) | Total number of created table. Shown as table |
hive.metastore.table.deleted (rate) | Total number of deleted table. Shown as table |
hive.metastore.table.init (rate) | Number of initialized table. Shown as table |
hive.server.api.operation.initialized.75percentile (gauge) | P75 time to init an operation. Shown as millisecond |
hive.server.api.operation.initialized.95percentile (gauge) | P95 time to init an operation. Shown as millisecond |
hive.server.api.operation.initialized.active_call (gauge) | Active initialized operation. Shown as user |
hive.server.api.operation.initialized.count (rate) | Number of operation initialized. Shown as operation |
hive.server.api.operation.initialized.max (gauge) | Max time to init an operation. Shown as millisecond |
hive.server.api.operation.initialized.mean (gauge) | Mean time to init an operation. Shown as millisecond |
hive.server.api.operation.initialized.meanrate (gauge) | Operation initialization rate. Shown as operation |
hive.server.api.operation.initialized.min (gauge) | Min time to init an operation. Shown as millisecond |
hive.server.api.operation.pending.75percentile (gauge) | P75 time in pending state for an sql operation. Shown as millisecond |
hive.server.api.operation.pending.95percentile (gauge) | P95 time in pending state for an sql operation. Shown as millisecond |
hive.server.api.operation.pending.active_call (gauge) | Active pending operation. Shown as operation |
hive.server.api.operation.pending.count (rate) | Number of operation in pending state. Shown as operation |
hive.server.api.operation.pending.max (gauge) | Max time in pending state for an sql operation. Shown as millisecond |
hive.server.api.operation.pending.mean (gauge) | Mean time in pending state for an sql operation. Shown as millisecond |
hive.server.api.operation.pending.meanrate (gauge) | Operation pending rate. Shown as operation |
hive.server.api.operation.pending.min (gauge) | Min time in pending state for an sql operation. Shown as millisecond |
hive.server.api.operation.running.75percentile (gauge) | P75 time in running state for an sql operation. Shown as millisecond |
hive.server.api.operation.running.95percentile (gauge) | P95 time in running state for an sql operation. Shown as millisecond |
hive.server.api.operation.running.active_call (gauge) | Active running operation. Shown as operation |
hive.server.api.operation.running.count (rate) | Number of operation in running state. Shown as millisecond |
hive.server.api.operation.running.max (gauge) | Max time in running state for an sql operation. Shown as millisecond |
hive.server.api.operation.running.mean (gauge) | Mean time in running state for an sql operation. Shown as millisecond |
hive.server.api.operation.running.meanrate (gauge) | Operation running rate. Shown as operation |
hive.server.api.operation.running.min (gauge) | Min time in running state for an sql operation. Shown as millisecond |
hive.server.api.queries.executing.active_call (gauge) | Active executing queries. Shown as query |
hive.server.api.sql_operation.pending.75percentile (gauge) | P75 time in pending state for an sql operation. Shown as millisecond |
hive.server.api.sql_operation.pending.95percentile (gauge) | P95 time in pending state for an sql operation. Shown as millisecond |
hive.server.api.sql_operation.pending.active_call (gauge) | Active pending sql operation. Shown as operation |
hive.server.api.sql_operation.pending.count (rate) | Number of sql operation in pending state. Shown as operation |
hive.server.api.sql_operation.pending.max (gauge) | Max time in pending state for an sql operation. Shown as millisecond |
hive.server.api.sql_operation.pending.mean (gauge) | Mean time in pending state for an sql operation. Shown as millisecond |
hive.server.api.sql_operation.pending.meanrate (gauge) | Pending sql operation rate. Shown as operation |
hive.server.api.sql_operation.pending.min (gauge) | Min time in pending state for an sql operation. Shown as millisecond |
hive.server.api.sql_operation.running.75percentile (gauge) | P75 time for running state for a sql operation. Shown as millisecond |
hive.server.api.sql_operation.running.95percentile (gauge) | P95 time for running state for a sql operation. Shown as millisecond |
hive.server.api.sql_operation.running.active_call (gauge) | Active running sql operation. Shown as operation |
hive.server.api.sql_operation.running.count (rate) | Number of sql operation in running state. Shown as operation |
hive.server.api.sql_operation.running.max (gauge) | Max time for running state for a sql operation. Shown as millisecond |
hive.server.api.sql_operation.running.mean (gauge) | Mean time for running state for a sql operation. Shown as millisecond |
hive.server.api.sql_operation.running.meanrate (gauge) | Running sql operation rate. Shown as operation |
hive.server.api.sql_operation.running.min (gauge) | Min time for running state for a sql operation. Shown as millisecond |
hive.server.memory.heap.committed (gauge) | Committed memory for the HiveServer2 or Metastore. Depends on the connected instance type Shown as byte |
hive.server.memory.heap.init (gauge) | Memory used at the initialization by the HiveServer2 or Metastore. Depends on the connected instance type Shown as byte |
hive.server.memory.heap.max (gauge) | Maximum memory that can be used by the HiveServer2 or Metastore. Depends on the connected instance type Shown as byte |
hive.server.memory.heap.used (gauge) | Memory used by the HiveServer2 or Metastore. Depends on the connected instance type Shown as byte |
hive.server.memory.non_heap.committed (gauge) | Committed memory for the HiveServer2 or Metastore. Depends on the connected instance type Shown as byte |
hive.server.memory.non_heap.init (gauge) | Memory used at the initialization by the HiveServer2 or Metastore. Depends on the connected instance type Shown as byte |
hive.server.memory.non_heap.max (gauge) | Maximum memory that can be used by the HiveServer2 or Metastore. Depends on the connected instance type Shown as byte |
hive.server.memory.non_heap.used (gauge) | Memory used by the HiveServer2 or Metastore. Depends on the connected instance type Shown as byte |
hive.server.memory.total.committed (gauge) | Total committed memory for the HiveServer2 or Metastore. Depends on the connected instance type Shown as byte |
hive.server.memory.total.init (gauge) | Total memory at the initialization by the HiveServer2 or Metastore. Depends on the connected instance type Shown as byte |
hive.server.memory.total.max (gauge) | Total maximum memory that can be used for the HiveServer2 or Metastore. Depends on the connected instance type Shown as byte |
hive.server.memory.total.used (gauge) | Total memory used by the HiveServer2 or Metastore. Depends on the connected instance type Shown as byte |
hive.server.open_operations (gauge) | Operation opened in the HiveServer2. Shown as operation |
hive.server.operation.completed.closed (rate) | Number of closed operation. Shown as operation |
hive.server.operation.completed.finished (rate) | Number of finished operation. Shown as operation |
hive.server.queries.compiling.75percentile (gauge) | P75 time for compiling a query. Shown as millisecond |
hive.server.queries.compiling.95percentile (gauge) | P95 time for compiling a query. Shown as millisecond |
hive.server.queries.compiling.active_call (gauge) | Active compiling queries. Shown as query |
hive.server.queries.compiling.count (rate) | Number of compiled query. Shown as query |
hive.server.queries.compiling.max (gauge) | Max time for compiling a query. Shown as millisecond |
hive.server.queries.compiling.mean (gauge) | Mean time for compiling a query. Shown as millisecond |
hive.server.queries.compiling.meanrate (gauge) | Compiling query rate. Shown as query |
hive.server.queries.compiling.min (gauge) | Min time for compiling a query. Shown as millisecond |
hive.server.queries.executing.75percentile (gauge) | P75 time for executing a query. Shown as millisecond |
hive.server.queries.executing.95percentile (gauge) | P95 time for executing a query. Shown as millisecond |
hive.server.queries.executing.count (rate) | Number of executed queries. Shown as query |
hive.server.queries.executing.max (gauge) | Max time for executing a query. Shown as millisecond |
hive.server.queries.executing.mean (gauge) | Mean time for executing a query. Shown as millisecond |
hive.server.queries.executing.meanrate (gauge) | Executing query rate. Shown as query |
hive.server.queries.executing.min (gauge) | Min time for executing a query. Shown as millisecond |
hive.server.queries.submitted.75percentile (gauge) | P75 time for a submitted query. Shown as millisecond |
hive.server.queries.submitted.95percentile (gauge) | P95 time for a submitted query. Shown as millisecond |
hive.server.queries.submitted.active_call (gauge) | Active submitted queries. Shown as query |
hive.server.queries.submitted.count (rate) | Number of submitted query. Shown as query |
hive.server.queries.submitted.max (gauge) | Max time for a submitted query. Shown as millisecond |
hive.server.queries.submitted.mean (gauge) | Mean time for a submitted query. Shown as millisecond |
hive.server.queries.submitted.meanrate (gauge) | Submitted query rate. Shown as query |
hive.server.queries.submitted.min (gauge) | Min time for a submitted query. Shown as millisecond |
hive.server.queries.succeeded.count (rate) | Number of succeeded queries. Shown as query |
hive.server.queries.succeeded.meanrate (gauge) | Succeeded queries rate. Shown as query |
hive.server.session.active (gauge) | Number of active session. Shown as session |
hive.server.session.active.time_mean (gauge) | Average time a session has been active. Shown as millisecond |
hive.server.session.open (gauge) | Number of opened session. Shown as session |
hive.server.session.open.time_mean (gauge) | Average time a session has been opened. Shown as millisecond |
hive.server.sql_operation.completed.closed (rate) | Number of closed sql operation. Shown as operation |
hive.server.sql_operation.completed.finished (rate) | Number of finished sql operation. Shown as operation |
hive.server.sql_operation.user.active (gauge) | Number of active user. Shown as user |
The Hive check does not include any events.
hive.can_connect
Returns CRITICAL
if the Agent is unable to connect to and collect metrics from the monitored HiveServer2/Hive Metastore instance, WARNING
if no metrics are collected, and OK
otherwise.
Statuses: ok, critical, warning
Need help? Contact Datadog support.