ZooKeeper

Supported OS Linux Mac OS

통합 버전6.0.0
이 페이지는 아직 한국어로 제공되지 않으며 번역 작업 중입니다. 번역에 관한 질문이나 의견이 있으시면 언제든지 저희에게 연락해 주십시오.

ZooKeeper Dashboard

Overview

The ZooKeeper check tracks client connections and latencies, monitors the number of unprocessed requests, and more.

Setup

Installation

The ZooKeeper check is included in the Datadog Agent package, so you don’t need to install anything else on your ZooKeeper servers.

Configuration

Inclusion list

As of version 3.5, ZooKeeper has a 4lw.commands.whitelist parameter. See ZooKeeper Cluster Options) for an example that allows four letter word commands. By default, only srvr is whitelisted. Add stat and mntr to the whitelist, as the integration is based on these commands.

Enabling SSL

ZooKeeper 3.5 introduced the ability to use SSL authentication. For information about setting up SSL with ZooKeeper, see the ZooKeeper SSL User Guide.

After you have ZooKeeper set up with SSL, you can also configure the Datadog Agent to connect to ZooKeeper using SSL. If you already have authentication set up using JKS files, follow the steps below to convert them to PEM files for TLS/SSL configuration.

The following example commands assume that your JKS truststore and keystore files are called:

  • server_truststore.jks
  • server_keystore.jks
  • client_truststore.jks
  • client_keystore.jks

It is also assumed that both sides’ keystore and truststore files have each other’s certificates with aliases server_cert and client_cert, meaning that a Java ZooKeeper client can already connect to a ZooKeeper server. If your private key has a password, make sure this password is included in the config.yaml file for config option tls_private_key_password.

To convert the JKS files to PEM files:

  1. Get the ca_cert.pem file from client_truststore.jks, since the client’s truststore contains the certificate of the server that is trustable:

    keytool -exportcert -file ca_cert.pem -keystore client_truststore.jks -alias server_cert -rfc

  2. Get the cert.pem file from client_keystore.jks, since the client’s keystore contains the cert of the client for alias client_cert:

    keytool -importkeystore -srckeystore client_keystore.jks -destkeystore cert.p12 -srcstoretype jks -deststoretype pkcs12 -srcalias client_cert

  3. Run the openssl pkcs12 command, which exports both the client cert and the private key for the certificate. The tls_cert config option is able to read and parse the PEM file which contains both the cert and private key. Add -nodes to this command if you want to get a non-password-protected file:

    openssl pkcs12 -in cert.p12 -out cert.pem

    Host

    To configure this check for an Agent running on a host:

    1. Edit the zk.d/conf.yaml file, in the conf.d/ folder at the root of your Agent’s configuration directory to start collecting your ZooKeeper metrics and logs. See the sample zk.d/conf.yaml for all available configuration options.

    2. Restart the Agent.

    Log collection

    Available for Agent versions >6.0

    1. ZooKeeper uses the log4j logger per default. To activate the logging into a file and customize the format edit the log4j.properties file:

        # Set root logger level to INFO and its only appender to R
  log4j.rootLogger=INFO, R
  log4j.appender.R.File=/var/log/zookeeper.log
  log4j.appender.R.layout=org.apache.log4j.PatternLayout
  log4j.appender.R.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %-5p [%t] %c{1}:%L - %m%n

    2. By default, Datadog’s integration pipeline supports the following conversion patterns:

        %d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L - %m%n
  %d [%t] %-5p %c - %m%n
  %r [%t] %p %c %x - %m%n

      Make sure you clone and edit the integration pipeline if you have a different format.

    3. Collecting logs is disabled by default in the Datadog Agent, enable it in your datadog.yaml file:

      logs_enabled: true

    4. Uncomment and edit this configuration block at the bottom of your zk.d/conf.yaml:

      logs:
  - type: file
    path: /var/log/zookeeper.log
    source: zookeeper
    service: myapp
    #To handle multi line that starts with yyyy-mm-dd use the following pattern
    #log_processing_rules:
    #  - type: multi_line
    #    name: log_start_with_date
    #    pattern: \d{4}\-(0?[1-9]|1[012])\-(0?[1-9]|[12][0-9]|3[01])

      Change the path and service parameter values and configure them for your environment. See the sample zk.d/conf.yaml for all available configuration options.

    5. Restart the Agent.

    Containerized

    For containerized environments, see the Autodiscovery Integration Templates for guidance on applying the parameters below.

    Metric collection
    ParameterValue
    <INTEGRATION_NAME>zk
    <INIT_CONFIG>blank or {}
    <INSTANCE_CONFIG>{"host": "%%host%%", "port": "2181"}
    Log collection

    Available for Agent versions >6.0

    Collecting logs is disabled by default in the Datadog Agent. To enable it, see Kubernetes Log Collection.

    ParameterValue
    <LOG_CONFIG>{"source": "zookeeper", "service": "<SERVICE_NAME>"}

    Validation

    Run the Agent’s status subcommand and look for zk under the Checks section.

    Data Collected

    Metrics

    zookeeper.add_dead_watcher_stall_time
    (gauge)    		[zookeeper version 3.6+ only]
    zookeeper.approximate_data_size
    (gauge)
    zookeeper.avg_latency
    (gauge)    		The amount of time it takes for the server to respond to a client request.
    Shown as millisecond
    zookeeper.bytes_received
    (gauge)    		Number of bytes received
    zookeeper.bytes_received_count
    (gauge)    		[zookeeper version 3.6+ only] The number of bytes received
    Shown as byte
    zookeeper.bytes_sent
    (gauge)    		Number of bytes sent
    zookeeper.close_session_prep_time
    (gauge)    		[zookeeper version 3.6+ only] Histogram of closesessionprep_time
    zookeeper.close_session_prep_time_count
    (gauge)    		[zookeeper version 3.6+ only] Total count of closesessionprep_time
    zookeeper.close_session_prep_time_sum
    (gauge)    		[zookeeper version 3.6+ only] Sum of closesessionprep_time
    zookeeper.commit_commit_proc_req_queued
    (gauge)    		[zookeeper version 3.6+ only] Histogram of commitcommitprocreqqueued
    zookeeper.commit_commit_proc_req_queued_count
    (gauge)    		[zookeeper version 3.6+ only] Total count of commitcommitprocreqqueued
    zookeeper.commit_commit_proc_req_queued_sum
    (gauge)    		[zookeeper version 3.6+ only] Sum of commitcommitprocreqqueued
    zookeeper.commit_count
    (gauge)    		[zookeeper version 3.6+ only] The number of commits performed on leader
    zookeeper.commit_process_time
    (gauge)    		[zookeeper version 3.6+ only] Histogram of commitprocesstime
    zookeeper.commit_process_time_count
    (gauge)    		[zookeeper version 3.6+ only] Total count of commitprocesstime
    zookeeper.commit_process_time_sum
    (gauge)    		[zookeeper version 3.6+ only] Sum of commitprocesstime
    zookeeper.commit_propagation_latency
    (gauge)    		[zookeeper version 3.6+ only] Histogram of commitpropagationlatency
    zookeeper.commit_propagation_latency_count
    (gauge)    		[zookeeper version 3.6+ only] Total count of commitpropagationlatency
    zookeeper.commit_propagation_latency_sum
    (gauge)    		[zookeeper version 3.6+ only] Sum of commitpropagationlatency
    zookeeper.concurrent_request_processing_in_commit_processor
    (gauge)    		[zookeeper version 3.6+ only] Histogram of concurrentrequestprocessingincommit_processor
    zookeeper.concurrent_request_processing_in_commit_processor_count
    (gauge)    		[zookeeper version 3.6+ only] Total count of concurrentrequestprocessingincommit_processor
    zookeeper.concurrent_request_processing_in_commit_processor_sum
    (gauge)    		[zookeeper version 3.6+ only] Sum of concurrentrequestprocessingincommit_processor
    zookeeper.connection_drop_count
    (gauge)    		[zookeeper version 3.6+ only] Count of connection drops
    zookeeper.connection_drop_probability
    (gauge)    		[zookeeper version 3.6+ only] Connection drop probability
    zookeeper.connection_rejected
    (gauge)    		[zookeeper version 3.6+ only] Connection rejected counts
    zookeeper.connection_request_count
    (gauge)    		[zookeeper version 3.6+ only] Number of incoming client connection requests
    zookeeper.connection_revalidate_count
    (gauge)    		[zookeeper version 3.6+ only] Count of connection revalidations
    zookeeper.connection_token_deficit
    (gauge)    		[zookeeper version 3.6+ only] Histogram of connectiontokendeficit
    zookeeper.connection_token_deficit_count
    (gauge)    		[zookeeper version 3.6+ only] Total count of connectiontokendeficit
    zookeeper.connection_token_deficit_sum
    (gauge)    		[zookeeper version 3.6+ only] Sum of connectiontokendeficit
    zookeeper.connections
    (gauge)    		The total count of client connections.
    Shown as connection
    zookeeper.datadog_client_exception
    (rate)    		The exception rate seen by the Datadog Agent when trying to collect stats.
    Shown as error
    zookeeper.dbinittime
    (gauge)    		[zookeeper version 3.6+ only] Histogram of dbinittime Time to reload database
    zookeeper.dbinittime_count
    (gauge)    		[zookeeper version 3.6+ only] Total count of dbinittime Time to reload database
    zookeeper.dbinittime_sum
    (gauge)    		[zookeeper version 3.6+ only] Sum of dbinittime Time to reload database
    zookeeper.dead_watchers_cleaner_latency
    (gauge)    		[zookeeper version 3.6+ only] Histogram of deadwatcherscleaner_latency
    zookeeper.dead_watchers_cleaner_latency_count
    (gauge)    		[zookeeper version 3.6+ only] Total count of deadwatcherscleaner_latency
    zookeeper.dead_watchers_cleaner_latency_sum
    (gauge)    		[zookeeper version 3.6+ only] Sum of deadwatcherscleaner_latency
    zookeeper.dead_watchers_cleared
    (gauge)    		[zookeeper version 3.6+ only]
    zookeeper.dead_watchers_queued
    (gauge)    		[zookeeper version 3.6+ only]
    zookeeper.diff_count
    (gauge)    		[zookeeper version 3.6+ only] Number of diff syncs performed
    zookeeper.digest_mismatches_count
    (gauge)    		[zookeeper version 3.6+ only]
    zookeeper.election_time
    (gauge)    		[zookeeper version 3.6+ only] Time between entering and leaving election
    zookeeper.election_time_count
    (gauge)    		[zookeeper version 3.6+ only] Time between entering and leaving election
    zookeeper.election_time_sum
    (gauge)    		[zookeeper version 3.6+ only] Time between entering and leaving election
    zookeeper.ensemble_auth_fail
    (gauge)    		[zookeeper version 3.6+ only]
    zookeeper.ensemble_auth_skip
    (gauge)    		[zookeeper version 3.6+ only]
    zookeeper.ensemble_auth_success
    (gauge)    		[zookeeper version 3.6+ only]
    zookeeper.ephemerals_count
    (gauge)
    zookeeper.follower_sync_time
    (gauge)    		[zookeeper version 3.6+ only] Time for follower to sync with leader
    zookeeper.follower_sync_time_count
    (gauge)    		[zookeeper version 3.6+ only] Time for follower to sync with leader
    zookeeper.follower_sync_time_sum
    (gauge)    		[zookeeper version 3.6+ only] Time for follower to sync with leader
    zookeeper.fsynctime
    (gauge)    		[zookeeper version 3.6+ only] Time to fsync transaction log
    zookeeper.fsynctime_count
    (gauge)    		[zookeeper version 3.6+ only] Time to fsync transaction log
    zookeeper.fsynctime_sum
    (gauge)    		[zookeeper version 3.6+ only] Time to fsync transaction log
    zookeeper.global_sessions
    (gauge)    		[zookeeper version 3.6+ only] Count of global sessions
    zookeeper.instances
    (gauge)
    zookeeper.jvm_buffer_pool_capacity_bytes
    (gauge)    		[zookeeper version 3.6+ only]
    zookeeper.jvm_buffer_pool_used_buffers
    (gauge)    		[zookeeper version 3.6+ only]
    zookeeper.jvm_buffer_pool_used_bytes
    (gauge)    		[zookeeper version 3.6+ only]
    zookeeper.jvm_classes_loaded
    (gauge)    		[zookeeper version 3.6+ only]
    zookeeper.jvm_classes_loaded_total
    (gauge)    		[zookeeper version 3.6+ only]
    zookeeper.jvm_classes_unloaded_total
    (gauge)    		[zookeeper version 3.6+ only]
    zookeeper.jvm_gc_collection_seconds_count
    (gauge)    		[zookeeper version 3.6+ only]
    zookeeper.jvm_gc_collection_seconds_sum
    (gauge)    		[zookeeper version 3.6+ only]
    zookeeper.jvm_info
    (gauge)    		[zookeeper version 3.6+ only]
    zookeeper.jvm_memory_bytes_committed
    (gauge)    		[zookeeper version 3.6+ only]
    zookeeper.jvm_memory_bytes_init
    (gauge)    		[zookeeper version 3.6+ only]
    zookeeper.jvm_memory_bytes_max
    (gauge)    		[zookeeper version 3.6+ only]
    zookeeper.jvm_memory_bytes_used
    (gauge)    		[zookeeper version 3.6+ only]
    zookeeper.jvm_memory_pool_allocated_bytes_total
    (gauge)    		[zookeeper version 3.6+ only]
    zookeeper.jvm_memory_pool_bytes_committed
    (gauge)    		[zookeeper version 3.6+ only]
    zookeeper.jvm_memory_pool_bytes_init
    (gauge)    		[zookeeper version 3.6+ only]
    Shown as byte
    zookeeper.jvm_memory_pool_bytes_max
    (gauge)    		[zookeeper version 3.6+ only]
    Shown as byte
    zookeeper.jvm_memory_pool_bytes_used
    (gauge)    		[zookeeper version 3.6+ only]
    Shown as byte
    zookeeper.jvm_threads_current
    (gauge)    		[zookeeper version 3.6+ only]
    zookeeper.jvm_threads_daemon
    (gauge)    		[zookeeper version 3.6+ only]
    zookeeper.jvm_threads_deadlocked
    (gauge)    		[zookeeper version 3.6+ only]
    zookeeper.jvm_threads_deadlocked_monitor
    (gauge)    		[zookeeper version 3.6+ only]
    zookeeper.jvm_threads_peak
    (gauge)    		[zookeeper version 3.6+ only]
    zookeeper.jvm_threads_started_total
    (gauge)    		[zookeeper version 3.6+ only]
    zookeeper.jvm_threads_state
    (gauge)    		[zookeeper version 3.6+ only]
    zookeeper.large_requests_rejected
    (gauge)    		[zookeeper version 3.6+ only]
    zookeeper.last_client_response_size
    (gauge)    		[zookeeper version 3.6+ only]
    zookeeper.latency.avg
    (gauge)    		The amount of time it takes for the server to respond to a client request.
    Shown as millisecond
    zookeeper.latency.max
    (gauge)    		The amount of time it takes for the server to respond to a client request.
    Shown as millisecond
    zookeeper.latency.min
    (gauge)    		The amount of time it takes for the server to respond to a client request.
    Shown as millisecond
    zookeeper.learner_commit_received_count
    (gauge)    		[zookeeper version 3.6+ only]
    zookeeper.learner_proposal_received_count
    (gauge)    		[zookeeper version 3.6+ only]
    zookeeper.local_sessions
    (gauge)    		[zookeeper version 3.6+ only] Count of local sessions
    zookeeper.local_write_committed_time_ms
    (gauge)    		[zookeeper version 3.6+ only] Histogram of localwritecommittedtimems
    zookeeper.local_write_committed_time_ms_count
    (gauge)    		[zookeeper version 3.6+ only] Total count of localwritecommittedtimems
    zookeeper.local_write_committed_time_ms_sum
    (gauge)    		[zookeeper version 3.6+ only] Sum of localwritecommittedtimems
    zookeeper.looking_count
    (gauge)    		[zookeeper version 3.6+ only] Number of transitions into looking state
    zookeeper.max_client_response_size
    (gauge)    		[zookeeper version 3.6+ only]
    zookeeper.max_file_descriptor_count
    (gauge)
    zookeeper.max_latency
    (gauge)    		The amount of time it takes for the server to respond to a client request.
    Shown as millisecond
    zookeeper.min_client_response_size
    (gauge)    		[zookeeper version 3.6+ only]
    zookeeper.min_latency
    (gauge)    		The amount of time it takes for the server to respond to a client request.
    Shown as millisecond
    zookeeper.netty_queued_buffer_capacity
    (gauge)    		[zookeeper version 3.6+ only] Histogram of nettyqueuedbuffer_capacity
    zookeeper.netty_queued_buffer_capacity_count
    (gauge)    		[zookeeper version 3.6+ only] Total count of nettyqueuedbuffer_capacity
    zookeeper.netty_queued_buffer_capacity_sum
    (gauge)    		[zookeeper version 3.6+ only] Sum of nettyqueuedbuffer_capacity
    zookeeper.node_changed_watch_count
    (gauge)    		[zookeeper version 3.6+ only] Histogram of nodechangedwatch_count
    zookeeper.node_changed_watch_count_count
    (gauge)    		[zookeeper version 3.6+ only] Total count of nodechangedwatch_count
    zookeeper.node_changed_watch_count_sum
    (gauge)    		[zookeeper version 3.6+ only] Sum of nodechangedwatch_count
    zookeeper.node_children_watch_count
    (gauge)    		[zookeeper version 3.6+ only] Histogram of nodechildrenwatch_count
    zookeeper.node_children_watch_count_count
    (gauge)    		[zookeeper version 3.6+ only] Total count of nodechildrenwatch_count
    zookeeper.node_children_watch_count_sum
    (gauge)    		[zookeeper version 3.6+ only] Sum of nodechildrenwatch_count
    zookeeper.node_created_watch_count
    (gauge)    		[zookeeper version 3.6+ only] Histogram of nodecreatedwatch_count
    zookeeper.node_created_watch_count_count
    (gauge)    		[zookeeper version 3.6+ only] Total count of nodecreatedwatch_count
    zookeeper.node_created_watch_count_sum
    (gauge)    		[zookeeper version 3.6+ only] Sum of nodecreatedwatch_count
    zookeeper.node_deleted_watch_count
    (gauge)    		[zookeeper version 3.6+ only] Histogram of nodedeletedwatch_count
    zookeeper.node_deleted_watch_count_count
    (gauge)    		[zookeeper version 3.6+ only] Total count of nodedeletedwatch_count
    zookeeper.node_deleted_watch_count_sum
    (gauge)    		[zookeeper version 3.6+ only] Sum of nodedeletedwatch_count
    zookeeper.nodes
    (gauge)    		The number of znodes in the ZooKeeper namespace (the data).
    Shown as node
    zookeeper.num_alive_connections
    (gauge)    		The total count of client connections.
    Shown as connection
    zookeeper.om_commit_process_time_ms
    (gauge)    		[zookeeper version 3.6+ only] Histogram of omcommitprocesstimems
    zookeeper.om_commit_process_time_ms_count
    (gauge)    		[zookeeper version 3.6+ only] Total count of omcommitprocesstimems
    zookeeper.om_commit_process_time_ms_sum
    (gauge)    		[zookeeper version 3.6+ only] Sum of omcommitprocesstimems
    zookeeper.om_proposal_process_time_ms
    (gauge)    		[zookeeper version 3.6+ only] Histogram of omproposalprocesstimems
    zookeeper.om_proposal_process_time_ms_count
    (gauge)    		[zookeeper version 3.6+ only] Total count of omproposalprocesstimems
    zookeeper.om_proposal_process_time_ms_sum
    (gauge)    		[zookeeper version 3.6+ only] Sum of omproposalprocesstimems
    zookeeper.open_file_descriptor_count
    (gauge)
    zookeeper.outstanding_changes_queued
    (gauge)    		[zookeeper version 3.6+ only]
    zookeeper.outstanding_changes_removed
    (gauge)    		[zookeeper version 3.6+ only]
    zookeeper.outstanding_requests
    (gauge)    		The number of queued requests when the server is under load and is receiving more sustained requests than it can process.
    Shown as request
    zookeeper.outstanding_tls_handshake
    (gauge)    		[zookeeper version 3.6+ only]
    zookeeper.packets.received
    (gauge)    		The number of packets received.
    Shown as packet
    zookeeper.packets.sent
    (gauge)    		The number of packets sent.
    Shown as packet
    zookeeper.packets_received
    (gauge)    		The number of packets received.
    Shown as packet
    zookeeper.packets_sent
    (gauge)    		The number of packets sent.
    Shown as packet
    zookeeper.pending_session_queue_size
    (gauge)    		[zookeeper version 3.6+ only] Histogram of pendingsessionqueue_size
    zookeeper.pending_session_queue_size_count
    (gauge)    		[zookeeper version 3.6+ only] Total count of pendingsessionqueue_size
    zookeeper.pending_session_queue_size_sum
    (gauge)    		[zookeeper version 3.6+ only] Sum of pendingsessionqueue_size
    zookeeper.prep_process_time
    (gauge)    		[zookeeper version 3.6+ only] Histogram of prepprocesstime
    zookeeper.prep_process_time_count
    (gauge)    		[zookeeper version 3.6+ only] Total count of prepprocesstime
    zookeeper.prep_process_time_sum
    (gauge)    		[zookeeper version 3.6+ only] Sum of prepprocesstime
    zookeeper.prep_processor_queue_size
    (gauge)    		[zookeeper version 3.6+ only] Histogram of prepprocessorqueue_size
    zookeeper.prep_processor_queue_size_count
    (gauge)    		[zookeeper version 3.6+ only] Total count of prepprocessorqueue_size
    zookeeper.prep_processor_queue_size_sum
    (gauge)    		[zookeeper version 3.6+ only] Sum of prepprocessorqueue_size
    zookeeper.prep_processor_queue_time_ms
    (gauge)    		[zookeeper version 3.6+ only] Histogram of prepprocessorqueuetimems
    zookeeper.prep_processor_queue_time_ms_count
    (gauge)    		[zookeeper version 3.6+ only] Total count of prepprocessorqueuetimems
    zookeeper.prep_processor_queue_time_ms_sum
    (gauge)    		[zookeeper version 3.6+ only] Sum of prepprocessorqueuetimems
    zookeeper.prep_processor_request_queued
    (gauge)    		[zookeeper version 3.6+ only]
    zookeeper.process_cpu_seconds_total
    (gauge)    		[zookeeper version 3.6+ only]
    zookeeper.process_max_fds
    (gauge)    		[zookeeper version 3.6+ only]
    zookeeper.process_open_fds
    (gauge)    		[zookeeper version 3.6+ only]
    zookeeper.process_resident_memory_bytes
    (gauge)    		[zookeeper version 3.6+ only]
    Shown as byte
    zookeeper.process_start_time_seconds
    (gauge)    		[zookeeper version 3.6+ only]
    Shown as second
    zookeeper.process_virtual_memory_bytes
    (gauge)    		[zookeeper version 3.6+ only]
    Shown as byte
    zookeeper.propagation_latency
    (gauge)    		[zookeeper version 3.6+ only] End-to-end latency for updates, from proposal on leader to committed-to-datatree on a given host
    zookeeper.propagation_latency_count
    (gauge)    		[zookeeper version 3.6+ only] End-to-end latency for updates, from proposal on leader to committed-to-datatree on a given host
    zookeeper.propagation_latency_sum
    (gauge)    		[zookeeper version 3.6+ only] End-to-end latency for updates, from proposal on leader to committed-to-datatree on a given host
    zookeeper.proposal_ack_creation_latency
    (gauge)    		[zookeeper version 3.6+ only] Histogram of proposalackcreation_latency
    zookeeper.proposal_ack_creation_latency_count
    (gauge)    		[zookeeper version 3.6+ only] Total count of proposalackcreation_latency
    zookeeper.proposal_ack_creation_latency_sum
    (gauge)    		[zookeeper version 3.6+ only] Sum of proposalackcreation_latency
    zookeeper.proposal_count
    (gauge)    		[zookeeper version 3.6+ only]
    zookeeper.proposal_latency
    (gauge)    		[zookeeper version 3.6+ only] Histogram of proposal_latency
    zookeeper.proposal_latency_count
    (gauge)    		[zookeeper version 3.6+ only] Total count of proposal_latency
    zookeeper.proposal_latency_sum
    (gauge)    		[zookeeper version 3.6+ only] Sum of proposal_latency
    zookeeper.quit_leading_due_to_disloyal_voter
    (gauge)    		[zookeeper version 3.6+ only]
    zookeeper.quorum_ack_latency
    (gauge)    		[zookeeper version 3.6+ only] Histogram of quorumacklatency
    zookeeper.quorum_ack_latency_count
    (gauge)    		[zookeeper version 3.6+ only] Total count of quorumacklatency
    zookeeper.quorum_ack_latency_sum
    (gauge)    		[zookeeper version 3.6+ only] Sum of quorumacklatency
    zookeeper.read_commit_proc_issued
    (gauge)    		[zookeeper version 3.6+ only] Histogram of readcommitproc_issued
    zookeeper.read_commit_proc_issued_count
    (gauge)    		[zookeeper version 3.6+ only] Total count of readcommitproc_issued
    zookeeper.read_commit_proc_issued_sum
    (gauge)    		[zookeeper version 3.6+ only] Sum of readcommitproc_issued
    zookeeper.read_commit_proc_req_queued
    (gauge)    		[zookeeper version 3.6+ only] Histogram of readcommitprocreqqueued
    zookeeper.read_commit_proc_req_queued_count
    (gauge)    		[zookeeper version 3.6+ only] Total count of readcommitprocreqqueued
    zookeeper.read_commit_proc_req_queued_sum
    (gauge)    		[zookeeper version 3.6+ only] Sum of readcommitprocreqqueued
    zookeeper.read_commitproc_time_ms
    (gauge)    		[zookeeper version 3.6+ only] Histogram of readcommitproctime_ms
    zookeeper.read_commitproc_time_ms_count
    (gauge)    		[zookeeper version 3.6+ only] Total count of readcommitproctime_ms
    zookeeper.read_commitproc_time_ms_sum
    (gauge)    		[zookeeper version 3.6+ only] Sum of readcommitproctime_ms
    zookeeper.read_final_proc_time_ms
    (gauge)    		[zookeeper version 3.6+ only] Histogram of readfinalproctimems
    zookeeper.read_final_proc_time_ms_count
    (gauge)    		[zookeeper version 3.6+ only] Total count of readfinalproctimems
    zookeeper.read_final_proc_time_ms_sum
    (gauge)    		[zookeeper version 3.6+ only] Sum of readfinalproctimems
    zookeeper.readlatency
    (gauge)    		[zookeeper version 3.6+ only] Histogram of readlatency Read request latency
    zookeeper.readlatency_count
    (gauge)    		[zookeeper version 3.6+ only] Total count of readlatency Read request latency
    zookeeper.readlatency_sum
    (gauge)    		[zookeeper version 3.6+ only] Sum of readlatency Read request latency
    zookeeper.reads_after_write_in_session_queue
    (gauge)    		[zookeeper version 3.6+ only] Histogram of readsafterwriteinsession_queue
    zookeeper.reads_after_write_in_session_queue_count
    (gauge)    		[zookeeper version 3.6+ only] Total count of readsafterwriteinsession_queue
    zookeeper.reads_after_write_in_session_queue_sum
    (gauge)    		[zookeeper version 3.6+ only] Sum of readsafterwriteinsession_queue
    zookeeper.reads_issued_from_session_queue
    (gauge)    		[zookeeper version 3.6+ only] Histogram of readsissuedfromsessionqueue
    zookeeper.reads_issued_from_session_queue_count
    (gauge)    		[zookeeper version 3.6+ only] Total count of readsissuedfromsessionqueue
    zookeeper.reads_issued_from_session_queue_sum
    (gauge)    		[zookeeper version 3.6+ only] Sum of readsissuedfromsessionqueue
    zookeeper.request_commit_queued
    (gauge)    		[zookeeper version 3.6+ only] Count of request commits queued
    zookeeper.request_throttle_wait_count
    (gauge)    		[zookeeper version 3.6+ only]
    zookeeper.requests_in_session_queue
    (gauge)    		[zookeeper version 3.6+ only] Histogram of requestsinsession_queue
    zookeeper.requests_in_session_queue_count
    (gauge)    		[zookeeper version 3.6+ only] Total count of requestsinsession_queue
    zookeeper.requests_in_session_queue_sum
    (gauge)    		[zookeeper version 3.6+ only] Sum of requestsinsession_queue
    zookeeper.response_packet_cache_hits
    (gauge)    		[zookeeper version 3.6+ only]
    zookeeper.response_packet_cache_misses
    (gauge)    		[zookeeper version 3.6+ only]
    zookeeper.response_packet_get_children_cache_hits
    (gauge)    		[zookeeper version 3.6+ only]
    zookeeper.response_packet_get_children_cache_misses
    (gauge)    		[zookeeper version 3.6+ only]
    zookeeper.revalidate_count
    (gauge)    		[zookeeper version 3.6+ only]
    zookeeper.server_state
    (gauge)
    zookeeper.server_write_committed_time_ms
    (gauge)    		[zookeeper version 3.6+ only] Histogram of serverwritecommittedtimems
    zookeeper.server_write_committed_time_ms_count
    (gauge)    		[zookeeper version 3.6+ only] Total count of serverwritecommittedtimems
    zookeeper.server_write_committed_time_ms_sum
    (gauge)    		[zookeeper version 3.6+ only] Sum of serverwritecommittedtimems
    zookeeper.session_queues_drained
    (gauge)    		[zookeeper version 3.6+ only] Histogram of sessionqueuesdrained
    zookeeper.session_queues_drained_count
    (gauge)    		[zookeeper version 3.6+ only] Total count of sessionqueuesdrained
    zookeeper.session_queues_drained_sum
    (gauge)    		[zookeeper version 3.6+ only] Sum of sessionqueuesdrained
    zookeeper.sessionless_connections_expired
    (gauge)    		[zookeeper version 3.6+ only]
    zookeeper.snap_count
    (gauge)    		[zookeeper version 3.6+ only] Number of snap syncs performed
    zookeeper.snapshottime
    (gauge)    		[zookeeper version 3.6+ only] Histogram of snapshottime Time to write a snapshot
    zookeeper.snapshottime_count
    (gauge)    		[zookeeper version 3.6+ only] Total count of snapshottime Time to write a snapshot
    zookeeper.snapshottime_sum
    (gauge)    		[zookeeper version 3.6+ only] Sum of snapshottime Time to write a snapshot
    zookeeper.stale_replies
    (gauge)    		[zookeeper version 3.6+ only]
    zookeeper.stale_requests
    (gauge)    		[zookeeper version 3.6+ only]
    zookeeper.stale_requests_dropped
    (gauge)    		[zookeeper version 3.6+ only]
    zookeeper.stale_sessions_expired
    (gauge)    		[zookeeper version 3.6+ only]
    zookeeper.startup_snap_load_time
    (gauge)    		[zookeeper version 3.6+ only] Histogram of startupsnapload_time
    zookeeper.startup_snap_load_time_count
    (gauge)    		[zookeeper version 3.6+ only] Total count of startupsnapload_time
    zookeeper.startup_snap_load_time_sum
    (gauge)    		[zookeeper version 3.6+ only] Sum of startupsnapload_time
    zookeeper.startup_txns_load_time
    (gauge)    		[zookeeper version 3.6+ only] Histogram of startuptxnsload_time
    zookeeper.startup_txns_load_time_count
    (gauge)    		[zookeeper version 3.6+ only] Total count of startuptxnsload_time
    zookeeper.startup_txns_load_time_sum
    (gauge)    		[zookeeper version 3.6+ only] Sum of startuptxnsload_time
    zookeeper.startup_txns_loaded
    (gauge)    		[zookeeper version 3.6+ only] Histogram of startuptxnsloaded
    zookeeper.startup_txns_loaded_count
    (gauge)    		[zookeeper version 3.6+ only] Total count of startuptxnsloaded
    zookeeper.startup_txns_loaded_sum
    (gauge)    		[zookeeper version 3.6+ only] Sum of startuptxnsloaded
    zookeeper.sync_process_time
    (gauge)    		[zookeeper version 3.6+ only] Histogram of syncprocesstime
    zookeeper.sync_process_time_count
    (gauge)    		[zookeeper version 3.6+ only] Total count of syncprocesstime
    zookeeper.sync_process_time_sum
    (gauge)    		[zookeeper version 3.6+ only] Sum of syncprocesstime
    zookeeper.sync_processor_batch_size
    (gauge)    		[zookeeper version 3.6+ only] Histogram of syncprocessorbatch_size
    zookeeper.sync_processor_batch_size_count
    (gauge)    		[zookeeper version 3.6+ only] Total count of syncprocessorbatch_size
    zookeeper.sync_processor_batch_size_sum
    (gauge)    		[zookeeper version 3.6+ only] Sum of syncprocessorbatch_size
    zookeeper.sync_processor_queue_and_flush_time_ms
    (gauge)    		[zookeeper version 3.6+ only] Histogram of syncprocessorqueueandflushtimems
    zookeeper.sync_processor_queue_and_flush_time_ms_count
    (gauge)    		[zookeeper version 3.6+ only] Total count of syncprocessorqueueandflushtimems
    zookeeper.sync_processor_queue_and_flush_time_ms_sum
    (gauge)    		[zookeeper version 3.6+ only] Sum of syncprocessorqueueandflushtimems
    zookeeper.sync_processor_queue_flush_time_ms
    (gauge)    		[zookeeper version 3.6+ only] Histogram of syncprocessorqueueflushtime_ms
    zookeeper.sync_processor_queue_flush_time_ms_count
    (gauge)    		[zookeeper version 3.6+ only] Total count of syncprocessorqueueflushtime_ms
    zookeeper.sync_processor_queue_flush_time_ms_sum
    (gauge)    		[zookeeper version 3.6+ only] Sum of syncprocessorqueueflushtime_ms
    zookeeper.sync_processor_queue_size
    (gauge)    		[zookeeper version 3.6+ only] Histogram of syncprocessorqueue_size
    zookeeper.sync_processor_queue_size_count
    (gauge)    		[zookeeper version 3.6+ only] Total count of syncprocessorqueue_size
    zookeeper.sync_processor_queue_size_sum
    (gauge)    		[zookeeper version 3.6+ only] Sum of syncprocessorqueue_size
    zookeeper.sync_processor_queue_time_ms
    (gauge)    		[zookeeper version 3.6+ only] Histogram of syncprocessorqueuetimems
    zookeeper.sync_processor_queue_time_ms_count
    (gauge)    		[zookeeper version 3.6+ only] Total count of syncprocessorqueuetimems
    zookeeper.sync_processor_queue_time_ms_sum
    (gauge)    		[zookeeper version 3.6+ only] Sum of syncprocessorqueuetimems
    zookeeper.sync_processor_request_queued
    (gauge)    		[zookeeper version 3.6+ only]
    zookeeper.time_waiting_empty_pool_in_commit_processor_read_ms
    (gauge)    		[zookeeper version 3.6+ only] Histogram of timewaitingemptypoolincommitprocessorreadms
    zookeeper.time_waiting_empty_pool_in_commit_processor_read_ms_count
    (gauge)    		[zookeeper version 3.6+ only] Total count of timewaitingemptypoolincommitprocessorreadms
    zookeeper.time_waiting_empty_pool_in_commit_processor_read_ms_sum
    (gauge)    		[zookeeper version 3.6+ only] Sum of timewaitingemptypoolincommitprocessorreadms
    zookeeper.timeouts
    (rate)    		The rate of timeouts the Datadog Agent received when trying to collect stats.
    Shown as occurrence
    zookeeper.tls_handshake_exceeded
    (gauge)    		[zookeeper version 3.6+ only]
    zookeeper.unrecoverable_error_count
    (gauge)    		[zookeeper version 3.6+ only]
    zookeeper.updatelatency
    (gauge)    		[zookeeper version 3.6+ only] Histogram of updatelatency Update request latency
    zookeeper.updatelatency_count
    (gauge)    		[zookeeper version 3.6+ only] Total count of updatelatency Update request latency
    zookeeper.updatelatency_sum
    (gauge)    		[zookeeper version 3.6+ only] Sum of updatelatency Update request latency
    zookeeper.uptime
    (gauge)    		[zookeeper version 3.6+ only] Uptime that a peer has been in a table leading/following/observing state
    zookeeper.watch_count
    (gauge)
    zookeeper.write_batch_time_in_commit_processor
    (gauge)    		[zookeeper version 3.6+ only] Histogram of writebatchtimeincommit_processor
    zookeeper.write_batch_time_in_commit_processor_count
    (gauge)    		[zookeeper version 3.6+ only] Total count of writebatchtimeincommit_processor
    zookeeper.write_batch_time_in_commit_processor_sum
    (gauge)    		[zookeeper version 3.6+ only] Sum of writebatchtimeincommit_processor
    zookeeper.write_commit_proc_issued
    (gauge)    		[zookeeper version 3.6+ only] Histogram of writecommitproc_issued
    zookeeper.write_commit_proc_issued_count
    (gauge)    		[zookeeper version 3.6+ only] Total count of writecommitproc_issued
    zookeeper.write_commit_proc_issued_sum
    (gauge)    		[zookeeper version 3.6+ only] Sum of writecommitproc_issued
    zookeeper.write_commit_proc_req_queued
    (gauge)    		[zookeeper version 3.6+ only] Histogram of writecommitprocreqqueued
    zookeeper.write_commit_proc_req_queued_count
    (gauge)    		[zookeeper version 3.6+ only] Total count of writecommitprocreqqueued
    zookeeper.write_commit_proc_req_queued_sum
    (gauge)    		[zookeeper version 3.6+ only] Sum of writecommitprocreqqueued
    zookeeper.write_commitproc_time_ms
    (gauge)    		[zookeeper version 3.6+ only] Histogram of writecommitproctime_ms
    zookeeper.write_commitproc_time_ms_count
    (gauge)    		[zookeeper version 3.6+ only] Total count of writecommitproctime_ms
    zookeeper.write_commitproc_time_ms_sum
    (gauge)    		[zookeeper version 3.6+ only] Sum of writecommitproctime_ms
    zookeeper.write_final_proc_time_ms
    (gauge)    		[zookeeper version 3.6+ only] Histogram of writefinalproctimems
    zookeeper.write_final_proc_time_ms_count
    (gauge)    		[zookeeper version 3.6+ only] Total count of writefinalproctimems
    zookeeper.write_final_proc_time_ms_sum
    (gauge)    		[zookeeper version 3.6+ only] Sum of writefinalproctimems
    zookeeper.znode_count
    (gauge)    		The number of znodes in the ZooKeeper namespace (the data).
    Shown as node
    zookeeper.zxid.count
    (gauge)
    zookeeper.zxid.epoch
    (gauge)

    Deprecated metrics

    The following metrics are still sent but will be removed eventually:

    • zookeeper.bytes_received
    • zookeeper.bytes_sent

    Events

    The ZooKeeper check does not include any events.

    Service Checks

    zookeeper.ruok
    Sends ruok to the monitored node. Returns OK with an imok response, WARN in the case of a different response and CRITICAL if no response is received.
    Statuses: ok, warning, critical

    zookeeper.mode
    Compares the current mode given by the stat command to the expected mode configured in zk.yaml. Returns OK when the modes match and CRITICAL otherwise. Only returns a status when there’s a value for expected_mode in zk.yaml
    Statuses: ok, critical

    Troubleshooting

    Need help? Contact Datadog support.

    PREVIEWING: esther/docs-9478-fix-split-after-example