flume

Supported OS Linux Mac OS Windows

Intégration0.0.1

Présentation

Ce check surveille Apache Flume.

Configuration

Le check Flume n’est pas inclus avec le package de l’Agent Datadog : vous devez donc l’installer.

Installation

Pour l’Agent v7.21+/6.21+, suivez les instructions ci-dessous afin d’installer le check Flume sur votre host. Consultez la section Utiliser les intégrations de la communauté pour effectuer une installation avec l’Agent Docker ou avec des versions antérieures de l’Agent.

  1. Exécutez la commande suivante pour installer l’intégration de l’Agent :

    datadog-agent integration install -t datadog-flume==<INTEGRATION_VERSION>
    
  2. Configurez votre intégration comme une intégration de base.

Configuration

  1. Configurez l’Agent Flume pour activer JMX en ajoutant les arguments JVM suivants à votre flume-env.sh :
export JAVA_OPTS="-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=5445 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false"
  1. Modifiez le fichier flume.d/conf.yaml dans le dossier conf.d/ à la racine du répertoire de configuration de votre Agent pour commencer à recueillir vos données de performance Flume. Consultez le fichier d’exemple flume.d/conf.yaml pour découvrir toutes les options de configuration disponibles.

    Ce check prévoit une limite de 350 métriques par instance. Le nombre de métriques renvoyées est indiqué sur la page d’informations. Choisissez les métriques qui vous intéressent en modifiant la configuration ci-dessous. Pour découvrir comment modifier la liste des métriques à recueillir, consultez la documentation relative aux checks JMX afin d’obtenir des instructions détaillées. Si vous devez surveiller davantage de métriques, contactez l’assistance Datadog.

  2. Redémarrez l’Agent.

Validation

Lancez la sous-commande status de l’Agent et cherchez flume dans la section Checks.

Métriques des composants

Les métriques récupérées par ce check dépendent de la source, du canal et du récepteur utilisés par votre Agent Flume. Pour obtenir la liste complète des métriques exposées pour chaque composant, consultez la section Available Component Metrics (en anglais) de la documentation Apache Flume. Pour obtenir la liste des métriques consultables dans Datadog, référez-vous à la rubrique Métriques de cette page.

Données collectées

Métriques

flume.channel.capacity
(gauge)
The maximum number of events that can be queued in the channel at any time. For channel types without a capacity limit the value will be zero.
Shown as event
flume.channel.fill_percentage
(gauge)
The channel fill percentage.
Shown as percent
flume.channel.size
(gauge)
The number of events currently queued in the channel.
Shown as event
flume.channel.event_put_attempt_count
(count)
The total number of events that have been attempted to be put into the channel.
Shown as event
flume.channel.event_put_success_count
(count)
The total number of events that have successfully been put into the channel.
Shown as event
flume.channel.event_take_attempt_count
(count)
The total number of attempts that have been made to take an event from the channel.
Shown as event
flume.channel.event_take_success_count
(count)
The total number of events that have successfully been taken from the channel.
Shown as event
flume.channel.kafka_commit_timer
(gauge)
The timer for the Kafka channel commits.
Shown as time
flume.channel.kafka_event_get_timer
(gauge)
The timer for the kafka channel retrieving events.
Shown as time
flume.channel.kafka_event_send_timer
(gauge)
The timer for the Kafka channel sending events.
Shown as time
flume.channel.rollbackcount
(count)
The count of rollbacks from the kafka channel.
Shown as event
flume.sink.event_write_fail
(count)
The total number of failed write events.
Shown as event
flume.sink.batch_empty_count
(count)
The number of append batches attempted containing zero events.
Shown as event
flume.sink.channel_read_fail
(count)
The number of failed read events from the channel.
Shown as event
flume.sink.batch_complete_count
(count)
The number of append batches attempted containing the maximum number of events supported by the next hop.
Shown as event
flume.sink.batch_underflow_count
(count)
The number of append batches attempted containing less than the maximum number of events supported by the next hop.
Shown as event
flume.sink.connection_closed_count
(count)
The number of connections closed by this sink.
Shown as connection
flume.sink.connection_failed_count
(count)
The number of failed connections.
Shown as connection
flume.sink.connection_created_count
(count)
The number of connections created by this sink. Only applicable to some sink types.
Shown as connection
flume.sink.event_drain_attempt_count
(count)
The total number of events that have been attempted to be drained to the next hop.
Shown as event
flume.sink.event_drain_success_count
(count)
The total number of events that have successfully been drained to the next hop
Shown as event
flume.sink.kafka_event_sent_timer
(gauge)
The timer for the Kafka sink sending events.
Shown as time
flume.sink.rollbackcount
(gauge)
The count of rollbacks from the Kafka sink.
Shown as event
flume.source.event_read_fail
(count)
The total number of failed read source events.
Shown as event
flume.source.channel_write_fail
(count)
The total number of failed channel write events.
Shown as event
flume.source.event_accepted_count
(count)
The total number of events successfully accepted, either through append batches or single-event appends.
Shown as event
flume.source.event_received_count
(count)
The total number of events received, either through append batches or single-event appends.
Shown as event
flume.source.append_accepted_count
(count)
The total number of single-event appends successfully accepted.
Shown as event
flume.source.append_received_count
(count)
The total number of single-event appends received.
Shown as event
flume.source.open_connection_count
(count)
The number of open connections
Shown as connection
flume.source.generic_processing_fail
(count)
The total number of generic processing failures.
Shown as event
flume.source.append_batch_accepted_count
(count)
The total number of append batches accepted successfully.
Shown as event
flume.source.append_batch_received_count
(count)
The total number of append batches received.
Shown as event
flume.source.kafka_commit_timer
(gauge)
The timer for the Kafka source committing events.
Shown as time
flume.source.kafka_empty_count
(count)
The count of empty events from the Kafka source.
Shown as event
flume.source.kafka_event_get_timer
(gauge)
The timer for the Kafka source retrieving events.
Shown as time

Événements

Flume n’inclut aucun événement.

Checks de service

flume.can_connect
Returns CRITICAL if the Agent is unable to connect to and collect metrics from the monitored Flume instance. Returns OK otherwise.
Statuses: ok, critical

Dépannage

Besoin d’aide ? Contactez l’assistance Datadog.

PREVIEWING: esther/docs-9518-update-example-control-sensitive-log-data