Fluentd and Fluent Bit Sources

Use Observability Pipelines’ Fluentd or Fluent Bit source to receive logs from the your Fluentd or Fluent Bit agent. Select and set up this source when you set up a pipeline.

Prerequisites

To use Observability Pipelines’s Fluentd or Fluent Bit source, you need the following information available:

  1. The Observability Pipelines Worker listens on this bind address to receive logs from your applications. For example, 0.0.0.0:8088. Later on, you configure your applications to send logs to this address.
  2. The appropriate TLS certificates and the password you used to create your private key if your forwarders are globally configured to enable SSL.

Set up the source in the pipeline UI

Select and set up this source when you set up a pipeline. The information below are for the source settings in the pipeline UI.

Optionally, toggle the switch to enable TLS. If you enable TLS, the following certificate and key files are required:

  • Server Certificate Path: The path to the certificate file that has been signed by your Certificate Authority (CA) Root File in DER or PEM (X.509) format.
  • CA Certificate Path: The path to the certificate file that is your Certificate Authority (CA) Root File in DER or PEM (X.509) format.
  • Private Key Path: The path to the .key private key file that belongs to your Server Certificate Path in DER or PEM (PKCS#8) format.

Send logs to the Observability Pipelines Worker over Fluent

Fluent Bit configuration

To configure Fluent Bit to send logs to the Observability Pipelines Worker, use the following output configuration:

[OUTPUT]
	Name          forward
	Match         *
	# Update these to point to your Observability Pipelines Worker
	Host          127.0.0.1
	Port          24224

Fluentd configuration

To configure Fluentd to send logs to the Observability Pipelines Worker, use the following output configuration:

<match *>
	  @type forward
	  <server>
		# Update these to point to your Observability Pipelines Worker
		name  local
		host  127.0.0.1
		port 24224
	  </server>
	  compress gzip
	</match>
PREVIEWING: esther/docs-9518-update-example-control-sensitive-log-data