Use Observability Pipelines’ Logstash source to receive logs from your Logstash agent. Select and set up this source when you set up a pipeline.

Prerequisites

To use Observability Pipelines’ Logstash source, you need the following information available:

• Logstash address, such as 0.0.0.0:8088. The Observability Pipelines Worker listens on this bind address to receive logs from your applications. Later on, you configure your applications to send logs to this address.
• The appropriate TLS certificates and the password you used to create your private key, if your forwarders are globally configured to enable SSL.

Set up the source in the pipeline UI

Select and set up this source when you set up a pipeline. The information below is for the source settings in the pipeline UI.

Optionally, toggle the switch to enable TLS. If you enable TLS, the following certificate and key files are required:

• Server Certificate Path: The path to the certificate file that has been signed by your Certificate Authority (CA) Root File in DER or PEM (X.509).
• CA Certificate Path: The path to the certificate file that is your Certificate Authority (CA) Root File in DER or PEM (X.509).
• Private Key Path: The path to the .key private key file that belongs to your Server Certificate Path in DER or PEM (PKCS #8) format.

Send logs to the Observability Pipelines Worker over Logstash

To configure Logstash to send logs to the Observability Pipelines Worker, use the following output configuration:

output {
	lumberjack {
		# update these to point to your Observability Pipelines Worker
		hosts => ["127.0.0.1"]
		port => 5044
		ssl_certificate => "/path/to/certificate.crt"
	}
}

Note: Logstash requires SSL to be configured.

Send logs using Filebeat to Observability Pipelines

Use the Logstash source to send logs to the Observability Pipelines Worker with Filebeat.

1. Set up Filebeat if you haven’t already.

2. In the filebeat.yml file:
   a. Comment out the Elasticsearch Output configuration section.
   b. Uncomment and configure the Logstash Output section:

   # ------------------------------ Logstash Output -------------------------------
   output.logstash:
   # The Logstash hosts
   hosts: ["<OPW_HOST>:9997"]
   

   <OPW_HOST> is the host IP address or the load balancer URL associated with the Observability Pipelines Worker.

   For CloudFormation installs, use the LoadBalancerDNS CloudFormation output for the URL.

   For Kubernetes installs, you can use the internal DNS record of the Observability Pipelines Worker service. For example: opw-observability-pipelines-worker.default.svc.cluster.local.

3. Set up a pipeline with the Logstash source.