Overview

Use Observability Pipelines’ sources to receive logs from your different log sources. Sources have different prerequisites and settings. Some sources also need to be configured to send logs to the Observability Pipelines Worker.

Select a source in the left navigation menu to see more information about it.

Standard metadata fields

All sources add the following standard metadata fields to ingested events:

For example, if this is the raw event:

{
  "foo": "bar"
}

Then the enriched event with the standard metadata fields is:

{
  "foo": "bar",
  "hostname": "ip-34-2-553.us.test",
  "timestamp": "2024-06-17T22:25:55.439Z",
  "source_type": "splunk_tcp"
}

You can see these standard metadata fields when you use the tap command to see the events sent through the source.

After events are ingested by the source, they get sent to different processors and destinations that might update those fields. For example, if the event is sent to the Datadog Logs destination, the timestamp field gets converted to UNIX format.

Note: The bytes in per second metric in the UI is for ingested raw events, not enriched events.

Further reading

Additional helpful documentation, links, and articles:

Set up pipelinesDOCUMENTATION Processors for your pipelinesDOCUMENTATION Observability Pipelines destinationsDOCUMENTATION