Overview

Use NetFlow Monitoring in Datadog to visualize and monitor your flow records from your NetFlow-enabled devices.

Installation

To use NetFlow Monitoring with Network Device Monitoring, ensure you are using the Agent version 7.45 or newer.

Note: Configuring metric collection from Network Device Monitoring is not a requirement for sending NetFlow data, although it is strongly recommended as this extra data can be used to enrich your flow records with information such as the device name, model, and vendor, as well as the inbound/outbound interface name.

Configuration

To configure your devices to send NetFlow, jFlow, sFlow, or IPFIX traffic to the Agent NetFlow server, your devices must be configured to send traffic to the IP address that the Datadog Agent is installed on, specifically the flow_type and port.

Edit your datadog.yaml Agent configuration file to enable NetFlow:

network_devices:
  netflow:
    enabled: true
    listeners:
      - flow_type: netflow9   # choices: netflow5, netflow9, ipfix, sflow5
        port: 2055            # devices need to be configured to the same port number
      - flow_type: netflow5
        port: 2056
      - flow_type: ipfix
        port: 4739
      - flow_type: sflow5
        port: 6343

After saving your changes, restart the Agent.

Aggregation

The Datadog Agent automatically aggregates the data received into NetFlow to limit the number of records sent to the platform while maintaining most of the information. By default, flow recordings that have the same identifiers, such as source, destination address, port, and protocol, are aggregated together in five minute intervals. Additionally, the Datadog Agent can detect ephemeral ports and remove them. As a result, you may see Flows with port:*.

Enrichment

Your NetFlow data is processed by the Datadog backend and enriched with the available metadata from your devices and interfaces. Enrichment is based on the NetFlow exporter IP and the interface indexes. To disambiguate possible collisions between reused private IPs, you can configure a different namespace for each Agent configuration file (with the setting network_devices.namespace).

If the NetFlow exporter IP is one of the device IPs, but not the one configured on the SNMP integration, Datadog attempts to locate the device that the exporter IP belongs to, and enriches your NetFlow data with it is as long as the match is unique.

Cloud provider IP enrichment

Datadog enriches IPs with public cloud provider service and region for IPv4 addresses, so you can filter for flow records from a specific service and region.

Port enrichment

Datadog enriches ports in NetFlow with IANA (Internet Assigned Numbers Authority) data to resolve well known port mappings (such as Postgres on 5432 and HTTPS on 443). This can be seen when searching for source or destination application names on NetFlow.

Custom port enrichment

You can also add your own custom enrichments to map ports and protocols to specific applications (for example, if a custom service runs on a specific port). This makes it easier for network engineers and their teams to interpret and query NetFlow data with human-readable names.

From the Configuration tab in NetFlow, click Add Enrichment to upload the CSV file containing your custom enrichments.

Visualization

You can access the data collected by NetFlow Monitoring on the NetFlow page. Hover over a flow from the list for additional information about hosts, pods, and containers, and access related network connections.

When creating a NetFlow monitor, you should consider the following fields with respect to the source IP or destination IP from the perspective of the device. These fields provide insights into network traffic patterns and help with optimizing performance and security.

Interface information

The following fields represent details about the ingress and egress interfaces.

Device information

The following fields represent details related to the device generating NetFlow records.

Flow details

The following fields represent characteristics of the network flow.

In addition to fields, you can also use out-of-the-box facets to start analyzing traffic patterns based on NetFlow destination and source IP addresses.

NetFlow Destination IP facets

NetFlow Source IP facets

By monitoring these key fields and using facets to analyze NetFlow events, organizations can gain visibility into their network infrastructure, optimize performance, and improve security posture.

This data is also available in dashboards and notebooks, enabling precise queries and correlation with other data sources. When creating a dashboard with NetFlow data, select NetFlow as the source in the Graph your data section.

Sampling rate

NetFlow’s sampling rate is taken into account in the computation of bytes and packets by default. The displayed values for bytes and packets are computed with the sampling rate applied. Additionally, you can query for Bytes (Adjusted) (@adjusted_bytes) and Packets (Adjusted) (@adjusted_packets) in dashboards and notebooks to visualize them.

To visualize the raw bytes/packets (sampled) sent by your devices, you can query for Bytes (Sampled) (@bytes) and Packets (Sampled) (@packets) in dashboards and notebooks.

Retention

NetFlow data is retained for 30 days by default, with options for 15, 30, 60, and 90 day retention.

Troubleshooting

NetFlow packet drops

NetFlow packet drops can occur when there are a high number of NetFlow packets per second, typically greater than 50,000. The following steps can help identify and mitigate NetFlow packet drops:

Identifying packet drops

Use the netstat -s command to see if there are any dropped UDP packets:

    netstat -s

Mitigation steps

1. Increase the Number of NetFlow Listeners

Increase the number of NetFlow listeners by using a configuration similar to the following: Datadog recommends setting the number of workers to match the number of CPU cores in your system:

      netflow:
        enabled: true
        listeners:
          - flow_type: netflow9
            port: 2055
            workers: 4 # 4 CPUs

2. Increase UDP Queue Length (Linux only)

Adjusting your system’s UDP queue length can help accommodate the higher volume of NetFlow packets. Increase the UDP receive buffer size to 25MB by executing the following commands:

    sudo sysctl -w net.core.rmem_max=26214400
    sudo sysctl -w net.core.rmem_default=26214400

3. Persisting the configuration (Linux only)

To make these changes permanent, add the following lines to your /etc/sysctl.conf file:

    net.core.rmem_max=26214400
    net.core.rmem_default=26214400

Further Reading

추가 유용한 문서, 링크 및 기사:

Using Profiles with Network Device MonitoringDOCUMENTATION Monitor NetFlow traffic data with DatadogBLOG Monitor and diagnose network performance issues with SNMP TrapsBLOG