Overview

Datadog’s Palo Alto Networks Firewall Log integration allows customers to ingest, parse, and analyze Palo Alto Networks firewall logs. This log integration relies on the HTTPS log templating and forwarding capability provided by PAN OS, the operating system that runs in Palo Alto firewalls. PAN-OS allows customers to forward threat, traffic, authentication, and other important log events.

Key use cases

Respond to high severity threat events

Firewall threat logs provide context on threats detected by a firewall, which can be filtered and analyzed by severity, type, origin IPs/countries, and more.

Make informed decisions on Firewall deployment

Firewall traffic logs can be used to measure the traffic and sessions passing through a firewall and also gives you the ability to monitor for anomalous throughput across firewall deployment.

Monitor authentication anomalies

Firewall authentication logs provide detailed information on users as they authenticate with Palo Alto Networks firewall. These logs can be used to monitor anomalous spikes in authentication traffic from specific protocols, users, locations, and more.

Setup

Log collection

1. Install the Datadog Agent on a machine that is reachable by the firewall and can connect to the internet.

2. In PanOS, Select Device » Server Profiles » Syslog , add a name for the server profile. Follow the Syslog log forwarding configuration steps. Same steps listed below.

3. Click Add and provide the following details of the server:

   • Name of the server
   • IP address of the machine with datadog agent
   • Transport as TCP
   • Port as 10518 and format as BSD

4. Copy and configure custom log format for the required log type. The formats below are mappings from the formats listed in the Palo Alto Networks Syslog Field Descriptions docs.

   Name	Format
   Traffic Log	View Payload timestamp=$time_generated, serial=$serial, type=$type, subtype=$subtype, time_generated=$time_generated, network.client.ip=$src, network.destination.ip=$dst, natsrc=$natsrc, natdst=$natdst, rule=$rule, usr.id=$srcuser, dstuser=$dstuser, app=$app, vsys=$vsys, from=$from, to=$to, inbound_if=$inbound_if, outbound_if=$outbound_if, logset=$logset, sessionid=$sessionid, repeatcnt=$repeatcnt, network.client.port=$sport, network.destination.port=$dport, natsport=$natsport natdport=$natdport, flags=$flags, proto=$proto, evt.name=$action, bytes=$bytes, network.bytes_read=$bytes_sent, network.bytes_written=$bytes_received, start=$start, elapsed=$elapsed, category=$category, seqno=$seqno, actionflags=$actionflags, network.client.geoip.country.name=$srcloc, dstloc=$dstloc, pkts_sent=$pkts_sent, pkts_received=$pkts_received, session_end_reason=$session_end_reason, device_name=$device_name, action_source=$action_source, src_uuid=$src_uuid, dst_uuid=$dst_uuid, tunnelid=$tunnelid, imsi= $imsi, monitortag=$monitortag, imei=$imei, parent_session_id=$parent_session_id, parent_start_time=$parent_start_time, tunnel=$tunnel, assoc_id=$assoc_id, chunks=$chunks chunks_sent=$chunks_sent chunks_received=$chunks_received
   Threat Log (and WildFire Submissions Log)	View Payload timestamp=$receive_time, serial=$serial, type=$type, subtype=$subtype, time_generated=$time_generated, network.client.ip=$src, network.destination.ip=$dst, natsrc=$natsrc, natdst=$natdst, rule=$rule, usr.id=$srcuser, dstuser=$dstuser, app=$app, vsys=$vsys, from=$from, to=$to, inbound_if=$inbound_if, outbound_if=$outbound_if, logset=$logset, sessionid=$sessionid, repeatcnt=$repeatcnt, network.client.port=$sport, network.destination.port=$dport, natsport=$natsport, natdport=$natdport, flags=$flags, proto=$proto, evt.name=$action, misc=$misc, threatid=$threatid, category=$category, severity=$severity, direction=$direction, seqno=$seqno, actionflags=$actionflags, network.client.geoip.country.name=$srcloc, dstloc=$dstloc, contenttype=$contenttype, pcap_id=$pcap_id, filedigest=$filedigest, cloud=$cloud, url_idx=$url_idx, http.useragent=$user_agent, filetype=$filetype, xff=$xff referer=$referer, sender=$sender, subject=$subject, recipient=$recipient, reportid=$reportid, vsys_name=$vsys_name, device_name=$device_name, src_uuid=$src_uuid, dst_uuid=$dst_uuid, http_method=$http_method, tunnel_id=$tunnel_id, imsi=$imsi, monitortag=$monitortag, imei=$imei, parent_session_id=$parent_session_id, parent_start_time=$parent_start_time, tunnel=$tunnel, thr_category=$thr_category, contentver=$contentver, assoc_id=$assoc_id, ppid=$ppid, http_headers=$http_headers
   Authentication Log	View Payload timestamp=$time_generated, serial=$serial, type=$type, subtype=$subtype, vsys=$vsys, network.client.ip=$ip, usr.id=$user, normalize_user=$normalize_user, object=$object, authpolicy=$authpolicy, repeatcnt=$repeatcnt, authid=$authid, vendor=$vendor , logset=$logset, serverprofile=$serverprofile, message=$message ,clienttype=$clienttype, evt.outcome=$event, factorno=$factorno, seqno=$seqno, actionflags=$actionflags, vsys_name=$vsys_name, device_name=$device_name, vsys_id=$vsys_id, evt.name=$authproto
   HIP Match Log	View Payload timestamp=$time_generated, serial=$serial, type=$type, subtype=$subtype, time_generated=$time_generated, usr.id=$srcuser, vsys=$vsys, machinename=$machinename, os=$os, network.client.ip=$src, matchname=$matchname, repeatcnt=$repeatcnt, matchtype=$matchtype, seqno=$seqno, actionflags=$actionflags, vsys_name=$vsys_name, device_name=$device_name, vsys_id=$vsys_id, srcipv6=$srcipv6, hostid=$hostid
   User-ID Log	View Payload timestamp=$time_generated, serial=$serial, type=$type, subtype=$subtype, vsys=$vsys, network.client.ip=$ip, usr.id=$user, datasourcename=$datasourcename, evt.name=$eventid, repeatcnt=$repeatcnt, timeout=$timeout, network.client.port=$beginport, network.destination.port=$endport, datasource=$datasource, datasourcetype=$datasourcetype, seqno=$seqno, actionflags=$actionflags, vsys_name=$vsys_name, device_name=$device_name, vsys_id=$vsys_id, factortype=$factortype, factorcompletiontime=$factorcompletiontime,, factorno=$factorno, ugflags=$ugflags, userbysource=$userbysource
   Tunnel Inspection Log	View Payload timestamp=$time_generated, serial=$serial, type=$type, subtype=$subtype, network.client.ip=$src, network.destination.ip=$dst, natsrc=$natsrc, natdst=$natdst, rule=$rule, usr.id=$srcuser, dstuser=$dstuser, app=$app, vsys=$vsys, from=$from, to=$to, inbound_if=$inbound_if, outbound_if=$outbound_if, logset=$logset, sessionid=$sessionid, repeatcnt=$repeatcnt, network.client.port=$sport, network.destination.port=$dport, natsport=$natsport, natdport=$natdport, flags=$flags, proto=$proto, evt.outcome=$action, severity=$severity, seqno=$seqno, actionflags=$actionflags, srcloc=$srcloc, dstloc=$dstloc, vsys_name=$vsys_name, device_name=$device_name, tunnelid=$tunnelid, monitortag=$monitortag, parent_session_id=$parent_session_id, parent_start_time=$parent_start_time, tunnel=$tunnel, bytes=$bytes, network.bytes_read=$bytes_sent, network.bytes_written=$bytes_received, packets=$packets, pkts_sent=$pkts_sent, pkts_received=$pkts_received, max_encap=$max_encap, unknown_proto=$unknown_proto, strict_check=$strict_check, tunnel_fragment=$tunnel_fragment, sessions_created=$sessions_created, sessions_closed=$sessions_closed, session_end_reason=$session_end_reason, evt.name=$action_source, start=$start, elapsed=$elapsed, tunnel_insp_rule=$tunnel_insp_rule
   SCTP Log	View Payload timestamp=$time_generated, serial=$serial, type=$type, network.client.ip=$src, network.destination.ip=$dst, rule=$rule, vsys=$vsys, from=$from, to=$to, inbound_if=$inbound_if, outbound_if=$outbound_if, logset=$logset, sessionid=$sessionid, repeatcnt=$repeatcnt, network.client.port=$sport, network.destination.port=$dport, proto=$proto, action=$action, vsys_name=$vsys_name, device_name=$device_name, seqno=$seqno, assoc_id=$assoc_id, ppid=$ppid, severity=$severity, sctp_chunk_type=$sctp_chunk_type, sctp_event_type=$sctp_event_type, verif_tag_1=$verif_tag_1, verif_tag_2=$verif_tag_2, sctp_cause_code=$sctp_cause_code, diam_app_id=$diam_app_id, diam_cmd_code=$diam_cmd_code, diam_avp_code=$diam_avp_code, stream_id=$stream_id, assoc_end_reason=$assoc_end_reason, op_code=$op_code, sccp_calling_ssn=$sccp_calling_ssn, sccp_calling_gt=$sccp_calling_gt, sctp_filter=$sctp_filter, chunks=$chunks, chunks_sent=$chunks_sent, chunks_received=$chunks_received, packets=$packets, pkts_sent=$pkts_sent, pkts_received=$pkts_received
   Config Log	View Payload timestamp=$time_generated, serial=$serial, type=$type, subtype=$subtype, network.client.ip=$host, vsys=$vsys, evt.name=$cmd, usr.id=$admin, client=$client, evt.outcome=$result, path=$path, before_change_detail=$before_change_detail, after_change_detail=$after_change_detail, seqno=$seqno, actionflags=$actionflags, vsys_name=$vsys_name, device_name=$device_name
   System Log	View Payload timestamp=$time_generated, serial=$serial, type=$type, subtype=$subtype, vsys=$vsys, evt.name=$eventid, object=$object, module=$module, severity=$severity, opaque=$opaque, seqno=$seqno, actionflags=$actionflags, vsys_name=$vsys_name, device_name=$device_name
   Correlated Events Log	View Payload timestamp=$time_generated, serial=$serial, type=$type, subtype=$subtype, vsys=$vsys, evt.name=$eventid, object=$object, module=$module, severity=$severity, opaque=$opaque, seqno=$seqno, actionflags=$actionflags, vsys_name=$vsys_name, device_name=$device_name
   GTP Log	View Payload timestamp=$start, serial=$serial, type=$type, subtype=$subtype, network.client.ip=$src, network.destination.ip=$dst, rule=$rule, app=$app, vsys=$vsys, from=$from, to=$to, inbound_if=$inbound_if, outbound_if=$outbound_if, logset=$logset, sessionid=$sessionid, network.client.port=$sport, network.destination.port=$dport, proto=$proto, evt.name=$action, event_type=$event_type, msisdn=$msisdn, apn=$apn, rat=$rat, msg_type=$msg_type, end_ip_adr=$end_ip_adr, teid1=$teid1, teid2=$teid2, gtp_interface=$gtp_interface, cause_code=$cause_code, severity=$severity, mcc=$mcc, mnc=$mnc, area_code=$area_code, cell_id=$cell_id, event_code=$event_code, srcloc=$srcloc, dstloc=$dstloc, imsi=$imsi, imei=$imei, start=$start, elapsed=$elapsed, tunnel_insp_rule=$tunnel_insp_rule
   Data Filtering Log	View Payload timestamp=$receive_time, serial=$serial, type=$type, subtype=$subtype, time_generated=$time_generated, network.client.ip=$src, network.destination.ip=$dst, natsrc=$natsrc, natdst=$natdst, rule=$rule, usr.id=$srcuser, dstuser=$dstuser, app=$app, vsys=$vsys, from=$from, to=$to, inbound_if=$inbound_if, outbound_if=$outbound_if, logset=$logset, sessionid=$sessionid, repeatcnt=$repeatcnt, network.client.port=$sport, network.destination.port=$dport, natsport=$natsport, natdport=$natdport, flags=$flags, proto=$proto, evt.name=$action, misc=$misc, threatid=$threatid, category=$category, severity=$severity, direction=$direction, seqno=$seqno, actionflags=$actionflags, network.client.geoip.country.name=$srcloc, dstloc=$dstloc, contenttype=$contenttype, pcap_id=$pcap_id, filedigest=$filedigest, cloud=$cloud, url_idx=$url_idx, http.useragent=$user_agent, filetype=$filetype, xff=$xff, referer=$referer, sender=$sender, subject=$subject, recipient=$recipient, reportid=$reportid, vsys_name=$vsys_name, device_name=$device_name, src_uuid=$src_uuid, dst_uuid=$dst_uuid, http_method=$http_method, tunnel_id=$tunnel_id, imsi=$imsi, monitortag=$monitortag, imei=$imei, parent_session_id=$parent_session_id, parent_start_time=$parent_start_time, tunnel=$tunnel, thr_category=$thr_category, contentver=$contentver, assoc_id=$assoc_id, ppid=$ppid, http_headers=$http_headers, url_category_list=$url_category_list, rule_uuid=$rule_uuid, http2_connection=$http2_connection, dynusergroup_name=$dynusergroup_name, xff_ip=$xff_ip, src_osfamily=$src_osfamily, src_osversion=$src_osversion, src_host=$src_host, src_mac=$src_mac, dst_osfamily=$dst_osfamily, dst_osversion=$dst_osversion, dst_host=$dst_host, dst_mac=$dst_mac, container_id=$container_id, pod_namespace=$pod_namespace, pod_name=$pod_name, src_edl=$src_edl, dst_edl=$dst_edl, hostid=$hostid, serialnumber=$serialnumber, domain_edl=$domain_edl, src_dag=$src_dag, dst_dag=$dst_dag, partial_hash=$partial_hash, high_res_timestamp=$high_res_timestamp, reason=$reason, justification=$justification
   URL Filtering Log	View Payload timestamp=$receive_time, serial=$serial, type=$type, subtype=$subtype, time_generated=$time_generated, network.client.ip=$src, network.destination.ip=$dst, natsrc=$natsrc, natdst=$natdst, rule=$rule, usr.id=$srcuser, dstuser=$dstuser, app=$app, vsys=$vsys, from=$from, to=$to, inbound_if=$inbound_if, outbound_if=$outbound_if, logset=$logset, sessionid=$sessionid, repeatcnt=$repeatcnt, network.client.port=$sport, network.destination.port=$dport, natsport=$natsport, natdport=$natdport, flags=$flags, proto=$proto, evt.name=$action, misc=$misc, threatid=$threatid, category=$category, severity=$severity, direction=$direction, seqno=$seqno, actionflags=$actionflags, network.client.geoip.country.name=$srcloc, dstloc=$dstloc, contenttype=$contenttype, pcap_id=$pcap_id, filedigest=$filedigest, cloud=$cloud, url_idx=$url_idx, http.useragent=$user_agent, filetype=$filetype, xff=$xff, referer=$referer, sender=$sender, subject=$subject, recipient=$recipient, reportid=$reportid, vsys_name=$vsys_name, device_name=$device_name, src_uuid=$src_uuid, dst_uuid=$dst_uuid, http_method=$http_method, tunnel_id=$tunnel_id, imsi=$imsi, monitortag=$monitortag, imei=$imei, parent_session_id=$parent_session_id, parent_start_time=$parent_start_time, tunnel=$tunnel, thr_category=$thr_category, contentver=$contentver, assoc_id=$assoc_id, ppid=$ppid, http_headers=$http_headers, url_category_list=$url_category_list, rule_uuid=$rule_uuid, http2_connection=$http2_connection, dynusergroup_name=$dynusergroup_name, xff_ip=$xff_ip, src_osfamily=$src_osfamily, src_osversion=$src_osversion, src_host=$src_host, src_mac=$src_mac, dst_osfamily=$dst_osfamily, dst_osversion=$dst_osversion, dst_host=$dst_host, dst_mac=$dst_mac, container_id=$container_id, pod_namespace=$pod_namespace, pod_name=$pod_name, src_edl=$src_edl, dst_edl=$dst_edl, hostid=$hostid, serialnumber=$serialnumber, domain_edl=$domain_edl, src_dag=$src_dag, dst_dag=$dst_dag, partial_hash=$partial_hash, high_res_timestamp=$high_res_timestamp, reason=$reason, justification=$justification
   GlobalProtect Log	View Payload timestamp=$receive_time, serial=$serial, type=$type, subtype=$subtype, time_generated=$time_generated, vsys=$vsys, evt.name=$eventid, stage=$stage, auth_method=$auth_method, tunnel_type=$tunnel_type, usr.id=$srcuser, srcregion=$srcregion, machinename=$machinename, public_ip=$public_ip, public_ipv6=$public_ipv6, private_ip=$private_ip, private_ipv6=$private_ipv6, hostid=$hostid, serialnumber=$serialnumber, client_ver=$client_ver, client_os=$client_os, client_os_ver=$client_os_ver, repeatcnt=$repeatcnt, reason=$reason, error=$error, opaque=$opaque, status=$status, location=$location, login_duration=$login_duration, connect_method=$connect_method, error_code=$error_code, portal=$portal, seqno=$seqno, actionflags=$actionflags, selection_type=$selection_type, response_time=$response_time, priority=$priority, attempted_gateways=$attempted_gateways, gateway=$gateway, dg_hier_level_1=$dg_hier_level_1, dg_hier_level_2=$dg_hier_level_2, dg_hier_level_3=$dg_hier_level_3, dg_hier_level_4=$dg_hier_level_4, vsys_name=$vsys_name, device_name=$device_name, vsys_id=$vsys_id

5. Click OK, this creates a syslog server profile.

6. Click on the Objects tab, this opens the log forwarding profile screen.

7. Create log forwarding profile by providing the name, log type and syslog profile

8. Create a pan.firewall.d/conf.yaml file at the root of the Agent’s configuration directory with the below content.

   logs:
   - type: tcp
     port: 10518
     service: "firewall"
     source: "pan.firewall"
   

9. Restart Agent.

Data Collected

Metrics

The metrics collected for this integration are determined by the relevant configured profiles on the network vendors page.

Logs

The PANOS integration collects logs from the Palo Alto Networks firewall integration and forwards them to Datadog.

Events

The PANOS integration does not send any events.

Service Checks

The PANOS integration does not include any service checks.

Further Reading

Additional helpful documentation, links, and articles:

• Log types and fields
• Logs Collection documentation

Troubleshooting

Need help? Contact Datadog support.