Overview
In the Datadog paradigm, any of the following are appropriate situations for declaring an incident:
- An issue is or may be impacting customers.
- You believe an issue (including an internal one) needs to be addressed as an emergency.
- You don’t know if you should call an incident - notify other people and increase severity appropriately.
You can declare an incident from multiple places within the Datadog platform, such as a graph widget on a dashboard, the Incidents UI, or any alert reporting into Datadog.
From the Incident page
In the Datadog UI, click Declare Incident to create an incident.
The Declare Incident modal displays a collapsible side panel that contains helper text and descriptions for the severities and statuses used by your organization. The helper text and descriptions are customizable in Incident Settings.
From a monitor
You can declare an incident directly from a monitor from the Actions dropdown. Select Declare incident to open an incident creation modal, and the monitor is added into the incident as a signal. You can also add a monitor to an existing incident.
From a Security Signal
Declare an incident directly from a Cloud SIEM or Cloud Security Management Threats signal side panel, by clicking Declare incident or Escalate Investigation. For more information, see Investigate Security Signals for Cloud Security Management.
Declare an incident from an Application Security Management signal through the actions listed in the signal side panel. Click Show all actions and click Declare Incident.
For more information, see Investigate Security Signals for Application Security Management.
From a case
Declare an incident from Case Management. From the individual case detail page, click Declare incident to escalate a case to an incident.
From a graph
You can declare an incident directly from a graph by clicking the export button on the graph and then clicking Declare incident. The incident creation modal appears, and the graph is added to the incident as a signal.
From a Synthetic test
Create incidents directly from a Synthetic test through the Actions dropdown. Select Declare incident to open an incident creation modal, where a summary of the test is added to your incident timeline, allowing you to pursue the investigation from there.
From the Datadog Clipboard
Use the Datadog Clipboard to gather multiple monitors and graphs and to generate an incident. To declare an incident from the Clipboard, copy a graph you want to investigate and open the Clipboard with the command Cmd/Ctrl + Shift + K
. Click Declare Incident or the export icon to add to the incident as a signal.
From Slack
If you have the Datadog integration enabled on Slack, you can declare a new incident with the slash command /datadog incident
from any Slack channel.
If the user declaring the incident connected their Slack to their Datadog account, by default, that user is listed as the Incident Commander. The Incident Commander (IC) can be changed later in-app if necessary. If the user declaring an incident is not a member of a Datadog account, then the IC is assigned to a generic Slack app user
and can be assigned to another IC in-app.
After you declare an incident from Slack, it generates an incident channel.
What’s next
Add helpful information to your incident and give context to everyone that is involved in the investigation.