Error outliers
Error outliers display fields such as faceted tags or attributes containing characteristics of errors that match the current query. Statistically overrepresented key:value
pairs among errors provide hints into the root causes of problems.
Typical examples of error outliers include env:staging
, docker_image:acme:3.1
, and http.useragent_details.browser.family:curl
.
In the banner card view, you can see:
- The field name
- The proportion of errors and overall logs that the field contributes to
In the full side panel view, you can see:
- The timeseries of error logs that contain the field
- Tags that are often associated with the error logs
- A comprehensive list of log patterns
APM outliers are available on all APM pages where the Watchdog Insights carousel is available:
Error outliers
Error outliers display fields such as tags containing characteristics of errors that match the current query. Statistically overrepresented key:value
pairs among errors provide hints into the root cause of problems.
Typical examples of error outliers include env:staging
, availability_zone:us-east-1a
, cluster_name:chinook
, and version:v123456
.
In the banner card view, you can see:
- The field name
- The proportion of errors and overall traces that the field contributes to
In the full side panel view, you can see:
- The timeseries of error traces that contain the field
- Tags that are often associated with the error traces
- A comprehensive list of related Error Tracking Issues and failing spans
Latency outliers
Latency outliers display fields such as tags that are associated with performance bottlenecks that match the current search query. key:value
pairs with worse performance than the baseline can provide hints into the performance bottlenecks among a subset of APM spans.
Latency outliers are computed for the span duration.
In the banner card view, you can see:
The field name
The latency distribution for spans containing the tag and the baseline for the rest of the data
A percentile of interest latency value for the outlier tag and the difference with the baseline for the rest of the data
In the full side panel, you can see a latency distribution graph for the tag and the baseline. The X axis has increments of p50
, p75
, p99
, and max
, along with a list of APM events that contain the field.
Lock contention outlier
In the banner card view, you can see:
- The name of the impacted service
- The number of threads impacted
- The potential CPU savings (and estimated cost savings)
In the full side panel, you can see instructions on how to resolve the lock contention:
Garbage collection outlier
In the banner card view, you can see:
- The name of the impacted service
- The amount of CPU time used to perform garbage collection
In the full side panel, you can see instructions on how to better configure garbage collection to free up some CPU time:
Regex compilation outlier
In the banner card view, you can see:
- The name of the impacted service
- The amount of CPU time spent on compiling regexes
In the full side panel, you can see instructions on how to improve regex compilation time, as well as examples of functions within your code that could be improved:
For Database Monitoring, Watchdog surfaces insights on the following metrics:
CPU
Commits
IO
Background
Concurrency
Idle
Find the databases impacted by one or multiple outliers by using the Insight carousel.
An overlay is then set on the databases, with pink pills highlighting the different Insights and giving more information about what happened.
Error outlier
Error outliers display fields such as faceted tags or attributes that contain characteristics of errors that match the current search query. Statistically overrepresented key:value
pairs among errors can provide hints into the root causes of issues. Typical examples of error outliers include env:staging
, version:1234
, and browser.name:Chrome
.
In the banner card view, you can see:
- The field name
- The proportion of total errors and overall RUM events that the field contributes to
- Related tags
In the full side panel, you can see a timeseries graph about the total number of RUM errors with the field, along with impact pie charts and a list of RUM events that contain the field.
Latency outlier
Latency outliers display fields such as faceted tags or attributes that are associated with performance bottlenecks that match the current search query. key:value
pairs with worse performance than the baseline can provide hints into the performance bottlenecks among a subset of real users.
Latency outliers are computed for Core Web Vitals such as First Contentful Paint, First Input Delay, Cumulative Layout Shift, and Loading Time. For more information, see Monitoring Page Performance.
In the banner card view, you can see:
- The field name
- The performance metric value containing the field and the baseline for the rest of the data
In the full side panel, you can see a timeseries graph about the performance metric. The X axis has increments of p50
, p75
, p99
, and max
, along with a list of RUM events that contain the field.
For serverless infrastructures, Watchdog surfaces the following insights:
Cold Start Ratio Up/Down
Error Invocation Ratio Up/Down
Memory Usage Up/Down
OOM Ratio Up/Down
Estimated Cost Up/Down
Init Duration Up/Down
Runtime Duration Up/Down
Find the serverless functions impacted by one or multiple outliers by using the Insights carousel.
An overlay is then set on the function, with pink pills highlighting the different insights and giving more information about what happened.
For Process Explorer, the Watchdog Insight carousel reflects all Process anomalies for the current context of the Process Explorer.
For Kubernetes Explorer, the Watchdog Insight carousel reflects all the Kubernetes anomalies for the current context of the Kubernetes Explorer.