Private Location Monitoring
Overview
With private locations, you have a set of out-of-the-box metrics to keep track of your private location health at a high level. You can visualize these metrics in the side panel of each private location on the Settings page or graph these metrics in a dashboard.
In Synthetics Settings, the Private Locations tab displays your private locations along with their reporting status and monitor status.
When you click on a private location, a panel containing Health and Metadata details appears. The table in the Health tab displays all reporting workers and the image version they are running. You can get a sense of how many workers you need to pull for the new image version.
In Monitors, you can see status warnings such as ALERT when something is going wrong with your private location. For example, the private location stops reporting, the private location becomes under-provisioned, or the private location worker runs an outdated image version.
Default monitors
When you create a private location, three monitors are added to your account:
| Monitor Name | Description |
|---|
| [Synthetic Private Locations] {{location_id.name}} stopped reporting | This monitor triggers a NO DATA alert when the synthetics.pl.worker.running metric stops reporting data for one of your private locations. This indicates that your private location workers may have been killed or stopped running. |
| [Synthetic Private Locations] {{location_id.name}} is underprovisioned | This monitor triggers an ALERT when the synthetics.pl.worker.remaining_slots metric goes below 1.5 on average for 30 minutes. This indicates that your private location is underprovisioned. Vertically or horizontally scale your private location to ensure that your private location has enough resources to execute all the tests that are assigned to it. |
| [Synthetic Private Locations] {{location_id.name}} uses an outdated image version | This monitor triggers an ALERT when the synthetics.pl.worker.outdated metric starts reporting 1 for one of your private locations. This indicates that at least one of your private location workers is running an outdated version of the private location image. Check for the latest image version in the Google Container Registry or the Windows Installer List and upgrade your workers to that image version by pulling the datadog/synthetics-private-location-worker image with the latest tag. |
By default, no handle is set in these monitors. To be alerted in case one of your monitors starts failing, add a handle in your monitors’ Notification section.
Monitors in the Monitors tab either have a group that corresponds to your private location ID or are tagged with location_id:<ID_OF_THE_PL>.
Monitor your private locations with the Datadog Agent
In addition to the out-of-the-box private location metrics, Datadog recommends installing the Datadog Agent alongside your private location.
The Datadog Agent enables in-depth visibility of your private locations by providing health metrics for the underlying workers (such as memory usage, limits, CPU, and disk). You can create a graph using these metrics and set an alert for low resources.
Further Reading
Additional helpful documentation, links, and articles:
