Overview
Superwise provides model observability for high-scale machine learning (ML) operations.
Superwise’s model observability gives you visibility and context into your models’ behaviors, so you can easily monitor model risks based on different use cases. With Superwise, data scientists, ML engineers, and business ops get model observability without alert fatigue, so you can be confident about your model management.
Superwise’s model metrics and incidents integration sends out-of-the-box model metrics, including drift, activity, incidents, and custom metrics, directly to Datadog. You get an overview of which models are not predicting the expected results, which can be configured to any use case, logic, segmentation, threshold, and sensitivity.
With the Datadog integration configured in Superwise, standard model metrics are sent to Datadog, and users get model observability dashboards in Datadog. You can configure any specific model metric and incident policy, and send them to Datadog for model observability that is tailored to your use case.
Setup
Go to Superwise portal and select Integrations.
Click Create a new channel and select Datadog.
Input your Datadog API and application keys, and click Test. A test request is sent to your Datadog account to validate the integration. If the request was successfully sent, there is a message in Superwise saying the test was delivered successfully. To finish the setup, click Create channel.
When setup is complete, the new Datadog integration widget is available:
Validation
In Datadog, go to Metrics Explorer and search for the metric superwise.integration.test to verify that the integration between Superwise and Datadog is working.
Data Collected
Metrics
superwise.metric.overall.data_drift
(gauge) | Average distribution shifts across all features.
Shown as day |
superwise.metric.overall.weighted_data_drift
(gauge) | Average distribution shifts across all features weighted by importance.
Shown as day |
superwise.metric.overall.quantity
(gauge) | Number of records sent.
Shown as day |
Events
The Superwise integration does not include any events.
Service Checks
The Superwise integration does not include any service checks.
Troubleshooting
Need help? See the Superwise documentation.
Further Reading
Additional helpful documentation, links, and articles:
