Overview

Weaviate is an open source, AI-native vector database that helps create AI-powered applications. With Datadog’s Weaviate integration, you can:

• Monitor usage statistics (such as duration of insert, delete, and maintenance operations) to identify potential storage issues, bottlenecks, and assess the impact of data modifications on system responsiveness.
• Track query latency, rate, and concurrent read/write requests to gain insight into the vector database’s overall responsiveness and load handling capabilities.
• Optimize write-heavy workloads with object statistics, like the average time taken for “put” (write) operations.
• Ensure smooth and efficient data ingestion with import-related metrics that offer insights into operations like data loading process.

This check monitors Weaviate through the Datadog Agent. For more information, see Weaviate monitoring. To learn more about Datadog’s suite of AI integrations, see this blog.

Setup

Follow the instructions below to install and configure this check for an Agent running on a host. For containerized environments, see the Autodiscovery Integration Templates for guidance on applying these instructions.

Installation

Starting from Agent release 7.47.0, the Weaviate check is included in the Datadog Agent package.

Note: This check requires Agent v7.47.0 or later.

Configuration

Weaviate can be configured to expose Prometheus-formatted metrics. The Datadog Agent can collect these metrics using the integration described below. Follow the instructions to configure data collection for your Weaviate instances. For the required configurations to expose the Prometheus metrics, see the Monitoring page in the Weaviate documentation.

In addition, a small subset of metrics can be collected by communicating with different API endpoints. Specifically:

• /v1/meta: Version information
• /v1/nodes: Node-specific metrics such as objects and shards
• /v1/.well-known/live: HTTP response time and service liveness

Note: This check uses OpenMetrics for metric collection, which requires Python 3.

Containerized

Metric collection

Make sure that the Prometheus-formatted metrics are exposed in your Weaviate cluster. You can configure and customize this by following the instructions on the Monitoring page in the Weaviate documentation. For the Agent to start collecting metrics, the Weaviate pods need to be annotated. For more information about annotations, refer to the Autodiscovery Integration Templates for guidance. You can find additional configuration options by reviewing the sample weaviate.d/conf.yaml

Note: The listed metrics can only be collected if they are available. Some metrics are generated only when certain actions are performed. For example, the object deletion metric is exposed only when an object is deleted.

The two most important parameters for configuring the Weaviate check are as follows:

• openmetrics_endpoint: This parameter should be set to the location where the Prometheus-formatted metrics are exposed. The default port is 2112, but it can be configured using the PROMETHEUS_MONITORING_PORT environment variable. In containerized environments, %%host%% should be used for host autodetection.
• weaviate_api_endpoint: This parameter is optional. By default, this parameter is set to <hostname>:8080 and it specifies the configuration of the RESTful API.

If authentication is required for the RESTful API endpoints, the check can be configured to provide an API key as part of the request header.

apiVersion: v1
kind: Pod
# (...)
metadata:
  name: '<POD_NAME>'
  annotations:
    ad.datadoghq.com/weaviate.checks: |
      {
        "weaviate": {
          "init_config": {},
          "instances": [
            {
              "openmetrics_endpoint": "http://%%host%%:2112/metrics",
              "weaviate_api_endpoint": "http://%%host%%:8080",
              "headers": {'Authorization': 'Bearer if_needed_for_auth'}
            }
          ]
        }
      }      
    # (...)
spec:
  containers:
    - name: 'weaviate'
# (...)

Note: You can set these annotations directly in your Weaviate Helm chart using annotations key.

Validation

Run the Agent’s status subcommand and look for weaviate under the Checks section.

Data Collected

Metrics

Events

The Weaviate integration does not include any events.

Service Checks

weaviate.openmetrics.health
Returns CRITICAL if the Agent is unable to connect to the Weaviate OpenMetrics endpoint, otherwise returns OK.
Statuses: ok, critical

weaviate.node.status
Returns CRITICAL if the node is UNAVAILABLE, WARNING if UNHEALTHY and OK if HEALTHY.
Statuses: ok, warning, critical

weaviate.liveness.status
Returns OK if liveness endpoint returns a 200 response, otherwise returns CRITICAL.
Statuses: ok, critical

Troubleshooting

Need help? Contact Datadog support.

Further Reading

Additional helpful documentation, links, and articles:

• Integration roundup: Monitoring your AI stack