Overview

See Monitoring ElastiCache performance metrics with Redis or Memcached to learn about key performance metrics, how to collect them, and how Coursera monitors ElastiCache using Datadog.

Setup

If you haven’t already, set up the Amazon Web Services integration first.

Installation without Datadog Agent

1. In the AWS integration page, ensure that ElastiCache is enabled under the Metric Collection tab.

2. Add the following permissions to your Datadog IAM policy in order to collect Amazon ElastiCache metrics. For more information, see the ElastiCache policies on the AWS website.

   AWS Permission	Description
   elasticache:DescribeCacheClusters	List and describe Cache clusters, to add tags and additional metrics.
   elasticache:ListTagsForResource	List custom tags of a cluster, to add custom tags.
   elasticache:DescribeEvents	Add events about snapshots and maintenances.

3. Install the Datadog - Amazon ElastiCache integration.

Installation with Datadog Agent (recommended)

Collecting native metrics with the Agent

The following diagram shows how Datadog collects metrics directly from CloudWatch with the native ElastiCache integration, and how it can additionally collect native metrics directly from the backend technology: Redis or Memcached. By collecting from the backend directly, you have access to a greater number of important metrics, and at a higher resolution.

How this works

Because the Agent metrics are tied to the EC2 instance where the agent is running and not to the actual ElastiCache instance, you need to use the cacheclusterid tag to connect all metrics together. Once the agent is configured with the same tags as the ElastiCache instance, combining Redis/Memcached metrics with ElastiCache metrics is straightforward.

Step-by-step

Since the Agent is not running on an actual ElastiCache instance, but on a remote machine, the key to setting up this integration correctly is telling the Agent where to collect the metrics from.

Gather connection details for your ElastiCache instance

First navigate to the AWS Console, open the ElastiCache section and then the Cache Clusters tab to find the cluster you want to monitor. It should look like:

Then click on the “node” link to access its endpoint URL:

Write down the endpoint URL (for example: replica-001.xxxx.use1.cache.amazonaws.com) and the cacheclusterid (for example: replica-001). You need these values to configure the agent and to create graphs and dashboards.

Configure the Agent

The Redis/Memcached integrations support the tagging of individual cache instances. Originally designed to allow the monitoring of multiple instances on the same machine, these tags can be used to filter and group metrics. Here is an example of a configuration for ElastiCache with Redis using redisdb.yaml. For information about where this file is stored based on your platform, see the Agent configuration directory.

init_config:

instances:
    # Endpoint URL from AWS console
    - host: replica-001.xxxx.use1.cache.amazonaws.com
      port: 6379
      # Cache Cluster ID from AWS console
      tags:
          - cacheclusterid:replicaa-001

Then restart the agent: sudo /etc/init.d/datadog-agent restart (on linux).

Visualize metrics together

After a few minutes, ElastiCache metrics and Redis or Memcached metrics can be accessed in Datadog for graphing, monitoring, etc.

Here’s an example of setting up a graph to combine cache hit metrics from ElastiCache with native latency metrics from Redis using the same cacheclusterid tag replicaa-001.

Data Collected

Metrics

Each of the metrics retrieved from AWS is assigned the same tags that appear in the AWS console, including but not limited to host name, security-groups, and more.

Note: Metrics for ElastiCache Serverless deployments are reported under the same aws.elasticache namespace. These metrics can be distinguished by tags:

• The existing ElastiCache metrics for self-designed caches use the tag cacheclusterid to identify an individual cache
• The serverless cache metrics use the tag clusterid to identify individual caches

Events

The Amazon ElastiCache integration includes events for cluster, cache security groups, and cache parameter groups. See example events below:

Service Checks

The Amazon ElastiCache integration does not include any service checks.

Troubleshooting

Need help? Contact Datadog support.

Further Reading

• Monitoring ElastiCache performance metrics with Redis or Memcached
• Collecting ElastiCache metrics + its Redis/Memcached metrics