Amazon Elastic File System

Overview

Amazon EFS provides simple, scalable file storage for use with AWS Lambda functions or Amazon EC2 instances.

Enable this integration to collect all your EFS metrics in Datadog.

Setup

Installation

If you haven’t already, set up the Amazon Web Services integration first.

Metric collection

  1. In the AWS integration page, ensure that EFS is enabled under the Metric Collection tab.

  2. Add those permissions to your Datadog IAM policy in order to collect Amazon EFS metrics:

    • elasticfilesystem:DescribeTags: Gets custom tags applied to file systems
    • elasticfilesystem:DescribeFileSystems: Provides a list of active file systems

    For more information, see the EFS policies on the AWS website.

  3. Install the Datadog - Amazon EFS integration.

Log collection

Enable logging

Configure Amazon EFS to send logs either to a S3 bucket or to CloudWatch.

Note: If you log to a S3 bucket, make sure that amazon_efs is set as Target prefix.

Send logs to Datadog

  1. If you haven’t already, set up the Datadog Forwarder Lambda function.

  2. Once the Lambda function is installed, manually add a trigger on the S3 bucket or CloudWatch log group that contains your Amazon EFS logs in the AWS console:

Amazon EFS for Lambda

Amazon EFS for Lambda allows you to connect an EFS to your Lambda functions. Organizations can use EFS for Lambda to simplify their machine learning and data processing workloads to be entirely serverless. To divide Lambda metrics and logs by EFS:

  1. Install the AWS Lambda integration and enable metric collection.

  2. Add this permission to your Datadog IAM policy:

    • elasticfilesystem:DescribeAccessPoints: Lists active EFS connected to Lambda functions

Data Collected

Metrics

aws.efs.burst_credit_balance
(gauge)		The average burst credit balance during the period.
Shown as byte
aws.efs.burst_credit_balance.maximum
(gauge)		The largest burst credit balance for any minute during the period.
Shown as byte
aws.efs.burst_credit_balance.minimum
(gauge)		The smallest burst credit balance for any minute during the period.
Shown as byte
aws.efs.client_connections
(count)		The number of Amazon EC2 instances that are connected to a file system.
aws.efs.data_read_iobytes
(count)		The total number of bytes associated with read operations.
Shown as byte
aws.efs.data_read_iobytes.average
(count)		The average size of read operations during the period.
Shown as byte
aws.efs.data_read_iobytes.maximum
(count)		The size of the largest read operation during the period.
Shown as byte
aws.efs.data_read_iobytes.minimum
(count)		The size of the smallest read operation during the period.
Shown as byte
aws.efs.data_read_iobytes.samplecount
(count)		The sample count of the number of data points used for read operation metrics.
Shown as unit
aws.efs.data_write_iobytes
(count)		The sum of the number bytes associated with write operations.
Shown as byte
aws.efs.data_write_iobytes.average
(count)		The average size of write operations during the period.
Shown as byte
aws.efs.data_write_iobytes.maximum
(count)		The size of the largest write operation during the period.
Shown as byte
aws.efs.data_write_iobytes.minimum
(count)		The size of the smallest write operation during the period.
Shown as byte
aws.efs.data_write_iobytes.samplecount
(count)		The sample count of the number of data points used for write operation metrics.
Shown as unit
aws.efs.metadata_iobytes
(count)		The total number of bytes associated with metadata operations.
Shown as byte
aws.efs.metadata_iobytes.average
(count)		The average metadata operation during the period.
Shown as byte
aws.efs.metadata_iobytes.maximum
(count)		The size of the largest metadata operation during the period.
Shown as byte
aws.efs.metadata_iobytes.minimum
(count)		The size of the smallest metadata operation during the period.
Shown as byte
aws.efs.metadata_iobytes.samplecount
(count)		The sample count of the number of data points used for metadata operation metrics.
Shown as unit
aws.efs.metered_iobytes
(gauge)		The average size of an operation during the period.
Shown as byte
aws.efs.metered_iobytes.maximum
(gauge)		The size of the largest operation during the period.
Shown as byte
aws.efs.metered_iobytes.minimum
(gauge)		The size of the smallest operation during the period.
Shown as byte
aws.efs.metered_iobytes.samplecount
(gauge)		A count of metadata operations.
Shown as byte
aws.efs.metered_iobytes.sum
(gauge)		The total number of bytes associated with metadata operations.
Shown as byte
aws.efs.percent_iolimit
(gauge)		Shows how close a file system is to reaching the I/O limit of the general purpose performance mode.
Shown as percent
aws.efs.permitted_throughput
(gauge)		The average throughput permitted during the period.
Shown as byte
aws.efs.permitted_throughput.maximum
(gauge)		The highest throughput permitted for any minute during the period.
Shown as byte
aws.efs.permitted_throughput.minimum
(gauge)		The smallest throughput permitted for any minute during the period.
Shown as byte
aws.efs.storage_bytes
(gauge)		The latest known metered size (in bytes) of data stored in the file system, including both storage classes.
Shown as byte
aws.efs.total_iobytes
(count)		The total number of bytes associated with all file system operations
Shown as byte
aws.efs.total_iobytes.average
(count)		The average size of an operation during the period
Shown as byte
aws.efs.total_iobytes.maximum
(count)		The size of the largest operation during the period.
Shown as byte
aws.efs.total_iobytes.minimum
(count)		The size of the smallest operation during a period.
Shown as byte
aws.efs.total_iobytes.samplecount
(count)		The sample count of the number of data points used for file system operation metrics.
Shown as unit

Each of the metrics retrieved from AWS are assigned the same tags that appear in the AWS console, including but not limited to host name, security-groups, and more.

Events

The Amazon Elastic File System integration does not include any events.

Service Checks

The Amazon Elastic File System integration does not include any service checks.

Troubleshooting

Need help? Contact Datadog support.

PREVIEWING: may/unit-testing