Instrumenting Java Serverless Applications Using the Datadog Forwarder
Overview
Some older versions of
datadog-lambda-java
import
log4j <=2.14.0
as a transitive dependency.
Upgrade instructions are below.
Prerequisites
The Datadog Forwarder Lambda function is required to ingest AWS Lambda traces, enhanced metrics, custom metrics, and logs.
To fully instrument your serverless application with distributed tracing, your Java Lambda functions must be using the Java 8 Corretto (java8.al2
), Java 11 (java11
) or Java 17 (java17
) runtime.
Configuration
Install
Install the Datadog Lambda Library locally by adding one of the following code blocks into pom.xml
(Maven) or build.gradle
(Gradle). Replace VERSION
below with the latest release (omitting the preceding v
):
Include the following dependency in your pom.xml
:
<dependency>
<groupId>com.datadoghq</groupId>
<artifactId>datadog-lambda-java</artifactId>
<version>VERSION</version>
</dependency>
Include the following in your build.gradle
:
dependencies {
implementation 'com.datadoghq:datadog-lambda-java:VERSION'
}
Instrument
Install the Datadog Lambda Layer on your function. The latest VERSION
is 15
.
arn:aws:lambda:<AWS_REGION>:464622532012:layer:dd-trace-java:<VERSION>
Configure the following environment variables on your function:
JAVA_TOOL_OPTIONS: -javaagent:"/opt/java/lib/dd-java-agent.jar" -XX:+TieredCompilation -XX:TieredStopAtLevel=1
DD_LOGS_INJECTION: true
DD_JMXFETCH_ENABLED: false
DD_TRACE_ENABLED: true
Wrap your Lambda handler function using the wrapper provided by the Datadog Lambda Library:
public class Handler implements RequestHandler<APIGatewayV2ProxyRequestEvent, APIGatewayV2ProxyResponseEvent> {
public Integer handleRequest(APIGatewayV2ProxyRequestEvent request, Context context){
DDLambda ddl = new DDLambda(request, context); //Required to initialize the trace
do_some_stuff();
make_some_http_requests();
ddl.finish(); //Required to finish the active span.
return new ApiGatewayResponse();
}
}
Subscribe
Subscribe the Datadog Forwarder Lambda function to each of your function’s log groups. This enables you to send metrics, traces, and logs to Datadog.
- Install the Datadog Forwarder if you haven’t.
- Subscribe the Datadog Forwarder to your function’s log groups.
Monitor Java Lambda function cold starts
Cold starts occur when your serverless applications receive sudden increases in traffic, including when the function was previously inactive or when it was receiving a relatively constant number of requests. Users may perceive cold starts as slow response times or lag. Datadog recommends you configure a monitor on Java Lambda function cold starts, and use Datadog Serverless Insights to keep cold starts to a minimum.
To create a Datadog monitor on Java Lambda function cold starts, follow the monitor creation steps with the following criteria:
- Metric Name:
aws.lambda.enhanced.invocations
- From:
runtime:java*
and cold_start:true
- Alert Grouping: Multi Alert, trigger a separate alert for each
function_arn
Tag
Although it is optional, Datadog recommends tagging your serverless applications with the reserved tags env
, service
, and version
. For more information about reserved tags, see the Unified Service Tagging documentation.
Explore
After configuring your function following the steps above, view your metrics, logs, and traces on the Serverless homepage.
Monitor custom business logic
To submit a custom metric, see the sample code below:
public class Handler implements RequestHandler<APIGatewayV2ProxyRequestEvent, APIGatewayV2ProxyResponseEvent> {
public Integer handleRequest(APIGatewayV2ProxyRequestEvent request, Context context){
DDLambda ddl = new DDLambda(request, context);
Map<String,Object> myTags = new HashMap<String, Object>();
myTags.put("product", "latte");
myTags.put("order","online");
// Submit a custom metric
ddl.metric(
"coffee_house.order_value", // Metric name
12.45, // Metric value
myTags); // Associated tags
URL url = new URL("https://example.com");
HttpURLConnection hc = (HttpURLConnection)url.openConnection();
hc.connect();
ddl.finish();
}
}
See the custom metrics documentation for more information on custom metric submission.
Connect logs and traces
To automatically connect Java Lambda function logs and traces, see Connecting Java Logs and Traces for instructions.
Failing to use the correct Java runtime can result in errors, for example: Error opening zip file or JAR manifest missing : /opt/java/lib/dd-java-agent.jar
. Make sure to use java8.al2
or java11
as your runtime, as described above.
Upgrading
The Apache Foundation has announced that log4j, a popular Java logging library, is vulnerable to remote code execution.
Some versions of datadog-lambda-java
include a transitive dependency on log4j that may be vulnerable. The vulnerable versions are:
The latest version of datadog-lambda-java
is . Use this version (omitting the preceeding v
) when following the upgrading instructions below.
If you do not wish to upgrade to 1.4.x
, 0.3.x
is updated with the latest log4j security patches as well.
You may find the latest version of 0.3.x
in the datadog-lambda-java
repository.
The version of the datadog-lambda-java
dependency in your Lambda function is set in pom.xml
(Maven) or build.gradle
(Gradle).
Your pom.xml
file contains a section similar to the following:
<dependency>
<groupId>com.datadoghq</groupId>
<artifactId>datadog-lambda-java</artifactId>
<version>VERSION</version>
</dependency>
Replace VERSION
with the latest version of datadog-lambda-java
(available above).
Then redeploy your Lambda function.
Your build.gradle
file contains a section similar to the following:
dependencies {
implementation 'com.datadoghq:datadog-lambda-java:VERSION'
}
Replace VERSION
with the latest version of datadog-lambda-java
(available above).
Then redeploy your Lambda function.
If you are upgrading from 0.3.x to 1.4.x and you wish to use the dd-trace-java
tracer, find the reference to the dd-trace-java
Lambda layer and change it to:
arn:aws:lambda:<AWS_REGION>:464622532012:layer:dd-trace-java:4