Instrumenting Python Serverless Applications
Instrumenting Python Serverless Applications
Agentically add Datadog to your Lambda Functions
Agentic onboarding for Datadog Serverless is in Preview. Use your favorite AI coding tool such as Cursor or Claude to bulk-add Datadog monitoring to your Lambda functions.
Request AccessSetup
The Datadog CLI modifies existing Lambda functions’ configurations to enable instrumentation without requiring a new deployment. It is the quickest way to get started with Datadog’s serverless monitoring.
Install the Datadog CLI client
npm install -g @datadog/datadog-ci @datadog/datadog-ci-plugin-lambdaIf you are new to Datadog serverless monitoring, launch the Datadog CLI in the interactive mode to guide your first installation for a quick start, and you can ignore the remaining steps. To permanently install Datadog for your production applications, skip this step and follow the remaining ones to run the Datadog CLI command in your CI/CD pipelines after your normal deployment.
datadog-ci lambda instrument -iConfigure the AWS credentials
Datadog CLI requires access to the AWS Lambda service, and depends on the AWS JavaScript SDK to resolve the credentials. Ensure your AWS credentials are configured using the same method you would use when invoking the AWS CLI.
Configure the Datadog site
export DATADOG_SITE="<DATADOG_SITE>"Replace
<DATADOG_SITE>with(ensure the correct SITE is selected on the right).Configure the Datadog API key
Datadog recommends saving the Datadog API key in AWS Secrets Manager for security and easy rotation. The key needs to be stored as a plaintext string (not a JSON blob). Ensure your Lambda functions have the required
secretsmanager:GetSecretValueIAM permission.export DATADOG_API_KEY_SECRET_ARN="<DATADOG_API_KEY_SECRET_ARN>"For quick testing purposes, you can also set the Datadog API key in plaintext:
export DATADOG_API_KEY="<DATADOG_API_KEY>"Instrument your Lambda functions
Note: Instrument your Lambda functions in a dev or staging environment first. If the instrumentation result is unsatisfactory, run
uninstrumentwith the same arguments to revert the changes.To instrument your Lambda functions, run the following command.
datadog-ci lambda instrument -f <functionname> -f <another_functionname> -r <aws_region> -v 123 -e 93To fill in the placeholders:
- Replace
<functionname>and<another_functionname>with your Lambda function names. Alternatively, you can use--functions-regexto automatically instrument multiple functions whose names match the given regular expression. - Replace
<aws_region>with the AWS region name.
Additional parameters can be found in the CLI documentation.
- Replace
The Datadog Serverless Plugin automatically configures your functions to send metrics, traces, and logs to Datadog through the Datadog Lambda Extension.
To install and configure the Datadog Serverless Plugin, follow these steps:
Install the Datadog Serverless Plugin:
serverless plugin install --name serverless-plugin-datadogUpdate your
serverless.yml:custom: datadog: site: <DATADOG_SITE> apiKeySecretArn: <DATADOG_API_KEY_SECRET_ARN>To fill in the placeholders:
- Replace
<DATADOG_SITE>with(ensure the correct SITE is selected on the right). - Replace
<DATADOG_API_KEY_SECRET_ARN>with the ARN of the AWS secret where your Datadog API key is securely stored. The key needs to be stored as a plaintext string (not a JSON blob). Thesecretsmanager:GetSecretValuepermission is required. For quick testing, you can instead useapiKeyand set the Datadog API key in plaintext.
For more information and additional settings, see the plugin documentation.
- Replace
The Datadog CloudFormation macro automatically transforms your SAM application template to install Datadog on your functions using Lambda layers, and configures your functions to send metrics, traces, and logs to Datadog through the Datadog Lambda Extension.
Install the Datadog CloudFormation macro
Run the following command with your AWS credentials to deploy a CloudFormation stack that installs the macro AWS resource. You only need to install the macro once for a given region in your account. Replace
create-stackwithupdate-stackto update the macro to the latest version.aws cloudformation create-stack \ --stack-name datadog-serverless-macro \ --template-url https://datadog-cloudformation-template.s3.amazonaws.com/aws/serverless-macro/latest.yml \ --capabilities CAPABILITY_AUTO_EXPAND CAPABILITY_IAMThe macro is now deployed and ready to use.
Instrument your Lambda functions
Add the
DatadogServerlesstransform after theAWS::Serverlesstransform under theTransformsection in yourtemplate.ymlfile for SAM.Transform: - AWS::Serverless-2016-10-31 - Name: DatadogServerless Parameters: stackName: !Ref "AWS::StackName" pythonLayerVersion: 123 extensionLayerVersion: 93 site: "<DATADOG_SITE>" apiKeySecretArn: "<DATADOG_API_KEY_SECRET_ARN>"To fill in the placeholders:
- Replace
<DATADOG_SITE>with(ensure the correct SITE is selected on the right). - Replace
<DATADOG_API_KEY_SECRET_ARN>with the ARN of the AWS secret where your Datadog API key is securely stored. The key needs to be stored as a plaintext string (not a JSON blob). Thesecretsmanager:GetSecretValuepermission is required. For quick testing, you can useapiKeyinstead and set the Datadog API key in plaintext.
More information and additional parameters can be found in the macro documentation.
- Replace
The Datadog CDK construct automatically installs Datadog on your functions using Lambda layers. It configures your functions to send metrics, traces, and logs to Datadog through the Datadog Lambda Extension.
TypeScript
Install the Datadog CDK constructs library
For AWS CDK v1:
npm install datadog-cdk-constructs --save-devFor AWS CDK v2:
npm install datadog-cdk-constructs-v2 --save-devInstrument your Lambda functions
For AWS CDK v1:
import { Datadog } from "datadog-cdk-constructs"; const datadog = new Datadog(this, "Datadog", { pythonLayerVersion: 123, extensionLayerVersion: 93, site: "<DATADOG_SITE>", apiKeySecretArn: "<DATADOG_API_KEY_SECRET_ARN>" }); datadog.addLambdaFunctions([<LAMBDA_FUNCTIONS>])For AWS CDK v2:
import { DatadogLambda } from "datadog-cdk-constructs-v2"; const datadogLambda = new DatadogLambda(this, "datadogLambda", { pythonLayerVersion: 123, extensionLayerVersion: 93, site: "<DATADOG_SITE>", apiKeySecretArn: "<DATADOG_API_KEY_SECRET_ARN>" }); datadogLambda.addLambdaFunctions([<LAMBDA_FUNCTIONS>])
Python
Install the Datadog CDK constructs library
For AWS CDK v1:
pip install datadog-cdk-constructsFor AWS CDK v2:
pip install datadog-cdk-constructs-v2Instrument your Lambda functions
For AWS CDK v1:
from datadog_cdk_constructs import Datadog datadog = Datadog(self, "Datadog", python_layer_version=123, extension_layer_version=93, site="<DATADOG_SITE>", api_key_secret_arn="<DATADOG_API_KEY_SECRET_ARN>", ) datadog.add_lambda_functions([<LAMBDA_FUNCTIONS>])For AWS CDK v2:
from datadog_cdk_constructs_v2 import DatadogLambda datadog = DatadogLambda(self, "datadogLambda", python_layer_version=123, extension_layer_version=93, site="<DATADOG_SITE>", api_key_secret_arn="<DATADOG_API_KEY_SECRET_ARN>", ) datadog.add_lambda_functions([<LAMBDA_FUNCTIONS>])
Go
Note: Go CDK constructs are only available for AWS CDK v2.
Install the Datadog CDK constructs library
go get github.com/DataDog/datadog-cdk-constructs-go/ddcdkconstruct/v3Instrument your Lambda functions
import ( "github.com/DataDog/datadog-cdk-constructs-go/ddcdkconstruct/v3" ) datadogLambda := ddcdkconstruct.NewDatadogLambda( stack, jsii.String("Datadog"), &ddcdkconstruct.DatadogLambdaProps{ PythonLayerVersion: jsii.Number(123), ExtensionLayerVersion: jsii.Number(93), Site: jsii.String("<DATADOG_SITE>"), ApiKeySecretArn: jsii.String("<DATADOG_API_KEY_SECRET_ARN>"), }) datadogLambda.AddLambdaFunctions(&[]interface{}{<LAMBDA_FUNCTIONS>}, nil)
To fill in the placeholders:
- Replace
<DATADOG_SITE>with(ensure the correct SITE is selected on the right). - Replace
<DATADOG_API_KEY_SECRET_ARN>with the ARN of the AWS secret where your Datadog API key is securely stored. The key needs to be stored as a plaintext string (not a JSON blob). Ensure your Lambda execution role has thesecretsmanager:GetSecretValueIAM permission in order to read the secret value. For quick testing, you can useapiKeyinstead and set the Datadog API key in plaintext.
More information and additional parameters can be found on the Datadog CDK documentation.
Install the Datadog Lambda Library
If you are deploying your Lambda function as a container image, you cannot use the Datadog Lambda library as a Lambda Layer. Instead, you must install the Datadog Lambda library as a dependency of your function within the image.
pip install datadog-lambdaNote that the minor version of the
datadog-lambdapackage always matches the layer version. For example,datadog-lambda v0.5.0matches the content of layer version 5.Install the Datadog Lambda Extension
Add the Datadog Lambda Extension to your container image by adding the following to your Dockerfile:
COPY --from=public.ecr.aws/datadog/lambda-extension:<TAG> /opt/. /opt/Replace
<TAG>with either a specific version number (for example,93) or withlatest. Alpine is also supported with specific version numbers (such as93-alpine) or withlatest-alpine. You can see a complete list of possible tags in the Amazon ECR repository.Redirect the handler function
- Set your image’s
CMDvalue todatadog_lambda.handler.handler. You can set this in AWS or directly in your Dockerfile. Note that the value set in AWS overrides the value in the Dockerfile if you set both. - Set the environment variable
DD_LAMBDA_HANDLERto your original handler, for example,myfunc.handler.
Note: If you are using a third-party security or monitoring tool that is incompatible with the Datadog handler redirection, you can apply the Datadog wrapper in your function code instead.
- Set your image’s
Configure the Datadog site, API key, and tracing in your Dockerfile
- Set the environment variable
DD_SITEto(ensure the correct SITE is selected on the right). - Set the environment variable
DD_API_KEY_SECRET_ARNwith the ARN of the AWS secret where your Datadog API key is securely stored. The key needs to be stored as a plaintext string (not a JSON blob). Thesecretsmanager:GetSecretValuepermission is required. For quick testing, you can useDD_API_KEYinstead and set the Datadog API key in plaintext. - Set the environment variable
DD_TRACE_ENABLEDtotrue.
- Set the environment variable
The lambda-datadog Terraform module wraps the aws_lambda_function resource and automatically configures your Lambda function for Datadog Serverless Monitoring by:
- Adding the Datadog Lambda layers
- Redirecting the Lambda handler
- Enabling the collection and sending of metrics, traces, and logs to Datadog
module "lambda-datadog" {
source = "DataDog/lambda-datadog/aws"
version = "4.0.0"
environment_variables = {
"DD_API_KEY_SECRET_ARN" : "<DATADOG_API_KEY_SECRET_ARN>"
"DD_ENV" : "<ENVIRONMENT>"
"DD_SERVICE" : "<SERVICE_NAME>"
"DD_SITE": "<DATADOG_SITE>"
"DD_VERSION" : "<VERSION>"
}
datadog_extension_layer_version = 93
datadog_python_layer_version = 123
# aws_lambda_function arguments
}
Replace the
aws_lambda_functionresource with thelambda-datadogTerraform module. Then, specify thesourceandversionof the module.Set the
aws_lambda_functionarguments:All of the arguments available in the
aws_lambda_functionresource are available in this Terraform module. Arguments defined as blocks in theaws_lambda_functionresource are redefined as variables with their nested arguments.For example, in
aws_lambda_function,environmentis defined as a block with avariablesargument. In thelambda-datadogTerraform module, the value for theenvironment_variablesis passed to theenvironment.variablesargument inaws_lambda_function. See inputs for a complete list of variables in this module.Fill in the environment variable placeholders:
- Replace
<DATADOG_API_KEY_SECRET_ARN>with the ARN of the AWS secret where your Datadog API key is securely stored. The key needs to be stored as a plaintext string (not a JSON blob). Thesecretsmanager:GetSecretValuepermission is required. For quick testing, you can instead use the environment variableDD_API_KEYand set your Datadog API key in plaintext. - Replace
<ENVIRONMENT>with the Lambda function’s environment, such asprodorstaging - Replace
<SERVICE_NAME>with the name of the Lambda function’s service - Replace
<DATADOG_SITE>with. (Ensure the correct Datadog site is selected on this page). - Replace
<VERSION>with the version number of the Lambda function
- Replace
Select the versions of the Datadog Extension Lambda layer and Datadog Python Lambda layer to use. If left blank the latest layer versions will be used.
datadog_extension_layer_version = 93
datadog_python_layer_version = 123
To configure Datadog using SST v3, follow these steps:
const app = new sst.aws.Function("MyApp", {
handler: "lambda_function.lambda_handler",
runtime: "python3.13",
environment: {
DD_ENV: "<ENVIRONMENT>",
DD_SERVICE: "<SERVICE_NAME>",
DD_VERSION: "<VERSION>",
DATADOG_API_KEY_SECRET_ARN: "<DATADOG_API_KEY_SECRET_ARN>",
DD_SITE: "<DATADOG_SITE>",
},
layers: [
$interpolate`arn:aws:lambda:${aws.getRegionOutput().name}:464622532012:layer:Datadog-Extension:93`,
$interpolate`arn:aws:lambda:${aws.getRegionOutput().name}:464622532012:layer:Datadog-<RUNTIME>:123`,
],
});
Configure the Datadog Lambda Library and Datadog Lambda Extension layers
- The available
<RUNTIME>options are:Python38,Python39,Python310,Python311,Python312,Python313,Python314.
- The available
Fill in the environment variable placeholders:
- Replace
<DATADOG_API_KEY_SECRET_ARN>with the ARN of the AWS secret where your Datadog API key is securely stored. The key needs to be stored as a plaintext string (not a JSON blob). Thesecretsmanager:GetSecretValuepermission is required. For quick testing, you can instead use the environment variableDD_API_KEYand set your Datadog API key in plaintext. - Replace
<ENVIRONMENT>with the Lambda function’s environment, such asprodorstaging - Replace
<SERVICE_NAME>with the name of the Lambda function’s service - Replace
<DATADOG_SITE>with. (Ensure the correct Datadog site is selected on this page). - Replace
<VERSION>with the version number of the Lambda function
- Replace
Install the Datadog Lambda library
The Datadog Lambda Library can be imported either as a layer (recommended) OR as a Python package.
The minor version of the
datadog-lambdapackage always matches the layer version. For example, datadog-lambda v0.5.0 matches the content of layer version 5.Option A: Configure the layers for your Lambda function using the ARN in the following format:
# Use this format for x86-based Lambda deployed in AWS commercial regions arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-<RUNTIME>:123 # Use this format for arm64-based Lambda deployed in AWS commercial regions arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-<RUNTIME>-ARM:123 # Use this format for x86-based Lambda deployed in AWS GovCloud regions arn:aws-us-gov:lambda:<AWS_REGION>:002406178527:layer:Datadog-<RUNTIME>:123 # Use this format for arm64-based Lambda deployed in AWS GovCloud regions arn:aws-us-gov:lambda:<AWS_REGION>:002406178527:layer:Datadog-<RUNTIME>-ARM:123Replace
<AWS_REGION>with a valid AWS region, such asus-east-1. The available<RUNTIME>options are:Python38,Python39,Python310,Python311,Python312,Python313,Python314.Option B: If you cannot use the prebuilt Datadog Lambda layer, alternatively install the
datadog-lambdapackage and its dependencies locally to your function project folder using your favorite Python package manager, such aspip.pip install datadog-lambda -t ./Note:
datadog-lambdadepends onddtrace, which uses native extensions; therefore it must be installed and compiled in a Linux environment on the right architecture (x86_64orarm64). For example, you can use dockerizePip for the Serverless Framework and –use-container for AWS SAM. For more details, see how to add dependencies to your function deployment package.See the latest release.
Install the Datadog Lambda Extension
Configure the layers for your Lambda function using the ARN in the following format:
# Use this format for x86-based Lambda deployed in AWS commercial regions arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension:93 # Use this format for arm64-based Lambda deployed in AWS commercial regions arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-ARM:93 # Use this format for x86-based Lambda deployed in AWS GovCloud regions arn:aws-us-gov:lambda:<AWS_REGION>:002406178527:layer:Datadog-Extension:93 # Use this format for arm64-based Lambda deployed in AWS GovCloud regions arn:aws-us-gov:lambda:<AWS_REGION>:002406178527:layer:Datadog-Extension-ARM:93Replace
<AWS_REGION>with a valid AWS region, such asus-east-1.Redirect the handler function
- Set your function’s handler to
datadog_lambda.handler.handler. - Set the environment variable
DD_LAMBDA_HANDLERto your original handler, for example,myfunc.handler.
Note: If you are using a third-party security or monitoring tool that is incompatible with the Datadog handler redirection, you can apply the Datadog wrapper in your function code instead.
- Set your function’s handler to
Configure the Datadog site, API key, and tracing
- Set the environment variable
DD_SITEto(ensure the correct SITE is selected on the right). - Set the environment variable
DD_API_KEY_SECRET_ARNwith the ARN of the AWS secret where your Datadog API key is securely stored. The key needs to be stored as a plaintext string, instead of being inside a json blob. Thesecretsmanager:GetSecretValuepermission is required. For quick testings, you can useDD_API_KEYinstead and set the Datadog API key in plaintext. - Set the environment variable
DD_TRACE_ENABLEDtotrue.
- Set the environment variable
(AWS Chalice only) Register the middleware
If you are using AWS Chalice, you must install
datadog-lambdausingpip, and registerdatadog_lambda_wrapperas a middleware in yourapp.py:from chalice import Chalice, ConvertToMiddleware from datadog_lambda.wrapper import datadog_lambda_wrapper app = Chalice(app_name='hello-chalice') app.register_middleware(ConvertToMiddleware(datadog_lambda_wrapper)) @app.route('/') def index(): return {'hello': 'world'}
Do not set the following environment variables in your serverless environment. They should only be set in non-serverless environments.
DD_AGENT_HOSTDD_TRACE_AGENT_URL
FIPS compliance
Datadog provides FIPS-compliant monitoring for AWS Lambda functions. For AWS GovCloud environments, the DD_LAMBDA_FIPS_MODE environment variable is enabled by default. When FIPS mode is enabled, AWS FIPS endpoints are used for Datadog API key lookups, and the Lambda metric helper function lambda_metric requires the FIPS-compliant extension for metric submission. While the FIPS-compliant Lambda components work with any Datadog site, end-to-end FIPS compliance requires using the US1-FED site. See AWS Lambda FIPS Compliance for more details.
AWS Lambda and VPC
If your Lambda functions are deployed in a Virtual Private Cloud (VPC) without access to the public internet, and you are using the datadoghq.com Datadog site, you can use AWS PrivateLink to send data.
If your Lambda functions are deployed in a VPC, and you are using a Datadog site that is not datadoghq.com, you can use a proxy to send data.
What’s next?
- Add custom tags to your telemetry by using the
DD_TAGSenvironment variable - Configure payload collection to capture your functions’ JSON request and response payloads
- If you are using the Datadog Lambda Extension, turn off the Datadog Forwarder’s Lambda logs
- See Configure Serverless Monitoring for AWS Lambda for further capabilities
Monitor custom business logic
To monitor your custom business logic, submit a custom metric or span using the sample code below. For additional options, see custom metric submission for serverless applications and the APM guide for custom instrumentation.
import time
from ddtrace import tracer
from datadog_lambda.metric import lambda_metric
def lambda_handler(event, context):
# add custom tags to the lambda function span,
# does NOT work when X-Ray tracing is enabled
current_span = tracer.current_span()
if current_span:
current_span.set_tag('customer.id', '123456')
# submit a custom span
with tracer.trace("hello.world"):
print('Hello, World!')
# submit a custom metric
lambda_metric(
metric_name='coffee_house.order_value',
value=12.45,
tags=['product:latte', 'order:online']
)
return {
'statusCode': 200,
'body': get_message()
}
# trace a function
@tracer.wrap()
def get_message():
return 'Hello from serverless!'
Further Reading
Additional helpful documentation, links, and articles:
| |
Request a personalized demo
AI-generated responses may be inaccurate. Verify important info.
How can I help you today?
Ask me anything about Datadog documentation