Integrating with Datadog

Datadog is a cloud monitoring platform you can use to provide observability for your Flightcontrol services.

AWS integration is part of Datadog, and you will need to follow these directions to get Datadog and AWS working together.

1. Sign up for Datadog
   • Note the “region” you select (Example: us1, us2, etc.)
2. Open the Account settings
   • NOTE - You’ll need to select the region you are using from the dropdown at top right
   • Create a new API key for AWS integration
   • Copy the API key
3. Go to /account/settings#integrations/amazon-web-services in Datadog
   • Under Configuration, click Automatically Using CloudFormation,
4. Paste the API key in the DdApiKey field
5. Click Create Stack
6. Back in Datadog, paste in the following:
   • AWS account ID (accessible in AWS from the top right menu)
   • AWS role name (DatadogIntegrationRole unless you changed it)
7. Wait until the Datadog CloudFormation stack has completed
   • If the Forwarder stack fails with error like this, you’ll need to request an increase of your Lambda Concurrency Limit (see instructions at the bottom of this page) Specified ReservedConcurrentExecutions for function decreases account's UnreservedConcurrentExecution below its minimum value of [50]
8. Click Install Integration, then you should see a success message
9. You should now see some infrastructure under the Infrastructure menu item.

To Set Up Log Forwarding

Log forwarding to Datadog requires the datadog-forwarder stack to be created in every region you want to forward logs from. The default aws datadog integration automatically installs it in us-east-1. See instructions below for installing it in a new region.

How to Install datadog-forwarder in a new region

1. Go to https://us5.datadoghq.com/logs/onboarding/cloud

2. Click on the CloudFormation stack link, then click Launch Stack

   

3. Change the AWS region to your desired region, using the AWS menu

   

4. In the CloudFormation create stack screen, make sure you:

   • Set DdApiKey to your api key (visible on https://us5.datadoghq.com/logs/onboarding/cloud)
   • Set DdSite to your account url (example: us5.datadoghq.com)

5. Click the orange Create Stack button.

How to set up automatic forwarding of some services

1. Go to the Datadog Account Settings (remember you’ll need to select your datadog region from the little dropdown)

2. Click on Collect Logs tab

   

3. Inside AWS, find the ARN value of the datadog-forwarder lambda function and paste that into this Datadog form.

   

4. Select all the services you’d like to receive logs from, then click Save

For more details on setting up log triggers, see Send AWS services logs with the datadog lambda function.

How to set up log forwarding for ECS logs

1. Find the datadog-forwarder lambda in the region you want
2. Click Add trigger
3. Click Select Trigger and select CloudWatch Logs
4. Click log group and find the log group for your ECS service in format /ecs/<service-id>-randomeTaskDefinition
5. Enter a name to remember this by in “filter name”
6. Click Add

To Increase Lambda Concurrency Limit

1. Go to Quotas

2. Select Concurrent Executions, then click Request quota increase

   

3. Enter a higher quota value, then click Request