Logging with Firelens

AWS Firelens is a log driver for ECS that increases your control over log routing and storage. A typical use case for Firelens is to route logs to a third-party service, such as Datadog, BetterStack, or any other logging service. See the web service type docs for specific configuration information.

Firelens uses Fluent Bit plugins for sending logs to other services. This is an open source tool that contains a number of third party output plugins that can be used with Firelens. After configuring the third-party service, you can use the following templates to route logs to the service.

Configuration Approaches

There are two ways to configure Firelens through Flightcontrol:

Inline Configuration (configSource: "inline"):
- Configuration is defined directly in the dashboard or in your flightcontrol.json file
- Example: The Datadog example below uses this approach
File-based Configuration (configSource: "file" & configPath: "path/to/file.conf"):
- Configuration is stored in a separate file in your repository
- Recommended for complex configurations (multiple outputs, filters, etc.)
- Better for version control and readability
- Example: The BetterStack example below uses this approach

Choose the approach that best fits your needs. For most third-party logging services, we recommend the file-based approach as it provides more flexibility and better maintainability.

When using Firelens with Flightcontrol, you don’t need to manually create or manage AWS ECS Task Definitions. Flightcontrol handles all the AWS infrastructure setup for you. You only need to configure the logging section in your flightcontrol.json file.

44+ Integrations

You can route logs to any Fluent Bit output plugin.

Configuration for common examples are listed here for easy reference:

Datadog Example

Add the following logging and envVariables fields in your Flightcontrol service config. Be sure to replace the values with the specific values for your Datadog instances.

See the Datadog documentation for more information.

{
  "$schema": "https://app.flightcontrol.dev/schema.json",
  "environments": [
    {
      "id": "production",
      "name": "Production",
      // ... other environment configuration ...
      "services": [
        {
          "id": "my-webapp",
          "name": "My Webapp",
          "type": "web",
          // ... other service configuration ...
          "logging": {
            "firelens": {
              "configSource": "inline",
              "config": [
                {
                  "name": "datadog",
                  "match": "*",
                  "options": {
                    "apikey": "${DATADOG_API_KEY}",
                    "Host": "http-intake.logs.us5.datadoghq.com",
                    "dd_service": "service name in Datadog",
                    "dd_source": "optional",
                    "dd_tags": "optional",
                    "TLS": "on",
                    "provider": "ecs"
                  }
                }
              ]
            },
            "envVariables": {
              "DATADOG_API_KEY": {
                "fromParameterStore": "/your-datadog-api-key"
              }
            }
          }
        }
      ]
    }
  ]
}

BetterStack Example

This example uses the config file approach, but you can also add these same values to inline config.

Create a file in your repository (e.g., firelens.conf) with the following content:

[OUTPUT]
    name    http
    match   *
    tls     On
    host    in.logs.betterstack.com
    port    443
    uri     /fluentbit
    header  Authorization Bearer ${LOGTAIL_SOURCE_TOKEN}
    header  Content-Type application/msgpack
    format  msgpack
    retry_limit 5
    net.dns.mode TCP

Reference this file in your Flightcontrol config. Here’s an example flightcontrol.json:

{
  "$schema": "https://app.flightcontrol.dev/schema.json",
  "environments": [
    {
      "id": "production",
      "name": "Production",
      // ... other environment configuration ...
      "services": [
        {
          "id": "my-webapp",
          "name": "My Webapp",
          "type": "web",
          // ... other service configuration ...
          "logging": {
            "firelens": {
              "configSource": "file",
              "configPath": "firelens.conf"
            },
            "envVariables": {
              "LOGTAIL_SOURCE_TOKEN": {
                "fromParameterStore": "/your-betterstack-token"
              }
            }
          }
        }
      ]
    }
  ]
}

See the BetterStack documentation for more information.

Source Maps Watch Paths