Understanding ECS-EC2 in Flightcontrol

When deploying applications with Flightcontrol, you have two main target options: fargate and ecs-ec2. While Fargate is the default and simplest option, ECS-EC2 offers more control, potential cost savings, and support for GPUs. However, it requires a deeper understanding of how AWS ECS and EC2 instances work together.

Fargate vs. ECS-EC2: Key Differences

Understanding the Relationship Between Cluster Instances and App Instances

One of the most common sources of confusion with ECS-EC2 is understanding how cluster instances (EC2 machines) relate to app instances (containers).

Key Concepts

1. Cluster Instances: These are the actual EC2 virtual machines that run in your AWS account
2. App Instances: These are the containers running your application code
3. Bin packing/task placement: This is the process of placing one or more app instances on each cluster instance

In Fargate, you don’t need to worry about cluster instances as AWS manages them automatically. In ECS-EC2, you must configure both:

{
  // other configuration...
  "services": [
    {
      "type": "web",
      "name": "api",
      "target": {
        "type": "ecs-ec2",
        "clusterInstanceSize": "t3.medium",
        "clusterMinInstances": 2,
        "clusterMaxInstances": 10
      },
      "minInstances": 2,
      "maxInstances": 4
      // other configuration...
    }
  ]
}

How Zero-Downtime Deployments Work

When you deploy a new version of your application, Flightcontrol performs a zero-downtime deployment:

1. It builds your new application version
2. It launches new app instances with the new version
3. At this step, you have twice your desired number of app instances running simultaneously.
   • The existing instances are still serving traffic
   • The new instances are booting up and passing health checks
4. Once all the new instances are healthy, they start serving traffic
5. Finally, the old instances are drained and terminated

Common Deployment Issues and Solutions

”Service Still Updating” Timeout Errors

This error typically occurs when there aren’t enough EC2 cluster instances to accommodate both the old and new app instances during deployment.

Solution: Ensure your clusterMaxInstances is at least 2x your app maxInstances value.

For example, if you have:

"minInstances": 2,
"maxInstances": 4

You should set:

"target": {
  "type": "ecs-ec2",
  "clusterMinInstances": 2,
  "clusterMaxInstances": 8  // At least 2x your instances.max
}

Slow Deployments

If your deployments are taking a long time, it might be because AWS needs to launch new EC2 instances, which can take several minutes.

Solution: Increase your clusterMinInstances to ensure there are always spare EC2 instances available for new deployments. For the fastest deployments, set clusterMinInstances to 2x the number of instances normally running.

Best Practices for ECS-EC2 Configuration

1. Set appropriate instance sizes: Choose an EC2 instance type that efficiently accommodates your app’s CPU and memory requirements.

2. Configure proper cluster capacity:

   • clusterMaxInstances: Set to at least 2x your maxInstances value

3. Understand resource allocation:

   • Each app instance requires the CPU and memory you specify
   • Multiple app instances can run on a single EC2 instance if there’s enough capacity
   • The ECS agent requires some overhead, roughly 300-500MB per instance

4. Monitor and optimize:

   • Regularly review your EC2 instance utilization
   • Adjust instance types and counts based on actual usage patterns
   • Consider Reserved Instances for predictable workloads to save costs

Running Multiple App Instances per EC2 Instance

Depending on how much CPU and memory your app instances need, you could benefit from running multiple app instances per EC2 instance. This reduces wasted memory for ECS agents. However, Flightcontrol deploys with high availability, so at least 1 instance is placed in each availability zone.

For example, if your app requires 0.5 vCPU and 1GB memory, and you use a t3.medium instance (2 vCPU, 4GB memory), you could potentially run up to 3 app instances per EC2 instance (accounting for ECS agent overhead).

When bin packing:

1. Ensure your clusterMaxInstances accounts for the worst-case scenario where only one app instance fits per EC2 instance during deployment
2. Monitor resource utilization closely to avoid performance issues

Conclusion

ECS-EC2 offers greater control and potential cost savings compared to Fargate, but requires more careful configuration. By understanding the relationship between cluster instances and app instances and following the best practices outlined above, you can avoid common deployment issues and optimize your infrastructure costs.