Configuring Scheduler Services with flightcontrol.json
Scheduler Service Attributes
The type for these services is scheduler
, and should be specified like the following:
type: 'scheduler'
Common Service Configuration
Before configuring the service-specific options below, you should review the Common Service Configuration that applies to all service types. This includes important settings for:
- Name
- Environment variables
- Build configuration
- Watch paths
- And more…
Scheduler-Specific Options
The following attributes are specific to static web services:
Target
target.type: 'fargate' | 'ecs-ec2'
- Example:
"target": {"type": "fargate"}
- Defaults to
fargate
- Fargate is a fully managed, pay-as-you-go compute engine that lets you focus on building apps without managing servers. It’s the easiest to use. The tradeoffs are that it does not support GPUs or custom instance types, and can be up to 1.5 times more expensive than ECS+EC2, depending on how well you optimize the EC2 compute.
- ECS+EC2 is AWS’s version of Kubernetes. It gives you advance control over the cluster instance type and supports GPUs. It enables leveraging Reserved Instances for 50% or more savings. The tradeoffs are that 1) you have to manage the instance size to ensure there is enough CPU and memory for your app, 2) autoscaling is up to 2x slower if you don’t have empty EC2 instances on standby, and 3) you have to manage the config options optimize compute across instances to minimize wasted resources.
If target is fargate
: no other fields
If target is ecs-ec2
:
target.clusterInstanceSize: string
- Example:
"clusterInstanceSize": "t3.medium"
- Supported values: all non-ARM based instance sizes. See all instance options.
target.clusterMinInstances: number
- Example:
"clusterMinInstances": 1
- Minimum
1
- Fastest deploys are possible when this is at least 2x the number of app instances running
- Faster autoscaling is enabled when this is higher than minimum app instances so that machines are already running
target.clusterMaxInstances: number
- Example:
"clusterMaxInstances": 5
- Minimum same as
clusterMinInstances
- Must be high enough to accommodate 2x your app max instances count, otherwise deploys may fail
Scheduler CPU
cpu: number
- Example:
"cpu": 0.25
- This is the AWS
vCPU
unit for each service instance. It does not correspond to number of cores. It’s an abstract unit of CPU power defined by Amazon - This value will be used as the default CPU for the job when the CPU is not set in the job
- Supported values:
- If target
fargate
:0.25, 0.5, 1, 2, 4, 8, 16
- For more details on Fargate configuration, see AWS’s Fargate pricing page
- If target
ecs-ec2
: range:0.125
to10
or the vCPU ofclusterInstanceSize
, whichever is less
- If target
Scheduler Memory
memory: number
- Example:
"memory": 1
- In gigabytes
- This value will be used as the default memory for the job when the memory is not set in the job
- Supported values:
- If target
fargate
:- With
cpu: 0.25
-0.5, 1, 2
- With
cpu: 0.5
-1...4
(intervals of 1) - With
cpu: 1
-2...8
(intervals of 1) - With
cpu: 2
-4...16
(intervals of 1) - With
cpu: 4
-8...30
(intervals of 1) - With
cpu: 8
-16...60
(intervals of 4) - With
cpu: 16
-32...120
(intervals of 8) - For more details on Fargate configuration, see AWS’s Fargate pricing page
- With
- If target
ecs-ec2
: range:0.125
to0.25
less than the memory ofclusterInstanceSize
. (0.25
GB is reserved for the ECS agent)
- If target
Scheduler GPU
gpu: integer
- Optional, with default:
0
- Range: 0 to 16
- This is how many GPUs to expose to your container
- This value will be used as the default GPU for the job when the GPU is not set in the job
- Supported values:
- If target
fargate
:0
or unset - If target
ecs-ec2
: Range0
to the GPU ofclusterInstanceSize
(requires a GPU compatible instance size)
- If target
Jobs
jobs: Record<string, JobConfig>
- Example:
{
"jobs": {
"process-reports": {
"startCommand": "php artisan reports:process",
"schedule": "0 * * * *"
}
}
}
- An object where each key is a unique
givenId
for the job - The
givenId
will be used to:- Display the job in the dashboard
- Generate CloudWatch log streams
- Reference the job in API calls
- See the JobConfig section below for all supported configuration options
Complete Example
Here’s a comprehensive example of a scheduler service configuration:
{
"type": "scheduler",
"jobs": {
"daily-report": {
"startCommand": "php artisan reports:generate-daily",
"schedule": "0 0 * * *",
"timezone": "America/New_York",
"timeout": 30,
"cpu": 1,
"memory": 2
},
"weekly-cleanup": {
"startCommand": "php artisan storage:cleanup",
"schedule": "0 0 * * 0",
"timezone": "UTC",
"timeout": 60,
"cpu": 0.5,
"memory": 1
},
"manual-data-sync": {
"startCommand": "php artisan data:sync",
"schedule": "manual",
"cpu": 2,
"memory": 4
}
}
}
Job Execution Behavior
Concurrency
- Each job runs in isolation
- Many jobs can run concurrently
Failure Handling
- Jobs are considered failed if they exit with a non-zero status code
- Failed jobs are recorded in our dashboard along with their error output
- No automatic retry mechanism is provided - use your application logic for retries
JobConfig
Each job in the jobs
object supports the following configuration options:
startCommand: string | string[]
- Example:
// String format "startCommand": "php artisan run:billing-cycle" // Array format (Custom Dockerfile only) "startCommand": ["php", "artisan", "run:billing-cycle"]
- The command to execute when the job runs
- For Nixpacks builds: Must be a string
- For Custom Dockerfile: Can be string or string array
- Executes from the application directory
- Must exit with code 0 for success, any other exit code marks the job as failed
schedule: string
- Example:
"schedule": "* * 1 * *"
- Supported values:
- Cron-like string:
* * * * *
, you can use crontab guru to design or test your schedule - Manually triggered job or command:
manual
- Cron-like string:
- Schedule is used to define the interval of running this job or to mark the job as manual and runs on demand
timezone: string
- Example:
"timezone": "America/New_York"
- Optional with default:
"UTC"
- Supports any of the available timezones here.
cpu: number | undefined
- Example:
"cpu": 0.25
- Optional, defaults to the CPU set in the base scheduler config
- This is the AWS
vCPU
unit for each service instance. It does not correspond to number of cores. It’s an abstract unit of CPU power defined by Amazon - Supported values:
- If target
fargate
:0.25, 0.5, 1, 2, 4, 8, 16
- For more details on Fargate configuration, see AWS’s Fargate pricing page
- If target
ecs-ec2
: range:0.125
to10
or the vCPU ofclusterInstanceSize
, whichever is less
- If target
memory: number | undefined
- Example:
"memory": 1
- Optional, defaults to the memory set in the base scheduler config
- In gigabytes
- Supported values:
- If target
fargate
:- With
cpu: 0.25
-0.5, 1, 2
- With
cpu: 0.5
-1...4
(intervals of 1) - With
cpu: 1
-2...8
(intervals of 1) - With
cpu: 2
-4...16
(intervals of 1) - With
cpu: 4
-8...30
(intervals of 1) - With
cpu: 8
-16...60
(intervals of 4) - With
cpu: 16
-32...120
(intervals of 8) - For more details on Fargate configuration, see AWS’s Fargate pricing page
- With
- If target
ecs-ec2
: range:0.125
to0.25
less than the memory ofclusterInstanceSize
. (0.25
GB is reserved for the ECS agent)
- If target
gpu: integer | undefined
- Optional, with default:
0
- Range: 0 to 16
- This is how many GPUs to expose to your container
- This value will be used as the default GPU for the job when the GPU is not set in the job
- Supported values:
- If target
fargate
:0
or unset - If target
ecs-ec2
: Range0
to the GPU ofclusterInstanceSize
(requires a GPU compatible instance size)
- If target
timeout: int
- Example:
"timeout": 10
- Timeout is in minutes
- Optional with default:
480
- Supported values:
1...1440
Container Insights
containerInsights: boolean
- Example:
"containerInsights": true
- Optional with default:
false
- Enables AWS Container Insights for your service. This will send metrics to CloudWatch for your service. For more, see our Container Insights guide.
Docker Labels
dockerLabels: Record<string, string>
- Example:
"dockerLabels": {"com.example.vendor": "ACME"}
- Optional
- Will apply the set labels to the container
Version History Count
versionHistoryCount: number
- Example:
"versionHistoryCount": 15
- Optional with default:
10
- How many previous container images to keep in ECR. This determines how far back you can rollback. ECR storage is $0.10 per GB / month, so this configuration is balance between cost and rollback history.
Runtime-only Environment variables
includeEnvVariablesInBuild: boolean
- Example:
"includeEnvVariablesInBuild": false
- Optional with default:
true
- Exclude environment variables from the build environment - see the Configuring Environment Variables page for more details.
Integrations
integrations: object
Under the integrations
key, you can configure integrations with third-party services. At this time, the only supported integration is with Sentry.
Upload Sentry Source Maps
uploadSentrySourceMap: boolean
- Example:
"integrations": { "uploadSentrySourceMap": true }
Extra options for Nixpacks build type
basePath?: string
(only supported when buildType: nixpacks
)
- Example:
"basePath": "./apps/frontend"
- Optional, defaults to
.
- The directory containing your application code, relative to the repository root
- Allows you to specify in which folder the following build commands should run
installCommand: string
- Example:
"installCommand": "./install.sh"
- Optional, intelligent default based on your language and framework detected at the
basePath
- What we use to install dependencies for your build
buildCommand: string
- Example:
"buildCommand": "blitz build"
- Optional, intelligent default based on your language and framework detected at the
basePath
- What we use to build your app
postBuildCommand: string
- Example:
"postBuildCommand": "./postBuildCommand.sh"
- Optional, Empty by Default
- Used as a build hook to run any operation after your build is complete
- Note:
buildCommand
must be set forpostBuildCommand
to be used
startCommand: string
- Example:
"startCommand": "blitz start"
- Optional, intelligent default based on your language and framework detected at the
basePath
- What we use to start your app
Extra options for Dockerfile build type
Make sure not to add an ENTRYPOINT
with a command that never exits. If you want to purpose the
same image for other services like a web server, use ENTRYPOINT
for jobs and use CMD
for your
server start command.
dockerfilePath: string
- Example:
"dockerfilePath": "packages/web/Dockerfile"
- Relative path to the Dockerfile from your repo root
- It’s recommended to use
ENTRYPOINT
instead ofCMD
for your start command - You can authenticate with Docker Hub by adding your Docker Hub credentials as
DOCKER_USERNAME
andDOCKER_PASSWORD
environment variables. If these env variables are present, we’ll rundocker login
with them. This will prevent Docker Hub rate limit issues.
dockerContext: string
- Example:
"dockerContext": "packages/web"
- Optional with default: the directory containing the Dockerfile
- The build context for the Docker build. This is the directory that will be sent to the Docker daemon for building the image.
startCommand: Array<string>
- Example:
"startCommand": ["node", "index.js"]
- Optional
- You can set this to override the CMD in your Dockerfile
- The command must be split into array parts
injectEnvVariablesInDockerfile: boolean
- Example:
"injectEnvVariablesInDockerfile": false
- Optional with default:
true
- Whether to inject environment variables automatically into Dockerfile or not
- It’s recommended to use Docker build secrets to control how environment variable are used during build, check the guide here
Extra options for From Service build type
containerImage.fromService: string
- Example:
"containerImage": { "fromService": "my-api" }
- Required (non-empty string)
- The Given ID of the service that will be used as the source for the container image. The service specified here needs to be built by Flightcontrol.
Extra options for From Repository build type
Make sure not to add an ENTRYPOINT
with a command that never exits. If you want to purpose the
same image for other services like a web server, use ENTRYPOINT
for jobs and use CMD
for your
server start command.
Example:
"containerImage": {
"registryId": "ecr-9l03731",
"repository": "node:18-slim",
"tag": "latest"
}
containerImage.registryId: string
- Example:
"registryId": "ecr-9l03731"
- Required (non-empty string)
- Registry ID in Flightcontrol, you can find this on the Registries page under Organization Settings
containerImage.repository: string
- Example:
"repository": "node:18-slim"
- Required (non-empty string)
- This is the URI of the image repository you wish to access
containerImage.tag?: string
- Example:
"tag": "latest"
- Optional with default:
"latest"
- Must be non-empty string when provided
- This is the tag of the image from the repository that you would like to use