Deploy SupportPilot to AWS

ECS on Fargate runs the container without managing servers, an Application Load Balancer fronts it, RDS for PostgreSQL stores checkpoints, Secrets Manager injects keys, and GitHub Actions deploys via OIDC.

Your progress

Prerequisites

An AWS account, the AWS CLI v2 configured, and Docker with buildx.
This guide uses the default VPC to stay short. For production, use private subnets + NAT (or VPC endpoints) and define everything in CDK/Terraform — see IaC.

Step-by-step

Variables

export AWS_REGION=us-east-1
export ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
export ECR=$ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com
export VPC_ID=$(aws ec2 describe-vpcs --filters Name=is-default,Values=true --query 'Vpcs[0].VpcId' --output text)
export SUBNETS=$(aws ec2 describe-subnets --filters Name=vpc-id,Values=$VPC_ID \
  --query 'Subnets[].SubnetId' --output text | tr '\t' ',')

Mark step complete

Push the image to ECR

aws ecr create-repository --repository-name supportpilot --image-scanning-configuration scanOnPush=true
aws ecr get-login-password | docker login --username AWS --password-stdin $ECR

docker buildx build --platform linux/amd64 -t $ECR/supportpilot:v1 --push .

Mark step complete

Security groups (ALB → app → DB)

ALB_SG=$(aws ec2 create-security-group --group-name sp-alb --description "ALB" --vpc-id $VPC_ID --query GroupId --output text)
APP_SG=$(aws ec2 create-security-group --group-name sp-app --description "App" --vpc-id $VPC_ID --query GroupId --output text)
DB_SG=$(aws ec2 create-security-group --group-name sp-db --description "DB" --vpc-id $VPC_ID --query GroupId --output text)

aws ec2 authorize-security-group-ingress --group-id $ALB_SG --protocol tcp --port 80 --cidr 0.0.0.0/0
aws ec2 authorize-security-group-ingress --group-id $APP_SG --protocol tcp --port 8080 --source-group $ALB_SG
aws ec2 authorize-security-group-ingress --group-id $DB_SG --protocol tcp --port 5432 --source-group $APP_SG

Each layer only accepts traffic from the layer in front of it — a classic interview talking point.

Mark step complete

Create RDS for PostgreSQL

export DB_PASS=$(openssl rand -hex 16)

aws rds create-db-instance --db-instance-identifier supportpilot-db \
  --engine postgres --engine-version 16 --db-instance-class db.t4g.micro \
  --allocated-storage 20 --storage-encrypted \
  --master-username agent --master-user-password "$DB_PASS" --db-name agent \
  --vpc-security-group-ids $DB_SG --no-publicly-accessible --backup-retention-period 7

aws rds wait db-instance-available --db-instance-identifier supportpilot-db
export DB_HOST=$(aws rds describe-db-instances --db-instance-identifier supportpilot-db \
  --query 'DBInstances[0].Endpoint.Address' --output text)

Mark step complete

Secrets Manager

KEY_ARN=$(aws secretsmanager create-secret --name supportpilot/anthropic-api-key \
  --secret-string "$ANTHROPIC_API_KEY" --query ARN --output text)
DB_ARN=$(aws secretsmanager create-secret --name supportpilot/database-url \
  --secret-string "postgresql://agent:$DB_PASS@$DB_HOST:5432/agent?sslmode=require" \
  --query ARN --output text)

Mark step complete

IAM roles

ECS uses two roles: the execution role (pull image, write logs, read secrets at start-up) and the task role (what your code may call at runtime, e.g. Bedrock).

cat > trust.json <<'EOF'
{"Version":"2012-10-17","Statement":[{"Effect":"Allow",
 "Principal":{"Service":"ecs-tasks.amazonaws.com"},"Action":"sts:AssumeRole"}]}
EOF
aws iam create-role --role-name sp-exec-role --assume-role-policy-document file://trust.json
aws iam attach-role-policy --role-name sp-exec-role \
  --policy-arn arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy
aws iam put-role-policy --role-name sp-exec-role --policy-name read-secrets --policy-document "{
  \"Version\":\"2012-10-17\",\"Statement\":[{\"Effect\":\"Allow\",
  \"Action\":\"secretsmanager:GetSecretValue\",\"Resource\":[\"$KEY_ARN\",\"$DB_ARN\"]}]}"

aws iam create-role --role-name sp-task-role --assume-role-policy-document file://trust.json

Mark step complete

Task definition

{
  "family": "supportpilot",
  "requiresCompatibilities": ["FARGATE"],
  "networkMode": "awsvpc",
  "cpu": "512",
  "memory": "1024",
  "runtimePlatform": { "cpuArchitecture": "X86_64", "operatingSystemFamily": "LINUX" },
  "executionRoleArn": "arn:aws:iam::ACCOUNT_ID:role/sp-exec-role",
  "taskRoleArn": "arn:aws:iam::ACCOUNT_ID:role/sp-task-role",
  "containerDefinitions": [{
    "name": "api",
    "image": "ACCOUNT_ID.dkr.ecr.REGION.amazonaws.com/supportpilot:v1",
    "portMappings": [{ "containerPort": 8080 }],
    "environment": [{ "name": "MODEL", "value": "anthropic:claude-sonnet-5-5" }],
    "secrets": [
      { "name": "ANTHROPIC_API_KEY", "valueFrom": "KEY_ARN" },
      { "name": "DATABASE_URL", "valueFrom": "DB_ARN" }
    ],
    "logConfiguration": {
      "logDriver": "awslogs",
      "options": {
        "awslogs-group": "/ecs/supportpilot",
        "awslogs-region": "REGION",
        "awslogs-stream-prefix": "api",
        "awslogs-create-group": "true"
      }
    }
  }]
}

sed -i.bak -e "s/ACCOUNT_ID/$ACCOUNT_ID/g" -e "s/REGION/$AWS_REGION/g" \
  -e "s|KEY_ARN|$KEY_ARN|" -e "s|DB_ARN|$DB_ARN|" taskdef.json
aws iam put-role-policy --role-name sp-exec-role --policy-name create-logs --policy-document \
  '{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Action":"logs:CreateLogGroup","Resource":"*"}]}'
aws ecs register-task-definition --cli-input-json file://taskdef.json

Mark step complete

Load balancer

ALB_ARN=$(aws elbv2 create-load-balancer --name supportpilot-alb \
  --subnets $(echo $SUBNETS | tr ',' ' ') --security-groups $ALB_SG \
  --query 'LoadBalancers[0].LoadBalancerArn' --output text)

# agent runs and SSE streams can be long — raise the 60s idle timeout
aws elbv2 modify-load-balancer-attributes --load-balancer-arn $ALB_ARN \
  --attributes Key=idle_timeout.timeout_seconds,Value=300

TG_ARN=$(aws elbv2 create-target-group --name supportpilot-tg --protocol HTTP --port 8080 \
  --vpc-id $VPC_ID --target-type ip --health-check-path /healthz \
  --query 'TargetGroups[0].TargetGroupArn' --output text)

aws elbv2 create-listener --load-balancer-arn $ALB_ARN --protocol HTTP --port 80 \
  --default-actions Type=forward,TargetGroupArn=$TG_ARN

Use HTTPS in productionRequest a certificate in ACM, add a 443 listener, and redirect 80 → 443. Add AWS WAF for rate limiting.

Mark step complete

Cluster and service

aws ecs create-cluster --cluster-name agents

aws ecs create-service --cluster agents --service-name supportpilot \
  --task-definition supportpilot --desired-count 2 --launch-type FARGATE \
  --network-configuration "awsvpcConfiguration={subnets=[$SUBNETS],securityGroups=[$APP_SG],assignPublicIp=ENABLED}" \
  --load-balancers "targetGroupArn=$TG_ARN,containerName=api,containerPort=8080" \
  --health-check-grace-period-seconds 60 \
  --deployment-configuration "deploymentCircuitBreaker={enable=true,rollback=true}"

aws ecs wait services-stable --cluster agents --services supportpilot

assignPublicIp=ENABLED lets tasks in default (public) subnets reach ECR and the LLM API. In private subnets, use a NAT gateway or VPC endpoints instead.

Mark step complete

Smoke test and autoscaling

URL=http://$(aws elbv2 describe-load-balancers --load-balancer-arns $ALB_ARN \
  --query 'LoadBalancers[0].DNSName' --output text)
curl -s $URL/healthz
curl -s $URL/chat -H 'content-type: application/json' \
  -d '{"thread_id":"aws-1","message":"Where is order A1002?"}'

aws logs tail /ecs/supportpilot --follow

# scale 2–10 tasks on CPU
aws application-autoscaling register-scalable-target --service-namespace ecs \
  --resource-id service/agents/supportpilot --scalable-dimension ecs:service:DesiredCount \
  --min-capacity 2 --max-capacity 10
aws application-autoscaling put-scaling-policy --service-namespace ecs \
  --resource-id service/agents/supportpilot --scalable-dimension ecs:service:DesiredCount \
  --policy-name cpu60 --policy-type TargetTrackingScaling \
  --target-tracking-scaling-policy-configuration \
  '{"TargetValue":60,"PredefinedMetricSpecification":{"PredefinedMetricType":"ECSServiceAverageCPUUtilization"}}'

Agent workloads are I/O-bound, so CPU can stay low under load. ALBRequestCountPerTarget is often a better scaling metric.

Mark step complete

CI/CD with GitHub Actions (OIDC)

Create an IAM OIDC provider for token.actions.githubusercontent.com and a deploy role whose trust policy is limited to your repo. Its permissions: ECR push, ecs:RegisterTaskDefinition, ecs:UpdateService, ecs:Describe* and iam:PassRole for the two task roles.

name: deploy-aws
on:
  push:
    branches: [main]
permissions:
  contents: read
  id-token: write
jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with: { python-version: "3.12" }
      - run: pip install -r requirements-dev.txt && pytest -q
      - uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: ${{ vars.AWS_DEPLOY_ROLE_ARN }}
          aws-region: us-east-1
      - id: ecr
        uses: aws-actions/amazon-ecr-login@v2
      - id: build
        run: |
          IMAGE=${{ steps.ecr.outputs.registry }}/supportpilot:${{ github.sha }}
          docker build -t $IMAGE . && docker push $IMAGE
          echo "image=$IMAGE" >> $GITHUB_OUTPUT
      - id: taskdef
        uses: aws-actions/amazon-ecs-render-task-definition@v1
        with:
          task-definition: taskdef.json
          container-name: api
          image: ${{ steps.build.outputs.image }}
      - uses: aws-actions/amazon-ecs-deploy-task-definition@v2
        with:
          task-definition: ${{ steps.taskdef.outputs.task-definition }}
          service: supportpilot
          cluster: agents
          wait-for-service-stability: true

Mark step complete

Using Amazon Bedrock models

Call Claude, Llama, Mistral or Amazon Nova through Bedrock with IAM instead of an external API key:

# requirements.txt: add langchain-aws
# MODEL=bedrock_converse:<model id or inference profile id from the Bedrock console>
aws iam put-role-policy --role-name sp-task-role --policy-name bedrock --policy-document \
  '{"Version":"2012-10-17","Statement":[{"Effect":"Allow",
    "Action":["bedrock:InvokeModel","bedrock:InvokeModelWithResponseStream"],"Resource":"*"}]}'

Remove the ANTHROPIC_API_KEY secret from the task definition and set MODEL — the graph code is unchanged. Scope Resource to specific model ARNs in production.

Alternatives on AWS

Option	When to choose it
ECS Fargate (this guide)	Long-running HTTP/SSE agents, steady traffic, full control of networking.
Amazon Bedrock AgentCore	Managed, session-isolated runtime for agents built with any framework (incl. LangGraph), plus managed memory, identity and tool gateway.
Lambda (+ Function URL / API Gateway)	Spiky, short agent runs (< 15 min). Watch cold starts and package size.
EKS	Existing Kubernetes platform or self-hosted models on GPUs.

Infrastructure as code

The CLI steps above are great for learning. For real environments, encode them in AWS CDK (the ApplicationLoadBalancedFargateService pattern creates the ALB, service, and log group in ~20 lines) or Terraform, so environments are reproducible and reviewable.

Cost & clean up

Always-on costs: ALB, 2 Fargate tasks, RDS. LLM tokens usually dominate once you have traffic.
Use Fargate Spot for non-production and Graviton (ARM64) for cheaper compute — build with --platform linux/arm64.

aws ecs update-service --cluster agents --service supportpilot --desired-count 0
aws ecs delete-service --cluster agents --service supportpilot --force
aws elbv2 delete-load-balancer --load-balancer-arn $ALB_ARN
aws elbv2 delete-target-group --target-group-arn $TG_ARN
aws rds delete-db-instance --db-instance-identifier supportpilot-db --skip-final-snapshot
aws ecr delete-repository --repository-name supportpilot --force
aws secretsmanager delete-secret --secret-id supportpilot/anthropic-api-key --force-delete-without-recovery
aws secretsmanager delete-secret --secret-id supportpilot/database-url --force-delete-without-recovery