Measuring and Cutting ECS Cost and Latency Across a Microservice Fleet

Shipping twelve Spring Boot APIs on AWS ECS Fargate is only half the job. The other half is proving they are fast enough and cheap enough under load — without guessing from a single curl.

After our Boot 4 migration, we ran a dedicated performance phase. The rule was simple: every change that claimed "faster" or "cheaper" needed a before/after benchmark with identical scripts.

Smoke is not a perf suite

Our deploy gate is HTTP smoke (Hurl): auth happy paths, health, critical reads/writes. That answers "is it broken?" It does not answer "is p95 better?"

A single smoke request is dominated by cold start noise, DynamoDB jitter, and ALB variance. You need concurrency and duration to see virtual threads, connection pools, and cache wins.

Before/after protocol

For every tuning wave:

1. Run before bench → export JSON summary
2. Apply change → smoke must stay green
3. Run after bench with the same VUs, duration, warm-up, and target URL
4. Fail the change if p95 regresses more than ~10%, or error rate rises (unless the change is pure memory rightsizing and latency stays within band)

We standardized on k6: free, scriptable, easy summary-export. A typical compare profile looked like 20 VUs for 2 minutes with a short warm-up, hitting the same read-heavy endpoints smoke already covers (no destructive writes in the default profile).

Where the time actually goes

Until you have Micrometer timers, be honest about wall-clock splits on I/O-heavy APIs:

That framing stops cargo-cult JVM flags. If DynamoDB dominates p95, rightsizing CPU from 512→1024 may do nothing useful while burning money.

The tuning waves that paid off

1. Instrument hot paths

Add Micrometer timers around DynamoDB and outbound HTTP on the busiest services (for us: payment, store, customer). Snapshot ECS CPU/memory utilization before you touch task definitions.

2. Rightsize Fargate from evidence

Only after baselines: reduce memory (or CPU) where utilization shows clear headroom. Re-bench immediately. Memory-only rightsizes often cut bill with little latency impact — but only if you measured first.

3. Extracted JAR + AOT cache

Layer Docker images so the application classes and Spring AOT cache warm faster. Across a fleet, startup and steady-state both improve when packaging is consistent.

4. Virtual threads on Tomcat

Enable virtual threads fleet-wide, then re-run the same k6 profile. Gains show up under concurrent I/O waits — exactly where food-delivery APIs live.

5. Selective hot-path cache

Cache only where the bench proves I/O-bound p95 and invalidation is clear. Caching "everything" is how you ship subtle correctness bugs.

6. GraalVM native later

Treat native images as a pilot on a small API after JVM wins soak — not the first lever.

Cost-aware defaults

For demos and early production, we still prefer:

• Smallest fit Fargate tasks until load proves otherwise
• DynamoDB on-demand
• No NAT Gateway for single public services
• Short CloudWatch log retention in non-prod

Performance work should reduce idle spend, not invent always-on capacity "for later."

Closing

Performance without a harness is storytelling. Pair smoke (correctness) with k6 (speed) and CloudWatch (cost), change one variable at a time, and keep the JSON reports. That is how a microservice fleet gets leaner on ECS without trading away reliability.