MervCodes

Tech Reviews From A Programmer

Blue-Green Deployment: Zero-Downtime Releases

1 min read

Shipping software used to mean a maintenance window: a banner warning users the site would be down at 2 a.m., a tense hour of manual steps, and a rollback plan nobody had rehearsed. Blue-green deployment replaces that ritual with something calmer. You run two production environments, release into the idle one, and flip traffic over in a single instant. Done well, users never notice a release happened at all.

This post walks through what blue-green deployment actually is, how to implement it, where it bites people, and how to decide whether it fits your system.

What Blue-Green Deployment Actually Means

The core idea is deceptively simple. You maintain two identical production environments, conventionally named blue and green. At any moment, one of them serves live traffic while the other sits idle. Say blue is live. When you want to release a new version, you deploy it to green — the idle environment. Because green takes no user traffic yet, you can deploy slowly, run smoke tests, warm caches, and check health endpoints without any pressure.

Once you're confident green is healthy, you switch the router so all incoming traffic now goes to green. The switch is atomic: one moment traffic hits blue, the next it hits green. Blue is now idle but still running the previous version. If green misbehaves, you flip the router back to blue and you're instantly on the old, known-good version.

That standby environment is the whole point. Rollback isn't a redeploy — it's a routing change that takes seconds. The old version is still warm and ready.

The Anatomy of a Switch

The "flip" happens at whatever layer routes traffic to your application. Common choices:

  • Load balancer target groups. On AWS, you point an Application Load Balancer listener at the blue target group, then swap it to the green target group. The change propagates near-instantly.
  • DNS. You update a DNS record to point at the new environment. Simple, but DNS TTLs and client-side caching mean the cutover is gradual and slow to reverse — usually the weakest option for true zero-downtime.
  • Service mesh or ingress. In Kubernetes, an Ingress or a mesh like Istio can shift traffic between two Services (or two Deployments behind one Service) by editing a selector or a routing rule.
  • Reverse proxy config. An nginx or HAProxy upstream block can be swapped and reloaded gracefully.

Prefer the load-balancer or ingress approach over DNS. DNS-based switches are hard to reverse quickly because you can't control how long clients cache the old record.

A Concrete Example

Here's a minimal blue-green flow using Kubernetes with two Deployments and one Service. The Service selector decides which pods receive traffic.

# service.yaml — the router
apiVersion: v1
kind: Service
metadata:
  name: web
spec:
  selector:
    app: web
    slot: blue   # currently pointing at blue
  ports:
    - port: 80
      targetPort: 8080

To release, you deploy the new version as the green Deployment, wait for its pods to pass readiness checks, then flip the selector:

# Deploy green, wait for rollout, then switch traffic
kubectl apply -f deploy-green.yaml
kubectl rollout status deployment/web-green

# Atomic cutover: repoint the Service at green
kubectl patch service web -p '{"spec":{"selector":{"slot":"green"}}}'

Rollback is the same command with blue — no rebuild, no image pull, no waiting.

Database Migrations: The Hard Part

Stateless app servers are easy to duplicate. Your database usually is not. Blue and green almost always share one database, and that shared state is where blue-green deployments go wrong.

The rule that saves you is backward-compatible, expand-then-contract migrations. Never ship a schema change that the currently-live version can't tolerate, because for a window of time both versions may be talking to the same database.

Break every breaking change into phases across multiple releases:

  1. Expand. Add the new column, table, or index. Make it nullable or give it a default. The old code ignores it; the new code can use it. Both versions work.
  2. Migrate and dual-write. Deploy code that writes to both old and new structures while reading from the old. Backfill existing rows.
  3. Switch reads. Deploy code that reads from the new structure.
  4. Contract. Once no running version references the old structure, drop it in a later release.

Renaming a column directly is the classic mistake: the instant you cut over, the old environment (your rollback target) starts throwing errors because the column it expects is gone. Expand-and-contract keeps rollback safe.

Handling In-Flight Requests and Sessions

At the moment of cutover, blue may still be processing requests. Don't hard-kill it. Drain it: stop sending new requests, let existing ones finish (a connection-draining timeout of 30–120 seconds is typical), then consider blue idle.

Sessions need thought too. If your app keeps session state in memory on the server, cutting over drops those sessions and logs users out. Externalize session state — Redis, a database, or signed cookies — so either environment can serve any user. This is just good twelve-factor practice, and blue-green forces the issue.

The same applies to background jobs, WebSocket connections, and long-running streams. Plan how each is drained or reconnected rather than assuming the switch only affects HTTP.

Blue-Green vs. Canary vs. Rolling

These strategies are often confused. They solve overlapping problems differently:

  • Rolling deployment replaces instances a few at a time. No second environment needed, but during the roll you're running mixed versions, and rollback means rolling back — slow.
  • Canary deployment sends a small percentage of traffic (say 5%) to the new version, watches metrics, then gradually increases. Excellent for catching problems with real traffic at limited blast radius, but more complex to orchestrate.
  • Blue-green switches 100% of traffic at once with instant rollback, at the cost of running two full environments.

They're not mutually exclusive. A common advanced pattern is blue-green with a canary step: shift a small slice of traffic to green first, watch, then complete the switch. You get instant rollback and limited initial blast radius.

Practical Advice From the Trenches

  • Automate the switch and the rollback. A cutover that requires a human running commands under pressure is a cutover that will eventually go wrong. Make "flip to green" and "flip back to blue" one button each.
  • Keep the environments genuinely identical. Configuration drift between blue and green is the silent killer. Use the same infrastructure-as-code to build both. If green has a different environment variable than blue had, your test on green proved nothing.
  • Run health checks before cutover, not after. Green should pass readiness and smoke tests while still idle. Only flip once it's proven healthy.
  • Watch metrics through the switch. Error rates, latency, and saturation should be on a dashboard you're actively watching for the first several minutes after cutover. Define in advance the threshold that triggers an automatic rollback.
  • Don't tear down the old environment immediately. Keep blue warm for at least a full release cycle — long enough to catch delayed failures like a memory leak or a slow query that only shows under sustained load.
  • Budget for the cost. Two production environments can mean roughly double the infrastructure. Some teams scale the idle environment down between releases and scale it back up just before deploying, trading a little switch speed for a lot of savings.

When Blue-Green Isn't the Right Fit

Blue-green shines for stateless web services with a clean routing layer. It's a poorer fit when:

  • Your database changes can't be made backward-compatible cheaply.
  • You have massive stateful services where duplicating the environment is prohibitively expensive.
  • You need gradual, metric-driven exposure — canary serves that better.

In those cases, reach for canary or rolling, or combine strategies. The goal is zero-downtime releases, not blue-green for its own sake.

FAQ

Does blue-green deployment double my infrastructure cost? During a release, yes — you're running two full environments. Many teams mitigate this by keeping the idle environment scaled down between deployments and scaling it up just before cutover, or by using autoscaling so the idle side costs little while dormant.

How is blue-green different from a canary release? Blue-green switches 100% of traffic at once and offers instant rollback by flipping back to the idle environment. Canary shifts traffic gradually (e.g., 5% → 25% → 100%) while watching metrics, limiting blast radius but making rollback a matter of dialing traffic back down. They can be combined.

What happens to database migrations during a blue-green deploy? Both environments typically share one database, so all schema changes must be backward-compatible. Use expand-and-contract migrations: add new structures first, migrate data, switch reads, and only remove old structures in a later release once no running version depends on them.

How fast is rollback? Seconds, if you switch at the load balancer or ingress layer, because the old version is already running and warm. Rollback is just a routing change — no rebuild or redeploy. DNS-based switches are much slower to reverse due to caching.

Do I lose in-flight requests when I switch? Not if you configure connection draining. The load balancer stops sending new requests to the old environment but lets existing ones finish within a timeout (commonly 30–120 seconds) before the environment is considered idle.

Can I use blue-green with a monolith? Yes. Blue-green cares about your routing and state layers, not your architecture style. A monolith with externalized sessions and backward-compatible migrations is a fine candidate — often easier than a sprawling set of microservices, each needing its own coordinated switch.

Blue-green deployment trades a bit of extra infrastructure for a lot of operational calm. Get the routing layer, the migration discipline, and the automation right, and releases stop being events you brace for — they become something you do on a Tuesday afternoon, with the confidence that undoing them is one button away.

Sources

Related Articles