ARC Zonal Shift: What Moves When You Evacuate an AZ

ARC zonal shift is the closest thing AWS gives you to a big red “stop sending traffic to that zone” button, and most teams have never held it down for longer than a coffee break. The AWS Architecture Blog just published a playbook for running multi-day AZ evacuation drills with ARC zonal shift, which is a great excuse to look at what actually happens when you shift away from an Availability Zone, and, more importantly, what does not.

What ARC zonal shift actually is

Zonal shift is a capability of Amazon Application Recovery Controller (ARC). You point it at a supported resource, name an AZ by its zone ID (think use1-az1, not us-east-1a), give it an expiry, and ARC steers that resource’s traffic away from the zone until the shift ends or you cancel it. The ARC developer guide covers the supported resource types; the ones that matter for most stacks are Application Load Balancers, Network Load Balancers and EKS clusters. There is no extra charge for the shift itself.

The design choice that makes it worth trusting: starting and ending a shift is a data-plane operation. It is built to work while the zone is impaired, which is exactly when control-plane APIs tend to get slow and grumpy. That distinction becomes the whole story once you look at what zonal shift leaves behind.

What moves on an ALB or NLB

For load balancers, two things happen. First, the load balancer’s IP address in the shifted AZ is pulled out of DNS, so new client connections resolve only to nodes in the healthy zones. Second, cross-zone routing to targets in the shifted AZ is blocked, so a load balancer node in AZ B will not quietly forward a request to a target in AZ A. Without that second half, cross-zone load balancing would happily keep feeding the broken zone through the back door.

Two caveats from the ALB zonal shift docs catch people out:

• Zonal shift integration is off by default on ALBs and NLBs. You have to enable the zonal_shift.config.enabled attribute before ARC can touch them.
• An ALB refuses the shift if a target group only has healthy targets in a single AZ. That is a guardrail, not a bug: shifting would leave zero capacity.

Also remember DNS caching. Clients that resolved the old IP set keep using it until their TTL expires, and badly behaved runtimes hold on far longer. Existing long-lived connections to the shifted node are not magically torn down either.

What moves on EKS

EKS is where zonal shift does the most work for you. Per the EKS zonal shift guide, when you shift a cluster away from an AZ, EKS:

• Cordons the nodes in that AZ, so the scheduler stops placing new pods there.
• Removes that zone’s pod endpoints from EndpointSlices, so in-cluster Service traffic stops landing on them.
• Suspends AZ rebalancing, so Auto Scaling does not “helpfully” relaunch capacity into the zone you are evacuating.
• Restricts managed node group Auto Scaling groups to the healthy AZs.

Note what it does not do: it does not drain or kill pods. They keep running in the shifted zone, just without traffic. That is deliberate. A fast restore means flipping endpoints back, not rescheduling half your cluster. It also means anything that does not go through a Service, such as a pod pulling from a queue, keeps working in that AZ. Plan for it.

What ARC zonal shift does not move: ECS, RDS and Aurora

Here is where people get burned. Zonal shift on the ALB stops traffic reaching ECS tasks in the evacuated AZ, but the ECS service still thinks those subnets are fair game. Tasks will be placed there, pass health checks and sit idle. Removing the AZ’s subnets from the service network configuration is a control-plane call you make yourself.

Databases are the same story. An RDS Multi-AZ primary or an Aurora writer living in the evacuated zone stays put. In the reference architecture from the AWS blog (ALB to ECS on Fargate, NLB to EKS, RDS Multi-AZ, Aurora PostgreSQL with the writer in AZ A and a reader in AZ B), you fail the database over explicitly. Aurora failover takes under 30 seconds when a reader is already provisioned in a healthy AZ, and roughly 10 minutes if a new instance has to be created. RDS then rebuilds the standby in the evacuated AZ, which is fine for a drill but worth knowing about.

The mental model: zonal shift handles the front door and the Kubernetes service mesh. Compute placement outside EKS and data-tier leadership are your job.

The CLI you will actually type

First, confirm what ARC can see and whether the integration is on:

aws arc-zonal-shift list-managed-resources

aws arc-zonal-shift get-managed-resource \
  --resource-identifier arn:aws:elasticloadbalancing:us-east-1:111122223333:loadbalancer/app/web/abc123

Start a 72-hour shift away from use1-az1. The comment shows up in the shift history, so make it useful:

aws arc-zonal-shift start-zonal-shift \
  --resource-identifier arn:aws:elasticloadbalancing:us-east-1:111122223333:loadbalancer/app/web/abc123 \
  --away-from use1-az1 \
  --expires-in 72h \
  --comment "drill: AZ evacuation, owner sre-oncall"

Extend, inspect and end it:

aws arc-zonal-shift update-zonal-shift \
  --zonal-shift-id <shift-id> --expires-in 24h

aws arc-zonal-shift list-zonal-shifts --status ACTIVE

aws arc-zonal-shift cancel-zonal-shift --zonal-shift-id <shift-id>

Each resource gets its own shift, so for an EKS cluster you run the same start-zonal-shift against the cluster ARN. Full parameter details live in the ARC zonal shift API reference.

Gotchas that show up on day two

A five-minute shift proves the plumbing. A multi-day one proves the architecture. The things that surface later include:

• Capacity math. Losing one of three AZs means the remaining two carry 50% more load each. If you did not pre-scale or test N-1, your drill becomes an incident.
• AZ ID versus name. Shifts use zone IDs, dashboards often use names, and the mapping differs per account. Map it before the drill, not during.
• Stale connections. Connection pools with no max lifetime keep talking to whatever they connected to on Monday.
• Restore order. Cancel the EKS shift before the load balancer shift so pods are reachable before external traffic arrives, then scale down gradually.

The zonal shift best practices page and the Advanced Multi-AZ Resilience Patterns whitepaper cover the static stability side in depth. Read them before you schedule the drill, not after the pager goes off.

At Enkompass (enkompass.net), this is our home turf. We have enabled zonal shift across ALB, NLB and EKS estates and walked teams through their first real AZ evacuations, database failovers included. If you’d rather have seasoned cloud people handle it instead of learning the hard way at 2 AM, let’s talk. Reach out through our contact page or call us directly at +1-828-436-5667 — we’ll help you turn this from a someday-project into a done-deal.