Governing EC2 T8i Instances: Alarms and Autoscaling

Migrating to EC2 T8i instances is the easy half. The hard half is keeping a burstable fleet burstable six months later, after three teams have added workloads to it and nobody has looked at a credit balance since the migration ticket closed. Burstable instances fail quietly: they do not page, they do not error, they just bill a little more every hour in a line item indistinguishable from normal EC2 usage. This is how you build the machinery that catches that.

Everything below assumes you have already moved a fleet to T8i. If you have not, start with the sizing and credit mechanics first — this article is about what happens after.

The governance model has four layers, and they are worth naming up front because teams routinely build one and assume they are covered. Detection tells you a fleet has started borrowing. Remediation stops the bleeding without a human in the loop. Prevention keeps workloads that were never burstable off the burstable family in the first place. Accountability makes sure the cost lands on a team that can do something about it. Skip any one of them and the other three decay within a quarter.

Detection: one alarm per fleet, not per instance

The signal is CPUSurplusCreditsCharged in the AWS/EC2 namespace. Nonzero means an instance in Unlimited mode borrowed credits, failed to repay them inside the repayment window, and you were billed. Per the Unlimited mode documentation, that is working as designed — which is exactly why it needs an alarm rather than a bug report.

Alarming on every instance ID is unmanageable in an autoscaled fleet where instance IDs churn. Alarm on the Auto Scaling group dimension instead, so the alarm survives instance replacement:

aws cloudwatch put-metric-alarm \
  --alarm-name t8i-fleet-surplus-charged \
  --namespace AWS/EC2 \
  --metric-name CPUSurplusCreditsCharged \
  --dimensions Name=AutoScalingGroupName,Value=api-asg \
  --statistic Sum \
  --period 3600 \
  --evaluation-periods 6 \
  --datapoints-to-alarm 4 \
  --threshold 0 \
  --comparison-operator GreaterThanThreshold \
  --treat-missing-data notBreaching \
  --alarm-actions arn:aws:sns:us-east-1:111122223333:finops-alerts

Six evaluation periods with four breaching datapoints is deliberate. One surplus hour after a deploy is noise. Four surplus hours in six is a pattern, and a pattern is what you want a human to look at. Alarm configuration options are covered in the CloudWatch alarms guide.

Remediation: demote chronic bursters automatically

An alert nobody acts on is a slightly more expensive alert. The safe automated response to sustained surplus charging is to flip the instance from unlimited to standard: the workload gets clamped to its baseline, the bleeding stops, and the resulting slowness forces the owning team to size the thing properly. No restart is required, so this is a reversible, non-destructive action.

Here is a Lambda that sweeps the fleet daily. Honor an opt-out tag so teams with a legitimate reason to burst can exempt themselves — on the record, in Git, rather than by arguing with your automation:

import datetime
import boto3

ec2 = boto3.client("ec2")
cloudwatch = boto3.client("cloudwatch")

EXEMPT_TAG = "BurstExempt"
LOOKBACK_HOURS = 24


def surplus_charged(instance_id):
    end = datetime.datetime.utcnow()
    start = end - datetime.timedelta(hours=LOOKBACK_HOURS)
    result = cloudwatch.get_metric_statistics(
        Namespace="AWS/EC2",
        MetricName="CPUSurplusCreditsCharged",
        Dimensions=[{"Name": "InstanceId", "Value": instance_id}],
        StartTime=start,
        EndTime=end,
        Period=3600,
        Statistics=["Sum"],
    )
    return sum(point["Sum"] for point in result["Datapoints"])


def lambda_handler(event, context):
    demoted = []
    pages = ec2.get_paginator("describe_instances").paginate(
        Filters=[
            {"Name": "instance-type", "Values": ["t8i.*"]},
            {"Name": "instance-state-name", "Values": ["running"]},
        ]
    )
    for page in pages:
        for reservation in page["Reservations"]:
            for instance in reservation["Instances"]:
                tags = {t["Key"]: t["Value"] for t in instance.get("Tags", [])}
                if tags.get(EXEMPT_TAG) == "true":
                    continue
                if surplus_charged(instance["InstanceId"]) > 0:
                    ec2.modify_instance_credit_specification(
                        InstanceCreditSpecifications=[{
                            "InstanceId": instance["InstanceId"],
                            "CpuCredits": "standard",
                        }]
                    )
                    demoted.append(instance["InstanceId"])
    return {"demoted": demoted}

Give the function ec2:DescribeInstances, ec2:ModifyInstanceCreditSpecification and cloudwatch:GetMetricStatistics, nothing more. Schedule it with an EventBridge rule — the EventBridge user guide covers scheduled rules — and have it publish the demotion list to the same SNS topic as the alarm. Run it in dry-run mode for two weeks first, logging what it would have demoted. You will learn more from that log than from the alarm.

Two design choices in that function are worth defending. First, it only ever demotes — it never promotes an instance back to Unlimited. Automation that can silently re-enable surplus billing is automation that will eventually re-enable surplus billing on the fleet you were trying to protect. Promotion stays a human decision made in the launch template, where it is reviewable. Second, it acts on the live instance rather than the launch template, which means the next instance replacement resets it. That is on purpose: the demotion is a circuit breaker, not a fix. If the same instance IDs keep showing up in the demotion list week after week, the launch template is wrong and somebody needs to change it.

Prevention: stop pretending every workload is burstable

Automation that demotes bursters treats the symptom. The cause is usually a workload on the wrong family. A mixed-instances policy lets an Auto Scaling group run T8i for the cheap baseline and fall back to a general-purpose family when capacity or demand calls for it:

aws autoscaling update-auto-scaling-group \
  --auto-scaling-group-name api-asg \
  --mixed-instances-policy '{
    "LaunchTemplate": {
      "LaunchTemplateSpecification": {
        "LaunchTemplateId": "lt-0abc123def4567890",
        "Version": "$Latest"
      },
      "Overrides": [
        { "InstanceType": "t8i.medium" },
        { "InstanceType": "m7i.large" },
        { "InstanceType": "m7a.large" }
      ]
    },
    "InstancesDistribution": {
      "OnDemandBaseCapacity": 1,
      "OnDemandPercentageAboveBaseCapacity": 100,
      "SpotAllocationStrategy": "price-capacity-optimized"
    }
  }'

Order the overrides deliberately — Auto Scaling prioritizes them in the order listed when the allocation strategy is prioritized. Details and the full option set live in the Amazon EC2 Auto Scaling user guide. The point of the fallback is not availability, it is economics: when the fleet needs sustained CPU, an m7i.large is cheaper than a t8i.medium paying surplus around the clock.

For fleets where you keep guessing, let Compute Optimizer settle it. Its findings are derived from actual utilization history, and “this was never a burstable workload” is a much easier conversation when the recommendation came from AWS rather than from you.

Accountability: tags, budgets, and anomaly detection

Surplus credits show up on the bill as EC2 usage, which means they land in whatever cost center owns the instance. If your burstable fleet is not tagged for chargeback, nobody feels the cost, and nothing changes. Activate cost allocation tags in Billing (see the cost allocation tags documentation), enforce them at launch through the launch template, and make Owner and Environment non-optional.

Then put two financial tripwires behind the technical ones. AWS Budgets gives you an absolute ceiling per tag or account — useful for environments that should never exceed a known number. Cost Anomaly Detection gives you the relative signal: a burstable fleet that starts charging surplus produces a slow upward drift that a fixed budget will not catch until month end but an anomaly monitor flags in days.

Close the loop with a weekly query so someone is actually reading the numbers:

aws ce get-cost-and-usage \
  --time-period Start=2026-09-15,End=2026-09-22 \
  --granularity DAILY \
  --metrics UnblendedCost \
  --group-by Type=DIMENSION,Key=INSTANCE_TYPE \
  --group-by Type=TAG,Key=Owner \
  --output json

The full Cost Explorer API reference covers the filter grammar if you want to scope this to a single account or linked payer.

What good governance looks like for EC2 T8i instances

A healthy burstable fleet has four properties: every instance has a deliberately chosen credit mode set in its launch template, a fleet-level alarm fires on sustained surplus charging, an automated sweep demotes repeat offenders to Standard mode with a documented exemption path, and every instance carries an owner tag that routes the cost to someone who can act on it.

None of that is exotic. It is four CloudWatch and Lambda pieces that take an afternoon to build and then run forever. The alternative is discovering in Q1 that a staging fleet has been paying surplus credits since September, which is a conversation nobody enjoys having twice.

At Enkompass (enkompass.net), this is our home turf. We’ve clawed back six figures in annual spend for clients whose bills were supposedly already optimized — then built the guardrails that keep the savings from quietly leaking back out. If you’d rather have seasoned cloud people handle it instead of learning the hard way at 2 AM, let’s talk. Reach out through our contact page or call us directly at +1-828-436-5667 — we’ll help you turn this from a someday-project into a done-deal.