On July 17, AWS Cost Explorer showed customers estimated bills of millions, billions, and in a few screenshots trillions of dollars. The numbers were wrong, nobody was charged, and AWS fixed the data within a day. The detail worth keeping: the people who noticed first were the ones with billing alerts configured. This post is the 101 on setting those up with Terraform's aws_budgets_budget, from a plain monthly cost alert to usage budgets and Savings Plan utilization tracking.

Anyone opening LinkedIn or X that morning saw the same feed: screenshots of AWS billing estimates with more digits than some national budgets. One user who paid $0.19 the previous month saw an estimate of nearly $2.5 billion. Others posted numbers up to $2.5 trillion.

What happened

At 1:33 AM Pacific, AWS posted to the Health Dashboard that Cost Explorer was "reflecting inaccurate estimated billing data." The root cause, in AWS's words, was "an issue with unit pricing within the estimated billing computation subsystem." In short:

  • The numbers were estimates, not actual usage or charges.
  • Nobody was charged, and no customer action was needed.
  • AWS backfilled the data, and the numbers were back to normal by July 18.

Here is the interesting part: many people learned about the bug from a billing threshold exceeded email. Accounts with alerts configured spotted the anomaly within minutes. Accounts without them found out from social media, or not at all.

This one was a false alarm. A forgotten NAT Gateway, a Lambda stuck in a retry loop, or a leaked access key produces a bill that is very much real. So this is a good week to set up billing alerts, and to do it with code instead of console clicks.

Why IaC and not the console

An alert created in the console lives in one account and in one person's memory. With Terraform:

  • It lives in version control. Who changed what, and when, is visible.
  • New account? Apply the same module and keep the standard.
  • A threshold change is a pull request that goes through review.

The resource that does the work is aws_budgets_budget, wrapping AWS Budgets.

1. A monthly cost budget with an email alert

A $100 monthly budget that emails you when actual spend crosses 80%:

resource "aws_budgets_budget" "monthly_cost" {
  name         = "monthly-cost-budget"
  budget_type  = "COST"
  limit_amount = "100"
  limit_unit   = "USD"
  time_unit    = "MONTHLY"

  notification {
    comparison_operator        = "GREATER_THAN"
    threshold                  = 80
    threshold_type             = "PERCENTAGE"
    notification_type          = "ACTUAL"
    subscriber_email_addresses = ["ops@example.com"]
  }
}

2. Forecast-based early warning

With notification_type = "FORECASTED", AWS warns you before the money is spent, when the end-of-month forecast is on track to blow the budget. Pairing it with an ACTUAL notification covers both directions:

resource "aws_budgets_budget" "with_forecast" {
  name         = "monthly-cost-budget"
  budget_type  = "COST"
  limit_amount = "100"
  limit_unit   = "USD"
  time_unit    = "MONTHLY"

  # Warn early if the forecast will exceed the budget
  notification {
    comparison_operator        = "GREATER_THAN"
    threshold                  = 100
    threshold_type             = "PERCENTAGE"
    notification_type          = "FORECASTED"
    subscriber_email_addresses = ["ops@example.com"]
  }

  # Warn again when actual spend crosses 90%
  notification {
    comparison_operator        = "GREATER_THAN"
    threshold                  = 90
    threshold_type             = "PERCENTAGE"
    notification_type          = "ACTUAL"
    subscriber_email_addresses = ["ops@example.com"]
  }
}

3. Slack or PagerDuty via SNS

Email gets lost. Send the alert to an SNS topic instead, then wire the topic to a Slack webhook or PagerDuty:

resource "aws_sns_topic" "billing_alerts" {
  name = "billing-alerts"
}

resource "aws_budgets_budget" "sns_alert" {
  name         = "monthly-cost-budget-sns"
  budget_type  = "COST"
  limit_amount = "100"
  limit_unit   = "USD"
  time_unit    = "MONTHLY"

  notification {
    comparison_operator       = "GREATER_THAN"
    threshold                 = 80
    threshold_type            = "PERCENTAGE"
    notification_type         = "ACTUAL"
    subscriber_sns_topic_arns = [aws_sns_topic.billing_alerts.arn]
  }
}

The failure mode here: the topic policy must grant SNS:Publish to the budgets.amazonaws.com service principal. Without it, Budgets cannot deliver and your alerts disappear silently. There is no error on the budget side.

4. A per-service budget

To watch a single service, say EC2, add a cost_filter:

resource "aws_budgets_budget" "ec2_only" {
  name         = "ec2-monthly-budget"
  budget_type  = "COST"
  limit_amount = "50"
  limit_unit   = "USD"
  time_unit    = "MONTHLY"

  cost_filter {
    name   = "Service"
    values = ["Amazon Elastic Compute Cloud - Compute"]
  }

  notification {
    comparison_operator        = "GREATER_THAN"
    threshold                  = 80
    threshold_type             = "PERCENTAGE"
    notification_type          = "ACTUAL"
    subscriber_email_addresses = ["ops@example.com"]
  }
}

5. A usage budget: 3 GB of S3

FinOps is not only about tracking dollars; tracking consumption is part of the job. With budget_type = "USAGE" the budget is defined in usage units instead of currency. A budget that caps S3 storage at 3 GB:

resource "aws_budgets_budget" "s3" {
  name         = "s3-3GB-limit"
  budget_type  = "USAGE"
  limit_amount = "3"
  limit_unit   = "GB"
  time_unit    = "MONTHLY"

  notification {
    comparison_operator        = "GREATER_THAN"
    threshold                  = 80
    threshold_type             = "PERCENTAGE"
    notification_type          = "ACTUAL"
    subscriber_email_addresses = ["ops@example.com"]
  }
}

Useful for side projects that want to stay inside the free tier, and for catching "why is this bucket growing" before it becomes a line item.

6. A Savings Plan utilization budget

The other half of FinOps: catching overspend matters, but so does not wasting the discount you prepaid for. If you bought a Savings Plan and your utilization is low, the commitment is burning money while your workloads run on-demand elsewhere. The SAVINGS_PLANS_UTILIZATION budget type tracks exactly that:

resource "aws_budgets_budget" "savings_plan_utilization" {
  name         = "savings-plan-utilization"
  budget_type  = "SAVINGS_PLANS_UTILIZATION"
  limit_amount = "100.0"
  limit_unit   = "PERCENTAGE"
  time_unit    = "MONTHLY"

  cost_types {
    include_credit             = false
    include_discount           = false
    include_other_subscription = false
    include_recurring          = false
    include_refund             = false
    include_subscription       = true
    include_support            = false
    include_tax                = false
    include_upfront            = false
    use_blended                = false
  }

  notification {
    comparison_operator        = "LESS_THAN"
    threshold                  = 90
    threshold_type             = "PERCENTAGE"
    notification_type          = "ACTUAL"
    subscriber_email_addresses = ["finops@example.com"]
  }
}

Note the inverted operator: comparison_operator = "LESS_THAN". The alert fires not when spend goes up but when utilization drops below 90%. The same pattern works for Reserved Instances with RI_UTILIZATION.

Wrapping up

July 17's trillion-dollar screenshots were a display bug. Nobody paid that money, and AWS corrected the data within a day. The lesson stands anyway: an anomaly in your bill should reach you through your own alerts, not through your timeline.

  • Define a monthly budget with aws_budgets_budget.
  • Pair FORECASTED and ACTUAL notifications for two-tier warning.
  • Route alerts through SNS to a channel your team actually watches.
  • Complete the picture with usage budgets (3 GB of S3) and Savings Plan utilization tracking.

Next time "my AWS bill exploded" trends, you can watch it with your coffee. Background reading on the incident: The Register and TechCrunch covered it as it unfolded.

Read this next

References