CosmoCode

State Management

Understand how Terraform tracks your infrastructure

7 min read

State Management

In the previous tutorial, we made our configs flexible with variables and outputs. Now let's talk about how Terraform remembers what it built.

Terraform state is the most important concept to understand. It's how Terraform knows what exists in the real world. Mess up state, mess up your infrastructure. It's like losing the only copy of a treasure map ā€” the treasure is still there, but good luck finding it. Respect state.

What is State?

"Where does Terraform remember what it created?"

When you run terraform apply, Terraform creates a terraform.tfstate file:

{
  "version": 4,
  "terraform_version": "1.7.0",
  "resources": [
    {
      "type": "aws_instance",
      "name": "web",
      "instances": [
        {
          "attributes": {
            "id": "i-0abc123def456789",
            "ami": "ami-0c55b159cbfafe1f0",
            "instance_type": "t2.micro",
            "public_ip": "54.123.45.67"
          }
        }
      ]
    }
  ]
}

State is a mapping: your config ā†” real infrastructure. It's Terraform's memory.

Why State Exists

1. Tracking What Exists

Without state, Terraform can't know:

  • What resources it created
  • What IDs those resources have
  • How to update or delete them

2. Performance

State caches resource attributes. Without it, Terraform would query every resource on every plan ā€” slow and API-intensive. Imagine asking your friend "where did you park?" every 5 seconds. That's Terraform without state.

3. Dependencies

State records resource relationships. Terraform knows the order to create, update, and destroy.

State File Location

By default: terraform.tfstate in your working directory.

project/
ā”œā”€ā”€ main.tf
ā”œā”€ā”€ terraform.tfstate      # Current state
ā””ā”€ā”€ terraform.tfstate.backup  # Previous state

Never commit state to Git. It often contains secrets (passwords, keys). Seriously. Never. Don't do it.

The Problem with Local State

"Can't I just keep the state file on my laptop?"

Sure, for learning. In teams, it's a disaster:

  • No sharing: Teammates can't see or apply changes
  • No locking: Two people apply at once = disaster
  • No backup: Delete the file, lose track of everything
  • Secrets exposed: State contains sensitive data in plain text

Remote Backends

"So where should I put it?"

Store state remotely. Multiple people can access it, with locking so two people can't apply at the same time.

S3 Backend (Recommended for AWS)

terraform {
  backend "s3" {
    bucket         = "my-terraform-state"
    key            = "prod/terraform.tfstate"
    region         = "us-east-1"
    encrypt        = true
    dynamodb_table = "terraform-locks"  # For locking
  }
}

First, create the bucket and DynamoDB table (yes, this is a chicken-and-egg problem ā€” you manage the state bucket with a separate Terraform project):

# state-setup/main.tf (separate project!)
resource "aws_s3_bucket" "terraform_state" {
  bucket = "my-terraform-state"
}

resource "aws_s3_bucket_versioning" "terraform_state" {
  bucket = aws_s3_bucket.terraform_state.id
  versioning_configuration {
    status = "Enabled"
  }
}

resource "aws_s3_bucket_server_side_encryption_configuration" "terraform_state" {
  bucket = aws_s3_bucket.terraform_state.id
  rule {
    apply_server_side_encryption_by_default {
      sse_algorithm = "aws:kms"
    }
  }
}

resource "aws_dynamodb_table" "terraform_locks" {
  name         = "terraform-locks"
  billing_mode = "PAY_PER_REQUEST"
  hash_key     = "LockID"

  attribute {
    name = "LockID"
    type = "S"
  }
}

Migrating to Remote Backend

  1. Add backend config to your project
  2. Run terraform init:
terraform init

Initializing the backend...
Do you want to copy existing state to the new backend?
  Enter a value: yes

Successfully configured the backend "s3"!

Terraform copies local state to S3.

State Locking

With DynamoDB configured, Terraform locks state during operations. It's like putting a "do not disturb" sign on a hotel room door:

Acquiring state lock. This may take a few moments...

If someone else is running, you'll wait:

Error: Error locking state: ConditionalCheckFailedException: The conditional request failed

Lock Info:
  ID:        abcd1234-5678-90ab-cdef
  Path:      my-terraform-state/prod/terraform.tfstate
  Operation: OperationTypeApply
  Who:       alice@laptop
  Created:   2024-01-15 10:30:00 UTC

Never force-unlock unless you're certain no one is running. You could corrupt state. I'm not kidding.

Other Backends

Terraform Cloud

terraform {
  cloud {
    organization = "my-org"
    workspaces {
      name = "my-workspace"
    }
  }
}

Best option for teams. Free tier available.

Azure Storage

terraform {
  backend "azurerm" {
    resource_group_name  = "terraform-state-rg"
    storage_account_name = "tfstate12345"
    container_name       = "tfstate"
    key                  = "prod.terraform.tfstate"
  }
}

GCS (Google Cloud)

terraform {
  backend "gcs" {
    bucket = "my-terraform-state"
    prefix = "prod"
  }
}

State Commands

These are your state debugging superpowers. Know them well.

terraform state list

List all resources Terraform knows about:

terraform state list

aws_instance.web
aws_security_group.web_sg
aws_eip.web_ip

terraform state show

Show details of a specific resource:

terraform state show aws_instance.web

# aws_instance.web:
resource "aws_instance" "web" {
    ami                          = "ami-0c55b159cbfafe1f0"
    id                           = "i-0abc123def456789"
    instance_type                = "t2.micro"
    public_ip                    = "54.123.45.67"
    ...
}

terraform state mv

Rename a resource without destroying and recreating it. Refactoring without the drama:

# Rename in config first, then:
terraform state mv aws_instance.web aws_instance.webserver

Useful when refactoring.

terraform state rm

Remove a resource from state (doesn't delete the actual resource in AWS ā€” just tells Terraform to forget about it):

terraform state rm aws_instance.web

Now Terraform doesn't manage it anymore. The EC2 instance still exists in AWS ā€” Terraform just pretends it doesn't know about it.

terraform state pull

Download current state (useful for debugging):

terraform state pull > state.json

terraform state push

Upload state. āš ļø Dangerous ā€” use this like you'd handle a loaded weapon:

terraform state push state.json

Refresh

Sync state with reality:

terraform refresh

This queries AWS and updates state with actual values. Deprecated in favor of:

terraform apply -refresh-only

Shows what would change, requires approval. Much safer.

State Inspection

Reading State Directly

State is just JSON. You can read it (but don't edit it manually unless you enjoy pain):

# Local state
cat terraform.tfstate | jq '.resources[0]'

# Remote state (S3)
aws s3 cp s3://my-terraform-state/prod/terraform.tfstate - | jq '.'

State in Multiple Environments

Separate state per environment:

# Production
terraform {
  backend "s3" {
    bucket = "my-terraform-state"
    key    = "prod/terraform.tfstate"  # Different key
    region = "us-east-1"
  }
}

# Staging
terraform {
  backend "s3" {
    bucket = "my-terraform-state"
    key    = "staging/terraform.tfstate"  # Different key
    region = "us-east-1"
  }
}

Or use workspaces (covered later).

Common State Problems

These will happen to you. Let's be ready.

Drift

"Someone logged into the AWS console and changed something manually!"

Yeah, that happens. A lot. Real infrastructure changed outside Terraform:

terraform plan

  # aws_instance.web has changed
  ~ resource "aws_instance" "web" {
      ~ instance_type = "t2.micro" -> "t2.large"  # Someone changed this!
    }

Plan: 0 to add, 1 to change, 0 to destroy.

Terraform wants to fix the drift. Options:

  1. Let Terraform revert the change
  2. Update your config to match reality
  3. Use terraform apply -refresh-only if you want to accept the change

Corrupted State

"My state file looks like garbled nonsense!"

State file got mangled? Try:

  1. Restore from backup: terraform.tfstate.backup
  2. Restore from S3 version history
  3. Import resources again (worst case)

Lost State

State completely gone? You need to import every resource. Painful but possible. It's the Terraform equivalent of rebuilding your contact list after losing your phone.

State Security

State contains secrets. Protect it:

1. Use Remote Backend

backend "s3" {
  encrypt = true  # Always
}

2. Restrict Access

# S3 bucket policy
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Deny",
      "Principal": "*",
      "Action": "s3:*",
      "Resource": [
        "arn:aws:s3:::my-terraform-state",
        "arn:aws:s3:::my-terraform-state/*"
      ],
      "Condition": {
        "Bool": {
          "aws:SecureTransport": "false"
        }
      }
    }
  ]
}

3. Enable Versioning

Restore previous versions if something goes wrong.

4. Don't Commit to Git

# .gitignore
*.tfstate
*.tfstate.*
.terraform/

What's Next?

State is no joke ā€” and now you understand why. You learned:

  • What state is and why it's Terraform's most critical file
  • Local vs remote state (spoiler: always go remote for teams)
  • S3 backend with DynamoDB locking
  • State commands for debugging and migration
  • How to handle drift, corruption, and accidental exposure

Sometimes you need to reference existing resources you didn't create. That's where data sources come in. Let's go!

Previousā† Variables & OutputsNextData Sources ā†’