What is Terraform State File? Explained With Examples

In this blog post, we'll dive deep into the Terraform state File concepts.

By the end of the blog you will learn,

1. What is a state file.
2. What is remote state file and its benefits.
3. What is remote state lock mechanism.

Terraform State File

So, what exactly is Terraform state, and why is it so important?

Terraform state is essentially a snapshot of your infrastructure. It's a JSON file that keeps track of the resources Terraform manages and their current configuration.

Think of it as Terraform's memory - it remembers what it has created, modified, or deleted, allowing it to make informed decisions about future changes.

The state file tracks all the infrastructure resource details and their current state by mapping the real-world resources to your configuration. It also includes metadata about the resources, allowing it to understand the dependencies between them.

You can consider it the source of truth for your infrastructure managed by Terraform.

When you make changes to Terraform configurations, Terraform uses the state file to determine what changes need to be made to the infrastructure.

One important thing to note is that the state file may contain sensitive information about your infrastructure, so it is crucial to keep it secure.

The Terraform state file is generated automatically when you execute the terraform apply command.

By default, the state file is stored locally where your Terraform files are, and the default name of the state file is terraform.tfstate.

Hands on Terraform state File

Let’s do a hands-on exercise to gain a better understanding of the Terraform state.

I am going to deploy an EC2 instance using Terraform to understand the state file using an example.

Here is the main.tf file.

resource "aws_instance" "terraform-state-test" {
  ami           = "ami-0cf2b4e024cdb6960"
  instance_type = "t2.micro"
}

This will create a t2.micro ubuntu instance on the AWS in the us-west-2 region

Initialize the Terraform code.

terraform init

To look at the preview of the infrastructure

terraform plan

Lets apply the configuration and create the instance.

terraform apply

Once the infrastructure is deployed, you can see the generated state file in the current directory, as shown below.

Let’s view what is inside the state file.

This is just a small portion of the state file; the actual state file is quite long and contains sensitive information.

Storing state files on your local system is not a good practice because:

1. You can't collaborate with other developers on infrastructure management unless you have a centralized state file.
2. You might lose the state file due to accidental deletions, file system corruption, etc.
3. You can't implement CI/CD effectively using local state files.

Terraform Remote State

Terraform remote state refers to the practice of storing Terraform state files in a remote, shared location instead of on a local machine. This approach is important for team collaboration and maintaining consistency in infrastructure management.

The following are the benefits of remote state.

1. Remote state is typically stored in a shared backend such as Amazon S3, Azure Blob Storage, Google Cloud Storage, or HashiCorp's Terraform Cloud. This ensures that all team members are working with the most up-to-date state information.
2. By using remote state, multiple team members can safely work on the same infrastructure without the risk of conflicting local state files.
3. Most remote backends support state locking, which prevents multiple users from modifying the state simultaneously, avoiding potential conflicts and data corruption.
4. Remote state can be encrypted at rest and in transit, providing better security for sensitive information compared to local state files.
5. Remote state facilitates easier integration with CI/CD pipelines, as the state is accessible from any machine running the pipeline.

To use remote state, you need to configure a backend in your Terraform configuration. Here's a simple example using an S3 backend

terraform {
  backend "s3" {
    bucket = "my-terraform-state"
    key    = "path/to/my/key"
    region = "us-east-1"
  }
}

This configuration tells Terraform to store the state file in the specified S3 bucket.

Remote State locking

Remote State locking is a mechanism that prevents multiple users or processes from modifying the same state simultaneously.

When one user or process acquires a lock on the state, others are prevented from acquiring the lock until it's released.

Terraform State File Best Practices

Here are some best practices for managing Terraform state files in a production environment:

1. Store Terraform state files in a remote backend (like AWS S3, Azure Blob Storage, or Google Cloud Storage) rather than locally. This ensures that the state file is shared among team members and is protected from local machine failures.
2. Use a backend that supports state locking (like AWS S3 with DynamoDB, Azure Blob Storage with Azure Cosmos DB, or Google Cloud Storage with Google Cloud Firestore) to prevent simultaneous modifications that could corrupt the state.
3. Ensure the remote state file is encrypted both at rest and in transit.
4. Restrict access to the Terraform state files using IAM policies, access control lists (ACLs), or similar mechanisms.
5. Use versioning for your state files to keep track of changes and to be able to roll back if needed. This is often supported by remote backends like S3 and Google Cloud Storage.
6. Regularly back up your state files to safeguard against data loss or corruption.
7. Maintain separate state files for different environments (e.g., development, staging, production) to avoid accidental changes to the wrong environment and to keep environments isolated.
8. Ensure sensitive information (like passwords and API keys) is not stored in the state file. Use Terraform variables or secrets management solutions for handling sensitive data.
9. Integrate state management into your CI/CD pipelines to automate tasks like state file backups, versioning, and validation. This helps in maintaining consistency and reliability in your infrastructure changes.