TerraWeek#Day 4-Importance of State in Terraform

Task 1:

What is Terraform state?

How did Terraform know which resources it was supposed to manage? You could have all sorts of infrastructure in your AWS account, deployed through a variety of mechanisms (some manually, some via Terraform, some via the CLI), so how does Terraform know which infrastructure it’s responsible for?

Every time you run Terraform, it records information about what infrastructure it created in a Terraform state file. By default, when you run Terraform in the folder /foo/bar, Terraform creates the file /foo/bar/terraform.tfstate. This file contains a custom JSON format that records a mapping from the Terraform resources in your configuration files to the representation of those resources in the real world. For example, let’s say your Terraform configuration contained the following:

resource "aws_instance" "example" {
  ami           = "ami-0fb653ca2d3203ac1"
  instance_type = "t2.micro"
}

After running terraform apply, here is a small snippet of the contents of the terraform.tfstate file (truncated for readability):

{
  "version": 4,
  "terraform_version": "1.2.3",
  "serial": 1,
  "lineage": "86545604-7463-4aa5-e9e8-a2a221de98d2",
  "outputs": {},
  "resources": [
    {
      "mode": "managed",
      "type": "aws_instance",
      "name": "example",
      "provider": "provider[\"registry.terraform.io/...\"]",
      "instances": [
        {
          "schema_version": 1,
          "attributes": {
            "ami": "ami-0fb653ca2d3203ac1",
            "availability_zone": "us-east-2b",
            "id": "i-0bc4bbe5b84387543",
            "instance_state": "running",
            "instance_type": "t2.micro",
            "(...)": "(truncated)"
          }
        }
      ]
    }
  ]
}

Using this JSON format, Terraform knows that a resource with type aws_instance and name example corresponds to an EC2 Instance in your AWS account with ID i-0bc4bbe5b84387543. Every time you run Terraform, it can fetch the latest status of this EC2 Instance from AWS and compare that to what’s in your Terraform configurations to determine what changes need to be applied. In other words, the output of the plan command is a diff between the code on your computer and the infrastructure deployed in the real world, as discovered via IDs in the state file.

the tfstate file, terraform apply will not perform any changes after applying again immediately, so what happens if we delete the tfstate file and then execute apply again?

Terraform can not read the tfstate file and will think this is the first time we have created this set of resources, so it will create all the resources described in the code again!

What’s more troublesome is that since the state information corresponding to the resources we created last time was deleted by us, we can no longer run terraform destroy to destroy and recycle these resources through execution, which actually results in resource leakage. So it is very important to keep this state file properly.

Terraform State Lock

Terraform state locking is a feature that ensures only one user or process can modify the Terraform state file at any given time, preventing conflicts. The state file represents the current state of infrastructure resources that Terraform manages and used to plan and apply changes to them.

When multiple users or processes attempt to modify the same state file, it can cause conflicts, leading to unpredictable and potentially harmful changes to infrastructure resources. Terraform state locking solves this problem by enabling only one user or process to modify the state file at a time.

State locking is crucial because it maintains consistency and reliability in the infrastructure resources managed by Terraform. In the absence of state locking, concurrent modifications to the state file can cause incomplete or inconsistent updates, data corruption, or even the destruction of resources.

Using state locking, Terraform users can be sure that changes to infrastructure resources are applied accurately and without any unintended consequences. Additionally, state locking simplifies collaboration between team members, as it eliminates the possibility of conflicts that can occur when multiple individuals work on the same infrastructure resources at the same time.

All in all, Terraform state locking is an indispensable feature for effectively managing infrastructure resources at scale and ensuring their reliability and consistency over time.

If you would like to learn more about Terraform State Locking and learn how to set up your own Locked state read the following article:

Task 2:

Understand the different methods of storing the state file (local or remote). Create a simple Terraform configuration file and initialize it to generate a local state file and provide the Terraform state command and mention it's purpose. Check usage of terraform state command.

Command: state list

The terraform state list command is used to list resources within a Terraform state.

Usage

Usage: terraform state list [options] [address...]

The command will list all resources in the state file matching the given addresses (if any). If no addresses are given, all resources are listed.

The resources listed are sorted according to module depth order followed by alphabetical. This means that resources that are in your immediate configuration are listed first, and resources that are more deeply nested within modules are listed last.

For complex infrastructures, the state can contain thousands of resources. To filter these, provide one or more patterns to the command. Patterns are in resource addressing format.

The command-line flags are all optional. The following flags are available:

-state=path - Path to the state file. Defaults to "terraform.tfstate". Ignored when remote state is used.
-id=id - ID of resources to show. Ignored when unset.

Example: All Resources

This example will list all resources, including modules:

$ terraform state list
aws_instance.foo
aws_instance.bar[0]
aws_instance.bar[1]
module.elb.aws_elb.main

Example: Filtering by Resource

This example will only list resources for the given name:

$ terraform state list aws_instance.bar
aws_instance.bar[0]
aws_instance.bar[1]

There are several options for storing Terraform state, including local storage, remote storage, and Terraform Cloud.

Local Storage

By default, Terraform stores state locally in a file called terraform.tfstate. This file is created in the same directory as your Terraform configuration files. While local storage is the simplest option, it is not recommended for production use, as it can be lost or corrupted if your local machine fails.

Remote Storage

Remote storage is a more secure and scalable option for storing Terraform state. Remote storage options include Amazon S3, Google Cloud Storage, and Azure Blob Storage. These services provide secure and durable storage for your Terraform state and can be configured to automatically back up your state file.

Best Practices:

Store state remotely for accessibility, collaboration, and durability.
Enable state locking to prevent conflicts during concurrent modifications.
Regularly back up the state file to prevent data loss.
Version control Terraform configurations along with the state file.

Task 3:

Explore remote state management options such as Terraform Cloud, AWS S3, Azure Storage Account or HashiCorp Consul. Choose one remote state management option and research its setup and configuration process.

To use remote storage, you will need to configure Terraform to use a remote backend. This can be done by adding the following code to your Terraform configuration files:

terraform {
  backend "s3" {
    bucket = "my-terraform-state"
    key    = "terraform.tfstate"
    region = "us-east-1"
  }
}

resource "aws_s3_bucket" "example" {
  bucket = "my-example-bucket"
  acl    = "private"
}

This code configures Terraform to use Amazon S3 as the remote backend, with the my-terraform-state-bucket bucket as the storage location. You will need to replace my-terraform-state-bucket it with the name of your own S3 bucket, and us-east-1 with the region where your bucket is located.

To configure remote state storage, you'll need to add a backend block to your Terraform configuration.

HashiCorp Consul.

Consul is a tool for service discovery, configuration and orchestration. The Key/Value store it provides is often used to store application configuration and information about the infrastructure necessary to process requests.

Terraform provides a Consul provider which can be used to interface with Consul from inside a Terraform configuration.

Consul is a distributed key-value store and service discovery tool developed by HashiCorp. Consul can be used as a remote backend for Terraform state storage, providing a highly available and scalable solution for managing state files. Consul is well-suited for large-scale, multi-region infrastructure deployments and can be used in conjunction with other HashiCorp tools like Vault for enhanced security and access control

Set up a Consul cluster and add the following backend block to your Terraform configuration:

terraform {
  backend "consul" {
    address = "your-consul-server-address"
    path    = "your/consul/path"
  }
}

provider "aws" {
  region = "us-west-2"
}

resource "aws_s3_bucket" "example" {
  bucket = "my-example-bucket"
}

Task 4:

Modify your Terraform configuration file to store the state remotely using the chosen remote state management option. Include the necessary backend configuration block in your Terraform configuration file.

terraform { backend "<chosen_backend>" { # Add required configuration options for the chosen backend } }

statefile as below:

Code/please refer--github:https://github.com/gsbarure/TerraWeek.git

linkdin:https://www.linkedin.com/in/gajanan-barure-7351a4140

Happy Learning :)

Thank you for reading!! Hope you find this helpful.

Terraweekday04#challenge90daysofdevops

Shubham Londhe