Terraform Remote Backend with State Locking in AWS and Azure

What is Terraform State File?

By default, when you run “terraform plan” or the “terrafom apply” commands, a record called terraform.tfstate file is created locally. This state file is used by Terraform to map resources to the configuration, keep track of metadata, and to improve performance for large infrastructures. State file can be used for scenarios like versioning, debugging, performance monitoring, rollbacks, rolling updates, immutable deployments, traceability, self-healing, etc.

What is Terraform Backend?

A “backend” in Terraform determines the handling of the state and the way certain operations are executed, enabling many essential features. Terraform uses the “local” backend as a normal behavior but state file can be stored remotely too.

There are multiple benefits to using a Remote backend:

1. Team Development — when working in a team, remote backends can keep the state of infrastructure at a centralized location
2. Sensitive Information — with remote backends your sensitive information would not be stored on local disk
3. Remote Operations — Infrastructure build could be a time-consuming task, some remote backends support remote execution of the tasks. You can then turn off your computer and your operation will still complete. Paired with remote state storage and locking above, this also helps in team environments.

What is Terraform State Locking?

Now your terraform state file is centrally managed and all the team members can access it and make changes to it. But if 2 changes are being made in parallel then that can corrupt the state file.

As a solution, terraform provides locking to prevent concurrent runs against the same state. Locking helps make sure that only one team member runs terraform configuration. Locking helps in preventing conflicts, data loss and state file corruption due to multiple runs on the same state file.

Terraform Remote Backend — AWS S3 and DynamoDB

Using the S3 backend resource in the configuration file, the state file can be saved in AWS S3. However, S3 doesn’t support the state locking functionality and this can be achieved by using DynamoDB.
DynamoDB supports state locking and consistency checking. A single DynamoDB table can be used to lock multiple remote state files. Terraform generates key names that include the values of the bucket and key variables.

Create S3 bucket permissions:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "s3:ListBucket",
      "Resource": "arn:aws:s3:::mybucket"
    },
    {
      "Effect": "Allow",
      "Action": ["s3:GetObject", "s3:PutObject"],
      "Resource": "arn:aws:s3:::mybucket/path/to/my/key"
    }
  ]
}

Create DynamoDB permissions:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "dynamodb:GetItem",
        "dynamodb:PutItem",
        "dynamodb:DeleteItem"
      ],
      "Resource": "arn:aws:dynamodb:*:*:table/mytable"
    }
  ]
}

Add S3 and DynamoDB details in backend S3 resource in Terraform configuration file:

terraform {
 backend “s3” {
    bucket = "mybucket"
    encrypt = true
    key    = "path/to/my/key"
    dynamodb_table = "Table_name"
    region = "us-east-1"
  }
}

Terraform Remote Backend — Azure Blob

Azure Blob Storage supports both state locking and consistency checking natively. When you store the Terraform state file in an Azure Storage Account, you get the benefits of RBAC (role-based access control) and data encryption.

Create a service principal for authentication:

az ad sp create-for-rbac — role=”Contributor” 
 — scopes=”/subscriptions/SUBSCRIPTION_ID” — name=”Any_name”

Configuring the Remote Backend to use Azure Storage:

terraform {
  backend "azurerm" {
    resource_group_name  = "StorageAccount-ResourceGroup"
    storage_account_name = "abcd1234"
    container_name       = "tfstate"
    key                  = "prod.terraform.tfstate"
  }
}

Terraform backend is a useful feature to solve pain points that afflict teams at a certain scale and makes it more friendly to use with multiple clouds. If your organization uses a hybrid setup the Terraform is one of the best choices for Infrastructure as a code.