How to use driftctl with Terragrunt

Let's see how easy it is for Terragrunt users to use driftctl, so they can aggregate all the generated states and track their drifted or unmanaged resources!

TL;DR

You can use driftctl with Terragrunt by using the glob pattern in the --from option: driftctl scan --from tfstate://path/to/states/**/*.tfstate!

Here are all the details.

A Terragrunt (Sample) Project

If you already have a Terragrunt project available: good, use it, and move to the next section!

If you don’t, you can find a simple one in this git repo: cloudskiff/driftctl-howto-terragrunt.

The sample project uses Terragrunt to store in a single directory (states/) all the Terraform states with predictable subdirectories. The directory base/ simply generates a random string, to be used to build a dynamic name for the S3 bucket in the s3/ directory.

It looks like this:

				
					❯  tree
.
├── base
│   ├── providers.tf
│   ├── random.tf
│   └── terragrunt.hcl
├── s3
│   ├── data.tf
│   ├── main.tf
│   ├── providers.tf
│   └── terragrunt.hcl
├── states
└── terragrunt.hcl

Our goal after this quick Terragrunt introduction is to observe a structure like this one (see the TF states and directories under ./states/:

				
					❯  tree
.
├── backend.tf
├── base
│   ├── backend.tf
│   ├── providers.tf
│   ├── random.tf
│   └── terragrunt.hcl
├── s3
│   ├── backend.tf
│   ├── data.tf
│   ├── main.tf
│   ├── providers.tf
│   └── terragrunt.hcl
├── states
│   ├── base
│   │   └── terraform.tfstate
│   └── s3
│       └── terraform.tfstate
└── terragrunt.hcl

Simply run terragrunt init in each folder:

				
					$ cd base
$ terragrunt init 
[...]
Terraform has been successfully initialized!
[...]

Confirm that a backend.tf file was correctly generated by Terragrunt:

				
					$ cat backend.tf 
# Generated by Terragrunt. Sig: nIlQXj57tbuaRZEa
terraform {
  backend "local" {
    path = "../states/base/terraform.tfstate"
  }
}

This ensures that a Terraform state will be stored under ../states/<service>/, as expected.

Now simply apply using Terragrunt as you would with Terraform:

				
					$ terragrunt apply
[...]
random_string.prefix: Creating...
random_string.prefix: Creation complete after 0s [id=j3dmhg]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

Outputs:

random_string = "j3dmhg"
$ cd ..

Confirm that a Terraform state now exists under states/base/:

				
					$ tree states/
states/
├── base
│   └── terraform.tfstate
└── s3

Now let’s do the same steps for the s3/ directory (remember to setup an authentication mechanism for AWS if you didn’t, like AWS_PROFILE):

				
					$ pwd
./howto-terragrunt/
$ cd s3
$ terragrunt init 
[...]
Terraform has been successfully initialized!
[...]
$ terragrunt apply
[...]
aws_s3_bucket.demo: Creating...
aws_s3_bucket.demo: Creation complete after 6s [id=j3dmhg-demo]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

As an alternative, you could have used terragrunt run-all apply from the root directory, which would have applied it all at once.

Now, we have all our Terraform states stored under states/ within named subfolders:

				
					$ tree states/
states/
├── base
│   └── terraform.tfstate
└── s3
    └── terraform.tfstate

Driftctl With a Terragrunt State Folder Structure

How can we use driftctl with such a Terraform state folder structure? States aren’t located at the same place, and no one wants to append dozens or hundreds of --from arguments to a CLI.

It’s very simple: by using a glob pattern in the --from option of the tool! Here’s a simple example if you want to load all the files with extension “*.tfstate” in any subdirectory found under ./states/:

				
					$ export AWS_PROFILE="terragrunt-demo" 
$ export AWS_REGION="us-east-1" 
$ pwd
./howto-terragrunt
$ driftctl scan --from tfstate://states/**/*.tfstate
Scanned states (2) 
Scanned resources (62)      
Found 62 resource(s)
 - 100% coverage
Congrats! Your infrastructure is fully in sync.

If you’re using S3 as a Terraform state storage backend, it works the same way:

				
					$ driftctl scan --from tfstate+s3://my-terragrunt-bucket/some-folder/**/*.tfstate
Scanned states (87) 
Scanned resources (7980)
Found 7980 resource(s)
 - 100% coverage
Congrats! Your infrastructure is fully in sync.

If you use Terragrunt to split your environments or any other kind of file/dir structure, it will work the same. Take a look at globster.xyz to get examples and ideas of patterns!

Key Takeaways & Next Steps

In this article, we quickly demonstrated how driftctl can use the many Terraform states Terragrunt can generate in slightly different locations than the original Terraform, using patterns to aggregate the right Terraform state files!

Now you know the right command to get the correct results for your use case, you might want to explore the next steps:

Generate a .driftignore file so you start tracking drifts from scratch (if you have too many unmanaged resources, for example)
Build your custom driftctl Docker image, so you can easily deploy the full thing as a recurring job, to get alerts when things go wrong!

Stay in touch

Get product updates and occasional news.