Getting started on driftctl: configure your Terraform Provider version.

Your second step towards getting started on driftctl and support the variety of existing deployments.

This post is part of the walkthrough that will get you started on driftctl in a couple of steps. If you haven’t already done so, you should consider setting up your AWS authentication to use driftctl in a secure way.

TL;DR

To run driftctl with different Terraform providers, you can use drifctl scan --tf-provider-version or the DTCL_TF_PROVIDER_VERSION environment variable.

Here are all the details.

Why use different Terraform provider versions?

Terraform provider versions are really important for performance and features: bugs are fixed, and new resources or services are supported. On the other hand, if you’re happy with a stable, working Terraform configuration using a specific Terraform provider version, then there’s no real reason to rush or schedule an update. Also, you may be stuck with an infrastructure still using AWS Provider v2.x, just because it’s working well as is, and nobody would win anything by upgrading to v3.x for now! It can be different customers with different profiles, but also different parts of a single infrastructure. It’s common to have very stable IAM and Security Group isolated repositories and states, and very dynamic other types of resources like S3, lambdas, or the latest and greatest service just released by AWS. In this situation, you’ll likely end up with a very static and outdated AWS provider in one case, and a very recent one in the other. There are probably as many possible setups out there as there are companies or projects. The point here is to highlight the fact that it’s perfectly normal to live with different Terraform configurations using different provider versions. This is why it is important to be able to use driftctl with different terraform providers.

Another aspect of those Terraform providers is their very fast release cycle (that’s the case at least for the AWS provider). Since we froze the code base for the first public driftctl version less than 6 months ago, no less than 30 (thirty!) versions were released.

So, odds are you’ll end up with different automated infrastructures using different Terraform provider versions, minor and major. And while in many cases it’s good enough to scan your AWS account(s) for unmanaged resources or drifts using a different Terraform provider version, in many others it’s not. Examples include resource attributes that became required, changed types (from a string to an array), or defaults. Not to mention resources that were simply not supported but now are! Running driftctl without the specific Terraform provider version you use may lead to inconsistent results (and in some worse case scenarios: errors).

driftctl default behavior

When you run driftctl without specifying anything, it will download and use the version that driftctl arbitrarily decided to be the default at that time, and report it at the end of the run:

				
					$ driftctl scan 
[...]
Found 25 resource(s)
 - 68% coverage
 - 17 managed by terraform
 - 8 not managed by terraform
 - 0 found in terraform state but missing on the cloud provider
Provider version used to scan: 3.19.0. Use --tf-provider-version to use another version.

Note that the default provider version is set to change over time, so you really should use the right version for you. Here’s how.

Use a specific Terraform provider version

Let’s say you want to scan an AWS account for drifted or unmanaged resources, maybe to generate a report, and this Terraform repository is using the latest 2.x generation of the provider (2.70.0) and uses some specific configuration related only to v2x.

You can run driftctl with the --tf-provider-version option:

				
					$ driftctl --tf-provider-version=2.70.0 scan
[...]
Found 25 resource(s)
 - 68% coverage
 - 17 managed by terraform
 - 8 not managed by terraform
 - 0 found in terraform state but missing on the cloud provider
Provider version used to scan: 2.70.0. Use --tf-provider-version to use another version.

You can also verify in the local cache what versions were downloaded and made available:

				
					❯  tree ~/.driftctl/plugins/linux_amd64/
├── terraform-provider-aws_v2.70.0_x4
├── terraform-provider-aws_v3.19.0_x5
└── terraform-provider-aws_v3.44.0_x5

Use an environment variable to fix the provider version

Another option is to use the DTCL_TF_PROVIDER_VERSION environment variable. I find it more useful when driftctl is integrated into scripts or CI pipelines, as we often have solutions to set or pass values on the fly.

So if we want to scan an AWS account with the 3.44.0 provider version using an environment variable, I would simply do something like this:

				
					$ DTCL_TF_PROVIDER_VERSION=3.44.0 driftctl scan --from tfstate+s3://my-state-bucket/terraform.tfstate
[...]
Provider version used to scan: 3.44.0. Use --tf-provider-version to use another version.

Now, driftctl scans your AWS account using the right provider version that generated the Terraform state we’re comparing to. Much fewer false positives!

Dynamically set the right Terraform provider version

As you know, driftctl doesn’t read the Terraform code or configuration, only the state file, so it’s the most independent possible from any local requirements like git availability, AWS credentials for a GitOps pipeline, etc. While it’s great for ease of integration or simplicity, it prevents driftctl to automatically detect the required version, as this information is not available in the state.

So we’re left to find a method to set the version in an automated way!

There’s a wide range of solutions, but here’s a very simple one that I use to dynamically set the required provider version in a pipeline where Terraform is already setup:

				
					❯  terraform version | grep "registry.terraform.io/hashicorp/aws" | awk '{print $4}' | cut -c 2-
3.43.0

Integrate it how you feel! An example is a simple export of the environment variable:

				
					❯  export DCTL_TF_PROVIDER_VERSION=`terraform version | grep "registry.terraform.io/hashicorp/aws" | awk '{print $4}' | cut -c 2-`

❯  echo $DCTL_TF_PROVIDER_VERSION
3.43.0

Now you can add a step in your pipeline with a simple and generic driftctl scan with a dynamic configuration that you don’t need to change every time you bump the provider version:

				
					$ driftctl scan --from tfstate+s3://bucket-with-remote-states/eu-west-3/*.tfstate
Found 84 resource(s)
 - 96% coverage
 - 81 managed by terraform
 - 3 not managed by terraform
 - 0 found in terraform state but missing on the cloud provider
You have diffs on computed fields, check the documentation for potential false positive drifts
Provider version used to scan: 3.43.0. Use --tf-provider-version to use another version.

Recap and next step

In this article, we discovered how to tell driftctl to use a specific Terraform provider version to scan the AWS account, using a CLI option as well as an environment variable. We concluded by giving a simple integration example in pipelines or recurring checks to dynamically set the provider version environment variable, so no one has to do any maintenance.

Following this walkthrough, your next step will be to scan your account for resources not managed by Infrastructure as Code, and from there create a clean state that you will use for further monitoring.

Stay in touch

Get product updates and occasional news.