Getting started on driftctl: start tracking drifts from a clean state

How to start tracking drifts from a clean baseline even if you are moving to Terraform.

This post is part of the walkthrough that will get you started on driftctl in a couple of steps. If you haven’t already done so, you should consider setting up your AWS authentication to use driftctl in a secure way and configuring your Terraform provider version as well.

TL;DR

Many of us have reasons to not target 100% IaC coverage of our AWS accounts. Some of us are moving to Terraform. In this case, driftctl continuously delivers drift notifications that we don’t care about. Behold! There’s a solution.

In this post, we’ll show how to start using driftctl with a clean state, by ignoring all the current resources that are not yet under control. This will be easily done in a couple of seconds by generating a .driftignore file from a driftctl JSON output.

We’ll start with a heavily drifted AWS account, and end up with a clean state so you can actually start tracking resources. We’ll do all the steps with both the CLI and Docker container version.

What kind of messages can you expect to find in the CLI?

Found 11 resource(s) – 73% coverage
– 8 managed by terraform
– 2 not managed by terraform
– 1 found in terraform state but missing on the cloud provider

At this stage and before digging into the specifics of how to create a clean baseline, it is probably useful to list the type of messages you can expect to find in the CLI upon running a scan :

“x resource(s) managed by Terraform”: this one is pretty clear. It displays the number of resources in your infra that are both listed in your terraform state and on your cloud provider.
“x resource(s) not managed by Terraform”: represents resources that are actually present on your cloud provider but not on your terraform state. Those resources are not under the control of your Infrastructure as Code and you might want to consider moving them to Terraform.
“x resource(s) found in Terraform state but missing on the cloud provider”: this is a list of resources that are defined as code in your terraform codebase, but do not appear on your cloud provider. Those resources were probably deleted manually after your last terraform apply directly on your cloud console and will be provisioned again at your next apply unless you modify your terraform code.
“Congratulations, your infrastructure is fully in sync”: when you see this message, this means that you have a 100% match between what is listed in your Infrastructure as Code and what is running on your cloud provider. Congratulations! Make sure to run driftctl on a regular basis, to spot any new changes and bring your control back to its maximum in case of discrepancies.

Day 1, too many unmanaged resources

Back to our tutorial : what are unmanaged resources? Those are simply resources (EC2 instances, Security Groups, S3 buckets…) that are not managed by Terraform. When we run driftctl against a bunch of Terraform states, unmanaged resources will show up.

But in many cases, it’s perfectly understandable and acceptable for the time being to have unmanaged resources in the AWS account.

Many of us:

do not have the luxury to have AWS accounts fully managed by Terraform,
use a mix of CloudFormation (CFN) and Terraform, or some legacy scripts running here and there,
work with different teams on shared AWS accounts,
are currently moving existing projects to Terraform ,
create new projects in existing AWS accounts,
will fix those drifts soon, but not now (“it’s in the backlog!”)

So, the driftctl experience is quite frustrating for now in this mixed situation: while we have very good reasons to want to know what goes wrong from now on, we are still getting a ton of noise in the output that renders the tool almost useless.

Sure, there’s the .driftignore file, that works similarly to the .gitignore file, and we could still fill the file manually. But it works with a handful of resources, definitely not with a lot of resources.

The good news is that driftctl v0.9.0 ships with a generator of .driftignore, so we can get started in no time and ignore all the drifts!

How to automatically generate a .driftignore file

If you’re like me, always trying stuff here and there, or if you are moving to Terraform, you’ll end up with a messy driftctl output like that:

It’s cool the first time we launch the tool, so we have a clear overview of drifted and unmanaged resources, based on the cumulative layers of Terraform states – but it’s pretty useless afterwards, as we won’t fix all the drifts right away.

Step 1 - Generate a JSON output

So let’s start by creating a workable JSON output of this scan:

Take a look at this drifts.json file. It contains the exact same output, but as a structured JSON:

Step 2 - Generate a .driftignore from JSON

Now we will simply use the new driftctl gen-drifignore command to generate a .driftignore file:

Take a look at the newly generated file, all the resources are now included with the proper format!

Profit!

Finally, we can enjoy a driftctl scan with all the drifts ignored, while keeping a complete list of them in the .driftignore file, so we don’t forget about them.

Wait! I'm using Docker.

Here’s a quick primer on how to achieve all the above steps very quickly using the official Docker image.

The pre-requisites for this example are:

~/.aws contains the AWS profiles, so we can share one named AWS_PROFILE
current directory contains a few Terraform states (available under /app inside the container)
~/.driftctl to reuse the local provider cache

Confirm we can actually run a successful driftctl scan using Docker:

Now we can easily generate the JSON file:

Finally generate the .driftignore file from the JSON:

We now have our .driftignore file fully generated in seconds, using driftctl and Docker!

Next Steps

Now we’re starting from a clean state, the next step could be to generate your own HTML reports and quickly spot any relevant information easy to read and to share.

Stay in touch

Get product updates and occasional news.