GET STARTED ON DRIFTCTL

Configure

Advanced usage

Getting started on driftctl: how to use driftctl HTML reports

Now that you are up and running, here's how to spot quickly relevant information easy to read and to share.

This post is part of the walkthrough that will get you started on driftctl. If you haven’t already done so, you should consider completing the previous steps on configuration and creating your first baseline to get the most out of the tool.

In this blog post, we’ll discover the HTML reporting feature for driftctl, how it helps to quickly see relevant information about resources not under Infrastructure-as-Code control, search features, some metrics, integration suggestions, and much more.

TL;DR: Generate an HTML report with: driftctl scan --output html://report.html and open it with your favorite browser.

The ASCII report problem

Using large shared AWS accounts, or accounts not fully automated, or when getting started with migrating to Infrastructure-as-Code, driftctl users often discover hundreds of resources to match or import. Here’s a driftctl CLI screenshot of a very poorly automated test AWS account, and it’s already a difficult mess to read, with hundreds of unmanaged resources (if not more):

In other cases, some users just don’t want to browse through cron logs or CI output archives. They just want something easily readable and easily shareable.

This is the reason why we created the driftctl HTML report!

A Sample Report

Here’s a sample report:

Walkthrough

Driftctl HTML reports include clearly and simply organized information.

Here’s the top of the report:

Displayed information is quick & simple to get at a glance:

The total number of resources driftctl handled (here, 80 resources in the AWS account were handled)
How many resources are managed by Infrastructure-as-Code (here, only 6 resources were controlled by Terraform)
How many resources are not managed by Infrastructure-as-Code (74 wild resources for my test account!)
How many resources are found in the Terraform state but missing on AWS (thankfully, none in this case)
Coverage percentage: how much of your AWS resources are under control?

The big list of unmanaged resources looks like this, with the resource ID (or name) on the left, and its type on the right:

If the driftctl output contained alerts, they will be displayed in the “Alerts” tab, too:

Now, let’s say that in the middle of hundreds of unmanaged resources, you want to quickly know if a certain string is present, like “Administrator”, to go straight to the point. Simply use the filter bar and the results will be displayed live!

Another useful option, if you’re interested in quickly discovering which EC2 key pairs are not under control and potentially a threat, just use the “Resource Type” box:

If for some reason, there’s a delta between your IaC and the reality, they will be displayed too (it happens for example when manual changes are not reverted by an automated terraform apply, and many other situations). In this case, a tag “Environment: Production” was added by someone or something on this EC2 Instance. If I were seeing this, I would be very worried!

That’s the sort of usage we have for those reports! Simple and direct, useful in seconds.

Integration idea: store daily reports on S3

We soon realized that it would be useful to quickly share & easily access reports across the team. So we decided to simply store on S3 an HTML report, daily, so that anyone from the team can easily refer to it to read clear information about what happened on a specific day. Historical drift and unmanaged data without too much work.

Here’s how we store daily HTML reports on an S3 bucket named “driftctl-reports”.

First, we need to generate the report, with a unique name. As we decided to generate one report a day, let’s use the timestamp as the unique string through the command date:

				
					$ driftctl scan --quiet --from tfstate://**/*.tfstate --output html://driftctl-report-`date '+%Y%m%d%H%M'`.html

This generates an HTML file, named “driftctl-report-202107061418.html” on July 6th, 2021 at 2.18 pm.

The next step is to easily upload it on S3, using the AWS CLI (adapt for your own bucket name):

				
					$ aws s3 cp driftctl-report-*.html s3://cs-driftctl-reports/reports/2021/
upload: ./driftctl-report-202107061418.html to s3://cs-driftctl-reports/reports/2021/driftctl-report-202107061418.html

Here’s a screenshot of reports stored on S3 that way:

What’s left to you now is to tighten the security if you need to:

driftctl needs wide read-only access but nothing more (AWS managed ReadOnlyAccess IAM policy is an example)
the AWS CLI needs write access to the bucket and nothing else (for example, create a read/write access IAM policy just for your bucket)

Actual GitHub Action Example

Here’s an example of how it can run every morning at 6 am on GitHub, with the correct Terraform provider version from the code.

For this example, I used a dedicated IAM user with full Read-Only access to resources for scanning, and write access to the bucket I’m using to store the reports. Those credentials are stored and accessed using GitHub Secrets, but there are many other different solutions probably well adapted for your specific use case.

Store this one under .github/workflows/driftctl-report.yml along with your other GitHub Actions:

				
					name: driftctl-report

# Store a report every day at 6am
on:
  schedule:
    - cron: "0 6 * * *"

jobs:
  scan:
    runs-on: ubuntu-latest
    steps:
      # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
      - uses: actions/checkout@v2
      # Use a specific version of Terraform
      - uses: hashicorp/setup-terraform@v1
        with:
          terraform_version: 1.0.0
      # We don't need to initialize the backend for this
      - name: initialize Terraform
        run: terraform init -backend=false
      # Let's collect the proper Terraform provider version from the actual code
      - name: setup AWS Terraform Provider version
        run: echo "DCTL_TF_PROVIDER_VERSION=`terraform version | grep "registry.terraform.io/hashicorp/aws" | awk '{print $4}' | cut -c 2- | head -1`" >> $GITHUB_ENV
      # Actually generate the report
      - name: Generate a report 
        env:
          AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
          AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          DCTL_FROM: tfstate+s3://<bucket-full-of-states>/**/*.tfstate
          AWS_REGION: eu-west-3
          DCTL_QUIET: true
        run: docker run --rm -e DCTL_TF_PROVIDER_VERSION -e DCTL_QUIET -e AWS_ACCESS_KEY_ID -e AWS_SECRET_ACCESS_KEY -e DCTL_FROM -e AWS_REGION -v $(pwd)/.driftignore:/app/.driftignore:ro -v $(pwd)/reports:/reports cloudskiff/driftctl:latest scan --output html:///reports/driftctl-report-`date '+%Y%m%d%H%M'`.html
      # Now store the report in a year-based directory on S3
      - name: Store the report on S3
        env: 
          AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
          AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          AWS_REGION: eu-west-3
        # the reports need to be chown'd
        run: |
          sudo chown -R $USER:$USER ${{ github.workspace }}/reports
          aws s3 cp reports/driftctl-report-*.html s3://mycompany-driftctl-reports/reports/`date '+%Y'`/

A successful run will look like this on your Actions tab:

This is only an example, your use-case might be much simpler or more complete as well; it will work very well on different CI/CD systems like Circle CI, GitLab, and others.

Bonus: use the S3 Bucket Browser or a similar solution to easily browse the reports graphically. Be careful though if you allow public access to those: the reports will give a lot of insider information about your infrastructure.

Wrap-Up and next step

In this blog post, we discovered Driftctl HTML output, what those reports contain, and how to easily filter data directly in our web browser. We also went a bit further and suggested integration ideas for a GitHub Action with remote storage on S3.

We are almost through this getting started post series. Your next possible steps could be to:

create a custom driftctl Docker image with our .driftignore file included (see this blog post)
create a driftctl scheduled run to be alerted when something is out of control (see driftctl CI integration documentation).

Stay in touch

Get product updates and occasional news.