GET STARTED ON DRIFTCTL
This post is part of the walkthrough that will get you started on driftctl. If you haven’t already done so, you should consider completing the previous steps on configuration and creating your first baseline to get the most out of the tool.
In this blog post, we’ll discover the HTML reporting feature for driftctl, how it helps to quickly see relevant information about resources not under Infrastructure-as-Code control, search features, some metrics, integration suggestions, and much more.
TL;DR: Generate an HTML report with: driftctl scan --output html://report.html
and open it with your favorite browser.
Using large shared AWS accounts, or accounts not fully automated, or when getting started with migrating to Infrastructure-as-Code, driftctl users often discover hundreds of resources to match or import. Here’s a driftctl CLI screenshot of a very poorly automated test AWS account, and it’s already a difficult mess to read, with hundreds of unmanaged resources (if not more):
In other cases, some users just don’t want to browse through cron logs or CI output archives. They just want something easily readable and easily shareable.
This is the reason why we created the driftctl HTML report!
Here’s a sample report:
Driftctl HTML reports include clearly and simply organized information.
Here’s the top of the report:
Displayed information is quick & simple to get at a glance:
The big list of unmanaged resources looks like this, with the resource ID (or name) on the left, and its type on the right:
If the driftctl output contained alerts, they will be displayed in the “Alerts” tab, too:
Now, let’s say that in the middle of hundreds of unmanaged resources, you want to quickly know if a certain string is present, like “Administrator”, to go straight to the point. Simply use the filter bar and the results will be displayed live!
Another useful option, if you’re interested in quickly discovering which EC2 key pairs are not under control and potentially a threat, just use the “Resource Type” box:
If for some reason, there’s a delta between your IaC and the reality, they will be displayed too (it happens for example when manual changes are not reverted by an automated terraform apply
, and many other situations). In this case, a tag “Environment: Production” was added by someone or something on this EC2 Instance. If I were seeing this, I would be very worried!
That’s the sort of usage we have for those reports! Simple and direct, useful in seconds.
We soon realized that it would be useful to quickly share & easily access reports across the team. So we decided to simply store on S3 an HTML report, daily, so that anyone from the team can easily refer to it to read clear information about what happened on a specific day. Historical drift and unmanaged data without too much work.
Here’s how we store daily HTML reports on an S3 bucket named “driftctl-reports”.
First, we need to generate the report, with a unique name. As we decided to generate one report a day, let’s use the timestamp as the unique string through the command date
:
$ driftctl scan --quiet --from tfstate://**/*.tfstate --output html://driftctl-report-`date '+%Y%m%d%H%M'`.html
This generates an HTML file, named “driftctl-report-202107061418.html” on July 6th, 2021 at 2.18 pm.
The next step is to easily upload it on S3, using the AWS CLI (adapt for your own bucket name):
$ aws s3 cp driftctl-report-*.html s3://cs-driftctl-reports/reports/2021/
upload: ./driftctl-report-202107061418.html to s3://cs-driftctl-reports/reports/2021/driftctl-report-202107061418.html
Here’s a screenshot of reports stored on S3 that way:
What’s left to you now is to tighten the security if you need to:
Here’s an example of how it can run every morning at 6 am on GitHub, with the correct Terraform provider version from the code.
For this example, I used a dedicated IAM user with full Read-Only access to resources for scanning, and write access to the bucket I’m using to store the reports. Those credentials are stored and accessed using GitHub Secrets, but there are many other different solutions probably well adapted for your specific use case.
Store this one under .github/workflows/driftctl-report.yml
along with your other GitHub Actions:
name: driftctl-report
# Store a report every day at 6am
on:
schedule:
- cron: "0 6 * * *"
jobs:
scan:
runs-on: ubuntu-latest
steps:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- uses: actions/checkout@v2
# Use a specific version of Terraform
- uses: hashicorp/setup-terraform@v1
with:
terraform_version: 1.0.0
# We don't need to initialize the backend for this
- name: initialize Terraform
run: terraform init -backend=false
# Let's collect the proper Terraform provider version from the actual code
- name: setup AWS Terraform Provider version
run: echo "DCTL_TF_PROVIDER_VERSION=`terraform version | grep "registry.terraform.io/hashicorp/aws" | awk '{print $4}' | cut -c 2- | head -1`" >> $GITHUB_ENV
# Actually generate the report
- name: Generate a report
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
DCTL_FROM: tfstate+s3:///**/*.tfstate
AWS_REGION: eu-west-3
DCTL_QUIET: true
run: docker run --rm -e DCTL_TF_PROVIDER_VERSION -e DCTL_QUIET -e AWS_ACCESS_KEY_ID -e AWS_SECRET_ACCESS_KEY -e DCTL_FROM -e AWS_REGION -v $(pwd)/.driftignore:/app/.driftignore:ro -v $(pwd)/reports:/reports cloudskiff/driftctl:latest scan --output html:///reports/driftctl-report-`date '+%Y%m%d%H%M'`.html
# Now store the report in a year-based directory on S3
- name: Store the report on S3
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
AWS_REGION: eu-west-3
# the reports need to be chown'd
run: |
sudo chown -R $USER:$USER ${{ github.workspace }}/reports
aws s3 cp reports/driftctl-report-*.html s3://mycompany-driftctl-reports/reports/`date '+%Y'`/
A successful run will look like this on your Actions tab:
This is only an example, your use-case might be much simpler or more complete as well; it will work very well on different CI/CD systems like Circle CI, GitLab, and others.
Bonus: use the S3 Bucket Browser or a similar solution to easily browse the reports graphically. Be careful though if you allow public access to those: the reports will give a lot of insider information about your infrastructure.
In this blog post, we discovered Driftctl HTML output, what those reports contain, and how to easily filter data directly in our web browser. We also went a bit further and suggested integration ideas for a GitHub Action with remote storage on S3.
We are almost through this getting started post series. Your next possible steps could be to:
.driftignore
file included (see this blog post)driftctl
scheduled run to be alerted when something is out of control (see driftctl CI integration documentation).Get product updates and occasional news.