How many Infrastructure as Code tools are you using on a single infrastructure?

Building driftctl, we often came across users with multiple Infrastructure as Code tools on the same infra. Here is what we learned while drilling into this.

Share on twitter
Share on reddit
Share on linkedin

A pretty wide variety of Infrastructure as Code tools at hand

DevOps teams have a pretty wide variety of Infrastructure as Code tools at hand. The range goes from so-called “configuration management tools”, such as Ansible, Chef, Puppet… mainly used to install and manage software on existing servers, to “provisioning tools” designed to spin up resources themselves like Terraform or Pulumi, not to mention cloud providers proprietary tools like AWS CloudFormation, ARM for Azure, or Google Cloud Deployment Manager.

If you add to that dev tools like AWS CDK, Terraform CDK, the serverless framework or even tools like Kubernetes that will spin up bits of infrastructure themselves, the possibilities and combinations are almost endless.

Building driftctl, a fully open-source tool that catches drift outside of infrastructure code, we have often come across use cases where users would have multiple IaC tools on the same infrastructure (mainly CloudFormation + Terraform as we only support AWS at this stage). This community feedback led us to a question: is this a common setup that has to be supported in our open source solution? So we decided to create a poll.

How many IaC tools are you using on the same infrastructure?

The question we asked was a simple one:

Looking at your IaC setup (or your customer’s), would you say that you use:

  • A single IaC tool (ex: CloudFormation only, Terraform only, Pulumi only…)
  • Multiple tools on purpose for different resources/use cases (ex: CloudFormation + Terraform or ARM+Terraform and/or the serverless framework)
  • Several tools today but plan to migrate to a single tool (ex migrating from a mix of CloudFormation + Terraform to Terraform only)

After posting this poll, we immediately got reactions on the question by itself : people would come up with comments and reactions on the first results because we did not mention all solutions at hand. Typically, the combination of configuration management tools and provisioning tools is still a classic:

The % split seemed a bit strange, at least to me. Perhaps it was prefacing the poll by naming CF & Terraform, as these have similar use cases. I also think of Chef and Helm as IaC, and for sure, multiple tools on purpose, depending on the context. Yes, I know Helm can be a provider in Terraform, and then we lose the flexibility that charts offer in the first place.

LinkedIn

What about Terraform + Puppet + Ansible .. all for different use cases

Twitter

We asked this question on several channels, like various Slacks dedicated to DevOps topics, LinkedIn, our Twitter account, and our community Discord server where our users discuss everything related to infrastructure drift. The very first thing we noticed was the fact that we gathered close to 350 responses within a couple of days, with results quite consistent across all channels. The topic is an interesting one and practitioners are eager to discuss it.

How many IaC tools are you using on the same infrastructure?​

A single IaC tool

62% of respondents declare using a single IaC tool: a common setup that we expected to emerge as the main one.

As we wanted people to answer the poll in under a minute, we kept it short and simple, and have no indication as to what tool emerges the most. Some people did give us a hint on their preferences, though…

While the number of answers collected comforts us thinking the poll was representative of reality, we are still wondering if there was a slight bias in this answer being more wishful thinking than a reality.

Multiple IaC tools on purpose for different resources/use cases

32% declare using multiple IaC tools on purpose for different resources/use cases. We’ll come back in detail on the reasons why teams embrace a variety of IaC tools, but it is already interesting to note here that for a significant number of organizations the hegemony of one single tool is not a goal to reach for.

 

Several tools today but planning to migrate to a single tool

6% of respondents declare using multiple tools today but planning to migrate to a single tool at some point. The reasons for that are various and can be directly related to the performances of the tools themselves and/or the skills of the team using them.

Why use multiple Infrastructure as Code tools on a single infrastructure?

The question is legit. There are several reasons why you would think that the usage of a single IaC tool is more convenient: one single standard for the whole team makes it easier to manage and track. Consequently, it makes your infrastructure simpler to check for compliance and security. There are also probably gains to be had on the training of the team, etc…

The funny thing about short polls is that they frustrate passionate people. They’d like to say more but the closed questions of the poll won’t let them. Some comments gave us very useful insights on why they use multiple Infrastructure as Code tools. Here are some quotes that we can probably categorize as patterns:

Many teams have a preferred tool for their core infrastructure, and are open to other tools for applications and non-core items:

We use terraform for the foundational infrastructure capable of supporting most tools and whatever other tool developers want for their applications: serverless, CDK, CFT, etc...

Slack SweetOps

This pattern also exists when 3rd party tools rely on a specific IaC language.

We use terraform for our 1st party IaC, but generally 3rd party tools that we use (convox for container platform, serverless framework for serverless platform, and various AWS tools/integrations) use Cloudformation as their deployment mechanism, so doubtful that we’d ever end up with a single IaC tool.

Driftctl community discord

Organizations also have to take into account the skill level of their teams and their ability to use one tool or another:

There are cases when both tools are needed. One tool like Terraform covers more than enough. Sometimes orgs use CloudFormation, but for AWS quickstarts or reference architecture (click through via GUI to create stacks) But that is because of lack of expertise and confidence, and results in half manual environments.

LinkedIn

And obviously, there is the special case of Kubernetes and the several ways it interacts with our infrastructures :

  • Resources created by Kubernetes for its own purpose (essentially storage, networking, etc.)
  • Resources created by Terraform but modified by Kubernetes (like a load balancer created by TF but its configuration changed on the fly by k8s for autoscaling or auto-configuration after a deployment)
  • Provisioning of cloud resources directly through Kubernetes

What about when it's Terraform plus bits of infra spun up by Kubernetes itself? I.e. load balancers and stuff.

Slack Faun

How this poll will impact drifctl roadmap in the future

Reflecting on those elements, we see that teams often use different Infrastructure as Code tools for different usage, and there is no reason for this to change. In this context, CloudFormation support is an obvious priority to bring our first users the best possible experience on their AWS infrastructures. 

A natural next step after this one would probably be to focus on a prototype of driftctl supporting multiple layers of Terraform states, CloudFormation and shared, dynamic and static resources from Kubernetes.

References

Our sincere thanks to the following communities for hosting this poll and to their members for providing us with valuable feedback:

France DevOps
Faun
CNCF
AWS users group
SweetOps
HUG

Stay in touch

Get product updates and occasional news.