Managing Credentials in Terraform Cloud & Enterprise

Managing Credentials in Terraform Cloud & Enterprise

There are several ways to manage credentials or other secret types in Terraform Cloud and Terraform Enterprise, either natively, or with purpose-built secrets management utilities like HashiCorp Vault, so this is a somewhat opinionated article that lists what I believe are currently the best options. Please note this is not a replacement for some other best practices, such as keeping your Terraform CLI and code up to date.

The described patterns follow some common principles. Credentials should be:

  • As unique as possible per workspace
  • Easy to rotate
  • As dynamic as possible
  • Protected with RBAC

There are also some nuances between Terraform Cloud and Terraform Enterprise that I will call out in each section.

In terms of security, both Cloud and Enterprise products encrypt their sensitive variables using the Vault Transit secrets engine and do not allow any external API call to decrypt these values. For more details please see the Data Security page.

If you still have questions by the end of this blog post, try attending one of our community office hours or booking a 1:1 technical session.

»Managing Credentials Using Only Terraform Workspaces

Using only Terraform workspaces, your cloud vendor’s Terraform provider, and the Terraform Cloud/Enterprise provider, you can set up a “Credentials” workspace that is able to generate new credentials and rotate the ones used by other workspaces. To avoid the secret zero problem, when setting up the credentials workspaces, you can make use of the Terraform Agents pattern described in the next section.

Note: In Terraform Cloud all workspaces live in the same Organization.

Terraform

Pros

  • There’s a separation between development environments.
  • It’s possible to use different types of secrets.

Cons

  • Requires a 3rd party API trigger to force a re-deployment of all “Credentials” workspaces.
  • Complex RBAC setup that does not exist in the free tier.
  • Secrets are stored in the “Credentials” workspace state.
  • “Credentials” workspaces require a Terraform Cloud/Enterprise User or Team token to access other workspaces that reside in another Organization.

»Managing Credentials Using Only Terraform Agents

Both Terraform Cloud Business tier and Terraform Enterprise support running your code using external agents. This feature is called Terraform Agents. Any cloud provider declared in your Terraform code is able to take advantage of the credentials set in the Terraform Agent environment, which means the credentials do not need to be set at the workspace level.

HashiCorp Solutions Engineer Andy Assareh has a repo and a recording to help walk you through this pattern.

Terraform

Pros

  • There’s a separation between development environments.
  • No credentials are set in project workspaces.
  • No credentials are stored in the workspace state.
  • One of the least complex patterns.

Cons

  • Only addresses cloud access credentials, not other kinds of secrets.
  • Requires Terraform Cloud Business tier.
  • Careful RBAC is required to ensure developers are not able to change Agent Pools in the workspace.
  • Difficult to scale and audit when using multiple accounts.
  • In Terraform Cloud Business tier, there are some limits on the number of Agents and Agent Pools an Organization can have.
  • Difficult to have a 1:1 mapping between workspace and Agent.

»Managing Credentials Using Vault

It’s no secret HashiCorp Vault is able to generate dynamic credentials for all major cloud vendors, databases, etc. For those who know about Vault, this integration with Terraform is the first solution they ask about, since secrets management is Vault’s primary use case. Here are a few patterns to make that integration work.

»Direct Integration with a Vault Plugin

There isn’t much to explain in this workflow. Most of the complexity is in setting up Vault authentication and some Terraform template code.

Terraform

Pros

Cons

  • Need to install a 3rd party plugin. More details are in the plugin repo.
  • Because we are using the Vault provider, secrets will be present in the state file.
  • Requires Terraform code changes to use the Vault provider.
  • Increase in complexity to set up Vault correctly.

»Integration with CI/CD

At the heart of this integration pattern, is the ability to confidently authenticate to Vault with an identity that is unique and combines workflow, repository, and branch/environment.

This is not the case with all CI/CD implementations, so I’m only mentioning two implementations where I know it’s possible.

Vault tokens should be very short-lived and linked to an entity, to restrict access in case any token gets leaked and allow for better auditing. Avoid storing Vault tokens as secrets!

Regarding the workflow, the CI/CD runner will need to authenticate to Vault and retrieve:

Pros

  • It’s possible to have 0 hardcoded or long-lived credentials, using only identities.
  • There’s a separation between development environments.
  • Different credentials for different branches.
  • Terraform Enterprise and Terraform Cloud credentials are not stored in Terraform state or the CI/CD platform.

Cons

  • Requires a CI/CD system able to assign an identity that combines a unique run id, workflow/pipeline, repository, and commit id.
  • May lead to secret sprawl.
  • Increase in pipeline complexity.

These pros and cons apply to the two subsections below.

»GitLab CI/CD Integration

GitLab runners are able to authenticate to Vault using the JWT auth backend, where you can configure separate roles for staging, dev, or production.

Once authentication has been solved, GitLab CI/CD will be able to retrieve the necessary secrets and interact with the Terraform Cloud/Terraform Enterprise API. Here’s an example.

For private repositories, you’ll need to have GitLab Premium.

Terraform,

»GitHub Actions Integration

For GitHub Actions to authenticate to Vault there are two options:

If you use AppRole, please make sure you also have a cron job set up to rotate the environment secrets, and use the “Repository Environments” to distinguish between production and non-production.

If you use the custom Vault GitHub Action authentication backend, right now it’s not possible to distinguish which branch is being executed.

»Pick the Solution that’s Right For You

There is no right answer or one-size-fits-all solution to managing your credentials and secrets within Terraform Cloud and Terraform Enterprise. This is highly dependent on your current requirements, environment, tools, etc, so it’s up to you to select the pattern that is best for you and your team, taking into account the pros and cons of each solution.

Please bear in mind that all of these solutions require you to plan and execute an efficient team/application onboarding process to be successful. To learn more about onboarding applications into Vault check out our other blog post Onboarding Applications to Vault Using Terraform: A Practical Guide or book a 1:1 session with one of our experts.


Source: HashiCorp Blog

16
Oct, 2021

Security in the Cloud: It’s Complicated

Security in the Cloud: It’s Complicated

When we launched the first HashiCorp State of Cloud Strategy survey earlier this year, we knew that security plays a key role in how organizations adopt cloud and multi-cloud architectures. But the survey of 3,200+ practitioners and decision makers from HashiCorp’s opt-in database revealed surprising complexity around the impact of security issues on cloud strategy.

In this blog post, we’ll take a deep dive into five key security findings from the survey. For more analysis around other aspects of the results, read this series of blog posts and check out the survey site for a visual presentation of key results and methodology.

»Security Concerns Are the Second Biggest Cloud Inhibitor

Even as cloud adoption continues to accelerate, that doesn’t mean it’s all smooth sailing. When asked about their organization’s primary cloud inhibitor, security concerns were the second most popular choice (47%), trailing only cost issues (51%). A closer look reveals some interesting differences in cloud program inhibitors by region, role, company size, and industry:

  • Security was the top concern for financial services organizations and healthcare/ biotech companies. Some 59% of finserv firms named security a top-three concern, as did 52% of healthcare/biotech respondents. Notably, security was named as a top-three cloud inhibitor by just 39% of entertainment/media firms and 41% of software and services companies.
  • Geographically, security concerns were most pervasive in the Asia-Pacific region, cited by 52% of respondents, while respondents in Europe/Middle East/Africa had the lowest percentage (39%).
  • Security concerns also varied significantly by organization size. Not surprisingly, perhaps, small businesses (<100 employees) were least concerned, with just 38% of these respondents calling security a top-3three cloud inhibitor, compared to 55% of large enterprises (>5,000 employees).
  • By role, business decision makers (40%) named security as a top-three cloud inhibitor notably less often than practitioners (47%) and technical decision makers (46%) did.

Why might business decision makers see security as less of an inhibitor than do practitioners or technical decision makers? Possibly because business decision makers must weigh the costs and benefits of moving to the cloud, whereas practitioners and technical decision makers are tasked with implementation and focus more on how to make it successful.

»Security Can Also be a Multi-Cloud Driver

But just as cost calculations in the cloud can be complicated, security isn’t just a cloud concern for the survey respondents, it can also be a cloud benefit.

When asked what business and technology factors were driving their multi-cloud adoption, security and governance was mentioned in the top three by 16% of respondents. As you might expect, cloud security was an even more important driver in the public sector (19%) and the financial industry (18%). One significant geographical difference was that only 11% of Latin American respondents cited security as a multi-cloud driver.

Security doesn’t live in isolation. According to one solution engineer, “The main thing is the security concerns, privacy, regulatory control of the data, and all costs involved around it.” But as another security engineer noted, “Cloud services allow more granular security controls.”

»Data/Privacy Protection Is the Top Cloud Security Concern

The concept of security can cover many things, so what exactly are practitioners and decision makers really worried about? A lot of different things, it turns out, and no single concern was mentioned by more than half of respondents.

What

»Skills Shortages Affect Cloud Security, Too

If those are the security threats, what’s keeping organizations from solving them? When it comes to security, the most common challenges involve staff and skilling issues, cited by more than a quarter (26%) of respondents.

What

Notably, a shortage of the proper skills was called out as a top-three challenge to operationalizing multi-cloud overall by more than half (57%) of respondents. That could be why almost 18% of respondents are using or plan to use commercial security tools as a service. Interestingly, only 3% of respondents were concerned about financial costs when it comes to security, possibly because security is considered so important that when a security issue is identified, a budget will be allocated to address it.

»Security Automation Tools Are Becoming More Essential

Survey respondents overwhelmingly agreed that the right tools are critical for managing multi-cloud environments and for every component of their cloud efforts, including provisioning, networking, security, and application deployment. Ninety-four percent of respondents called infrastructure automation tools “important” or “extremely important” to operationalize their multi-cloud environments.

Surprisingly, despite the importance of security in cloud environments, just about half (50%) of respondents use infrastructure automation tools in the security arena, trailing provisioning (75%), application deployment (69%), and networking (58%). That may be about to change, however, as 44% of respondents plan to use infrastructure automation tools for security, easily topping the other four components.

What

»Conclusion

Survey respondents from different industries expressed varied cloud inhibitors, security concerns, business challenges, and provided insight into cloud adoption. Not too surprisingly, a current area of focus across industries is a shortage of the skills needed to help implement their cloud initiatives. As noted above, that could be why so many companies are looking to leverage security automation tools.

For more insights into how companies are transitioning to the cloud and multi-cloud environments, check out the full HashiCorp State of Cloud Strategy Survey. And read more survey analysis in the HashiCorp blog.


Source: HashiCorp Blog