AWS Secrets Manager with Terraform: Tutorial & examples

AWS Secrets Manager with Terraform: Tutorial & examples

AWS Secrets Manager is an excellent service for managing sensitive information like database credentials, certificates, passwords, and tokens in the cloud. While infrastructure as code (IaC) tools like Terraform can integrate with secret management services like Secrets Manager, KMS, or Parameter Store, Terraform’s native secrets management has some limitations.

This article discusses best practices for secret management in Terraform. We cover how Terraform can be configured to create and retrieve secrets using Secrets Manager and discuss the challenges of Terraform’s native secrets management approach. We also look at how good third-party tools can address Terraform’s limitations and provide an example of how such a tool can streamline secret management for multiple platforms.

Summary of secrets supported in Terraform

Let’s take a quick look at the different secrets that can be used within Terraform and their use cases in an AWS environment.

Summary of secret management best practices in Terraform

The table below gives an overview of the best practices to follow while managing secrets in Terraform. We will explain each in detail in the following section.

Best practices for secret management in Terraform

Now that you’ve gotten a brief idea about the best practices, let’s take a deeper dive into each one, with examples.

Learn how WhatNot stopped secrets sprawl using Doppler

Read Case Study

Avoid storing secrets in plain text

Storing secrets as plain text inside code exposes the secrets to anyone having access to your version control system when you push the code. This can be dangerous if the repository is public and can be avoided by using secrets management services like AWS Secrets Manager, AWS Parameter Store, or Hashicorp Vault.

Here’s an example of storing secrets using AWS Secrets Manager in Terraform.

1. Create a main.tf file and add the following code to create your secret.

2. Create a variables.tf file as follows:

3. Now, use terraform plan and pass the secret as a variable to review your changes:

4. Finally, perform terraform apply to deploy the configuration.

Use the Terraform data module to retrieve secrets

The Terraform data module allows Terraform to use external information in its configuration. This can be done by specifying the appropriate provider, such as an AWS Secrets Manager data source.

The following example demonstrates how you can use the AWS Secrets Manager data source to retrieve secrets into your Terraform configuration from AWS.

1. Create a main.tf file and add the following code specifying the Secrets Manager data module:

2. If you have stored secrets as JSON, you can use the following method instead, which utilizes jsondecode:

Pass secrets to Terraform as input variables, variable definition files, or environment variables

Secrets can be passed to Terraform as input variables, variable definition files, or environment variables.

Input variables

Users can pass the secrets with the -var command line option while running Terraform commands. The following example shows how users can pass input variables during the Terraform plan state and apply the Terraform configuration to deploy resources.

Variable definition files

Users can enter the secrets inside a .tfvars file and specify this on the command line using the -var-file argument. However, this requires users to be careful to not push the tfvars file to the version control system and maintain it themselves.

Let’s go through an example by creating a secrets.tfvars file with the following content:

Apply the configuration to deploy the changes using the following command:

Environment variables

Users can export secrets in the command line as TF_VAR_ followed by the variable name before running the Terraform commands. The following example shows how users can use environment variables to pass secrets to Terraform:

Store Terraform state in a remote backend instead of a local state file

Since a Terraform state file contains sensitive information about your resource configurations, it is recommended to store the state files in a remote backend like S3 bucket and avoid pushing them to the version control system.

To create a new S3 backend for your Terraform project, follow these steps:

1. Add the following code inside your main.tf file after specifying your bucket name:

2. Next, perform terraform init to initialize your remote backend:

Note that it is recommended to have versioning enabled on your S3 bucket so that you can recover old state files in case.

Use tools that can inject secrets dynamically for team collaboration

Integration with tools like Doppler can help inject secrets from multiple providers dynamically into your Terraform code, helping keep a single source of truth for the secrets across your projects.

The following example demonstrates how users can make use of Doppler CLI to dynamically inject secrets into Terraform.

1. First, make sure you have Doppler CLI installed on your system and configure it by doing doppler login and doppler setup to grant Doppler API access to the required secrets.

2. Next, integrate Doppler with AWS Secrets Manager. Doppler supports integration with multiple services; check out this article to learn how to integrate Doppler with AWS Secrets Manager.

3. Once you’ve finished the integration, create your secret from the Doppler dashboard. You could choose to generate a random secret value as follows for added security:

4. Finally, run the following command to inject the secret inside your Terraform configuration:

Let’s verify this by running the command from the terminal.

As you can see, Doppler dynamically injected the secrets into the Terraform configuration while keeping the secret as a sensitive value.

Challenges of Terraform’s native secrets management approach

Terraform’s native secrets management has the following drawbacks:

• Insecure by default: By default, secrets are stored as plain text in Terraform. Users have to encrypt manually to create a secure approach.
• Limited scope: Terraform’s secrets capability is limited to Terraform configurations only; it cannot be used to store or manage secrets outside Terraform.
• Tightly coupled: The secrets created using Terraform are tightly coupled to Terraform configurations. Terraform’s native approach does not provide a centralized way to store secrets across multiple accounts or environments.
• Version control issues: Since Terraform stores secrets as plain text by default, they need to be excluded from version control. This causes issues with collaboration and auditing.

How Doppler helps overcome Terraform’s limitations

Doppler can help improve Terraform by providing the following advantages:

• Secure by default: All the secrets stored using Doppler are encrypted at rest and in transit. Users do not have to manually encrypt secrets.
• Broad scope: Doppler is a dedicated secrets management solution that can help store and manage secrets across all infrastructure, applications, and teams.
• Decoupled from configurations: Secrets in Doppler are decoupled from Terraform configurations and can be used across multiple accounts or environments. This makes configurations more robust and reusable.
• Better version control: Since Doppler stores secrets externally, Terraform configurations can be stored within version control as usual, helping improve collaboration and auditing.
• Support for multiple providers: Doppler integrates with various secret services and cloud providers beyond the limited set supported by Terraform natively.
• Advanced features: Doppler provides additional features like secret rotation, auditing, access control, etc., helping achieve a robust secret management solution.
• Simple integration: Doppler integrates easily with Terraform using its Terraform provider. Secrets can be fetched directly from Doppler into Terraform configurations.

Conclusion

This article provides an in-depth look at using AWS Secrets Manager with Terraform. While Terraform can be used to provision and manage secrets in Secrets Manager, Terraform’s native approach has some limitations. We described these limitations in detail and went through best practices and security recommendations that can help improve Terraform’s secret management capability.

The article also highlights how combining Terraform with tools like Doppler can make secret management more efficient and secure. Since Doppler integrates with multiple secret management services, it can be a reliable option for managing secrets across different cloud providers, empowering teams to scale their deployments efficiently while maintaining security and control over sensitive data.

If you’re interested in learning more about Doppler, check out the free team trial for 14 days without a credit card or request a demo to try Doppler for the enterprise.