AWS Secrets Manager: Tutorial & best practices

AWS Secrets Manager: Tutorial & best practices

AWS Secrets Manager is an Amazon Web Services (AWS) managed service for secure storage, access management, and rotation of sensitive values, known as "secrets." Typical examples of secrets include credentials, passwords, API keys, and database connection strings.

Getting secrets management right is a key component of cloud security posture, but it can be challenging to manage keys at scale securely. This is particularly true when considering the many secrets management solutions available on modern cloud platforms.

In this guide, we'll explore the ins and outs of AWS Secrets Manager, including its use cases, features and limitations, how it differs from related AWS services, best practices, and a real-world example with instructions to get you started.

Summary of key AWS Secret Manager best practices

The table below summarizes key AWS Secrets Manager best practices that this article will explain in more detail. These concepts can help administrators leverage AWS Secrets effectively and avoid common secrets management pitfalls.

Secrets management overview

Secrets management is a critical practice in information security and data protection. AWS Secrets Manager is one solution in the broader secrets management domain. To better understand the purpose of Secrets Manager in AWS, let’s take a step back and review secrets management fundamentals.

Secrets management involves the secure generation, storage, access control, rotation, and compliance monitoring of sensitive information ("secrets") within an organization's computing environment. Secrets computing includes confidential data that is crucial for the configuration and operation of systems, such as:

• Passwords
• Database connection strings
• API tokens, encryption seed values
• Secure Shell (SSH) keys
• SSL/TLS certificates

The primary intention of secrets management is data protection. Secrets management safeguards sensitive data from exposure and potential breaches. Encrypting secrets at rest and in transit ensures that even if an attacker gains access to the storage, the information remains unreadable.

In addition to data protection, securely storing and managing secrets ensures that organizations can recover and restore their systems and services in case of data loss or infrastructure failures.

Use cases: AWS Secrets Manager vs. KMS, ACM, and Parameter Store

In addition to AWS Secrets Manager, other solutions like Key Management Service (KMS), AWS Certificate Manager (ACM), and Parameter Store also handle sensitive data. AWS Secrets Manager stands out due to its specialized focus on secret management.

Here's how Secrets Manager differs from other AWS services:

• AWS Parameter Store: While AWS Systems Manager's Parameter Store can store secrets, it lacks features like automated rotation, and it's more suitable for static configuration values. It’s important to note AWS Parameter store is not capable of automatic cross-region and cross-account replication. If you require parameters to be available across regions or even AWS accounts, you will need an external process to cover this limitation.
• AWS KMS: KMS manages the generation and storage of symmetric and asymmetric encryption keys. KMS differs from Secrets Manager and Parameter Store in that KMS does not directly manage secrets. Instead, KMS APIs are used to perform data encryption using these KMS-managed keys
• ACM: ACM manages x509 SSL/TLS certificates for securing network traffic, whereas Secrets Manager manages a broader range of secrets. ACM certificates are supported for use with AWS Elastic Load Balancer, AWS CloudFront, and AWS API Gateway. In contrast, AWS ACM Certificate will not work for other use cases such as code signing or TLS client mutual authentication.

11 essential AWS secrets management best practices

Securely managing secrets is a complex task at scale. It takes a combination of tools, configuration, and process. The 11 best practices below can help administrators get AWS secrets management right.

Enable secret rotation

Enabling automatic rotation for credentials and other secrets whenever possible to enhance security is critical. AWS Secrets Manager can automate this process for supported services.

Doppler’s secrets management platform provides a unified “single pane of glass” with pre-built integrations for all the popular cloud service providers and several favorite SaaS applications. With Doppler’s cross-platform capability and library of SaaS integrations, teams can ensure that secrets are rotated everywhere within their environment. For example, Doppler supports native integrations with popular platforms like Kubernetes, Gitlab, CircleCI, Cloudflare, and Heroku.

Leverage tags

Implement a tagging strategy to organize and manage your secrets effectively. This helps in tracking and categorizing secrets for easier management.

Doppler has robust support for tagging and labeling. Depending on the use case (cloud infrastructure, Kubernetes resource management, etc.) utilizing a single secrets management platform across disparate technology stacks can ensure that your engineering team’s policy around tags and labels, as it relates to secrets, can be enforced and utilized uniformly.

Implement monitoring and alerts

Set up CloudWatch Logs and CloudTrail to monitor and audit the usage of secrets. Create alarms and alerts for suspicious or unauthorized access.

The centralized, cross-platform nature of Doppler, with its built-in support for auditing, eliminates the need to configure auditing and alerting for each cloud provider or SaaS application. Information security teams, auditors, and compliance automation platforms can automatically consume Doppler’s built-in audit API & fetch audit information for all your configured secrets.

Define secret rotation policies

Define and implement clear policies for secret rotation, including frequency and procedures. Regularly review and update these policies to align with changing security requirements.

One of Doppler’s most valuable features is its pre-built secrets management integrations. The engineering and testing of the Doppler’s product provides a level of abstraction and assurance that secret rotation will happen as defined in the configuration. Doppler’s consistent support for secrets rotation ensures that secrets will be rotated precisely as configured, mitigating security risks.

Secure secret storage

Secret leakage can lead to unauthorized access and further system compromise. Ensure that the secrets stored in AWS Secrets Manager are encrypted at rest and in transit. Use AWS KMS for added security.

Enforce strict access controls

Implement strict access controls and IAM policies to restrict who can manage and retrieve secrets. Avoid using overly permissive policies.

Relative to both storage and access control: Doppler; combined with AWS Secrets Manager or AWS Parameter Store, offers an additional layer of resiliency and flexibility in that after integrating with Doppler, applications can either directly consume secrets from Doppler or push them to the AWS services and continue consuming them from there without any disruption to previous workflows.

Standardize naming conventions

Establish consistent naming conventions for secrets to make them easily identifiable and prevent naming conflicts.

Use of a singular secrets management platform such as Doppler will go a long way in ensuring consistent naming conventions across a tech stack.

Have a backup and recovery plan

Regularly back up your secrets to prevent data loss. Have a disaster recovery plan to restore secrets in case of accidental deletion or other incidents.

Doppler’s solution provides highly available access to secrets through failover and fallback features.

Failover is built directly into Doppler’s routing layer via edge workers. If an outage affects Doppler’s primary servers, Doppler’s edge routing layer will automatically direct traffic to failover servers. The failover servers run hot in distinct availability zones to reduce blast radius while ensuring minimal downtime.

Doppler’s CLI has a built-in “fallback” capability. If your machine cannot reach Doppler’s APIs, as a result of losing internet access, the CLI will automatically read your secrets from a local, encrypted backup. This backup file is updated on each successful secret fetch. This functionality can create a local backup.

Test secret rotation

Given the complexity of secrets management, validating configurations before production deployment is essential. Teams should test secret rotation procedures in a non-production environment to ensure they work as expected and do not disrupt critical services.

Maintain clear documentation

Maintain thorough documentation of your secrets, their purposes, and their dependencies to facilitate troubleshooting and onboarding of new team members.

Doppler simplifies secrets documentation, including behavioral quirks and differences across environments. The Doppler application supports Secrets notes for every secret, scoped at the project level. See Doppler Secrets Notes for more information.

Regularly audit secrets management practices

Periodically review and audit your secrets management practices to identify and address any security gaps or compliance issues.

A single platform for secrets management is powerful in this respect. Doppler can be used to audit and remediate secrets and privileges across and outside an organization's tech stack.

8 AWS Secrets Manager limitations

AWS Secrets Manager is a secure and resilient service, however it does come with some inherent limitations. In this section we’ll break down 8 AWS Secrets Management limitations to consider as you design a secrets management strategy.

Access control

Secrets Manager, while a powerful tool for managing sensitive information, has limitations regarding access control. Access control in AWS Secrets Manager relies solely upon AWS Identity and Access Management (IAM) policies. IAM does provide robust access management capabilities. However, setting up fine-grained access control for individual secret versions can be challenging.

Managing access across multiple AWS accounts while ensuring the principle of least privilege can be complex, requiring meticulous policy configurations. To get it right, organizations must invest time and effort in carefully defining, testing, and maintaining IAM policies to ensure secure access to secrets, especially in multi-tenant or highly regulated environments where strict access controls are paramount.

AWS regional usage constraints

Every AWS Secrets Manager secret is specific to a single AWS region, which means that secrets stored in one region are not readily accessible from another. This may pose challenges for organizations that require multi-region architectures, as they must update secrets in the origin region to replicate secrets across regions to ensure redundancy and availability in case of regional failures.

Managing cross-region secrets replication and synchronization adds complexity to deployment strategies. Consequently, organizations must carefully plan and implement strategies to address these regional usage constraints for productive and reliable secret management across AWS regions.

Costs and pricing model

Costs associated with Secrets Manager can vary depending on the number of secrets, versions, and cross-region replicas. Understanding the pricing model and monitoring usage is essential to avoid unexpected costs.

Cross-account access

Secrets Manager allows cross-account access to secrets; however, enabling cross-account access necessitates meticulous IAM (identity and access management) configuration. Organizations must craft IAM policies that define the permissions and restrictions for cross-account users or services, ensuring the principle of least privilege is enforced.

This meticulous setup is vital to maintaining robust security practices, preventing unauthorized access to sensitive data while allowing legitimate cross-account entities to retrieve the necessary secrets. While cross-account access can be a powerful tool for collaboration and access control in complex AWS environments, it underscores the importance of well-thought-out IAM policy design and ongoing monitoring to maintain a secure secrets management infrastructure.

Focused on AWS resources

AWS Secrets Manager may not be the most suitable solution for handling secrets associated with non-AWS resources or on-premises systems. While it can store and rotate secrets used by AWS services, extending its capabilities to manage secrets for external or on-premises systems often requires additional workarounds or custom solutions.

Organizations with hybrid or multi-cloud environments may find that Secrets Manager's functionality is limited when dealing with secrets for resources hosted outside of the AWS ecosystem. In such cases, alternative, more versatile, platform-agnostic solutions may be a more practical choice to ensure consistent and secure secret management across diverse infrastructure environments.

Logging and auditing

AWS Secrets Manager offers valuable logging and auditing capabilities, allowing organizations to monitor and track access to their stored secrets and credentials. However, it's important to note that Secrets Manager's built-in logging and auditing have certain limitations. While it provides essential visibility into secret usage, including who accessed secrets and when, it may not offer the depth of auditing required for comprehensive security and compliance needs.

Organizations often find it necessary to configure additional services such as AWS CloudTrail to address these limitations and achieve a more comprehensive auditing solution. CloudTrail can capture more granular details about API calls and events across the AWS environment, providing a more extensive and detailed audit trail that complements Secrets Manager's basic logging features. By integrating Secrets Manager with CloudTrail, organizations can enhance their security posture and ensure robust monitoring and auditing of sensitive credentials and secrets.

Automated rotation may require additional customization

Secrets Manager provides built-in rotation support for popular database engines like RDS, DocumentDB, and Redshift, streamlining the process of regularly updating and securing credentials. However, regarding non-database services or custom requirements, Secrets Manager offers flexibility through custom Lambda functions.

These lambda functions can be configured to manage secret rotation for various services, allowing organizations to adapt Secrets Manager to their unique needs. Whether automating the rotation of API keys, certificates, or other sensitive information, AWS Secrets Manager's customization capabilities ensure that security best practices are upheld across various AWS resources and applications.

Secret names must be unique

Secret names must be unique within an AWS account. This uniqueness is crucial to avoid naming conflicts and ensure the precise identification of secrets within the infrastructure. Secret names are subject to certain naming constraints, which must be adhered to during creation. It's important to carefully choose secret names that are both descriptive and compliant with these constraints from the outset, as they cannot be changed once a secret is created. A thoughtful naming convention for secrets facilitates effective secret management and maintaining a secure and organized AWS environment.

How to use AWS Secrets Manager to securely store and retrieve database credentials

This example demonstrates how to use AWS Secrets Manager to securely store and retrieve database credentials (username and password) for an Amazon RDS instance.

Prerequisites

• You will need an AWS account with appropriate permissions to use AWS Secrets Manager and create an RDS instance.
• AWS CLI installed and configured with your AWS credentials.

Step 1: Create a secret

1. Log in to the AWS Management Console.
2. Navigate to the AWS Secrets Manager service.
3. Click on the "Store a new secret" button.
4. Choose the "Credentials for RDS database" option.
5. Fill in the following information:
   • RDS DB instance: Select your RDS instance from the dropdown menu.
   • Username: Enter the database username.
   • Password: Enter the database password.
6. Click "Next."
7. Give your secret a name and description. For example, you can name it "MyDatabaseSecret."
8. Review the configuration and click "Store."

Step 2: Accessing the secret

Now that you've stored your database credentials securely in AWS Secrets Manager, let's retrieve and use them in an application or script:

Using AWS CLI:

Open your terminal and run the following AWS CLI commands to retrieve the secret value:

Replace MyDatabaseSecret with the name you provided for your secret.

The output will include the secret value, which you can parse in your script or application to obtain the username and password.

Using Python:

You can use the AWS SDK for Python (Boto3) to retrieve the secret in a Python script:

Conclusion

Automation of secrets management tasks is now critically important for organizations processing sensitive data in the cloud.

When selecting and implementing a secret management automation solution it is essential to weigh the pros and cons of available solutions to fully understand if a given solution meets an organization's current and future needs. Cloud provider secrets management services come with several limitations. Ensure you properly discover and understand all of your use cases before implementing your solution to secrets management.

Proper discovery, planning, implementation, and testing is key to developing a secure and robust solution to help safeguard critical assets and mitigate the ever-evolving security threats related to cloud computing. As technology advances, a properly selected and implemented secrets management solution remains a trusted ally in the ongoing battle to protect sensitive information.

Tools like Doppler, with its pre-built integrations, cross-platform interoperability across clouds and SaaS Applications, provisioning and rotation automation, in-platform secrets documentation, and auditing capabilities add a lot to the overall success of your secrets management efforts.