No Touch Secrets with Doppler

Managing a secrets management platform is undoubtedly a complex and challenging task for many organizations. The landscape is cluttered with a variety of systems, integration methods, authentication methods, compliance standards and frameworks, and edge cases, making it all too easy for an organization to inadvertently expose a sensitive credential. However, it’s important to recognize that it doesn’t have to be this way. By implementing a robust secrets management strategy with a common sense approach to authentication, logging, alerting, and access control, organizations can empower their developers to focus on what they do best—building innovative solutions—while allowing platform, DevOps, and Site Reliability Engineering (SRE) teams to dedicate their efforts to high-value, strategic work that drives the business forward.

In the upcoming series of blog posts, I aim to share my insights and experiences on how I have effectively utilized Doppler to create a seamless “no-touch” solution for deploying AWS infrastructure. This approach employs a combination of widely used technologies, including Docker, Doppler, and Terraform, showcasing how these tools can work together to simplify the secrets management process and enhance operational efficiency. Stay tuned for a detailed exploration of this methodology and its practical applications.

No-Touch Secrets Management

No-touch secrets management represents a sophisticated philosophy that prioritizes security by ensuring that developers never have direct access to the actual contents of sensitive information, such as authentication credentials and API keys. This approach is fundamentally designed to minimize the risk of exposure to secrets, thereby enhancing overall security protocols. A prime illustration of this concept can be found in the use of AWS credential pairs, where the developer interacts with the system without ever needing to see or handle the credentials themselves. This method not only safeguards the integrity of the secrets but also streamlines the development process by allowing developers to focus on building applications without the inherent risks associated with managing and accessing sensitive data directly.

An additional important aspect to keep in mind when contemplating no-touch secrets management is the capability to implement frequently rotating secrets. This practice is particularly effective when these secrets are associated with short Time-to-Live (TTL) values. By ensuring that secrets are rotated regularly and have limited lifespans, organizations can significantly enhance their security posture, minimizing the risk of unauthorized access and data breaches. This dynamic approach not only strengthens the overall security framework but also ensures that sensitive information remains protected against potential vulnerabilities.

Architectural Overview: Bringing the No-Touch Solution to Life

With this idea of no-touch secrets management in mind, it's time to translate these principles into a concrete architectural solution. The following high-level architecture diagram serves as a visual blueprint that encapsulates the core components and interactions of our innovative approach.

Architecture Diagram Breakdown

The architecture presented showcases a fairly standard three-tiered web architecture design that effectively incorporates several critical components, including cross-region Amazon RDS replication, to ensure data availability and resilience across different geographical locations. In addition, it employs Elastic Load Balancing, which plays a vital role in the efficient distribution of incoming traffic to various backend servers, thus enhancing performance and reliability. Network segmentation is achieved through a custom-defined virtual private network, providing an added layer of security and organization to the infrastructure. Positioned directly in front of the Load Balancer is a Web Application Firewall (WAF), which is a crucial element in safeguarding applications from various online threats and vulnerabilities. This WAF will be the focal point of our discussion as we delve deeper into this subject over the course of the series.

The WAF

The AWS serves as a robust security layer designed to protect applications from a variety of prevalent security vulnerabilities. This includes safeguarding against threats such as Cross-Site Scripting (XSS) and SQL Injection (SQLi), as well as implementing effective bot detection measures. In addition to these protective features, AWS also provides an essential visibility layer, enabling comprehensive traffic monitoring. This dual functionality not only enhances the security posture of applications but also allows for real-time insights into traffic patterns and potential vulnerabilities.

In our case, the AWS WAF deployment, configuration, and custom rules are all part of a singular deployment package that runs as part of a CICD pipeline. Infrastructure Security Engineers are able to write, test, and deploy using a terraform pipeline that leverages Doppler secrets management GitHub integration. This allows the GitHub secrets configuration to always be updated with the relevant secrets values.

The Issue

The central issue at hand revolves around our established protocol of refraining from deploying any changes into production environments without conducting thorough tests beforehand, correct? This principle aligns with widely accepted practices observed throughout various industries. However, it is important to note that non-production environments often receive less stringent security scrutiny compared to their production counterparts.

To facilitate our development processes, we maintain a lower-level 'dev' account that lacks a comprehensive Continuous Integration and Continuous Deployment (CICD) pipeline. Our developers utilize AWS credential pairs, which grant them access to a 'sandbox' environment specifically designed for testing new features and functionalities. This approach enables them to experiment and innovate without the risk of impacting live systems.

With that in mind, we have developed an internal tool specifically for Doppler administrators that streamlines the process of configuring access to designated configurations. This tool also allows administrators to issue Doppler tokens to developers on an ‘as-needed’ basis. These Doppler tokens are instrumental, as they provide tightly scoped access to the Doppler platform all while ensuring that developers can work securely within the defined parameters while safeguarding sensitive resources.

The Followup

In the next part of this series, we will conduct a technical deep dive into the code development of our infrastructure automation tools. We will explore how these tools provide developers and engineers with essential access to the AWS testing environment while maintaining the integrity of sensitive information. Additionally, we will examine the evolution of this tool now that Doppler has introduced new auto-rotating functionality, making life even easier for our development team. Stay tuned to the Doppler blog for insights on secrets management and all things cloud security.