The Secret Life of AWS: The Plaintext Password (AWS Secrets Manager)
.jpg)
The Secret Life of AWS: The Plaintext Password (AWS Secrets Manager)
Why hardcoding credentials is a breach waiting to happen.
#AWS #SecretsManager #Security #DevOps
Part 45 of The Secret Life of AWS
Timothy successfully connected his Inventory microservice to the Checkout database across the new Transit Gateway. The network was secure. The traffic was private.
He opened his Lambda function configuration to add the database connection string. In the Environment Variables section, he typed:DB_HOST: 10.0.5.12 DB_USER: admin DB_PASS: SuperSecretDatabasePassword123!
He clicked Deploy. The function ran, connected to the database, and returned a 200 OK.
Margaret walked by and glanced at his screen. "Timothy," she said, "you just secured the entire network, but you left the master database password sitting in plain text."
"It's an environment variable," Timothy defended. "It's injected at runtime. It's not like I committed it to the Git repository." He paused. "Well, not the production password, anyway."
"Environment variables in Lambda are stored in plain text," Margaret explained. "Anyone with the lambda:GetFunctionConfiguration IAM permission can open the console and read that password. If a junior developer needs to check the timeout settings of this function, they will also see the master database credentials. You have decoupled the secret from the code, but you have not secured the secret."
"We need to move from static configuration to dynamic retrieval," she continued. "Secrets should never be stored in plaintext anywhere—not in code, not in environment variables, and not in configuration files."
The Secure Vault (AWS Secrets Manager)
Margaret opened the AWS Secrets Manager console and clicked Store a new secret.
She selected "Credentials for Amazon RDS database" and inputted the username and password. She selected the Checkout database instance, allowing Secrets Manager to map the credentials directly to the infrastructure.
She named the secret prod/checkout/db-credentials.
"Now the password is encrypted at rest using AWS Key Management Service (KMS)," Margaret said. "It is no longer a string of text floating in a configuration menu. It is a secure, managed object."
Dynamic Retrieval and Caching
"But how does the application get the password?" Timothy asked. "If I don't give it the environment variable, the connection will fail."
"The application has to ask for it," Margaret explained. "At runtime, your Lambda function will make an API call to Secrets Manager using the AWS SDK. It requests the secret named prod/checkout/db-credentials. Secrets Manager decrypts the payload and hands it to your function in memory."
"Doesn't that add latency to every single Lambda execution?" Timothy pointed out.
"It would, if we fetched it on every request," Margaret agreed. "To avoid the latency and the per-API-call cost, we cache the secret in memory outside the main Lambda handler function. The execution environment fetches it once when it spins up, and reuses it for subsequent requests."
Timothy thought about the architecture. "That means the Lambda function needs IAM permissions to read the secret."
"Exactly," Margaret nodded. "We are relying on the Principle of Least Privilege again."
She opened the IAM console and updated the Lambda Execution Role. She added a policy that allowed the secretsmanager:GetSecretValue action, strictly scoped to the exact Amazon Resource Name (ARN) of the prod/checkout/db-credentials secret.
"Now, the function has the cryptographic authority to fetch its own password," she said. "If an attacker views the Lambda configuration, they just see the ARN of the secret. If they try to read the secret themselves, IAM denies them."
Cost vs. Value (Parameter Store)
Timothy looked at the billing documentation. "Secrets Manager costs $0.40 per secret per month, plus API costs. Is there a free way to do this?"
"For simple, non-database secrets like third-party API keys, we could use AWS Systems Manager Parameter Store," Margaret acknowledged. "It is free for standard parameters. But we are using Secrets Manager here because of its integration with RDS. It doesn't just store the password; it changes it."
Automated Rotation & Versioning
"Right now, how often do you change that database password?" Margaret asked.
Timothy winced. "Never. If I change it, I have to find every application using it, update their environment variables, and redeploy them all at the exact same time."
"With Secrets Manager, you configure a rotation schedule—say, every 30 days," Margaret explained.
"Secrets Manager will automatically generate a new, complex password, update it in the RDS database, and update the stored secret. Because your applications fetch the password dynamically, they will instantly start using the new password. Zero downtime. Zero code changes."
"What if the rotation fails and the database rejects the new password?" Timothy asked.
"Secrets Manager maintains multiple versions of each secret," Margaret replied. "If a rotation fails, we can instantly roll back to the previous version to restore connectivity."
Timothy deleted the plaintext environment variables from his Lambda function. He updated his code to fetch the credentials via the SDK.
The network was private. The identity was scoped. And now, the credentials were secure.
Key Concepts
- AWS Secrets Manager: A service that helps you protect secrets needed to access your applications by enabling you to rotate, manage, and retrieve database credentials and API keys dynamically.
- AWS Systems Manager Parameter Store: An alternative service that provides secure, hierarchical storage for configuration data management and secrets management (free for standard parameters, but lacks native RDS rotation).
- Plaintext Credentials: Unencrypted sensitive information stored directly in code, configuration files, or environment variables. This is a severe security vulnerability.
- Dynamic Retrieval: The architectural pattern of fetching secrets via an API call at runtime, keeping them in memory only for as long as needed.
- Secret Caching: Storing retrieved secrets in memory (outside the Lambda handler) to reduce API latency and AWS costs on subsequent invocations.
- Automatic Rotation: The process where Secrets Manager automatically updates a secret on a scheduled basis, updating both the secure vault and the target resource simultaneously.
Aaron Rose is a software engineer and technology writer at tech-reader.blog and the author of Think Like a Genius.
Comments
Post a Comment