How to store application secrets on AWS

Why this matters

A secret in a config file is a secret you cannot rotate cleanly. It ends up in a repo, a Dockerfile, a CI variable, a screenshot in a ticket. Once it lands in git history, deleting the line does nothing, because the value is still in the commit. The fix is to take secrets out of the code path and put them somewhere that can replace them on demand.

AWS gives you two managed stores. Secrets Manager is built for credentials: it keeps versions, rotates them on a schedule, and supports cross-account access. SSM Parameter Store holds plain strings and SecureString values encrypted with KMS. Parameter Store is simpler and cheaper, Secrets Manager carries rotation.

The store is the easy half. Permissions and retrieval are the hard half. Your app should read the secret at runtime through an IAM role, cache it in memory, and handle the value changing underneath it. A vault full of secrets with a role that can read every one of them is worse than a plaintext file, because it looks safe.

Before you start

List every secret the app needs: database credentials, third-party API keys, signing keys, SMTP passwords, webhook secrets. For each one, note which service reads it, which environment it belongs to, and whether it can rotate automatically. Rotation decides the store. An RDS password rotates with the built-in Secrets Manager function, while a vendor API key with no rotation endpoint needs a written manual procedure.

Pick a naming scheme first. Paths like prod/orders/db and staging/orders/db sort well in the console, read clearly in a policy ARN, and make it easy to grant a role access to one environment only. Decide on a customer managed KMS key per environment and create it before the first secret, because the key is chosen at creation time.

Check how the app gets AWS credentials. On Lambda, ECS and EC2 it is the execution role or instance profile. On EKS it is IAM Roles for Service Accounts. In CI it should be OIDC federation to a role. If someone has to paste an access key into a pipeline variable to read the secret, fix that before you migrate anything.

How to do it

The work splits into four moves: create the store and the key, grant the smallest permission that works, change the app to fetch and cache, then turn on rotation and watch it. Do one secret end to end in a dev account before you migrate the rest, so you have a policy and a code path to copy.

Secrets Manager stores a string or a binary blob, and each version carries staging labels. AWSCURRENT is the live value and AWSPREVIOUS is the one before it. Store structured credentials as JSON with fields like username, password, host and port, because the rotation functions expect that shape and the app can build a connection string from one blob. AWSPREVIOUS stays readable during rotation, which gives a rolling deploy time to catch up.

Fetch the secret when the process starts, keep it in memory, and fetch again when an authentication attempt fails with a credential error. That covers rotation without a restart. Do not fetch on every request: it adds latency and cost and puts Secrets Manager in the request path.

Keep config in Parameter Store. Endpoints, queue names and feature flags do not need rotation, and moving them out of Secrets Manager keeps that list short enough for a person to review.

1. Create a KMS key per environment. Make a symmetric customer managed key in KMS for each environment. Keep the key policy tight at first and add application roles one at a time later. Do not share a key between prod and dev.
2. Store the first secret with a clear path. Create it with aws secretsmanager create-secret and a name like prod/orders/db. Put username, password, host and port in a JSON secret string, with the same field names in every environment.
3. Scope the read policy to one ARN. Allow secretsmanager:GetSecretValue on that exact secret ARN, not a wildcard. Allow kms:Decrypt only for the key that encrypted it, and tag secrets by environment if you want to grant by tag.
4. Attach the policy to the compute role. In Lambda that is the execution role, in ECS the task role, in EKS a service account role. Never attach a secret-reading policy to a human user or a shared CI user.
5. Replace hardcoded values with a startup fetch. Swap the config loader for a Secrets Manager call that runs once at startup and keeps the parsed value in memory. Log the secret name, never the value.
6. Turn on rotation and test it early. Enable rotation with the RDS template or your own Lambda function. Run it on a short schedule first, confirm the app reconnects, then set the interval your team can support.
7. Move plain config to Parameter Store. Put non-sensitive settings in String parameters and low-sensitivity values in SecureString under a path like /app/prod/. This keeps the Secrets Manager list short and reviewable.
8. Sweep the old copies out. Search repo history for the old values, rotate anything you find, and clean CI variables, .env files, task definition environment blocks and machine images. A new store does nothing while the old copy is still live.

How to check it worked

CloudTrail records Secrets Manager API calls, including GetSecretValue. Filter event history by event name and read the principal on each line. Every entry should be a role you recognize. A developer SSO role or an old CI user showing up there is a permission to remove.

Test the denial path. Run aws secretsmanager get-secret-value with the secret name and a profile that should not have access, and confirm you get AccessDenied. Then call it through the app's own role and code path and confirm it works. A policy you never test negatively is a guess.

Force a rotation with aws secretsmanager rotate-secret and watch the logs. The connection should drop, the retry should refetch, and traffic should resume without a deploy. If the app needs a restart to pick up the new password, your cache has no refresh path.

Add gitleaks or trufflehog to the pipeline so a committed key fails the build. Run it across history once, not just the current tree.

Mistakes to avoid

The usual failure is an app role with secretsmanager:* on *. It gets written during a deadline and stays for years. Scope to exact secret ARNs and to kms:Decrypt on one key, and keep the policy in your repo so a change gets reviewed. The other one is a CI job still holding a long-lived access key. Move it to OIDC.

Secrets leak through logs more often than through breaches: an exception that prints the connection string, a debug line with a token, a request dump in CloudWatch. Log the secret name and version, never the value, and scrub it in error handlers. Watch the cache too. A value cached with no expiry turns rotation into an outage at an odd hour.

Do not put plaintext secrets in Lambda environment variables or ECS environment blocks. ECS has a secrets block that resolves from Secrets Manager at task start, so use it. When KMS returns AccessDenied, fix the key policy for that one role instead of granting kms:* to make the error go away.

Deleting a secret with force delete skips the recovery window and breaks anything still reading it, so schedule deletion and let the window run. If you want an outside read-only pass over the IAM and KMS side before a customer security review, the Cloud security review package at $6,500/month covers identity, exposure, encryption and logging across up to five accounts with a ranked fix list. It is one option among several, and the steps above still apply.

Frequently asked questions