How to manage encryption keys on Google Cloud

Why this matters

Cloud KMS is the root of trust for encrypted data in Google Cloud. When you use Google-managed encryption, Google handles keys and you handle access to data. When you use customer-managed encryption keys (CMEK), you take on key lifecycle, permissions and destruction. That changes your failure modes. A bad IAM grant can expose data without touching a bucket. A destroyed key version can make backups useless.

Key management is mostly about separation and evidence. The team that builds an application should not be able to grant itself decrypt rights on the key that protects production data. The person who can rotate a key should not be the same person who can destroy old versions without review. You want a short list of principals with crypto access, and logs that show every use.

Cloud KMS has some sharp edges. Key rings and keys cannot be deleted. Key versions can be destroyed, and destruction is scheduled with a short cancellation window. Disabling a version is reversible; destroying it is not. Plan for those facts before you create anything, because cleanup later is not possible.

Before you start

Inventory the data stores that need CMEK. Common ones are Cloud Storage buckets, BigQuery datasets, Cloud SQL instances, Compute Engine persistent disks, Spanner databases and Pub/Sub topics. Some services support CMEK only at creation or only for new data. Check the service documentation before you promise a migration date.

Choose a protection level. Software keys are the default and work for most workloads. Cloud HSM keys keep key material in hardware security modules and cost more. External Key Manager keeps the key material outside Google Cloud and adds an availability dependency on your external key store. Pick the lowest level that meets the requirement, not the highest by habit.

Decide location and granularity. A key must be in the same location as the resource it protects, so a multi-region bucket needs a multi-region key and a regional disk needs a regional key. Use separate keys per environment and data class, for example prod-payments, prod-analytics, dev-payments. One key for everything makes blast radius large and makes rotation hard to reason about.

Set up the permission model before the first key. Put key rings in a dedicated security or platform project. Give key administrators roles/cloudkms.admin on that project or on specific key rings. Give workloads roles/cloudkms.cryptoKeyEncrypterDecrypter on specific keys, not on the project. Keep a break-glass group with admin rights, and log its use. Enable the Cloud KMS API and turn on Data Access audit logs for the key project if you need call-level visibility.

How to do it

Work from a small test key and a test resource first. Create the key ring, create a key, grant one service account access, attach the key to a test bucket or dataset, write and read an object, then check the audit log. That sequence catches location mismatches and missing service agent permissions before you touch production.

After the test, repeat the same pattern for each data store. Keep the key resource names in your infrastructure as code, not in a wiki page. If you use Terraform, the google_kms_key_ring and google_kms_crypto_key resources are the source of truth, and the CMEK settings on buckets, datasets and instances point at those names. A plan review is where you catch a key in the wrong location.

1. Create a key ring in the target location. Run gcloud kms keyrings create with a name that includes the environment and purpose, and pass the location that matches your data. Key ring names cannot be reused or deleted, so get the name right the first time.
2. Create keys with a clear purpose. Use gcloud kms keys create with --purpose=encryption, the protection level you chose and a rotation period. Set --next-rotation-time so the first rotation does not wait for a default. A key can hold many versions; the primary version encrypts new data.
3. Grant key-level IAM to groups and service accounts. Bind roles/cloudkms.cryptoKeyEncrypterDecrypter on the key, not the project. Give encrypter-only or decrypter-only roles where a workload only writes or only reads. Reserve roles/cloudkms.admin for a small group and review the policy regularly.
4. Attach the key to each Google Cloud resource. Set the default key on a bucket, the dataset default key in BigQuery, or the disk encryption key on a Cloud SQL instance. Confirm the service agent for that product has permission on the key. Test with a new object or row, not an existing one.
5. Turn on audit logs and route them somewhere durable. Enable Data Access audit logs for Cloud KMS in the key project, then create a log sink to a locked bucket or a SIEM. Alert on destroy, disable and setIamPolicy events at minimum. Raw logs in a project that an attacker can edit are not evidence.
6. Schedule rotation and rewrap where needed. Automatic rotation creates a new primary version and leaves old versions for decryption. For envelope encryption, rewrap data encryption keys with the new primary. For CMEK-backed services, follow the product guide for re-encrypting existing data before you disable old versions.
7. Restrict version destruction to a formal process. Limit the cloudkms.cryptoKeyVersions.destroy permission with a custom role. Require a ticket, a data owner approval and a backup check before anyone schedules destruction. Know that a scheduled destruction can be cancelled, but only inside the window.
8. Document key ownership and location per data store. Keep a register that maps each bucket, dataset and instance to its key, key ring, location and owner. Update it when a service is added. This register is what you use during an incident, not the console search bar.

How to check it worked

Test both the allowed and the denied path. Use a service account from each workload to encrypt and decrypt a small object. Then use a service account that should not have access and confirm you get a PermissionDenied error. A key that works for everyone is a finding, not a success.

Inspect key state. Run gcloud kms keys versions list on each key and confirm the primary version is the one you expect, the next rotation time is set, and older versions are still enabled for decrypt. Check that no version is scheduled for destruction unless there is a ticket for it.

Check the resource side. Describe the bucket, dataset or instance and confirm the CMEK key name is the one in your register. For Cloud Storage, the bucket shows the default KMS key. For BigQuery, the dataset shows the default key. For Cloud SQL, the instance shows the disk encryption key.

Read the logs. Filter Cloud Audit Logs for the key resource and look for Decrypt calls from unexpected principals or at odd hours. Confirm your alert policy fires on a test disable of a non-production key version. If the alert does not fire, fix the log sink or the metric filter before you rely on it.

Mistakes to avoid

Using one key for every project and environment. It feels simple until you need to prove which team could decrypt which data. Separate keys also let you disable one workload without breaking another.

Giving Owner or Editor at the project level to application teams. Those roles include key administration permissions, so a developer can change IAM on the key that protects production. Use key-level roles and a separate admin group.

Disabling old key versions right after rotation. Rotation does not re-encrypt existing ciphertext. If you disable the version that wrapped an old object, reads fail. Re-encrypt or rewrap first, then disable after a soak period.

Treating key destruction as routine cleanup. A destroyed version cannot be restored, and any data encrypted with it is gone. Also remember that key rings and keys cannot be deleted at all, so name them carefully and expect them to stay in the project. If you want an outside read on your key setup, Cloudverg's Cloud security review at $6,500/month is one option among others.

Frequently asked questions