How to turn on audit logging in Google Cloud

Why this matters

Google Cloud writes audit logs for every project, but not every type is on. Admin Activity logs, which record changes like creating a VM or changing an IAM policy, are always on. System Event logs are always on too. Data Access logs, which record reads, writes and queries against your data, are off by default. That gap matters when someone asks who downloaded a file or ran a BigQuery query.

Turning on Data Access audit logs gives you a record with the principal, the resource, the method, the time and the source IP. During an incident, that record lets you separate a noisy service account from a compromised one. It also shows whether a person read data before deleting it. Admin Activity alone cannot answer that.

Audit logs are not the same as application logs. They come from Google Cloud's control plane and data plane, and they use a fixed log name pattern. You turn them on per service, not with one global switch. That is good for cost and bad for anyone who wants a single click. Plan the scope before you start.

Before you start

You need permission to change audit settings on the project, folder or organization. A role like Logging Admin works, or a custom role with logging.config.update. If you edit the IAM policy directly, you also need permission to set the policy. To read the logs afterward, Logs Viewer is enough.

Decide the scope first. A single project is fine for a test. For production, set it at the folder or organization level so new projects inherit the setting. Audit configuration is inherited down the resource hierarchy. A child resource can add its own audit config, so keep a written list of where you changed it.

Pick services and log types. Data Access has three types in the console: ADMIN_READ, DATA_READ and DATA_WRITE. Not every service supports all three. Enable what you need for Cloud Storage, BigQuery, Secret Manager or whatever holds sensitive data. Do not enable every service on day one.

Check retention. Admin Activity logs go to the _Required log bucket, which keeps them for 400 days. Data Access logs go to the _Default bucket, which keeps them for 30 days by default. If you need longer retention, create a log sink to Cloud Storage, BigQuery or Pub/Sub before you rely on the logs.

How to do it

Use the console for a small change and Terraform for repeatable coverage. The console path is IAM & Admin, then Audit Logs. Select the project, folder or organization at the top. The page lists Google Cloud services. Pick one service, check the Data Access types you want, and save. Repeat per service.

Exclusions are part of the same page. Use them to drop high-volume, low-value events before they are written. A common exclusion is a service account that reads the same bucket thousands of times a day, or health check reads from a monitoring system. Exclusion filters use the Logging query language. Test the filter in Logs Explorer first so you do not drop the events you need.

For code, use the google_project_iam_audit_config resource in Terraform. Set project, service and one audit_log_config block per log type. That keeps the setting reviewable in a pull request. If you edit the IAM policy JSON by hand, download it with gcloud projects get-iam-policy, add auditConfigs, and upload it with gcloud projects set-iam-policy. Hand edits are easy to break, so use them only for a quick test.

After the audit config is on, create a sink. A sink sends matching log entries to a destination you control. Filter on logName:"cloudaudit.googleapis.com" to capture all audit logs, or narrow it to data_access. Grant the sink's writer identity access to the destination. Without that grant, the sink is created but no logs arrive.

1. Choose the resource level. Decide whether the setting belongs on one project, a folder or the whole organization. Use the highest level that matches your policy so new projects inherit it. Write down the level you chose.
2. Open the Audit Logs page. In the Google Cloud console, go to IAM & Admin, then Audit Logs. Pick the project, folder or organization in the scope selector. The page shows a table of services.
3. Enable Data Access log types. Select a service such as Cloud Storage or BigQuery. Check ADMIN_READ, DATA_READ or DATA_WRITE for the operations you need. If a checkbox is unavailable, that service does not support that type.
4. Add an exclusion for noise. Use the Exclusions tab or section to filter out known high-volume principals or methods. Build the filter in Logs Explorer first and confirm it matches only the events you want to drop. Keep a comment with the reason for each exclusion.
5. Save and repeat. Save the service configuration, then repeat for each service that holds sensitive data. Do not enable every service at once. Add services as you confirm the log volume and owners.
6. Create a log sink. Create a sink to Cloud Storage, BigQuery or Pub/Sub for retention or analysis. Use a filter such as logName:"cloudaudit.googleapis.com" to include audit logs. Grant the sink writer identity write access to the destination.
7. Set an alert for audit changes. Create a log-based alert for changes to audit configuration or IAM policy. A filter on method names like SetIamPolicy catches the events that could turn logging off. Route the alert to a channel someone actually watches.

How to check it worked

First, confirm the configuration. Run gcloud projects get-iam-policy PROJECT_ID --format=json and look for auditConfigs. You should see your service and the log types you enabled. For a folder or organization, use the matching get-iam-policy command with the folder or organization ID.

Second, generate an event. Read a Cloud Storage object, run a BigQuery query, or access a secret. Wait a few minutes. Then open Logs Explorer and run a query like logName:"cloudaudit.googleapis.com%2Fdata_access" AND protoPayload.serviceName="storage.googleapis.com". Set the time range to the last hour.

You can check from the command line too. gcloud logging read 'logName:"cloudaudit.googleapis.com%2Fdata_access" AND protoPayload.serviceName="storage.googleapis.com"' --limit=5 --format=json prints recent entries. Look at protoPayload.authenticationInfo.principalEmail, protoPayload.methodName and protoPayload.status. A successful read has a method like storage.objects.get.

If nothing appears, check three things. The service may not support the log type you enabled. The operation may not be covered by that type. An exclusion may be dropping it. Also confirm you are looking at the right project and time range.

Mistakes to avoid

Turning on Data Access for every service and every principal. BigQuery and Cloud Storage can produce a lot of entries, and the useful signal disappears. Start with the data stores that matter, use exclusions for known noise, and add services after you see the volume.

Relying on the default retention. Data Access logs in the _Default bucket expire after 30 days. Many investigations start later than that. Create a sink to a Cloud Storage bucket with a retention policy or to BigQuery, and restrict who can delete from the destination.

Leaving log access wide open or too narrow. Audit logs contain principal emails, resource names and IP addresses. Give Logs Viewer to the security and operations people who need it, not to every project member. At the same time, do not lock the logs so tightly that the on-call engineer cannot read them during an incident.

Forgetting alerting and ownership. A log nobody reads is storage, not monitoring. Set alerts for IAM changes, audit config changes, bucket permission changes and access from unexpected networks. Assign an owner for each alert. You can build that in-house, hire a consultant, or use a fixed scope review. Cloudverg's Cloud security review is $6,500 at a fixed price and checks logging, alerting and backup against CIS Benchmarks as one option.

Frequently asked questions