How-to guide

Control logging and monitoring costs on Google Cloud

Learn how to cut Cloud Logging and Cloud Monitoring spend on Google Cloud using exclusion filters, retention limits, and metric controls.

By the Cloudverg engineering team · · 6 min read

Why this matters

Logging and monitoring bills grow without anyone deciding to spend more. A service ships, its client library logs every request at info level, and nothing breaks. The bill moves a little. A few months later logging is a real line item and nobody remembers approving it.

Google Cloud bills this in two parts. Ingestion is charged per volume above the free monthly allotment for each project. Storage is charged when you keep logs longer than the retention on a log bucket. Monitoring sits alongside it: metric ingestion above its own free allotment, uptime check executions, and custom metrics, which include log-based metrics. Each unit is small, which is exactly why the charge hides.

The answer is not to log less. It is to decide what you keep, for how long, and who owns that decision. Most teams find a few sources carrying the bulk of the volume: health check probes, debug lines in chatty services, GKE system logs, VPC Flow Logs, and data access audit logs. Fix those and the rest is upkeep.

Before you start

Get the numbers first. In Cloud Billing, open Reports, group by SKU, and look for the logging and monitoring lines. If you export billing data to BigQuery, query it instead so you can slice by project and by label. You want to know which projects produce the volume, not just the total.

Then map your log buckets. The Logs Storage page lists each bucket with its retention and region. The _Required bucket holds audit logs and cannot be excluded. The _Default bucket holds whatever the _Default sink routes to it, at the retention you set. Any bucket you created yourself carries its own retention and its own storage charge.

Write down what you must keep before you touch a filter. Audit and security requirements come first, then debugging needs for the past few days, then everything else. Talk to the people who handle incidents, because they are the ones who will miss a log you dropped at two in the morning.

Check what runs on the hosts as well. The Ops Agent on Compute Engine, the logging components on GKE clusters, and any third party forwarder all decide what gets sent before the log router sees it.

How to do it

Work from source to sink. A log line that never leaves the VM costs nothing to ingest, store or index, so cut at the workload or the agent first. Then use the log router for what is left. Exclusion filters on a sink are the main tool, and they cost nothing to run.

Exclusion filters use the same filter language as the Logs Explorer. You can match on severity, log name, resource labels or text inside the payload. Build the filter in the Logs Explorer first and watch the volume estimate as you type, then attach it to the sink once you are sure it matches what you think it matches.

Monitoring has separate controls. Metrics management lets you stop ingesting metric types that nobody charts. Uptime checks bill per execution, so trimming locations and stretching the period lowers cost without losing the signal. Log-based metrics bill as custom metrics, and that catches people out: a metric built on a busy log line can cost more than the logs themselves.

  1. Find the loudest log sources. Use the logs ingestion view in the console, or group the billing export by log name, to see which sources dominate. Write the top few down before you change anything.
  2. Turn down log levels at the source. Change application config so debug and trace lines are off in production and can be switched on per request. A logger that writes on every request in a hot loop is the usual culprit.
  3. Exclude health check and probe traffic. Load balancer health checks, readiness probes and uptime checks write steady logs with no debugging value. Match them on user agent or log name in an exclusion filter on the _Default sink, and give the filter a name that explains the reason.
  4. Set retention per bucket. Shorten retention on _Default to what debugging actually needs, then route the few logs that must live longer into their own bucket with a sink. Storage cost follows retention, so this is the second biggest lever after ingestion.
  5. Trim GKE logging components. When you create or update a cluster you can choose system logs, workload logs, both, or neither. Switch logging off for clusters you only use for batch work, and exclude the kube-system chatter you never read.
  6. Sample platform logs. VPC Flow Logs, firewall rules logging and Cloud NAT logging all take a sample rate or an errors-only setting. Lower the rate on noisy subnets and keep full detail where you actually investigate.
  7. Review data access audit logs. Admin Activity audit logs are free to ingest. Data Access logs are the ones that cost, and you enable them per service in the IAM audit configuration, so turn on only what security asks for.
  8. Trim monitoring. Add metric exclusion rules for metric types nobody looks at, and cut the locations on uptime checks. Delete alerting policies and custom dashboards that reference nothing live.

How to check it worked

Give it a week before you judge anything. In Cloud Billing Reports, filter to the logging and monitoring SKUs and compare the daily trend with the week before. Ingestion drops first. Storage follows more slowly, because it reflects what has already piled up.

Then check that you did not break incident response. Run the queries from your on-call runbook in Logs Explorer and confirm they still return entries. Fire a test alert and confirm the notification arrives. Open the dashboards that use log-based metrics and check that they still draw.

Put a guardrail in place. A Cloud Billing budget with alerts on the logging project catches a jump before month end, and an alerting policy on log volume catches the service that starts logging in a loop. Review the filters once a quarter, since new services arrive with new habits.

Mistakes to avoid

Filters that are too wide. A substring match on a common word catches the failures you needed. Test the filter in Logs Explorer over a window that includes an incident, and look at what disappears, not only at what stays.

Switching audit logs off to save money. Admin Activity audit logs are not charged for ingestion and often answer the question you get asked later. Data Access logs are where the cost sits, and those you scope per service in the IAM audit configuration.

Assuming exclusions fix the metric bill. Log-based metrics are billed as custom metrics, so a metric whose filter matches a busy log line keeps costing whatever you do with the sink. Go through the metrics list and delete the ones nobody queries.

Cleanups with no owner. A one-off pass helps for a quarter, then a new service ships with debug logging on. Name an owner for the filters and the retention settings, and put the review on the calendar. If you want an outside pass on the whole bill, the Cloud cost audit from Cloudverg is $4,500/month at a fixed scope, and it is one option among doing the work in house or hiring a contractor.

Frequently asked questions

Are exclusion filters free in Cloud Logging?

Yes. An exclusion filter on a sink costs nothing to run and removes the matched entries before they reach a log bucket, so both ingestion and storage go down. Logs that other sinks still route are billed as before.

Do I get charged for Admin Activity audit logs?

Admin Activity and System Event audit logs are not charged for ingestion while they stay in the _Required bucket. Data Access audit logs are charged, and they are usually the expensive part of the audit picture, so enable them only for the services you need.

Why did monitoring cost rise after I added a log-based metric?

Log-based metrics are billed as custom metrics in Cloud Monitoring, so a metric on a high volume log line adds cost even if no dashboard charts it. Narrow the metric filter or delete the metric if nobody queries it.

Monthly plan

Want help with this? Cloud cost audit

$4,500/month

2 weeks · Starts within 10 business days

A two-week, read-only review of your AWS, Azure, Google Cloud or Oracle bill, with every saving ranked by dollars per month.

Renews monthly. Cancel any time.
What is included

Related guides

More cloud costs guides   Cloud cost audit

Smallest way to start

Get a senior architect on your problem this week

Ninety minutes a month on the cloud, AI or security question that is blocking you, a pre-read of your material and a written summary within two business days.

$450/month

Renews monthly. Cancel any time.