Control logging and monitoring costs on Azure

Why this matters

Azure Monitor is easy to turn on and hard to see. A diagnostic setting takes a minute. A workspace keeps collecting after the incident is over. The bill shows up later, split across meters that finance may not map to a team. Logs usually drive the spend, not metrics. Every verbose app, chatty container and debug line adds volume.

The cost shape is simple. You pay for data sent to a Log Analytics workspace, for how long it stays queryable, and for extras such as export, alert rules and some features. A single noisy table can dominate. Container Insights and Application Insights are common sources because they collect by default and can multiply with replicas.

You need controls before an incident. During a failure, nobody wants to debate retention. Put guardrails in place, then give teams a clear path to collect more when they need it. The goal is not the lowest bill. It is enough telemetry to run the service, with no surprise line item.

Before you start

Get the current picture. In Azure Cost Management, open cost analysis, filter by the subscription or resource group, and group by service name or meter. Look for Azure Monitor, Log Analytics and Application Insights. Export a month of daily costs so you can compare later.

List every Log Analytics workspace and the resources that send to it. Include subscription-level diagnostic settings, not only resource-level ones. Check which tables are largest and which teams own them. You need names, not guesses.

Confirm access. You need Monitoring Contributor or Log Analytics Contributor to change diagnostic settings and workspace settings. Cost Management Reader is enough to inspect spend. If you use Azure Policy to enforce diagnostic settings, check the policy assignments first, or your changes may be reverted.

Agree on a review window and a rollback plan. Pick one workspace or one noisy table for the first pass. Decide who signs off on dropping log categories. Keep a copy of the current settings, either as a template or an export, before you edit.

How to do it

Work from volume down. Find the top tables, fix the biggest source, then move to retention and alerts. Do not start by cutting retention everywhere. That hides the problem and creates gaps in investigations.

Use table plans and data collection rules as the main controls. Diagnostic settings decide what leaves a resource. The workspace decides what you keep and how you can query it. Both matter.

Treat every change as a test. Change one setting, watch ingestion for a day, then check that alerts, workbooks and dashboards still work. A cheaper table that breaks an alert is a bad trade.

1. Map every diagnostic setting. Use the Azure portal or Azure Resource Graph to list diagnostic settings at subscription and resource scope. Note the workspace, the categories selected, and the resources with no setting. Look for duplicate paths where an agent and a diagnostic setting send the same records.
2. Rank tables by ingestion. In the workspace, query the Usage table for Quantity by DataType over the last 30 days. Sort by volume, not row count. Pick two or three tables that account for most of the data. Confirm the owning team and what queries or alerts depend on each table.
3. Set the right table plan. Keep Analytics for tables used by alerts and fast investigations. Move high-volume, rarely queried tables to Basic Logs if the query limits fit. Use Archive for older data you must keep but seldom read. Check that each plan supports the queries and alerts you rely on before you switch.
4. Drop noise with DCR transforms. Create or edit a data collection rule and add a transform that filters rows you do not need, such as debug or verbose events. Test the transform on a small scope first. Do not filter audit or security events without the owner's approval.
5. Tune Application Insights sampling. Enable adaptive sampling for high-volume applications and set a maximum sampling rate for predictable spend. Use fixed-rate sampling when you need a known percentage. Check that error and dependency views still show enough detail after sampling.
6. Use Container Insights presets. Switch Container Insights to a lower-cost collection preset where it fits. Remove performance counters and stdout logs you never query. If you need custom metrics, collect only the ones used by alerts or capacity planning.
7. Set retention and a daily cap. Set interactive retention per table to match how far back your team actually searches. Put older data in Archive or export only if a real process needs it. Set a workspace daily cap as a guardrail and alert well before it trips.
8. Clean up orphaned settings and alerts. Remove diagnostic settings that point to deleted resources or unused workspaces. Delete duplicate alert rules and unused action groups. Check features with separate meters, such as data export and managed Prometheus, and turn off what nobody uses.
9. Add budget alerts. Create an Azure budget for the subscription or resource group and filter to monitoring services. Send alerts to the owning team, not only finance. Review the budget after each change so the threshold matches the new baseline.

How to check it worked

Compare like with like. In Cost Management, view daily cost for Azure Monitor before and after the change. Give it a full billing cycle. A drop in one day can be noise from a quiet weekend.

Check ingestion by table in the workspace. Query the Usage table and compare Quantity for the tables you changed. If a table went to Basic Logs, run the queries your team uses and confirm they still return results within the limits.

Test the alarms. Trigger a test alert for each critical rule, or replay a known event. Confirm the action group fires and the on-call person gets it. If you added sampling, check that error counts and failed request rates still line up with what the service reports.

Look for blind spots. Review the workspace Operation table for daily cap events and failed ingestion. Ask each owner to confirm the remaining logs are enough for incident response. Put a monthly review on the calendar, because new resources and new log categories will creep back.

Mistakes to avoid

Turning on all log categories. Many resources offer an allLogs category. It is convenient and expensive. Start with audit and the categories an investigation actually uses. Add more when a real question needs them.

Leaving two collection paths. An agent, a diagnostic setting and a container insight can all send similar records. Duplicate data is hard to spot by table name. Check the resource ID and the collection path, not just the total volume.

Using the daily cap as a budget. The cap stops ingestion when it is reached. That can silence the logs you need during an incident. Use budgets and alerts to warn you, and keep the cap high enough that it only catches runaway growth.

Forgetting the long tail. Data export, archive restore, alert rules and managed Prometheus can add cost after the log bill is under control. Review every monitoring feature with its own meter. Then assign an owner who checks the bill each month. If you want an outside review, Cloudverg's Cloud cost audit at $4,500/month covers up to three subscriptions and includes a ranked savings report.

Frequently asked questions