Why this matters
Spot VMs on Azure sell spare capacity at a lower rate. Azure can take that capacity back when it needs it, and you get about 30 seconds of notice. That short window is the design constraint for everything else in this guide. If a workload can lose a machine in 30 seconds and still finish its work, spot is a good fit. If it cannot, the lower rate turns into incidents and reruns.
Good candidates look like batch rendering, video transcoding, CI build agents, log processing and stateless web tiers behind a load balancer. Bad candidates look like a single database instance, a domain controller, or a long job that writes its output at the end. The test is simple: kill the process without warning and see what you lose.
Azure exposes spot in four places. Standalone Spot VMs, Spot instances inside a Virtual Machine Scale Set, spot node pools in AKS and spot pools in Azure Batch. Each has its own knobs for eviction and replacement. VMSS and AKS give you the most control, and standalone VMs give you the least.
Before you start
Inventory the workload before you touch pricing. Write down what holds state, how long one unit of work takes and how a restart picks up. A job that runs for 40 minutes and checkpoints every 5 minutes is workable. A job that runs for 40 minutes and writes at the end is not.
Move state off the machine. Sessions go to Azure Cache for Redis or Azure SQL. Uploads go to Blob Storage. Work items go to Storage Queue or Service Bus. Keep only files you can rebuild on the local disk, and remember that an ephemeral OS disk disappears with the VM.
Check your spot quota. Spot vCPU quota is separate from regular quota and counted per VM family per region. Request an increase before you build, and pick two or three alternative sizes for when capacity is tight.
Decide the eviction policy and max price now. Delete removes the VM and its OS disk. Deallocate stops the VM and keeps the managed OS disk, which you keep paying for. Set the max price above the current spot rate, or use pay-as-you-go as the ceiling, so Azure evicts you for capacity rather than price.
How to do it
Build the eviction handler before the first spot VM. Azure posts a scheduled event to the Instance Metadata Service a short time before it reclaims an instance. A small process polls the link-local endpoint at 169.254.169.254 with the Metadata:true header and looks for EventType Preempt. On that event, stop taking new work and finish or checkpoint the current item. If the handler cannot finish in the window, you need smaller units of work.
Use a Virtual Machine Scale Set for anything that must stay available. A single Spot VM is one eviction away from an outage. With flexible orchestration you can mix spot and regular instances in one scale set, and Spot priority mix lets you set a base count of regular instances under the spot capacity. That base is your fallback during a regional capacity squeeze.
AKS needs a few extra settings. Add the spot taint to the node pool, tolerate it only on workloads that can restart, and give the same toleration to the CNI, CSI and log DaemonSets. Set terminationGracePeriodSeconds below the eviction window and add PodDisruptionBudgets so the scheduler keeps enough replicas running.
- Split state from compute. Move sessions, uploads and queue state to Azure SQL, Azure Cache for Redis, Blob Storage or Service Bus. Keep only rebuildable files on the instance. If the process cannot restart cleanly, fix that before you ask for spot capacity.
- Add a Scheduled Events handler. Poll the Instance Metadata Service every few seconds for scheduled events. On Preempt, stop taking work, checkpoint the current item and exit. Log the event with the instance ID.
- Build on a scale set, not a single VM. Use a Virtual Machine Scale Set with flexible orchestration so you can mix spot and regular instances. Set the eviction policy to Delete and the max price above the current spot rate. A standalone Spot VM is fine only for work you can lose whole.
- Keep a regular capacity floor. Set a base count of regular instances under the spot capacity so traffic has somewhere to go during a mass eviction. On AKS, add a small regular node pool and let the autoscaler add nodes when spot nodes disappear. Cap the regular pool so a bad day does not double your bill.
- Taint spot nodes in AKS. Add the spot taint to the node pool and tolerate it only on workloads that can restart. Set terminationGracePeriodSeconds to fit inside the 30 second window and add PodDisruptionBudgets. Give CNI, CSI and log DaemonSets the same toleration.
- Run an eviction drill under load. Trigger Simulate eviction on one instance while a load test runs. Confirm the handler fires, the checkpoint lands and a replacement instance picks up work. Record the drain time and the work lost.
- Alert on evictions and capacity. Create alerts for eviction count, scale set capacity below your floor and failed checkpoints. Send them to the team that owns the workload.
How to check it worked
Force an eviction during a load test. The portal and CLI offer Simulate eviction on spot instances in a scale set. Run one while traffic is flowing and watch the logs for your Scheduled Events handler. Confirm the checkpoint lands, the instance leaves the load balancer and a replacement picks up work.
Measure what you lose. Time the drain from the first event to the last write, and count the work items that had to restart. If the loss is higher than you planned, shorten the checkpoint interval or split the job into smaller pieces. The 30 second window does not move.
Check Azure Monitor and Cost Management. The scale set metrics show evictions, and AzureActivity records the events if you send activity logs to a Log Analytics workspace. In Cost Management, filter by pricing model or by your spot tag and confirm the instances are billed at the spot rate. Look for managed disks left behind by Deallocate and delete them.
Repeat the drill on a schedule. Spot capacity changes with regional demand, so a test that passed last quarter can fail today. Monthly is enough for most teams.
Mistakes to avoid
The worst mistake is using one Spot VM for something that has to stay up. Put two or more behind a load balancer, or keep a regular instance as failover. A single instance with no fallback is a scheduled outage with a random timer.
Setting the max price too low is the second. You evict yourself when the market rate passes your cap. Leave headroom or set pay-as-you-go as the ceiling, so Azure only takes the VM when it needs the capacity.
Do not ignore the 30 second limit. Long shutdown hooks, slow database connection drains and large file uploads overrun it. Cut work into small chunks and make writes idempotent so a repeated unit does no harm.
In AKS, check the DaemonSets. If the CNI or CSI agent cannot tolerate the spot taint, nodes come up without networking or storage and pods stay pending. Watch for orphaned disks after evictions and for autoscaler settings that replace spot nodes with regular ones without a cap.
If you want a second set of eyes on what spot really saves, the Cloud cost audit from Cloudverg is a fixed price review at $4,500 that runs for two weeks and ranks each finding by monthly cost. Azure Cost Management reports are the other option, and they work fine when your tags are clean.
Frequently asked questions
Can Azure evict a Spot VM even when my max price is high?
Yes. The max price is a cap on what you pay, not a reservation. Azure can reclaim the capacity when it needs it, and the 30 second notice still applies. Set the cap high to avoid price evictions, but plan for capacity evictions anyway.
What is the difference between Delete and Deallocate eviction policy?
Delete removes the VM and its OS disk, so compute and disk billing stop. Deallocate stops the VM but keeps the managed OS disk, and you keep paying for that disk. Use Deallocate only when the disk must survive the eviction.
Do Spot VMs come with an SLA?
No. Spot VMs carry no availability SLA. Pair them with regular capacity or another pricing model when the workload needs an uptime commitment.