Spot VMs can cut GPU compute costs by 60-91%, but eviction handling decides whether savings stick. Use this 2026 AWS, GCP, and Azure cheat sheet. Read now.
What Spot GPU Capacity Actually Buys You
Spot VMs let you run GPU workloads on spare cloud capacity at a steep discount. The headline range is large—often cited as roughly 60–91% off equivalent on-demand GPU compute—but the real number depends on region, GPU class, and how long you can tolerate interruption. Spot is a price-for-interruptibility trade: you pay less because the provider can reclaim the machine when demand rises.
That trade works well for training jobs that checkpoint, batch inference that can restart, and experiments you can pause. It fails when a job assumes exclusive, uninterrupted access to a GPU for hours without a recovery path. Serverless wrappers around spot capacity can hide provisioning, but they do not remove the eviction risk. The bill only stays low if your design survives reclaim events without re-running expensive work from scratch.
Eviction Handling Decides Whether Savings Stick
A spot discount is temporary revenue for the cloud; an unhandled eviction is permanent waste for you. When a node is reclaimed mid-epoch, unsaved gradients, half-written checkpoints, or orphaned GPU memory can wipe out days of discounted runtime. Savings “stick” only when interruption is a first-class event in the pipeline, not an ops surprise.
Treat eviction as an expected signal, not an exception. Capture progress on a durable store outside the spot instance. Make every unit of work idempotent so a retry does not double-bill or corrupt outputs. Prefer short, restartable steps over long monolithic runs. Monitor reclaim notices where the platform offers them, and fail over to a smaller on-demand or reserved slice only for the final stretch if a deadline is hard. Without those patterns, spot becomes a lottery ticket that sometimes costs more than on-demand after retries and human cleanup.
AWS, GCP, and Azure: Shared Cheat Sheet
Across AWS, GCP, and Azure the product names differ, but the cost mechanics are the same: spare GPU capacity at a discount, reclaim under pressure, and optional capacity types that trade price for stability. Map your workload to three levers—price, interruptibility, and placement—then pick the control that matches each stage of the job rather than forcing one SKU for everything.
- Price: Use spot or equivalent spare capacity for the bulk of training or batch inference where restarts are cheap.
- Interruptibility: Wire platform reclaim notices, health probes, and job queues so work drains cleanly before the GPU disappears.
- Placement: Spread across zones and GPU families so a single pool drain does not stop the whole pipeline; fall back to on-demand only for latency-sensitive or deadline-bound slices.
Serverless GPU products on these clouds often sit on top of the same capacity markets. They simplify scale-up and scale-to-zero, but you still own checkpoint strategy, timeout budgets, and whether a cold start after eviction is acceptable. Read the interruption model for the specific offering you use; do not assume “serverless” means “never interrupted.”
A Practical Path to Reliable Spot Savings
Start by classifying each GPU job: restart-safe, partially restart-safe, or never-interrupt. Put only the first two on spot. Add checkpoint frequency that matches your cost of lost work—more frequent checkpoints cost storage and I/O, but they cap how much discounted time you can lose. Keep a small on-demand or committed buffer for production inference and final evaluation so user-facing latency does not depend on spare capacity.
Instrument cost and interruption together: track GPU-hours billed, reclaim rate, and wasted compute from incomplete jobs. If reclaims climb and wasted hours dominate, tighten checkpointing, shorten tasks, or move fragile stages off spot. Cloud cost management for GPUs in 2026 is less about chasing the deepest list price and more about making discounted capacity survivable. When eviction handling is solid, the 60–91% range becomes a planning band you can actually realize; when it is not, the discount is fiction on the invoice.