H100 prices span roughly $3.78 to $5.49 per GPU-hour across Lambda, Modal, and Replicate, with different idle-billing tradeoffs. Full breakdown.
What the H100 range actually tells you
Across Lambda, Modal, and Replicate, H100 capacity sits in a narrow band of roughly $3.78 to $5.49 per GPU-hour. That spread is real, but the sticker price alone is a weak decision input. Serverless GPU billing is rarely pure “pay only while tokens flow.” Providers differ in how they charge for cold starts, warm retention, minimum billed increments, and any period the instance sits allocated but idle. A cheaper headline rate can lose to a slightly higher rate if your workload leaves the GPU warm between bursts or if you are billed in coarse time slices.
Treat the matrix as three columns of unit price plus a fourth column you fill in yourself: expected utilization. Utilization is the fraction of billed GPU time that is doing useful work. If you only compare per-hour rates, you optimize for continuous training-style jobs and misprice chatty inference, batch pipelines with gaps, and interactive notebooks.
Idle billing is the real differentiator
Idle billing is the policy that decides whether you pay when the GPU is reserved but not computing. Some setups scale to zero aggressively and charge almost nothing between requests; others keep capacity warm for lower latency and bill for that warmth. The same H100 hour can therefore cost very different amounts depending on whether your traffic is steady, spiky, or long-tailed.
Map your traffic shape before you pick a provider. Continuous or near-continuous jobs care most about the lowest GPU-hour rate. Sparse traffic cares most about scale-to-zero behavior and how long a warm instance lingers after the last request. Medium traffic often sits in the middle: you want fast restarts without paying for full idle hours. When two providers quote similar H100 rates, the idle and warm-pool rules usually decide the monthly bill.
- Steady training or long batch jobs: prioritize lowest effective $/GPU-hour and simple, transparent increments.
- Spiky inference: prioritize fast cold starts, short or zero idle charges, and fine-grained billing.
- Latency-sensitive products: budget explicitly for warm capacity; do not assume scale-to-zero is free of product cost.
How to use a pricing matrix without fooling yourself
Build a small spreadsheet with four inputs: peak concurrent H100s, average busy minutes per hour, warm/idle minutes you are willing to fund, and request volume. Convert each provider’s H100 rate in the $3.78–$5.49 band into an estimated monthly cost under those assumptions. Then stress-test the model: cut busy time in half, double concurrent GPUs, and add a warm floor. The ranking often flips when idle policy changes even if unit prices barely move.
Also separate “list price” from “billable shape.” Minimum charge windows, rounding up to the next second or minute, and multi-GPU packing rules all change effective cost. Two providers can both advertise an H100 near the middle of the range and still diverge on short jobs because one bills a full minute for a five-second call while the other tracks closer to actual runtime.
Practical selection checklist
Start with workload class, not brand preference among Lambda, Modal, and Replicate. If the job runs for hours with high occupancy, take the lower end of the H100 range and verify multi-GPU and storage I/O are not hidden bottlenecks. If the job is request-driven, score cold-start latency and idle policy before you celebrate a lower $/hour number. If you need both low latency and cost control, design an explicit warm pool with a hard concurrency cap so idle spend is intentional, not accidental.
Revisit the matrix whenever traffic shape changes. A cheat sheet freezes rates and idle tradeoffs at a moment in time; your utilization does not. Re-run the same utilization model quarterly—or after any major product launch—so you are always choosing on effective cost, not on whichever H100 number looks cheapest in isolation.