Stop adding more GPUs: Weka's new storage platform reduces load by caching 100% of AI model's pre-calculated…

**Weka** has introduced a storage platform aimed at production AI inference that claims to cut GPU load by caching **100%** of a model’s pre-calculated tokens. The pitch is direct: stop treating more GPUs as the default fix when memory and recomputation are the bottleneck. Long context windows and multi-turn conversations already force models to reprocess information they have seen, and that work burns the most expensive resource in the stack.

The product mechanic is token-level caching off the GPU. Instead of keeping all prior-token state only in GPU memory, the platform stores those pre-calculated tokens on cheaper storage and serves them back so the model does not recompute them. That extends effective memory beyond what fits on the accelerator. The stated design choice is simple: treat GPU memory as something to protect and extend, not as the hard ceiling that must grow with every longer context or longer chat.

For engineers running multi-turn or long-context systems, that targets a real cost and capacity problem. GPU memory is both scarce and expensive, and it is the resource that runs out first under production load. Every recomputed token uses memory and compute that could have served another user or produced a new response. If caching can absorb that reuse work, the same GPU fleet can carry more concurrent sessions without a proportional hardware bump.

In market terms, this sits against the common response of buying more GPUs when context and traffic grow. Weka is arguing for a storage-backed path: use lower-cost capacity to hold pre-calculated tokens so GPU spend stays focused on new work. That competes less with a new model release and more with “scale the cluster” as the default ops answer when memory pressure shows up.

What to watch next is whether that **100%** token-cache claim holds under real multi-turn and long-context traffic, and how much GPU memory and compute it actually frees for additional users. Builders should measure recompute share on their own workloads first, then test whether off-GPU token caching reduces load enough to delay another GPU buy—not assume the savings until they show up in latency, throughput, and cost per session.