Compute is the part of the stack most providers are actively solving for. In inference, the layer beneath it has just as much to say about how much a fleet can serve.

Ask a room of AI cloud operators what storage means to their business and the answers fall along a spectrum.

At one end it is a cost to be controlled — overhead sitting next to GPUs, judged on price per terabyte and kept as lean as the workload allows. At the other end it is an enabler — capacity they sell to tenants, certainly, but judged mainly by how productive it makes the accelerators it sits beneath.

Most providers sit somewhere in the middle, and where exactly usually depends on who is answering. But among the providers we work with at Dell, the ones building the most durable businesses tend to sit closer to the second end, and there is a practical reason for that. Treating storage as an enabler puts it in the same conversation as the GPUs it supports, which is where decisions about what a fleet can actually do get made.

Making GPUs productive was once a question of getting data to them quickly enough — a pipeline problem, largely settled during training. It now extends into inference itself, which is why a successful provider is no longer just measured by the number of GPUs in its fleet, but by how much work each of those completes in a day once real tenants are running on it — how many sessions it can hold at once, how quickly it responds when someone’s context runs long, how much of its time goes to producing output rather than reproducing work it had already done. That is inference efficiency.

Providers can’t forecast what tenants will run

Inference efficiency is harder to engineer for a provider than for almost anyone else, because the work arriving is never theirs to plan. An enterprise running its own models knows roughly what its sessions look like. A provider does not. That is the real design constraint for a service provider business: the fleet is bought and built before anyone knows the shape of the work it will carry.

Capacity planning helps, but no forecast tells a provider what a tenant will run next quarter. What is within their control is how well the infrastructure uses itself — whether each part of the system is doing the work it was built for, rather than absorbing work that belongs somewhere else. That is where storage shows up as an architectural decision.

When GPU memory is the only place context lives

The most expensive real estate in any AI system is the memory attached directly to the GPU. It is fast, it is physically close to the processor, and there is not much of it. While a model is actively generating a response, that is exactly where its context belongs — nothing else is close enough to keep the GPU fed token by token. The harder question is what happens to context once a session goes quiet but is not finished.

That context is the KV cache — the running record of what a model has already worked out in a session, kept close at hand so it does not have to be recalculated with every token. Keeping all of it in GPU memory was reasonable when sessions were short. It is harder to sustain now that tenants are running agents that carry document histories, tool outputs and chains of reasoning across long sessions, and send the same prompts and reference material through thousands of times a day. When GPU memory is the only place context can live, a provider has two options for any piece of it: keep it there, or throw it away.

Throwing it away is what usually happens. The memory fills, the system discards older context, and the model recomputes it when a session needs it again. That recomputation is work already completed once, performed a second time, on compute power that could have been serving someone else. Response times stretch under load, and token throughput suffers.

None of this is a failure of the GPUs, or of the memory attached to them. Both are doing what they were designed for. What is missing is somewhere for context to go when it is no longer in active use but will be needed again — somewhere slower than GPU memory, and far faster than working it out from scratch.

Giving context somewhere else to go

The correction is unglamorous, which is probably why it took a while to get attention. Keep active context in GPU memory, where it belongs, and give everything behind it a home on storage designed to hold it and return it at the speed inference requires.

Nothing is discarded, and nothing has to be recomputed. Context that is not in active use moves to infrastructure built to hold it, and the GPU gets its memory back for the work only it can do — which shows up as faster responses and more users served per GPU.

Shared storage, separate tenants

Moving context onto shared storage raises an obvious question for anyone running a multi-tenant fleet: shared between whom, and separated how. It is the right question, and it is the reason why choosing the right storage architecture matters.

Start with the first. Context in GPU memory lives and dies with one session on one node; context in storage persists beyond it, and can be picked up by whichever node serves that tenant next. A returning user, a resumed session, a document revisited across a working day — each starts from what is already there, and across thousands of sessions that is work the fleet never has to do twice. That is the sharing worth having, and it stops at the tenant.

Access zones are how Dell’s file storage holds that line: each is a partition of the cluster with its own authentication boundary, so separation is enforced rather than assumed. The number of zones a cluster supports is the ceiling on how many tenants can hold isolated context on it, and that ceiling is now 500. Tenant traffic can run over TLS, so nothing else on a shared network can read it in transit. Inside the zone, tenants now manage themselves: their own quotas, so one team’s long-running agents cannot exhaust the space another team is relying on, and their own snapshot schedules and restores, driven from Kubernetes rather than through a support request to the provider.

That is what makes offloading context practical at a service provider’s scale. Context persists for the tenant, stays within the tenant, and every GPU serving them gets its memory back.

Storage here is not standing in for compute or competing with it for budget. It is taking a job off the GPU’s plate so it can do more of what it is uniquely suited to. The two work alongside each other, and the performance of the whole system depends on both. Storage earns that position through integration — with the serving and routing software that decides what gets reused, the network fabric that carries it, and a direct path into GPU memory that skips the CPU entirely. It has to behave less like a repository than like an extension of memory, returning context faster than the GPU could recompute it. Storage that can do that is no longer just capacity sitting underneath the fleet. It is part of how inference runs.

All of it comes back to the two ends of that spectrum: whether storage is treated as a line beneath GPUs, or as part of what makes them productive. There is a version of the next few years in which providers compete mainly on how much accelerated compute they can secure. There is another, more likely in my view, in which they compete on how much service they can deliver from what they already have.

The second version rewards the storage as an enabler view: a fleet built this way absorbs more of what tenants throw at it before the experience degrades.

Dell builds AI storage not as the layer beneath compute, but as an enabler increasingly shaping inference efficiency. Read on to learn more about how the storage engines in the Dell AI Data Platform — PowerScale, ObjectScale and Lightning — move context out of GPU memory onto shared storage over GPU-direct paths, across file, object and parallel file, to lower query response times and improve token throughput. Register here for From Ambition to AI at Scale , where Dell and NVIDIA go deeper on the architecture behind this.