As public sector organizations transition from artificial intelligence pilots to full enterprise rollouts, the core conversation is pivoting. Instead of focusing merely on which model to deploy, leadership is grappling with the exact cost of each answer, whether those expenses can be reliably forecasted a full fiscal year ahead, and the strategic tipping point for owning infrastructure rather than renting it.

September marked a definitive shift in how the federal government acquires AI capabilities. Throughout the previous year, agencies accessed frontier AI models at nominal rates. Under the General Services Administration’s OneGov initiative, tools like ChatGPT Enterprise, Claude, and Gemini for Government were accessible for a dollar or less. Those promotional windows are now transitioning to standard commercial frameworks, highlighted by GSA's 27-month agreement with OpenAI. This new structure relies on a distinct economic model: a 50% discount on token-based consumption, devoid of platform-access fees, minimum order requirements, or mandatory spending commitments, allowing agencies to pay strictly for utilization.

While this arrangement offers undeniable flexibility, it introduces a fundamentally different fiscal dynamic. A flat-rate subscription functions as a neat budget line item, whereas a consumption meter demands active forecasting. Because this updated agreement extends its reach to state, local, and tribal governments, the financial implications stretch far beyond federal offices.

Public sector artificial intelligence has officially transitioned into a variable expense dictated by tokens—the discrete word fragments processed by AI models. Crucially, the vast majority of government agencies currently lack the internal instrumentation necessary to forecast these expenditures accurately.

The Trap: Falling Token Prices, Rising Bills

Conventional wisdom suggests that sinking token prices should simplify technology budgets. However, that logic only tells half the story. Speaking at Dell Technologies World 2026, Dell Vice Chairman and COO Jeff Clarke revealed that while unit token costs had plummeted by roughly 80% year-over-year, token consumption dedicated strictly to reasoning had surged by an astonishing 320 times.

In essence, plunging unit prices are being thoroughly eclipsed by exploding consumption volumes.

Corporate giants experienced this firsthand when Uber CTO Praveen Neppalli Naga revealed that aggressive internal deployment of AI coding assistants had completely exhausted the company's annual AI allocation, forcing an immediate budgetary reset. Given the blistering pace of AI adoption, it is hardly surprising that a 2026 FinOps Foundation survey found 98% of respondents tasked with managing AI expenses, up dramatically from just 31% two years prior.

While a multinational corporation like Uber can absorb unexpected budgetary overruns, government bodies possess no such financial cushion. Operating strictly under legislative appropriations, federal overspending constitutes a legal violation rather than a mere fiscal variance. At the state or county level, identical overruns manifest as disruptive supplemental budget requests, postponed initiatives, or public services that grind to a halt mid-year.

This reality underscores why tokenomics—the underlying economics governing the production and consumption of AI tokens—cannot be deferred until after system deployment. Within government administration, strict cost predictability remains an absolute prerequisite for deployment.

According to NASCIO’s 2026 State CIO Top Ten Priorities, artificial intelligence has claimed the number one ranking for the very first time. Concurrently, budgetary control and fiscal management sit firmly at number three, forcing state technology leaders to successfully deliver AI solutions while simultaneously unearthing sweeping operational savings.

In government, cost predictability is a condition of deployment.

Four Decisions That Move the Number

When moving from a localized pilot to an enterprise-wide deployment, four strategic choices dictate the ultimate financial outcome, with model branding playing a surprisingly minor role:

  • Model size, measured correctly: Headline parameter metrics frequently deceive. True operational cost is determined by the specific volume of parameters computed per token. Mixture-of-experts architectures can boast a trillion parameters while actively engaging only a tiny fraction per token. The pragmatic approach involves deploying a dual-model strategy: a highly capable model reserved for complex queries alongside a compact, rapid model handling summaries, classifications, and basic searches. Rerouting routine tasks away from expensive flagship models represents the most reliable efficiency gain available.
  • Reasoning complexity: Advanced reasoning models expend significant tokens deliberating before generating output, mirroring the surge observed by Clarke. Agentic systems compound this dynamic by chaining prompts, invoking external tools, and executing self-corrections until satisfactory results emerge. Because complex agentic workflows rarely conclude within a single inference cycle, raw cost-per-token metrics become obsolete. The only metric that truly matters is the cost per completed outcome.
  • Data access patterns: This variable remains heavily influenced by agency decisions yet rarely audited. Retrieval mechanisms—which query internal records to ground AI responses—drive utility but quickly consume finite working memory. Because memory capacity expands alongside retrieved text volume, quadrupling retrieved data cuts the server's simultaneous request capacity by roughly 75%. Agencies that optimize retrieval protocols prior to hardware procurement ultimately purchase substantially less infrastructure.
  • Deployment location: Metered cloud inference aligns well with variable, experimental demand characteristic of early pilots. Steady-state workloads, which comprise the vast majority of government operations, demand a different approach, making infrastructure placement the most high-leverage decision an agency can make.

While the first three decisions minimize the computational requirements of individual queries, the fourth decision fundamentally alters how those computations are financed.

Operating on a cloud meter means expenditures scale linearly with consumption. Conversely, investing in owned infrastructure establishes a largely fixed cost baseline where every incremental token costs little more than the electricity required to power the hardware. Public utilities have successfully structured operations around this exact economic divide for a century: maintain owned capacity for base loads while procuring peak demand externally.

At a national level, sovereign AI implies trusted domestic capabilities designed to safeguard sensitive data while reflecting local institutional standards. For state and county agencies, the practical definition centers entirely on custody. Retaining ownership of the physical hardware, selecting preferred models, managing lifecycle transitions, and storing every prompt and response within local logs under strict retention schedules provides unmatched operational control. Certain public records introduce strict regulatory mandates; for instance, federal tax information may only enter cloud environments via FedRAMP-authorized frameworks backed by mandatory advance notifications to the IRS.

This focus on custody is viable only if internally managed models match required capability thresholds—a performance gap that has narrowed dramatically. Independent research group Epoch AI reported that since January 2026, leading open-weight models—freely downloadable and operable in private environments—have lagged behind elite closed-source alternatives by an average of just four months. Furthermore, Stanford’s 2026 AI Index pegs the performance advantage of the top closed model over its premier open counterpart at a modest 3.3%.

Government procurement officers will inevitably note regional caveats. By mid-2026, top-tier open models closest to the technological frontier originated primarily from Chinese developers, prompting multiple state administrations to restrict their usage on official devices. U.S.-developed open alternatives from NVIDIA, Google, OpenAI, and specialized laboratories operate just a step behind. Crucially, NVIDIA has published the underlying training datasets for its largest Nemotron models, allowing agencies to audit their software stack completely. For typical government use cases—such as drafting documents, summarizing records, searching archives, and classifying text—the primary test is not whether a model achieves global supremacy, but whether it is sufficiently capable. A model lagging the frontier by a few months simply represents the frontier of yesterday.

Furthermore, the physical infrastructure required is frequently far more modest than anticipated. Dell's internal support assistant launched successfully within a single on-premises server rack, requiring no bleeding-edge GPUs, specialized power upgrades, or advanced cooling systems. Comprehensive analysis by Signal65 and Futurum, commissioned by Dell and NVIDIA, demonstrated that executing agentic workloads on enterprise workstations and servers can reduce costs by up to 87% compared to commercial cloud APIs over a two-year period. However, utilization remains the definitive qualifier: an idle GPU generates zero utility tokens while continuing to depreciate, meaning owned capacity outperforms cloud meters only when workloads maintain steady operational volume.

Consequently, the ideal strategy avoids binary choices. Flexible commercial terms lacking minimum spend requirements accommodate unpredictable variables like complex reasoning tasks, exploratory experiments, and seasonal usage spikes. Meanwhile, owned hardware infrastructure perfectly absorbs predictable base workloads, including employee productivity assistants, continuous document searching, and strictly regulated data operations destined to remain on-premises.

A model a few months behind the frontier is simply the frontier of a few months ago.

Building the Number Into the Plan

Four distinct practices separate defensible AI budget line items from unpleasant financial surprises:

  • Size based on concurrency rather than headcount: Total infrastructure capacity should be provisioned according to simultaneous active requests rather than total employee login counts. Because the vast majority of any user session involves reading and typing, only a small fraction of personnel actively await model outputs at any given millisecond.
  • Transform pilots into budget forecasting tools: Instrument proofs-of-concept from day one to track peak concurrent requests, queue depth, working-memory utilization, time-to-first-response, and average prompt lengths. These five metrics convert preliminary pilots into defensible appropriation requests while clearly identifying true base-load workloads.
  • Establish hard ceilings on metered consumption: Embracing arrangements with no minimum commitments offers valuable buyer flexibility but introduces profound exposure to public appropriations. Integrating workflow-specific spending caps, team-level quotas, and automated consumption alerts during the initial design phase prevents costly postmortem discoveries.
  • Embed record placement within established policy workflows: Numerous state governments currently maintain formal AI use-case inventories. Incorporating simple tracking fields specifying execution locations and anticipated consumption profiles requires minimal effort while ensuring cost visibility occurs during initial project approval rather than at fiscal year-end.

Cost Per Outcome, Not Cost Per Token

Reflecting on operational shifts, Uber's leadership noted that expenses continued to decline alongside growing adoption because the company approached efficiency as an engineering challenge rather than a strict accounting constraint.

Governments that successfully scale artificial intelligence will not be those that merely hunted for the cheapest unit token prices. Instead, victors will be organizations that calculated—prior to scaling—the exact token volume required to achieve specific outcomes, alongside the safest and most cost-effective production environment: cloud meters, owned infrastructure, or a strategic hybrid of both. Within the public sector, this rigorous discipline does far more than safeguard financial allocations; it honors a long-term service commitment to citizens that agencies can sustain across entire fiscal cycles.

Resources such as Dell Technologies and NVIDIA's joint executive guide, Overcoming Pilot Fatigue: A Roadmap for Scaling AI in State and Local Government, provide deep insights into resolving pilot bottlenecks. Additional whitepapers like Sovereign AI on Your Terms explore maintaining absolute governance over critical data assets, hardware infrastructure, and foundational models, while NVIDIA's specialized Tokenomics Guide offers robust forecasting tools for estimating computational demand.