AI Data Center Overbuild

How Much AI Data Center Capacity Do We Actually Need — and Who Pays for the Rest?

1. Introduction

Across the United States, and increasingly across Europe and Asia, a building boom is underway on a scale rarely seen in peacetime. The buildings are data centers — vast, windowless halls of computers — and the announced numbers are staggering. The fourteen largest operators alone are on track to spend close to three-quarters of a trillion dollars in 2026. Industry analysts project the United States could add more than 75 gigawatts of new data center capacity by 2035, an investment on the order of four and a half trillion dollars. For scale, a single gigawatt is roughly the output of a large nuclear reactor.

This paper asks a simple question that is surprisingly hard to get a straight answer to: how much of this do we actually need? Not how much is being announced, or financed, or promised to investors — how much is genuinely required to deliver AI to the people and businesses who use it.

The question matters because the buildout is not free, and the bill is not landing where you might expect. As later sections show, much of the cost is already being passed to ordinary electricity customers, while the benefits and the public subsidies flow to a small number of technology firms. If even part of the projected demand turns out to be avoidable — met by better engineering rather than more concrete and steel — then the public is paying for infrastructure that smarter design could have made unnecessary.

Our argument is not that AI data centers are unnecessary. They plainly are necessary; training advanced models and serving complex requests requires enormous, centralized compute. Our argument is narrower and, we think, harder to dismiss: a large and growing share of AI's workload is lightweight enough to run on devices people already own, and shifting that share off the cloud would avoid a meaningful slice of the projected buildout — a slice measured in tens of gigawatts and hundreds of billions of dollars. That gap, between what is being committed and what is strictly necessary, is what we call the overbuild.

2. Two Kinds of AI Work

To see where the overbuild hides, you have to separate two very different jobs that data centers do for AI. They are usually lumped together, but they behave nothing alike.

Training

Training is how a model is created. It means running enormous clusters of specialized chips flat-out for weeks or months to build the model in the first place. Training is extraordinarily compute-intensive, it has to happen in large centralized facilities, and nothing about it can move onto your phone. This part of the buildout is genuinely required, and this paper does not contest it.

Inference

Inference is what happens every time someone actually uses a model — asks a question, requests a summary, generates an image. Each individual inference is far smaller than a training run, but inference happens billions of times a day, and that volume has made it by far the larger load. Measured across a model's working life, inference now accounts for an estimated 80 to 90 percent of all AI compute, because a model is trained once but then runs continuously for years. McKinsey projects global AI inference capacity rising from about 21 gigawatts in 2025 to more than 93 gigawatts by 2030 — growing faster than training, and pulling further ahead each year.

This distinction is the whole game. Training must stay in the data center for the foreseeable future. Inference — or at least a meaningful portion of it — does not. And inference is where the growth is.

The trend, and where it points

Measured by the work AI actually does, inference is not catching up to training — it has already overwhelmed it. Across a model's productive life, inference accounts for an estimated 80 to 90 percent of total compute, because a model is trained once but then runs continuously, answering queries every second for years. Industry analysts now put inference at roughly 85 percent of enterprise AI budgets in 2026. The rule of thumb is stark: for every dollar spent training a model, fifteen to twenty dollars are spent running it. Training is the one-time cost of building the engine; inference is the fuel burned every mile thereafter, and the car never stops driving.

Table 1: Training/inference split two ways. "Operational" share reflects total lifetime compute and budget (where inference dominates); "capacity" share reflects live data-center power. Both rise in absolute terms. Operational figures from Telnyx, Introl, ICONIQ; capacity and gigawatt figures from McKinsey. All figures rounded.

And the gap is widening fast. Goldman Sachs projects that total token consumption — the raw volume of inference work — will multiply twenty-four-fold between 2026 and 2030, reaching some 120 quadrillion tokens per month. The accelerant is the shift to agentic AI: a single task handed to an autonomous agent fans out into many model calls, consuming five to thirty times the tokens of a simple chatbot exchange. The cost collapse is what makes it possible — the price of a typical query fell more than 280-fold between late 2022 and late 2024, and continues to drop by roughly an order of magnitude a year — but cheap inference does not shrink total demand, it multiplies it.

One clarification matters for the math that follows. The 80-to-90-percent figure measures inference's share of total operational compute — the right lens for understanding where AI's work actually goes. But the question of how much we need to build is a question about live power capacity, where training's brief, intense runs keep its share higher. For that reason, when we model avoidable construction in the next sections, we deliberately use the more conservative capacity figure — McKinsey's roughly 93 gigawatts of projected inference capacity in 2030 — rather than the larger operational share. We would rather understate the avoidable buildout than overstate it. Even on that conservative basis, the numbers are large.

3. Requests Are Not Compute

It is tempting to say: if half of what people ask AI to do is simple, then half the inference load could move to their devices. That would be wrong, and the error matters enough that we want to confront it directly before building any estimate on top of it.

A request and a unit of compute are not the same thing. The requests that are easy to run locally — a quick factual question, a short rewrite, a simple summary — are precisely the ones that consume very little compute. The requests that dominate the load are the heavy ones: long documents, extended reasoning, image and video generation. And the gap between light and heavy is enormous. Peer-reviewed benchmarking finds a standard model answering a simple query uses roughly 0.3 watt-hours, while a reasoning model working through a complex prompt can use 33 to 40 watt-hours — a difference of seventy to a hundred times. Even under efficient, optimized conditions, a heavy reasoning query still costs on the order of thirteen times a light one.

So offloading half of all requests does not offload half of all compute. It offloads the cheap half. When you account for the heavy tail staying in the cloud, shifting 50% of inference requests to local devices removes something closer to 10 to 25 percent of inference compute, not 50 percent. We treat that smaller, honest figure as the real lever — and even it turns out to be large.

4. The Offload Scenario

Here is the scenario we will be working with. We assume, as a goal, that 50% of inference requests by volume could be served on-device rather than in the cloud. Two facts make this less speculative than it sounds. First, real usage data — OpenAI's own published taxonomy of how people use ChatGPT — shows that roughly half of all conversations fall into inherently lightweight categories: quick factual lookups, short edits, how-to questions, simple writing help. Second, this is not hypothetical: at its June 2026 developers conference, Apple rebuilt Siri on an explicit three-tier architecture that routes each request to on-device models, to its Private Cloud Compute, or to a frontier cloud model, depending on what the request actually needs. The architecture exists, ships on hundreds of millions of devices, and is the design this paper's scenario assumes.

Translating that 50% of requests into compute, and applying it to McKinsey's projected 93 gigawatts of inference demand in 2030, gives the following picture. We show three cases rather than one number, because the honest answer is a range, and because the conclusion holds across all of it.

Table 2: Avoidable cloud inference capacity under three offload scenarios, applied to projected 2030 inference demand (~93 GW). Capital at $38B/GW (Epoch AI). All figures are estimates; see appendices for the full model.

Read the table from the bottom up. Even in the conservative case — where only a tenth of inference compute shifts local — about 9 gigawatts of cloud capacity and roughly $350 billion of construction become unnecessary. The target case, which we treat as a likely probability given the status quo, avoids around 23 gigawatts and $885 billion. And in the optimistic case, where on-device AI matures to absorb nearly half of inference compute, roughly 40 gigawatts and $1.5 trillion of buildout could be avoided.

We lean toward the optimistic case as the more likely long-run outcome, for one reason: on-device AI is improving faster than almost anything else in technology. Small models are gaining capability every few months, AI accelerators are now standard in consumer chips, and the gap with the cloud is closing. We think a near future in which local inference handles the bulk of everyday requests is not just plausible, but likely. We hold that as a forward view, and we have shown the conservative floor precisely so the argument does not depend on it. Even the floor in this case is enormous.

5. Who Pays for the Buildout

So far this is an engineering argument. It becomes a public argument when you look at who is paying for the capacity being built, because the answer is increasingly: everyone, through their electricity bills, whether or not they use AI at all.

The mechanism is called cost-shifting. When a data center needs new power lines, substations, and generation, utilities often spread those costs across the entire customer base rather than billing the data center alone. The Union of Concerned Scientists found that ratepayers across seven states in the PJM grid region were put on the hook for $4.3 billion in electricity infrastructure approved in a single year solely to connect data centers. PJM's own market monitor calculated that removing data centers from the region's demand forecasts would cut capacity payments by roughly $9.3 billion — a 64% reduction — with that burden flowing to 65 million residential and commercial customers.

The effect shows up on individual bills. Electricity prices rose around 13% nationally in 2025, with data centers cited as a primary driver, at a time when an estimated 80 million American households already struggle to pay their utility bills. Researchers at Carnegie Mellon and NC State estimate data centers could raise the average electricity bill by 8% by 2030, and by more than 25% in concentrated markets such as northern Virginia — a state where data centers already consume nearly 40% of all electricity. These are not abstractions to the people receiving the bills: one Virginia homeowner saw his monthly electricity charge jump from about $100 to $281 in a single winter.

Meanwhile, the public subsidizes the construction itself. State tax breaks for data centers reached an estimated $2.5 billion in a single year in Georgia, $1.6 billion in Virginia, and $1 billion in Texas. One analysis in Ohio found state tax breaks working out to nearly $1 million for every job a data center created. A Harvard researcher studying the pattern described it plainly as a cost shift: in effect, the public is subsidizing someone else's AI.

In fairness, this is beginning to change. More than 300 bills across 30-plus states have been introduced to rein in the cost-shifting, and several major firms have pledged to cover their own power costs and forgo tax breaks. We weigh those responses honestly. But pledges and pending bills are not yet results, and the costs documented above are already on the bills.

6. The Opportunity Cost

Numbers in the hundreds of billions are hard to comprehend. So consider what the avoidable buildout represents in terms of public goods — not because the money would literally be redirected, but to make the scale easier to understand. The comparison below is an illustration of magnitude, nothing more: it asks what sums of this size look like when measured against things communities actually build.

A new public high school in the United States costs roughly $50 million on average. A mid-sized hospital costs around $112 million.

Table 3: The avoidable buildout expressed as an illustration of scale against public-infrastructure costs. This is a magnitude comparison, not a claim that funds would be redirected. School ~$50M, hospital ~$112M (national averages, 2025–2026). US stock: ~6,100 hospitals, ~24,000 public high schools.

The right-hand column is the one that stops you in your tracks. The United States has roughly 6,100 hospitals and about 24,000 public high schools in total. Even the target case represents more capital than it would take to rebuild every hospital in the country. The optimistic case exceeds what it would cost to replace every public high school in America. The point is to illustrate that the scale of capital being committed to compute is on the same order as the nation's entire stock of its most important public buildings. When a society commits resources at that scale, it is more than fair to ask whether all of it is necessary. These are not tinker toys being played with by billionaires — these are real resources and real lives being redirected to overbuild AI data center infrastructure. Those resources and lives could focus instead on the greater needs for the people and the planet.

7. The Honest Counterarguments

An argument like this is only as credible as its treatment of the objections. There are several serious ones, and we take them in turn.

"Cheaper inference just creates more demand" (the Jevons paradox)

The strongest objection is that making inference cheaper and more local will simply increase total AI use, wiping out the savings — the classic pattern in which efficiency breeds consumption. There is real evidence for this; data center power demand is growing around 15% a year partly for this reason. Our response: even if total AI use keeps climbing, moving inference onto devices people already own still shifts that load off the cloud and avoids the marginal data center that would otherwise have to be built. The opportunity-cost argument concerns committed capital and capacity, and it holds regardless of whether total usage grows. Jevons changes the size of the pie; it does not change the fact that a slice of it need not be baked in a data center.

"We'll need all of it anyway"

Deloitte argues directly that most computation will remain on expensive data center chips, and that we will need essentially all the facilities now planned. We don't dismiss this; for total compute, it may be right. But our claim is about the avoidable margin, not the whole. Even Deloitte concedes that some inference moves to the edge. We have tried to quantify that margin conservatively rather than assume the buildout vanishes — and a conservative margin still runs to hundreds of billions of dollars.

"You can't run real AI on a phone"

Two honest limits are real. Training cannot be offloaded at all, which is why we exclude it entirely from the pool. And on-device models still struggle with very long contexts and the largest multimodal tasks, which is exactly why our compute discount is so steep — why 50% of requests becomes only 10 to 25 percent of compute. The heavy work stays in the cloud in our model. We are claiming the light work can move to local inference, not that the data center disappears.

"Local models aren't good enough"

Today, for the hardest tasks, that's true. Our scenario is partly forward-looking, and we've labeled it that way. But the direction is not in serious dispute: small models are improving rapidly, and the hardware to run them is now standard in new phones and laptops. The companion paper to this one documents how quickly that gap is closing.

8. A Path Forward

If even part of the projected buildout is avoidable, the implication is not to halt data center construction — it is to stop treating ever-more centralized capacity as the only way to meet AI demand, and to put on-device inference on equal footing as a deliberate strategy.

Build less by computing smarter

Every inference request served locally is one that does not require new cloud capacity, new transmission, or new ratepayer-funded grid upgrades. On-device AI is not only greener and more private; it is a way to avoid building infrastructure we would otherwise pay for twice — once in construction, once on our bills.

Make those who benefit pay the full cost

The cost-shifting documented here is a policy choice, not a law of nature. Requiring data centers to fund the grid capacity they trigger — as a wave of new state legislation now attempts — would both protect ratepayers and force a more honest accounting of which capacity is truly worth building.

Count avoided demand as infrastructure

Efficiency and edge computing should be weighed in capacity planning the way conservation is weighed against new power plants. A gigawatt of inference that never needs a data center is as real as a gigawatt that gets built — and far cheaper.

This is the work that EcoInference.ai (and many others) want to advance: making capable, low-resource AI run on the devices people already own, so that a large share of the world's inference compute load never has to touch a data center at all. We are early in this process, and this needs to be part of an even larger effort to improve the efficiency of all types of compute everywhere. But the direction is clear, and the stakes — measured in gigawatts, dollars, and the paths not taken that follow — are now large enough that the public deserves clear and accurate answers, not corporate platitudes from billionaires.

How much AI data center capacity do we actually need? Less than is being built. And who is paying way more than their share for this — in money, lost opportunities, and limited resources? You and I...