The Case for Greener AI

Understanding the Hidden Environmental Cost of Artificial Intelligence

1. Introduction

AI is having a moment. Hundreds of millions of people use AI tools daily — to draft email, answer questions, write code, summarize documents. For each of those interactions, a request leaves the user's device, travels to a data center, gets processed by machines consuming thousands of watts, and returns. It feels instant. It feels free. The costs are real; they're just somewhere else.

This paper examines where those costs land — specifically, energy and water — and makes the case that for a large and growing share of everyday AI use, routing inference through a remote data center is not necessary. On-device AI exists. It is improving rapidly. And using it, where appropriate, is a practical lever for reducing AI's environmental footprint without meaningful sacrifice of capability.

A note on scope: this paper addresses inference — the act of generating a response to a query. It does not cover training, which has its own substantial footprint. Training happens once (or infrequently); inference happens billions of times per day. That's where scale makes the aggregate numbers meaningful.

2. Where Does AI Actually Run?

Most consumer AI — ChatGPT, Gemini, Claude, Copilot — runs in cloud data centers operated by large technology companies. When you ask a question, your request leaves your device, travels to a server facility, gets processed, and the response travels back. The computation happens there, not here.

This is the current default for two reasons: cloud models are larger and more capable, and the infrastructure already exists. A smartphone's battery and processor cannot run a 70-billion-parameter frontier model. That is a real constraint.

But the constraint is not absolute, and it is weakening. Capable smaller models now run on consumer hardware — phones, laptops, edge devices — with reasonable response times and meaningful accuracy. For tasks that don't require frontier-scale reasoning, the cloud route may never have been necessary.

On-device AI is not hypothetical. It is in use today, in narrow domains. The question is how far that domain can expand — and how quickly.

3. The Energy Gap

Two large AI providers have published official per-query energy figures. These are the most reliable numbers available, and they are the ones used here.

• OpenAI: approximately 0.34 Wh per query
• Google Gemini: approximately 0.24 Wh per query

A single query is a thin slice of a session. A ten-minute working session involves many queries at varying complexity. Extending the per-query figures across a realistic session — and comparing them to measured on-device consumption — yields the following estimates. Session figures for cloud are derived from the per-query disclosures above; on-device figures are based on measured smartphone consumption during active local inference.

For context: a typical smartphone battery holds approximately 18.5 Wh (5,000 mAh). A complex ten-minute cloud AI session consumes roughly 38% of that battery's total capacity — in energy terms, even though the phone itself is not doing the computation.

4. The Water Nobody Talks About

Energy gets most of the attention. Water is less visible but equally real.

Data centers use water in two ways: directly, for cooling the servers themselves, and indirectly, because generating electricity consumes water at the power plant. Both matter.

Direct cooling water

• OpenAI: approximately 0.000085 gallons per query (roughly 1/15 of a teaspoon)
• Google: approximately 0.26 mL per query (roughly 5 drops)

Indirect water (power generation)

The US federal estimate for water withdrawal associated with electricity generation is approximately 1.19 gallons per kWh. This figure, applied to the session energy estimates above, yields the indirect water cost of each session.

On-device AI does not eliminate water use entirely — the electricity that charges your phone has a water footprint — but the orders-of-magnitude reduction is real. Most smartphone charging happens at the grid edge, and on-device inference draws only a fraction of the energy of a cloud session.

5. The Capability Trade-Off

The honest position: cloud AI is more capable, for now. Frontier models have more parameters, more training data, and more computational budget than anything running on a consumer device. For tasks requiring deep reasoning, broad world knowledge, or sophisticated multi-step planning, the cloud remains ahead.

The honest follow-up: that gap is narrowing fast, and for a large share of everyday tasks, it may not be the relevant comparison.

Drafting a routine email, summarizing a document you provide, answering a factual question, generating a first draft of code — these tasks do not require a frontier model. On-device models, running today on consumer hardware, handle them with reasonable accuracy.

For tasks where precision is critical and the domain is well-defined — financial calculations, unit conversions, statistical analysis — on-device models can call reliable software libraries (numpy, scipy, and equivalent tools) rather than attempting to compute the answer by inference. This hybrid approach produces deterministic, verifiable results. The model handles language and intent; the library handles the arithmetic. The result is often more trustworthy than a large cloud model reasoning through the same calculation.

The capability gap that remains is real. It should not be overstated. And it should be continuously re-evaluated, because the trajectory is one-directional.

6. Why It Matters at Scale

Individual sessions are small. Aggregate use is not.

Current estimates place global AI chatbot query volume at approximately 3.5 billion per day. At a weighted average of roughly 0.30 Wh per query, that works out to approximately 1.05 million kWh per day — equivalent to powering roughly 36,000 average US homes around the clock.

The associated water use — direct cooling plus indirect power-generation water — is approximately 1.5 million gallons per day. That is roughly equivalent to the daily water use of 5,000 US households.

The trajectory makes the floor meaningful. Global AI chatbot use approximately doubled between 2025 and 2026. IEEE Spectrum projects that all generative AI queries could approach 329 billion per day by 2030 — roughly 100 times today's volume. If the mix of cloud versus on-device inference stays constant, the aggregate footprint scales with it.

It does not have to stay constant.

7. A Path Forward

The goal is not to discourage AI use. The goal is to shift where appropriate inference happens — and to make that shift practical.

What you can do

• Use on-device options when they exist. Apple Intelligence, on-device Gemini Nano features, and similar capabilities process many tasks locally. Prefer them for routine work.
• Reserve cloud AI for tasks that need it. Frontier reasoning, synthesis across large bodies of knowledge, and highly complex generation are legitimate uses of cloud capacity. Quick lookups and routine drafting generally are not.
• Ask providers where your inference runs. Transparency pressure matters. Companies that publish their per-query energy figures (as OpenAI and Google have) deserve acknowledgment; those that don't should be asked.

People already working on this

EcoInference.ai is a research and engineering effort to make on-device AI practical for a mass consumer audience. The work includes tooling for efficient edge inference, methodology for comparing on-device and cloud approaches, and published research making the environmental case with evidence. If you're working on related problems — sustainable AI infrastructure, edge inference tooling, energy-aware model design — we'd like to hear from you: info@ecoinference.ai.

The infrastructure for on-device AI exists. The models are capable enough for a meaningful share of everyday tasks. The environmental argument is clear. What remains is making the default easier to change.

For a deeper look at the scale of data center buildout this shift could avoid — and who currently pays for the capacity that smarter design could make unnecessary — see the companion paper: AI Data Center Overbuild →