Does every AI request really need a data center?
For most everyday tasks — no. On-device AI uses a fraction of the energy, keeps your data on your device, and works without a network connection. We're building the tools to make it practical.
Read the white papersThe gap is closing
Cloud AI has a meaningful head start in raw capability. That gap is narrowing fast. Today's on-device models handle a large and growing share of everyday tasks — drafting text, answering questions, summarizing documents — without touching a server.
Where precision matters most, on-device models can call reliable software libraries (numpy, scipy, and similar tools) for deterministic, verifiable results rather than probabilistic guesses from a distant data center.
When the cloud goes down, your AI shouldn't
Cloud AI is a single point of failure. One provider outage and every dependent user, app, and business stops cold — simultaneously. On-device AI has no such dependency. No network required. No provider uptime to monitor. No rate limits, surprise model changes, or cascading failures from a data center under pressure.
This isn't a hypothetical risk. In the twelve months ending mid-2026, ChatGPT, Claude, and Gemini each experienced significant outages — including a week in May 2026 when all three reported disruptions within the same five-day window. Third-party tracking shows AI-platform disruption days rising as adoption scales, with resolution times frequently measured in hours.
The structural argument is simple: centralization concentrates risk. On-device AI distributes it back to zero. Even a hybrid design — cloud for heavy tasks, local as fallback — gives graceful degradation instead of total failure when a provider goes dark.
On-device AI also changes what happens to your data. With cloud AI, everything you type is sent to a remote server — which may include health records, legal documents, financial details, or a child's schoolwork. Once it leaves the device, you are trusting a vendor's policies on logging, retention, and training use. With on-device AI, nothing is transmitted. It's privacy you can verify rather than trust — a property of the architecture, not a promise buried in a terms-of-service. For doctor's offices, law firms, schools, and anyone handling data subject to GDPR, HIPAA, or US state privacy laws, that distinction carries real legal weight.
Greener, more reliable, and more private — three arguments, one architecture. Less energy consumed. Less water used. Still working when the internet isn't. And your data never left the device.
What we're doing
EcoInference.ai is a research and engineering effort to make on-device AI practical at consumer scale. We build tools, publish findings, and make the case — with evidence — that reducing AI's environmental footprint doesn't require sacrificing usefulness. Four white papers are published so far: The Case for Greener AI examines the energy and water cost of cloud inference; AI Data Center Overbuild asks how much of the projected buildout is actually necessary — and who is paying for it; The Price of a Throwaway Video measures the outsized energy cost of AI video generation and where most of that cost is wasted; and The Regeneration Tax asks how many times people re-roll AI outputs before keeping one — a hidden multiplier no vendor has ever disclosed.
The work is led by Mark J. Divitt, a software engineer with 40+ years of experience in large-scale application development, software architecture, and infrastructure, now focused on what happens when that infrastructure is no longer necessary. More about Mark →
The tools are open source
The argument on this page is only worth as much as the software behind it, so that software is now public. EcoInference is a set of three applications — for iPhone, for Android, and for the desktop — that run a language model directly on the machine in front of you. Ask a question and the answer is produced by hardware you already own. Nothing is transmitted, and it works with the network switched off. Where a small local model genuinely struggles, it can hand the question to a cloud model instead, but only where you can see that happening and overrule it.
The code is published under the MIT license, and you can find it here on GitHub: three independent native clients, an on-device tool-calling system that lets the model run real computations rather than guess at them, and the routing logic that decides what stays local. Anyone may take it, change it, rename it, and build a commercial product on it, owing nothing in return.
It is published as a finished thing to fork rather than a project to join — pull requests are not accepted. Reviewing them properly takes time this effort does not have, and saying so plainly is fairer than letting contributions sit unanswered. What ships alongside the code matters more than the license: roughly thirteen thousand words of engineering notes, including ten debugging case studies written up with the wrong turns left in, performance measured on real hardware rather than estimated, and a candid list of what remains unfinished. For anyone else building in this space, that record is probably worth more than the source.
One caveat, stated plainly because it is easy to miss: the MIT license covers our code, not the models it runs. Gemma's weights carry Google's own terms, which are not an open-source license and include restrictions on use.
Interested in collaborating?
We're looking for partners working on sustainable AI, edge inference, and related tooling. If that's you, reach out at info@ecoinference.ai.