Local AI tools: run AI on your own computer
Everything you type into a cloud AI assistant leaves your computer. For most questions that is fine. For client contracts, medical notes, legal drafts or unpublished financials, it is a problem you should not have to think about twice.
Local AI means running a model on your own machine. No subscription, no upload, no per-token bill. The trade-off is honest and specific: you need the hardware, the answers are weaker than a frontier model, and you maintain it yourself.
| Tool | Cost | Runs on | Best for |
|---|---|---|---|
| Ollama | Free, open source | Mac, Windows, Linux | Command-line and API access to open models |
| LM Studio | Free for personal use | Mac, Windows, Linux | A normal desktop app with a model browser |
| Jan | Free, open source | Mac, Windows, Linux | A chat interface that looks like a cloud assistant |
| GPT4All | Free | Windows, Mac, Linux | Older machines without a dedicated GPU |
| Whisper | Free, open source | Any modern machine | Transcribing audio without uploading it |
What hardware you actually need
The model size is what decides everything. As a rough guide:
- 8GB of RAM: small models (7 to 8 billion parameters, quantised). Good enough for summarising, rewriting and simple questions. Slow but usable.
- 16GB of RAM: comfortable for everyday small models, and you can stretch to larger ones if you accept waiting.
- 32GB of RAM, or a GPU with 8 to 16GB of video memory: larger models that start to feel close to a mid-tier cloud assistant.
- A modern Mac with unified memory is the easiest way in, because the memory is shared between processor and graphics, which is exactly what these models want.
Where local AI is genuinely worse
Do not expect a small local model to plan a complex project, write working code across several files, or reason through a legal question the way the paid frontier models do. It will be noticeably weaker at long context, multi-step reasoning and current information, and it will not know anything that happened after it was trained.
Where it is genuinely better: privacy, offline use on a plane or in a basement office, bulk processing you would not want to pay per token for, and never being rate limited.
The honest use cases
- Private documents. Drafts under NDA, client files, anything with personal data in it.
- Transcription. Whisper locally means meeting recordings never leave the laptop, and there is no per-minute charge.
- Bulk, boring jobs. Rewriting 400 product descriptions overnight costs nothing but electricity.
- Learning. Experimenting with prompts and models without watching a usage meter.
A sane way to start
- Install one app, not four. LM Studio or Jan if you want a normal window, Ollama if you want a command line and an API.
- Download one small model and use it for a week on real work.
- Keep the cloud subscription for the hard 20% of tasks and use the local model for the routine 80%.
- Revisit in six months. Open models improve quickly, and your hardware does not get slower.
What we would pay for
Nothing. The software is free. The real cost is the machine you already own, plus an hour of setup. Pair a local model for private and bulk work with one $20 cloud assistant for the tasks that actually need a frontier model, and you have both privacy and capability.
One note: AI pricing moves fast. We read these numbers off the vendors' own pricing pages in October 2026, so check the current price before you buy.
Where to check the current price
These are the official pages we read while writing this guide. Plain links, no tracking, no middlemen. If a number here disagrees with what you see, trust their page and tell us so we can fix it.
Frequently asked questions
Is local AI actually free?
The software is. Ollama, Jan, GPT4All and Whisper are open source with no subscription. You pay with your own hardware, electricity and setup time, and you miss the capability of the paid frontier models.
How much RAM do I need to run a local model?
Around 8GB runs small quantised models for basic writing and summarising. 16GB is comfortable. For larger models that feel closer to a cloud assistant, plan on 32GB of memory or a GPU with 8 to 16GB of video memory.
Is local AI as good as ChatGPT?
No. A model that fits on a laptop is noticeably weaker at multi-step reasoning, long documents and current information. It is better at privacy, offline use and bulk processing, which is usually why people install one.
Can I use local AI for client work?
That is the main reason to use it. Nothing leaves your machine, so client files, recordings and drafts stay private. Check the licence of the specific model you download, since some restrict commercial use.
Which is easier, local AI or a cloud subscription?
The cloud subscription, by a wide margin. Local tools need a download, a model file, and some patience with speed. The payoff is privacy and no per-use cost, which is worth it only if those two things matter to you.