Local AI: the best open-weight LLMs for writing in English

The short answer

If you want one recommendation rather than a methodology, this is it:

And if that variant is too large for your hardware, the practical answer changes with your memory budget rather than with your taste:

Everything below explains how those numbers were produced, and where they stop being reliable.

How to read a row

The unit of this ranking is not "a model". It is a variant: model or checkpoint, plus quantization, plus reasoning mode.

That distinction matters more than it sounds. The same model family, shipped at two different quantizations, can behave differently in long-form prose. The same weights with reasoning enabled and disabled can produce noticeably different text. Averaging those together would produce a tidier table and a less honest one, so we never merge them. If you see two rows that look like the same model, look at the quantization tag and the reasoning marker — they are different things to download and different things to run.

Two markers appear throughout:

• 💡 — the variant was run with its reasoning mode active.
• ↯ — the variant was run with reasoning disabled.

There is also an unknown state, and we use it deliberately. Asking a runtime for high reasoning effort is a request, not a guarantee. When a runtime accepts the parameter but reports zero reasoning tokens, we cannot honestly claim the model reasoned, and we cannot honestly claim it didn't. Those rows stay marked unknown until the evidence is unambiguous. You can filter them in or out.

Why the scores include cloud opponents

The scores shown here are the scores from the general English leaderboard, not a fresh tournament run among local models only. We filter and re-number; we do not re-score.

This is a deliberate choice with a clear trade-off. Keeping the original scores means every local variant sits on the same scale as every other model we test, including large cloud systems, and it means each variant keeps its full match history rather than discarding the majority of its comparisons. You get a common yardstick and more data behind each number. The cost is that a rank of, say, third here means third among eligible local variants, while the underlying score was earned in a wider field. The rank column is local; the score column is global.

What counts as "local" here

Four conditions, applied strictly:

1. Open weights. The model files must be publicly downloadable and runnable outside the vendor's infrastructure.
2. Genuinely runnable offline. Not a gated API with an open-sounding name.
3. Under 256 billion total parameters. Strictly below the threshold.
4. Already evaluated on this site. No estimates, no placeholder entries in the ranking itself.

The parameter limit applies to total parameters, including for mixture-of-experts models. This is worth stating plainly because it is a common and consequential error: an MoE model that activates a small fraction of its parameters per token still has to have all of its weights in memory. Active parameters affect speed, not the size of the thing you need to load. Where the active count is known, we show it separately, in its own column, so it can inform your speed expectations without being mistaken for a memory figure.

One thing the criteria do not require: that we generated the model's responses on a machine in this office. What makes a model eligible is that the weights are available and the model can be run locally — not the historical location of the hardware that produced its texts. Where a variant's throughput was measured on our own hardware, the speed figure is labelled as an AIWB measurement. Figures taken from elsewhere are labelled as external or estimated. We would rather show a labelled estimate than a confident invention.

Open source is not the same as open weights

"Open source" is the phrase people search for, so it is worth untangling. Open source is a licensing claim broader than merely publishing weights; how that framework applies to all model artefacts is still contested. Most models in this ranking publish their weights under licences that range from permissive to distinctly restrictive — commercial-use limits, acceptable-use clauses, redistribution conditions.

We rank on writing quality. The licence is a separate question, and an important one if you are shipping a product rather than drafting an email. Check it before you build on a model; it has no bearing on how well the model writes.

Eligible, but not yet tested

Models appear here only once they have been evaluated. Anything that meets the criteria but hasn't been through the corpus yet sits in a short waiting list, without a score and without a rank, so you can see what's coming rather than wonder whether we've noticed it.

Quantization, without the marketing

Quantization stores a model's weights using fewer bits per value. A model trained in 16-bit precision can be repackaged at 8 bits, or 4, shrinking the download and, more importantly, the memory footprint. On the kind of inference most people run at home — one user, short batches, bottlenecked by how fast memory can be read rather than by raw compute — a smaller file often runs faster too, because there is simply less data to move per token.

It also changes the model. Sometimes barely; sometimes visibly.

What we can say with confidence is that the size of the effect is not a constant. It varies with the model, the quantization method, the task, the language, the context length and the level of quantization. This is why you will not find a "Q4 keeps 98% of quality" claim on this page. That number does not exist as a general fact, and repeating it is how people end up disappointed by a specific model on a specific task.

The tags you'll see in the filter:

• Q8 — an eight-bit variant. We treat it as the practical high-quality reference point in this ranking: it is the version we reach for when memory allows. That is a working convention, not a claim that Q8 is mathematically equivalent to BF16 or FP16. Whether the remaining gap is measurable in English prose is an empirical question, and we prefer to answer it with paired tests rather than assertion.
• Q4 QAT — quantization-aware training. The model was prepared or trained with four-bit execution in mind, which makes it a distinct checkpoint rather than a post-hoc compression of an existing one. Selected by default alongside Q8, because it is what most people with limited memory should actually be looking at.
• Q4_K_M — a four-bit GGUF quantization applied after training, with mixed precision across different parts of the model. This is not QAT, and conflating the two is a genuine source of confusion. Available in the filter, unticked by default.
• Q6 — a middle ground that will appear in the filter automatically as soon as we have evaluated variants to put in it. It isn't hidden; there is simply nothing there yet.

Filters apply as you change them, and the state is carried in the URL, so a filtered view can be linked or bookmarked.

What our own data says about it

Any claim we make about quantization "in our tests" comes from paired comparisons only: same model, same reasoning mode, same language, different quantization. Nothing else is a fair test.

Where the pairs exist and the sample counts are adequate, the comparison stands on its own. Where they don't, we say the result is provisional or simply not yet measurable. A ranking that only ever produces confident conclusions isn't measuring anything.

One deliberate omission: we do not cite MMLU, GPQA, HumanEval, AIME or coding and maths benchmarks as evidence about prose. They measure other things competently. They do not tell you whether a model can hold a register across 800 words of English, and using them as a proxy is the single most common failure in this category.

Will it fit? Memory, honestly

The most common local-AI mistake is comparing a file size to a VRAM number and concluding that a model fits.

The weights are the floor, not the total. You also need room for the runtime itself, for the KV cache — the growing store of intermediate state that the model keeps for every token in the conversation, which scales with your context length — and for the context you actually intend to use. Batch size, model architecture and any vision component all move the number. A model that loads comfortably at 4K context can fail at 32K on the same card.

It's also worth keeping four kinds of memory distinct, because vendors and forums routinely blur them:

• Dedicated VRAM on a discrete GPU: the fastest, and usually the smallest.
• System RAM: plentiful, far slower to feed a GPU.
• Unified memory, where CPU and GPU share one pool: large capacities become reachable, but bandwidth is not the same as dedicated VRAM bandwidth, and treating the two as interchangeable will produce bad predictions.
• CPU offload, where part of the model sits in system RAM: this can make an otherwise impossible model load, at a speed penalty that is often severe. "It runs" and "it's usable" are different claims.

Fitting is necessary. It is not sufficient. Leave yourself a real margin.

Dedicated GPU

If you have a discrete graphics card, your budget is its VRAM, and the useful question is which variant gives the best English writing quality while still leaving headroom for your context.

Each tier gives the best-writing variant that fits with realistic headroom, plus a faster alternative for when responsiveness matters more than the last increment of quality. Speeds are shown where we have measured them; otherwise the speed field stays empty.

Unified memory

Apple Silicon, AMD's Strix Halo / Ryzen AI Max+ platforms and NVIDIA's DGX Spark all share memory between CPU and GPU rather than giving the GPU a separate pool. In practice this changes what's possible: capacities that no consumer graphics card offers become available to a model, which is why these machines can host variants that a discrete GPU of similar price cannot.

The trade is bandwidth. Feeding a model from unified memory is generally slower than feeding it from dedicated VRAM, and the gap shows up directly in tokens per second. A large model that fits is still a large model being read slowly. We keep the specific capacities, tiers and recommendations in the panel below rather than in this text, because vendor configurations change and a page that hard-codes them ages badly.

Our own throughput measurements on Strix Halo hardware are marked as AIWB measurements. Numbers for machines we don't have are marked as external sources or estimates, and we would rather leave a cell empty than fill it with a plausible guess.

Is running an LLM locally actually difficult?

Less than it was, and less than the folklore suggests — but the honest answer has two halves.

Installing a runtime is now genuinely easy. Tools like Ollama will download a model and give you a working chat in a few commands, and llama.cpp gives you finer control if you want it. Nothing here requires compiling anything or understanding CUDA.

The part that still takes judgement is everything this page is about: choosing which variant to download, deciding how much context you need, and knowing whether your memory can hold both. That's where most people's first attempt goes wrong — they pull the largest model that appears to fit, run it near the edge of memory, and conclude that local AI is slow or unstable. It isn't; the configuration was.

At a high level, the path is: install a runtime, pick a variant from the ranking that fits your memory tier with headroom to spare, run it, then set your context length to what you actually need rather than the maximum the model advertises. If it's too slow, step down a quantization level before you step down a model. If it's fitting only barely, step down anyway.

Why bother running it locally at all

The cloud models at the top of our general leaderboard are strong. Local models are chosen for reasons that have nothing to do with beating them.

When the runtime and its integrations stay local, your text need not leave the machine — which is the whole argument for anyone drafting confidential, legal, medical or unpublished material. It works with no connection, on a plane or a bad hotel connection. It doesn't change underneath you: a checkpoint you downloaded behaves the same next month, where a cloud endpoint can be updated, deprecated or retired without warning. There's no per-request cost and no rate limit. And you can inspect, fine-tune and integrate it on your own terms.

Set against that: you are trading some quality for that control, and you're providing the hardware. Whether the trade is worth it is exactly the question this ranking exists to inform.

Quality against speed

These are separate axes, and we keep them separate on purpose. The ranking column measures how well a variant writes English. The speed figures measure how fast it produces tokens on specific hardware. Nothing in this page combines them into a single "best" score, because the right balance depends on what you're doing.

Drafting a long article, where you'll read and edit the output carefully, rewards quality. Interactive back-and-forth — rephrasing, brainstorming, dialogue — rewards responsiveness, and a slightly weaker model that answers in two seconds may serve you better than a stronger one that takes thirty. Both variants are listed in every hardware tier for that reason.

What this ranking does not tell you

It measures English writing quality, judged through blind pairwise comparisons on a native English corpus. That is a real and specific thing, and it is not a general measure of intelligence.

A model that ranks well here may be mediocre at code, weak at maths, or unreliable at long-context retrieval. Our German, Spanish and French rankings are built on separate native corpora and produce genuinely different orders — a model's English performance does not predict its French performance, and we don't ask it to. Scores carry uncertainty that shrinks as comparison volumes grow; a small gap between adjacent rows with modest sample counts is not a meaningful difference, and the volume column is there so you can judge that for yourself.

You can read the actual texts and inspect the head-to-head comparisons behind any row. That is the part we'd most encourage: the scores are a summary, and the writing is the evidence.

FAQ

What is the best local AI for writing in English? The live summary above names the current top-ranked variant with its score and sample count. Bear in mind that the top variant is only the best answer if it fits your memory with room for your context; the per-tier recommendations above are more useful for most readers.

What does "local LLM" mean? An LLM is a large language model — the type of AI that generates text. A local LLM is one you download and run on your own computer, offline, rather than calling a service over the internet.

Why is there a 256B parameter limit? The threshold keeps this page centred on models that remain plausible on personal or compact workstation hardware. It applies to total parameters, including for mixture-of-experts models, because all the weights must be in memory even when only a fraction is active per token. Larger systems are outside this page's scope rather than impossible in absolute terms.

Does quantization ruin quality? Not inherently, and not by a fixed amount. The effect depends on the model, the method, the task and the language. We show quantizations as separate ranked rows so you can see the difference where we've measured it, instead of trusting a general percentage.

What's the difference between Q4 QAT and Q4_K_M? Q4 QAT is a distinct checkpoint prepared or trained for four-bit execution. Q4_K_M is a four-bit GGUF quantization applied to an existing model after training, using mixed precision across its layers. They are not interchangeable and we never merge them into one row.

How much VRAM do I need? More than the model file. Budget for the weights plus the runtime plus the KV cache, which grows with your context length. The dedicated GPU tab gives recommendations for common VRAM sizes with realistic headroom built in.

Is unified memory as good as VRAM? For capacity, often better — unified-memory machines can hold models no consumer graphics card can. For bandwidth, generally not, and that difference shows up as slower generation. Don't treat the two as equivalent.

Are open-weight models open source? Not automatically. Publishing weights does not by itself establish an open-source licence, and model licences vary considerably in what they permit commercially. The licence is independent of writing quality — check it separately.

Why do these scores include comparisons against cloud models? Because it keeps every variant on one scale and preserves its full match history. The rank is local; the score comes from the general English leaderboard.

How often does this page update? Continuously. Rankings, filters and hardware recommendations are generated from live data, and new variants enter the ranking as soon as they've been evaluated on the English corpus.