JetBrains released Mellum2, a 12B-parameter open Mixture-of-Experts model for text and code workflows that activates only 2.5B parameters per token.
What Mellum2 is designed to do
JetBrains Mellum2 is a 12B-parameter open Mixture-of-Experts model built for text and code workflows. MoE means the full parameter count is not active on every token: the model routes each step through a subset of experts, and Mellum2 activates only 2.5B parameters per token. That split is the core product idea—capacity for rich code and language patterns without paying the full 12B compute cost on every keystroke or completion.
For IDE-style use, latency matters as much as raw quality. Autocomplete, inline edits, and short refactor suggestions feel useful only when they return quickly enough to stay in the flow. A design that keeps active compute near 2.5B parameters per token is aimed at that constraint: enough model to handle multi-file context and programming idioms, while staying light enough for frequent, interactive calls.
Why MoE helps low-latency code work
Dense models of similar total size run all weights for every token. That is simple to serve but expensive when the UI fires many small requests. MoE separates “how much the model can know” from “how much it runs right now.” The 12B total budget can specialize experts for different languages, styles, or task shapes; the 2.5B active path is what the GPU or CPU actually evaluates on each step.
In practice, that favors interactive loops over one-shot batch jobs. You get more headroom to run the model locally or on modest remote instances, or to leave more memory for project indexes and editor state. Throughput for long offline generation still depends on your stack, but the per-token active size is a better match for chat-in-the-editor and continuous suggestion than a fully dense model of the same headline size.
Where it fits in a real workflow
Treat Mellum2 as a code-aware assistant layer, not a replacement for tests or review. Strong fits include completing APIs you already use, explaining a function in place, proposing small patches, and drafting boilerplate that you will edit. Weaker fits include unsupervised large refactors, security-critical changes without human checks, and anything that needs live facts outside the model’s training and your prompt context.
- Keep prompts short and local: current file, signature, error message, and the few related symbols.
- Prefer many small completions over one huge “rewrite the module” ask when latency is the goal.
- Validate output with the same tools you already trust—compiler, linter, unit tests—before you commit.
Because the model is open, you can evaluate it on your own repos, languages, and privacy rules instead of sending every snippet to a closed API. That matters when code cannot leave the network, or when you want stable latency under your own load limits rather than a shared queue.
How to evaluate it without chasing hype
Judge Mellum2 on tasks that match low-latency code work: time-to-first-token and full response on typical editor prompts, quality on your primary languages, and behavior with incomplete or noisy context. Compare active parameter cost and memory footprint against denser alternatives you already run, not only against larger models that you would never host the same way.
If responses are fast but shallow, tighten the prompt and tool context rather than only increasing max tokens. If quality is fine but latency slips, reduce context window and concurrent requests first—MoE still has routing and expert load costs. Used that way, a 12B MoE that activates 2.5B parameters per token is a practical option for teams that want stronger code assistance without turning every suggestion into a heavy inference job.