Mellum2.1 GGUF is available: an 8.07GB file does not guarantee an 8GB GPU is enough

Are you downloading weights or a complete coding assistant?

JetBrains released Mellum2.1 Thinking as an open-weight model for coding agents that read code, edit files and use tools. The weights are the model itself; operating on a project also requires a model runtime or service and a client that executes tool requests.

The JetBrains announcement links to its official Hugging Face collection. That collection now contains the original weights and a GGUF repository whose file list shows downloadable builds. The announcement and original model card still say GGUF is coming soon, so use the current repository when choosing a download.

Why does a model with 2.5B active parameters have files larger than 8GB?

Mellum2.1 uses a mixture of experts, or MoE: each computation activates a subset of expert modules. It has about 12B total parameters and activates about 2.5B per token. Using fewer parameters for computation does not mean the remaining weights need no storage.

The official file list shows Q4_K_M at about 8.07GB, BF16 at about 24.3GB and MXFP4_MOE at about 7.03GB. GGUF is a model-file format read by local inference runtimes. Quantization reduces weight precision to shrink files and can also change outputs relative to the original model.

These are file sizes, not whole-machine memory requirements. Inference also needs cache and software overhead; longer contexts generally consume more memory. The listed 131,072-token context limit does not mean every device should be configured to use that full length.

Where should a personal computer or a team service start?

For a personal computer, read the official GGUF card’s llama.cpp or Ollama examples, then choose a file according to your operating system, runtime version and available memory. The card also names LM Studio. Listing a runtime does not establish that we have verified compatibility on every machine.

For a shared inference service, consult the original Thinking model card’s vLLM example. This is a different route from the GGUF instructions. Do not combine a file from one route with settings from the other without checking the runtime’s requirements.

Consider a hypothetical developer with an 8GB GPU who chooses to load the entire 8.07GB file into VRAM. The file size alone leaves cache and the actual loading method unaccounted for. The next useful step is to check the chosen runtime’s memory guidance, then validate a small task.

After chat works, what else does tool calling need?

In tool calling, the model requests an action, the client executes it, and the result returns to the model. The original model card configures the qwen3 reasoning parser for vLLM. For tool calling it also lists enable-auto-tool-choice and the hermes tool parser; the complete command is in the source.

First, check the model name, weight version and parser settings against the documentation for the service you intend to use. A running server or a single chat response does not establish that coding tools are connected.

Next, use an isolated example supported by your client to check that it recognizes the tool name and arguments and returns the execution result to the model. This separates a model’s proposed action from an action actually processed by the client.

Then review file-editing, command-execution and confirmation permissions for the real project. This is a suggested validation sequence; we have not run installation commands or performed tool tests.

How can you decide whether it belongs in your workflow?

JetBrains publishes its own coding evaluation results, but a benchmark cannot determine whether your project’s tasks will succeed. A more useful comparison holds a small maintenance task fixed and observes correct edits, tool calls, failure handling, total time and resource use.

The official model uses Apache 2.0; read the license file and its obligations. Open weights do not remove hardware, cloud-compute or operational costs. The current model pages also list no deployed Hugging Face inference provider, so this release is not a promise of a free hosted API.

Align model choice, runtime capacity and the client’s toolchain before expanding use. We have no measured conclusions about speed, coding quality after quantization or compatibility with a particular device. Keep your own small-task results as the basis for that decision.