MiMo-V2.6-Pro
1T-A42BXiaomi MiMo
Xiaomi's open-weight flagship, near the top of the open-model rankings this autumn. It has about 1.02 trillion parameters, roughly 42B of them active per token, and ships under the MIT licence. The released weights are already 4-bit (MXFP4 experts), and the GGUF builds repack them rather than quantize again. At that level it needs about 604 GB at 4K context, more than any single machine in this index. The largest Mac Studio gives its GPU about 384 GB, and four 80 GB H100s hold 320 GB. Eight 80 GB cards (640 GB) are about the smallest server that holds it, and even that is tight on this site's rule. It is listed so the calculator can say so. Only 10 of its 70 layers keep a growing cache, so even 1M tokens adds only about 50 GB. llama.cpp runs it as `mimo2`, and vision loads through a separate encoder. Sizes come from a published GGUF file; no per-level quality loss has been published.
1049K
Max Context
1
Quant Variants
GGUF MXFP4
Best Quality
100.0%
Accuracy Retained
Quantization Variants
Per-quant VRAM, quality loss, and inference speed on RTX 4090
Measured = run on this site · Estimated = not run here: calculated, or a figure this site has not verified · Community = public reports
Similar models
Compare with Mistral LargeMistral Large 3 675B Instruct
Mistral AI
Mistral 3 flagship MoE (41B active / 675B total) with vision encoder. FP8 on 8×H200; GGUF quant for research clusters only. Its attention is DeepSeek-style MLA, so the real KV cache is smaller than the figure below, which uses an unverified conventional layout. At 32K that is a rounding error beside ~390 GB of weights; at the full 256K window, treat the cache figure as an upper bound.
Muse Spark Open 816B-A42B
Meta Muse
Meta's largest open-weight model, released in October under the Muse Model License. It is an 816B mixture-of-experts model with about 42B parameters active per token, built for coding and agent work, with image and audio input. It is a single-machine model only at its smallest build. At Q2_K (ggml-org's 304 GB file) it needs about 319 GB at 4K context and still only 327 GB at 128K, because just 16 of its 62 layers keep a growing cache — comfortable on a 512 GB Mac Studio and tight across four 80 GB H100s. The full 1M window brings it to about 389 GB. Q4_K_S (470 GB) and Q8_0 (872 GB) fit no single machine in this index. The vision and audio encoders ship as separate files of 2–4 GB and are not included. Sizes come from the published GGUF files; Meta's benchmark claims are its own and no per-level quality loss has been published. The licence is not a standard open-source one: Meta's repository is gated, so downloads need an accepted access request; products built on it must show "Built with Muse Spark Open"; and a company with over $20M in annual revenue must sign a separate agreement with Meta before offering it as an API service.
Nemotron 3.5 Bolt 74B-A9B
NVIDIA Nemotron
NVIDIA's Nemotron 3.5 Bolt is a 74B mixture-of-experts model with about 9B parameters active per token, built to fit a 24 GB card. It is compressed from Nemotron 3.5 Super VL and ships in one build, NVIDIA's own native 2-bit file of 22.0 GB. Only 8 of its 88 layers keep a growing cache, so context costs little: about 23.1 GB at 4K, 24.1 GB at 128K and 25.2 GB at its full 256K window. That is the 24 GB NVIDIA designed for, but over this site's comfortable line, so a 24 GB card reads as tight; a 32 GB card or a 48 GB Mac runs it comfortably. The image encoder is a separate 1.7 GB file. NVIDIA's benchmark claims are its own, and no quality loss for the 2-bit build has been published.
Nemotron 3.5 Super VL 120B-A12B
NVIDIA Nemotron
NVIDIA's Nemotron 3.5 Super VL is a 120B mixture-of-experts vision-language model with about 12B parameters active per token. It is built for coding, tool use and document work, with a context window of up to 1M tokens. Only 8 of its 88 layers keep a growing cache, so even the full window is cheap: at Q4_0 (ggml-org's 71.8 GB file) it needs about 75 GB at 4K and 84 GB at 1M. That runs comfortably on a 128 GB Mac or a DGX Spark 128G, and tightly on one 80 GB card. Q8_0 (128 GB) wants a 256 GB Mac. The vision encoder is a separate 1–1.7 GB file. Sizes come from the published GGUF files; NVIDIA's benchmark claims are its own and no per-level quality loss has been published.
Running MiMo-V2.6-Pro locally
At MXFP4 and 4K of context, MiMo-V2.6-Pro needs about 603.8 GB — 548.5 GB of weights, 0.42 GB of KV cache and a 54.9 GB activation buffer. MXFP4 is the format MiMo-V2.6-Pro was released in, so it is the reference here rather than a community requantization. No card in this index clears that comfortably at MXFP4, so at that level it is a multi-GPU or CPU-offload proposition. These are calculated figures, not measurements: the estimate stops counting a card as comfortable at 88% of its VRAM, which is roughly the room a desktop session needs.
What longer context costs
Going from 4K to 32K adds about 1.36 GB, taking the total to 605.3 GB. That is gentler than the parameter count suggests: only 10 of 70 layers keep a KV cache that grows with context, and 60 more cap theirs at a sliding window, so the cache scales at a fraction of the usual rate. The model's native window is 1M; holding all of it at MXFP4 would need about 659 GB.
Which build to download
This index only tracks MiMo-V2.6-Pro in GGUF, across 1 level (MXFP4). No throughput has been measured for this model on this site's hardware. GGUF runs on llama.cpp and Ollama across NVIDIA, AMD and Apple silicon; AWQ and GPTQ target vLLM on CUDA and ROCm; EXL2 is ExLlamaV2 — CUDA only, and now archived, so local use rather than serving.
Common questions
- How much VRAM does MiMo-V2.6-Pro need?
- About 603.8 GB at MXFP4 with 4K of context and batch 1, rising to roughly 605.3 GB at 32K. That figure is weights plus KV cache plus a 10% activation buffer, calculated from the model's architecture rather than measured on a card.
- Can MiMo-V2.6-Pro run on a single GPU?
- Not comfortably on any single card in this index at MXFP4. It needs about 603.8 GB, which means splitting across GPUs or offloading layers to system RAM.
- Which quantization of MiMo-V2.6-Pro should I use?
- MXFP4 is the only level in the index for this model, so there is nothing to choose between here. Note this is a mixture-of-experts model — all parameters must be resident even though only a fraction are active per token, so the memory cost follows the total, not the active count.
Where this model fits
Sized at MXFP4 with a 4K context window, smallest card first. Comfortable means the estimate uses at most 88% of the memory.
- Ships in
- GGUF
- Try it yourself
- Size it in the calculatorGenerate the run command
How to actually run this
Deployment guides for this model and this class of hardware.
This page's figures change when the model or the runtime does.Last updated 2026-10-20 RSS → /feed.xml