How much VRAM does DBRX Instruct need?

About 84.3 GB at Q4_K_M with 4K of context and batch 1, rising to roughly 89.1 GB at 32K. That figure is weights plus KV cache plus a 10% activation buffer, calculated from the model's architecture rather than measured on a card.

Will DBRX Instruct run on a 128GB GPU?

Yes — at Q4_K_M and 4K context it needs about 84.3 GB, which leaves 43.7 GB spare on a DGX Spark 128G, or on any of the other 128 GB cards here (4 in all). That is the smallest size in this index that clears it comfortably; 8 of 78 do.

Which quantization of DBRX Instruct should I use?

Q4_K_M has the lowest published quality loss (2.5%), and Q4_K_M is the level most people run. All 2 levels in the index are Q4_K_M, Q3_K_M. Note this is a mixture-of-experts model — all parameters must be resident even though only a fraction are active per token, so the memory cost follows the total, not the active count.

DBRX Instruct

132B MoE

Databricks

⚠ Superseded · Prefer GLM-4.5-Air

This index would suggest GLM-4.5-Air instead today — longer context (32K → 128K tokens). The numbers below are still accurate; they are just for a model you probably should not start with. Compare the two →

MoE flagship (~36B active). Needs multi-GPU; strong code and reasoning at scale.

Pro GPU

33K

Max Context

Quant Variants

GGUF Q4_K_M

Best Quality

97.5%

Accuracy Retained

Calculate VRAM Hugging Face Compare

Quantization Variants

Per-quant VRAM, quality loss, and inference speed on RTX 4090

Measured = run on this site · Estimated = not run here: calculated, or a figure this site has not verified · Community = public reports

Format	Level	BPW	VRAM	PPL Loss	Speed	Source	Actions
GGUF	Q4_K_M	4.85	78.5 GB	2.5%	—	Estimated	Calc HF
GGUF	Q3_K_M	3.87	63.2 GB	5.8%	—	Estimated	Calc HF

Similar models

Compare with Qwen3 235B-A22B

235B-A22B

Qwen3 235B-A22B Instruct

Alibaba Qwen3

Pro GPU

115.0 GBmin VRAM97.8%accuracy

Qwen3 flagship MoE (22B active / 235B total). Q4_K_M ~142GB; rivals DeepSeek-R1 class models.

671B MoE

DeepSeek-V3

DeepSeek

Pro GPU

310.0 GBmin VRAM98.0%accuracy

DeepSeek-V3 frontier MoE (~37B active / 671B total). MLA + FP8; multi-node GPU cluster required at Q4. MLA cache: a GGUF converted by current llama.cpp (attn_k_b / attn_v_b) holds about 2 GB at 32K, which the estimates here assume; an older conversion with a single attn_kv_b tensor caches full keys and values — about 150 GB at 32K.

671B MoE

DeepSeek-R1

DeepSeek

Pro GPU

310.0 GBmin VRAM98.2%accuracy

DeepSeek-R1 reasoning model built on V3 MoE. Chain-of-thought at frontier scale — use distill variants for local GPUs. MLA cache: a GGUF converted by current llama.cpp (attn_k_b / attn_v_b) holds about 2 GB at 32K, which the estimates here assume; an older conversion with a single attn_kv_b tensor caches full keys and values — about 150 GB at 32K.

675B MoE

Mistral Large 3 675B Instruct

Mistral AI

Pro GPU

312.0 GBmin VRAM97.9%accuracy

Mistral 3 flagship MoE (41B active / 675B total) with vision encoder. FP8 on 8×H200; GGUF quant for research clusters only. Its attention is DeepSeek-style MLA, so the real KV cache is smaller than the figure below, which uses an unverified conventional layout. At 32K that is a rounding error beside ~390 GB of weights; at the full 256K window, treat the cache figure as an upper bound.

Running DBRX Instruct locally

At Q4_K_M and 4K of context, DBRX Instruct needs about 84.3 GB — 76.0 GB of weights, 0.63 GB of KV cache and a 7.7 GB activation buffer. The smallest cards in this index that clear that comfortably are 128 GB ones — 4 of them, from the DGX Spark 128G up — and 8 of the 78 cards here do. These are calculated figures, not measurements: the estimate stops counting a card as comfortable at 88% of its VRAM, which is roughly the room a desktop session needs.

What longer context costs

Going from 4K to 32K adds about 4.37 GB, taking the total to 89.1 GB. Weights do not move with context — only the KV cache does, and it grows linearly, so this is the number to watch when planning for long documents. The model's native window is 32K; holding all of it at Q4_K_M would need about 89 GB.

Which build to download

This index only tracks DBRX Instruct in GGUF, across 2 levels (Q4_K_M, Q3_K_M). Q4_K_M carries the lowest published perplexity loss at 2.5%. No throughput has been measured for this model on this site's hardware. GGUF runs on llama.cpp and Ollama across NVIDIA, AMD and Apple silicon; AWQ and GPTQ target vLLM on CUDA and ROCm; EXL2 is ExLlamaV2 — CUDA only, and now archived, so local use rather than serving.

Common questions

How much VRAM does DBRX Instruct need?: About 84.3 GB at Q4_K_M with 4K of context and batch 1, rising to roughly 89.1 GB at 32K. That figure is weights plus KV cache plus a 10% activation buffer, calculated from the model's architecture rather than measured on a card.
Will DBRX Instruct run on a 128GB GPU?: Yes — at Q4_K_M and 4K context it needs about 84.3 GB, which leaves 43.7 GB spare on a DGX Spark 128G, or on any of the other 128 GB cards here (4 in all). That is the smallest size in this index that clears it comfortably; 8 of 78 do.
Which quantization of DBRX Instruct should I use?: Q4_K_M has the lowest published quality loss (2.5%), and Q4_K_M is the level most people run. All 2 levels in the index are Q4_K_M, Q3_K_M. Note this is a mixture-of-experts model — all parameters must be resident even though only a fraction are active per token, so the memory cost follows the total, not the active count.

Where this model fits

Sized at Q4_K_M with a 4K context window, smallest card first. Comfortable means the estimate uses at most 88% of the memory.

Runs comfortably on: DGX Spark 128G128GB · +43.7 Mac M3 Max 128G128GB · +11.7 Mac M4 Max 128G128GB · +11.7 Mac M5 Max 128G128GB · +11.7 Mac M2 Ultra 192G192GB · +59.7
Just misses: A100 80G+4.3GB over H100 80G+4.3GB over
Ships in: GGUF
Try it yourself: Size it in the calculator Generate the run command

How to actually run this

Deployment guides for this model and this class of hardware.

This page's figures change when the model or the runtime does.Last updated 2026-10-20 RSS → /feed.xml