Back to Quant Hub

Mistral 7B Instruct v0.3

7B

Mistral AI

Classic Mistral 7B v0.3. Still a reliable baseline for local chat APIs.

⬇ 32.9K HF downloads♥ 41 likesbartowski/Mistral-7B-Instruct-v0.3-GGUF· stats from 6/24/2026

Consumer GPUMac / Apple SiliconCPU / VPS

33K

Max Context

3

Quant Variants

GGUF Q6_K

Best Quality

99.1%

Accuracy Retained

Calculate VRAM Hugging Face Compare

Quantization Variants

Per-quant VRAM, quality loss, and inference speed on RTX 4090

Format	Level	BPW	VRAM	PPL Loss	Speed	Actions
GGUF	Q4_K_M	4.85	5.2 GB	3.2%	158 tok/s	Calc HF
GGUF	Q6_K	6.56	6.8 GB	0.9%	135 tok/s	Calc HF
AWQ	INT4	4	4.6 GB	4.5%	225 tok/s	Calc HF