Qwen2.5 7B Instruct

Alibaba Qwen2.5

Alibaba's highly optimized 7B. Punches well above its weight, especially in coding.

⬇ 173.2K HF downloads♥ 157 likesQwen/Qwen2.5-7B-Instruct-GGUF· stats from 6/24/2026

Consumer GPUMac / Apple SiliconCPU / VPS

131K

Max Context

Quant Variants

GGUF Q6_K

Best Quality

99.3%

Accuracy Retained

Quantization Variants

Per-quant VRAM, quality loss, and inference speed on RTX 4090

Format	Level	BPW	VRAM	PPL Loss	Speed	Actions
GGUF	Q4_K_M	4.85	5.4 GB	3.0%	155 tok/s	Calc HF
GGUF	Q6_K	6.56	7.0 GB	0.7%	132 tok/s	Calc HF
AWQ	INT4	4	4.8 GB	4.2%	222 tok/s	Calc HF
EXL2	4.65bpw	4.65	5.2 GB	2.2%	245 tok/s	Calc HF