Phi-3 Medium 14B Instruct

14B

Microsoft Phi

Microsoft's mid-size Phi-3. Excellent quality-per-GB on 16GB cards.

Consumer GPUMac / Apple Silicon

131K

Max Context

Quant Variants

GGUF Q6_K

Best Quality

99.2%

Accuracy Retained

Quantization Variants

Per-quant VRAM, quality loss, and inference speed on RTX 4090

Format	Level	BPW	VRAM	PPL Loss	Speed	Actions
GGUF	Q4_K_M	4.85	9.8 GB	3.0%	102 tok/s	Calc HF
GGUF	Q6_K	6.56	12.8 GB	0.8%	88 tok/s	Calc HF
AWQ	INT4	4	8.8 GB	4.2%	135 tok/s	Calc HF