Phi-4 14B

Running Phi-4 14B locally

At Q4_K_M and 4K of context, Phi-4 14B needs about 10.2 GB — 8.5 GB of weights, 0.78 GB of KV cache and a 0.9 GB activation buffer. The smallest cards in this index that clear that comfortably are 12 GB ones — 10 of them, from the RTX 3060 12G up — and 65 of the 78 cards here do. These are calculated figures, not measurements: the estimate stops counting a card as comfortable at 88% of its VRAM, which is roughly the room a desktop session needs.

What longer context costs

Going from 4K to 32K adds about 5.47 GB, taking the total to 16.2 GB. Weights do not move with context — only the KV cache does, and it grows linearly, so this is the number to watch when planning for long documents. The model's native window is 16K; holding all of it at Q4_K_M would need about 13 GB.

Which build to download

This index tracks 2 formats for it — GGUF, AWQ — across 3 levels. Q5_K_M carries the lowest published perplexity loss at 1.3%. The fastest level listed is AWQ INT4 at about 112 tok/s on an RTX 4090, batch 1 — an estimate, not a run on this site's hardware. GGUF runs on llama.cpp and Ollama across NVIDIA, AMD and Apple silicon; AWQ and GPTQ target vLLM on CUDA and ROCm; EXL2 is ExLlamaV2 — CUDA only, and now archived, so local use rather than serving.

Common questions

How much VRAM does Phi-4 14B need?

About 10.2 GB at Q4_K_M with 4K of context and batch 1, rising to roughly 16.2 GB at 32K. That figure is weights plus KV cache plus a 10% activation buffer, calculated from the model's architecture rather than measured on a card.

Will Phi-4 14B run on a 12GB GPU?

Yes — at Q4_K_M and 4K context it needs about 10.2 GB, which leaves 1.8 GB spare on a RTX 3060 12G, or on any of the other 12 GB cards here (10 in all). That is the smallest size in this index that clears it comfortably; 65 of 78 do.

Which quantization of Phi-4 14B should I use?

Q5_K_M has the lowest published quality loss (1.3%), and Q4_K_M is the level most people run. All 3 levels in the index are Q4_K_M, Q5_K_M, AWQ INT4.