Methodology

This calculator estimates how much GPU or unified memory an open-weight LLM needs for inference, then matches it against real hardware offers. Everything below is an estimate. Real usage depends on the runtime (llama.cpp, Ollama, LM Studio, vLLM, MLX), its version and its settings.

Model data

• Parameter count: the exact total from the Hugging Face safetensors metadata of the official repository (or an ungated mirror with identical weights for Meta models). For multimodal checkpoints it includes the vision encoder, so text-only use needs slightly less.
• Attention layout (layers, KV heads, head size, sliding-window and hybrid layers, MLA rank): from each model's config.json.
• Active parameters for mixture-of-experts models: from the model card or the model name (for example A3B means about 3B active). Left empty when the card doesn't say.
• Official checkpoint size: the sum of the repository's safetensors files.

Memory formula

• Weights = parameters × bits per weight ÷ 8. Bits per weight are the averages in llama.cpp's quantize README: IQ2_XXS 2.3824, Q2_K 3.1593, Q3_K_M 3.996, IQ4_XS 4.4597, Q4_K_M 4.8944, Q5_K_M 5.7036, Q6_K 6.5633, Q8_0 8.5008, F16 16.0005. Real GGUF files mix tensor types, so sizes vary by a few percent.
• KV cache = 2 (keys and values) × KV heads × head size × bytes per value × context tokens, summed over full-attention layers. Sliding-window layers count only their window. Hybrid models (Gated DeltaNet, Mamba, conv or KDA layers) count only their attention layers. MLA models (DeepSeek V3, Kimi, GLM-5, Mistral Small 4) store the compressed latent: layers × (KV rank + RoPE dims) × bytes × tokens. DeepSeek V4's compressed sparse attention isn't modelled.
• Runtime overhead: our rough estimate of 1 GB plus 3% of the weights for compute buffers and the GPU context.

Hardware data

• Devices, specs and prices come from AI Hardware Deals through its public offers feed, refreshed here every 12 hours. Only real offers that pass its anti-scam checks are used; demo and rejected listings are excluded.
• Usable memory: discrete GPUs get their VRAM minus 0.5 GB. Macs get about 75% of unified memory, the commonly reported macOS default GPU limit (it can be raised with sysctl iogpu.wired_limit_mb). Other unified-memory machines (DGX Spark and GB10 boxes, Strix Halo mini PCs) get their memory minus 8 GB for the OS. These are our estimates.
• Multi-GPU: splitting a model over N identical cards uses N × usable VRAM minus 0.5 GB per card. It needs a board, power supply and cooling that can take the cards.
• Prices are converted to EUR at fixed approximate rates. Click a price to see the original listing. Prices change quickly and used listings sell out.
• Speed ceiling = memory bandwidth ÷ bytes of active weights per token. It is an upper bound, shown only for devices with a sourced bandwidth figure.

Not covered

Offloading layers or MoE experts to system RAM (llama.cpp --n-cpu-moe) can run models larger than your VRAM at lower speed. That is not modelled yet. Neither is fine-tuning or training memory.

Contact

Found a wrong number or want a model added? Email us.