Models / DeepSeek V4 Flash (0731)
Can I run DeepSeek V4 Flash (0731) locally?
DeepSeek · 304B parameters · mixture of experts, 13B active per token · up to 1024K context · license mit · Hugging Face
At Q4_K_M with a 32K context, DeepSeek V4 Flash (0731) needs about 193 GB of GPU or unified memory by our estimate. The official checkpoint (FP8) is 167 GB on disk.
This model uses compressed/sparse attention; its KV cache is not modelled here (it is small compared to the weights).
Memory by quantization and context
| Quantization | Weights | Total at 8K | Total at 32K | Total at 128K |
|---|---|---|---|---|
| Q3_K_M (~4.0 bit) | 152 GB | 157 GB | 157 GB | 157 GB |
| Q4_K_M (~4.9 bit, popular default) | 186 GB | 193 GB | 193 GB | 193 GB |
| Q5_K_M (~5.7 bit) | 217 GB | 224 GB | 224 GB | 224 GB |
| Q6_K (~6.6 bit) | 250 GB | 258 GB | 258 GB | 258 GB |
| Q8_0 (~8.5 bit, near-lossless) | 323 GB | 334 GB | 334 GB | 334 GB |
| F16 / BF16 (16 bit) | 608 GB | 628 GB | 628 GB | 628 GB |
Bits per weight from llama.cpp's quantize docs. F16 KV cache. Totals include our runtime-overhead estimate.
Cheapest hardware that fits (Q4_K_M, 32K)
- 3× NVIDIA A100 80GB PCIe: about €44,997 (used, ebay.nl, 237 GB usable)
- NVIDIA DGX Station (GB300): about €83,738 (new, configurator.exxactcorp.com, 740 GB usable)
Runs in
| Tool | Platforms | How | Source |
|---|---|---|---|
| colibri Disk-streaming MoE engine | Linux, Windows, macOS | coli setup --model deepseek-v4-flash (experts stream from SSD; 16/32 GB RAM) | source |
| LM Studio Desktop app | macOS, Windows (x64, arm64), Linux (x64, arm64) | lmstudio-community build: lmstudio-community/DeepSeek-V4-Flash-0731-GGUF | source |
| llama.cpp Inference engine (C/C++) | Linux, Windows, macOS, Android (build) | GGUF: ggml-org/DeepSeek-V4-Flash-0731-GGUF | source |
| vLLM Serving engine | Linux | named in the official model card | source |
| SGLang Serving engine | Linux | named in the official model card | source |
Only tools with a public source are listed: an official build (ggml-org, lmstudio-community, mlx-community, bartowski or Unsloth on Hugging Face), the model card, ollama.com or the tool's own README. Apps built on llama.cpp, such as Jan and PocketPal AI, can usually load the same GGUF files. All local apps.
Try other settings
This model uses compressed/sparse attention; its KV cache is not modelled here (it is small compared to the weights).
2 hardware options fit
Cheapest real offer per device from AI Hardware Deals, converted to EUR at approximate rates. Multi-GPU rows multiply the card price.
| Setup | Headroom | ||||
|---|---|---|---|---|---|
| NVIDIA DGX Station (GB300) unified memory | 1 device | 740 GB | 547 GB | €83,738 new, configurator.exxactcorp.com | ≤ 893 tok/s |
| Apple Mac Studio M5 Ultra 512GB (2026) unified memory | 1 device | 384 GB | 191 GB | no verified price | ≤ 151 tok/s |
Speed ceiling is memory bandwidth divided by the bytes of (active) weights read per token, shown only where we have a sourced bandwidth figure. Real decode speed is lower, especially with long contexts or when splitting across cards. All memory numbers are estimates; see the methodology.
FAQ
- How much VRAM does DeepSeek V4 Flash (0731) need?
- About 193 GB at Q4_K_M with a 32K-token context (186 GB weights, KV cache not modelled, 6.6 GB overhead). These are estimates.
- Can I run DeepSeek V4 Flash (0731) on a single GPU?
- Not at Q4_K_M with 32K context on any single GPU in our catalog. Use several GPUs, a high-memory unified-memory machine, a smaller quantization or a shorter context.
- Which apps can run DeepSeek V4 Flash (0731) locally?
- Verified so far: colibri, LM Studio, llama.cpp, vLLM, SGLang. Each entry links to its source on this page.