Models / DeepSeek V4.1 Flash
Can I run DeepSeek V4.1 Flash locally?
DeepSeek · 763B parameters · mixture of experts, 16B active per token · up to 1024K context · license mit · Hugging Face
At Q4_K_M with a 32K context, DeepSeek V4.1 Flash needs about 482 GB of GPU or unified memory by our estimate. The official checkpoint (FP8) is 510 GB on disk.
The checkpoint also holds a 196B-parameter Engram conditional memory that is only sparsely read per token (model card).
This model uses compressed/sparse attention; its KV cache is not modelled here (it is small compared to the weights).
Memory by quantization and context
| Quantization | Weights | Total at 8K | Total at 32K | Total at 128K |
|---|---|---|---|---|
| Q3_K_M (~4.0 bit) | 381 GB | 394 GB | 394 GB | 394 GB |
| Q4_K_M (~4.9 bit, popular default) | 467 GB | 482 GB | 482 GB | 482 GB |
| Q5_K_M (~5.7 bit) | 544 GB | 561 GB | 561 GB | 561 GB |
| Q6_K (~6.6 bit) | 626 GB | 646 GB | 646 GB | 646 GB |
| Q8_0 (~8.5 bit, near-lossless) | 811 GB | 836 GB | 836 GB | 836 GB |
| F16 / BF16 (16 bit) | 1526 GB | 1573 GB | 1573 GB | 1573 GB |
Bits per weight from llama.cpp's quantize docs. F16 KV cache. Totals include our runtime-overhead estimate.
Cheapest hardware that fits (Q4_K_M, 32K)
- NVIDIA DGX Station (GB300): about €83,738 (new, configurator.exxactcorp.com, 740 GB usable)
Runs in
| Tool | Platforms | How | Source |
|---|---|---|---|
| colibri Disk-streaming MoE engine | Linux, Windows, macOS | deepseek_v41 engine, one-off preparation, ~18 GB RAM plus expert cache | source |
Ollama: cloud only, it does not download the model to your machine.
Only tools with a public source are listed: an official build (ggml-org, lmstudio-community, mlx-community, bartowski or Unsloth on Hugging Face), the model card, ollama.com or the tool's own README. Apps built on llama.cpp, such as Jan and PocketPal AI, can usually load the same GGUF files. All local apps.
Try other settings
This model uses compressed/sparse attention; its KV cache is not modelled here (it is small compared to the weights).
1 hardware options fit
Cheapest real offer per device from AI Hardware Deals, converted to EUR at approximate rates. Multi-GPU rows multiply the card price.
| Setup | Headroom | ||||
|---|---|---|---|---|---|
| NVIDIA DGX Station (GB300) unified memory | 1 device | 740 GB | 258 GB | €83,738 new, configurator.exxactcorp.com | ≤ 725 tok/s |
Speed ceiling is memory bandwidth divided by the bytes of (active) weights read per token, shown only where we have a sourced bandwidth figure. Real decode speed is lower, especially with long contexts or when splitting across cards. All memory numbers are estimates; see the methodology.
FAQ
- How much VRAM does DeepSeek V4.1 Flash need?
- About 482 GB at Q4_K_M with a 32K-token context (467 GB weights, KV cache not modelled, 15.0 GB overhead). These are estimates.
- Can I run DeepSeek V4.1 Flash on a single GPU?
- Not at Q4_K_M with 32K context on any single GPU in our catalog. Use several GPUs, a high-memory unified-memory machine, a smaller quantization or a shorter context.
- Which apps can run DeepSeek V4.1 Flash locally?
- Verified so far: colibri. Each entry links to its source on this page.