Models / DeepSeek V3.1
Can I run DeepSeek V3.1 locally?
DeepSeek · 685B parameters · mixture of experts, 37B active per token · up to 160K context · license mit · Hugging Face
At Q4_K_M with a 32K context, DeepSeek V3.1 needs about 435 GB of GPU or unified memory by our estimate. The official checkpoint (FP8) is 689 GB on disk.
Memory by quantization and context
| Quantization | Weights | Total at 8K | Total at 32K | Total at 128K |
|---|---|---|---|---|
| Q3_K_M (~4.0 bit) | 342 GB | 354 GB | 355 GB | 362 GB |
| Q4_K_M (~4.9 bit, popular default) | 419 GB | 433 GB | 435 GB | 442 GB |
| Q5_K_M (~5.7 bit) | 488 GB | 504 GB | 506 GB | 513 GB |
| Q6_K (~6.6 bit) | 562 GB | 580 GB | 582 GB | 589 GB |
| Q8_0 (~8.5 bit, near-lossless) | 727 GB | 751 GB | 753 GB | 759 GB |
| F16 / BF16 (16 bit) | 1369 GB | 1412 GB | 1413 GB | 1420 GB |
Bits per weight from llama.cpp's quantize docs. F16 KV cache. Totals include our runtime-overhead estimate.
Cheapest hardware that fits (Q4_K_M, 32K)
- NVIDIA DGX Station (GB300): about €83,738 (new, configurator.exxactcorp.com, 740 GB usable)
Runs in
| Tool | Platforms | How | Source |
|---|---|---|---|
| Ollama Desktop app + CLI | macOS, Windows, Linux | ollama run deepseek-v3.1:671b | source |
| llama.cpp Inference engine (C/C++) | Linux, Windows, macOS, Android (build) | GGUF: bartowski/deepseek-ai_DeepSeek-V3.1-GGUF | source |
| MLX (mlx-lm) Apple framework + CLI | macOS (Apple Silicon only) | MLX conversion: mlx-community/DeepSeek-V3.1-4bit | source |
Only tools with a public source are listed: an official build (ggml-org, lmstudio-community, mlx-community, bartowski or Unsloth on Hugging Face), the model card, ollama.com or the tool's own README. Apps built on llama.cpp, such as Jan and PocketPal AI, can usually load the same GGUF files. All local apps.
Try other settings
1 hardware options fit
Cheapest real offer per device from AI Hardware Deals, converted to EUR at approximate rates. Multi-GPU rows multiply the card price.
| Setup | Headroom | ||||
|---|---|---|---|---|---|
| NVIDIA DGX Station (GB300) unified memory | 1 device | 740 GB | 305 GB | €83,738 new, configurator.exxactcorp.com | ≤ 314 tok/s |
Speed ceiling is memory bandwidth divided by the bytes of (active) weights read per token, shown only where we have a sourced bandwidth figure. Real decode speed is lower, especially with long contexts or when splitting across cards. All memory numbers are estimates; see the methodology.
FAQ
- How much VRAM does DeepSeek V3.1 need?
- About 435 GB at Q4_K_M with a 32K-token context (419 GB weights, 2.3 GB KV cache, 13.6 GB overhead). These are estimates.
- Can I run DeepSeek V3.1 on a single GPU?
- Not at Q4_K_M with 32K context on any single GPU in our catalog. Use several GPUs, a high-memory unified-memory machine, a smaller quantization or a shorter context.
- Which apps can run DeepSeek V3.1 locally?
- Verified so far: Ollama, llama.cpp, MLX (mlx-lm). Each entry links to its source on this page.