Models / Kimi K2.6
Can I run Kimi K2.6 locally?
Moonshot AI · 1.03T parameters · mixture of experts, 32B active per token · up to 256K context · license other · Hugging Face
At Q4_K_M with a 32K context, Kimi K2.6 needs about 650 GB of GPU or unified memory by our estimate. The official checkpoint (COMPRESSED-TENSORS) is 595 GB on disk.
Memory by quantization and context
| Quantization | Weights | Total at 8K | Total at 32K | Total at 128K |
|---|---|---|---|---|
| Q3_K_M (~4.0 bit) | 513 GB | 530 GB | 532 GB | 539 GB |
| Q4_K_M (~4.9 bit, popular default) | 628 GB | 649 GB | 650 GB | 657 GB |
| Q5_K_M (~5.7 bit) | 732 GB | 756 GB | 757 GB | 764 GB |
| Q6_K (~6.6 bit) | 842 GB | 869 GB | 871 GB | 878 GB |
| Q8_0 (~8.5 bit, near-lossless) | 1091 GB | 1125 GB | 1127 GB | 1134 GB |
| F16 / BF16 (16 bit) | 2054 GB | 2117 GB | 2119 GB | 2126 GB |
Bits per weight from llama.cpp's quantize docs. F16 KV cache. Totals include our runtime-overhead estimate.
Cheapest hardware that fits (Q4_K_M, 32K)
- NVIDIA DGX Station (GB300): about €83,738 (new, configurator.exxactcorp.com, 740 GB usable)
Runs in
We have not found a verified local runner for Kimi K2.6 yet.
Only tools with a public source are listed: an official build (ggml-org, lmstudio-community, mlx-community, bartowski or Unsloth on Hugging Face), the model card, ollama.com or the tool's own README. Apps built on llama.cpp, such as Jan and PocketPal AI, can usually load the same GGUF files. All local apps.
Try other settings
1 hardware options fit
Cheapest real offer per device from AI Hardware Deals, converted to EUR at approximate rates. Multi-GPU rows multiply the card price.
| Setup | Headroom | ||||
|---|---|---|---|---|---|
| NVIDIA DGX Station (GB300) unified memory | 1 device | 740 GB | 89.6 GB | €83,738 new, configurator.exxactcorp.com | ≤ 363 tok/s |
Speed ceiling is memory bandwidth divided by the bytes of (active) weights read per token, shown only where we have a sourced bandwidth figure. Real decode speed is lower, especially with long contexts or when splitting across cards. All memory numbers are estimates; see the methodology.
FAQ
- How much VRAM does Kimi K2.6 need?
- About 650 GB at Q4_K_M with a 32K-token context (628 GB weights, 2.3 GB KV cache, 19.8 GB overhead). These are estimates.
- Can I run Kimi K2.6 on a single GPU?
- Not at Q4_K_M with 32K context on any single GPU in our catalog. Use several GPUs, a high-memory unified-memory machine, a smaller quantization or a shorter context.
- Which apps can run Kimi K2.6 locally?
- We have not verified a local app for Kimi K2.6 yet.