Models / Qwen3-Coder 480B-A35B
Can I run Qwen3-Coder 480B-A35B locally?
Qwen · 480B parameters · mixture of experts, 35B active per token · up to 256K context · license apache-2.0 · Hugging Face
At Q4_K_M with a 32K context, Qwen3-Coder 480B-A35B needs about 312 GB of GPU or unified memory by our estimate. The official checkpoint is 960 GB on disk.
Memory by quantization and context
| Quantization | Weights | Total at 8K | Total at 32K | Total at 128K |
|---|---|---|---|---|
| Q3_K_M (~4.0 bit) | 240 GB | 250 GB | 256 GB | 281 GB |
| Q4_K_M (~4.9 bit, popular default) | 294 GB | 306 GB | 312 GB | 337 GB |
| Q5_K_M (~5.7 bit) | 342 GB | 356 GB | 362 GB | 387 GB |
| Q6_K (~6.6 bit) | 394 GB | 409 GB | 415 GB | 440 GB |
| Q8_0 (~8.5 bit, near-lossless) | 510 GB | 529 GB | 535 GB | 560 GB |
| F16 / BF16 (16 bit) | 960 GB | 992 GB | 998 GB | 1023 GB |
Bits per weight from llama.cpp's quantize docs. F16 KV cache. Totals include our runtime-overhead estimate.
Cheapest hardware that fits (Q4_K_M, 32K)
- 4× NVIDIA A100 80GB PCIe: about €59,996 (used, ebay.nl, 316 GB usable)
- NVIDIA DGX Station (GB300): about €83,738 (new, configurator.exxactcorp.com, 740 GB usable)
Runs in
| Tool | Platforms | How | Source |
|---|---|---|---|
| Ollama Desktop app + CLI | macOS, Windows, Linux | ollama run qwen3-coder:480b | source |
| LM Studio Desktop app | macOS, Windows (x64, arm64), Linux (x64, arm64) | lmstudio-community build: lmstudio-community/Qwen3-Coder-480B-A35B-Instruct-MLX-6bit | source |
| llama.cpp Inference engine (C/C++) | Linux, Windows, macOS, Android (build) | GGUF: bartowski/Qwen_Qwen3-Coder-480B-A35B-Instruct-GGUF | source |
| MLX (mlx-lm) Apple framework + CLI | macOS (Apple Silicon only) | MLX conversion: mlx-community/Qwen3-Coder-480B-A35B-Instruct-4bit | source |
| KTransformers CPU+GPU hybrid engine | Linux, Windows | named in the official model card | source |
Only tools with a public source are listed: an official build (ggml-org, lmstudio-community, mlx-community, bartowski or Unsloth on Hugging Face), the model card, ollama.com or the tool's own README. Apps built on llama.cpp, such as Jan and PocketPal AI, can usually load the same GGUF files. All local apps.
Try other settings
2 hardware options fit
Cheapest real offer per device from AI Hardware Deals, converted to EUR at approximate rates. Multi-GPU rows multiply the card price.
| Setup | Headroom | ||||
|---|---|---|---|---|---|
| NVIDIA DGX Station (GB300) unified memory | 1 device | 740 GB | 428 GB | €83,738 new, configurator.exxactcorp.com | ≤ 332 tok/s |
| Apple Mac Studio M5 Ultra 512GB (2026) unified memory | 1 device | 384 GB | 72.1 GB | no verified price | ≤ 56 tok/s |
Speed ceiling is memory bandwidth divided by the bytes of (active) weights read per token, shown only where we have a sourced bandwidth figure. Real decode speed is lower, especially with long contexts or when splitting across cards. All memory numbers are estimates; see the methodology.
FAQ
- How much VRAM does Qwen3-Coder 480B-A35B need?
- About 312 GB at Q4_K_M with a 32K-token context (294 GB weights, 8.3 GB KV cache, 9.8 GB overhead). These are estimates.
- Can I run Qwen3-Coder 480B-A35B on a single GPU?
- Not at Q4_K_M with 32K context on any single GPU in our catalog. Use several GPUs, a high-memory unified-memory machine, a smaller quantization or a shorter context.
- Which apps can run Qwen3-Coder 480B-A35B locally?
- Verified so far: Ollama, LM Studio, llama.cpp, MLX (mlx-lm), KTransformers. Each entry links to its source on this page.