Models / Qwen3.6 35B-A3B
Can I run Qwen3.6 35B-A3B locally?
Qwen · 36B parameters · mixture of experts, 3B active per token · up to 256K context · license apache-2.0 · Hugging Face
At Q4_K_M with a 32K context, Qwen3.6 35B-A3B needs about 24.3 GB of GPU or unified memory by our estimate. The official checkpoint is 71.9 GB on disk.
Hybrid model: only the full-attention layers keep a growing KV cache; linear/conv layers keep a small fixed state (not counted).
Memory by quantization and context
| Quantization | Weights | Total at 8K | Total at 32K | Total at 128K |
|---|---|---|---|---|
| Q3_K_M (~4.0 bit) | 18.0 GB | 19.7 GB | 20.2 GB | 22.2 GB |
| Q4_K_M (~4.9 bit, popular default) | 22.0 GB | 23.8 GB | 24.3 GB | 26.3 GB |
| Q5_K_M (~5.7 bit) | 25.6 GB | 27.6 GB | 28.1 GB | 30.1 GB |
| Q6_K (~6.6 bit) | 29.5 GB | 31.5 GB | 32.1 GB | 34.1 GB |
| Q8_0 (~8.5 bit, near-lossless) | 38.2 GB | 40.5 GB | 41.0 GB | 43.0 GB |
| F16 / BF16 (16 bit) | 71.9 GB | 75.2 GB | 75.7 GB | 77.7 GB |
Bits per weight from llama.cpp's quantize docs. F16 KV cache. Totals include our runtime-overhead estimate.
Cheapest hardware that fits (Q4_K_M, 32K)
- 2× NVIDIA Tesla V100 16GB (PCIe or SXM2): about €550 (used, ebay.co.uk, 30.0 GB usable)
- 2× Intel Arc A770 16GB: about €604 (new, alternate.de, 30.0 GB usable)
- NVIDIA Tesla V100 32GB (PCIe or SXM2): about €621 (used, ebay.co.uk, 31.5 GB usable)
- 3× Intel Arc B580 12GB: about €960 (new, alternate.de, 33.0 GB usable)
- 2× NVIDIA Tesla T4 16GB: about €1,042 (used, ebay.co.uk, 30.0 GB usable)
Runs in
| Tool | Platforms | How | Source |
|---|---|---|---|
| Strata Single-model engine + one-click installer | Windows 10/11, Linux, macOS (Apple Silicon, experimental) | Supported by Strata; v0.1.45 added image input and several requests at once | source |
| colibri Disk-streaming MoE engine | Linux, Windows, macOS | coli setup --model qwen36-35b (runs from RAM; 10/20 GB) | source |
| Ollama Desktop app + CLI | macOS, Windows, Linux | ollama run qwen3.6:35b-a3b | source |
| LM Studio Desktop app | macOS, Windows (x64, arm64), Linux (x64, arm64) | lmstudio-community build: lmstudio-community/Qwen3.6-35B-A3B-MLX-4bit | source |
| llama.cpp Inference engine (C/C++) | Linux, Windows, macOS, Android (build) | GGUF: ggml-org/Qwen3.6-35B-A3B-GGUF | source |
| MLX (mlx-lm) Apple framework + CLI | macOS (Apple Silicon only) | MLX conversion: mlx-community/Qwen3.6-35B-A3B-4bit | source |
| KTransformers CPU+GPU hybrid engine | Linux, Windows | named in the official model card | source |
| vLLM Serving engine | Linux | named in the official model card | source |
| SGLang Serving engine | Linux | named in the official model card | source |
Only tools with a public source are listed: an official build (ggml-org, lmstudio-community, mlx-community, bartowski or Unsloth on Hugging Face), the model card, ollama.com or the tool's own README. Apps built on llama.cpp, such as Jan and PocketPal AI, can usually load the same GGUF files. All local apps.
Try other settings
Estimated memory needed24.3 GB
Hybrid model: only the full-attention layers keep a growing KV cache; linear/conv layers keep a small fixed state (not counted).
44 hardware options fit
Cheapest real offer per device from AI Hardware Deals, converted to EUR at approximate rates. Multi-GPU rows multiply the card price.
| Setup | Headroom | ||||
|---|---|---|---|---|---|
| NVIDIA Tesla V100 16GB (PCIe or SXM2) server GPU (passive cooling) | 2× cards | 30.0 GB | 5.7 GB | €550 used, ebay.co.uk | n/a |
| Intel Arc A770 16GB consumer GPU | 2× cards | 30.0 GB | 5.7 GB | €604 new, alternate.de | n/a |
| NVIDIA Tesla V100 32GB (PCIe or SXM2) server GPU (passive cooling) | 1 device | 31.5 GB | 7.2 GB | €621 used, ebay.co.uk | n/a |
| NVIDIA Tesla T4 16GB server GPU (passive cooling) | 2× cards | 30.0 GB | 5.7 GB | €1,042 used, ebay.co.uk | n/a |
| AMD Instinct MI100 32GB server GPU (passive cooling) | 1 device | 31.5 GB | 7.2 GB | €1,130 used, ebay.co.uk | n/a |
| AMD Radeon RX 7900 XT 20GB consumer GPU | 2× cards | 38.0 GB | 13.7 GB | €1,778 new, mindfactory.de | n/a |
| Intel Arc Pro B60 24GB workstation GPU | 2× cards | 46.0 GB | 21.7 GB | €1,798 new, mindfactory.de | n/a |
| AMD Radeon RX 9070 XT 16GB consumer GPU | 2× cards | 30.0 GB | 5.7 GB | €1,858 new, mindfactory.de | n/a |
| AMD Radeon Pro W7800 32GB workstation GPU | 1 device | 31.5 GB | 7.2 GB | €2,415 new, grafikkarten.com | n/a |
| Framework Desktop (Ryzen AI Max+ 395, 128GB) unified memory | 1 device | 120 GB | 95.7 GB | €2,875 used, ebay.co.uk | ≤ 139 tok/s |
| NVIDIA GeForce RTX 3090 Ti 24GB consumer GPU | 2× cards | 46.0 GB | 21.7 GB | €2,998 used, kleinanzeigen.de | n/a |
| GMKtec EVO-X2 (Ryzen AI Max+ 395, 128GB) unified memory | 1 device | 120 GB | 95.7 GB | €3,175 new, newegg.com | ≤ 139 tok/s |
| Apple Mac mini M5 Pro 64GB (2026) unified memory | 1 device | 48.0 GB | 23.7 GB | €3,224 new, cyberport.de | ≤ 167 tok/s |
| NVIDIA GeForce RTX 5080 16GB consumer GPU | 2× cards | 30.0 GB | 5.7 GB | €3,328 new, alternate.de | n/a |
| NVIDIA A100 40GB PCIe server GPU (passive cooling) | 1 device | 39.5 GB | 15.2 GB | €3,548 new, ebay.com.au | n/a |
| NVIDIA L4 24GB server GPU (passive cooling) | 2× cards | 46.0 GB | 21.7 GB | €3,682 used, ebay.fr | n/a |
| ASUS Ascent GX10 (GB10, 128GB) unified memory | 1 device | 120 GB | 95.7 GB | €4,136 new, alternate.de | ≤ 149 tok/s |
| NVIDIA A40 48GB server GPU (passive cooling) | 1 device | 47.5 GB | 23.2 GB | €4,255 used, ebay.co.uk | n/a |
| NVIDIA RTX A6000 48GB workstation GPU | 1 device | 47.5 GB | 23.2 GB | €4,349 used, ebay.ca | n/a |
| NVIDIA A10 24GB server GPU (passive cooling) | 2× cards | 46.0 GB | 21.7 GB | €4,506 new, ebay.co.uk | n/a |
| NVIDIA RTX A5000 24GB workstation GPU | 2× cards | 46.0 GB | 21.7 GB | €4,696 used, ebay.co.uk | n/a |
| Apple Mac Studio M5 Ultra 96GB (2026, base Ultra) unified memory | 1 device | 72.0 GB | 47.7 GB | €4,784 new, apple.com | ≤ 654 tok/s |
| NVIDIA GeForce RTX 3090 24GB consumer GPU | 2× cards | 46.0 GB | 21.7 GB | €5,198 used, alternate.de | n/a |
| HP Z2 Mini G1a (Ryzen AI Max+ PRO 395, 128GB) unified memory | 1 device | 120 GB | 95.7 GB | €5,686 new, mindfactory.de | ≤ 139 tok/s |
| NVIDIA GeForce RTX 4090 24GB consumer GPU | 2× cards | 46.0 GB | 21.7 GB | €5,800 new, idealo.de | n/a |
| Lenovo ThinkStation PGX (GB10, 128GB) unified memory | 1 device | 120 GB | 95.7 GB | €5,829 new, karley.de | ≤ 149 tok/s |
| Apple Mac Studio M4 Max 128GB (2025) unified memory | 1 device | 96.0 GB | 71.7 GB | €5,979 used, ebay.co.uk | ≤ 297 tok/s |
| Apple Mac Studio M5 Max 128GB (2026) unified memory | 1 device | 96.0 GB | 71.7 GB | €6,189 new, macconsultshop.de | ≤ 335 tok/s |
| Dell Pro Max with GB10 (FCM1253, 128GB) unified memory | 1 device | 120 GB | 95.7 GB | €6,663 new, eurieka.ie, excl. VAT | ≤ 149 tok/s |
| Apple Mac Studio M3 Ultra 256GB (2025) unified memory | 1 device | 192 GB | 168 GB | €6,724 used, apple.com | ≤ 446 tok/s |
| Gigabyte AI TOP ATOM (GB10, 128GB) unified memory | 1 device | 120 GB | 95.7 GB | €6,795 new, grafikkarten.com | ≤ 149 tok/s |
| NVIDIA DGX Spark (GB10, 128GB) unified memory | 1 device | 120 GB | 95.7 GB | €6,999 new, grafikkarten.com | ≤ 149 tok/s |
| HP ZGX Nano G1n AI Station (GB10, 128GB) unified memory | 1 device | 120 GB | 95.7 GB | €7,127 new, office-partner.de | ≤ 149 tok/s |
| NVIDIA GeForce RTX 5090 32GB consumer GPU | 1 device | 31.5 GB | 7.2 GB | €7,262 new, alternate.de | n/a |
| MSI EdgeXpert MS-C931 (GB10, 128GB) unified memory | 1 device | 120 GB | 95.7 GB | €7,308 new, buyzero.de | ≤ 149 tok/s |
| NVIDIA RTX 6000 Ada 48GB workstation GPU | 1 device | 47.5 GB | 23.2 GB | €9,393 new, jacob.de | n/a |
| Apple Mac Studio M5 Ultra 256GB (2026) unified memory | 1 device | 192 GB | 168 GB | €9,830 new, abt.com | ≤ 654 tok/s |
| NVIDIA A100 80GB PCIe server GPU (passive cooling) | 1 device | 79.5 GB | 55.2 GB | €14,999 used, ebay.nl | n/a |
| NVIDIA DGX Station (GB300) unified memory | 1 device | 740 GB | 716 GB | €83,738 new, configurator.exxactcorp.com | ≤ 3868 tok/s |
| NVIDIA GeForce RTX 4080 Super 16GB consumer GPU | 2× cards | 30.0 GB | 5.7 GB | no verified price | n/a |
| NVIDIA GeForce RTX 4070 Ti Super 16GB consumer GPU | 2× cards | 30.0 GB | 5.7 GB | no verified price | n/a |
| AMD Radeon RX 7900 XTX 24GB consumer GPU | 2× cards | 46.0 GB | 21.7 GB | no verified price | n/a |
| AMD Radeon Pro W7900 48GB workstation GPU | 1 device | 47.5 GB | 23.2 GB | no verified price | n/a |
| Apple Mac Studio M5 Ultra 512GB (2026) unified memory | 1 device | 384 GB | 360 GB | no verified price | ≤ 654 tok/s |
Speed ceiling is memory bandwidth divided by the bytes of (active) weights read per token, shown only where we have a sourced bandwidth figure. Real decode speed is lower, especially with long contexts or when splitting across cards. All memory numbers are estimates; see the methodology.
FAQ
- How much VRAM does Qwen3.6 35B-A3B need?
- About 24.3 GB at Q4_K_M with a 32K-token context (22.0 GB weights, 0.7 GB KV cache, 1.7 GB overhead). These are estimates.
- Can I run Qwen3.6 35B-A3B on a single GPU?
- Yes, at Q4_K_M and 32K context a NVIDIA Tesla V100 32GB (PCIe or SXM2) (32 GB) has enough memory, from about €621.
- Which apps can run Qwen3.6 35B-A3B locally?
- Verified so far: Strata, colibri, Ollama, LM Studio, llama.cpp, MLX (mlx-lm), KTransformers, vLLM, SGLang. Each entry links to its source on this page.