Models / Qwen3.8-Flash-Next 125B-A6B
Can I run Qwen3.8-Flash-Next 125B-A6B locally?
Qwen · 180B parameters · mixture of experts, 6B active per token · up to 256K context · license other · Hugging Face
At Q4_K_M with a 32K context, Qwen3.8-Flash-Next 125B-A6B needs about 115 GB of GPU or unified memory by our estimate. The official checkpoint is 360 GB on disk.
The published checkpoint is 180B parameters: the 125B language model, a 51B n-gram embedding table and a 4B multi-token-prediction head. Local runners like Strata keep the n-gram table on the SSD and the experts in system RAM, so the real GPU need is far below the full-weights number shown here.
Hybrid model: only the full-attention layers keep a growing KV cache; linear/conv layers keep a small fixed state (not counted).
Memory by quantization and context
| Quantization | Weights | Total at 8K | Total at 32K | Total at 128K |
|---|---|---|---|---|
| Q3_K_M (~4.0 bit) | 89.9 GB | 93.8 GB | 94.4 GB | 96.8 GB |
| Q4_K_M (~4.9 bit, popular default) | 110 GB | 115 GB | 115 GB | 118 GB |
| Q5_K_M (~5.7 bit) | 128 GB | 133 GB | 134 GB | 136 GB |
| Q6_K (~6.6 bit) | 148 GB | 153 GB | 154 GB | 156 GB |
| Q8_0 (~8.5 bit, near-lossless) | 191 GB | 198 GB | 199 GB | 201 GB |
| F16 / BF16 (16 bit) | 360 GB | 372 GB | 373 GB | 375 GB |
Bits per weight from llama.cpp's quantize docs. F16 KV cache. Totals include our runtime-overhead estimate.
Cheapest hardware that fits (Q4_K_M, 32K)
- 4× NVIDIA Tesla V100 32GB (PCIe or SXM2): about €2,484 (used, ebay.co.uk, 124 GB usable)
- Framework Desktop (Ryzen AI Max+ 395, 128GB): about €2,875 (used, ebay.co.uk, 120 GB usable)
- GMKtec EVO-X2 (Ryzen AI Max+ 395, 128GB): about €3,175 (new, newegg.com, 120 GB usable)
- ASUS Ascent GX10 (GB10, 128GB): about €4,136 (new, alternate.de, 120 GB usable)
- 4× AMD Instinct MI100 32GB: about €4,521 (used, ebay.co.uk, 124 GB usable)
Runs in
| Tool | Platforms | How | Source |
|---|---|---|---|
| Strata Single-model engine + one-click installer | Windows 10/11, Linux, macOS (Apple Silicon, experimental) | The model Strata is built for: one-click setup, 12 GB+ GPU, 32-64 GB RAM | source |
| colibri Disk-streaming MoE engine | Linux, Windows, macOS | coli setup --model qwen38-flash-next (experts stream from SSD; 24/32 GB RAM) | source |
| Ollama Desktop app + CLI | macOS, Windows, Linux | ollama run qwen3.8-flash-next:125b-a6b-q4_K_M | source |
| LM Studio Desktop app | macOS, Windows (x64, arm64), Linux (x64, arm64) | lmstudio-community build: lmstudio-community/Qwen3.8-Flash-Next-GGUF | source |
| llama.cpp Inference engine (C/C++) | Linux, Windows, macOS, Android (build) | GGUF: ggml-org/Qwen3.8-Flash-Next-GGUF | source |
| MLX (mlx-lm) Apple framework + CLI | macOS (Apple Silicon only) | MLX conversion: mlx-community/Qwen3.8-Flash-Next-4bit | source |
| KTransformers CPU+GPU hybrid engine | Linux, Windows | named in the official model card | source |
| vLLM Serving engine | Linux | named in the official model card | source |
| SGLang Serving engine | Linux | named in the official model card | source |
| TokenSpeed Serving engine | Linux | named in the official model card | source |
Only tools with a public source are listed: an official build (ggml-org, lmstudio-community, mlx-community, bartowski or Unsloth on Hugging Face), the model card, ollama.com or the tool's own README. Apps built on llama.cpp, such as Jan and PocketPal AI, can usually load the same GGUF files. All local apps.
Try other settings
Hybrid model: only the full-attention layers keep a growing KV cache; linear/conv layers keep a small fixed state (not counted).
15 hardware options fit
Cheapest real offer per device from AI Hardware Deals, converted to EUR at approximate rates. Multi-GPU rows multiply the card price.
| Setup | Headroom | ||||
|---|---|---|---|---|---|
| Framework Desktop (Ryzen AI Max+ 395, 128GB) unified memory | 1 device | 120 GB | 4.8 GB | €2,875 used, ebay.co.uk | ≤ 70 tok/s |
| GMKtec EVO-X2 (Ryzen AI Max+ 395, 128GB) unified memory | 1 device | 120 GB | 4.8 GB | €3,175 new, newegg.com | ≤ 70 tok/s |
| ASUS Ascent GX10 (GB10, 128GB) unified memory | 1 device | 120 GB | 4.8 GB | €4,136 new, alternate.de | ≤ 74 tok/s |
| HP Z2 Mini G1a (Ryzen AI Max+ PRO 395, 128GB) unified memory | 1 device | 120 GB | 4.8 GB | €5,686 new, mindfactory.de | ≤ 70 tok/s |
| Lenovo ThinkStation PGX (GB10, 128GB) unified memory | 1 device | 120 GB | 4.8 GB | €5,829 new, karley.de | ≤ 74 tok/s |
| Dell Pro Max with GB10 (FCM1253, 128GB) unified memory | 1 device | 120 GB | 4.8 GB | €6,663 new, eurieka.ie, excl. VAT | ≤ 74 tok/s |
| Apple Mac Studio M3 Ultra 256GB (2025) unified memory | 1 device | 192 GB | 76.8 GB | €6,724 used, apple.com | ≤ 223 tok/s |
| Gigabyte AI TOP ATOM (GB10, 128GB) unified memory | 1 device | 120 GB | 4.8 GB | €6,795 new, grafikkarten.com | ≤ 74 tok/s |
| NVIDIA DGX Spark (GB10, 128GB) unified memory | 1 device | 120 GB | 4.8 GB | €6,999 new, grafikkarten.com | ≤ 74 tok/s |
| HP ZGX Nano G1n AI Station (GB10, 128GB) unified memory | 1 device | 120 GB | 4.8 GB | €7,127 new, office-partner.de | ≤ 74 tok/s |
| MSI EdgeXpert MS-C931 (GB10, 128GB) unified memory | 1 device | 120 GB | 4.8 GB | €7,308 new, buyzero.de | ≤ 74 tok/s |
| Apple Mac Studio M5 Ultra 256GB (2026) unified memory | 1 device | 192 GB | 76.8 GB | €9,830 new, abt.com | ≤ 327 tok/s |
| NVIDIA A100 80GB PCIe server GPU (passive cooling) | 2× cards | 158 GB | 42.8 GB | €29,998 used, ebay.nl | n/a |
| NVIDIA DGX Station (GB300) unified memory | 1 device | 740 GB | 625 GB | €83,738 new, configurator.exxactcorp.com | ≤ 1934 tok/s |
| Apple Mac Studio M5 Ultra 512GB (2026) unified memory | 1 device | 384 GB | 269 GB | no verified price | ≤ 327 tok/s |
Speed ceiling is memory bandwidth divided by the bytes of (active) weights read per token, shown only where we have a sourced bandwidth figure. Real decode speed is lower, especially with long contexts or when splitting across cards. All memory numbers are estimates; see the methodology.
FAQ
- How much VRAM does Qwen3.8-Flash-Next 125B-A6B need?
- About 115 GB at Q4_K_M with a 32K-token context (110 GB weights, 0.8 GB KV cache, 4.3 GB overhead). These are estimates.
- Can I run Qwen3.8-Flash-Next 125B-A6B on a single GPU?
- Not at Q4_K_M with 32K context on any single GPU in our catalog. Use several GPUs, a high-memory unified-memory machine, a smaller quantization or a shorter context.
- Which apps can run Qwen3.8-Flash-Next 125B-A6B locally?
- Verified so far: Strata, colibri, Ollama, LM Studio, llama.cpp, MLX (mlx-lm), KTransformers, vLLM, SGLang, TokenSpeed. Each entry links to its source on this page.