Models / GLM-5.3-Flash 320B-A18B
Can I run GLM-5.3-Flash 320B-A18B locally?
Z.ai · 321B parameters · mixture of experts, 18B active per token · up to 1024K context · license mit · Hugging Face
At Q4_K_M with a 32K context, GLM-5.3-Flash 320B-A18B needs about 204 GB of GPU or unified memory by our estimate. The official checkpoint (FP8) is 328 GB on disk.
Hybrid model: only the latent-attention (MLA) layers keep a growing KV cache; the linear-attention layers keep a small fixed state (not counted).
Memory by quantization and context
| Quantization | Weights | Total at 8K | Total at 32K | Total at 128K |
|---|---|---|---|---|
| Q3_K_M (~4.0 bit) | 161 GB | 166 GB | 167 GB | 168 GB |
| Q4_K_M (~4.9 bit, popular default) | 197 GB | 204 GB | 204 GB | 205 GB |
| Q5_K_M (~5.7 bit) | 229 GB | 237 GB | 237 GB | 238 GB |
| Q6_K (~6.6 bit) | 264 GB | 273 GB | 273 GB | 274 GB |
| Q8_0 (~8.5 bit, near-lossless) | 341 GB | 353 GB | 353 GB | 354 GB |
| F16 / BF16 (16 bit) | 643 GB | 663 GB | 663 GB | 664 GB |
Bits per weight from llama.cpp's quantize docs. F16 KV cache. Totals include our runtime-overhead estimate.
Cheapest hardware that fits (Q4_K_M, 32K)
- 3× NVIDIA A100 80GB PCIe: about €44,997 (used, ebay.nl, 237 GB usable)
- NVIDIA DGX Station (GB300): about €83,738 (new, configurator.exxactcorp.com, 740 GB usable)
Runs in
| Tool | Platforms | How | Source |
|---|---|---|---|
| colibri Disk-streaming MoE engine | Linux, Windows, macOS | glm53-flash engine, converted shard by shard to ~195 GB; 25 GB RAM | source |
| llama.cpp Inference engine (C/C++) | Linux, Windows, macOS, Android (build) | GGUF: ggml-org/GLM-5.3-Flash-GGUF | source |
| MLX (mlx-lm) Apple framework + CLI | macOS (Apple Silicon only) | MLX conversion: mlx-community/GLM-5.3-Flash-4bit | source |
| KTransformers CPU+GPU hybrid engine | Linux, Windows | named in the official model card | source |
| vLLM Serving engine | Linux | named in the official model card | source |
| SGLang Serving engine | Linux | named in the official model card | source |
| TokenSpeed Serving engine | Linux | named in the official model card | source |
Ollama: cloud only, it does not download the model to your machine.
Only tools with a public source are listed: an official build (ggml-org, lmstudio-community, mlx-community, bartowski or Unsloth on Hugging Face), the model card, ollama.com or the tool's own README. Apps built on llama.cpp, such as Jan and PocketPal AI, can usually load the same GGUF files. All local apps.
Try other settings
Hybrid model: only the latent-attention (MLA) layers keep a growing KV cache; the linear-attention layers keep a small fixed state (not counted).
2 hardware options fit
Cheapest real offer per device from AI Hardware Deals, converted to EUR at approximate rates. Multi-GPU rows multiply the card price.
| Setup | Headroom | ||||
|---|---|---|---|---|---|
| NVIDIA DGX Station (GB300) unified memory | 1 device | 740 GB | 536 GB | €83,738 new, configurator.exxactcorp.com | ≤ 645 tok/s |
| Apple Mac Studio M5 Ultra 512GB (2026) unified memory | 1 device | 384 GB | 180 GB | no verified price | ≤ 109 tok/s |
Speed ceiling is memory bandwidth divided by the bytes of (active) weights read per token, shown only where we have a sourced bandwidth figure. Real decode speed is lower, especially with long contexts or when splitting across cards. All memory numbers are estimates; see the methodology.
FAQ
- How much VRAM does GLM-5.3-Flash 320B-A18B need?
- About 204 GB at Q4_K_M with a 32K-token context (197 GB weights, 0.4 GB KV cache, 6.9 GB overhead). These are estimates.
- Can I run GLM-5.3-Flash 320B-A18B on a single GPU?
- Not at Q4_K_M with 32K context on any single GPU in our catalog. Use several GPUs, a high-memory unified-memory machine, a smaller quantization or a shorter context.
- Which apps can run GLM-5.3-Flash 320B-A18B locally?
- Verified so far: colibri, llama.cpp, MLX (mlx-lm), KTransformers, vLLM, SGLang, TokenSpeed. Each entry links to its source on this page.