How much VRAM does Step 3.5 Flash 196B-A11B need?

About 128 GB at Q4_K_M with a 32K-token context (122 GB weights, 1.7 GB KV cache, 4.7 GB overhead). These are estimates.

Can I run Step 3.5 Flash 196B-A11B on a single GPU?

Not at Q4_K_M with 32K context on any single GPU in our catalog. Use several GPUs, a high-memory unified-memory machine, a smaller quantization or a shorter context.

Which apps can run Step 3.5 Flash 196B-A11B locally?

Verified so far: llama.cpp, MLX (mlx-lm), vLLM, SGLang. Each entry links to its source on this page.

Models / Step 3.5 Flash 196B-A11B

Can I run Step 3.5 Flash 196B-A11B locally?

StepFun · 199B parameters · dense · up to 256K context · license apache-2.0 · Hugging Face

At Q4_K_M with a 32K context, Step 3.5 Flash 196B-A11B needs about 128 GB of GPU or unified memory by our estimate. The official checkpoint is 399 GB on disk.

12 full-attention and 33 sliding-window (512-token) layers with 8 KV groups, from config.json (the 3 MTP layers are not counted).

Memory by quantization and context

Quantization	Weights	Total at 8K	Total at 32K	Total at 128K
Q3_K_M (~4.0 bit)	99.6 GB	104 GB	105 GB	110 GB
Q4_K_M (~4.9 bit, popular default)	122 GB	127 GB	128 GB	133 GB
Q5_K_M (~5.7 bit)	142 GB	148 GB	149 GB	154 GB
Q6_K (~6.6 bit)	164 GB	170 GB	171 GB	176 GB
Q8_0 (~8.5 bit, near-lossless)	212 GB	220 GB	221 GB	226 GB
F16 / BF16 (16 bit)	399 GB	412 GB	413 GB	418 GB

Bits per weight from llama.cpp's quantize docs. F16 KV cache. Totals include our runtime-overhead estimate.

Cheapest hardware that fits (Q4_K_M, 32K)

Apple Mac Studio M3 Ultra 256GB (2025): about €6,724 (used, apple.com, 192 GB usable)
Apple Mac Studio M5 Ultra 256GB (2026): about €9,830 (new, abt.com, 192 GB usable)
3× NVIDIA A40 48GB: about €12,764 (used, ebay.co.uk, 141 GB usable)
3× NVIDIA RTX A6000 48GB: about €13,047 (used, ebay.ca, 141 GB usable)
4× NVIDIA A100 40GB PCIe: about €14,191 (new, ebay.com.au, 156 GB usable)

Runs in

Tool	Platforms	How	Source
llama.cpp Inference engine (C/C++)	Linux, Windows, macOS, Android (build)	GGUF: ggml-org/Step-3.5-Flash-GGUF	source
MLX (mlx-lm) Apple framework + CLI	macOS (Apple Silicon only)	MLX conversion: mlx-community/Step-3.5-Flash-4bit	source
vLLM Serving engine	Linux	named in the official model card	source
SGLang Serving engine	Linux	named in the official model card	source

Only tools with a public source are listed: an official build (ggml-org, lmstudio-community, mlx-community, bartowski or Unsloth on Hugging Face), the model card, ollama.com or the tool's own README. Apps built on llama.cpp, such as Jan and PocketPal AI, can usually load the same GGUF files. All local apps.

Try other settings

ModelQuantizationContext lengthKV cacheMax GPUs

Estimated memory needed128 GB

Weights 122 GBKV cache at 32K 1.7 GBRuntime overhead 4.7 GB

12 full-attention and 33 sliding-window (512-token) layers with 8 KV groups, from config.json (the 3 MTP layers are not counted).

5 hardware options fit

Cheapest real offer per device from AI Hardware Deals, converted to EUR at approximate rates. Multi-GPU rows multiply the card price.

Also show hardware that doesn't fit Include legacy server cards (M40, P40, P100, MI50, MI60)

	Setup		Headroom
Apple Mac Studio M3 Ultra 256GB (2025) unified memory	1 device	192 GB	63.7 GB	€6,724 used, apple.com	≤ 7 tok/s
Apple Mac Studio M5 Ultra 256GB (2026) unified memory	1 device	192 GB	63.7 GB	€9,830 new, abt.com	≤ 10 tok/s
NVIDIA A100 80GB PCIe server GPU (passive cooling)	2× cards	158 GB	29.7 GB	€29,998 used, ebay.nl	n/a
NVIDIA DGX Station (GB300) unified memory	1 device	740 GB	612 GB	€83,738 new, configurator.exxactcorp.com	≤ 58 tok/s
Apple Mac Studio M5 Ultra 512GB (2026) unified memory	1 device	384 GB	256 GB	no verified price	≤ 10 tok/s

Speed ceiling is memory bandwidth divided by the bytes of (active) weights read per token, shown only where we have a sourced bandwidth figure. Real decode speed is lower, especially with long contexts or when splitting across cards. All memory numbers are estimates; see the methodology.

FAQ

How much VRAM does Step 3.5 Flash 196B-A11B need?: About 128 GB at Q4_K_M with a 32K-token context (122 GB weights, 1.7 GB KV cache, 4.7 GB overhead). These are estimates.
Can I run Step 3.5 Flash 196B-A11B on a single GPU?: Not at Q4_K_M with 32K context on any single GPU in our catalog. Use several GPUs, a high-memory unified-memory machine, a smaller quantization or a shorter context.
Which apps can run Step 3.5 Flash 196B-A11B locally?: Verified so far: llama.cpp, MLX (mlx-lm), vLLM, SGLang. Each entry links to its source on this page.