Models / Nemotron 3 Super 120B-A12B
Can I run Nemotron 3 Super 120B-A12B locally?
NVIDIA · 124B parameters · mixture of experts, 12B active per token · up to 256K context · license other · Hugging Face
At Q4_K_M with a 32K context, Nemotron 3 Super 120B-A12B needs about 79.2 GB of GPU or unified memory by our estimate. The official checkpoint is 247 GB on disk.
Hybrid model: only the attention layers keep a growing KV cache; the other layers keep a small fixed state (not counted).
Memory by quantization and context
| Quantization | Weights | Total at 8K | Total at 32K | Total at 128K |
|---|---|---|---|---|
| Q3_K_M (~4.0 bit) | 61.7 GB | 64.7 GB | 64.9 GB | 65.7 GB |
| Q4_K_M (~4.9 bit, popular default) | 75.6 GB | 79.0 GB | 79.2 GB | 80.0 GB |
| Q5_K_M (~5.7 bit) | 88.1 GB | 91.8 GB | 92.0 GB | 92.8 GB |
| Q6_K (~6.6 bit) | 101 GB | 106 GB | 106 GB | 107 GB |
| Q8_0 (~8.5 bit, near-lossless) | 131 GB | 136 GB | 137 GB | 137 GB |
| F16 / BF16 (16 bit) | 247 GB | 256 GB | 256 GB | 257 GB |
Bits per weight from llama.cpp's quantize docs. F16 KV cache. Totals include our runtime-overhead estimate.
Cheapest hardware that fits (Q4_K_M, 32K)
- 3× NVIDIA Tesla V100 32GB (PCIe or SXM2): about €1,863 (used, ebay.co.uk, 93.0 GB usable)
- Framework Desktop (Ryzen AI Max+ 395, 128GB): about €2,875 (used, ebay.co.uk, 120 GB usable)
- GMKtec EVO-X2 (Ryzen AI Max+ 395, 128GB): about €3,175 (new, newegg.com, 120 GB usable)
- 3× AMD Instinct MI100 32GB: about €3,390 (used, ebay.co.uk, 93.0 GB usable)
- 4× Intel Arc Pro B60 24GB: about €3,596 (new, mindfactory.de, 92.0 GB usable)
Runs in
| Tool | Platforms | How | Source |
|---|---|---|---|
| Ollama Desktop app + CLI | macOS, Windows, Linux | ollama run nemotron-3-super:120b-a12b | source |
| LM Studio Desktop app | macOS, Windows (x64, arm64), Linux (x64, arm64) | lmstudio-community build: lmstudio-community/NVIDIA-Nemotron-3-Super-120B-A12B-GGUF | source |
| llama.cpp Inference engine (C/C++) | Linux, Windows, macOS, Android (build) | GGUF: unsloth/NVIDIA-Nemotron-3-Super-120B-A12B-GGUF | source |
| MLX (mlx-lm) Apple framework + CLI | macOS (Apple Silicon only) | MLX conversion: mlx-community/NVIDIA-Nemotron-3-Super-120B-A12B-4bit | source |
| vLLM Serving engine | Linux | named in the official model card | source |
| SGLang Serving engine | Linux | named in the official model card | source |
Only tools with a public source are listed: an official build (ggml-org, lmstudio-community, mlx-community, bartowski or Unsloth on Hugging Face), the model card, ollama.com or the tool's own README. Apps built on llama.cpp, such as Jan and PocketPal AI, can usually load the same GGUF files. All local apps.
Try other settings
Hybrid model: only the attention layers keep a growing KV cache; the other layers keep a small fixed state (not counted).
21 hardware options fit
Cheapest real offer per device from AI Hardware Deals, converted to EUR at approximate rates. Multi-GPU rows multiply the card price.
| Setup | Headroom | ||||
|---|---|---|---|---|---|
| Framework Desktop (Ryzen AI Max+ 395, 128GB) unified memory | 1 device | 120 GB | 40.8 GB | €2,875 used, ebay.co.uk | ≤ 35 tok/s |
| GMKtec EVO-X2 (Ryzen AI Max+ 395, 128GB) unified memory | 1 device | 120 GB | 40.8 GB | €3,175 new, newegg.com | ≤ 35 tok/s |
| ASUS Ascent GX10 (GB10, 128GB) unified memory | 1 device | 120 GB | 40.8 GB | €4,136 new, alternate.de | ≤ 37 tok/s |
| HP Z2 Mini G1a (Ryzen AI Max+ PRO 395, 128GB) unified memory | 1 device | 120 GB | 40.8 GB | €5,686 new, mindfactory.de | ≤ 35 tok/s |
| Lenovo ThinkStation PGX (GB10, 128GB) unified memory | 1 device | 120 GB | 40.8 GB | €5,829 new, karley.de | ≤ 37 tok/s |
| Apple Mac Studio M4 Max 128GB (2025) unified memory | 1 device | 96.0 GB | 16.8 GB | €5,979 used, ebay.co.uk | ≤ 74 tok/s |
| Apple Mac Studio M5 Max 128GB (2026) unified memory | 1 device | 96.0 GB | 16.8 GB | €6,189 new, macconsultshop.de | ≤ 84 tok/s |
| Dell Pro Max with GB10 (FCM1253, 128GB) unified memory | 1 device | 120 GB | 40.8 GB | €6,663 new, eurieka.ie, excl. VAT | ≤ 37 tok/s |
| Apple Mac Studio M3 Ultra 256GB (2025) unified memory | 1 device | 192 GB | 113 GB | €6,724 used, apple.com | ≤ 112 tok/s |
| Gigabyte AI TOP ATOM (GB10, 128GB) unified memory | 1 device | 120 GB | 40.8 GB | €6,795 new, grafikkarten.com | ≤ 37 tok/s |
| NVIDIA DGX Spark (GB10, 128GB) unified memory | 1 device | 120 GB | 40.8 GB | €6,999 new, grafikkarten.com | ≤ 37 tok/s |
| HP ZGX Nano G1n AI Station (GB10, 128GB) unified memory | 1 device | 120 GB | 40.8 GB | €7,127 new, office-partner.de | ≤ 37 tok/s |
| MSI EdgeXpert MS-C931 (GB10, 128GB) unified memory | 1 device | 120 GB | 40.8 GB | €7,308 new, buyzero.de | ≤ 37 tok/s |
| NVIDIA A40 48GB server GPU (passive cooling) | 2× cards | 94.0 GB | 14.8 GB | €8,509 used, ebay.co.uk | n/a |
| NVIDIA RTX A6000 48GB workstation GPU | 2× cards | 94.0 GB | 14.8 GB | €8,698 used, ebay.ca | n/a |
| Apple Mac Studio M5 Ultra 256GB (2026) unified memory | 1 device | 192 GB | 113 GB | €9,830 new, abt.com | ≤ 163 tok/s |
| NVIDIA A100 80GB PCIe server GPU (passive cooling) | 1 device | 79.5 GB | 0.3 GB | €14,999 used, ebay.nl | n/a |
| NVIDIA RTX 6000 Ada 48GB workstation GPU | 2× cards | 94.0 GB | 14.8 GB | €18,786 new, jacob.de | n/a |
| NVIDIA DGX Station (GB300) unified memory | 1 device | 740 GB | 661 GB | €83,738 new, configurator.exxactcorp.com | ≤ 967 tok/s |
| AMD Radeon Pro W7900 48GB workstation GPU | 2× cards | 94.0 GB | 14.8 GB | no verified price | n/a |
| Apple Mac Studio M5 Ultra 512GB (2026) unified memory | 1 device | 384 GB | 305 GB | no verified price | ≤ 163 tok/s |
Speed ceiling is memory bandwidth divided by the bytes of (active) weights read per token, shown only where we have a sourced bandwidth figure. Real decode speed is lower, especially with long contexts or when splitting across cards. All memory numbers are estimates; see the methodology.
FAQ
- How much VRAM does Nemotron 3 Super 120B-A12B need?
- About 79.2 GB at Q4_K_M with a 32K-token context (75.6 GB weights, 0.3 GB KV cache, 3.3 GB overhead). These are estimates.
- Can I run Nemotron 3 Super 120B-A12B on a single GPU?
- Yes, at Q4_K_M and 32K context a NVIDIA A100 80GB PCIe (80 GB) has enough memory, from about €14,999.
- Which apps can run Nemotron 3 Super 120B-A12B locally?
- Verified so far: Ollama, LM Studio, llama.cpp, MLX (mlx-lm), vLLM, SGLang. Each entry links to its source on this page.