{"updated":"2026-10-17","site":"https://canirun.branolabs.com","note":"Estimated inference memory in GB (weights + F16 KV cache + runtime overhead). See /about.","quantBitsPerWeight":{"IQ2_XXS":2.3824,"Q2_K":3.1593,"Q3_K_M":3.996,"IQ4_XS":4.4597,"Q4_K_M":4.8944,"Q5_K_M":5.7036,"Q6_K":6.5633,"Q8_0":8.5008,"F16":16.0005},"quantSource":"https://github.com/ggml-org/llama.cpp/blob/master/tools/quantize/README.md","tools":[{"id":"strata","name":"Strata","url":"https://github.com/Niko1221/Strata","repo":"Niko1221/Strata","kind":"Single-model engine + one-click installer","what":"An inference engine written for one model, Qwen3.8-Flash-Next (125B MoE, 6B active). It keeps the most-used experts on a 12 GB+ graphics card, all 24,576 experts in system RAM and the 29 GB n-gram lookup table on the SSD, and uses the model's own MTP layer to guess and check several tokens per pass. Version 0.1.45 also runs Qwen3.6-35B-A3B.","platforms":["Windows 10/11","Linux","macOS (Apple Silicon, experimental)"],"hardware":"NVIDIA RTX 20-50 series or AMD RX 6800 / 7000 / 9000 with 12 GB+ VRAM, 32-64 GB RAM, about 80 GB disk. Community-tested: older Tesla/GTX cards, Intel Arc, Strix Halo.","formats":"GGUF quants of Qwen3.8-Flash-Next (ISTA-DASLab GSQ-RCO, Unsloth, UkisAI Swift 1.5); uses parts of llama.cpp/ggml","api":"OpenAI-compatible at http://127.0.0.1:8080/v1, Anthropic /v1/messages, Responses API, browser chat","license":"MIT","best":"Running Qwen3.8-Flash-Next fast on a gaming PC: the project measured 53-94 tokens/s on an RTX 5070 12 GB with 64 GB RAM.","checked":"v0.1.45, released 17 Oct 2026","sources":[{"label":"README","url":"https://github.com/Niko1221/Strata"},{"label":"Model sizes","url":"https://github.com/Niko1221/Strata/blob/main/docs/MODELS.md"},{"label":"v0.1.45 release notes","url":"https://github.com/Niko1221/Strata/releases/tag/v0.1.45"},{"label":"Paper (PDF)","url":"https://github.com/Niko1221/Strata/blob/main/docs/paper/Strata-Paper.pdf"}],"page":"https://canirun.branolabs.com/tools#strata"},{"id":"colibri","name":"colibri","url":"https://github.com/JustVugg/colibri","repo":"JustVugg/colibri","kind":"Disk-streaming MoE engine","what":"A dependency-free C engine that runs very large mixture-of-experts models on ordinary machines. It keeps the dense part in RAM and streams the experts from the disk when they are needed, so a fast NVMe SSD sets the speed. One engine per model family: GLM-5.2/5.3, GLM-5.3-Flash, Kimi K3, DeepSeek V4 / V4.1 Flash, MiMo-V2.6, Qwen3.8-Flash-Next, Qwen3.6 (also Qwen3-Coder and Qwen3.8-27B), Inkling and OLMoE.","platforms":["Linux","Windows","macOS"],"hardware":"CPU only works; 8 GB RAM minimum (16 GB+ better) and 22 GB disk for the smallest model. Optional GPU: CUDA on Linux, Vulkan on Linux/Windows (any vendor, integrated too), Metal for some engines on Apple Silicon.","formats":"Its own int4/int8 containers, converted once from the Hugging Face safetensors (pre-converted copies on Hugging Face)","api":"OpenAI-compatible (coli serve, /v1/chat/completions), web dashboard, terminal chat","license":"Apache-2.0","best":"Trying huge MoE models with little RAM, if you accept slow output: in its own tests small models run at 6-30 tokens/s and the 300B-750B ones at roughly 0.05-4 tokens/s on ordinary machines.","checked":"v2.0.0, released 6 Oct 2026","sources":[{"label":"README and model tables","url":"https://github.com/JustVugg/colibri"},{"label":"Project site","url":"https://justvugg.github.io/colibri"},{"label":"Quick start","url":"https://github.com/JustVugg/colibri/blob/main/docs/quickstart.md"}],"page":"https://canirun.branolabs.com/tools#colibri"},{"id":"ollama","name":"Ollama","url":"https://ollama.com","repo":"ollama/ollama","kind":"Desktop app + CLI","what":"The easiest way to download and run a model with one command (ollama run model:tag). It has its own model library; some large models there are cloud-only and do not run locally.","platforms":["macOS","Windows","Linux"],"hardware":"CPU, NVIDIA, AMD and Apple Silicon","formats":"Models from the Ollama library (GGUF-based; MLX builds on Mac for some models)","api":"Local REST API on port 11434, OpenAI-compatible endpoint","license":"MIT","best":"Beginners and quick local APIs.","checked":"Library tags checked 17 Oct 2026","sources":[{"label":"GitHub","url":"https://github.com/ollama/ollama"},{"label":"Model library","url":"https://ollama.com/library"}],"page":"https://canirun.branolabs.com/tools#ollama"},{"id":"lmstudio","name":"LM Studio","url":"https://lmstudio.ai","repo":"lmstudio-ai/lms","kind":"Desktop app","what":"A desktop app to find, download and chat with models, with a local server. It uses llama.cpp and, on Apple Silicon, MLX under the hood. Ready builds are published in the lmstudio-community Hugging Face org.","platforms":["macOS","Windows (x64, arm64)","Linux (x64, arm64)"],"hardware":"CPU, NVIDIA, AMD, Apple Silicon","formats":"GGUF and MLX","api":"OpenAI-compatible local server; lms CLI","license":"Free app (closed source); lms CLI is MIT","best":"A polished GUI on Mac and Windows.","checked":"Site checked 17 Oct 2026","sources":[{"label":"lmstudio.ai","url":"https://lmstudio.ai"},{"label":"lmstudio-community models","url":"https://huggingface.co/lmstudio-community"}],"page":"https://canirun.branolabs.com/tools#lmstudio"},{"id":"llamacpp","name":"llama.cpp","url":"https://github.com/ggml-org/llama.cpp","repo":"ggml-org/llama.cpp","kind":"Inference engine (C/C++)","what":"The engine behind GGUF and most local apps (Ollama, LM Studio, Jan, PocketPal and parts of Strata build on it). llama-server gives a local OpenAI-compatible API; llama-cli a terminal chat.","platforms":["Linux","Windows","macOS","Android (build)"],"hardware":"CPU, NVIDIA CUDA, AMD, Apple Metal, Vulkan and more","formats":"GGUF","api":"llama-server: OpenAI-compatible","license":"MIT","best":"Maximum control and the widest hardware support.","checked":"17 Oct 2026","sources":[{"label":"GitHub","url":"https://github.com/ggml-org/llama.cpp"},{"label":"Quantization types","url":"https://github.com/ggml-org/llama.cpp/blob/master/tools/quantize/README.md"}],"page":"https://canirun.branolabs.com/tools#llamacpp"},{"id":"mlx","name":"MLX (mlx-lm)","url":"https://github.com/ml-explore/mlx-lm","repo":"ml-explore/mlx-lm","kind":"Apple framework + CLI","what":"Apple's machine-learning framework; mlx-lm runs and serves LLMs on Apple Silicon. Converted models live in the mlx-community Hugging Face org.","platforms":["macOS (Apple Silicon only)"],"hardware":"M1-M5 Macs with unified memory","formats":"MLX safetensors (4/6/8-bit, mxfp4)","api":"mlx_lm.server: OpenAI-compatible","license":"MIT","best":"The fastest option on many Macs.","checked":"17 Oct 2026","sources":[{"label":"GitHub","url":"https://github.com/ml-explore/mlx-lm"},{"label":"mlx-community models","url":"https://huggingface.co/mlx-community"}],"page":"https://canirun.branolabs.com/tools#mlx"},{"id":"ktransformers","name":"KTransformers","url":"https://github.com/kvcache-ai/ktransformers","repo":"kvcache-ai/ktransformers","kind":"CPU+GPU hybrid engine","what":"A research project for CPU-GPU heterogeneous inference: big MoE models run with the experts on the CPU (INT4/INT8) and the rest on a GPU.","platforms":["Linux","Windows"],"hardware":"x86 CPU (AMX/AVX-512, AVX2) plus NVIDIA or AMD (ROCm) GPU","formats":"Hugging Face weights, CPU-side INT4/INT8, GPTQ","api":"OpenAI-compatible server","license":"Apache-2.0","best":"One GPU plus lots of system RAM for large MoE models.","checked":"17 Oct 2026","sources":[{"label":"GitHub","url":"https://github.com/kvcache-ai/ktransformers"}],"page":"https://canirun.branolabs.com/tools#ktransformers"},{"id":"vllm","name":"vLLM","url":"https://vllm.ai","repo":"vllm-project/vllm","kind":"Serving engine","what":"A high-throughput serving engine for many users at once. Most new open models ship vLLM recipes on day one.","platforms":["Linux"],"hardware":"Mainly datacenter and workstation GPUs (NVIDIA, AMD and others)","formats":"Hugging Face safetensors, FP8, AWQ/GPTQ, NVFP4","api":"OpenAI-compatible","license":"Apache-2.0","best":"Serving a model to a team when it fits in GPU memory.","checked":"17 Oct 2026","sources":[{"label":"GitHub","url":"https://github.com/vllm-project/vllm"},{"label":"Recipes","url":"https://recipes.vllm.ai"}],"page":"https://canirun.branolabs.com/tools#vllm"},{"id":"sglang","name":"SGLang","url":"https://sglang.io","repo":"sgl-project/sglang","kind":"Serving engine","what":"A fast serving framework for LLMs and multimodal models, with per-model cookbooks.","platforms":["Linux"],"hardware":"Mainly datacenter and workstation GPUs","formats":"Hugging Face safetensors, FP8 and more","api":"OpenAI-compatible","license":"Apache-2.0","best":"High-throughput serving, agent workloads.","checked":"17 Oct 2026","sources":[{"label":"GitHub","url":"https://github.com/sgl-project/sglang"}],"page":"https://canirun.branolabs.com/tools#sglang"},{"id":"tokenspeed","name":"TokenSpeed","url":"https://lightseek.org/blog/lightseek-tokenspeed.html","repo":"lightseekorg/tokenspeed","kind":"Serving engine","what":"An inference engine from the non-profit LightSeek Foundation, focused on datacenter GPUs (NVIDIA Blackwell, AMD Instinct).","platforms":["Linux"],"hardware":"Datacenter GPUs","formats":"Hugging Face weights, FP8/FP4","api":"OpenAI-compatible","license":"MIT","best":"Datacenter serving; not a home-PC tool.","checked":"17 Oct 2026","sources":[{"label":"GitHub","url":"https://github.com/lightseekorg/tokenspeed"}],"page":"https://canirun.branolabs.com/tools#tokenspeed"},{"id":"jan","name":"Jan","url":"https://jan.ai","repo":"janhq/jan","kind":"Desktop app","what":"An open-source ChatGPT-style app that runs models offline with a bundled llama.cpp engine. Any GGUF that mainline llama.cpp supports should work.","platforms":["Windows","macOS","Linux"],"hardware":"CPU, NVIDIA, AMD, Apple Silicon","formats":"GGUF","api":"Local OpenAI-compatible server","license":"Open source (see repo)","best":"An open-source alternative to LM Studio.","checked":"17 Oct 2026","sources":[{"label":"GitHub","url":"https://github.com/janhq/jan"}],"page":"https://canirun.branolabs.com/tools#jan"},{"id":"pocketpal","name":"PocketPal AI","url":"https://github.com/a-ghorbani/pocketpal-ai","repo":"a-ghorbani/pocketpal-ai","kind":"Phone app","what":"Runs small GGUF models fully offline on a phone, using llama.cpp. You pick and download models from Hugging Face inside the app.","platforms":["iOS / iPadOS","Android"],"hardware":"Phone CPU, GPU (Metal on iOS, OpenCL on Adreno) and Qualcomm Hexagon NPU","formats":"GGUF","license":"MIT","best":"Small models (about 1-8B, such as Gemma 4 E2B/E4B or Ling-3.0-tiny GGUFs) on a phone.","checked":"17 Oct 2026","sources":[{"label":"GitHub","url":"https://github.com/a-ghorbani/pocketpal-ai"},{"label":"App Store","url":"https://apps.apple.com/us/app/pocketpal-ai/id6502579498"},{"label":"Google Play","url":"https://play.google.com/store/apps/details?id=com.pocketpalai"}],"page":"https://canirun.branolabs.com/tools#pocketpal"}],"apiOnly":[{"name":"Qwen3.8-Flash","by":"Qwen (Qwen Cloud)","note":"The production API version built on Qwen3.8-Flash-Next, with 1M context by default and built-in tools. Run Qwen3.8-Flash-Next locally instead.","url":"https://www.qwencloud.com/models/qwen3.8-flash","source":"https://huggingface.co/Qwen/Qwen3.8-Flash-Next"}],"models":[{"slug":"gemma-4-e2b","name":"Gemma 4 E2B","org":"Google","repo":"google/gemma-4-E2B-it","dataRepo":"google/gemma-4-E2B-it","use":["chat","vision","edge"],"paramsTotal":5123178051,"paramsActiveB":null,"activeSource":null,"moe":false,"nativeDtypes":["BF16"],"nativeQuant":null,"checkpointGB":10.25,"maxContext":131072,"license":"apache-2.0","released":"2026-03-02","downloads":2668931,"kv":{"mode":"gqa","fullLayers":7,"slidingLayers":28,"window":512,"kvHeads":1,"headDim":256},"url":"https://canirun.branolabs.com/model/gemma-4-e2b","runsIn":[{"tool":"ollama","how":"ollama run gemma4:e2b","url":"https://ollama.com/library/gemma4","status":"ok"},{"tool":"lmstudio","how":"lmstudio-community build: lmstudio-community/gemma-4-E2B-it-MLX-4bit","url":"https://huggingface.co/lmstudio-community/gemma-4-E2B-it-MLX-4bit","status":"ok"},{"tool":"llamacpp","how":"GGUF: ggml-org/gemma-4-E2B-it-GGUF","url":"https://huggingface.co/ggml-org/gemma-4-E2B-it-GGUF","status":"ok"},{"tool":"mlx","how":"MLX conversion: mlx-community/gemma-4-e2b-it-4bit","url":"https://huggingface.co/mlx-community/gemma-4-e2b-it-4bit","status":"ok"}],"estimatesGB":{"Q4_K_M":{"ctx8192":{"total":4.3,"weights":3.13,"kv":0.07,"overhead":1.09},"ctx32768":{"total":4.48,"weights":3.13,"kv":0.25,"overhead":1.09}},"Q8_0":{"ctx8192":{"total":6.68,"weights":5.44,"kv":0.07,"overhead":1.16},"ctx32768":{"total":6.86,"weights":5.44,"kv":0.25,"overhead":1.16}}}},{"slug":"ling-3-0-tiny","name":"Ling-3.0-tiny 7.9B-A1.3B","org":"inclusionAI (Ant Group)","repo":"inclusionAI/Ling-3.0-tiny","dataRepo":"inclusionAI/Ling-3.0-tiny","use":["chat","reasoning","edge"],"paramsTotal":7893392800,"paramsActiveB":1.3,"activeSource":"model card (7.9B total, 1.3B activated)","moe":true,"nativeDtypes":["F32","BF16"],"nativeQuant":null,"checkpointGB":15.79,"maxContext":131072,"license":"mit","released":"2026-08-10","downloads":11738,"kv":{"mode":"mla","layers":6,"dim":576,"note":"Hybrid model: only the latent-attention (MLA) layers keep a growing KV cache; the linear-attention layers keep a small fixed state (not counted)."},"url":"https://canirun.branolabs.com/model/ling-3-0-tiny","runsIn":[{"tool":"llamacpp","how":"GGUF: bartowski/Ling-3.0-tiny-GGUF","url":"https://huggingface.co/bartowski/Ling-3.0-tiny-GGUF","status":"ok"},{"tool":"vllm","how":"named in the official model card","url":"https://huggingface.co/inclusionAI/Ling-3.0-tiny","status":"ok"},{"tool":"sglang","how":"named in the official model card","url":"https://huggingface.co/inclusionAI/Ling-3.0-tiny","status":"ok"}],"estimatesGB":{"Q4_K_M":{"ctx8192":{"total":6.03,"weights":4.83,"kv":0.06,"overhead":1.14},"ctx32768":{"total":6.2,"weights":4.83,"kv":0.23,"overhead":1.14}},"Q8_0":{"ctx8192":{"total":9.7,"weights":8.39,"kv":0.06,"overhead":1.25},"ctx32768":{"total":9.87,"weights":8.39,"kv":0.23,"overhead":1.25}}}},{"slug":"gemma-4-e4b","name":"Gemma 4 E4B","org":"Google","repo":"google/gemma-4-E4B-it","dataRepo":"google/gemma-4-E4B-it","use":["chat","vision","edge"],"paramsTotal":7996156490,"paramsActiveB":null,"activeSource":null,"moe":false,"nativeDtypes":["BF16"],"nativeQuant":null,"checkpointGB":15.99,"maxContext":131072,"license":"apache-2.0","released":"2026-03-02","downloads":4205756,"kv":{"mode":"gqa","fullLayers":7,"slidingLayers":35,"window":512,"kvHeads":2,"headDim":256},"url":"https://canirun.branolabs.com/model/gemma-4-e4b","runsIn":[{"tool":"ollama","how":"ollama run gemma4:e4b","url":"https://ollama.com/library/gemma4","status":"ok"},{"tool":"lmstudio","how":"lmstudio-community build: lmstudio-community/gemma-4-E4B-it-MLX-4bit","url":"https://huggingface.co/lmstudio-community/gemma-4-E4B-it-MLX-4bit","status":"ok"},{"tool":"llamacpp","how":"GGUF: ggml-org/gemma-4-E4B-it-GGUF","url":"https://huggingface.co/ggml-org/gemma-4-E4B-it-GGUF","status":"ok"},{"tool":"mlx","how":"MLX conversion: mlx-community/gemma-4-e4b-it-4bit","url":"https://huggingface.co/mlx-community/gemma-4-e4b-it-4bit","status":"ok"}],"estimatesGB":{"Q4_K_M":{"ctx8192":{"total":6.19,"weights":4.89,"kv":0.15,"overhead":1.15},"ctx32768":{"total":6.55,"weights":4.89,"kv":0.51,"overhead":1.15}},"Q8_0":{"ctx8192":{"total":9.91,"weights":8.5,"kv":0.15,"overhead":1.25},"ctx32768":{"total":10.26,"weights":8.5,"kv":0.51,"overhead":1.25}}}},{"slug":"llama-3-1-8b","name":"Llama 3.1 8B","org":"Meta","repo":"meta-llama/Llama-3.1-8B-Instruct","dataRepo":"unsloth/Llama-3.1-8B-Instruct","use":["chat"],"paramsTotal":8030261248,"paramsActiveB":null,"activeSource":null,"moe":false,"nativeDtypes":["BF16"],"nativeQuant":null,"checkpointGB":16.06,"maxContext":131072,"license":"llama3.1","released":"2025-02-15","downloads":166887,"kv":{"mode":"gqa","fullLayers":32,"slidingLayers":0,"window":null,"kvHeads":8,"headDim":128},"url":"https://canirun.branolabs.com/model/llama-3-1-8b","runsIn":[{"tool":"ollama","how":"ollama run llama3.1:8b","url":"https://ollama.com/library/llama3.1","status":"ok"},{"tool":"lmstudio","how":"lmstudio-community build: lmstudio-community/Meta-Llama-3.1-8B-Instruct-GGUF","url":"https://huggingface.co/lmstudio-community/Meta-Llama-3.1-8B-Instruct-GGUF","status":"ok"},{"tool":"llamacpp","how":"GGUF: ggml-org/Meta-Llama-3.1-8B-Instruct-Q4_0-GGUF","url":"https://huggingface.co/ggml-org/Meta-Llama-3.1-8B-Instruct-Q4_0-GGUF","status":"ok"},{"tool":"mlx","how":"MLX conversion: mlx-community/Llama-3.1-8B-Instruct-4bit","url":"https://huggingface.co/mlx-community/Llama-3.1-8B-Instruct-4bit","status":"ok"}],"estimatesGB":{"Q4_K_M":{"ctx8192":{"total":7.13,"weights":4.91,"kv":1.07,"overhead":1.15},"ctx32768":{"total":10.36,"weights":4.91,"kv":4.29,"overhead":1.15}},"Q8_0":{"ctx8192":{"total":10.86,"weights":8.53,"kv":1.07,"overhead":1.26},"ctx32768":{"total":14.08,"weights":8.53,"kv":4.29,"overhead":1.26}}}},{"slug":"qwen3-8b","name":"Qwen3 8B","org":"Qwen","repo":"Qwen/Qwen3-8B","dataRepo":"Qwen/Qwen3-8B","use":["chat"],"paramsTotal":8190735360,"paramsActiveB":null,"activeSource":null,"moe":false,"nativeDtypes":["BF16"],"nativeQuant":null,"checkpointGB":16.38,"maxContext":40960,"license":"apache-2.0","released":"2025-04-27","downloads":9336872,"kv":{"mode":"gqa","fullLayers":36,"slidingLayers":0,"window":null,"kvHeads":8,"headDim":128},"url":"https://canirun.branolabs.com/model/qwen3-8b","runsIn":[{"tool":"ollama","how":"ollama run qwen3:8b","url":"https://ollama.com/library/qwen3","status":"ok"},{"tool":"llamacpp","how":"GGUF: ggml-org/Qwen3-8B-GGUF","url":"https://huggingface.co/ggml-org/Qwen3-8B-GGUF","status":"ok"},{"tool":"mlx","how":"MLX conversion: mlx-community/Qwen3-8B-4bit","url":"https://huggingface.co/mlx-community/Qwen3-8B-4bit","status":"ok"},{"tool":"ktransformers","how":"named in the official model card","url":"https://huggingface.co/Qwen/Qwen3-8B","status":"ok"},{"tool":"vllm","how":"named in the official model card","url":"https://huggingface.co/Qwen/Qwen3-8B","status":"ok"},{"tool":"sglang","how":"named in the official model card","url":"https://huggingface.co/Qwen/Qwen3-8B","status":"ok"}],"estimatesGB":{"Q4_K_M":{"ctx8192":{"total":7.37,"weights":5.01,"kv":1.21,"overhead":1.15},"ctx32768":{"total":10.99,"weights":5.01,"kv":4.83,"overhead":1.15}},"Q8_0":{"ctx8192":{"total":11.17,"weights":8.7,"kv":1.21,"overhead":1.26},"ctx32768":{"total":14.8,"weights":8.7,"kv":4.83,"overhead":1.26}}}},{"slug":"qwen3-5-9b","name":"Qwen3.5 9B","org":"Qwen","repo":"Qwen/Qwen3.5-9B","dataRepo":"Qwen/Qwen3.5-9B","use":["chat","vision"],"paramsTotal":9653104368,"paramsActiveB":null,"activeSource":null,"moe":false,"nativeDtypes":["BF16","F32"],"nativeQuant":null,"checkpointGB":19.31,"maxContext":262144,"license":"apache-2.0","released":"2026-02-27","downloads":8262238,"kv":{"mode":"gqa","fullLayers":8,"slidingLayers":0,"window":null,"kvHeads":4,"headDim":256,"note":"Hybrid model: only the full-attention layers keep a growing KV cache; linear/conv layers keep a small fixed state (not counted)."},"url":"https://canirun.branolabs.com/model/qwen3-5-9b","runsIn":[{"tool":"ollama","how":"ollama run qwen3.5:9b","url":"https://ollama.com/library/qwen3.5","status":"ok"},{"tool":"lmstudio","how":"lmstudio-community build: lmstudio-community/Qwen3.5-9B-MLX-4bit","url":"https://huggingface.co/lmstudio-community/Qwen3.5-9B-MLX-4bit","status":"ok"},{"tool":"llamacpp","how":"GGUF: bartowski/Qwen_Qwen3.5-9B-GGUF","url":"https://huggingface.co/bartowski/Qwen_Qwen3.5-9B-GGUF","status":"ok"},{"tool":"mlx","how":"MLX conversion: mlx-community/Qwen3.5-9B-MLX-4bit","url":"https://huggingface.co/mlx-community/Qwen3.5-9B-MLX-4bit","status":"ok"},{"tool":"ktransformers","how":"named in the official model card","url":"https://huggingface.co/Qwen/Qwen3.5-9B","status":"ok"},{"tool":"vllm","how":"named in the official model card","url":"https://huggingface.co/Qwen/Qwen3.5-9B","status":"ok"},{"tool":"sglang","how":"named in the official model card","url":"https://huggingface.co/Qwen/Qwen3.5-9B","status":"ok"}],"estimatesGB":{"Q4_K_M":{"ctx8192":{"total":7.35,"weights":5.91,"kv":0.27,"overhead":1.18},"ctx32768":{"total":8.16,"weights":5.91,"kv":1.07,"overhead":1.18}},"Q8_0":{"ctx8192":{"total":11.83,"weights":10.26,"kv":0.27,"overhead":1.31},"ctx32768":{"total":12.64,"weights":10.26,"kv":1.07,"overhead":1.31}}}},{"slug":"gemma-4-12b","name":"Gemma 4 12B","org":"Google","repo":"google/gemma-4-12B-it","dataRepo":"google/gemma-4-12B-it","use":["chat","vision"],"paramsTotal":11959730224,"paramsActiveB":null,"activeSource":null,"moe":false,"nativeDtypes":["BF16"],"nativeQuant":null,"checkpointGB":23.92,"maxContext":262144,"license":"apache-2.0","released":"2026-05-23","downloads":1675469,"kv":{"mode":"gqa","fullLayers":8,"slidingLayers":40,"window":1024,"kvHeads":8,"headDim":256},"url":"https://canirun.branolabs.com/model/gemma-4-12b","runsIn":[{"tool":"ollama","how":"ollama run gemma4:12b","url":"https://ollama.com/library/gemma4","status":"ok"},{"tool":"lmstudio","how":"lmstudio-community build: lmstudio-community/gemma-4-12B-it-MLX-4bit","url":"https://huggingface.co/lmstudio-community/gemma-4-12B-it-MLX-4bit","status":"ok"},{"tool":"llamacpp","how":"GGUF: ggml-org/gemma-4-12B-it-GGUF","url":"https://huggingface.co/ggml-org/gemma-4-12B-it-GGUF","status":"ok"},{"tool":"mlx","how":"MLX conversion: mlx-community/gemma-4-12B-it-qat-4bit","url":"https://huggingface.co/mlx-community/gemma-4-12B-it-qat-4bit","status":"ok"}],"estimatesGB":{"Q4_K_M":{"ctx8192":{"total":9.41,"weights":7.32,"kv":0.87,"overhead":1.22},"ctx32768":{"total":11.02,"weights":7.32,"kv":2.48,"overhead":1.22}},"Q8_0":{"ctx8192":{"total":14.96,"weights":12.71,"kv":0.87,"overhead":1.38},"ctx32768":{"total":16.57,"weights":12.71,"kv":2.48,"overhead":1.38}}}},{"slug":"phi-4","name":"Phi-4 14B","org":"Microsoft","repo":"microsoft/phi-4","dataRepo":"microsoft/phi-4","use":["chat","reasoning"],"paramsTotal":14659507200,"paramsActiveB":null,"activeSource":null,"moe":false,"nativeDtypes":["BF16"],"nativeQuant":null,"checkpointGB":29.32,"maxContext":16384,"license":"mit","released":"2024-12-11","downloads":355684,"kv":{"mode":"gqa","fullLayers":40,"slidingLayers":0,"window":null,"kvHeads":10,"headDim":128},"url":"https://canirun.branolabs.com/model/phi-4","runsIn":[{"tool":"ollama","how":"ollama run phi4:14b","url":"https://ollama.com/library/phi4","status":"ok"},{"tool":"lmstudio","how":"lmstudio-community build: lmstudio-community/phi-4-GGUF","url":"https://huggingface.co/lmstudio-community/phi-4-GGUF","status":"ok"},{"tool":"llamacpp","how":"GGUF: bartowski/phi-4-GGUF","url":"https://huggingface.co/bartowski/phi-4-GGUF","status":"ok"},{"tool":"mlx","how":"MLX conversion: mlx-community/phi-4-4bit","url":"https://huggingface.co/mlx-community/phi-4-4bit","status":"ok"}],"estimatesGB":{"Q4_K_M":{"ctx8192":{"total":11.92,"weights":8.97,"kv":1.68,"overhead":1.27},"ctx32768":{"total":16.95,"weights":8.97,"kv":6.71,"overhead":1.27}},"Q8_0":{"ctx8192":{"total":18.72,"weights":15.58,"kv":1.68,"overhead":1.47},"ctx32768":{"total":23.76,"weights":15.58,"kv":6.71,"overhead":1.47}}}},{"slug":"gpt-oss-20b","name":"gpt-oss-20b","org":"OpenAI","repo":"openai/gpt-oss-20b","dataRepo":"openai/gpt-oss-20b","use":["chat","reasoning"],"paramsTotal":20914757184,"paramsActiveB":3.6,"activeSource":"model card","moe":true,"nativeDtypes":["BF16","U8"],"nativeQuant":"mxfp4","checkpointGB":13.76,"maxContext":131072,"license":"apache-2.0","released":"2025-08-04","downloads":5978429,"kv":{"mode":"gqa","fullLayers":12,"slidingLayers":12,"window":128,"kvHeads":8,"headDim":64},"url":"https://canirun.branolabs.com/model/gpt-oss-20b","runsIn":[{"tool":"ollama","how":"ollama run gpt-oss:20b","url":"https://ollama.com/library/gpt-oss","status":"ok"},{"tool":"lmstudio","how":"lmstudio-community build: lmstudio-community/gpt-oss-20b-GGUF","url":"https://huggingface.co/lmstudio-community/gpt-oss-20b-GGUF","status":"ok"},{"tool":"llamacpp","how":"GGUF: ggml-org/gpt-oss-20b-GGUF","url":"https://huggingface.co/ggml-org/gpt-oss-20b-GGUF","status":"ok"},{"tool":"mlx","how":"MLX conversion: mlx-community/gpt-oss-20b-MXFP4-Q8","url":"https://huggingface.co/mlx-community/gpt-oss-20b-MXFP4-Q8","status":"ok"},{"tool":"vllm","how":"named in the official model card","url":"https://huggingface.co/openai/gpt-oss-20b","status":"ok"}],"estimatesGB":{"Q4_K_M":{"ctx8192":{"total":14.38,"weights":12.8,"kv":0.2,"overhead":1.38},"ctx32768":{"total":14.99,"weights":12.8,"kv":0.81,"overhead":1.38}},"Q8_0":{"ctx8192":{"total":24.1,"weights":22.22,"kv":0.2,"overhead":1.67},"ctx32768":{"total":24.7,"weights":22.22,"kv":0.81,"overhead":1.67}}}},{"slug":"lfm2-5-24b-a2b","name":"LFM2.5 24B-A2B","org":"Liquid AI","repo":"LiquidAI/LFM2.5-24B-A2B","dataRepo":"LiquidAI/LFM2.5-24B-A2B","use":["chat","agents","edge"],"paramsTotal":23971585280,"paramsActiveB":2,"activeSource":"model card","moe":true,"nativeDtypes":["F32","BF16"],"nativeQuant":null,"checkpointGB":47.94,"maxContext":131072,"license":"other","released":"2026-10-12","downloads":11926,"kv":{"mode":"gqa","fullLayers":10,"slidingLayers":0,"window":null,"kvHeads":8,"headDim":64,"note":"Hybrid model: only the full-attention layers keep a growing KV cache; linear/conv layers keep a small fixed state (not counted)."},"url":"https://canirun.branolabs.com/model/lfm2-5-24b-a2b","runsIn":[{"tool":"ollama","how":"ollama run lfm2.5:24b","url":"https://ollama.com/library/lfm2.5","status":"ok"},{"tool":"llamacpp","how":"GGUF: ggml-org/LFM2.5-24B-A2B-GGUF","url":"https://huggingface.co/ggml-org/LFM2.5-24B-A2B-GGUF","status":"ok"},{"tool":"mlx","how":"MLX conversion: mlx-community/LFM2.5-24B-A2B-Preview-bf16","url":"https://huggingface.co/mlx-community/LFM2.5-24B-A2B-Preview-bf16","status":"ok"},{"tool":"vllm","how":"named in the official model card","url":"https://huggingface.co/LiquidAI/LFM2.5-24B-A2B","status":"ok"},{"tool":"sglang","how":"named in the official model card","url":"https://huggingface.co/LiquidAI/LFM2.5-24B-A2B","status":"ok"}],"estimatesGB":{"Q4_K_M":{"ctx8192":{"total":16.27,"weights":14.67,"kv":0.17,"overhead":1.44},"ctx32768":{"total":16.78,"weights":14.67,"kv":0.67,"overhead":1.44}},"Q8_0":{"ctx8192":{"total":27.4,"weights":25.47,"kv":0.17,"overhead":1.76},"ctx32768":{"total":27.91,"weights":25.47,"kv":0.67,"overhead":1.76}}}},{"slug":"devstral-small-2-24b","name":"Devstral Small 2 24B","org":"Mistral AI","repo":"mistralai/Devstral-Small-2-24B-Instruct-2512","dataRepo":"mistralai/Devstral-Small-2-24B-Instruct-2512","use":["coding"],"paramsTotal":24011361840,"paramsActiveB":null,"activeSource":null,"moe":false,"nativeDtypes":["BF16","F8_E4M3"],"nativeQuant":"fp8","checkpointGB":51.59,"maxContext":393216,"license":"apache-2.0","released":"2025-11-28","downloads":291025,"kv":{"mode":"gqa","fullLayers":40,"slidingLayers":0,"window":null,"kvHeads":8,"headDim":128},"url":"https://canirun.branolabs.com/model/devstral-small-2-24b","runsIn":[{"tool":"ollama","how":"ollama run devstral-small-2:24b","url":"https://ollama.com/library/devstral-small-2","status":"ok"},{"tool":"lmstudio","how":"lmstudio-community build: lmstudio-community/Devstral-Small-2-24B-Instruct-2512-GGUF","url":"https://huggingface.co/lmstudio-community/Devstral-Small-2-24B-Instruct-2512-GGUF","status":"ok"},{"tool":"llamacpp","how":"GGUF: ggml-org/Devstral-Small-2-24B-Instruct-2512-GGUF","url":"https://huggingface.co/ggml-org/Devstral-Small-2-24B-Instruct-2512-GGUF","status":"ok"},{"tool":"mlx","how":"MLX conversion: mlx-community/Devstral-Small-2-24B-Instruct-2512-4bit","url":"https://huggingface.co/mlx-community/Devstral-Small-2-24B-Instruct-2512-4bit","status":"ok"},{"tool":"vllm","how":"named in the official model card","url":"https://huggingface.co/mistralai/Devstral-Small-2-24B-Instruct-2512","status":"ok"},{"tool":"sglang","how":"named in the official model card","url":"https://huggingface.co/mistralai/Devstral-Small-2-24B-Instruct-2512","status":"ok"}],"estimatesGB":{"Q4_K_M":{"ctx8192":{"total":17.47,"weights":14.69,"kv":1.34,"overhead":1.44},"ctx32768":{"total":21.5,"weights":14.69,"kv":5.37,"overhead":1.44}},"Q8_0":{"ctx8192":{"total":28.62,"weights":25.51,"kv":1.34,"overhead":1.77},"ctx32768":{"total":32.65,"weights":25.51,"kv":5.37,"overhead":1.77}}}},{"slug":"gemma-4-26b-a4b","name":"Gemma 4 26B-A4B","org":"Google","repo":"google/gemma-4-26B-A4B-it","dataRepo":"google/gemma-4-26B-A4B-it","use":["chat","vision"],"paramsTotal":25805936206,"paramsActiveB":4,"activeSource":"name (A4B)","moe":true,"nativeDtypes":["BF16"],"nativeQuant":null,"checkpointGB":51.61,"maxContext":262144,"license":"apache-2.0","released":"2026-03-11","downloads":10744917,"kv":{"mode":"gqa","fullLayers":5,"slidingLayers":25,"window":1024,"kvHeads":8,"headDim":256},"url":"https://canirun.branolabs.com/model/gemma-4-26b-a4b","runsIn":[{"tool":"ollama","how":"ollama run gemma4:26b-a4b","url":"https://ollama.com/library/gemma4","status":"ok"},{"tool":"lmstudio","how":"lmstudio-community build: lmstudio-community/gemma-4-26B-A4B-it-QAT-MLX-4bit","url":"https://huggingface.co/lmstudio-community/gemma-4-26B-A4B-it-QAT-MLX-4bit","status":"ok"},{"tool":"llamacpp","how":"GGUF: ggml-org/gemma-4-26B-A4B-it-GGUF","url":"https://huggingface.co/ggml-org/gemma-4-26B-A4B-it-GGUF","status":"ok"},{"tool":"mlx","how":"MLX conversion: mlx-community/gemma-4-26b-a4b-it-4bit","url":"https://huggingface.co/mlx-community/gemma-4-26b-a4b-it-4bit","status":"ok"}],"estimatesGB":{"Q4_K_M":{"ctx8192":{"total":17.81,"weights":15.79,"kv":0.55,"overhead":1.47},"ctx32768":{"total":18.81,"weights":15.79,"kv":1.55,"overhead":1.47}},"Q8_0":{"ctx8192":{"total":29.79,"weights":27.42,"kv":0.55,"overhead":1.82},"ctx32768":{"total":30.8,"weights":27.42,"kv":1.55,"overhead":1.82}}}},{"slug":"qwen3-8-27b","name":"Qwen3.8 27B","org":"Qwen","repo":"Qwen/Qwen3.8-27B","dataRepo":"Qwen/Qwen3.8-27B","use":["chat","coding","vision"],"paramsTotal":27781427952,"paramsActiveB":null,"activeSource":null,"moe":false,"nativeDtypes":["BF16"],"nativeQuant":null,"checkpointGB":55.56,"maxContext":262144,"license":"apache-2.0","released":"2026-08-05","downloads":7795543,"kv":{"mode":"gqa","fullLayers":16,"slidingLayers":0,"window":null,"kvHeads":4,"headDim":256,"note":"Hybrid model: only the full-attention layers keep a growing KV cache; linear/conv layers keep a small fixed state (not counted)."},"url":"https://canirun.branolabs.com/model/qwen3-8-27b","runsIn":[{"tool":"colibri","how":"Qwen3.6 engine, converted to int4/int8 (20-30 GB RAM)","url":"https://github.com/JustVugg/colibri#which-model-for-my-machine","status":"ok"},{"tool":"ollama","how":"ollama run qwen3.8:27b","url":"https://ollama.com/library/qwen3.8","status":"ok"},{"tool":"lmstudio","how":"lmstudio-community build: lmstudio-community/Qwen3.8-27B-MLX-4bit","url":"https://huggingface.co/lmstudio-community/Qwen3.8-27B-MLX-4bit","status":"ok"},{"tool":"llamacpp","how":"GGUF: ggml-org/Qwen3.8-27B-GGUF","url":"https://huggingface.co/ggml-org/Qwen3.8-27B-GGUF","status":"ok"},{"tool":"mlx","how":"MLX conversion: mlx-community/Qwen3.8-27B-4bit","url":"https://huggingface.co/mlx-community/Qwen3.8-27B-4bit","status":"ok"},{"tool":"vllm","how":"named in the official model card","url":"https://huggingface.co/Qwen/Qwen3.8-27B","status":"ok"},{"tool":"sglang","how":"named in the official model card","url":"https://huggingface.co/Qwen/Qwen3.8-27B","status":"ok"},{"tool":"tokenspeed","how":"named in the official model card","url":"https://huggingface.co/Qwen/Qwen3.8-27B","status":"ok"}],"estimatesGB":{"Q4_K_M":{"ctx8192":{"total":19.04,"weights":17,"kv":0.54,"overhead":1.51},"ctx32768":{"total":20.65,"weights":17,"kv":2.15,"overhead":1.51}},"Q8_0":{"ctx8192":{"total":31.94,"weights":29.52,"kv":0.54,"overhead":1.89},"ctx32768":{"total":33.55,"weights":29.52,"kv":2.15,"overhead":1.89}}}},{"slug":"qwen3-6-27b","name":"Qwen3.6 27B","org":"Qwen","repo":"Qwen/Qwen3.6-27B","dataRepo":"Qwen/Qwen3.6-27B","use":["chat","coding","vision"],"paramsTotal":27781427952,"paramsActiveB":null,"activeSource":null,"moe":false,"nativeDtypes":["BF16"],"nativeQuant":null,"checkpointGB":55.56,"maxContext":262144,"license":"apache-2.0","released":"2026-04-21","downloads":2298346,"kv":{"mode":"gqa","fullLayers":16,"slidingLayers":0,"window":null,"kvHeads":4,"headDim":256,"note":"Hybrid model: only the full-attention layers keep a growing KV cache; linear/conv layers keep a small fixed state (not counted)."},"url":"https://canirun.branolabs.com/model/qwen3-6-27b","runsIn":[{"tool":"ollama","how":"ollama run qwen3.6:27b","url":"https://ollama.com/library/qwen3.6","status":"ok"},{"tool":"lmstudio","how":"lmstudio-community build: lmstudio-community/Qwen3.6-27B-MLX-4bit","url":"https://huggingface.co/lmstudio-community/Qwen3.6-27B-MLX-4bit","status":"ok"},{"tool":"llamacpp","how":"GGUF: ggml-org/Qwen3.6-27B-GGUF","url":"https://huggingface.co/ggml-org/Qwen3.6-27B-GGUF","status":"ok"},{"tool":"mlx","how":"MLX conversion: mlx-community/Qwen3.6-27B-4bit","url":"https://huggingface.co/mlx-community/Qwen3.6-27B-4bit","status":"ok"},{"tool":"ktransformers","how":"named in the official model card","url":"https://huggingface.co/Qwen/Qwen3.6-27B","status":"ok"},{"tool":"vllm","how":"named in the official model card","url":"https://huggingface.co/Qwen/Qwen3.6-27B","status":"ok"},{"tool":"sglang","how":"named in the official model card","url":"https://huggingface.co/Qwen/Qwen3.6-27B","status":"ok"}],"estimatesGB":{"Q4_K_M":{"ctx8192":{"total":19.04,"weights":17,"kv":0.54,"overhead":1.51},"ctx32768":{"total":20.65,"weights":17,"kv":2.15,"overhead":1.51}},"Q8_0":{"ctx8192":{"total":31.94,"weights":29.52,"kv":0.54,"overhead":1.89},"ctx32768":{"total":33.55,"weights":29.52,"kv":2.15,"overhead":1.89}}}},{"slug":"qwen3-coder-30b-a3b","name":"Qwen3-Coder 30B-A3B","org":"Qwen","repo":"Qwen/Qwen3-Coder-30B-A3B-Instruct","dataRepo":"Qwen/Qwen3-Coder-30B-A3B-Instruct","use":["coding"],"paramsTotal":30532122624,"paramsActiveB":3.3,"activeSource":"model card","moe":true,"nativeDtypes":["BF16"],"nativeQuant":null,"checkpointGB":61.07,"maxContext":262144,"license":"apache-2.0","released":"2025-07-31","downloads":468166,"kv":{"mode":"gqa","fullLayers":48,"slidingLayers":0,"window":null,"kvHeads":4,"headDim":128},"url":"https://canirun.branolabs.com/model/qwen3-coder-30b-a3b","runsIn":[{"tool":"colibri","how":"coli setup --model qwen3-coder-30b (runs from RAM; 8/18 GB)","url":"https://github.com/JustVugg/colibri#which-model-for-my-machine","status":"ok"},{"tool":"ollama","how":"ollama run qwen3-coder:30b","url":"https://ollama.com/library/qwen3-coder","status":"ok"},{"tool":"lmstudio","how":"lmstudio-community build: lmstudio-community/Qwen3-Coder-30B-A3B-Instruct-MLX-4bit","url":"https://huggingface.co/lmstudio-community/Qwen3-Coder-30B-A3B-Instruct-MLX-4bit","status":"ok"},{"tool":"llamacpp","how":"GGUF: ggml-org/Qwen3-Coder-30B-A3B-Instruct-Q8_0-GGUF","url":"https://huggingface.co/ggml-org/Qwen3-Coder-30B-A3B-Instruct-Q8_0-GGUF","status":"ok"},{"tool":"mlx","how":"MLX conversion: mlx-community/Qwen3-Coder-30B-A3B-Instruct-4bit","url":"https://huggingface.co/mlx-community/Qwen3-Coder-30B-A3B-Instruct-4bit","status":"ok"},{"tool":"ktransformers","how":"named in the official model card","url":"https://huggingface.co/Qwen/Qwen3-Coder-30B-A3B-Instruct","status":"ok"}],"estimatesGB":{"Q4_K_M":{"ctx8192":{"total":21.05,"weights":18.68,"kv":0.81,"overhead":1.56},"ctx32768":{"total":23.46,"weights":18.68,"kv":3.22,"overhead":1.56}},"Q8_0":{"ctx8192":{"total":35.22,"weights":32.44,"kv":0.81,"overhead":1.97},"ctx32768":{"total":37.64,"weights":32.44,"kv":3.22,"overhead":1.97}}}},{"slug":"glm-4-7-flash","name":"GLM-4.7-Flash 30B-A3B","org":"Z.ai","repo":"zai-org/GLM-4.7-Flash","dataRepo":"zai-org/GLM-4.7-Flash","use":["chat","coding"],"paramsTotal":31221488576,"paramsActiveB":3,"activeSource":"model card (30B-A3B)","moe":true,"nativeDtypes":["BF16","F32"],"nativeQuant":null,"checkpointGB":62.44,"maxContext":202752,"license":"mit","released":"2026-01-19","downloads":1223759,"kv":{"mode":"mla","layers":47,"dim":576},"url":"https://canirun.branolabs.com/model/glm-4-7-flash","runsIn":[{"tool":"ollama","how":"ollama run glm-4.7-flash:q4_K_M","url":"https://ollama.com/library/glm-4.7-flash","status":"ok"},{"tool":"lmstudio","how":"lmstudio-community build: lmstudio-community/GLM-4.7-Flash-MLX-8bit","url":"https://huggingface.co/lmstudio-community/GLM-4.7-Flash-MLX-8bit","status":"ok"},{"tool":"llamacpp","how":"GGUF: ggml-org/GLM-4.7-Flash-GGUF","url":"https://huggingface.co/ggml-org/GLM-4.7-Flash-GGUF","status":"ok"},{"tool":"mlx","how":"MLX conversion: mlx-community/GLM-4.7-Flash-4bit","url":"https://huggingface.co/mlx-community/GLM-4.7-Flash-4bit","status":"ok"},{"tool":"vllm","how":"named in the official model card","url":"https://huggingface.co/zai-org/GLM-4.7-Flash","status":"ok"},{"tool":"sglang","how":"named in the official model card","url":"https://huggingface.co/zai-org/GLM-4.7-Flash","status":"ok"}],"estimatesGB":{"Q4_K_M":{"ctx8192":{"total":21.12,"weights":19.1,"kv":0.44,"overhead":1.57},"ctx32768":{"total":22.45,"weights":19.1,"kv":1.77,"overhead":1.57}},"Q8_0":{"ctx8192":{"total":35.61,"weights":33.18,"kv":0.44,"overhead":2},"ctx32768":{"total":36.95,"weights":33.18,"kv":1.77,"overhead":2}}}},{"slug":"gemma-4-31b","name":"Gemma 4 31B","org":"Google","repo":"google/gemma-4-31B-it","dataRepo":"google/gemma-4-31B-it","use":["chat","vision"],"paramsTotal":31273088876,"paramsActiveB":null,"activeSource":null,"moe":false,"nativeDtypes":["BF16"],"nativeQuant":null,"checkpointGB":62.55,"maxContext":262144,"license":"apache-2.0","released":"2026-03-11","downloads":9113281,"kv":{"mode":"gqa","fullLayers":10,"slidingLayers":50,"window":1024,"kvHeads":16,"headDim":256},"url":"https://canirun.branolabs.com/model/gemma-4-31b","runsIn":[{"tool":"ollama","how":"ollama run gemma4:31b","url":"https://ollama.com/library/gemma4","status":"ok"},{"tool":"lmstudio","how":"lmstudio-community build: lmstudio-community/gemma-4-31B-it-MLX-8bit","url":"https://huggingface.co/lmstudio-community/gemma-4-31B-it-MLX-8bit","status":"ok"},{"tool":"llamacpp","how":"GGUF: ggml-org/gemma-4-31B-it-GGUF","url":"https://huggingface.co/ggml-org/gemma-4-31B-it-GGUF","status":"ok"},{"tool":"mlx","how":"MLX conversion: mlx-community/gemma-4-31b-it-4bit","url":"https://huggingface.co/mlx-community/gemma-4-31b-it-4bit","status":"ok"}],"estimatesGB":{"Q4_K_M":{"ctx8192":{"total":22.89,"weights":19.13,"kv":2.18,"overhead":1.57},"ctx32768":{"total":26.91,"weights":19.13,"kv":6.21,"overhead":1.57}},"Q8_0":{"ctx8192":{"total":37.41,"weights":33.23,"kv":2.18,"overhead":2},"ctx32768":{"total":41.44,"weights":33.23,"kv":6.21,"overhead":2}}}},{"slug":"nemotron-3-nano-30b-a3b","name":"Nemotron 3 Nano 30B-A3B","org":"NVIDIA","repo":"nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16","dataRepo":"nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16","use":["chat","reasoning"],"paramsTotal":31577937344,"paramsActiveB":3,"activeSource":"name (A3B)","moe":true,"nativeDtypes":["F32","BF16"],"nativeQuant":null,"checkpointGB":63.16,"maxContext":262144,"license":"other","released":"2025-12-04","downloads":1164980,"kv":{"mode":"gqa","fullLayers":6,"slidingLayers":0,"window":null,"kvHeads":2,"headDim":128,"note":"Hybrid model: only the attention layers keep a growing KV cache; the other layers keep a small fixed state (not counted)."},"url":"https://canirun.branolabs.com/model/nemotron-3-nano-30b-a3b","runsIn":[{"tool":"ollama","how":"ollama run nemotron-3-nano:30b","url":"https://ollama.com/library/nemotron-3-nano","status":"ok"},{"tool":"lmstudio","how":"lmstudio-community build: lmstudio-community/NVIDIA-Nemotron-3-Nano-30B-A3B-MLX-4bit","url":"https://huggingface.co/lmstudio-community/NVIDIA-Nemotron-3-Nano-30B-A3B-MLX-4bit","status":"ok"},{"tool":"llamacpp","how":"GGUF: ggml-org/NVIDIA-Nemotron-3-Nano-30B-A3B-GGUF","url":"https://huggingface.co/ggml-org/NVIDIA-Nemotron-3-Nano-30B-A3B-GGUF","status":"ok"},{"tool":"mlx","how":"MLX conversion: mlx-community/NVIDIA-Nemotron-3-Nano-30B-A3B-MLX-6Bit","url":"https://huggingface.co/mlx-community/NVIDIA-Nemotron-3-Nano-30B-A3B-MLX-6Bit","status":"ok"},{"tool":"vllm","how":"named in the official model card","url":"https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16","status":"ok"},{"tool":"sglang","how":"named in the official model card","url":"https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16","status":"ok"}],"estimatesGB":{"Q4_K_M":{"ctx8192":{"total":20.95,"weights":19.32,"kv":0.05,"overhead":1.58},"ctx32768":{"total":21.1,"weights":19.32,"kv":0.2,"overhead":1.58}},"Q8_0":{"ctx8192":{"total":35.61,"weights":33.55,"kv":0.05,"overhead":2.01},"ctx32768":{"total":35.76,"weights":33.55,"kv":0.2,"overhead":2.01}}}},{"slug":"qwen3-32b","name":"Qwen3 32B","org":"Qwen","repo":"Qwen/Qwen3-32B","dataRepo":"Qwen/Qwen3-32B","use":["chat","coding"],"paramsTotal":32762123264,"paramsActiveB":null,"activeSource":null,"moe":false,"nativeDtypes":["BF16"],"nativeQuant":null,"checkpointGB":65.52,"maxContext":40960,"license":"apache-2.0","released":"2025-04-27","downloads":3041641,"kv":{"mode":"gqa","fullLayers":64,"slidingLayers":0,"window":null,"kvHeads":8,"headDim":128},"url":"https://canirun.branolabs.com/model/qwen3-32b","runsIn":[{"tool":"ollama","how":"ollama run qwen3:32b","url":"https://ollama.com/library/qwen3","status":"ok"},{"tool":"lmstudio","how":"lmstudio-community build: lmstudio-community/Qwen3-32B-MLX-4bit","url":"https://huggingface.co/lmstudio-community/Qwen3-32B-MLX-4bit","status":"ok"},{"tool":"llamacpp","how":"GGUF: ggml-org/Qwen3-32B-GGUF","url":"https://huggingface.co/ggml-org/Qwen3-32B-GGUF","status":"ok"},{"tool":"mlx","how":"MLX conversion: mlx-community/Qwen3-32B-4bit","url":"https://huggingface.co/mlx-community/Qwen3-32B-4bit","status":"ok"},{"tool":"ktransformers","how":"named in the official model card","url":"https://huggingface.co/Qwen/Qwen3-32B","status":"ok"},{"tool":"vllm","how":"named in the official model card","url":"https://huggingface.co/Qwen/Qwen3-32B","status":"ok"},{"tool":"sglang","how":"named in the official model card","url":"https://huggingface.co/Qwen/Qwen3-32B","status":"ok"}],"estimatesGB":{"Q4_K_M":{"ctx8192":{"total":23.79,"weights":20.04,"kv":2.15,"overhead":1.6},"ctx32768":{"total":30.24,"weights":20.04,"kv":8.59,"overhead":1.6}},"Q8_0":{"ctx8192":{"total":39,"weights":34.81,"kv":2.15,"overhead":2.04},"ctx32768":{"total":45.45,"weights":34.81,"kv":8.59,"overhead":2.04}}}},{"slug":"qwen3-6-35b-a3b","name":"Qwen3.6 35B-A3B","org":"Qwen","repo":"Qwen/Qwen3.6-35B-A3B","dataRepo":"Qwen/Qwen3.6-35B-A3B","use":["chat","coding","vision"],"paramsTotal":35951822704,"paramsActiveB":3,"activeSource":"name (A3B)","moe":true,"nativeDtypes":["BF16"],"nativeQuant":null,"checkpointGB":71.9,"maxContext":262144,"license":"apache-2.0","released":"2026-04-15","downloads":3743201,"kv":{"mode":"gqa","fullLayers":10,"slidingLayers":0,"window":null,"kvHeads":2,"headDim":256,"note":"Hybrid model: only the full-attention layers keep a growing KV cache; linear/conv layers keep a small fixed state (not counted)."},"url":"https://canirun.branolabs.com/model/qwen3-6-35b-a3b","runsIn":[{"tool":"strata","how":"Supported by Strata; v0.1.45 added image input and several requests at once","url":"https://github.com/Niko1221/Strata/releases/tag/v0.1.45","status":"ok"},{"tool":"colibri","how":"coli setup --model qwen36-35b (runs from RAM; 10/20 GB)","url":"https://github.com/JustVugg/colibri#which-model-for-my-machine","status":"ok"},{"tool":"ollama","how":"ollama run qwen3.6:35b-a3b","url":"https://ollama.com/library/qwen3.6","status":"ok"},{"tool":"lmstudio","how":"lmstudio-community build: lmstudio-community/Qwen3.6-35B-A3B-MLX-4bit","url":"https://huggingface.co/lmstudio-community/Qwen3.6-35B-A3B-MLX-4bit","status":"ok"},{"tool":"llamacpp","how":"GGUF: ggml-org/Qwen3.6-35B-A3B-GGUF","url":"https://huggingface.co/ggml-org/Qwen3.6-35B-A3B-GGUF","status":"ok"},{"tool":"mlx","how":"MLX conversion: mlx-community/Qwen3.6-35B-A3B-4bit","url":"https://huggingface.co/mlx-community/Qwen3.6-35B-A3B-4bit","status":"ok"},{"tool":"ktransformers","how":"named in the official model card","url":"https://huggingface.co/Qwen/Qwen3.6-35B-A3B","status":"ok"},{"tool":"vllm","how":"named in the official model card","url":"https://huggingface.co/Qwen/Qwen3.6-35B-A3B","status":"ok"},{"tool":"sglang","how":"named in the official model card","url":"https://huggingface.co/Qwen/Qwen3.6-35B-A3B","status":"ok"}],"estimatesGB":{"Q4_K_M":{"ctx8192":{"total":23.82,"weights":22,"kv":0.17,"overhead":1.66},"ctx32768":{"total":24.33,"weights":22,"kv":0.67,"overhead":1.66}},"Q8_0":{"ctx8192":{"total":40.52,"weights":38.2,"kv":0.17,"overhead":2.15},"ctx32768":{"total":41.02,"weights":38.2,"kv":0.67,"overhead":2.15}}}},{"slug":"llama-3-3-70b","name":"Llama 3.3 70B","org":"Meta","repo":"meta-llama/Llama-3.3-70B-Instruct","dataRepo":"unsloth/Llama-3.3-70B-Instruct","use":["chat"],"paramsTotal":70553706496,"paramsActiveB":null,"activeSource":null,"moe":false,"nativeDtypes":["BF16"],"nativeQuant":null,"checkpointGB":141.11,"maxContext":131072,"license":"llama3.3","released":"2024-12-06","downloads":47941,"kv":{"mode":"gqa","fullLayers":80,"slidingLayers":0,"window":null,"kvHeads":8,"headDim":128},"url":"https://canirun.branolabs.com/model/llama-3-3-70b","runsIn":[{"tool":"ollama","how":"ollama run llama3.3:70b","url":"https://ollama.com/library/llama3.3","status":"ok"},{"tool":"lmstudio","how":"lmstudio-community build: lmstudio-community/Llama-3.3-70B-Instruct-GGUF","url":"https://huggingface.co/lmstudio-community/Llama-3.3-70B-Instruct-GGUF","status":"ok"},{"tool":"llamacpp","how":"GGUF: bartowski/Llama-3.3-70B-Instruct-GGUF","url":"https://huggingface.co/bartowski/Llama-3.3-70B-Instruct-GGUF","status":"ok"},{"tool":"mlx","how":"MLX conversion: mlx-community/Llama-3.3-70B-Instruct-4bit","url":"https://huggingface.co/mlx-community/Llama-3.3-70B-Instruct-4bit","status":"ok"},{"tool":"vllm","how":"named in the official model card","url":"https://huggingface.co/unsloth/Llama-3.3-70B-Instruct","status":"ok"}],"estimatesGB":{"Q4_K_M":{"ctx8192":{"total":48.14,"weights":43.16,"kv":2.68,"overhead":2.29},"ctx32768":{"total":56.2,"weights":43.16,"kv":10.74,"overhead":2.29}},"Q8_0":{"ctx8192":{"total":80.9,"weights":74.97,"kv":2.68,"overhead":3.25},"ctx32768":{"total":88.96,"weights":74.97,"kv":10.74,"overhead":3.25}}}},{"slug":"llama-4-scout","name":"Llama 4 Scout 17B-16E","org":"Meta","repo":"meta-llama/Llama-4-Scout-17B-16E-Instruct","dataRepo":"unsloth/Llama-4-Scout-17B-16E-Instruct","use":["chat","vision"],"paramsTotal":108641793536,"paramsActiveB":17,"activeSource":"name (17B active)","moe":true,"nativeDtypes":["BF16"],"nativeQuant":null,"checkpointGB":217.28,"maxContext":10485760,"license":"other","released":"2025-04-05","downloads":2839,"kv":{"mode":"gqa","fullLayers":48,"slidingLayers":0,"window":null,"kvHeads":8,"headDim":128},"url":"https://canirun.branolabs.com/model/llama-4-scout","runsIn":[{"tool":"ollama","how":"ollama run llama4:scout","url":"https://ollama.com/library/llama4","status":"ok"},{"tool":"lmstudio","how":"lmstudio-community build: lmstudio-community/Llama-4-Scout-17B-16E-MLX-text-8bit","url":"https://huggingface.co/lmstudio-community/Llama-4-Scout-17B-16E-MLX-text-8bit","status":"ok"},{"tool":"llamacpp","how":"GGUF: ggml-org/Llama-4-Scout-17B-16E-Instruct-GGUF","url":"https://huggingface.co/ggml-org/Llama-4-Scout-17B-16E-Instruct-GGUF","status":"ok"},{"tool":"mlx","how":"MLX conversion: mlx-community/Llama-4-Scout-17B-16E-Instruct-4bit","url":"https://huggingface.co/mlx-community/Llama-4-Scout-17B-16E-Instruct-4bit","status":"ok"}],"estimatesGB":{"Q4_K_M":{"ctx8192":{"total":71.07,"weights":66.47,"kv":1.61,"overhead":2.99},"ctx32768":{"total":75.9,"weights":66.47,"kv":6.44,"overhead":2.99}},"Q8_0":{"ctx8192":{"total":121.52,"weights":115.44,"kv":1.61,"overhead":4.46},"ctx32768":{"total":126.35,"weights":115.44,"kv":6.44,"overhead":4.46}}}},{"slug":"glm-4-5-air","name":"GLM-4.5-Air","org":"Z.ai","repo":"zai-org/GLM-4.5-Air","dataRepo":"zai-org/GLM-4.5-Air","use":["chat","coding"],"paramsTotal":110468824832,"paramsActiveB":12,"activeSource":"model card","moe":true,"nativeDtypes":["BF16","F32"],"nativeQuant":null,"checkpointGB":220.94,"maxContext":131072,"license":"mit","released":"2025-07-20","downloads":751454,"kv":{"mode":"gqa","fullLayers":46,"slidingLayers":0,"window":null,"kvHeads":8,"headDim":128},"url":"https://canirun.branolabs.com/model/glm-4-5-air","runsIn":[{"tool":"lmstudio","how":"lmstudio-community build: lmstudio-community/GLM-4.5-Air-MLX-8bit","url":"https://huggingface.co/lmstudio-community/GLM-4.5-Air-MLX-8bit","status":"ok"},{"tool":"llamacpp","how":"GGUF: ggml-org/GLM-4.5-Air-GGUF","url":"https://huggingface.co/ggml-org/GLM-4.5-Air-GGUF","status":"ok"},{"tool":"mlx","how":"MLX conversion: mlx-community/GLM-4.5-Air-4bit","url":"https://huggingface.co/mlx-community/GLM-4.5-Air-4bit","status":"ok"},{"tool":"vllm","how":"named in the official model card","url":"https://huggingface.co/zai-org/GLM-4.5-Air","status":"ok"},{"tool":"sglang","how":"named in the official model card","url":"https://huggingface.co/zai-org/GLM-4.5-Air","status":"ok"}],"estimatesGB":{"Q4_K_M":{"ctx8192":{"total":72.16,"weights":67.58,"kv":1.54,"overhead":3.03},"ctx32768":{"total":76.79,"weights":67.58,"kv":6.17,"overhead":3.03}},"Q8_0":{"ctx8192":{"total":123.45,"weights":117.38,"kv":1.54,"overhead":4.52},"ctx32768":{"total":128.08,"weights":117.38,"kv":6.17,"overhead":4.52}}}},{"slug":"gpt-oss-120b","name":"gpt-oss-120b","org":"OpenAI","repo":"openai/gpt-oss-120b","dataRepo":"openai/gpt-oss-120b","use":["chat","coding","reasoning"],"paramsTotal":116829156672,"paramsActiveB":5.1,"activeSource":"model card","moe":true,"nativeDtypes":["BF16","U8"],"nativeQuant":"mxfp4","checkpointGB":65.25,"maxContext":131072,"license":"apache-2.0","released":"2025-08-04","downloads":3308713,"kv":{"mode":"gqa","fullLayers":18,"slidingLayers":18,"window":128,"kvHeads":8,"headDim":64},"url":"https://canirun.branolabs.com/model/gpt-oss-120b","runsIn":[{"tool":"ollama","how":"ollama run gpt-oss:120b","url":"https://ollama.com/library/gpt-oss","status":"ok"},{"tool":"lmstudio","how":"lmstudio-community build: lmstudio-community/gpt-oss-120b-MLX-8bit","url":"https://huggingface.co/lmstudio-community/gpt-oss-120b-MLX-8bit","status":"ok"},{"tool":"llamacpp","how":"GGUF: ggml-org/gpt-oss-120b-GGUF","url":"https://huggingface.co/ggml-org/gpt-oss-120b-GGUF","status":"ok"},{"tool":"mlx","how":"MLX conversion: mlx-community/gpt-oss-120b-MXFP4-Q8","url":"https://huggingface.co/mlx-community/gpt-oss-120b-MXFP4-Q8","status":"ok"},{"tool":"vllm","how":"named in the official model card","url":"https://huggingface.co/openai/gpt-oss-120b","status":"ok"}],"estimatesGB":{"Q4_K_M":{"ctx8192":{"total":74.93,"weights":71.48,"kv":0.31,"overhead":3.14},"ctx32768":{"total":75.83,"weights":71.48,"kv":1.21,"overhead":3.14}},"Q8_0":{"ctx8192":{"total":129.17,"weights":124.14,"kv":0.31,"overhead":4.72},"ctx32768":{"total":130.08,"weights":124.14,"kv":1.21,"overhead":4.72}}}},{"slug":"mistral-small-4-119b","name":"Mistral Small 4 119B","org":"Mistral AI","repo":"mistralai/Mistral-Small-4-119B-2603","dataRepo":"mistralai/Mistral-Small-4-119B-2603","use":["chat","coding"],"paramsTotal":119401317952,"paramsActiveB":6.5,"activeSource":"model card","moe":true,"nativeDtypes":["BF16","F8_E4M3"],"nativeQuant":"fp8","checkpointGB":241.85,"maxContext":1048576,"license":"apache-2.0","released":"2026-01-23","downloads":72871,"kv":{"mode":"mla","layers":36,"dim":320},"url":"https://canirun.branolabs.com/model/mistral-small-4-119b","runsIn":[{"tool":"lmstudio","how":"lmstudio-community build: lmstudio-community/Mistral-Small-4-119B-2603-GGUF","url":"https://huggingface.co/lmstudio-community/Mistral-Small-4-119B-2603-GGUF","status":"ok"},{"tool":"llamacpp","how":"GGUF: bartowski/mistralai_Mistral-Small-4-119B-2603-GGUF","url":"https://huggingface.co/bartowski/mistralai_Mistral-Small-4-119B-2603-GGUF","status":"ok"},{"tool":"mlx","how":"MLX conversion: mlx-community/Mistral-Small-4-119B-2603-4bit","url":"https://huggingface.co/mlx-community/Mistral-Small-4-119B-2603-4bit","status":"ok"},{"tool":"vllm","how":"named in the official model card","url":"https://huggingface.co/mistralai/Mistral-Small-4-119B-2603","status":"ok"},{"tool":"sglang","how":"named in the official model card","url":"https://huggingface.co/mistralai/Mistral-Small-4-119B-2603","status":"ok"}],"estimatesGB":{"Q4_K_M":{"ctx8192":{"total":76.43,"weights":73.05,"kv":0.19,"overhead":3.19},"ctx32768":{"total":77,"weights":73.05,"kv":0.75,"overhead":3.19}},"Q8_0":{"ctx8192":{"total":131.87,"weights":126.88,"kv":0.19,"overhead":4.81},"ctx32768":{"total":132.44,"weights":126.88,"kv":0.75,"overhead":4.81}}}},{"slug":"nemotron-3-super-120b-a12b","name":"Nemotron 3 Super 120B-A12B","org":"NVIDIA","repo":"nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16","dataRepo":"nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16","use":["chat","reasoning"],"paramsTotal":123611012096,"paramsActiveB":12,"activeSource":"name (A12B)","moe":true,"nativeDtypes":["F32","BF16"],"nativeQuant":null,"checkpointGB":247.23,"maxContext":262144,"license":"other","released":"2026-03-10","downloads":1065408,"kv":{"mode":"gqa","fullLayers":8,"slidingLayers":0,"window":null,"kvHeads":2,"headDim":128,"note":"Hybrid model: only the attention layers keep a growing KV cache; the other layers keep a small fixed state (not counted)."},"url":"https://canirun.branolabs.com/model/nemotron-3-super-120b-a12b","runsIn":[{"tool":"ollama","how":"ollama run nemotron-3-super:120b-a12b","url":"https://ollama.com/library/nemotron-3-super","status":"ok"},{"tool":"lmstudio","how":"lmstudio-community build: lmstudio-community/NVIDIA-Nemotron-3-Super-120B-A12B-GGUF","url":"https://huggingface.co/lmstudio-community/NVIDIA-Nemotron-3-Super-120B-A12B-GGUF","status":"ok"},{"tool":"llamacpp","how":"GGUF: unsloth/NVIDIA-Nemotron-3-Super-120B-A12B-GGUF","url":"https://huggingface.co/unsloth/NVIDIA-Nemotron-3-Super-120B-A12B-GGUF","status":"ok"},{"tool":"mlx","how":"MLX conversion: mlx-community/NVIDIA-Nemotron-3-Super-120B-A12B-4bit","url":"https://huggingface.co/mlx-community/NVIDIA-Nemotron-3-Super-120B-A12B-4bit","status":"ok"},{"tool":"vllm","how":"named in the official model card","url":"https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16","status":"ok"},{"tool":"sglang","how":"named in the official model card","url":"https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16","status":"ok"}],"estimatesGB":{"Q4_K_M":{"ctx8192":{"total":78.96,"weights":75.63,"kv":0.07,"overhead":3.27},"ctx32768":{"total":79.16,"weights":75.63,"kv":0.27,"overhead":3.27}},"Q8_0":{"ctx8192":{"total":136.36,"weights":131.35,"kv":0.07,"overhead":4.94},"ctx32768":{"total":136.56,"weights":131.35,"kv":0.27,"overhead":4.94}}}},{"slug":"qwen3-5-122b-a10b","name":"Qwen3.5 122B-A10B","org":"Qwen","repo":"Qwen/Qwen3.5-122B-A10B","dataRepo":"Qwen/Qwen3.5-122B-A10B","use":["chat","coding","vision"],"paramsTotal":125086497008,"paramsActiveB":10,"activeSource":"name (A10B)","moe":true,"nativeDtypes":["BF16","F32"],"nativeQuant":null,"checkpointGB":250.17,"maxContext":262144,"license":"apache-2.0","released":"2026-02-24","downloads":699279,"kv":{"mode":"gqa","fullLayers":12,"slidingLayers":0,"window":null,"kvHeads":2,"headDim":256,"note":"Hybrid model: only the full-attention layers keep a growing KV cache; linear/conv layers keep a small fixed state (not counted)."},"url":"https://canirun.branolabs.com/model/qwen3-5-122b-a10b","runsIn":[{"tool":"ollama","how":"ollama run qwen3.5:122b-a10b","url":"https://ollama.com/library/qwen3.5","status":"ok"},{"tool":"lmstudio","how":"lmstudio-community build: lmstudio-community/Qwen3.5-122B-A10B-GGUF","url":"https://huggingface.co/lmstudio-community/Qwen3.5-122B-A10B-GGUF","status":"ok"},{"tool":"llamacpp","how":"GGUF: bartowski/Qwen_Qwen3.5-122B-A10B-GGUF","url":"https://huggingface.co/bartowski/Qwen_Qwen3.5-122B-A10B-GGUF","status":"ok"},{"tool":"mlx","how":"MLX conversion: mlx-community/Qwen3.5-122B-A10B-4bit","url":"https://huggingface.co/mlx-community/Qwen3.5-122B-A10B-4bit","status":"ok"},{"tool":"ktransformers","how":"named in the official model card","url":"https://huggingface.co/Qwen/Qwen3.5-122B-A10B","status":"ok"},{"tool":"vllm","how":"named in the official model card","url":"https://huggingface.co/Qwen/Qwen3.5-122B-A10B","status":"ok"},{"tool":"sglang","how":"named in the official model card","url":"https://huggingface.co/Qwen/Qwen3.5-122B-A10B","status":"ok"}],"estimatesGB":{"Q4_K_M":{"ctx8192":{"total":80.03,"weights":76.53,"kv":0.2,"overhead":3.3},"ctx32768":{"total":80.63,"weights":76.53,"kv":0.81,"overhead":3.3}},"Q8_0":{"ctx8192":{"total":138.11,"weights":132.92,"kv":0.2,"overhead":4.99},"ctx32768":{"total":138.71,"weights":132.92,"kv":0.81,"overhead":4.99}}}},{"slug":"ling-3-0-flash","name":"Ling-3.0-flash 124B-A5.1B","org":"inclusionAI (Ant Group)","repo":"inclusionAI/Ling-3.0-flash","dataRepo":"inclusionAI/Ling-3.0-flash","use":["chat","coding","reasoning","agents"],"paramsTotal":127486405600,"paramsActiveB":5.1,"activeSource":"model card (124B total, 5.1B activated)","moe":true,"nativeDtypes":["F32","BF16"],"nativeQuant":null,"checkpointGB":254.98,"maxContext":262144,"license":"mit","released":"2026-08-02","downloads":10250,"kv":{"mode":"mla","layers":7,"dim":576,"note":"Hybrid model: only the latent-attention (MLA) layers keep a growing KV cache; the linear-attention layers keep a small fixed state (not counted)."},"url":"https://canirun.branolabs.com/model/ling-3-0-flash","runsIn":[{"tool":"llamacpp","how":"GGUF: bartowski/Ling-3.0-flash-GGUF","url":"https://huggingface.co/bartowski/Ling-3.0-flash-GGUF","status":"ok"},{"tool":"vllm","how":"named in the official model card","url":"https://huggingface.co/inclusionAI/Ling-3.0-flash","status":"ok"},{"tool":"sglang","how":"named in the official model card","url":"https://huggingface.co/inclusionAI/Ling-3.0-flash","status":"ok"}],"estimatesGB":{"Q4_K_M":{"ctx8192":{"total":81.4,"weights":78,"kv":0.07,"overhead":3.34},"ctx32768":{"total":81.6,"weights":78,"kv":0.26,"overhead":3.34}},"Q8_0":{"ctx8192":{"total":140.6,"weights":135.47,"kv":0.07,"overhead":5.06},"ctx32768":{"total":140.8,"weights":135.47,"kv":0.26,"overhead":5.06}}}},{"slug":"mistral-medium-3-5-128b","name":"Mistral Medium 3.5 128B","org":"Mistral AI","repo":"mistralai/Mistral-Medium-3.5-128B","dataRepo":"mistralai/Mistral-Medium-3.5-128B","use":["chat","coding"],"paramsTotal":127704210176,"paramsActiveB":null,"activeSource":null,"moe":false,"nativeDtypes":["BF16","F8_E4M3"],"nativeQuant":"fp8","checkpointGB":267.21,"maxContext":262144,"license":"other","released":"2026-03-31","downloads":74696,"kv":{"mode":"gqa","fullLayers":88,"slidingLayers":0,"window":null,"kvHeads":8,"headDim":128},"url":"https://canirun.branolabs.com/model/mistral-medium-3-5-128b","runsIn":[{"tool":"ollama","how":"ollama run mistral-medium-3.5:128b","url":"https://ollama.com/library/mistral-medium-3.5","status":"ok"},{"tool":"llamacpp","how":"GGUF: bartowski/mistralai_Mistral-Medium-3.5-128B-GGUF","url":"https://huggingface.co/bartowski/mistralai_Mistral-Medium-3.5-128B-GGUF","status":"ok"},{"tool":"mlx","how":"MLX conversion: mlx-community/Mistral-Medium-3.5-128B-4bit","url":"https://huggingface.co/mlx-community/Mistral-Medium-3.5-128B-4bit","status":"ok"},{"tool":"vllm","how":"named in the official model card","url":"https://huggingface.co/mistralai/Mistral-Medium-3.5-128B","status":"ok"},{"tool":"sglang","how":"named in the official model card","url":"https://huggingface.co/mistralai/Mistral-Medium-3.5-128B","status":"ok"}],"estimatesGB":{"Q4_K_M":{"ctx8192":{"total":84.43,"weights":78.13,"kv":2.95,"overhead":3.34},"ctx32768":{"total":93.28,"weights":78.13,"kv":11.81,"overhead":3.34}},"Q8_0":{"ctx8192":{"total":143.72,"weights":135.7,"kv":2.95,"overhead":5.07},"ctx32768":{"total":152.58,"weights":135.7,"kv":11.81,"overhead":5.07}}}},{"slug":"qwen3-8-flash-next","name":"Qwen3.8-Flash-Next 125B-A6B","org":"Qwen","repo":"Qwen/Qwen3.8-Flash-Next","dataRepo":"Qwen/Qwen3.8-Flash-Next","use":["chat","coding","vision","agents"],"paramsTotal":179999981459,"paramsActiveB":6,"activeSource":"model card (125B with 6B activated, plus 51B n-gram embedding and 4B MTP)","moe":true,"nativeDtypes":["BF16","I64"],"nativeQuant":null,"checkpointGB":360,"maxContext":262144,"license":"other","released":"2026-08-24","downloads":2457201,"kv":{"mode":"gqa","fullLayers":12,"slidingLayers":0,"window":null,"kvHeads":2,"headDim":256,"note":"Hybrid model: only the full-attention layers keep a growing KV cache; linear/conv layers keep a small fixed state (not counted)."},"note":"The published checkpoint is 180B parameters: the 125B language model, a 51B n-gram embedding table and a 4B multi-token-prediction head. Local runners like Strata keep the n-gram table on the SSD and the experts in system RAM, so the real GPU need is far below the full-weights number shown here.","url":"https://canirun.branolabs.com/model/qwen3-8-flash-next","runsIn":[{"tool":"strata","how":"The model Strata is built for: one-click setup, 12 GB+ GPU, 32-64 GB RAM","url":"https://github.com/Niko1221/Strata","status":"ok"},{"tool":"colibri","how":"coli setup --model qwen38-flash-next (experts stream from SSD; 24/32 GB RAM)","url":"https://github.com/JustVugg/colibri#which-model-for-my-machine","status":"ok"},{"tool":"ollama","how":"ollama run qwen3.8-flash-next:125b-a6b-q4_K_M","url":"https://ollama.com/library/qwen3.8-flash-next","status":"ok"},{"tool":"lmstudio","how":"lmstudio-community build: lmstudio-community/Qwen3.8-Flash-Next-GGUF","url":"https://huggingface.co/lmstudio-community/Qwen3.8-Flash-Next-GGUF","status":"ok"},{"tool":"llamacpp","how":"GGUF: ggml-org/Qwen3.8-Flash-Next-GGUF","url":"https://huggingface.co/ggml-org/Qwen3.8-Flash-Next-GGUF","status":"ok"},{"tool":"mlx","how":"MLX conversion: mlx-community/Qwen3.8-Flash-Next-4bit","url":"https://huggingface.co/mlx-community/Qwen3.8-Flash-Next-4bit","status":"ok"},{"tool":"ktransformers","how":"named in the official model card","url":"https://huggingface.co/Qwen/Qwen3.8-Flash-Next","status":"ok"},{"tool":"vllm","how":"named in the official model card","url":"https://huggingface.co/Qwen/Qwen3.8-Flash-Next","status":"ok"},{"tool":"sglang","how":"named in the official model card","url":"https://huggingface.co/Qwen/Qwen3.8-Flash-Next","status":"ok"},{"tool":"tokenspeed","how":"named in the official model card","url":"https://huggingface.co/Qwen/Qwen3.8-Flash-Next","status":"ok"}],"estimatesGB":{"Q4_K_M":{"ctx8192":{"total":114.63,"weights":110.12,"kv":0.2,"overhead":4.3},"ctx32768":{"total":115.23,"weights":110.12,"kv":0.81,"overhead":4.3}},"Q8_0":{"ctx8192":{"total":198.21,"weights":191.27,"kv":0.2,"overhead":6.74},"ctx32768":{"total":198.81,"weights":191.27,"kv":0.81,"overhead":6.74}}}},{"slug":"step-3-5-flash","name":"Step 3.5 Flash 196B-A11B","org":"StepFun","repo":"stepfun-ai/Step-3.5-Flash","dataRepo":"stepfun-ai/Step-3.5-Flash","use":["chat","coding","reasoning","agents"],"paramsTotal":199384301376,"paramsActiveB":11,"activeSource":"model card (~11B active per token)","moe":false,"nativeDtypes":["BF16","F32"],"nativeQuant":null,"checkpointGB":398.77,"maxContext":262144,"license":"apache-2.0","released":"2026-02-01","downloads":261975,"kv":{"mode":"gqa","fullLayers":12,"slidingLayers":33,"window":512,"kvHeads":8,"headDim":128,"note":"12 full-attention and 33 sliding-window (512-token) layers with 8 KV groups, from config.json (the 3 MTP layers are not counted)."},"url":"https://canirun.branolabs.com/model/step-3-5-flash","runsIn":[{"tool":"llamacpp","how":"GGUF: ggml-org/Step-3.5-Flash-GGUF","url":"https://huggingface.co/ggml-org/Step-3.5-Flash-GGUF","status":"ok"},{"tool":"mlx","how":"MLX conversion: mlx-community/Step-3.5-Flash-4bit","url":"https://huggingface.co/mlx-community/Step-3.5-Flash-4bit","status":"ok"},{"tool":"vllm","how":"named in the official model card","url":"https://huggingface.co/stepfun-ai/Step-3.5-Flash","status":"ok"},{"tool":"sglang","how":"named in the official model card","url":"https://huggingface.co/stepfun-ai/Step-3.5-Flash","status":"ok"}],"estimatesGB":{"Q4_K_M":{"ctx8192":{"total":127.11,"weights":121.98,"kv":0.47,"overhead":4.66},"ctx32768":{"total":128.32,"weights":121.98,"kv":1.68,"overhead":4.66}},"Q8_0":{"ctx8192":{"total":219.69,"weights":211.87,"kv":0.47,"overhead":7.36},"ctx32768":{"total":220.9,"weights":211.87,"kv":1.68,"overhead":7.36}}}},{"slug":"minimax-m2-7","name":"MiniMax M2.7","org":"MiniMax","repo":"MiniMaxAI/MiniMax-M2.7","dataRepo":"MiniMaxAI/MiniMax-M2.7","use":["chat","coding","agents"],"paramsTotal":228689764864,"paramsActiveB":null,"activeSource":null,"moe":true,"nativeDtypes":["F32","BF16","F8_E4M3"],"nativeQuant":"fp8","checkpointGB":230.13,"maxContext":204800,"license":"other","released":"2026-04-09","downloads":581161,"kv":{"mode":"gqa","fullLayers":62,"slidingLayers":0,"window":null,"kvHeads":8,"headDim":128},"url":"https://canirun.branolabs.com/model/minimax-m2-7","runsIn":[{"tool":"ollama","how":"Ollama lists this model as a cloud model only (no local download)","url":"https://ollama.com/library/minimax-m2.7","status":"cloud"},{"tool":"lmstudio","how":"lmstudio-community build: lmstudio-community/MiniMax-M2.7-GGUF","url":"https://huggingface.co/lmstudio-community/MiniMax-M2.7-GGUF","status":"ok"},{"tool":"llamacpp","how":"GGUF: bartowski/MiniMaxAI_MiniMax-M2.7-GGUF","url":"https://huggingface.co/bartowski/MiniMaxAI_MiniMax-M2.7-GGUF","status":"ok"},{"tool":"mlx","how":"MLX conversion: mlx-community/MiniMax-M2.7-4bit","url":"https://huggingface.co/mlx-community/MiniMax-M2.7-4bit","status":"ok"},{"tool":"vllm","how":"named in the official model card","url":"https://huggingface.co/MiniMaxAI/MiniMax-M2.7","status":"ok"},{"tool":"sglang","how":"named in the official model card","url":"https://huggingface.co/MiniMaxAI/MiniMax-M2.7","status":"ok"}],"estimatesGB":{"Q4_K_M":{"ctx8192":{"total":147.19,"weights":139.91,"kv":2.08,"overhead":5.2},"ctx32768":{"total":153.43,"weights":139.91,"kv":8.32,"overhead":5.2}},"Q8_0":{"ctx8192":{"total":253.38,"weights":243.01,"kv":2.08,"overhead":8.29},"ctx32768":{"total":259.62,"weights":243.01,"kv":8.32,"overhead":8.29}}}},{"slug":"deepseek-v4-flash","name":"DeepSeek V4 Flash (0731)","org":"DeepSeek","repo":"deepseek-ai/DeepSeek-V4-Flash-0731","dataRepo":"deepseek-ai/DeepSeek-V4-Flash-0731","use":["chat","coding","reasoning"],"paramsTotal":304180418494,"paramsActiveB":13,"activeSource":"DeepSeek-V4-Pro model card table","moe":true,"nativeDtypes":["BF16","I64","F32","F8_E4M3","I8"],"nativeQuant":"fp8","checkpointGB":166.89,"maxContext":1048576,"license":"mit","released":"2026-07-31","downloads":4327965,"kv":{"mode":"unknown","note":"This model uses compressed/sparse attention; its KV cache is not modelled here (it is small compared to the weights)."},"url":"https://canirun.branolabs.com/model/deepseek-v4-flash","runsIn":[{"tool":"colibri","how":"coli setup --model deepseek-v4-flash (experts stream from SSD; 16/32 GB RAM)","url":"https://github.com/JustVugg/colibri#which-model-for-my-machine","status":"ok"},{"tool":"lmstudio","how":"lmstudio-community build: lmstudio-community/DeepSeek-V4-Flash-0731-GGUF","url":"https://huggingface.co/lmstudio-community/DeepSeek-V4-Flash-0731-GGUF","status":"ok"},{"tool":"llamacpp","how":"GGUF: ggml-org/DeepSeek-V4-Flash-0731-GGUF","url":"https://huggingface.co/ggml-org/DeepSeek-V4-Flash-0731-GGUF","status":"ok"},{"tool":"vllm","how":"named in the official model card","url":"https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731","status":"ok"},{"tool":"sglang","how":"named in the official model card","url":"https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731","status":"ok"}],"estimatesGB":{"Q4_K_M":{"ctx8192":{"total":192.68,"weights":186.1,"kv":null,"overhead":6.58},"ctx32768":{"total":192.68,"weights":186.1,"kv":null,"overhead":6.58}},"Q8_0":{"ctx8192":{"total":333.92,"weights":323.22,"kv":null,"overhead":10.7},"ctx32768":{"total":333.92,"weights":323.22,"kv":null,"overhead":10.7}}}},{"slug":"mimo-v2-6-flash","name":"MiMo-V2.6-Flash 309B-A15B","org":"Xiaomi","repo":"XiaomiMiMo/MiMo-V2.6-Flash-MOPD","dataRepo":"XiaomiMiMo/MiMo-V2.6-Flash-MOPD","use":["chat","coding","vision","agents"],"paramsTotal":310756322688,"paramsActiveB":15,"activeSource":"model card (309B total / 15B activated)","moe":true,"nativeDtypes":["F32","BF16","F8_E4M3","U8"],"nativeQuant":"fp8","checkpointGB":172.93,"maxContext":1048576,"license":"mit","released":"2026-09-27","downloads":9570,"kv":{"mode":"gqa","fullLayers":9,"slidingLayers":39,"window":128,"kvHeads":4,"headDim":160,"note":"9 full-attention layers (4 KV heads, 192-dim keys and 128-dim values, averaged to 160) and 39 sliding-window layers with a 128-token window, from config.json. The sliding layers use 8 KV heads; with a 128-token window that cache stays tiny."},"url":"https://canirun.branolabs.com/model/mimo-v2-6-flash","runsIn":[{"tool":"colibri","how":"coli setup --model mimo-v2.6-flash (experts stream from SSD; 32/52 GB RAM)","url":"https://github.com/JustVugg/colibri/blob/main/docs/mimo.md","status":"ok"},{"tool":"llamacpp","how":"GGUF: ggml-org/MiMo-V2.6-Flash-RL-GGUF","url":"https://huggingface.co/ggml-org/MiMo-V2.6-Flash-RL-GGUF","status":"ok"},{"tool":"vllm","how":"named in the official model card","url":"https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Flash-MOPD","status":"ok"},{"tool":"sglang","how":"named in the official model card","url":"https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Flash-MOPD","status":"ok"}],"estimatesGB":{"Q4_K_M":{"ctx8192":{"total":197.03,"weights":190.12,"kv":0.2,"overhead":6.7},"ctx32768":{"total":197.59,"weights":190.12,"kv":0.77,"overhead":6.7}},"Q8_0":{"ctx8192":{"total":341.32,"weights":330.21,"kv":0.2,"overhead":10.91},"ctx32768":{"total":341.88,"weights":330.21,"kv":0.77,"overhead":10.91}}}},{"slug":"glm-5-3-flash","name":"GLM-5.3-Flash 320B-A18B","org":"Z.ai","repo":"zai-org/GLM-5.3-Flash","dataRepo":"zai-org/GLM-5.3-Flash","use":["chat","coding","vision","agents"],"paramsTotal":321323031390,"paramsActiveB":18,"activeSource":"model card (320B total, 18B active)","moe":true,"nativeDtypes":["BF16","F8_E4M3","F32"],"nativeQuant":"fp8","checkpointGB":328.34,"maxContext":1048576,"license":"mit","released":"2026-08-25","downloads":6574089,"kv":{"mode":"mla","layers":11,"dim":512,"note":"Hybrid model: only the latent-attention (MLA) layers keep a growing KV cache; the linear-attention layers keep a small fixed state (not counted)."},"url":"https://canirun.branolabs.com/model/glm-5-3-flash","runsIn":[{"tool":"colibri","how":"glm53-flash engine, converted shard by shard to ~195 GB; 25 GB RAM","url":"https://github.com/JustVugg/colibri/blob/main/docs/glm53-flash.md","status":"ok"},{"tool":"ollama","how":"Ollama lists this model as a cloud model only (no local download)","url":"https://ollama.com/library/glm-5.3-flash","status":"cloud"},{"tool":"llamacpp","how":"GGUF: ggml-org/GLM-5.3-Flash-GGUF","url":"https://huggingface.co/ggml-org/GLM-5.3-Flash-GGUF","status":"ok"},{"tool":"mlx","how":"MLX conversion: mlx-community/GLM-5.3-Flash-4bit","url":"https://huggingface.co/mlx-community/GLM-5.3-Flash-4bit","status":"ok"},{"tool":"ktransformers","how":"named in the official model card","url":"https://huggingface.co/zai-org/GLM-5.3-Flash","status":"ok"},{"tool":"vllm","how":"named in the official model card","url":"https://huggingface.co/zai-org/GLM-5.3-Flash","status":"ok"},{"tool":"sglang","how":"named in the official model card","url":"https://huggingface.co/zai-org/GLM-5.3-Flash","status":"ok"},{"tool":"tokenspeed","how":"named in the official model card","url":"https://huggingface.co/zai-org/GLM-5.3-Flash","status":"ok"}],"estimatesGB":{"Q4_K_M":{"ctx8192":{"total":203.58,"weights":196.59,"kv":0.09,"overhead":6.9},"ctx32768":{"total":203.85,"weights":196.59,"kv":0.37,"overhead":6.9}},"Q8_0":{"ctx8192":{"total":352.77,"weights":341.44,"kv":0.09,"overhead":11.24},"ctx32768":{"total":353.05,"weights":341.44,"kv":0.37,"overhead":11.24}}}},{"slug":"qwen3-coder-480b-a35b","name":"Qwen3-Coder 480B-A35B","org":"Qwen","repo":"Qwen/Qwen3-Coder-480B-A35B-Instruct","dataRepo":"Qwen/Qwen3-Coder-480B-A35B-Instruct","use":["coding"],"paramsTotal":480154875392,"paramsActiveB":35,"activeSource":"name (A35B)","moe":true,"nativeDtypes":["BF16"],"nativeQuant":null,"checkpointGB":960.31,"maxContext":262144,"license":"apache-2.0","released":"2025-07-22","downloads":42106,"kv":{"mode":"gqa","fullLayers":62,"slidingLayers":0,"window":null,"kvHeads":8,"headDim":128},"url":"https://canirun.branolabs.com/model/qwen3-coder-480b-a35b","runsIn":[{"tool":"ollama","how":"ollama run qwen3-coder:480b","url":"https://ollama.com/library/qwen3-coder","status":"ok"},{"tool":"lmstudio","how":"lmstudio-community build: lmstudio-community/Qwen3-Coder-480B-A35B-Instruct-MLX-6bit","url":"https://huggingface.co/lmstudio-community/Qwen3-Coder-480B-A35B-Instruct-MLX-6bit","status":"ok"},{"tool":"llamacpp","how":"GGUF: bartowski/Qwen_Qwen3-Coder-480B-A35B-Instruct-GGUF","url":"https://huggingface.co/bartowski/Qwen_Qwen3-Coder-480B-A35B-Instruct-GGUF","status":"ok"},{"tool":"mlx","how":"MLX conversion: mlx-community/Qwen3-Coder-480B-A35B-Instruct-4bit","url":"https://huggingface.co/mlx-community/Qwen3-Coder-480B-A35B-Instruct-4bit","status":"ok"},{"tool":"ktransformers","how":"named in the official model card","url":"https://huggingface.co/Qwen/Qwen3-Coder-480B-A35B-Instruct","status":"ok"}],"estimatesGB":{"Q4_K_M":{"ctx8192":{"total":305.65,"weights":293.76,"kv":2.08,"overhead":9.81},"ctx32768":{"total":311.89,"weights":293.76,"kv":8.32,"overhead":9.81}},"Q8_0":{"ctx8192":{"total":528.6,"weights":510.21,"kv":2.08,"overhead":16.31},"ctx32768":{"total":534.84,"weights":510.21,"kv":8.32,"overhead":16.31}}}},{"slug":"deepseek-v3-1","name":"DeepSeek V3.1","org":"DeepSeek","repo":"deepseek-ai/DeepSeek-V3.1","dataRepo":"deepseek-ai/DeepSeek-V3.1","use":["chat","coding","reasoning"],"paramsTotal":684531386000,"paramsActiveB":37,"activeSource":"model card","moe":true,"nativeDtypes":["BF16","F8_E4M3","F32"],"nativeQuant":"fp8","checkpointGB":688.59,"maxContext":163840,"license":"mit","released":"2025-08-21","downloads":350794,"kv":{"mode":"mla","layers":61,"dim":576},"url":"https://canirun.branolabs.com/model/deepseek-v3-1","runsIn":[{"tool":"ollama","how":"ollama run deepseek-v3.1:671b","url":"https://ollama.com/library/deepseek-v3.1","status":"ok"},{"tool":"llamacpp","how":"GGUF: bartowski/deepseek-ai_DeepSeek-V3.1-GGUF","url":"https://huggingface.co/bartowski/deepseek-ai_DeepSeek-V3.1-GGUF","status":"ok"},{"tool":"mlx","how":"MLX conversion: mlx-community/DeepSeek-V3.1-4bit","url":"https://huggingface.co/mlx-community/DeepSeek-V3.1-4bit","status":"ok"}],"estimatesGB":{"Q4_K_M":{"ctx8192":{"total":432.94,"weights":418.8,"kv":0.58,"overhead":13.56},"ctx32768":{"total":434.66,"weights":418.8,"kv":2.3,"overhead":13.56}},"Q8_0":{"ctx8192":{"total":750.78,"weights":727.38,"kv":0.58,"overhead":22.82},"ctx32768":{"total":752.51,"weights":727.38,"kv":2.3,"overhead":22.82}}}},{"slug":"glm-5-3","name":"GLM-5.3","org":"Z.ai","repo":"zai-org/GLM-5.3","dataRepo":"zai-org/GLM-5.3","use":["chat","coding"],"paramsTotal":753329940480,"paramsActiveB":null,"activeSource":null,"moe":true,"nativeDtypes":["BF16","F8_E4M3","F32"],"nativeQuant":"fp8","checkpointGB":755.63,"maxContext":1048576,"license":"other","released":"2026-08-25","downloads":1856325,"kv":{"mode":"mla","layers":78,"dim":576},"url":"https://canirun.branolabs.com/model/glm-5-3","runsIn":[{"tool":"colibri","how":"coli setup --model glm-5.3 (experts stream from SSD; 16/24 GB RAM)","url":"https://github.com/JustVugg/colibri#which-model-for-my-machine","status":"ok"},{"tool":"ollama","how":"Ollama lists this model as a cloud model only (no local download)","url":"https://ollama.com/library/glm-5.3","status":"cloud"},{"tool":"llamacpp","how":"GGUF: unsloth/GLM-5.3-GGUF","url":"https://huggingface.co/unsloth/GLM-5.3-GGUF","status":"ok"},{"tool":"mlx","how":"MLX conversion: mlx-community/GLM-5.3-4bit","url":"https://huggingface.co/mlx-community/GLM-5.3-4bit","status":"ok"},{"tool":"ktransformers","how":"named in the official model card","url":"https://huggingface.co/zai-org/GLM-5.3","status":"ok"},{"tool":"vllm","how":"named in the official model card","url":"https://huggingface.co/zai-org/GLM-5.3","status":"ok"},{"tool":"sglang","how":"named in the official model card","url":"https://huggingface.co/zai-org/GLM-5.3","status":"ok"},{"tool":"tokenspeed","how":"named in the official model card","url":"https://huggingface.co/zai-org/GLM-5.3","status":"ok"}],"estimatesGB":{"Q4_K_M":{"ctx8192":{"total":476.45,"weights":460.89,"kv":0.74,"overhead":14.83},"ctx32768":{"total":478.66,"weights":460.89,"kv":2.94,"overhead":14.83}},"Q8_0":{"ctx8192":{"total":826.24,"weights":800.49,"kv":0.74,"overhead":25.01},"ctx32768":{"total":828.45,"weights":800.49,"kv":2.94,"overhead":25.01}}}},{"slug":"deepseek-v4-1-flash","name":"DeepSeek V4.1 Flash","org":"DeepSeek","repo":"deepseek-ai/DeepSeek-V4.1-Flash","dataRepo":"deepseek-ai/DeepSeek-V4.1-Flash","use":["chat","coding","vision","reasoning"],"paramsTotal":763205315794,"paramsActiveB":16,"activeSource":"model card table (8B / 16B activated; the higher figure is used)","moe":true,"nativeDtypes":["BF16","F32","F8_E4M3","I8"],"nativeQuant":"fp8","checkpointGB":510.3,"maxContext":1048576,"license":"mit","released":"2026-09-10","downloads":1306744,"kv":{"mode":"unknown","note":"This model uses compressed/sparse attention; its KV cache is not modelled here (it is small compared to the weights)."},"note":"The checkpoint also holds a 196B-parameter Engram conditional memory that is only sparsely read per token (model card).","url":"https://canirun.branolabs.com/model/deepseek-v4-1-flash","runsIn":[{"tool":"colibri","how":"deepseek_v41 engine, one-off preparation, ~18 GB RAM plus expert cache","url":"https://github.com/JustVugg/colibri/blob/main/docs/deepseek-v41.md","status":"ok"},{"tool":"ollama","how":"Ollama lists this model as a cloud model only (no local download)","url":"https://ollama.com/library/deepseek-v4.1-flash","status":"cloud"}],"estimatesGB":{"Q4_K_M":{"ctx8192":{"total":481.94,"weights":466.93,"kv":null,"overhead":15.01},"ctx32768":{"total":481.94,"weights":466.93,"kv":null,"overhead":15.01}},"Q8_0":{"ctx8192":{"total":836.31,"weights":810.98,"kv":null,"overhead":25.33},"ctx32768":{"total":836.31,"weights":810.98,"kv":null,"overhead":25.33}}}},{"slug":"kimi-k2","name":"Kimi K2 Instruct","org":"Moonshot AI","repo":"moonshotai/Kimi-K2-Instruct","dataRepo":"moonshotai/Kimi-K2-Instruct","use":["chat","coding","agents"],"paramsTotal":1026408235864,"paramsActiveB":32,"activeSource":"model card","moe":true,"nativeDtypes":["F32","BF16","F8_E4M3"],"nativeQuant":"fp8","checkpointGB":1029.19,"maxContext":131072,"license":"other","released":"2025-07-11","downloads":348867,"kv":{"mode":"mla","layers":61,"dim":576},"url":"https://canirun.branolabs.com/model/kimi-k2","runsIn":[{"tool":"llamacpp","how":"GGUF: unsloth/Kimi-K2-Instruct-GGUF","url":"https://huggingface.co/unsloth/Kimi-K2-Instruct-GGUF","status":"ok"},{"tool":"mlx","how":"MLX conversion: mlx-community/Kimi-K2-Instruct-4bit","url":"https://huggingface.co/mlx-community/Kimi-K2-Instruct-4bit","status":"ok"},{"tool":"ktransformers","how":"named in the official model card","url":"https://huggingface.co/moonshotai/Kimi-K2-Instruct","status":"ok"},{"tool":"vllm","how":"named in the official model card","url":"https://huggingface.co/moonshotai/Kimi-K2-Instruct","status":"ok"},{"tool":"sglang","how":"named in the official model card","url":"https://huggingface.co/moonshotai/Kimi-K2-Instruct","status":"ok"}],"estimatesGB":{"Q4_K_M":{"ctx8192":{"total":648.37,"weights":627.96,"kv":0.58,"overhead":19.84},"ctx32768":{"total":650.1,"weights":627.96,"kv":2.3,"overhead":19.84}},"Q8_0":{"ctx8192":{"total":1124.96,"weights":1090.66,"kv":0.58,"overhead":33.72},"ctx32768":{"total":1126.68,"weights":1090.66,"kv":2.3,"overhead":33.72}}}},{"slug":"deepseek-v4-pro","name":"DeepSeek V4 Pro","org":"DeepSeek","repo":"deepseek-ai/DeepSeek-V4-Pro","dataRepo":"deepseek-ai/DeepSeek-V4-Pro","use":["chat","coding","reasoning"],"paramsTotal":1598839674782,"paramsActiveB":49,"activeSource":"model card","moe":true,"nativeDtypes":["BF16","I64","F32","F8_E4M3","I8"],"nativeQuant":"fp8","checkpointGB":864.72,"maxContext":1048576,"license":"mit","released":"2026-04-22","downloads":373563,"kv":{"mode":"unknown","note":"This model uses compressed/sparse attention; its KV cache is not modelled here (it is small compared to the weights)."},"url":"https://canirun.branolabs.com/model/deepseek-v4-pro","runsIn":[{"tool":"ollama","how":"Ollama lists this model as a cloud model only (no local download)","url":"https://ollama.com/library/deepseek-v4-pro","status":"cloud"},{"tool":"mlx","how":"MLX conversion: mlx-community/DeepSeek-V4-Pro","url":"https://huggingface.co/mlx-community/DeepSeek-V4-Pro","status":"ok"}],"estimatesGB":{"Q4_K_M":{"ctx8192":{"total":1008.52,"weights":978.17,"kv":null,"overhead":30.35},"ctx32768":{"total":1008.52,"weights":978.17,"kv":null,"overhead":30.35}},"Q8_0":{"ctx8192":{"total":1750.89,"weights":1698.93,"kv":null,"overhead":51.97},"ctx32768":{"total":1750.89,"weights":1698.93,"kv":null,"overhead":51.97}}}},{"slug":"kimi-k3","name":"Kimi K3","org":"Moonshot AI","repo":"moonshotai/Kimi-K3","dataRepo":"moonshotai/Kimi-K3","use":["chat","coding","agents"],"paramsTotal":2779931837184,"paramsActiveB":104,"activeSource":"model card","moe":true,"nativeDtypes":["F32","BF16","U8"],"nativeQuant":"compressed-tensors","checkpointGB":1560.94,"maxContext":1048576,"license":"other","released":"2026-06-13","downloads":1049317,"kv":{"mode":"mla","layers":24,"dim":576,"note":"Hybrid model: only the latent-attention (MLA) layers keep a growing KV cache; the linear-attention layers keep a small fixed state (not counted)."},"url":"https://canirun.branolabs.com/model/kimi-k3","runsIn":[{"tool":"colibri","how":"coli setup --model kimi-k3 (experts stream from SSD; 32/64 GB RAM, about 9 s per token)","url":"https://github.com/JustVugg/colibri#which-model-for-my-machine","status":"ok"},{"tool":"ollama","how":"Ollama lists this model as a cloud model only (no local download)","url":"https://ollama.com/library/kimi-k3","status":"cloud"},{"tool":"llamacpp","how":"Unsloth GGUF needs Unsloth's llama.cpp fork (PR #48), not mainline yet","url":"https://huggingface.co/unsloth/Kimi-K3-GGUF","status":"fork"},{"tool":"vllm","how":"named in the official model card","url":"https://huggingface.co/moonshotai/Kimi-K3","status":"ok"},{"tool":"sglang","how":"named in the official model card","url":"https://huggingface.co/moonshotai/Kimi-K3","status":"ok"},{"tool":"tokenspeed","how":"named in the official model card","url":"https://huggingface.co/moonshotai/Kimi-K3","status":"ok"}],"estimatesGB":{"Q4_K_M":{"ctx8192":{"total":1753.01,"weights":1700.76,"kv":0.23,"overhead":52.02},"ctx32768":{"total":1753.69,"weights":1700.76,"kv":0.91,"overhead":52.02}},"Q8_0":{"ctx8192":{"total":3043.8,"weights":2953.96,"kv":0.23,"overhead":89.62},"ctx32768":{"total":3044.48,"weights":2953.96,"kv":0.91,"overhead":89.62}}}}]}