Can I Run Nanbeige4.3 33B Locally? (2026): Yes, on 24GB
Nanbeige put Nanbeige4.3-33B on Hugging Face on 24 October 2026. It is a dense 33B model on the standard Llama architecture, with a 512K context and agent benchmarks that beat Qwen3.8 27B on the lab's own table. Stock llama.cpp loads it. We read every GGUF size from the Hugging Face API and computed the KV cache from config.json. The short answer: a 24GB card runs a 4-bit file with about 32K of context. The catch is the cache, not the weights.
Bottom Line
- Yes, on a 24GB card. bartowski’s IQ4_XS (17.36 GiB) plus a 32K-token cache at q8_0 (~3.45 GiB) is about 20.8 GiB. Q4_K_M (19.36 GiB) fits with about 16K tokens.
- 32GB runs 6-bit. Q6_K (27.02 GiB) fits an RTX 5090 with about 16K tokens. Use 6-bit if you can: this model loses more quality per bit than most.
- The cache is the catch. Nanbeige needs 208 KiB per token at f16. Qwen3.8 27B needs 64 KiB. At 131K tokens that is 26 GiB vs 8 GiB.
- Stock llama.cpp loads it. Use release b11845 or newer. It is a plain Llama architecture. GGUF and MLX 4-bit builds were on Hugging Face within a day.
- It is not fast. A dense 33B reads every weight per token. One DGX Spark build measured 7.3 tok/s at Q6_K, and 12.6 tok/s with a 3B draft model (community-reported).
- Be careful with tool access. In one community test it resisted 58.5% of prompt-injection attacks. The Qwen model in that test resisted 99.0%.
Ready to buy? See the tested hardware list with current prices.
What Nanbeige4.3 33B Is
Figures come from the model card, its config.json and the Hugging Face API, read on 31 October 2026.
| Spec | Value |
|---|---|
| Repo | Nanbeige/Nanbeige4.3-33B (plus a -Base checkpoint) |
| Created on Hugging Face | 24 October 2026 |
| Maker | Nanbeige, the research team at the BOSS Zhipin recruitment app |
| Architecture | LlamaForCausalLM, dense (no experts) |
| Parameters | 33B (card) |
| Layers | 52 |
| Attention | 56 query heads, 8 KV heads, head size 128, full attention on every layer |
| Context | 524,288 tokens |
| Vocabulary | 166,912 tokens |
| Weights as shipped | BF16, about 66 GB |
| Licence | Apache 2.0 |
| Thinking | On by default; enable_thinking=False turns it off |
The card calls it a “teacher” model. Nanbeige plans to distill it into a Nanbeige4.3-3B.
What the card claims. These are Nanbeige’s numbers, all at the highest thinking effort, with the same harness for every model. We ran no benchmark.
| Benchmark (card) | Nanbeige4.3 33B | Qwen3.8 27B | Gemma4 31B |
|---|---|---|---|
| Terminal-Bench 2.1 | 80.9 | 73.0 | 47.2 |
| DeepSWE | 65.5 | 42.5 | 2.7 |
| SWE-Bench Pro | 60.3 | 61.7 | 36.9 |
| GDPval | 84.8 | 86.4 | 71.4 |
| Toolathlon-Verified | 70.4 | 68.5 | 27.8 |
| GPQA Diamond | 89.5 | 89.2 | 91.3 |
| AIME 2026 | 95.2 | 94.5 | 89.2 |
Nanbeige wins most agent rows. Qwen3.8 27B still wins SWE-Bench Pro and GDPval. Independent tests are thin so far, so treat these as claims.
Every GGUF Size
bartowski’s files, read from the Hugging Face API on 31 October 2026. The card says they were made with llama.cpp b11845, “you’ll need that release or newer.”
| Quant | Size (GiB) | Fits |
|---|---|---|
| Q8_0 | 32.78 | 48GB+ |
| Q6_K_L | 28.32 | 32GB card, short context |
| Q6_K | 27.02 | 32GB card (~16K tokens) |
| Q5_K_M | 23.35 | 32GB card (~64K tokens) |
| Q4_K_L | 20.81 | 24GB card, ~8K tokens |
| Q4_K_M | 19.36 | 24GB card, ~16K tokens |
| IQ4_XS | 17.36 | 24GB card, ~32K tokens |
| IQ3_M | 16.58 | 20GB card |
| Q3_K_M | 14.83 | 16-20GB card, large quality loss |
| IQ2_M | 11.35 | 12-16GB card, not recommended |
“Fits” is our estimate: weights plus a q8_0 KV cache, before ~1 GiB of compute buffers. It is not a measured minimum.
This model is sensitive to quantization. AtomicChat measured that plain llama.cpp Q6_K picks the original’s next token only 95.5% of the time; “most models reach 98% at 6 bits.” Their plain Q4_K_M scored 89.9%. Their tuned AD-Q4_K_M (21.2 GB) reached 92.6%. Their AD-IQ2_S fell to 80.5%. So 2-bit and 3-bit files lose more here than on Qwen.
Why the Cache Is the Real Limit
The weights are only part of the cost. This is our arithmetic from config.json:
- Nanbeige4.3 33B: 2 (K and V) x 52 layers x 8 KV heads x 128 x 2 bytes = 212,992 bytes = 208 KiB per token at f16.
- Qwen3.8 27B: full attention on only 16 of 64 layers, 4 KV heads, head size 256 = 64 KiB per token.
| Context | Nanbeige f16 | Nanbeige q8_0 | Qwen3.8 27B f16 |
|---|---|---|---|
| 16,384 | 3.25 GiB | 1.73 GiB | 1.00 GiB |
| 32,768 | 6.50 GiB | 3.45 GiB | 2.00 GiB |
| 131,072 | 26.0 GiB | 13.8 GiB | 8.00 GiB |
| 524,288 (max) | 104 GiB | 55.3 GiB | 32.0 GiB |
q8_0 cache is 53.1% of f16, not 50%. Our KV cache guide shows why.
This is the thing the benchmark table does not tell you. On a 24GB card, Qwen3.8 27B at Q4 leaves room for about 64K tokens at f16. Nanbeige at Q4 leaves room for about 16K-32K at q8_0. The full 512K context needs ~104 GiB of cache alone. One DGX Spark build ran 262K tokens and measured 100,247 MiB for the whole server.
What Fits Your Machine
| Machine | Best file | Context (our estimate) | Verdict |
|---|---|---|---|
| 12-16GB card | IQ2_M (11.35 GiB) | 8K | No. Run Qwen3.8 27B on 12GB instead |
| 24GB card (RTX 3090 / 4090) | IQ4_XS or Q4_K_M | 16K-32K at q8_0 | Yes |
| 32GB card (RTX 5090 / R9700) | Q6_K or Q5_K_M | 16K (Q6_K), 64K (Q5_K_M) | Yes, best single card |
| 32GB Mac (~25 GiB for the GPU) | IQ4_XS | ~16K at q8_0 | Tight |
| 36GB Mac (~27 GiB for the GPU) | Q4_K_M | ~32K at q8_0 | Yes |
| 64GB+ Mac / Strix Halo | Q8_0 | 32K+ at q8_0 | Yes, slow |
| DGX Spark (128GB) | Q6_K or Q8_0 | 262K | Yes, measured below |
Mac GPU limits are the logged values in our 36GB Mac guide. Raise them with the iogpu.wired_limit_mb sysctl at your own risk.
For a 24GB card, a used RTX 3090 averaged $1,563 on resale trackers as of October 2026. For 6-bit quality, an RTX 5090 is the single card that fits Q6_K; marketplace listings sat near $6,332 on 30 October 2026. Check current listings before you buy.
How Fast It Runs
Measured (community-reported). The dedene DGX Spark build uses llama.cpp b11870 on one DGX Spark (~273 GB/s):
| Setup (Q6_K, thinking off) | Decode tok/s | Prefill at 41K tok/s |
|---|---|---|
| No drafting | 7.3 | 494 |
| 3B draft model (Nanbeige4.1-3B) | 12.6 | 420 |
| 3B draft + n-gram | 12.6 | 429 |
The same repo lists speed per quant with the drafter (code / English): Q8_0 15 / 10, Q6_K 16 / 11, Q5_K_M 18 / 12, Q4_K_M 19 / 14.
Estimated, not measured. A dense model reads all its weights for every token. So decode speed is close to memory bandwidth divided by file size. The Spark reached 78% of that limit (7.3 of 9.4 tok/s). At the same 78%, Q4_K_M (20.79 GB) gives:
| Card | Bandwidth | Q4_K_M estimate |
|---|---|---|
| RTX 3090 | 936 GB/s | ~35 tok/s |
| R9700 | 640 GB/s | ~24 tok/s |
| RTX 5090 | 1,792 GB/s | ~67 tok/s |
Your number will differ. Run llama-bench -m <file> -p 512 -n 128 and compare.
Run It
llama-server -hf bartowski/Nanbeige_Nanbeige4.3-33B-GGUF:IQ4_XS \
--jinja -c 32768 -ngl 99 -fa on -ctk q8_0 -ctv q8_0- Thinking is on by default. Turn it off with
--chat-template-kwargs '{"enable_thinking": false}'. - Sampling from the card: agent and tool use
--temp 1.0 --top-p 0.95; chat and reasoning--temp 0.6 --top-p 0.95. - Tool calls: the card recommends
tool_call_format="xml". - Draft model: Nanbeige4.1-3B helps on a Spark, but llama.cpp refuses it as is. The two files pad the vocabulary to different sizes (166,912 vs 166,144).
Should You Switch From Qwen3.8 27B?
The same DGX Spark author ran an agent test suite on 25-26 October against Qwen3.8 Flash Next (a different, faster MoE model). Community-reported:
| Test | Nanbeige4.3 33B | Qwen3.8 Flash Next |
|---|---|---|
| BFCL multi-turn | 56.5 | 54.5 |
| AgentDojo, tasks done (no attack) | 79.2 | 84.4 |
| AgentDojo, attacks resisted | 58.5 | 99.0 |
| IFEval prompt strict | 74.7 | 85.8 |
| Decode tok/s (weighted) | 12.3 | 43.6 |
- Switch if you run an agent harness on a 32GB card and want the best dense model for tool use. The lab’s own agent scores lead.
- Stay on Qwen3.8 27B if you need long context on 24GB, or if the agent reads untrusted web pages or email. The prompt-injection result is the biggest risk we found. Read our prompt injection guide.
Sources
- Nanbeige/Nanbeige4.3-33B model card and config.json (read 2026-10-31)
- bartowski GGUF files (sizes from the Hugging Face API, 2026-10-31; b11845 note)
- AtomicChat GGUF (KLD and top-1 table, community-measured)
- dedene/Nanbeige4.3-33B-DGX-Spark (DGX Spark speeds, memory, agent tests, community-reported)
- daily.dev release roundup (2026-10-26)
- Prices: resale tracker (used RTX 3090) and GPU Poet (RTX 5090), read 2026-10-30
See Also
- Can I Run SenseNova Loop 9B Locally? — a 9B that computes like a 14B, GGUF sizes and the patched-build caveat
- Best Local LLM for 24GB VRAM
- Is 24GB VRAM Enough?
- Best Local LLM for RTX 5090
- Qwen 3.8 27B GSQ-RCO Quants
- llama.cpp KV Cache Quantization