Xing4.0 29B A4B on RTX 3090 (2026): Fits, Needs a Fork

Bottom Line

• Yes, one RTX 3090 runs Xing4.0-29B-A4B fully on the GPU. The official 4-bit GGUF is 18.72 GiB. China Telecom’s own llama.cpp guide was tested on an RTX 3090 at 64K context.
• You cannot run it in stock llama.cpp, Ollama or LM Studio yet. Support is an open, unmerged pull request (#29012, opened 2026-09-17). Today you need the xing4_0-port fork or the vendor’s prebuilt Windows package.
• Long context is cheap on this model. It uses MLA attention. By our math the full 256K cache is about 11.25 GiB in f16, and about 6 GiB at q8_0.
• A 16GB card needs a community quant, not the official file. Two exist at 12.04 GiB and 12.92 GiB.
• It is not a clear upgrade over Qwen3.6-35B-A3B. The vendor’s own table has Qwen winning 5 of 9 benchmarks, including SWE-bench Verified.

We have not found a public tokens-per-second number measured on an RTX 3090. The speed figures below come from other hardware and are marked community-reported.

What Xing4.0-29B-A4B Is

Xing4.0 is the new name of China Telecom’s TeleChat series. The Hugging Face repo XingChen-AGI/Xing4.0-29B-A4B was created on 2026-09-16 under the Apache-2.0 license.

From the model card and config.json, read 2026-09-18:

Spec	Value
Total / active parameters	29B / 4B per token (card); 29.51B per the llama.cpp PR
Architecture	Xing4_0ForCausalLM, MoE with MLA attention, mHC hyper-connections, MTP
Layers	40 (first 2 dense)
Experts	64 routed + 1 shared, 4 active per token
Attention	MLA: kv_lora_rank 512, qk_rope_head_dim 64, 32 heads
Context	256K native (max_position_embeddings 262,144); card says extensible to 512K
Vocab	131,072
MTP layers	1 (num_nextn_predict_layers)

The card says the model was trained entirely on Huawei Ascend NPUs. It also says the model has “targeted adaptation and format alignment” for OpenCode, Claude Code, OpenClaw and Hermes. We did not test that claim.

Can You Load It Today?

Runtime	Status on 2026-09-18
llama.cpp (stock)	No. PR #29012 “Add Support for Xing4.0” is open and unmerged. The repo’s bot flagged it as a large, multi-backend PR.
llama.cpp fork shuxiaoqiong/llama.cpp, branch xing4_0-port	Yes. This is the branch the official guide builds, with CPU and CUDA support.
Vendor prebuilt Windows package	Yes, on NVIDIA cards from the 3090 generation (sm_86) and newer, per the guide.
Ollama / LM Studio	No. Both load GGUF through llama.cpp-based engines, and we found no Xing4.0 support in either.
MLX (Apple Silicon)	No. mlx-lm PR #1901 “Add xing4_0 model support” is open. The MLX quant publisher says the files “will not load anywhere” until it merges.
vLLM, SGLang, KTransformers	Listed as supported on the model card. We did not test them.

If you pull the GGUF into Ollama or LM Studio now, expect an unknown-architecture error. That is not a broken download.

What Fits: File Sizes

File sizes come from the Hugging Face API file trees, read 2026-09-18.

Repo	File	Size
XingChen-AGI (official)	IQ4_NL, 3 split files	18.72 GiB
SolenopsisCampo	Q4_K_M	17.65 GiB
SolenopsisCampo	IQ3_M	12.92 GiB
SolenopsisCampo	IQ3_XXS	10.79 GiB
jmarceno	dynamic 3.31 bpw (IQ4_XS/IQ2_XXS experts, Q8_0 attention)	12.04 GiB
SolenopsisCampo	f16	58.16 GiB
suzu89 (MLX, Mac)	4-bit / 6-bit / 8-bit	15.51 / 22.37 / 29.23 GiB

The MLX quants drop the MTP layer. Its publisher reports that it is a 41st layer with 1.71B parameters.

KV Cache: Why MLA Makes Context Cheap

MLA stores one small compressed vector per token per layer instead of full keys and values for all 32 heads. From config.json, that vector is kv_lora_rank (512) plus qk_rope_head_dim (64) = 576 values.

40 layers x 576 values x 2 bytes (f16) = 46,080 bytes per token (45 KiB)
q8_0 (34 bytes per 32 values): 40 x 576 x 34/32 = 24,480 bytes per token

This is our arithmetic, and it assumes the fork caches the compressed form. The evidence says it does. If it stored full keys and values, the cache would be about 800 KiB per token and 64K context alone would need about 50 GiB. The official guide runs 64K on a 24GB card, and jmarceno reports running 262K across two 12GB cards.

Context	f16 cache (est.)	q8_0 cache (est.)
32K	1.41 GiB	0.75 GiB
64K	2.81 GiB	1.49 GiB
128K	5.63 GiB	2.99 GiB
256K	11.25 GiB	5.98 GiB

Fit by Card (estimates)

Each sum is file size plus our cache estimate. We exclude compute buffers, so treat each row as a floor.

Hardware	Recommended file	Context that fits	Sum
16GB card (RTX 5060 Ti 16GB, 4060 Ti 16GB)	jmarceno 12.04 GiB or IQ3_M 12.92 GiB	64K at q8_0; 128K is tight with the 12.04 file	13.5-15.0 GiB
24GB card (RTX 3090, 4090)	official IQ4_NL 18.72 GiB	128K at q8_0, 64K at f16	21.5-21.7 GiB
24GB card, full 256K	IQ4_NL + q8_0 cache	does not fit	24.70 GiB
32GB card (RTX 5090)	Q4_K_M 17.65 GiB	full 256K at f16	28.90 GiB
Mac, 24GB+ unified	MLX 4-bit 15.51 GiB	only after mlx-lm #1901 merges	15.6 GB peak at short prompts (publisher)

The official Windows script defaults to a 262,144 context. On a 24GB card our estimate says that does not fit with the 4-bit file. The vendor’s own run-server.bat uses 65,536 with a q8_0 cache, so start there.

Speed: What Has Been Reported

No one has published a tok/s number on an RTX 3090 that we could find. Here is what exists:

Source	Hardware	File	Reply speed
llama.cpp PR #29012 (the port’s author)	GPU not stated	IQ4_NL, 19 GiB	151.65 tok/s (tg128)
jmarceno GGUF card	2x RTX 3060 12GB	12.04 GiB dynamic	65.9 tok/s (tg128); 52.9 tok/s in llama-server
suzu89 MLX card	M5 Max, 128GB	4-bit MLX	21.2 tok/s

All three are community-reported. The PR author also reports perplexity going from 8.67 (F16) to 8.77 (IQ4_NL), about 1% worse.

With 4B active parameters, a 3090 should land between the two-3060 result and the PR figure. That is our inference, not a measurement. Measure your own with llama-bench from the fork build.

The model card claims MTP. We found no test of MTP speculative decoding for this model in llama.cpp. jmarceno’s launch command turns it off with --spec-type none, so do not count on an MTP speedup yet.

Xing4.0 vs Qwen3.6-35B-A3B

The model card compares Xing4.0 against Qwen3.6-35B-A3B and Gemma4-26B-A4B. All scores are vendor-reported by China Telecom.

Benchmark	Xing4.0-29B-A4B	Qwen3.6-35B-A3B	Winner
IFBench	69.67	65.50	Xing
AIME2026	90.00	92.70	Qwen
AA.LCR	61.00	62.00	Qwen
Tau3-Bench	64.63	67.20	Qwen
Claw-Eval	76.55	74.54	Xing
SWE-bench Verified	75.00	76.00	Qwen
Terminal-Bench 2.1	57.50	51.50	Xing
SWE-bench Multilingual	66.00	67.20	Qwen
DeepresearchBII	60.80	59.70	Xing

The thing most launch coverage skips: on the vendor’s own table, Qwen3.6-35B-A3B wins 5 of the 9 rows. That includes SWE-bench Verified, the coding number most people care about. The headline Claw-Eval 76.55 is a 2-point lead.

So who should try it?

• You run OpenClaw or terminal agents. Claw-Eval (+2.0) and Terminal-Bench 2.1 (+6.0) are the rows where Xing leads. Test it on your own tasks.
• You want a smaller file on 24GB. The official Xing 4-bit file is 18.72 GiB. Our MoE vs dense on 24GB page puts Qwen3.6-35B-A3B Q4_K_M files at 20-22GB. Xing leaves more room for context.
• Everyone else: stay on Qwen3.6-35B-A3B or the dense Qwen3.8-27B until llama.cpp merges support. They run in Ollama and LM Studio today.

One more practical note. The MLX quant’s publisher reports that the model “tends to write its reasoning trace in Chinese even for English prompts.” Turn thinking off (enable_thinking: false) if you read the trace.

How to Run It on an RTX 3090

1. Download the three xing4_0-29b-IQ4_NL-0000X-of-00003.gguf files from XingChen-AGI/Xing4.0-29B-A4B-GGUF.
2. On Windows, use the vendor’s prebuilt package from the official guide. On Linux, build shuxiaoqiong/llama.cpp on branch xing4_0-port with CUDA on.
3. Start the server at 64K with a q8_0 cache, as the vendor’s script does:

llama-server -m xing4_0-29b-IQ4_NL-00001-of-00003.gguf -ngl 999 -fa on \
  -c 65536 -ctk q8_0 -ctv q8_0 --jinja

4. Use the card’s sampling: temperature 0.8, top_p 0.95, repetition penalty 1.05 for coding and agents; temperature 1.0 for general reasoning.
5. Raise -c to 131072 only after you check free VRAM with nvidia-smi.

For what each flag does, see llama.cpp flags explained.

What to Buy

A used RTX 3090 is the card this model was tested on. It holds the official 4-bit file with room for 128K of q8_0 context.

Prices as of September 2026: our August reference range for a used 3090 was $1,000 to $1,300. On 2026-09-16 the gpudojo tracker showed used listings from $1,399. That is one scraped tracker, so check current listings. Read how to buy a used RTX 3090 safely first.

Check RTX 3090 24GB prices on Amazon

On a 16GB budget, the RTX 5060 Ti 16GB runs the 12 GiB community quants at 64K context. As of September 2026 it sells for $589 to $805, well above its $429 MSRP. You give up some quality for the smaller file: jmarceno measured +19.9% perplexity against the official IQ4_NL.

Check RTX 5060 Ti 16GB prices on Amazon

Do not buy a card for this model alone. Its runtime support is days old, and the Qwen models above run on the same hardware today.

Sources

• Hugging Face API, model card and config.json, XingChen-AGI/Xing4.0-29B-A4B (created 2026-09-16, Apache-2.0, 31,215,031,088 safetensors parameters including the MTP layer), read 2026-09-18
• Hugging Face API file trees: XingChen-AGI/Xing4.0-29B-A4B-GGUF, SolenopsisCampo/Xing4.0-29B-A4B-GGUF, jmarceno/Xing4.0-29B-A4B-GGUF, suzu89/Xing4.0-29B-A4B-{4,6,8}bit-MLX, read 2026-09-18
• jmarceno GGUF card: recipe, perplexity, 2x RTX 3060 speed and VRAM (community-reported), read 2026-09-18
• suzu89 MLX card: mlx-lm support status, M5 Max speed, MTP layer size (community-reported), read 2026-09-18
• Official llama.cpp deploy guide, XingChen-AGI/Xing4.0-29B-A4B GitHub tutorial/llama.cpp/README_EN.md (RTX 3090 test machine, 65,536 context, q8_0 cache), read 2026-09-18
• GitHub API: llama.cpp PR #29012 (open, created 2026-09-17) and issue #28153 (open); mlx-lm PR #1901 (open, created 2026-09-18), read 2026-09-18
• Price reference: used RTX 3090 and RTX 5060 Ti 16GB ranges from our tracked price sheet (Aug 2026, 3090 re-checked 2026-09-16)

See Also

• Qwen 3.8 27B on RTX 3090
• Best local LLM for the RTX 3090
• MoE vs dense on a 24GB card
• Qwen 3.6 vs Gemma 4 for agents
• Best local LLM for the RTX 5060 Ti 16GB
• Best 20B-35B local LLMs
• Lowest KV cache models for long context