DeepSeek V4.1 Flash GSQ-RCO Quants (2026): 12GB GPU, No Mac

Bottom Line

• A 12GB NVIDIA card now runs DeepSeek V4.1 Flash. It needs the ISTA-DASLab GSQ-RCO files, the Strata engine, 46-64GB of RAM and an NVMe drive.
• Speed is slow but usable. Strata’s author reports 9 to 11 tok/s on an RTX 3060 12GB and 4 to 6 tok/s on an RTX 5070 12GB. These are the engine author’s numbers.
• The backbone is 112.43 to 144.57 GiB. The shared Engram file is 103.00 GiB, and it stays on the SSD. Byte counts are from the Hugging Face API.
• Stock llama.cpp does not load these files. Upstream merged V4.1 support on 2026-10-25, but the ISTA files use different metadata keys and a separate Engram file.
• No Mac runs them. Strata’s Mac guide says so, and its DeepSeek engine ships for x86_64 Windows and Linux only.
• The “100% of the base model” claim rests on three benchmarks. Strata measured the same 2.00 bpw file at +15.5% perplexity.

Every size on this page is a byte count from the Hugging Face API, read 2026-10-25. We did not run these files. Speeds come from Strata’s own documentation and are labelled as such.

This page updates our earlier verdict, Can You Run DeepSeek V4.1 Flash Locally?. That page said no consumer machine runs the model. For a PC with an NVIDIA card, that is no longer true.

What ISTA-DASLab Actually Shipped

The repo is ISTA-DASLab/DeepSeek-V4.1-Flash-1017-GSQ-RCO-GGUF. It was created 2026-10-23 and last modified the same day. The API reports 412,392 downloads and 249 likes. The license is MIT.

The base model is deepseek-ai/DeepSeek-V4.1-Flash-1017. DeepSeek published it on 2026-10-17 and calls it “a minor update” to V4.1 Flash. Its checkpoint is 48 safetensors files and 510,296,708,312 bytes. That is the same 510.30 GB our earlier page measured for the September release.

The card describes a 763.2B-parameter MoE with 384 routed experts per layer, 6 active per token. It splits the weights in two:

• Backbone, 551.9B parameters. These are the transformer weights. The bpw target applies to them only.
• Engram, 196.6B parameters. These are two n-gram lookup tables. The card holds them at IQ4_NL, 4.50 bpw, in every build.

Two methods made the backbone. GSQ (arXiv 2604.18556) quantizes each tensor at a given GGUF type. RCO (arXiv 2605.00649) picks the type for each tensor under an exact size budget. Dense tensors draw from Q5_K, Q6_K and Q8_0. Routed experts draw from IQ1_S to IQ3_S.

File Sizes, in GiB

The card lists sizes in decimal GB. Your tools report GiB. These are the raw bytes from the API tree, converted.

File	Bytes	GiB
GSQ-RCO-IQ1_M-00001-of-00004	39,358,830,464	36.66
GSQ-RCO-IQ1_M-00002-of-00004	39,176,113,248	36.49
GSQ-RCO-IQ1_M-00003-of-00004	39,997,334,720	37.25
GSQ-RCO-IQ1_M-00004-of-00004	2,188,841,824	2.04
GSQ-RCO-IQ2_XXS-00001-of-00004	39,075,714,624	36.39
GSQ-RCO-IQ2_XXS-00002-of-00004	39,388,449,504	36.68
GSQ-RCO-IQ2_XXS-00003-of-00004	39,246,891,616	36.55
GSQ-RCO-IQ2_XXS-00004-of-00004	20,527,837,312	19.12
GSQ-RCO-IQ2_XS-00001-of-00004	39,535,776,992	36.82
GSQ-RCO-IQ2_XS-00002-of-00004	39,494,617,536	36.78
GSQ-RCO-IQ2_XS-00003-of-00004	39,423,838,496	36.72
GSQ-RCO-IQ2_XS-00004-of-00004	36,771,591,232	34.25
DeepSeek-V4.1-Flash-Engram-IQ4_NL.gguf	110,595,290,816	103.00

The repo also holds a README, two banner images, one benchmark plot and three tensor-allocation/*.rco-allocation.txt files of about 33 KB each.

Summed per build:

Build	bpw (target / measured)	Backbone	+ Engram	Total
IQ1_M	1.75 / 1.7499	112.43 GiB	103.00 GiB	215.43 GiB
IQ2_XXS	2.00 / 2.0038	128.75 GiB	103.00 GiB	231.74 GiB
IQ2_XS	2.25 / 2.2501	144.57 GiB	103.00 GiB	247.57 GiB

The measured bpw values come from the allocation files. Strata also downloads Unsloth’s DSpark drafter, 7,976,914,240 bytes (7.43 GiB). Strata’s own disk totals are 239, 257 and 274 GB.

Against Unsloth’s uniform-type GGUFs of the same base model, read from the same API. Unsloth ships the Engram table as the last shard, so we subtracted it to compare backbones:

Unsloth build	Backbone	Engram shard	Nearest ISTA build	Backbone
UD-IQ1_S	113.98 GiB	IQ4_NL	IQ1_M	112.43 GiB
UD-IQ1_M	130.21 GiB	IQ4_NL	IQ2_XXS	128.75 GiB
UD-IQ2_XXS	153.95 GiB	Q5_K	IQ2_XS	144.57 GiB

ISTA’s “2-bit” backbone is smaller than Unsloth’s “1-bit” UD-IQ1_M backbone. The names describe different things. Read the byte counts.

The Thing Nobody Else Says About the 2-Bit File

The card calls IQ2_XXS a 2.00 bpw build. The allocation file shows what that means for the experts.

We read tensor-allocation/DeepSeek-V4.1-Flash-1017-GSQ-RCO-IQ2_XXS.rco-allocation.txt. The model has 40 layers and three routed-expert tensors per layer, so 120 expert tensors. Their types:

Build	IQ1_S	IQ1_M	IQ2_XXS	IQ2_XS	IQ2_S	IQ3_XXS
IQ1_M (1.75)	83	15	20	—	2	—
IQ2_XXS (2.00)	—	60	42	18	—	—
IQ2_XS (2.25)	—	—	76	27	16	1

Half the expert tensors in the recommended “2-bit” build are IQ1_M. In the 1.75 bpw build, 83 of 120 are IQ1_S, the smallest type in the ladder. Every quantized dense tensor is Q5_K or Q8_0 in all three builds. RCO did not pick Q6_K for any tensor.

The layout is not even across depth. In the IQ2_XXS build, layers 0-6 are IQ1_M only. IQ2_XS first appears at layer 17. Only layer 39 is IQ2_XS in all three expert tensors. RCO spends more bits deeper in the network.

What the Low Bit Rate Costs You

This is the card’s own table, measured at reasoning effort 50. We reproduce it as published.

Variant	bpw	AIME25	GPQA-D	LCB v6	Task avg	Recovery
Base (FP8 / FP4)	n/a	100.00	90.91	89.71	93.54	100.0%
GSQ-RCO IQ1_M	1.75	100.00	88.38	88.57	92.32	98.7%
GSQ-RCO IQ2_XXS	2.00	100.00	90.91	89.71	93.54	100.0%

The card has no IQ2_XS row yet. It marks that row “TBD”.

Read this table with two facts in mind:

1. The base model scores 100.00 on AIME25. A quant cannot show a gain there, and a small loss can hide. One of the three columns is at its ceiling.
2. Strata measured the same files differently. Its DeepSeek guide reports, over 4,995 positions against the original model:

Build	KL divergence	Perplexity change	Strata’s notes
IQ1_M (1.75)	0.217	+21%	code +8.9%, agent work +10.2%, plain English prose +53%
IQ2_XXS (2.00)	0.170	+15.5%	code +8.6%, agent work +5.3%
IQ2_XS (2.25)	0.107	+7.3%	code +3.2%, agent work +2.2%

Both sources can be right. Three reasoning benchmarks can hold steady while the token distribution drifts. For code and agent work, IQ2_XXS is fine. For long prose, avoid IQ1_M. If you have the RAM, IQ2_XS has the lowest drift.

One more cost applies to small cards. Strata’s “Faster” mode uses 5 of the model’s 6 experts per token. It is the default on a PC with 16GB of graphics memory or less. Neither table above measured that mode. Use --full-quality if you want the model ISTA benchmarked.

Which Runtime Loads These Files

This is the part that decides whether you can run anything.

Strata: yes, on Windows and Linux. The Strata engine has a separate DeepSeek binary, strata-dsv41. Release 0.2.5 (2026-10-25) ships it for x86_64 Linux with CUDA, x86_64 Windows with CUDA, and x86_64 Linux with HIP for the R9700. Its README lists NVIDIA RTX 30, 40 and 50 cards and the Radeon AI PRO R9700. The app fixes context at 8K tokens.

Stock llama.cpp: no. Upstream merged DeepSeek V4.1 support in PR #31086 at 01:43 UTC on 2026-10-25. We read the merged loader. It requires the key deepseek41.attention.kv_source_layers. The ISTA shard header stores deepseek41.kv_source_layer_ids instead. Upstream also expects the Engram table inside the model’s own shard set as per_layer_token_embd. The ISTA files point to a separate file through deepseek41.engram.file. The ISTA card agrees: “these GGUFs do not run in llama.cpp yet.”

Unsloth’s GGUFs and llama.cpp: maybe. Unsloth’s Engram shard uses the same tensor name as the merged code. We did not test those files on master.

Mac: no. Strata’s Mac guide says “DeepSeek V4.1-Flash does not run on a Mac yet.” Its Strix Halo guide says the same. Its DeepSeek packages are x86_64 only, so a DGX Spark (ARM) is out too.

Context Memory

We did not compute a KV-cache figure for this model, and we give none. The config.json uses compressed sparse attention: a 128-token sliding window, per-layer compress_ratios, four kv_source_layer_ids and an indexer. The cache code in llama.cpp is a dedicated DSv4 path. We could not reduce it to one honest bytes-per-token number.

One measured hint exists. Strata saves a paused conversation at about 17 MB per 6.5K tokens, by its own docs. That is 17,000,000 / 6,500 = about 2.6 KB per token. It is saved state, not a KV-cache measurement. It does show that context is small next to a 112-145 GiB backbone at Strata’s 8K default.

Fit by Memory Tier

The Engram file stays on SSD in every row. Both ISTA and Strata say it is read a few rows per token. “Resident” means backbone plus the 7.43 GiB drafter fits in VRAM plus RAM. That is our arithmetic, not a test, and it ignores OS and buffer overhead.

Your machine	IQ1_M (112.43)	IQ2_XXS (128.75)	IQ2_XS (144.57)	Runs today?
12GB NVIDIA + 46-64GB RAM + NVMe	SSD-streamed	SSD-streamed	SSD-streamed	Yes, Strata. 4-11 tok/s reported
24GB NVIDIA + 128GB RAM	Resident (119.86 of 152)	Resident (136.18 of 152)	No (152.00 of 152)	Yes, Strata
96GB VRAM (RTX PRO 6000) + 128GB RAM	Resident	Resident	Resident	Not listed by Strata
4x Radeon AI PRO R9700 (128GB VRAM)	In VRAM (119.86 of 128)	Needs some RAM	Needs some RAM	Yes, Strata. 74-77 tok/s reported
64GB / 96GB unified Mac	Too small	Too small	Too small	No
128GB unified Mac	Too tight to use	Too small	Too small	No runtime
192GB Ryzen AI Max+ PRO 495 box	Fits on paper	Fits on paper	Fits on paper	No. Strata: no Strix Halo yet
256GB Mac Studio M5 Ultra	Fits on paper	Fits on paper	Fits on paper	No runtime
DGX Spark 128GB	Too tight	Too small	Too small	No. ARM, no package

“Fits on paper” is not “runs.” The big unified-memory machines have the room and no engine. The small PC has no room and the only engine. That is the inverse of most model launches this year.

Strata’s README speed table, measured with the 2.00 bpw set on Strata 0.2.5:

Hardware	Output	Prompt
RTX 5070 12GB, 64GB RAM	4 to 6 tok/s	about 110 tok/s (4K)
RTX 3060 12GB, 46GB RAM	9 to 11 tok/s	about 155 tok/s (4K)
1x Radeon AI PRO R9700, 251GB RAM	21 to 25 tok/s	320 to 360 tok/s (8K)
4x Radeon AI PRO R9700	74 to 77 tok/s	400 to 450 tok/s (8K)

These are the engine author’s measurements. Nobody has published an independent run yet. The docs say a 3.6 GB/s NVMe drive was in the 5070 box. Under about 40GB of RAM, Strata calls the setup “untested and slow.”

What to Buy

Prices are US street figures as of October 2026, from our hardware price reference.

If you own a 12GB+ RTX 30, 40 or 50 card: buy RAM and a drive, not a GPU. The model streams from SSD when RAM runs short. One build needs 239-274 GB free. All three need 533 GB. A 2TB NVMe drive holds a full set plus your other models.

Check SanDisk 2TB NVMe prices on Amazon

Strata’s fastest 12GB result had 46GB of RAM, and its 5070 result had 64GB. A 64GB DDR5 kit is the step to take. Kits are expensive in the 2026 shortage. A 64GB DDR5-5200 kit listed at $919.99 at Newegg on 2026-10-01.

Check Crucial DDR5 64GB kit prices on Amazon

If you have no supported card: an RTX 3060 12GB is the cheapest entry. It is the card in Strata’s fastest 12GB result. New units run $419-599+, and used units average about $307. Do not buy an Intel Arc B580 for this model. Strata ships its DeepSeek engine only for CUDA and for HIP on the R9700.

Check RTX 3060 12GB prices on Amazon

If you own a 128GB or 256GB Mac: do not buy anything for this model yet. Wait for a runtime. Until then, use the API or run DeepSeek V4-Flash, which does load on unified memory.

How to Run It

Strata’s Terminal app picks the files and shows its memory plan before it downloads anything:

strata launch --family deepseek41 --model ist-das-2.00bpw --plan

Drop --plan to download and start. Add --full-quality to use all 6 experts on a small card.

If you already have the files, point Strata at the folder. Keep the Engram file beside the shards:

hf download ISTA-DASLab/DeepSeek-V4.1-Flash-1017-GSQ-RCO-GGUF \
  --include "*IQ2_XXS*" "*Engram*" --local-dir ds41

strata launch --model-path ds41 --model ist-das-2.00bpw

The card’s llama-cli example uses -lm mmap --lazy-mode on. Do not expect it to work in stock llama.cpp. The card says the flags come from the build ISTA tested with, and the loader mismatch above still applies.

Audit the build before you argue about it. Each allocation file lists all 1,008 tensors with their types and the measured bpw. Almost no other quant repo publishes this.

Sources

• Hugging Face API, ISTA-DASLab/DeepSeek-V4.1-Flash-1017-GSQ-RCO-GGUF and its /tree/main listings, recursed into assets/ and tensor-allocation/ (all byte counts, createdAt, downloads, likes), read 2026-10-25
• ISTA-DASLab model card README.md (parameters, build table, results table, runtime note), read 2026-10-25
• tensor-allocation/*.rco-allocation.txt for IQ1_M, IQ2_XXS and IQ2_XS, read 2026-10-25
• GGUF headers of the IQ2_XXS shards 1 and 4 and the Engram file (range requests only), read 2026-10-25
• Hugging Face API and README for deepseek-ai/DeepSeek-V4.1-Flash-1017, plus its config.json, read 2026-10-25
• Hugging Face API, unsloth/DeepSeek-V4.1-Flash-1017-GGUF tree and engram/README.md, read 2026-10-25
• GitHub API, ggml-org/llama.cpp PR #31086 (merged 2026-10-25 01:43 UTC) and its file patches; draft PR #30817, read 2026-10-25
• Strata README, docs/DEEPSEEK41.md, docs/MAC.md, docs/STRIX_HALO.md and release v0.2.5 notes and assets, read 2026-10-25
• OpenClaw DC hardware price reference, last updated 2026-10-23

See Also

• Can You Run DeepSeek V4.1 Flash Locally? — the September verdict on the official 510GB checkpoint
• Qwen3.8-27B vs DeepSeek V4.1 Flash — the 24GB-card alternative
• DeepSeek V4-Flash vs gpt-oss-120b — the big MoE a Mac can actually load
• Can DeepSeek V4 Flash code like Claude? — the previous Flash, tested on a 128GB Mac
• Qwen 3.8 27B GSQ-RCO Quants — the same lab’s method on a 27B dense model
• Qwen3.8 Flash Next 2-Bit — another GSQ-RCO build whose n-gram shard stays on SSD
• Can you run a 160GB MoE on 8GB VRAM? — how expert streaming from SSD works
• Best local LLM for RTX 3060 12GB — what else the entry card runs
• How much SSD for a local model library? — sizing the drive