Can I Run LFM2.5 24B A2B Locally? (2026): Yes, on 16GB

Bottom Line

• Yes, a 16GB GPU runs LFM2.5-24B-A2B fully. Liquid AI’s QAD-Q4_0 GGUF is 12.65 GiB. On a 16GB card it leaves room for about 64K tokens of f16 context with stock llama.cpp settings.
• The context is cheap. Only 10 of 40 layers keep a KV cache. The full 131,072-token window costs 2.5 GiB in f16. With attention in all 40 layers, it would cost 4x that.
• 8GB and 12GB GPUs work too, with expert offload. Each token reads 4 of 64 experts per layer, so experts in system RAM cost little speed. Use --n-cpu-moe.
• On a Mac, buy 24GB. A 24GB Mac fits Q4_K_M plus the full 131K context. A 16GB Mac fits only the 3-bit file.
• CPU-only works on 32GB of RAM. Q4_K_M has a bandwidth ceiling of about 60 tok/s on dual-channel DDR5-5600, because only 1.48 GB is read per token.
• The license has a revenue cap. Commercial use is free only for companies under $10 million in annual revenue.

What the Model Is

All figures come from the LFM2.5-24B-A2B model card, its config.json and the Hugging Face API, read in October 2026.

Spec	Value
Repo	LiquidAI/LFM2.5-24B-A2B
Created on Hugging Face	12 October 2026
Total parameters	23,971,585,280 (card: 24.0B)
Active parameters	2.45B per token
Layers	40: 30 short-convolution blocks + 10 attention (GQA) blocks
Experts	64 per MoE block, top-4 routing; the first 2 layers are dense
Context	131,072 tokens
Thinking	Always on; the template opens every answer with <think>
Languages	16, including English, Chinese, Hindi and Arabic
License	LFM Open License v1.0 (revenue cap, see below)

The card calls it the “largest and final model in the LFM2.5 family”. Liquid AI trained it with agentic reinforcement learning inside OpenClaw, Hermes Agent, Pi and OpenCode. So it knows those harnesses’ tools and prompts.

The 2.45B figure checks out. From config.json, each MoE layer holds 64 experts of 3 x 2,048 x 1,536 weights. That is 22.95B expert weights across 38 MoE layers. The other 1.02B weights are shared. Per token: 1.02B shared + 38 layers x 4 experts x 9.44M = 2.45B.

Every GGUF Size, in Bytes

Sizes from the Hugging Face file trees for LiquidAI/LFM2.5-24B-A2B-GGUF and unsloth/LFM2.5-24B-A2B-GGUF. “Read per token” is our estimate: 2.45B active weights at the file’s average bits per weight.

File	Source	Bytes	GiB	Bits/weight	Read per token
UD-IQ2_XXS	Unsloth	7,468,302,592	6.96	2.49	0.76 GB
UD-IQ2_M	Unsloth	7,535,313,152	7.02	2.51	0.77 GB
UD-Q2_K_XL	Unsloth	8,042,561,760	7.49	2.68	0.82 GB
UD-IQ3_XXS	Unsloth	8,959,328,480	8.34	2.99	0.92 GB
UD-Q3_K_XL	Unsloth	10,947,100,928	10.20	3.65	1.12 GB
QAD-Q4_0	Liquid AI	13,578,190,880	12.65	4.53	1.39 GB
Q4_K_M	Liquid AI	14,526,258,464	13.53	4.85	1.48 GB
UD-Q4_K_XL	Unsloth	14,876,901,632	13.86	4.96	1.52 GB
Q5_K_M	Liquid AI	17,029,602,592	15.86	5.68	1.74 GB
UD-Q5_K_XL	Unsloth	17,918,820,608	16.69	5.98	1.83 GB
Q6_K	Liquid AI	19,689,405,728	18.34	6.57	2.01 GB
UD-Q6_K_XL	Unsloth	21,861,204,224	20.36	7.30	2.23 GB
Q8_0	Liquid AI	25,493,732,640	23.74	8.51	2.61 GB
BF16	Liquid AI	47,962,094,880	44.67	16.01	4.90 GB

QAD-Q4_0 is not an ordinary Q4_0. The GGUF card says it is a “Quantization-Aware Distillation checkpoint”, “trained to recover the accuracy lost to Q4_0 quantization”. It is also 0.88 GiB smaller than Q4_K_M. Liquid AI publishes no score for it, so we cannot say how close it gets.

Why the Context Costs So Little

LFM2.5 is a hybrid. Config.json lists layer_types: 30 conv layers and 10 full_attention layers. Only the attention layers keep a cache that grows with context.

1. The growing KV cache. Config.json gives 32 attention heads, 8 KV heads and a hidden size of 2,048. So the head dimension is 2,048 / 32 = 64.

f16:  2 (K and V) x 10 layers x 8 KV heads x 64 dims x 2 bytes = 20,480 bytes = 20 KiB per token
q8_0: same 10,240 values x 1.0625 bytes                       = 10,880 bytes per token

f16 cache:   32K = 640 MiB    64K = 1,280 MiB    131,072 = 2,560 MiB
q8_0 cache:  32K = 340 MiB    131,072 = 1,360 MiB

2. The fixed convolution state. conv_L_cache is 3. The llama.cpp source sizes this state as hidden size x (L_cache - 1).

30 conv layers x 2,048 x 2 x 4 bytes (F32) = 491,520 bytes = 0.47 MiB per sequence

That is small enough to ignore, even with several server slots.

3. The compute buffer. We did not measure it. The output logits alone need 512 x 128,000 x 4 bytes = 250 MiB at llama.cpp’s default 512-token batch. We budget 1 GiB for the compute buffer and the GPU driver together.

4. llama.cpp’s safety margin. The --fit option is on by default. It keeps 1,024 MiB free on each GPU (fit_params_target in common/common.h). You can lower it with --fit-target.

Fit Table by Hardware

GPU rows: card memory minus 1 GiB --fit margin, minus 1 GiB for compute and driver, minus weights. Mac rows use the Metal working-set limit instead. Every row is our estimate, not a measurement.

Hardware	Best file	Fits fully?	Context that fits	Speed ceiling (estimate)
8GB GPU (RTX 4060)	Q4_K_M	No; offload experts	32K f16 with --n-cpu-moe 26	~104 tok/s (DDR5)
12GB GPU (Arc B580, RTX 3060)	Q4_K_M	No; offload experts	32K f16 with --n-cpu-moe 15	~150-170 tok/s
12GB GPU, all on GPU	UD-IQ3_XXS (8.34 GiB)	Yes	About 64K f16	~490 tok/s (B580)
16GB GPU (RTX 5060 Ti 16GB)	QAD-Q4_0	Yes	About 64K f16, or about 128K q8_0	~320 tok/s
16GB GPU	Q4_K_M	Yes	About 24K f16 at stock margin	~300 tok/s
24GB GPU (RTX 3090)	Q6_K	Yes	Full 131K f16	~465 tok/s
16GB Mac	UD-Q3_K_XL	Yes, tight	32K f16	~137 tok/s (M6, 153 GB/s)
24GB Mac	Q4_K_M or MLX 4-bit	Yes	Full 131K f16	~115 tok/s (M6, 170 GB/s)
32GB Mac	Q6_K	Yes	Full 131K f16	~84 tok/s (M6)
16GB RAM, CPU only	UD-Q3_K_XL	Yes, tight	32K f16	~80 tok/s DDR5, ~46 DDR4
32GB RAM, CPU only	Q4_K_M or Q6_K	Yes	Full 131K f16	~60 tok/s DDR5 (Q4_K_M)

How we got the rows:

• 16GB GPU, QAD-Q4_0: 16,384 - 1,024 - 1,024 - 12,949 MiB = 1,387 MiB for cache. That holds 64K f16 (1,280 MiB) or about 128K q8_0 (1,360 MiB). Q4_K_M leaves only 483 MiB, so it gets about 24K f16.
• 24GB GPU, Q6_K: 24,576 - 2,048 - 18,777 = 3,751 MiB. The full 131K f16 cache needs 2,560 MiB. Q8_0 (24,313 MiB) does not fit.
• 16GB Mac: a llama.cpp log from a 16GB M2 Mac mini (issue #20704) reports recommendedMaxWorkingSetSize = 12713.12 MB. That is 12,124 MiB. UD-Q3_K_XL (10,440 MiB) + 32K cache (640) + a 512 MiB compute buffer = 11,592 MiB. Q4_K_M and the 12.64 GiB MLX 4-bit build do not fit.
• 24GB and 32GB Macs: we assume the same ~75% limit: about 18 GiB and 24 GiB. Q4_K_M + 131K f16 + 512 MiB = 16.5 GiB. Q6_K + 131K f16 + 512 MiB = 21.3 GiB.
• CPU only: we leave about 4 GiB for the operating system. On 16GB, Q4_K_M (13.53 GiB) leaves too little. On 32GB, Q8_0 (23.74 GiB) fits with a short context, but Q6_K is the safer pick.

How to Run It on 8GB and 12GB GPUs

On a small card, offload whole experts, not whole layers. The --n-cpu-moe N flag keeps the expert weights of the first N layers in system RAM. Attention and shared weights stay on the GPU.

Why this model suits it. Per layer, the experts weigh about 349 MiB at Q4_K_M. But each token reads only 4 of the 64 experts: about 23 MB per layer. So a layer of experts in RAM costs 23 MB of RAM traffic per token, not 349 MB.

Our sum for an 8GB card: 8,192 - 1,024 margin - 1,024 compute - 640 for 32K of cache = 5,504 MiB. The shared weights take about 590 MiB (1.02B weights at 4.85 bits). That leaves room for about 14 MoE layers of experts. The other 24 go to RAM. The first 2 layers are dense, so the flag value is 24 + 2 = 26.

# 8GB GPU, about 8.2 GiB of experts in system RAM
llama-server -hf LiquidAI/LFM2.5-24B-A2B-GGUF:Q4_K_M --jinja -ngl 99 \
  --n-cpu-moe 26 -c 32768 -fa on

# 12GB GPU, about 4.4 GiB of experts in system RAM
llama-server -hf LiquidAI/LFM2.5-24B-A2B-GGUF:Q4_K_M --jinja -ngl 99 \
  --n-cpu-moe 15 -c 32768 -fa on

These values are starting points from our arithmetic. Lower N one step at a time until speed drops or VRAM spills. Our —n-cpu-moe guide shows the sweep. Use 16GB of system RAM or more.

Speed ceiling on an RTX 4060 with 26 offloaded: 0.94 GB per token on the GPU at 272 GB/s, plus 0.55 GB per token in RAM. On DDR5-5600 dual channel (89.6 GB/s) the ceiling is about 104 tok/s. On DDR4-3200 (51.2 GB/s) it is about 70 tok/s.

Runtime Support, Checked Today

Runtime	Status	Source
llama.cpp	Supported. The LFM2-MoE architecture landed in PR #16464, merged 7 October 2025, first tag b6709	GitHub compare against release tags
Official GGUF	Liquid AI publishes 7 GGUF files, including QAD-Q4_0	Hugging Face file tree
Ollama	The library lists lfm2.5:24b (Q4_K_M, 15GB), 24b-a2b-q4_0 (14GB) and 24b-a2b-q8_0 (25GB), all “128K context window”	ollama.com/library/lfm2.5/tags
MLX	Liquid AI publishes MLX 4-bit, 5-bit, 6-bit, 8-bit and BF16 builds. The 4-bit weights total 13,567,294,335 bytes	Hugging Face API
LM Studio	Listed on the model card as supported. We did not test it	Model card

LFM2.5 doubled the vocabulary from 65K to 128K tokens. We did not find which llama.cpp build first reads the new tokenizer. Use a build from October 2026. The GGUF card says each file carries its sampling settings as metadata, and llama.cpp applies them by default: temperature 0.1, top-k 50, repeat penalty 1.05.

Speed: One Vendor Number, the Rest Is Arithmetic

Liquid AI’s card reports 184 tok/s on an Apple M5 Max and 114 tok/s on an AMD Ryzen AI Max+ 395. That is llama.cpp, Q4_K_M and a 4K-token input, on Liquid AI’s own benchmark system. We found no independent measured numbers as of 17 October 2026.

Ceilings for other hardware are bandwidth divided by bytes read per token. Real runs land below them.

Hardware	Arithmetic	Ceiling
Ryzen AI Max+ 395 (256 GB/s), Q4_K_M	256 / 1.48 GB	~173 tok/s; vendor measured 114
RTX 5060 Ti 16GB (448 GB/s), QAD-Q4_0	448 / 1.39 GB	~323 tok/s
RTX 3090 (936 GB/s), Q6_K	936 / 2.01 GB	~465 tok/s
Mac mini M6 24GB (170 GB/s), Q4_K_M	170 / 1.48 GB	~115 tok/s
CPU, DDR5-5600 dual channel (89.6 GB/s), Q4_K_M	89.6 / 1.48 GB	~60 tok/s
CPU, DDR4-3200 dual channel (51.2 GB/s), Q4_K_M	51.2 / 1.48 GB	~34 tok/s

The vendor’s Strix Halo figure is 66% of our ceiling. Small MoE models rarely reach the bandwidth ceiling, because fixed overhead per token matters more. On fast GPUs, expect a much lower share.

Thinking costs time. The model always reasons before it answers. A 1,000-token reasoning trace at 60 tok/s takes about 17 seconds before the first word of the answer.

What the Benchmarks Say

These are Liquid AI’s numbers from the model card. We did not run them. All four models ran in vLLM.

Benchmark	LFM2.5-24B-A2B	Qwen3.6-35B-A3B	Gemma-4-26B-A4B-it	Nemotron-3.5-Lightning (30B-A3B)
LiveClaw	48.09	38.19	41.25	37.72
BFCLv4	65.01	65.00	57.09	53.78
ToolSandbox	80.71	75.02	72.09	75.13
AutomationBench	27.69	8.72	5.46	2.61
IFEval	96.75	93.06	92.00	94.19
GPQA Diamond	74.42	83.69	74.90	72.42
WebArena Verified	26.75	37.62	25.62	28.69
SWE-Bench Verified (OpenCode)	51.00	66.20	25.80	53.40
SciCode	32.71	37.35	35.42	27.78

Where it is strong: tool calling and instruction following. It leads all four tool-use rows above.

Where it is weaker: coding and knowledge. Qwen3.6-35B-A3B scores 15 points higher on SWE-Bench Verified and 9 points higher on GPQA Diamond. The card says it “is not recommended for knowledge-heavy tasks on its own”. It advises pairing it with retrieval or web search.

The License: Free Under $10 Million

LFM2.5 uses the LFM Open License v1.0, not Apache 2.0. We read the full text in the repo’s LICENSE file.

• Personal, research and non-commercial use: allowed, with no revenue limit.
• Commercial use: allowed only if you or your legal entity do not exceed the “Threshold”. The license defines it as annual revenue of $10,000,000 or more. Your legal entity includes companies that control you or that you control.
• Over $10 million: “Any Commercial Use … by a Legal Entity that exceeds the Threshold is not licensed.” You need a separate deal with Liquid AI.
• Qualified non-profits: the threshold does not apply to their non-commercial or research use.
• Redistribution: include the license and mark any files you changed.
• Patents: if you sue anyone claiming the model infringes a patent, your patent license ends.

For a home lab or a small business, none of this blocks you.

What to Buy

If you own an 8GB or 12GB card, try it first. Use the --n-cpu-moe commands above. Each token reads so little from RAM that the loss is small.

If you are buying for this model, buy 16GB. It is the smallest tier that runs a 4-bit file fully on the GPU with 64K of context. Prices are as of October 2026 and move weekly.

Hardware	Price (as of October 2026)	What it runs
Intel Arc B580 12GB	$329.99 Newegg; $369.99 Amazon	Q4_K_M with expert offload, or UD-IQ3_XXS fully
RTX 5060 Ti 16GB	$589-805 range; $789 at B&H on 24 September	QAD-Q4_0 fully, 64K f16 context
RTX 3090 24GB (used)	about $1,518 average used	Q6_K and the full 131K context
Mac mini M6 24GB	$1,099	Q4_K_M and the full 131K context

The RTX 5060 Ti 16GB is the pick for this model. Buy the 16GB card, not the 8GB RTX 5060 Ti with the same name. On a smaller budget, the Arc B580 12GB runs Q4_K_M with expert offload. If you want Q6_K and the full 131K context on one card, a used RTX 3090 24GB does it.

On a Mac, skip 16GB. The Mac mini M6 starts at $899 with 16GB, which fits only the 3-bit file. The 24GB configuration costs $1,099 (Apple configurator, read 22 September 2026) and holds Q4_K_M with the full context.

Honest Caveats

1. We did not run the model. Every fit row is arithmetic from the file sizes, config.json and the llama.cpp source.
2. The compute buffer is a budget, not a measurement. We used 1 GiB on GPUs and 512 MiB on Macs. Check the buffer sizes in your llama-server log.
3. The --n-cpu-moe values assume equal-size layers. Real Q4_K_M files mix quant types per tensor. Sweep the value.
4. The Mac limits for 24GB and 32GB are assumptions. We found a log for 16GB only and assumed the same ~75% share.
5. Liquid AI publishes no score for QAD-Q4_0. Compare it with Q4_K_M on your own tasks before you trust it.
6. The benchmark table is the vendor’s. Liquid AI chose the benchmarks and the comparison models.

Sources

• LFM2.5-24B-A2B model card: parameters, layers, experts, context, benchmarks, edge speed numbers, runtime list
• config.json: layer_types, heads, KV heads, hidden size, conv_L_cache, experts, moe_intermediate_size, num_dense_layers
• Hugging Face API: creation date 12 October 2026, 23,971,585,280 parameters
• File trees for LiquidAI GGUF, Unsloth GGUF and MLX 4-bit: byte sizes
• LiquidAI GGUF card: QAD-Q4_0 description and sampling metadata
• LICENSE: LFM Open License v1.0
• llama.cpp source on GitHub, master branch: src/llama-hparams.cpp (conv state size), common/common.h (--fit 1,024 MiB target, default batch 512), common/arg.cpp (server slots auto by default)
• llama.cpp PR #16464: LFM2-MoE support, merged 7 October 2025
• llama.cpp issue #20704: Metal working set on a 16GB M2 Mac mini
• Ollama lfm2.5 tags: tag names and sizes
• Memory bandwidth: NVIDIA (RTX 3090 936 GB/s, RTX 5060 Ti 448 GB/s), Guru3D (RTX 4060 272 GB/s), Apple (Mac mini M6 153-170 GB/s); system RAM figures are channel arithmetic
• OpenClaw DC hardware price reference, last updated 13 October 2026: Arc B580, RTX 5060 Ti, RTX 3090 and Mac mini M6 rows

See Also

• Best local LLM for 16GB VRAM: what else fits the tier this model wins
• Best local LLM for 12GB VRAM: the 12GB alternatives without offload
• Best local LLM for 8GB VRAM: what fits without expert offload
• Best local LLM for 24GB VRAM: the RTX 3090 tier
• Best local LLMs for 16GB RAM: the CPU-only and 16GB Mac tier
• Best local LLMs for 32GB RAM: where Q6_K fits without a GPU
• —n-cpu-moe explained: how to sweep the offload value
• Run an LLM without a GPU: the CPU-only setup
• Which RTX 5060 Ti 16GB to buy: pick the card for this model