← All guides
Guides October 25, 2026

Can I Run Ornith 1.5 122B Locally? (2026): Yes, Q6 on 128GB

Ornith AI put Ornith-1.5-122B on Hugging Face on 16 October 2026. It is a further-trained Qwen3.5-122B-A10B: the same mixture-of-experts layout, about 10B active parameters per token, MIT licence. We summed every GGUF file in bytes from the Hugging Face API, did the cache arithmetic from config.json, and checked llama.cpp, Ollama and MLX. The short answer: a 128GB machine runs it at 6 bits with the full 262K context.

Bottom Line

  • Yes. A 128GB machine runs Ornith 1.5 122B at 6 bits with the full context. Q6_K is 95.35 GiB. The 262,144-token cache adds only 6.0 GiB.
  • The cheapest 128GB route is a Strix Halo mini PC. One owner reports UD-Q6_K_XL at ~25 tok/s with 262K context on Ubuntu (community-reported).
  • A 96GB RTX PRO 6000 holds the official Q5_K_M (82.62 GiB) with the full context, all on the GPU.
  • A 24GB or 32GB GPU works with system RAM. Put the experts in RAM with --n-cpu-moe. 64GB of RAM holds a 4-bit file.
  • 64GB of VRAM or a 64GB Mac means 3 bits or less. No 4-bit GGUF plus a long context fits.
  • The cache is small. Only 12 of 48 layers keep a KV cache: 24,576 bytes per token.
  • Every common runtime loads it today: llama.cpp, Ollama (ornith-1.5:122b), LM Studio, MLX, vLLM and SGLang.

Ready to buy? See the tested hardware list with current prices.

What Ornith 1.5 122B Is

Figures come from the model card, its config.json and the Hugging Face API, read on 25 October 2026.

SpecValue
Repoornith-ai/Ornith-1.5-122B
Created on Hugging Face16 October 2026
Base modelQwen3.5-122B-A10B (AtomicChat GGUF card)
Parameters in the checkpoint125,086,497,008 (BF16, includes vision and MTP weights)
Active per token~10B (the base model’s “A10B”); our count from config.json is 9.46B
Layers48: 36 Gated DeltaNet (linear) + 12 full attention
Experts256 routed + 1 shared per layer, 8 routed active
Extra layer1 multi-token-prediction (MTP) layer
Context262,144 tokens; card says YaRN x4 reaches about 1M
InputsText, image and video (vision encoder, 27 blocks)
LicenceMIT (base Qwen3.5 is Apache 2.0)
Downloads48,477 (API, 25 October)

It is a training change, not an architecture change. We compared its text config with Qwen’s Qwen3.5-122B-A10B config.json. Every size field matches. Only three default values are written out explicitly. So Ornith needs exactly the same memory as Qwen3.5-122B.

What the card says it changes. Ornith-1.0 added continued pretraining, mid-training and post-training on top of Qwen3.5 and Gemma 4. The card says Ornith-1.5 adds a self-improvement loop. The model generates its own training tasks, builds scaffolds for them and learns from the rollouts by reinforcement learning.

What the card claims. These are Ornith’s own numbers, not ours:

Benchmark (card)Ornith-1.5-122BQwen3.8-Flash-Next (125B)
SWE-bench Verified (non-hack)84.582.8
SWE-bench Pro (no-hack)76.943.9
Terminal-Bench 2.181.283.4
HLE with tools48.038.0

Treat them with care. Unsloth’s first README showed the 9B model’s scores by mistake; Unsloth fixed it on 20 October. A thread on the Unsloth repo is titled “Another benchmaxxed fine-tune by Ornith?”. We did not run any benchmark.

Every GGUF Size, in Bytes

We summed every file in the Hugging Face file trees on 25 October 2026. GiB = bytes / 1,073,741,824. Split files are added together. The vision projector (mmproj) is a separate 0.85 GiB file; add it only if you send images.

FileRepoBytesGiB
UD-IQ1_Munsloth40,356,745,47237.59
UD-Q2_K_XLunsloth44,533,952,76841.48
IQ2_Mbartowski46,383,407,42443.20
AD-IQ3_XXS-IQ2_SAtomicChat47,643,074,62444.37
UD-IQ3_XXSunsloth49,515,667,71246.12
UD-Q3_K_XLunsloth59,761,254,54555.66
AD-IQ4_XS-IQ3_SAtomicChat62,151,958,78457.88
i1-IQ4_XSmradermacher66,800,964,28862.21
IQ4_XSbartowski70,214,361,60065.39
Q4_K_Mornith-ai (official)75,843,911,64870.64
Q4_K_Mbartowski78,921,736,73673.50
UD-Q4_K_XLunsloth79,713,771,66574.24
Q5_K_Mornith-ai (official)88,710,135,77682.62
UD-Q5_K_XLunsloth94,611,939,50588.11
Q6_Kornith-ai (official)102,380,498,91295.35
UD-Q6_K_XLunsloth114,222,407,953106.38
Q8_0ornith-ai (official)132,555,710,048123.45
BF16ornith-ai (official)249,362,980,832232.24

Bartowski also publishes IQ1_S (27.95 GiB) to Q8_0 (123.49 GiB). His card marks both 1-bit files “not recommended”.

Two files called Q4_K_M differ by 2.86 GiB. Bartowski’s card says some of his files use a layout computed for this model. It gives more bits to the most sensitive weights. Compare files by bytes, not by name.

How much quality each step costs. AtomicChat measured its files against the BF16 original on 4 x H100. “Top-1” is how often the quant picks the same next token:

AtomicChat fileGiBTop-1 agreement
AD-IQ3_XXS-IQ2_S44.3786.05%
AD-IQ4_XS-IQ3_S57.8890.96%
AD-Q5_K-Q4_K73.0794.19%
AD-Q6_K-Q5_K86.8595.53%
Q8_0123.4596.28%

The jump from 3 bits to 4-5 bits is the big one. Above 5 bits the gain is small.

The Unsloth 4-bit file is bigger than Qwen’s. Our 96GB VRAM page measured Qwen3.5-122B UD-Q4_K_XL at 71.74 GiB. Ornith’s UD-Q4_K_XL is 74.24 GiB, 2.50 GiB more. Bartowski’s and AtomicChat’s cards say their files carry the MTP layer. We think that explains most of the gap, but we did not open the tensors.

Why the Context Costs So Little

Only the 12 full-attention layers keep a cache that grows with context. The 36 Gated DeltaNet layers keep a fixed state.

1. The KV cache. Config.json gives num_key_value_heads 2 and head_dim 256. layer_types lists 12 full_attention layers (full_attention_interval 4).

f16: 2 (K and V) x 2 heads x 256 x 2 bytes x 12 layers = 24,576 bytes per token

32,768 tokens   = 0.75 GiB
131,072 tokens  = 3.0 GiB
262,144 tokens  = 6.0 GiB

AtomicChat’s card says “24 KB of attention cache per token”. That matches.

2. The fixed DeltaNet state. Config.json gives 64 value heads of 128 x 128, stored in float32 (mamba_ssm_dtype).

Recurrent state: 36 layers x 64 heads x 128 x 128 x 4 bytes = 144 MiB
Conv state:      36 layers x 3 x 12,288 channels x 4 bytes  =   5 MiB
Total per conversation                                     ~ 149 MiB

3. The full budget. Weights + cache + 0.15 GiB state + a compute buffer. We budget 2 GiB for the buffer. We did not measure it.

4. Past 262K. The card’s YaRN x4 setting targets about 1M tokens. That cache would be about 24 GiB. The card warns that static YaRN “can slightly hurt quality on ordinary-length inputs”.

Fit Table by Hardware

Mac rows use the default Metal limit. PC rows use VRAM plus system RAM, minus about 8 GiB for the OS. Every row is our arithmetic unless it says community-reported.

HardwareUsable memoryBest fileContextNotes
24GB GPU + 64GB DDR5~80 GiBIQ4_XS (65.39)262KExperts in RAM
32GB GPU + 64GB DDR5~88 GiBQ4_K_M (70.64)262KExperts in RAM
24GB GPU + 128GB DDR5~144 GiBQ6_K (95.35)262KExperts in RAM
2 x 32GB GPUs (64GB VRAM)~62 GiBUD-Q3_K_XL (55.66)64KAll on GPU, 3-bit
64GB Macdefault limit not checkedAD-IQ3_XXS-IQ2_S (44.37)shortAtomicChat’s pick; 86% top-1
RTX PRO 6000 96GB~95 GiBQ5_K_M (82.62)262KAll on GPU
128GB Mac (M5 Max)107.52 GiB default limitQ6_K (95.35)262KMLX-4bit (64.83) also fits
DGX Spark121.63 GiB totalQ6_K (95.35)262KUD-Q6_K_XL (106.38) is tight
Strix Halo 128GB (Linux)~124GB GPU-visible (community)UD-Q6_K_XL (106.38)262KMeasured, see below
256GB Mac Studio M5 Ultra256GB unifiedQ8_0 (123.45)262K8-bit needs ~131.5 GiB

The 128GB Mac fits Q6_K without a tweak. A 128GB M5 Max reports a 115,448.73 MB working set in llama.cpp issue #21680. llama.cpp prints that in units of 10^6 bytes, so it is 107.52 GiB. Q6_K + 6.0 GiB cache + 2 GiB buffer is about 103.5 GiB. UD-Q6_K_XL needs a raised limit. AtomicChat’s card gives sudo sysctl iogpu.wired_limit_mb=118000 for a 128GB Mac.

The DGX Spark figure is a historical reading. The 121.63 GiB MemTotal comes from the Baekpica VQ-GGUF card. The OS lives inside it. UD-Q6_K_XL with the full cache needs about 114.4 GiB, which leaves about 7 GiB for Linux. Q6_K leaves about 18 GiB.

Windows Strix Halo boxes hold less. The same owner says Windows caps GPU memory at 112GB, against 124GB on Ubuntu. Q6 did not load on Windows (community-reported). On Windows, take Q5_K_M (82.62 GiB).

The 64GB tier is a poor fit. A 4-bit file (62-71 GiB) plus cache does not fit 64GB of VRAM. You drop to 3 bits, where AtomicChat measured 86-91% top-1 agreement. A 24GB card plus 64GB of cheap RAM gets a 4-bit file for less money.

The card’s “single 80GB GPU” line needs a quant. The model card says the model “serves on a single 80GB GPU”. Its BF16 weights are 232.99 GiB and its official FP8 checkpoint is 123.82 GiB. Only the official NVFP4 checkpoint, at 72.30 GiB, is smaller than 80GB. That leaves little room for the 6.0 GiB full-context cache.

How Fast Will It Be?

We found one measured number on local hardware. An owner posted it on the Unsloth GGUF discussion page on 21 October (community-reported):

SetupPrompt processingGeneration
Strix Halo, UD-Q6_K_XL, 262,144 context, MTP on, ~128K filled~400 tok/s~25 tok/s

The same poster measured Qwen3.8-Flash-Next at Q6_K_XL on the same box at about 350 tok/s and 21 tok/s, without MTP. Another owner in that thread found Ornith slower than Flash-Next at a larger quant. We cannot settle that from two posts.

The rows below are bandwidth ceilings, not predictions. Real speed lands below them.

Weights read per token (our count from config.json):
  8 routed experts x 3 x 3,072 x 1,024 x 48 layers = 3.62B   (lives in RAM if offloaded)
  attention, shared expert, router, output head     = 5.84B   (always on the GPU)
  total                                             = 9.46B

Q4_K_M at ~5.02 bits/weight (bartowski's card):
  9.46B x 5.02 / 8 = 5.94 GB per token
  DGX Spark,        273 GB/s  -> ~46 tok/s ceiling
  M5 Max (40-core), 614 GB/s  -> ~103 tok/s ceiling
  RTX PRO 6000,   1,792 GB/s  -> ~302 tok/s ceiling

PC, experts in RAM, dual-channel DDR5-4800 = 76.8 GB/s:
  3.62B x 5.02 / 8 = 2.27 GB per token -> ~34 tok/s ceiling

The PC route works because the experts are small. Only 3.62B routed weights are read per token. The 5.84B always-read weights need about 3.7 GB at Q4 on the GPU. A 24GB card holds them, the 6.0 GiB cache and many experts besides.

MTP should add speed. Bartowski’s card says the MTP layers are in every quant. Turn them on with --spec-type draft-mtp. AtomicChat measured +18% to 35% on the 35B sibling. They have not measured the 122B yet.

Software Support, Checked 25 October 2026

RuntimeStatusSource
llama.cppSupported. Architecture qwen35moe, from PR #19468, merged 10 February 2026GitHub
llama.cpp build to useb11716+ (bartowski), b11736 tested (AtomicChat)GGUF cards
MTP speculative decoding--spec-type draft-mtp; MTP weights are inside the GGUFsbartowski card, HF thread
Ollamaornith-1.5:122b (73-77GB), 122b-q4_K_M (77GB), 122b-nvfp4 (MLX, 73GB)ollama.com
LM Studio, koboldcpp, JanListed by bartowski’s card. We did not test thembartowski card
MLXOfficial Ornith-1.5-122B-MLX-4bit (64.83 GiB) and -MLX-8bit (121.68 GiB); mlx-lm has qwen3_5_moe.pyHF API, mlx-lm source
vLLM, SGLangvLLM 0.19.1+, SGLang 0.5.9+; official FP8 (123.82 GiB) and NVFP4 (72.30 GiB) checkpointsmodel card, HF API

The 77GB Ollama tag matches the official Q4_K_M file (75.84 GB) plus the 0.91 GB vision projector.

Run the official Q6_K on a 128GB machine with:

llama-server -hf ornith-ai/Ornith-1.5-122B-GGUF:Q6_K -c 262144 --spec-type draft-mtp

On a PC with a 24-32GB card, add -ngl 99 --n-cpu-moe 48 and lower --n-cpu-moe until the GPU is full. Our —n-cpu-moe guide shows how. Ornith is a thinking model. To turn thinking off per request, AtomicChat’s card passes "chat_template_kwargs": {"enable_thinking": false}.

What to Buy

Prices as of October 2026, from our price reference. The read date is in brackets.

OptionPriceWhat it runs
Strix Halo 128GB mini PC (GMKtec EVO-X2)$3,499.99 (gmktec.com, 23 September)UD-Q6_K_XL, full context, Linux
NVIDIA DGX Spark 128GB$4,699 list, sold out; $5,399.99 at Newegg (13 September)Q6_K, full context
Mac Studio M5 Max 128GB (40-core GPU)$5,099 (apple.com, 21 September)Q6_K or MLX-4bit, full context
RTX PRO 6000 96GB$14,999.99 in stock (Central Computer, 30 September)Q5_K_M, full context
RTX 3090 24GB (used) + DDR5 64GB (2 x 32GB)$1,541 average (23 October) + $919.99 (Newegg, 1 October)IQ4_XS, experts in RAM

A 128GB unified box is the simplest buy. A Ryzen AI Max+ 395 128GB mini PC holds the 6-bit file and the full 262K context. One owner measured about 25 tok/s. Run Linux, not Windows, for the extra 12GB.

Pick the DGX Spark for CUDA. The NVIDIA DGX Spark 128GB holds Q6_K with about 18 GiB to spare. Its 273 GB/s sets a Q4_K_M ceiling near 46 tok/s, so buy it for CUDA, not for speed.

For speed, one 96GB card wins. The RTX PRO 6000 96GB holds Q5_K_M and the full cache in VRAM. Its 1,792 GB/s gives a ceiling about 6.5 times a Spark’s. It costs about four Strix Halo boxes.

Already own a 24GB or 32GB card? Buy RAM, not a new machine. 64GB of DDR5 runs a 4-bit file with the experts in RAM. 128GB of DDR5 runs Q6_K.

The 128GB Mac Studio M4 Max is discontinued. Apple now sells the M5 Max. Check the memory size on any clearance listing.

Honest Caveats

  1. We did not run the model. Every fit row is arithmetic from file sizes and config.json.
  2. The only speed number is one owner’s post. No llama.cpp benchmark for Ornith 1.5 122B on a GPU was published where we looked.
  3. The 2 GiB compute buffer is a budget. Check the buffer sizes in your llama-server log.
  4. The benchmark table is Ornith’s. Community threads question it. Test the model on your own tasks.
  5. The DGX Spark and Strix Halo memory limits are second-hand. The Spark figure is a historical MemTotal reading. The 124GB Strix Halo ceiling is one owner’s report.
  6. Our 9.46B active count includes the output head. The base model’s name says 10B. The speed ceilings move by about 5% either way.

Sources

See Also

Read next

Can I Run MiMo V2.6 Flash Locally (2026): Needs 256GB
MiMo-V2.6-Flash is a 309B MoE with 15B active. The checkpoint already ships at about 4 bits, so the floor is 165.53 GiB, and the MLX build peaks at 164-167 GB on a 256GB M3 Ultra. No GGUF exists yet. Everyone below 256GB should run the 9B distill.
Can I Run Qwen3.8 Flash Next Locally (2026): Yes, on 128GB
Qwen3.8-Flash-Next has only 6B active parameters, but a 51B n-gram embedding table makes even the 1-bit GGUF 67.56 GiB. It needs a 128GB box. llama.cpp support is merged, and Strix Halo and DGX Spark owners report 17-47 tok/s.
Can I Run GLM-5.3-Flash Locally (2026): 128GB Minimum
GLM-5.3-Flash is 321B parameters with 18B active, MIT licensed, and released natively in FP8. The smallest usable GGUF is 86.69 GiB, so 128GB is the floor. And stock llama.cpp still cannot load it — the support PR is open, not merged.
DeepSeek V4.1 Flash GSQ-RCO Quants (2026): 12GB GPU, No Mac
ISTA-DASLab shrank DeepSeek V4.1 Flash 1017 to 1.75-2.25 bits per weight. The backbone is 112-145 GiB, the Engram table is 103 GiB on SSD, and the files run today only in Strata on a 12GB+ NVIDIA card. Stock llama.cpp cannot load them, and no Mac runs them yet.