1.58-Bit LLMs Explained: How 3 Values Use Fewer Bits

Bottom Line

• “1.58-bit” means three values per weight: -1, 0 or +1. Three equal choices carry log2(3) = 1.585 bits.
• Real files are bigger than 1.58. Bytes hold whole bits, so the usual packing costs 1.6 to 2.06 bits per weight, plus the scale values.
• The 1.585 limit assumes the three values are equally likely. They are not. Zero is often the most common one, and that lowers the true limit.
• Intel’s BITCOS format uses that fact. It stored one model at 1.485 bits per weight, exactly at the new limit. No math was broken.
• The saving is small on your machine: about 0.34 GB on a 27B model, and no local runtime supports it yet.

Where 1.58 Comes From

A normal model stores each weight as a 16-bit number. A 4-bit quant rounds it to one of 16 levels. A ternary model goes further. Every weight is one of three values:

Microsoft’s BitNet b1.58 paper (February 2024) trained models this way from the start. The name comes from information theory. If each of the three values is equally likely, one weight carries log2(3) = 1.585 bits. That is the least space you can use on average, if you know nothing else about the weights.

Ternary math also needs no multiplication. Adding, subtracting or skipping is cheaper than multiplying, which is why ternary models are interesting for CPUs and small GPUs.

Why Your File Is Bigger Than 1.58

Computers store whole bits. You cannot write 1.585 bits to memory. The formats use tricks to get close:

The scales are the other cost. Each group of weights shares a 16-bit scale number. Ternary Bonsai 2 27B uses one scale per 128 weights, which adds 16/128 = 0.125 bits. Its model card gives about 1.71 bits per weight from that sum, and 1.72 for the whole model.

2-bit packing wastes space but runs fast. Unpacking five trits from one byte takes more work than reading a 2-bit code. On some CPUs that extra work costs more than the memory it saves.

For what the GGUF names mean, see our GGUF quant names explained page.

How BITCOS Gets Below 1.58

The key fact: in real ternary models, the three values are not equally likely. Zero is often the most common one. The BITCOS paper measured 29 ternary models from 7 families. The share of zeros ran from 29.7% to 51.5%.

When one value is more common, you can store it more cheaply. That is how every compression format works.

BITCOS (BITmap and COmpacted Signs) splits each weight into two questions:

1. Is it zero? One bit for every weight. This is the bitmap.
2. Is it + or -? One sign bit, stored only for the nonzero weights.

If a share z of the weights is zero, the cost is 1 + (1 - z) = 2 - z bits per weight. With 51.5% zeros, that is about 1.485 bits.

Why this does not break information theory

The true limit is the entropy of the weights. For a ternary weight with a zero share z, and + and - equally likely:

H(z) = -z log2(z) - (1 - z) log2((1 - z) / 2)

We computed it for the cases in the paper:

1.585 was never a hard limit. It was the limit for one weight mix. When half the weights are zero, the limit is 1.5 bits, and BITCOS reaches it exactly. At other zero shares, BITCOS sits a little above the true limit.

When BITCOS loses

BITCOS beats five-trit packing (1.625) only when more than 37.5% of the weights are zero. It won on 26 of the 29 models. One of the 3 losers is Bonsai 27B. With 29.7% zeros, BITCOS made that file bigger: 1.828 bits per weight with scales, against 1.750 for five-trit packing.

The paper came out two days before Ternary Bonsai 2 was published, so its table is almost certainly the earlier Bonsai 27B. Bonsai 2’s zero share is not published.

What It Saves on Your Machine

For a 27.36B ternary model, the weight size is 27.36 billion x bits / 8. We computed:

The jump that matters is from 4-bit to ternary, not from 1.58 to 1.485. Ternary cuts a 27B model from about 15 GB to about 6 GB. BITCOS on top saves about 0.34 GB in its best case. Your KV cache at long context is usually bigger than that.

Speed follows size, up to a point. Decoding one token reads every weight once, so a smaller file can be faster on the same memory bandwidth. But unpacking costs work too. In the paper, BITCOS gave no gain on Intel’s Lunar Lake CPU, because that chip ran out of instructions before it ran out of bandwidth.

To see how a ternary 27B fits a 12GB card, read Bonsai 2 27B on an RTX 3060 12GB.

Can You Use BITCOS Today?

No, not in a local app. As of 1 November 2026:

• We found no BITCOS issue or pull request in llama.cpp, bitnet.cpp, the Prism ML llama.cpp fork or vLLM.
• Ollama and LM Studio build on llama.cpp, so they have no support either.
• The only public code is Intel’s research GPU kernels (TernOCL and TernSYCL) for Intel Xe2 GPUs.

BITCOS is lossless and needs no retraining. It changes only how the weights are stored. If a runtime adds it, existing ternary models could use it without new training.

What the Speed Numbers Mean

The paper reports decode speedups of up to 1.18x on CPUs and 1.27x on GPUs. Read them carefully:

• They compare BITCOS with Intel’s own 2-bit kernels, not with a 1.58-bit format.
• All tests ran on Intel hardware in vLLM: two server and desktop CPUs, and two Arc GPUs. There are no NVIDIA, AMD or Apple results.
• The best gains came on the sparsest models. A model with fewer zeros gains less, or loses.

Honest Caveats

1. The 1.485 figure counts the ternary values only. With the 16-bit scales that every real file stores, that model is 1.610 bits per weight.
2. We did not run BITCOS. The table numbers come from the paper. The memory and entropy numbers are our own calculations from the formulas shown.
3. The size of the zero share decides everything. It depends on how the model was trained, and most model cards do not publish it.
4. The Bonsai 2 card has changed since our RTX 3060 page. It now says the Q2_0 file runs on stock llama.cpp from build b11556. The smaller PQ2_0 and PTQ1_0 files still need the Prism ML fork.

Sources

• Georganas, Heinecke and Dubey (Intel): Breaking the 1.58-bit Barrier for Ternary LLMs: BITCOS, 29 models, 1.485 bits, speedups, Table I
• Ma et al.: The Era of 1-bit LLMs (BitNet b1.58): ternary weights
• llama.cpp ggml-common.h: TQ1_0 1.6875 and TQ2_0 2.0625 bits per weight
• microsoft/BitNet: bitnet.cpp
• Prism ML: Ternary Bonsai 2 27B GGUF: bits per weight and file sizes
• libxsmm/TernOCL: Intel’s BITCOS GPU kernels

See Also

• Quantization in plain English: what Q4, Q5 and Q8 cost you
• GGUF quant names explained: TQ1_0, TQ2_0, PQ2_0 and the rest
• Bonsai 2 27B on an RTX 3060 12GB: a ternary 27B on a budget card
• Best local LLM by RAM: what fits in each memory tier