Check KV Cache Size in llama.cpp and Ollama: Read the Log

Bottom Line

• The number is in the startup log. Look for llama_kv_cache: size = … MiB (N cells, L layers, …). The cell count is the context the cache holds.
• Ollama shows the same line. Ollama 0.34.0 runs llama.cpp’s llama-server underneath and logs at verbosity 4.
• llama-server may shrink your context without a visible message. --fit is on by default. Its “context size reduced” line prints only at -lv 4.
• llama-server defaults to 4 slots. n_ctx is a pool shared by 4 conversations. Set -np 1 for one user.
• Sliding-window models print two caches. gpt-oss shows a full cache and a small SWA cache. --swa-full doubles the total.
• Ollama multiplies context by parallel slots. OLLAMA_NUM_PARALLEL=4 at 32K sends -c 128000.

The Lines to Read in llama.cpp

We read these format strings from llama.cpp master on 2026-09-12. Your build prints the same fields if it is recent.

The full size = line has this shape:

llama_kv_cache: size = <MiB> MiB (<cells> cells, <layers> layers, <seqs>/<streams> seqs), K (<type>): <MiB> MiB, V (<type>): <MiB> MiB

Check three fields against your plan:

1. cells — is it the context you asked for?
2. K and V type — is it f16, or the q8_0 you set with -ctk and -ctv?
3. device — does the KV buffer size line say CUDA0, or CPU?

A CPU KV buffer on a GPU machine means the cache lives in system RAM. That is a speed problem even when the model fits.

The Exit Table: llama-server’s Memory Breakdown

When llama-server stops, it prints a table headed memory breakdown [MiB]. The columns are total, free, self, model, context and compute, plus unaccounted. The row for each device reads self = model + context + compute + unaccounted.

The context column holds the KV cache for that device. It is the fastest way to see the split across two GPUs.

Trap 1: --fit Shrinks the Context Quietly

--fit defaults to on. It adjusts arguments you did not set so the model fits free device memory. -c defaults to 0, which means “use the model’s trained context.”

So if you omit -c on a 262K-context model, --fit can cut the context until the model fits. The floor is --fit-ctx, default 4096. The line that reports the cut is:

context size reduced from <N> to <M> -> need <X> MiB less memory in total

That line uses LOG_TRC, which is verbosity level 4. The default verbosity is 3. At default settings the cut happens and the line does not print.

Three ways to see or stop it:

If you set -c and the model does not fit, --fit changes layer placement instead. Watch for layers moving to the CPU.

Trap 2: llama-server Reserves Context for 4 Users

llama-server’s --parallel default is -1, meaning auto. The server source sets auto to 4 slots with a unified KV cache.

With a unified cache, n_ctx is one pool that the 4 slots share. n_ctx_seq is what one conversation can use. If you run one chat client, start with -np 1.

This matters most for the sliding-window cache, because its size scales with the slot count. See the next section.

Trap 3: Sliding-Window Models Have Two Caches

gpt-oss alternates full-attention and 128-token sliding-window layers: 18 of each across 36 layers. llama.cpp builds two caches for it.

The source sizes the SWA cache as the window times the sequences, plus the micro-batch, rounded up to a multiple of 256. It is capped at the full context:

size_swa = pad_256( min(n_ctx, n_swa × n_seq + n_ubatch) )

With the default n_ubatch of 512, that gives:

What the arithmetic predicts for gpt-oss 120B at 131,072 cells, f16 cache. This is our calculation from the model’s config.json (8 KV heads, head dim 64), not a captured log. Each layer costs 2 KiB per cell.

So a normal start should print about 4.6 GiB of KV cache. If your log shows about 9.0 GiB, a full-size SWA cache is on. llama.cpp prints a warning when that happens: using full-size SWA cache. The memory side of this model is covered in how much VRAM gpt-oss 120B needs at full context.

Ollama: the Same Lines, One More Multiplier

We read Ollama’s source at release v0.34.0 (published 2026-09-05). Its scheduler starts llm.NewLlamaServer, which runs llama.cpp’s upstream llama-server binary. The launch flags include:

-c <context × parallel>  -np <parallel>  --log-verbosity 4

Two consequences follow from that.

1. The llama_kv_cache lines appear in Ollama’s server log. Verbosity 4 also prints the --fit trace lines.
2. The cache holds context × OLLAMA_NUM_PARALLEL. The default parallel count is 1. At 32K and a parallel count of 4, the cache holds 128,000 cells.

Ollama always passes -c, so --fit does not cut the context. It changes layer placement instead.

Step 1 — check the context and the CPU split:

ollama ps

The columns are NAME, ID, SIZE, PROCESSOR, CONTEXT and UNTIL. CONTEXT is per request. Anything other than 100% GPU under PROCESSOR means part of the model runs on the CPU.

Step 2 — read the log:

Step 3 — know the default context. Ollama’s docs set it by VRAM: 4K below 24 GiB, 32K from 24 to 48 GiB, 256K at 48 GiB and above. A 16GB card starts at 4K. Agent and coding tools need far more; Ollama’s docs recommend at least 64,000 tokens. Set it with OLLAMA_CONTEXT_LENGTH=64000 ollama serve.

Ollama forces one slot for some model families: mllama, qwen3vl, qwen3vlmoe, qwen35, qwen35moe, qwen3next, lfm2, lfm2moe, nemotron_h, nemotron_h_moe and nemotron_h_omni. For those, OLLAMA_NUM_PARALLEL does not multiply the cache.

When the Number Is Too Big

If the cache is right and still does not fit, the model needs more memory than the card has. The 16GB VRAM picks and the lowest-KV-cache models are the next pages to read.

FAQ

See Also

• How much VRAM gpt-oss 120B needs at full context — the arithmetic this page checks
• How much VRAM for 128K context — the KV cost per model at long context
• KV cache quantization: Q8 vs Q4 — the flag that halves the number you just read
• Context window traps in local agents — why 4K defaults break tool use
• Lowest-KV-cache models for long context — models that keep the cache small
• YaRN and RoPE scaling — the other way a context setting fails without an error