Agent Context Budget Checker

A local coding agent decides when to compact from its own idea of the context window. The server decides when to stop from the context it actually loaded. When the two numbers differ, the agent fills the server, replies get cut off ("Response truncated due to token limits"), and compaction never starts. Enter both numbers to see which one breaks first.

How the check works

compaction point = the agent's rule applied to the window it assumes
  Qwen Code: min(0.85 x window, window - 20,000 - 13,000)   (0.85 x window when that is <= 0)
  Pi:        window - 16,384
  other:     0.85 x window
compacts in time  = compaction point < server context
reply room        = server context - compaction point
conversation room = compaction point (or server context) - fixed prompt

Worked example: Qwen Code on Ollama at 32K

Qwen Code settingServerCompacts atReply room thenResult
No contextWindowSize32K850,000noneServer fills first
contextWindowSize 32,76832K27,8524,916Replies get cut
contextWindowSize 65,53664K32,53633,000Works

Measure your fixed prompt

Questions

What does "Response truncated due to token limits" mean?

Qwen Code prints it when the server ends a reply with finish_reason "length" (packages/core/src/core/openaiContentGenerator/converter.ts maps "length" to MAX_TOKENS). The reply hit a cap: the max_tokens limit, or the server's context filled up while the model was still writing. llama-server stops a reply when prompt plus reply reaches its context size and reports "length". A thinking model spends its reasoning tokens from the same room, so a long think on a nearly full context ends this way.

Why did raising num_ctx and num_predict not fix it?

Because Qwen Code does not read the server's context. For an Ollama tag such as qwen3.6:27b it looks the name up in its own table and plans for 1,000,000 tokens, so it compacts at 850,000 tokens. A 32K server fills long before that. Set contextWindowSize in the model's generationConfig to the server's real context, so compaction starts in time.

What context should I set for a local coding agent?

At least 64K. Ollama's own docs say coding tools "should be set to at least 64000 tokens". With Qwen Code and contextWindowSize set to 65,536, compaction starts at 32,536 tokens and leaves 33,000 tokens for a reply, enough for the 32,768 the Qwen3.6 card recommends. At 32K the same setup leaves 4,916 tokens.

What is Ollama's default context length?

It depends on VRAM: under 24 GiB it is 4K, 24 to 48 GiB gives 32K, and 48 GiB or more gives 256K (docs/context-length.mdx). The FAQ page still says 4096, but the server code (envconfig/config.go) follows the VRAM rule. Check the CONTEXT column of ollama ps to see what you got.

How do I measure my agent's system prompt and tool definitions?

Restart the server, start the agent, and send one short message. On llama-server, read the "prompt eval time = ... / N tokens" log line; N is the fixed prompt plus your message. On Ollama's native API the same number is prompt_eval_count. Take the first request only: later requests reuse the cached prefix and show fewer tokens.

Related