Local Model Team Planner

A local agent team is several models with jobs: a small router, a planner, a coder, a reviewer. Each machine has to hold every model it keeps loaded, file plus KV cache. Pick the models and the machines; the planner adds them up per machine, picks the largest file that still fits next to the others, and gives each machine its endpoint.

Machines

#NameTypeRAMVRAMServer
1
2
3

Roles

RoleModelContextRuns on
Router
Planner
Coder
Reviewer

KV cache at f16, the llama.cpp default. One model in two roles on the same machine loads once; its cache holds both contexts.

How the planner adds it up

per model   = GGUF file + KV cache at its context     (same math as /calculator/)
per machine = sum of the models it keeps loaded
usable      = Mac: RAM - 4 GiB | GPU: VRAM - 1 GiB (+70% of RAM, slow) | no GPU: 70% of RAM, slow
file choice = start every model at its largest file; while less than 2 GiB is left, step
              the biggest one down, never below its Q4-class file

The viewer setups, run through it

Keep several models loaded: the setting per server

ServerDefault addressListen on the networkSeveral models at once
LM Studiolocalhost:1234/v1"Serve on Local Network", or lms server start --bind 0.0.0.0JIT Auto-Evict keeps one model by default. Load each one with lms load (no TTL) or turn Auto-Evict off.
Ollama127.0.0.1:11434OLLAMA_HOST=0.0.0.0:11434OLLAMA_MAX_LOADED_MODELS, default 3 x GPUs (3 on CPU), "provided they fit". A negative keep_alive keeps a model loaded indefinitely.
llama-server127.0.0.1:9931--host 0.0.0.0One server per model on its own --port, or router mode with --models-max (default 4, least recently used unloads first).

llama-server's default port is 9931 on current master (common.h and the server README, read 2026-11-21). Older builds and the Docker examples use 8080; check llama-server --help on your build. LM Studio's network-serving page recommends turning on authentication.

Questions

How much memory do I need to run several local models at once?

Add up every model that has to stay loaded: its file plus its KV cache at the context you give it. On one 64GB Mac (60 GiB usable after a 4 GiB reserve), a router, planner, coder and reviewer need 55.7 GiB when the planner and reviewer share one Qwen3.8 27B, which loads once. If the sum is larger than the memory, the server unloads one model to load the next, and every switch between roles reloads a file from disk.

Can Ollama, LM Studio and llama.cpp keep several models loaded at the same time?

Yes, with limits. Ollama loads up to OLLAMA_MAX_LOADED_MODELS models "provided they fit in available memory"; the default is 3 times the number of GPUs, or 3 for CPU inference. LM Studio keeps only one model loaded through JIT by default (Auto-Evict); models loaded with lms load stay until you unload them. llama-server in router mode loads up to --models-max models (default 4) and unloads the least recently used one when a new model does not fit.

Should I split one model across two computers?

Usually not. llama.cpp's RPC backend can spread one model's weights and KV cache across machines in proportion to their memory, but its own README calls it "fragile and insecure" and says never to run it on an open network. Giving each machine its own whole model and its own endpoint is simpler: only the prompt and the reply cross the network, as text. Split one model only when it fits on no single machine you own.

How do I use a model loaded on my other Mac?

In LM Studio, turn on LM Link on both machines (it needs an LM Studio account login). It builds a private Tailscale network, and a model on the other Mac then appears in the local server at localhost:1234 as if it were local. Without LM Link, run the server on the other machine on the local network (LM Studio "Serve on Local Network", OLLAMA_HOST=0.0.0.0, or llama-server --host 0.0.0.0) and point your agent at that machine's address.

How do I point OpenClaw or Hermes at a model on another machine?

OpenClaw: add a provider under models.providers with baseUrl set to the other machine's LAN or tailnet address and api set to "openai-completions" (or "openai-responses" for LM Studio), then give each agent its model with agents.entries.<id>.model. Hermes: set provider: custom and base_url under model: for the main model; the delegation: block takes its own base_url for subagents, and auxiliary.<task> blocks set a cheaper model for side jobs such as compression.

Sources

Related