A local agent team is several models with jobs: a small router, a planner, a coder, a reviewer. Each machine has to hold every model it keeps loaded, file plus KV cache. Pick the models and the machines; the planner adds them up per machine, picks the largest file that still fits next to the others, and gives each machine its endpoint.
| # | Name | Type | RAM | VRAM | Server |
|---|---|---|---|---|---|
| 1 | |||||
| 2 | |||||
| 3 |
| Role | Model | Context | Runs on |
|---|---|---|---|
| Router | |||
| Planner | |||
| Coder | |||
| Reviewer |
KV cache at f16, the llama.cpp default. One model in two roles on the same machine loads once; its cache holds both contexts.
per model = GGUF file + KV cache at its context (same math as /calculator/)
per machine = sum of the models it keeps loaded
usable = Mac: RAM - 4 GiB | GPU: VRAM - 1 GiB (+70% of RAM, slow) | no GPU: 70% of RAM, slow
file choice = start every model at its largest file; while less than 2 GiB is left, step
the biggest one down, never below its Q4-class file | Server | Default address | Listen on the network | Several models at once |
|---|---|---|---|
| LM Studio | localhost:1234/v1 | "Serve on Local Network", or lms server start --bind 0.0.0.0 | JIT Auto-Evict keeps one model by default. Load each one with lms load (no TTL) or turn Auto-Evict off. |
| Ollama | 127.0.0.1:11434 | OLLAMA_HOST=0.0.0.0:11434 | OLLAMA_MAX_LOADED_MODELS, default 3 x GPUs (3 on CPU), "provided they fit". A negative keep_alive keeps a model loaded indefinitely. |
| llama-server | 127.0.0.1:9931 | --host 0.0.0.0 | One server per model on its own --port, or router mode with --models-max (default 4, least recently used unloads first). |
llama-server's default port is 9931 on current master (common.h and the server README, read 2026-11-21). Older builds and the Docker examples use 8080; check llama-server --help on your build. LM Studio's network-serving page recommends turning on authentication.
Add up every model that has to stay loaded: its file plus its KV cache at the context you give it. On one 64GB Mac (60 GiB usable after a 4 GiB reserve), a router, planner, coder and reviewer need 55.7 GiB when the planner and reviewer share one Qwen3.8 27B, which loads once. If the sum is larger than the memory, the server unloads one model to load the next, and every switch between roles reloads a file from disk.
Yes, with limits. Ollama loads up to OLLAMA_MAX_LOADED_MODELS models "provided they fit in available memory"; the default is 3 times the number of GPUs, or 3 for CPU inference. LM Studio keeps only one model loaded through JIT by default (Auto-Evict); models loaded with lms load stay until you unload them. llama-server in router mode loads up to --models-max models (default 4) and unloads the least recently used one when a new model does not fit.
Usually not. llama.cpp's RPC backend can spread one model's weights and KV cache across machines in proportion to their memory, but its own README calls it "fragile and insecure" and says never to run it on an open network. Giving each machine its own whole model and its own endpoint is simpler: only the prompt and the reply cross the network, as text. Split one model only when it fits on no single machine you own.
In LM Studio, turn on LM Link on both machines (it needs an LM Studio account login). It builds a private Tailscale network, and a model on the other Mac then appears in the local server at localhost:1234 as if it were local. Without LM Link, run the server on the other machine on the local network (LM Studio "Serve on Local Network", OLLAMA_HOST=0.0.0.0, or llama-server --host 0.0.0.0) and point your agent at that machine's address.
OpenClaw: add a provider under models.providers with baseUrl set to the other machine's LAN or tailnet address and api set to "openai-completions" (or "openai-responses" for LM Studio), then give each agent its model with agents.entries.<id>.model. Hermes: set provider: custom and base_url under model: for the main model; the delegation: block takes its own base_url for subagents, and auxiliary.<task> blocks set a cheaper model for side jobs such as compression.