Best Local LLM for 36GB RAM (2026): Qwen3.8 27B on MLX
On a 36GB Mac, run Qwen3.8 27B in MLX 4-bit. The 14.95 GiB model leaves room for about 181K tokens of context inside the 27.00 GiB that macOS gives the GPU. Run it in MLX, not in llama.cpp: on a 36GB M4 Max, one tester measured about 35 tok/s in MLX and 6.5-8.8 tok/s with the same model in llama.cpp. For speed, Qwen3.6 35B-A3B holds its full 262K window and runs at 40-52 tok/s.
Bottom Line
- Best local LLM for 36GB RAM: Qwen3.8 27B in MLX 4-bit. The
mlx-community/Qwen3.8-27B-4bitweights are 14.95 GiB. About 181K tokens of f16 context fit next to them. - The runtime matters more than the quant on this tier. On a 36GB M4 Max, one tester measured the dense 27B at 6.5-8.8 tok/s in llama.cpp and about 35 tok/s in MLX (llama.cpp #27525, community-reported).
- Speed pick: Qwen3.6 35B-A3B. The 20.61 GiB UD-Q4_K_M file holds the full 262K window. A 36GB M4 Max ran a Q4_K_M build of it at 40-52 tok/s in llama.cpp (same issue).
- The GPU gets 27.00 GiB, not 36. macOS keeps 25% by default. Every “fits” verdict below uses 27.00 GiB minus 1 GiB of buffers.
- 8-bit 27B does not fit. Q8_0 is 27.05 GiB and MLX 8-bit is 27.48 GiB. Both are over the limit before context.
- Every 36GB Mac is a Max chip or an M3 Pro. Today Apple sells 36GB only on the M5 Max with the 32-core GPU: $2,499 in the Mac Studio, $4,099 in the 14-inch MacBook Pro (apple.com, as of October 2026).
Still choosing a machine? See the tested hardware list, or compare every Mac tier in best Mac for local LLM.
Which Macs Have 36GB
36GB is an Apple-only size. It exists on four chips. Bandwidth decides the speed of every model on this page, and it differs by 3x across them.
| Chip | GPU cores | Memory bandwidth | Where it ships | Apple sells it now? |
|---|---|---|---|---|
| M5 Max | 32 | 460 GB/s | Mac Studio, MacBook Pro 14/16 | Yes |
| M4 Max | 32 | 410 GB/s | MacBook Pro 14/16 (2024) | No |
| M3 Max | 30 | 300 GB/s | MacBook Pro 14/16 (Nov 2023) | No |
| M3 Pro | 18 | 150 GB/s | MacBook Pro 14/16 (Nov 2023) | No |
Sources: apple.com/mac-studio/specs and apple.com/macbook-pro/specs for the M5 Max; Apple Support tech specs for the M4 Max and M3 Pro / M3 Max MacBook Pros.
The M3 Pro is the outlier. It has the same 36GB as an M5 Max at one third of the bandwidth. The model picks below are the same on it, but expect about one third of the speed.
How Much of 36GB the GPU Can Use
macOS reports a “recommended max working set” to Metal. llama.cpp prints it at startup.
- Logged value on a 36GB Mac:
recommendedMaxWorkingSetSize = 28991.03 MB, on an M3 Pro (llama.cpp #13865). - Convert it correctly. llama.cpp divides bytes by 106 and calls the result “MB”. So 28,991.03 “MB” is 27.00 GiB, which is 75.0% of 36 GiB.
- Our budget: 27.00 GiB minus 1 GiB for compute buffers (our assumption) leaves 26.00 GiB for weights plus KV cache.
The full table for 16GB to 128GB Macs is on best local LLM for MacBook. A 32GB Mac gets 24.96 GiB (78%). So 36GB adds only 2.04 GiB of GPU memory over 32GB, not 4.
What Fits in 36GB
File sizes are byte counts from the Hugging Face API, read 2026-10-28. GiB = 230 bytes. KV cache per token comes from each model’s config.json: Qwen3.8 27B has 16 full-attention layers × 4 KV heads × 256 head dim, so 64 KiB per token at f16. Qwen3.6 35B-A3B has 10 × 2 × 256, so 20 KiB per token. Context = (26.00 GiB − file) ÷ cache per token. This is our arithmetic.
| Model | File | Size (GiB) | Context that fits (f16 cache) | Speed on a 36GB Max chip |
|---|---|---|---|---|
| Qwen3.8 27B (dense) | MLX 4-bit (mlx-community) | 14.95 | About 181K | About 35 tok/s, M4 Max, MLX (omlx) — community-reported |
| Qwen3.8 27B | Q4_1 GGUF (Unsloth) | 16.34 | About 158K | Not measured on 36GB; 13.53 tok/s on an M1 Max (400 GB/s) |
| Qwen3.8 27B | UD-Q4_K_XL GGUF (Unsloth) | 16.35 | About 158K | 7.0-7.1 tok/s, M4 Max, llama.cpp (8.5-8.8 with MTP) |
| Qwen3.8 27B | UD-Q6_K GGUF (Unsloth) | 20.47 | About 90K | 6.5 tok/s for a Q6_K file, M4 Max, llama.cpp |
| Qwen3.8 27B | Q8_0 GGUF / MLX 8-bit | 27.05 / 27.48 | Does not fit | — |
| Qwen3.6 35B-A3B (MoE) | UD-Q4_K_M GGUF (Unsloth) | 20.61 | Full 262K | 40-52 tok/s, M4 Max, a Q4_K_M build, llama.cpp |
| Qwen3.6 35B-A3B | UD-Q5_K_XL GGUF (Unsloth) | 24.77 | About 64K | Not measured |
| Qwen3.6 35B-A3B | UD-Q6_K GGUF (Unsloth) | 27.30 | Does not fit | — |
| gpt-oss 20B (MoE) | MXFP4 (ggml-org) | 11.28 | Full 128K | 92.36 tok/s, M4 Max 36GB (#15396) |
| gpt-oss 120B (MoE) | MXFP4 (ggml-org) | 59.03 | Does not fit | — |
Speed sources: the M4 Max rows are from llama.cpp #27525 (short prompts, server-reported decode). The M1 Max Q4_1 row is from a reply in the same issue. All are community-reported, on one machine each. Qwen3.8 27B also has a linear-attention state that does not grow with context; we leave it out of the context math.
Why the Runtime Decides the Winner
The finding. In issue #27525, the tester ran Qwen3.8 27B on a 36GB M4 Max (410 GB/s) for two weeks. llama.cpp’s Metal backend gave 6.5-8.8 tok/s on every config: Q4, Q5 and Q6 files, flash attention on and off, 12 threads, mlock, two builds. The MLX server omlx gave about 35 tok/s on the same machine. The MoE model was not affected: 40-52 tok/s.
What it means in bandwidth. 7 tok/s × 17 GB read per token is about 120 GB/s. That is 29% of the chip’s 410 GB/s. MLX reached about 68%, by the tester’s figures.
The partial fix in llama.cpp. A second tester replied that the XL quant layout is the problem on Metal. On an M1 Max (400 GB/s), standard files ran at 14.29 tok/s (IQ4_NL) and 13.53 tok/s (Q4_1) — about 2x the XL files. That is still well below MLX.
So pick the model by runtime:
| You use | Run this on 36GB | Why |
|---|---|---|
| MLX (LM Studio MLX engine, mlx-lm, omlx) | Qwen3.8 27B 4-bit | The strongest model that fits, at about 35 tok/s |
| llama.cpp, Ollama, LM Studio GGUF | Qwen3.6 35B-A3B UD-Q4_K_M | The MoE path is fast on Metal; the dense path is not |
| Either, and you want maximum speed | gpt-oss 20B MXFP4 | 92 tok/s and the full 128K window |
These numbers come from one machine per tester. They can change with a new llama.cpp release. Measure your own: run llama-bench -m <file> -p 512 -n 128 and compare it with mlx_lm.generate --model mlx-community/Qwen3.8-27B-4bit --max-tokens 128 on the same prompt. For runtime setup, see Ollama vs LM Studio vs llama.cpp vs oMLX.
Speed on the M5 Max 36GB (an Estimate)
No one has published a Qwen3.8 27B measurement on the 32-core M5 Max that we could find. Generation speed scales roughly with memory bandwidth. So the M4 Max MLX figure gives about 35 × 460 ÷ 410 ≈ 39 tok/s on the M5 Max. The MoE gives about 45-58 tok/s by the same ratio. These are our estimates, not measurements. On an M3 Max (300 GB/s), expect about 26 tok/s for the 27B in MLX; on an M3 Pro (150 GB/s), about 13.
Can You Raise the Limit for 8-bit?
You can give the GPU more memory:
sudo sysctl iogpu.wired_limit_mb=NN is in MiB. It does not rescue 8-bit on this tier. Our rule is to leave at least 8 GiB for macOS on a machine you also work on. That gives N = 28672 (28 GiB), only 1 GiB over the default. Q8_0 at 27.05 GiB plus 1 GiB of buffers already needs 28.05 GiB, so no context fits. The tester in #27525 set N = 36000, which leaves macOS under 1 GiB. We do not recommend that. If you want 8-bit 27B or 35B, buy 48GB. See best local LLM for 48GB.
36GB or 48GB: What $600 Buys
Apple list prices, read on apple.com on 2026-10-27, as of October 2026:
| Machine | 36GB (M5 Max, 32-core GPU, 460 GB/s) | 48GB (M5 Max, 40-core GPU, 614 GB/s) | Difference |
|---|---|---|---|
| Mac Studio | $2,499 (512GB SSD) | $3,099 (512GB SSD) | +$600 |
| MacBook Pro 14-inch | $4,099 (2TB SSD) | $4,699 (2TB SSD) | +$600 |
The step buys two things. It adds 10.44 GiB of GPU memory (37.44 GiB default, logged on a 48GB Mac) and 33% more bandwidth. The extra memory runs Qwen3.6 35B-A3B at Q8_0 (34.37 GiB) and Qwen3.8 27B at Q8_0 with long context.
Buy 36GB if you run 4-bit models and want the cheapest Max-chip bandwidth. The $2,499 Mac Studio is the cheapest way Apple sells 460 GB/s. Buy 48GB if you want 8-bit quality on the 27B or 35B class. Neither tier runs gpt-oss 120B; that needs 128GB. Compare the full lineup in which Mac Studio to buy.
See Also
- Best Local LLM for MacBook (2026) — every MacBook RAM tier, with the logged Metal limits
- Best Mac for Local LLM (2026) — Mac mini vs MacBook vs Mac Studio, by tier
- Best Local LLM for 32GB RAM — the tier just below 36GB
- Best Local LLM for 48GB RAM — the tier above, and the 8-bit picks
- M4 Max 36GB vs 128GB for Local AI — the memory decision on the M4 Max
- M5 Max MacBook Pro for Local LLMs — the current 36GB laptop
- Best Local LLM for 24GB VRAM — the same Qwen3.8 27B pick on a GPU
- GGUF Quant Names Explained — what Q4_1, IQ4_NL and UD-Q4_K_XL mean
Sources
- Apple: Mac Studio specs, MacBook Pro specs, configurator prices read 2026-10-27; Apple Support tech specs 121553 (M4 Max) and 117736 (M3 Pro / M3 Max), read 2026-10-28
- Hugging Face API file trees, read 2026-10-28: unsloth/Qwen3.8-27B-GGUF, unsloth/Qwen3.6-35B-A3B-GGUF, mlx-community/Qwen3.8-27B-4bit, mlx-community/Qwen3.8-27B-8bit;
config.jsonof Qwen/Qwen3.8-27B and Qwen/Qwen3.6-35B-A3B - llama.cpp: issue #27525 (M4 Max 36GB speeds and the M1 Max reply), issue #13865 (36GB working set), discussion #15396 (gpt-oss speeds)