I Recommended the 96 GiB VRAM Carveout. Then a 21 GiB Model Hung on Load.

In May I published a setup for this box that said, in so many words: give the iGPU the maximum, 96 GiB, because "the math is one-sided: more VRAM, more options", and run llama.cpp with --no-mmap, because "standard mmap behavior is incompatible with how Strix Halo's BIOS-allocated VRAM works in practice."

The carveout advice was wrong. With 96 GiB reserved as VRAM, a 21 GiB model would not load through mmap on this box, and the --no-mmap flag only worked around that failure; I had presented the flag as a fact about the hardware. The June runs below show what actually happened, and on 11 August the llama.cpp maintainers changed the default load path on CUDA/ROCm and Vulkan integrated GPUs for a related reason.

What happened in June

On June 18 I tried to load a 35B mixture-of-experts model at Q4, 20.82 GiB on disk, through llama.cpp with the ROCm backend and memory mapping at the default of the time, which was on. The box is a Bosgame M5 with the Ryzen AI MAX+ 395 and 128 GB of memory, on Fedora 43 with kernel 7.0.11 at the time, with the 96 GiB carveout from the May post still set.

The load hung. My notes record a 300-second wait with the model GTT-resident and the process in a CPU busy-wait, and no error in the llama.cpp output. I tried it on two ROCm stacks to rule out a bad build, the system ROCm 6.4 with a binary from May and TheRock 7.13 with a fresh self-compiled b9637, and both hung the same way. A 0.9 GiB sanity model loaded on the same binary and ran at 4,565 t/s prompt processing and 144 t/s generation, so the build itself was fine.

What sent me back to it was a thread on r/LocalLLaMA where an owner of the same class of machine was benchmarking 27B and 35B models on ROCm without the load hang, and the common flags in that benchmark table had --mmap 0 in every row.

What the kernel log said

llama.cpp printed no error. dmesg recorded 39 copies of one line in the two seconds after the load started:

amdgpu: SVM mapping failed, exceeds resident system memory limit

plus a single line saying a user buffer address was already allocated by SVM. SVM is the path ROCm uses to make host memory visible to the GPU, and the resident system memory limit is how much of that memory the driver is allowed to pin for it. The mapping failed repeatedly, and from the outside that looked like a hang.

The three loads

Same binary, same model, same ROCm stack, with --no-mmap: the model loaded and ran at 888 t/s prompt processing and 46 t/s generation. That was the workaround I had already been shipping in May. With no-mmap, the same model loaded, so that setting avoided the failing path in this run.

The failure depended on model size and the log talked about a memory limit, so the carveout was the obvious suspect. The next day I dropped it in the BIOS from 96 GiB to 1 GiB, changed nothing else, and loaded the same 20.82 GiB model with memory mapping on. It loaded on ROCm at 1,137 t/s prompt processing and 48 t/s generation, and on Vulkan at 1,077 and 57 t/s.

The three recorded outcomes line up like this:

Of the three configurations, only the large carveout together with mmap failed. The May post had treated the big carveout as free capacity and the flag as a fixed property of the hardware, and the June runs contradict both.

Why "more VRAM" is less memory on this hardware

The CPU and the GPU in a Strix Halo box draw from the same physical 128 GB. The BIOS carveout fences off a fixed amount of that at boot as dedicated VRAM, and that amount is gone from what Linux can see and share, whether the GPU ever fills it or not. With 96 GiB carved out, the shared side of the machine is small, and it is the shared side that memory-mapped model files and host-side registrations have to fit into.

The kernel parameter ttm.pages_limit is a separate thing. It is the ceiling on how much system memory the GPU driver may use through GTT, and by default the kernel sets it to about half of system memory. Raising it lets the driver use nearly the whole machine. It counts 4 KiB pages, so the value looks nothing like a gigabyte figure. The older amdgpu.gttsize takes megabytes; on my kernel both are set and they agree. The May post already included both parameters. The June report records the resulting 124 GiB GTT ceiling beside the 96 GiB carveout. The part I had not understood is that the June box had a 124 GiB configured GTT limit and a 96 GiB carveout at the same time, and I had taken the limit for actually available shared memory.

AMD's own Strix Halo optimization guide for ROCm 7.2, dated 20 February 2026, says to keep the dedicated VRAM reservation in the BIOS small, 0.5 GB as its example, and raise the shared TTM/GTT limit instead. The Linux community guides I have seen say the same. Some current Windows guides still recommend 96 GiB, and so did my May post.

Upstream changed the default on 11 August

I was not the only one to set a 96 GiB carveout and lose a model load to it. Other Strix Halo owners have reported loads failing under the same split, usually with much larger models than mine; one report from 6 August on another Bosgame board pairs a 96 GiB carveout with a 39.6 GiB model and the same kernel line. A separate llama.cpp issue tracks large-model mmap hangs on ROCm even without that carveout constraint, and it is still open as of this month.

On 11 August 2026 the llama.cpp maintainers merged PR #26081, which adds a --load-mode auto default. Their reasoning, in short: an integrated GPU has to copy the model into device-visible shared memory anyway, so with mmap the model is "in RAM twice" during loading, and that default was "leading to issues for users that don't know about this." With auto, llama.cpp skips mmap when a device reports that it does not support it, and the change flags CUDA/ROCm and Vulkan integrated GPUs that way. Other backends were left alone.

That is their explanation, not my measurement. That separate issue also gives the new default a clean test on a machine without my carveout mistake. A report from 13 August on a Framework Desktop found that forcing --load-mode mmap still hung on ROCm 7.2.4 and 7.14, while --no-mmap and the new auto default both loaded in about 23 seconds. On that current Strix Halo build you have to ask for mmap to hit this. On the June build, mmap was what you got without asking.

So the load hang now mostly affects older builds, runtimes that bundle their own copy of llama.cpp until they pick up the change, and anyone who forces mmap explicitly.

What this box runs today

I read these values back on 24 August 2026, on kernel 7.1.5, Fedora 44, llama.cpp build 10587 on Vulkan. The BIOS and kernel side:

The Setup Pack carries these values and this correction; the full command line, the Fedora and Ubuntu steps, and the systemd units are there.

On the llama.cpp side, build 10587 marks --mmap and --no-mmap as deprecated. The current spelling of no-mmap is --load-mode none, and the default is --load-mode auto, which after the 11 August change means no mmap on this integrated GPU. All but one active model profile on this box still carries no-mmap = on from the older configuration. On this build that matches what the default already does, and I have not run an A/B between the explicit flag and auto under the 1 GiB carveout, so I am not going to claim the two are identical in speed or behaviour. The flags stay until I have measured that.

What I can't tell you

I do not know whether the model was physically in memory once or twice under mmap. The maintainers' description above says the model is in RAM twice during loading on an iGPU. My runs show that the mmap path hit the resident-memory limit, while the no-mmap path loaded successfully. They do not include a footprint measurement that would tell you how many bytes each path actually held, and adding up RAM and GTT counters does not answer it, because pages mapped into more than one address space get counted more than once. A real answer needs /proc/PID/smaps for the mapped file plus the amdgpu pool counters, on the same model and build, with mmap on and off.

If you have measured that, I want your numbers: model size, build, carveout, ttm.pages_limit, and the smaps and pool figures for both paths. I will add them to the live bench page with credit.

The other thing I have not measured is Vulkan under pressure. The hang I measured was on ROCm, with llama.cpp, on Linux. I never loaded through Vulkan under the 96 GiB carveout, so I cannot tell you what a starved shared side does there, and I do not know what Ollama, LM Studio, or a Windows driver do in the same situation. Reducing the carveout made the workaround unnecessary for this model on that build; it does not prove that for every model or profile.

If your model hangs on load

These checks are for Linux, and the error string is specific to ROCm with llama.cpp. I have not measured Windows VGM settings or drivers, and I am not assuming they behave the same way.

Check three values separately, because one of them can look fine while another is the problem. mem_info_vram_total under /sys/class/drm/ for your GPU tells you the fixed carveout. free -g tells you what Linux can actually see and share after that carveout. mem_info_gtt_total tells you the configured GTT ceiling. In June my ceiling read 124 GiB and the box still could not map a 21 GiB model, because the 96 GiB carveout had left too little shared memory underneath that ceiling.

If a load hangs on ROCm with llama.cpp, read dmesg. If you see SVM mapping failed, exceeds resident system memory limit, --load-mode none (the old --no-mmap) gets you running today, and a current upstream llama.cpp build containing PR #26081 should already select that path unless you force mmap. With the raised TTM limit already in place, the one change that made mmap load this model on this build was reducing the carveout to 1 GiB, then rebooting.

If the kernel log shows something else, you have a different problem. Post that kernel log in the comments so we can identify it.