Laguna-XS runs beautifully on my 5090, and I am keeping the model everyone calls outdated

This is the second time I have run this experiment, and I am writing it up because the conclusion held from a completely different direction.

A while back I argued that "which model is best" has no answer until you name the workload. The example then was GLM-4.7-Flash against Qwen3.6-27B. The newer Qwen scored higher on BFCL, tied GLM exactly on the actual job my agent does, and cost me latency for the tie. The headline number pointed one way and the right answer pointed the other. This post is the same lesson with a different model and a cleaner shape. Last time the newer model tied and cost latency. This time it fits my hardware beautifully and fails on the one axis the job cares about most.

The job has not changed. I run a small read-only agent that watches my home and a bit of farm infrastructure. You talk to it over chat, it queries a metrics database and a weather service, and that is all it does. Pure tool-calling, no writes, nothing it can break. It runs GLM-4.7-Flash on a single RTX 5090 with 32GB of VRAM, a consumer desktop.

The candidate this round is Laguna-XS-2.1, a coding-focused model from poolside, released 2026-06-20 under the openmdw-1.1 license. It is a Mixture of Experts, about 33.4B parameters total across 256 experts with a small fraction active per token. It is a reasoning model with native interleaved thinking, built for agentic coding: call a tool, read the result, reason, call the next one. On paper it is exactly the kind of newer, well-regarded model worth a look.

Getting it to load at all

Getting it running took one non-obvious step. llama.cpp only merged support for the "laguna" architecture on 2026-07-22. My server binary was build b10075, compiled 2026-07-20, and it refused to load the model. I had missed architecture support by two days. The weights existed, the GGUF existed, and the binary still could not read it, because the support had not shipped at the time I built. I staged the current release, b10098, next to the old one, and it loaded fine. The lesson is small but worth keeping: for a brand-new architecture, check your binary's build date against the date arch support merged, not just that a GGUF is sitting on disk.

What the card can actually hold

Everything below was measured on the card, flash attention on, Q8 KV cache.

At Q6_K, the highest-quality quant that fits, with a 128k context window, the model sits at 30.8 GB of 32, leaving about 1.7 GB spare. Drop to Q4_K_M and I can load the model's full 262k native context for 27.4 GB, roughly 5 GB to spare. Generation runs around 229 tokens per second, faster than the 201 I get from the GLM it might replace. Tool calls parse correctly through llama.cpp's built-in template on b10098, despite poolside's warning that parsing off vLLM can be fragile. That warning did not bite here.

So on fit, this is a clean win. The 5090 holds this model at maximum quality and 128k context with headroom, or at long context with more headroom, and runs it faster than the incumbent. If "fits the card" were the question, Laguna would win it outright.

Fit is not fitness, though, and the first crack showed up before I ran a single benchmark. Laguna reasons hard by default, and poolside's default max output is 32768 tokens. My first smoke test asked for a plain Fibonacci function with a 2048-token budget. It spent all 2048 tokens thinking and produced zero answer: finish reason length, 6188 characters of reasoning content, empty output. For fast interactive use you turn thinking off with enable_thinking=false. For genuinely hard tasks you leave it on and budget 8k to 16k or more output tokens. Fine once you know it, but it tells you what kind of model this is.

The benchmark, and the number I did not trust

I ran BFCL v3 end to end, all 1298 cases, on the same harness that measured GLM and Qwen. For Laguna I used poolside's own recommended settings: temperature 1.0, top_k 20, top_p 1.0, thinking on. Here is every category next to the two models from the earlier post.

The 67.95% overall reads like a collapse. Two rows drive it, parallel at 0.0% and irrelevance at 60.0%, and both looked wrong enough that I read the cases behind them before trusting the headline.

Verification 1: the parallel 0% is a harness artifact

BFCL's "parallel" category requires the model to emit several tool calls in a single response. Laguna scored 0.0% across all 200 parallel cases, an absurd number for a model this capable, so I read the transcripts. On every one of the 200, Laguna emitted exactly one tool call, and that first call was correct every time: right function, right arguments. It is an interleaved caller by design. It makes one call, waits for the result, then decides the next. My grader is single-turn, so it only ever sees the first call and scores the case zero even though the call is right. That category cannot be read as a failure. It is my harness failing to score an agentic model, not the model failing the task. I set parallel aside.

Verification 2: the irrelevance 60% is real

The other suspicious row was irrelevance, where Laguna scored 60.0% against GLM's 85.4% and Qwen's 87.5%. BFCL's "irrelevance" category measures the opposite of the usual skill: it checks whether the model correctly makes no call when no tool is warranted. I went through these cases too, hoping for another artifact. There was not one. Laguna failed 96 of the 240 irrelevance cases by calling a tool anyway. And it really made the calls: str_to_int, investment_calculator, magnetic_field_intensity, and others, fired on questions that asked for none of them. It reaches for a tool about 40% of the time when the right move is to sit still. The over-calling is the model, not the harness.

Why that one row decides the job

For a coding assistant, an eager reach for tools is often a feature. For mine it is the disqualifying trait. My agent is read-only monitoring, and a spurious tool call there becomes a fabricated reading. On a farm a fabricated reading gets acted on: a tank that is not low, a temperature that was never measured. The whole reason the agent runs GLM-4.7-Flash is that GLM is strong on exactly this axis, doing nothing when nothing is called for. Laguna is markedly worse at it. However good it is at writing code, a model that over-calls is the wrong shape for a monitor.

The caveats, honestly

Three things keep me fair to Laguna here. First, it ran at temperature 1.0, poolside's shipped config, while GLM and Qwen ran at 0.2. Higher temperature makes spurious calls more likely, so some of the irrelevance gap is probably temperature rather than the model. I used 1.0 because it is what poolside recommends, but it is a confound and I want to name it. Second, the parallel result and the interleaved style mean my single-turn harness genuinely cannot score Laguna's agentic loop. Doing that fairly needs real multi-turn tool execution, which I have not built. On the three single-call categories the harness is fair, and there Laguna is right in the mix: 91.0 on simple, 89.5 on multiple, 75.6 on live_simple. Third, my grader is my own, not the official BFCL scorer, so every number above is conditional on it.

Fit and fitness are different questions

Laguna-XS-2.1 is a strong, current, well-regarded coding model, and on my hardware it is a clean fit: maximum quality at 128k context, faster than what I run now, room to spare, the full 262k context if I want it. For coding and agentic development, its actual purpose, it may be excellent, and I am not judging that here. For this job it is the wrong tool. It over-calls on the single axis that matters most for a read-only monitor, and its interleaved style does not match a batch-query agent. So it stays a coding model on the shelf, and the monitoring agent keeps GLM.

Two models now, two different failure shapes, one result. The first time a newer model tied on the real work and cost me latency. This time a newer model fits the card better than the incumbent, runs faster, and still loses on the axis the job is built around. Neither outcome shows up in a top-line benchmark score. Both show up the moment you attach the actual workload. The raw 67.95% would have told me Laguna was hopeless. The transcripts told me it was fine at the task and wrong for the job, which is a more useful thing to know.

Test it on your own job

There is a broader lesson under both of these experiments. Qwen3.6 and Laguna did not come to me at random. They are models the local-LLM community recommends, the ones that come up when people ask what to run. GLM-4.7-Flash is the one that gets waved off as older and less interesting. Yet the two recommended models both measured worse on my actual workload than the one that got dismissed, and I would not have learned that from any comment thread or benchmark leaderboard.

A recommendation online is someone else's workload on someone else's hardware, stated with confidence. It is a starting list, not an answer. The only way to know what runs best for your job is to run the candidates yourself, on your own workload, on your own machine, and read the transcripts when a number looks wrong. The model everyone keeps telling me to replace has won every time. On my hardware, for my job, it is still the best tool I have found.