Strata is a new inference engine written for one model, Qwen3.8-Flash-Next, using its own Metal kernels plus parts of ggml. On my 128GB M5 Max MacBook Pro, with the full 262K window and vision loaded, it decodes about 25% faster than oMLX once the context holds real content: 78 vs 63 tok/s at 32K, 75 vs 61 at 128K, and 74 vs 57.5 at 245K.
The shape of the curve is the interesting part. Strata loses about 13% from an empty context to 245K. oMLX loses about 46% over the same range. On an empty context oMLX is faster, 106 vs 84 tok/s, and the lines cross somewhere before 32K. Prompt reading (prefill) is about 12% slower on Strata, and answer quality on my small test suite came out the same.
The catch is concurrency: on a Mac, Strata serves one request at a time. More on that at the end, because it decides where each engine fits.
Setup
Strata is MIT licensed and about a month old. I pinned v0.2.2 and built from source. One script does the whole setup:
git clone https://github.com/Niko1221/Strata && cd Strata && git checkout v0.2.2
./setup.sh --setup --model UD-Q4_K_XL --context 262144 --vision yes --no-startThat run builds the Metal engine, downloads the model (about 90 GB) and the vision projector, repacks the weights into a native Metal file, fetches the MTP draft layer, and writes a start script. After that, ./setup.sh on its own starts the server on the port in its config. Load took about 30 seconds. It exposes an OpenAI-compatible API, an Anthropic-compatible /v1/messages endpoint, and a browser chat page.
The model is unsloth's UD-Q4_K_XL GGUF of Qwen3.8-Flash-Next in 4 shards, with the BF16 mmproj for vision. Strata's setup writes the MTP draft layer as mtp-q2_0.gguf, 0.889 GB, and speculates 2 tokens per step (--spec 2). The oMLX side is 0.7.1rc1 with Jundot/Qwen3.8-Flash-Next-oQ4e-mtp at 4.72 bits per weight, mtp_enabled and qwen4_ple_ssd_offload both true, the same setup as the last post. These are different quants of the same model, both around 4 bits, so some of the gap in either direction could be the quant rather than the runtime.
Strata wires the model into RAM and keeps it there. Free memory went from 91% to 29% when it loaded, and the docs put the resident set at about 91 GB. oMLX, in my config, unloads after 30 idle minutes, so the laptop feels different with Strata up: less a server that comes and goes, more an appliance that happens to have a keyboard.
One tip: by default Strata caps the thinking budget to leave room for the answer. With max_tokens 8000 it caps thinking at 6000. The config key is answer_room.
On stability, the Strata docs describe kernel panics on M5 Max in earlier versions, traced to memory pressure and fixed with footprint caps and a wired (mlocked) arena. My two-hour run was clean.
Method
Same harness for both engines, timed from the streamed response. Prefill is prompt tokens divided by time to first token. Decode is generated tokens minus one, divided by the time from first token to last.
The context filler is real C++ source, llama.cpp's src/ tree, cut to exact token counts with the model's tokenizer: empty, 32,768, 131,072 and 245,760 tokens. The question at each depth is "Explain in about 400 words what the C++ code above does", 1,024 generated tokens, temperature 0. A warm-up request runs first. Each depth is repeated non-streamed as a check, and the numbers matched. The vision check reads a serial number and device ID off a photo of a sensor label, and both engines got it right.
One note on the prefix cache: each deeper prompt starts with the previous one, so the 128K and 245K prompts partly hit the cache on both engines. Compare prefill at 32K, where the cache was cold.
The numbers
Decode, streamed, in tok/s as oMLX / Strata: empty 106.2 / 84.3, 32K 63.0 / 78.4, 128K 61.2 / 75.2, 245K 57.5 / 73.7.
- Temperature 0.7 on an empty context: oMLX 81.3, Strata 97.4.
- Prefill at 32K with a cold cache: oMLX 1,631 tok/s, Strata 1,437, about 12% slower.
- Time to first token at 245K, with 131K of it cached: oMLX 76 s, Strata 86 s.
Quality on my own small suite, 3 reps each, came out the same within noise. Tool calling: 21/24 on both, missing the same case. Reasoning puzzles: 52/54 on oMLX, 53/54 on Strata. Agent tasks against a real metrics server: 29/30 on both, with Strata averaging 34 s per task against 40 s. Strata used about 20% fewer tokens per reasoning item, likely because of the thinking cap.
A correction to the last post: on a server that has already done some work, oMLX's empty-context speed is 106 tok/s, well above the 64.5 I reported then. The first long generation after the model loads runs slower, and my earlier number came from a freshly loaded server. The deeper numbers match.
Where it fits
Strata's docs say --parallel (batch slots) is not ported to Metal yet, so on a Mac it serves one request at a time and other clients wait their turn. oMLX handles up to 8 at once in my config, and my assistant plus a couple of research agents share this machine. So oMLX stays my daily server for now, and Strata is what I switch to for one long job: a quarter-million tokens of code and a question, where its flatter curve is worth about 16 tok/s at the deep end.
I did not expect a month-old engine built for a single model to hold its speed this well across the whole window. Hand-written kernels for one architecture, ahead of a general MLX server at long context on the same hardware, is the part I keep thinking about. I am looking forward to parallel slots landing on Mac.
Comments
// Comments are reviewed before appearing. No spam. No noise.