ResearchPractical AIProjectsOpen SourceAbout
Research/Engine or Model? The Same Giant Coder Model Delivered 6.8 Times the Correct Answers per Hour on Niko1221's Strata Engine Than on Stock llama.cpp
Local AIInference EnginesMixture of ExpertsBatch ThroughputPre-RegisteredOriginal Research

Engine or Model? The Same Giant Coder Model Delivered 6.8 Times the Correct Answers per Hour on Niko1221's Strata Engine Than on Stock llama.cpp

My earlier studies compared a 9B model on stock llama.cpp with a giant Coder model on Niko1221's Strata engine, so every gap mixed the model and the engine together. This one runs the same Coder file on both engines over the same queue of short business tasks. Niko1221's Strata engine delivered about 6.8 times the correct answers per hour, with accuracy within a point. The planned 2x2 became three cells, because the 9B does not load on that engine.

Bart Porcincula · Published

Finding

On the first 483 items both runs finished, on an RTX 5060 Ti 16 GB at concurrency 1, the Qwen3.8-Flash-Next Coder IQ1_M delivered 1,452 correct items per hour on Niko1221's Strata engine (Terminal 0.1.54, reused from the 8 h run) against 214 on stock llama.cpp b11753 (2 h run, experts on CPU, no multi-token prediction): ratio 6.78 (95% CI 6.69 to 6.86, paired bootstrap, 10,000 draws). Accuracy was 87.99% against 88.82%, a gap of -0.83 points (CI -2.07 to +0.21), most of it on intent classification (-2.13 points). The 9B file does not load on Niko1221's Strata engine (exit 5, model type not supported), so the 9B engine ratio and the engine-by-model interaction were not measurable. The 9B on stock was ahead of the Coder by 11.3 times on stock but only 1.68 times on Niko1221's Strata engine.

Same Coder IQ1_M on Niko1221's Strata engine vs stock llama.cpp, 16 GB VRAM (Nov 2026)

One run per cell, one machine, different days. RTX 5060 Ti 16 GB, 31.6 GB RAM. Coder on Niko1221's Strata engine: Terminal 0.1.54, thinking off through a pass-through proxy, 8 h run reused from the earlier study. Coder on stock llama.cpp b11753: --cpu-moe, 32K context, one slot, thinking off, no multi-token prediction, 2 h run. 9B Q4_K_M on stock llama.cpp reused from the earlier study. Same shuffled queue of synthetic items (seed 0), temperature 0, concurrency 1, 300 s cap. Compared on the first K items both runs finished. Hypotheses were written before the first scored run.

MeasurementValueNote
Items finished: Coder on stock llama.cpp in 2 h483median 9.3 s per item, p95 35 s
Correct items per hour (first 483 items)Niko1221's Strata engine 1,452 vs stock 214
Strata / stock correct per hour (headline)6.7895% CI 6.69 to 6.86; interval covers item sampling only
Accuracy (first 483 items)Strata 87.99% vs stock 88.82%gap -0.83 points, CI -2.07 to +0.21
Accuracy gap by taskintent -2.13, extraction -0.61, log triage 0.00log triage 100% on both
9B on Niko1221's Strata enginedoes not loadexit 5, model type not supported; reported, not worked around
9B / Coder correct per hour on stock11.33CI 10.99 to 11.65, first 483 items
9B / Coder correct per hour on Niko1221's Strata engine1.68from the earlier 8 h runs, first 14,022 items
Lowest free RAMstock 140 MiB, Strata 437 MiBboth cells ran at the memory edge

The short version

My last three studies compared a 9B model on stock llama.cpp with a giant coding model on Niko1221's Strata engine. Each of those arms changed the model and the serving engine together, so a gap between them could not be assigned to either one. This study holds the model file fixed and changes only the engine.

  • The engine is most of the speed. The same Coder file delivered 1,452 correct answers per hour on Niko1221's Strata engine and 214 on stock llama.cpp. The ratio was 6.78 (95% interval 6.69 to 6.86).
  • Accuracy barely moved. 87.99% against 88.82%, a gap of 0.83 points. The interval runs from -2.07 to +0.21, so a small real loss is not ruled out. Most of it was in intent classification (-2.13 points); log triage was 100% on both.
  • The 9B does not load on that engine. It exited with "model type not supported", so I could not run the cell that would have separated engine from model for the small model. I reported that and did not work around it.
  • The earlier model gap was mostly an engine gap. On stock llama.cpp the 9B delivered 11.3 times the Coder's correct answers per hour. On Niko1221's Strata engine it was 1.68 times.

Takeaway: when two local setups differ in both model and engine, the comparison mostly measures the engine. The same large model on the right runtime was several times faster than on the general-purpose one.

What changed from the plan

The design was a 2x2: two models (the 9B and the Coder) on two engines. Before any scored item, a launch check showed the 9B Q4_K_M file will not start on Niko1221's Strata engine, which accepts a fixed set of model families, large mixture-of-experts models among them. That removed one cell. The plan was reduced to three cells before the first scored run, and the hypotheses that needed the missing cell were marked not measurable in advance.

How it was run

Same card (RTX 5060 Ti, 16 GB), one run per cell, concurrency 1, temperature 0, thinking off in every cell. The queue was the same 30,000-item shuffled pool of short business tasks as the eight-hour batch study (intent labels, invoice extraction, log triage). The Coder on stock llama.cpp b11753 ran for two hours with the experts kept on the CPU, one slot and no multi-token prediction. The other two cells were reused from the earlier eight-hour runs. The headline compared correct per hour on the first 483 items both runs finished, with a paired bootstrap (10,000 draws, seed 0).

What the hypotheses said

HypothesisVerdict
The Coder has at least 1.3 times more correct per hour on Niko1221's Strata engineSupported (6.78)
The engine changes Coder accuracy by under 1 pointSupported as written (-0.83), but the interval reaches -2.07
The 9B gains under 1.3 times on that engine, or does not loadSupported (does not load)
The 9B is ahead of the Coder on correct per hour on both enginesSupported (11.3 times on stock, 1.68 times on Niko1221's Strata engine)
The 9B-over-Coder ratio differs by under 1.3 times between enginesNot measurable (no 9B cell on that engine)

What the engine did and did not change

The stock run took a median of 9.3 seconds per item with a slowest 5% at 35 seconds. On Niko1221's Strata engine the median was 1.7 seconds and the slowest 5% were at 4.1 seconds or more. Per-item time on the small intent and triage items is where stock llama.cpp lost most, and the ratio was similar by task (6.6 to 6.8 times). I did not separate which feature of Niko1221's Strata engine produces the gain.

What I am not claiming

  • This is one run per cell. The two Coder cells ran on different days; versions are pinned, but drift is a confound. The interval covers which items were sampled and holds per-item time fixed, so it understates run-to-run noise.
  • K is only 483, because stock llama.cpp ran at about 15 seconds per item in the two-hour window. The window was not extended.
  • Both Coder cells ran with very little free RAM (140 MiB on stock, 437 MiB on Niko1221's Strata engine). A stock run with more headroom, or with multi-token prediction available, might be faster.
  • This is a deployment comparison, each engine with its best-effort flags, not a kernel-level one.
  • The tasks are synthetic and short. The Coder is a heavily compressed (IQ1_M) quantisation, and the 9B is a different quantisation (Q4_K_M), so the model effect here is "this file against that file".
  • Nothing here says Niko1221's Strata engine is faster for models it does not support.

Cite this research

Bart Porcincula. "Engine or Model? The Same Giant Coder Model Delivered 6.8 Times the Correct Answers per Hour on Niko1221's Strata Engine Than on Stock llama.cpp." Published 2026-11-02; updated 2026-11-02. https://porsync.com/research/same-giant-model-two-engines-niko1221-strata-vs-stock-llama-cpp-correct-items-per-hour

Permanent article URL