# Bart Porcincula > Bart Porcincula is a Philippines-based Senior AI Engineer who runs large language models locally (llama.cpp, Ollama, GGUF) on a single consumer 16 GB GPU. This site is his portfolio of original research and projects on running, training and evaluating small AI models, with measured results, including the ones that didn't work. Builder: Bart Porcincula (Senior AI Engineer, based in the Philippines) Focus: running and evaluating open language models locally (llama.cpp, Ollama, GGUF) on a single consumer 16 GB GPU About: https://porsync.com/about Open-source projects: https://porsync.com/open-source Lab and current hardware: https://porsync.com/lab Dated experiment status notes: https://porsync.com/lab/notes RSS: https://porsync.com/feed.xml Contact: connect@porsync.com Website: https://porsync.com Full text: https://porsync.com/llms-full.txt This site is a personal portfolio. It publishes original research on running, training and evaluating small AI models, and technical write-ups of projects. It is read-only: there are no forms, bookings or services. --- ## Research (original experiments, with the numbers) ### The Same Coding Model Improved When Its Harness Changed A separate 288-attempt follow-up put Aider ahead on three models and both Patchsmith versions near ceiling on Qwen3.6. Harness changes helped some models, without establishing a general winner. Finding: In the separate n09-n16 cohort, Patchsmith v5b/v6/Aider passed 13/19/23 of 24 for gpt-oss, 14/15/17 for Qwen3-Coder, 24/24/23 for Qwen3.6 and 18/17/23 for Edge0. Edge0 v5b is not blind; cohorts are never pooled. Two simultaneous v6 changes and unequal budgets prevent a causal harness ranking. Published: 2026-11-19 URL: https://porsync.com/research/same-coding-model-different-harness ### A 120B Model Fit on My 16 GB Card. Reliable Coding Was Still Unresolved Edge0 ran interactively. A separate routing-accounting retry passed 15/15 final checks, with two frontier repairs, but showed no speed win. The router remains frontier-only. Finding: Edge0-120B-q3k passed 27/50 and 21/50 harder synthetic coding attempts in separate live-context runs without execution errors; earlier runs encountered OOM responses and prefill timeouts. Separate CLI accounting passed 21/21 direct, 36/36 shadow and 15/15 retry final checks; two retry tasks needed frontier repair. The retry showed no speed win or promotion conclusion. Broader comparisons are outside this selected release. Published: 2026-11-15 URL: https://porsync.com/research/edge0-120b-16gb-fits-but-agent-reliability-unresolved ### Expert Caching Made Tokens Faster. Coding Throughput Did Not Improve Clearly A 6,000 MiB cache improved generation speed, but passed 8/50 repair attempts versus 9/50 without it. A larger timing-only cache was faster, with no agent result. Finding: For Qwen3-30B-A3B Q4_K_M on RTX 5060 Ti 16 GB, a 6,000 MiB cache raised later-request decode speed from 30–33 to 41–50 tokens/s, while verified coding throughput was 12.35 versus 12.93 passes per task-hour; correctness equivalence is not established. Published: 2026-11-15 URL: https://porsync.com/research/expert-cache-faster-tokens-no-coding-throughput-gain ### The AI Could Read the Answer. The Search Pipeline Had to Find It First I tested a chatbot on 96 questions about a fictional knowledge base. Giving it the whole parent article after search raised accuracy from 78% to 89%; giving it the answer sections directly reached 99%. Finding: On 96 questions over a synthetic HTML knowledge base, Claude Haiku 5.5 scored 78% with heading-based retrieval and 89% when the top three parent articles were included; the paired difference was 10.4 percentage points (95% bootstrap interval 3 to 19). Given gold sections, it scored 99%. Published: 2026-11-13 URL: https://porsync.com/research/knowledge-base-ai-retrieval-parent-article-expansion ### A 30B Model Fit on My 16 GB Card. Eight Layers Changed Its Speed On one 16 GB GPU, moving eight layers of a 30B model off the card cut short-prompt generation from 21.7 to 11.4 tokens per second while saving GPU memory. This measures speed and memory, not answer quality. Finding: In three short-prompt timing runs per setting on one 16 GB RTX 5060 Ti, Muse Glimmer 30B generated 21.7 tokens per second with 52 layers on the GPU and 11.4 with 44; peak GPU memory was 15,873 and 13,650 MiB respectively. A separate 100,449-token synthetic prompt ran once at 9.4 tokens per second after prefill. Published: 2026-11-13 URL: https://porsync.com/research/30b-model-16gb-layer-offload-speed-memory ### Two Local Models Followed Every Written Rule and Missed Many Implied Ones Qwen3.6-27B and Gemma 4 12B passed all 252 explicit-rule checks across roughly 4K, 16K and 32K tokens. With the rules implied, Qwen passed 52–55% and Gemma passed 24–29%. Giving each model its own extracted rule list never raised its score. Finding: At each of three context lengths, both models passed 42 of 42 explicit-rule probes. Qwen passed 23, 22 and 22 with implied rules and 19, 20 and 21 with a same-model extraction pass. Gemma passed 12, 10 and 11 with implied rules and 12, 8 and 10 with extraction. Published: 2026-11-09 URL: https://porsync.com/research/local-ai-stated-vs-implied-rules-replication ### A Small Model Recovered More Details When a Different Model Wrote Its Summary The same Qwen3.5-9B answered 17.5% of detail questions from its own short structured summary and 77.5% from Qwen3.6-27B’s. Asking the 9B to write a longer summary raised its score to 78.8%. Finding: On 240 matched probes per condition, Qwen3.5-9B passed 42 from its own short structured summary, 186 from the 27B writer’s short summary, 189 from its own longer summary, 209 from the 27B writer’s longer summary and 211 from the full transcript. Published: 2026-11-09 URL: https://porsync.com/research/local-ai-summary-writer-detail-recovery ### The Quote Was Real. The Contract Answer Was Still Wrong. A literal quote check caught zero of 49 answers on contract fields labeled absent across three local deployments. It also discarded seven answers that passed the recall scorer. The same payment-allocation passage fooled all three deployments on a governing-law question. Finding: On the same 630 field–excerpt pairs, the quote check kept all 19 labeled-absent answers from Qwen3.5-9B, all 17 from Gemma 4 12B and all 13 from Qwen3.6-27B. Requiring a quote changed overlap recall by −6.7, +1.9 and −1.0 percentage points, respectively. Strict value match fell on all three deployments. Published: 2026-11-09 URL: https://porsync.com/research/local-ai-contract-extraction-quote-check-replication ### Gemma Invented Zero Search Parameters and Returned 307 Invalid Replies Gemma 4 12B made 885 tool calls without inventing an argument key. It still failed the final-answer format in 307 of 600 attempts, compared with 72 on Qwen3.5-9B. A third local deployment stopped at the development gate after four searches used category values outside the catalogue. Finding: On the same 150 synthetic shopping requests under four tool descriptions, Gemma 4 12B passed 241 of 600 tasks and Qwen3.5-9B passed 527. Gemma produced 76, 77, 77 and 77 invalid finals across the four conditions. Qwen produced 19, 7, 28 and 18. Both invented zero argument keys. GPT-OSS 20B passed 6 of 10 development tasks against a threshold of 7 and recorded no test outcomes. Published: 2026-11-09 URL: https://porsync.com/research/local-ai-tool-filters-gemma-replication ### Three Local Models Read the Same Receipts. Repair Fixed No Totals. Gemma 4 12B and Qwen3.6-27B got all 42 printed totals right on their first attempt. Qwen3.5-9B repeated its one wrong total, which passed the arithmetic check. An instruction and a repair pass changed some line-item scores while exact-total counts held across all three models. Finding: On 42 eligible CORD v2 receipt transcripts, baseline, instructed and repaired exact-total counts were 41/42 throughout for Qwen3.5-9B and 42/42 throughout for Gemma 4 12B and Qwen3.6-27B. Arithmetic contradictions moved from 4 to 4 to 3, 2 to 1 to 1, and 0 to 0 to 0 respectively. Six repairs ran across the three deployments, with zero total fixes or breakages. Published: 2026-11-09 URL: https://porsync.com/research/receipt-arithmetic-checks-three-local-models ### A Numbers-Only Instruction Halved Flags on One Local Model The original 13-to-6 drop in checker flags reproduced on Qwen3.5-9B. Gemma 4 12B went from zero to one, while Qwen3.6-27B went from six to five. A review of passing summaries found claims that misused valid source numbers. Finding: On the same 200 five-year CO₂ tables from 113 countries, a numbers-only instruction changed numeric-membership flags from 13 to 6 for Qwen3.5-9B, 0 to 1 for Gemma 4 12B, and 6 to 5 for Qwen3.6-27B. All three paired 95% country-cluster intervals included zero. In a seeded sample of 50 passing summaries per model, agent review found 11, one and two clear contradictions respectively. Published: 2026-11-09 URL: https://porsync.com/research/numeric-summary-checks-three-local-models ### The Numbers Matched the Table. The AI Summary Still Got the Facts Wrong. I tested summaries of 200 public CO2 tables with a small local model and a numeric checker. The checker rejected supported approximations while letting false comparisons and trends through. In an agent review of 50 passing summaries, 11 contained clear factual contradictions. A clean numeric-membership score measured much less than factual accuracy. Finding: On 200 five-year CO2 snapshots across 113 countries, a frozen numeric-membership checker rejected 13/200 plain Qwen3.5-9B summaries and 6/200 with an added numbers-only instruction. The paired country-cluster bootstrap interval for the change was −7.32 to 0.00 percentage points. Agent review of all 21 flagged numeric occurrences found 15 false flags, four legitimate flags and two uncertain cases; review of a seeded 50 passing summaries found 11 clear factual contradictions. Guarded outputs had zero membership failures by construction, which does not establish factual correctness. Published: 2026-11-08 URL: https://porsync.com/research/ai-summary-numbers-match-table-claims-still-wrong ### The AI's Receipt Total Added Up. It Was Still Wrong. On 42 eligible receipt transcripts, a local model produced four arithmetic contradictions. The only wrong total passed the arithmetic check: the model had multiplied line amounts by quantities again, then added its mistaken numbers correctly. One repair fixed a line amount, but total accuracy stayed 41/42. Finding: On 42 eligible CORD v2 test receipts supplied as human-transcribed annotated text, Qwen3.5-9B Q4_K_M produced four baseline arithmetic contradictions and one wrong total. The wrong total was internally consistent and escaped the checker. One re-ask reduced contradictions to three and improved macro item F1 from 89.47% to 90.67%, while exact total accuracy remained 41/42. Two identical-prompt repeats matched baseline on all receipts. An arithmetic instruction corrected one tax/service swap without changing primary endpoints. This selected subset and its single wrong total cannot establish a general detection rate. Published: 2026-11-08 URL: https://porsync.com/research/ai-receipt-total-adds-up-still-wrong ### No Invented Search Filters, but 72 Answers Failed the Format Check I tested whether telling a local AI that a search tool's filters were its only filters would stop it inventing new ones. Across 600 attempts on 150 synthetic shopping requests, it invented none. The more visible problem was finishing: 72 replies failed the required output format, and one further reply searched the wrong category. The shortest added instruction had the fewest format failures in this run, but that pattern did not recur on Gemma 4 12B (76 versus 77 invalid finals). The original filter-error question hit a zero-error baseline. Finding: On 150 generated product-search requests run once under each of four tool-description conditions, Qwen3.5-9B Q4_K_M made 1,696 tool calls with zero invented parameters. Among 333 readable finals on 400 unsupported-filter attempts, the corrected structured-field scorer found zero false application claims or silent constraint drops; 67 unsupported attempts had invalid finals. Across all 600 attempts, 72 failed the output contract and one further attempt used incorrect category values, leaving 527 task successes. Invalid-final counts were 19/150 with plain descriptions, 7/150 with an only-filters sentence, 28/150 with an added disclosure instruction, and 18/150 with schema annotations plus server rejection. All arms shared an instruction to disclose unverifiable requirements; one catalogue, narrow templates, and a post-start scoring correction limit the conclusion. Published: 2026-11-08 URL: https://porsync.com/research/local-ai-tool-filters-valid-parameters-invalid-answers ### The Same Coding Model Got Six Times as Many Incidents Right per Hour on a Different Engine An earlier test found a nearly sevenfold speed gap between two ways of running the same large model on one 16 GB card. I repeated the comparison on incident triage. Niko1221's Strata engine delivered 1,278 correct incidents per hour versus 208 for my original llama.cpp setup, with less than a point of observed accuracy difference. Tuning llama.cpp later narrowed the gap. Finding: With Coder IQ1_M on the same first 762 synthetic incidents, Niko1221's Strata engine delivered 6.14 times the correct items per hour of stock llama.cpp configured with all experts on the CPU (95% paired interval 5.95 to 6.34). Accuracy was 82.68% versus 81.89%, a difference of +0.79 points (95% interval -0.26 to +1.84). Published: 2026-11-07 URL: https://porsync.com/research/same-coder-two-engines-incident-triage-16gb ### Tuning llama.cpp Cut a Sixfold Speed Gap to 2.7 on My 16 GB Card My first llama.cpp setup kept all of the model's experts on the CPU. Moving some onto the GPU more than doubled its output. Niko1221's Strata engine still got 2.71 times as many incidents right per hour as the tuned setup. I also turned off its multi-token prediction and tested a small draft model in stock llama.cpp to see how much each accounted for. Finding: On 762 matched synthetic incidents with Coder IQ1_M, tuning stock llama.cpp increased correct throughput from 208 to 472 per hour (2.27×). Niko1221's Strata engine reached 1,017 with multi-token prediction off (2.15× tuned stock) and 1,278 with it on (2.71× tuned stock, 95% paired interval 2.66 to 2.76). Adding a 0.8B draft model to tuned stock reached 528 per hour (1.12×). Published: 2026-11-07 URL: https://porsync.com/research/tuned-llama-cpp-vs-strata-16gb ### A 74B Coding Model on a 16 GB Card: It Fit With Room for a 116K-Token Prompt I ran a community build of Nemotron Bolt on a 16 GB graphics card. It used 14.8 GB of video memory with 116,100 tokens in the prompt and generated about 50 tokens a second. Its coding and tool-call scores came within 2.5 points of the builder's reported scores. On 24 everyday questions, my smaller 9B model got five more right. Finding: Nemotron Bolt REAP50 IQ2_XXS ran on an RTX 5060 Ti 16 GB with stock llama.cpp b11753. At a 128K context setting with 116.1K tokens actually in the prompt, it used 14.8 GB of video memory and decoded at 50.3 tokens per second. It scored 86.0% on HumanEval+, 69.6% on MBPP+ and 90.2% on BFCL simple call correctness. It answered 19 of 24 non-code questions correctly versus 24 of 24 for Qwen3.5-9B. Published: 2026-11-07 URL: https://porsync.com/research/nemotron-bolt-74b-on-a-16gb-graphics-card ### Giving a Local AI the Runbook Raised Incident-Triage Accuracy to 57%. A Bigger Model Reached 80% I gave two local models the same incident reports, retrieved runbook pages and system state. The 9B got all five required fields right on 56.9% of incidents. The bigger Coder got 80.3% right. The 9B finished more correct incidents per hour across different engines, but the accuracy gap was far outside the five-point margin I set for swapping one for the other. Finding: On 3,000 synthetic incidents, Qwen3.5-9B with retrieved documents and system state scored 56.9% versus 80.3% for Coder IQ1_M on Niko1221's Strata engine with the same prompt. The paired gap was -23.4 percentage points (90% CI -25.0 to -21.9), outside the pre-set ±5-point equivalence margin. The 9B delivered 1,301 correct tasks per hour versus 1,233 for the Coder. Published: 2026-11-07 URL: https://porsync.com/research/local-model-incident-triage-retrieval-16gb ### Can a Small Local AI Make the Code Change? 360 Attempts Across Six Jobs I gave two local models and Claude the same four small bugs and two small features, repeating each job 20 times per model. The 9B and Claude passed every attempt. The larger local Coder passed 119 of 120 directly. On the remaining attempt, the runner verified that it had undone the local edits, then Claude solved the job from a fresh copy. Finding: On six fixed synthetic Python tasks repeated 20 times per setup, Qwen3.5-9B through Ollama and Claude Sonnet 5 each passed 120 of 120 attempts directly. Coder IQ1_M on Niko1221's Strata engine passed 119 of 120 directly and 120 of 120 with one measured Claude fallback. Throughput including fallback was 193.11, 185.72 and 104.38 accepted attempts per hour for the 9B, Claude and Coder respectively. Published: 2026-11-07 URL: https://porsync.com/research/local-coding-agent-six-tasks-360-attempts ### Under Load, the Speed Gap Holds but the Latency Target Does Not: The Same Giant Coder Model on Two Engines, Tested with Detonate My last study measured two engines one request at a time. This one sends several requests at once, using the open-source Detonate load tester, and repeats the whole sweep to see what is stable. Niko1221's Strata engine generated about 3 to 4 times the tokens per second of stock llama.cpp at every load level, in both runs. Neither engine met a normal latency target beyond one user, each ran a single slot, and stock llama.cpp met it for one user in the first run but not in the second. Finding: Same Qwen3.8-Flash-Next Coder IQ1_M file, RTX 5060 Ti 16 GB, one slot on both engines, Detonate v0.1.0 concurrency sweeps, 256-token prompts and 64-token answers, 60 s per level, two runs per engine (run 1 at 1, 2, 4 and 8 users; run 2 at 1, 2, 3, 4 and 8). Output speed: Niko1221's Strata engine (Terminal 0.1.54) 31-33 tokens per second at every level in both runs, against 7-10 for stock llama.cpp b11753 (3.1 to 3.6 times in run 1, 3.7 to 4.3 times in run 2). Strata's numbers repeated almost exactly; stock's did not. Time to first token at the 95th percentile, run 1 then run 2: Strata 0.88 / 0.85 s at 1 user, 3.0 / 3.0 s at 2, 7.2 / 6.9 s at 4, 39.8 / 30.6 s at 8; stock 1.78 / 4.12 s at 1 user, 21.7 / 12.1 s at 2, 28.8 / 27.3 s at 4, 61.1 / 57.1 s at 8. Run 2 added 3 users: Strata 5.0 s, stock 28.0 s. Against the default target (95th-percentile first token within 2 s and per-token time within 100 ms) Strata serves 1 user in both runs and fails from 2 up. Stock served 1 user in run 1 (narrowly, 1.78 s) and none in run 2. Both engines break (over 30 s) at 8 users in both runs. No errors at any level. Published: 2026-11-03 URL: https://porsync.com/research/giant-coder-model-two-engines-concurrent-load-detonate-niko1221-strata-vs-stock-llama-cpp ### Run Eight Requests at Once and a 9B Model Delivers 1.32 Times the Correct Answers per Hour, Not the 2 Times I Predicted. The Giant Coder Model Got Slower. My earlier batch studies sent one request at a time. A batch job has nobody waiting, so it can send several at once. I tested 1, 2, 4 and 8 parallel slots for the 9B model and 1 and 2 for the giant Coder model on Niko1221's Strata engine, over the same queue of short business tasks. Batching helped the 9B by 1.32 times at eight slots, accuracy stayed within a third of a point, and the price was a 9-fold rise in the slowest 5% of response times. The Coder got slower at two slots. Finding: On an RTX 5060 Ti 16 GB, Qwen3.5-9B Q4_K_M on stock llama.cpp b11753 delivered 2,584 correct items per hour at one slot and 3,408 at eight, on the first 8,180 items both runs finished: ratio 1.32 (paired bootstrap, 10,000 draws; the interval is very narrow because wall time is held fixed, so it covers item sampling only). Accuracy was 83.67% against 83.36%, a gap of -0.31 points (CI -0.51 to -0.10). The curve was 1.12 times at two slots, 1.26 times at four and 1.05 times more from four to eight. My pre-written prediction that four slots would reach at least 2 times was refuted. Median seconds per item rose from 1.36 at one slot to 4.31 at eight, and the slowest 5% from 2.27 to 20.95 seconds. The Coder IQ1_M on Niko1221's Strata engine (Terminal 0.1.54) delivered 0.75 times as many correct items per hour at two slots as at one on 2,612 matched items (accuracy -0.38 points, CI -0.88 to +0.08), with 410 MiB of free RAM at its lowest. At each model's best slot count the 9B delivered 3,328 correct items per hour against the Coder's 1,534, and at one slot the ratio was 1.68 times, as in the earlier study. Published: 2026-11-03 URL: https://porsync.com/research/batching-slots-small-model-1-32x-correct-answers-per-hour-giant-coder-model-slower-16gb ### Reasoning on Two Model and Engine Setups: Coder on Strata Was Faster, While the Thinking 9B Was More Accurate I gave a 9B model with thinking turned on and Niko1221's Strata Coder with thinking off the same queue of reasoning puzzles for six hours each on one 16 GB graphics card. The Coder on Strata delivered about 11.9 times the correct answers per hour compared with the 9B on llama.cpp. Model, engine and thinking mode all changed together. The thinking 9B was 10.4 points more accurate, with the gap coming from constraint puzzles and log diagnosis. There was no difficulty level where thinking paid back its time. Finding: Over a fixed 6 h window each, on an RTX 5060 Ti 16 GB at concurrency 1, on the first 633 items both arms finished, Niko1221's Strata Coder IQ1_M (thinking off) delivered 1,159 correct items per hour against 97.7 for Qwen3.5-9B Q4_K_M with thinking on: Coder/9B 11.85 (95% CI 10.13 to 13.74, paired bootstrap, 10,000 draws). This comparison changes engine and thinking mode as well as model. A later test on short business tasks found the same Coder 6.8 times faster on Strata than stock llama.cpp; it does not isolate the engine effect on these reasoning tasks. The 9B was more accurate, 93.05% against 82.62% (+10.43 points, CI +7.74 to +13.11). The Coder led on correct per hour at every tier of 2, 4, 6 and 8 reasoning steps (15.0 times down to 10.7 times), so there was no crossover. The 9B's median item took 14.9 s against 2.44 s, and 6.95% of its items hit the 16,000-token limit with an empty reply, counted as wrong. Published: 2026-11-02 URL: https://porsync.com/research/thinking-small-model-vs-direct-giant-model-fixed-six-hour-budget-reasoning-tasks ### Engine or Model? The Same Giant Coder Model Delivered 6.8 Times the Correct Answers per Hour on Niko1221's Strata Engine Than on Stock llama.cpp My earlier studies compared a 9B model on stock llama.cpp with a giant Coder model on Niko1221's Strata engine, so every gap mixed the model and the engine together. This one runs the same Coder file on both engines over the same queue of short business tasks. Niko1221's Strata engine delivered about 6.8 times the correct answers per hour, with accuracy within a point. The planned 2x2 became three cells, because the 9B does not load on that engine. Finding: On the first 483 items both runs finished, on an RTX 5060 Ti 16 GB at concurrency 1, the Qwen3.8-Flash-Next Coder IQ1_M delivered 1,452 correct items per hour on Niko1221's Strata engine (Terminal 0.1.54, reused from the 8 h run) against 214 on stock llama.cpp b11753 (2 h run, experts on CPU, no multi-token prediction): ratio 6.78 (95% CI 6.69 to 6.86, paired bootstrap, 10,000 draws). Accuracy was 87.99% against 88.82%, a gap of -0.83 points (CI -2.07 to +0.21), most of it on intent classification (-2.13 points). The 9B file does not load on Niko1221's Strata engine (exit 5, model type not supported), so the 9B engine ratio and the engine-by-model interaction were not measurable. The 9B on stock was ahead of the Coder by 11.3 times on stock but only 1.68 times on Niko1221's Strata engine. Published: 2026-11-02 URL: https://porsync.com/research/same-giant-model-two-engines-niko1221-strata-vs-stock-llama-cpp-correct-items-per-hour ### An Eight-Hour Batch Window on One 16 GB Card: The Small Model Delivered 1.68 Times the Correct Answers per Hour, the Giant One Was More Accurate I gave a 9B model and Niko1221's Strata Coder the same shuffled queue of 30,000 short business tasks (intent classification, invoice extraction, log triage) for eight hours each on one 16 GB graphics card, and counted correct items per hour. The 9B finished 24,805 items and the Coder 14,022. The Coder was 4.2 points more accurate, all of it on extraction, but the 9B delivered about 1.68 times the correct answers per hour. The models used different engines; a later same-Coder test measured a 6.8-fold engine gap. Finding: Over a fixed 8 h window each, on an RTX 5060 Ti 16 GB at concurrency 1, Qwen3.5-9B Q4_K_M delivered 2,592 correct items per hour (83.6% accuracy) against 1,541 (87.9%) for Niko1221's Strata Coder IQ1_M. On the first 14,022 items both finished, Coder/9B correct per hour was 0.596 (95% CI 0.591 to 0.601, paired bootstrap, 10,000 draws), and the Coder's accuracy lead was +4.18 points (CI +3.69 to +4.69). By task, the ratio was 0.258 on intent, 0.906 on extraction and 0.540 on log triage; the accuracy gap came entirely from extraction (+11.7 points). Neither arm had a failed request. Published: 2026-11-01 URL: https://porsync.com/research/eight-hour-batch-window-small-model-vs-giant-moe-correct-items-per-hour ### A Giant Coding Model on a 16 GB Graphics Card: Fast Enough to Chat With, Too Slow to Finish a Coding Benchmark Niko1221's Strata engine is an open-source runtime that squeezes a very large mixture-of-experts model onto one consumer graphics card by keeping the hot parts in video memory and the rest in system memory and on disk. I ran its coding model on a 16 GB card with 32 GB of RAM. It decoded at 42-58 tokens per second, but my coding benchmark was stopped by a rule I wrote beforehand: at 12 finished cases it projected 11.9 hours against an 8-hour budget. There is no benchmark score, and I am not reporting one. Finding: Niko1221's Strata engine (v0.2.8, Terminal 0.1.54) served its Coder model at the IQ1_M quantisation on an RTX 5060 Ti 16 GB with 31.6 GB RAM at 42 tokens per second cold and 56-58 warm (1.38 times), with 15.2 of 16.3 GB of video memory in use and the pagefile active. Aider Polyglot (225 cases, whole-file edits, one thread) was stopped by the pre-set usability cut after 12 finished cases: mean 171.6 s per case, projected 11.9 h against the 8 h budget. 4 of 12 passed at the last attempt and 2 of 12 on the first try; that sample is too small and too biased toward early finishers to count as a score, so the comparison with the 9B reference (27.6%, gate 22.1%) was not made. Published: 2026-10-30 URL: https://porsync.com/research/giant-moe-on-16gb-gpu-32gb-ram-too-slow-for-coding-benchmark ### One Long AI Session or a Fresh Start Each Step? What It Costs on a Single Graphics Card Part 1 of 5. If an AI helps with a long job, you can keep one growing conversation or start clean on every step and re-supply what matters. I timed both over 60 steps. On the 9B model, the long conversation was the cheapest, as long as the server remembers the start of the prompt. On the 27B, running at the memory edge of the card, starting fresh was cheaper. Finding: Over 60 steps of a ~1,500-token note and ~150-token reply on one 16 GB card, Qwen3.5-9B finished in 179 s as one growing session with the prompt cache on, against 190 s starting fresh each step and 337 s starting fresh while re-reading five earlier notes; with the cache off the long session took 1,440 s (8 times slower). On Qwen3.6-27B at the memory edge the order flipped: 3,744 s fresh against 5,159 s for the long session. Published: 2026-10-30 URL: https://porsync.com/research/long-ai-session-vs-fresh-start-each-step-cost ### Does a Small AI Model Stop Following a Rule as the Conversation Gets Long? Not Up to 64,000 Tokens. Part 2 of 5. A common worry is that a model forgets standing instructions as the context grows. I stated one rule once, buried it among filler tasks up to 64,000 tokens, and then tested whether the model followed it. It followed the rule every time at every length, wherever the rule sat. The catch, in part 3, is how the rule is phrased. Finding: Qwen3.5-9B followed a once-stated rule on 42 of 42 probes at every context length from 4K to 64K tokens, quoted it back on 42 of 42, and showed no drop when the rule sat in the middle of the context instead of the start (also 42 of 42 at every length). With no failures in 42 contexts per cell, the true failure rate is bounded at roughly 7%, not zero. Published: 2026-10-30 URL: https://porsync.com/research/small-model-context-cliff-does-it-follow-a-stated-rule-at-64k ### A Small AI Follows a Stated Rule 100% of the Time and the Same Rule Only Implied 24% of the Time Part 3 of 5. In part 2 a rule stated outright was followed every time. Here the same six rules are given only as a reason (for example, the importer rejects commas in amounts) instead of an order. The model followed them about a quarter of the time, at every context length, and a pre-step that asks it to list the standing rules did not close the gap. Finding: Qwen3.5-9B followed a rule stated outright on 100% of probes and the same rule implied by a stated reason on 24% at 4K (26% at 16K and 32K), a gap of 76 points (95% interval 62 to 88) that did not grow with length; a pre-pass in which the 9B listed the standing rules scored 21% at 4K (a change of -2 points, interval includes zero), and a 27B model's list scored 33% (+10 points, interval also includes zero). Published: 2026-10-30 URL: https://porsync.com/research/ai-follows-stated-rules-but-not-implied-ones ### Paste the Whole Folder or Retrieve the Relevant Notes? On a Small Model, Pasting Won Up to 120 Notes Part 4 of 5. The standard advice for working over a pile of notes is retrieval: find the few relevant ones and show only those. I tested that against simply pasting the whole folder, from 10 to 120 notes (about 46,000 tokens). Pasting was more accurate at every size. Retrieval tied it on single-note questions and was 2 to 20 times faster. The entire gap was questions that need two notes. Finding: On 60 questions per size over a synthetic project-notes folder (10, 30, 60 and 120 notes), Qwen3.5-9B with the whole folder pasted scored 97%, 95%, 83% and 77%, against 67% for the best retrieval arm at every size from 30 up (70-77% at 10 notes), with retrieval at 100% on single-note and distractor questions and 0% on multi-hop questions except at 10 notes, where top-8 scored 10% and hybrid scored 30%; pasting was 2.0 to 20.1 times slower per question (1.5 s to 18.4 s against 0.7 to 0.9 s). Published: 2026-10-30 URL: https://porsync.com/research/paste-the-whole-folder-or-retrieve-the-notes-small-model ### What Does a Small AI Keep When It Summarises Its Own Work? Mostly the Dates. Part 5 of 5. When a conversation gets long, the standard fix is to have the model compress it. I planted six kinds of detail across 20 steps of work log, had the 9B compress its own context three ways, then asked for each detail back. Uncompressed it got 88%. A free summary got 3%, a structured template 20% and a copy-exactly template 33%. A files-plus-tool approach got 12%, because the model almost never opened a file. Finding: Across 40 generated streams of 20 work-log steps (about 16K tokens), 6 planted details each and 1,440 probes, Qwen3.5-9B passed 88% with no compression, 3% after a free-form summary, 20% after a structured template, 33% after a copy-verbatim template, 12% with a file list and a read_file tool, and 27% with the file list plus a template; only the deadline date survived a template reliably (85 to 90%), and the model opened a file on 8.8% of probes. Published: 2026-10-30 URL: https://porsync.com/research/what-a-small-ai-keeps-when-it-summarises-its-own-context ### Run the Same AI Prompt Twice: How Much of a "Change" Is Just Noise? Part 1 of 6. Before you can say an edit to a prompt changed the answers, you need to know how much they change when you edit nothing. I measured that on 500 support tickets. Edits that look cosmetic, like reordering the options, changed more answers than a deliberate improvement did, and the improvement made the model slightly less accurate. Finding: On 500 support tickets with Qwen3.5-9B, re-running an identical prompt changed 0 of 500 queue answers when one request ran at a time and 21 of 500 (4.2%) when four ran together; reversing the option order changed 111, adding a one-line definition per option changed 135 and lowered queue accuracy by about 4 points, and a whitespace-only edit changed 38. Published: 2026-10-29 URL: https://porsync.com/research/same-prompt-twice-how-much-changes-small-model ### One Polite Sentence Redirected a Small AI Classifier on 71% of Tested Tickets Part 2 of 6. If a model sorts text that strangers write, anyone can append a sentence and steer it. I tested six such sentences on 500 tickets and four defences. A polite sentence asking for a different team worked on 71% of tickets. Telling the model its output must be a valid label did nothing, and wrapping the ticket in markers helped on exactly one kind of attack. Finding: Appending one sentence to a support ticket moved a Qwen3.5-9B queue answer to the sentence's target on 71.2% of 500 tickets and the priority answer to "high" on 64-69% (baseline accuracy 32% and 40%); constraining the output to valid labels left the rate at 67.3% against 67.2%, and delimiters plus a data note cut one attack from 33.0% to 6.4% but did not move another (71.2% to 72.4%). Published: 2026-10-29 URL: https://porsync.com/research/one-appended-sentence-redirects-small-model-ticket-sorting ### When the Answer Isn't in the Contract, a Small AI Makes One Up 6% of the Time Part 3 of 6. Pulling dates and terms out of contracts is a common AI job, and most of the time the field is simply not in the excerpt. The baseline already instructed the model to use null when a field was absent; a JSON schema did not materially change the result, and a stricter rule that required a quoted source cost accuracy without reducing the invention. Finding: On 315 contract-field instances where the answer was absent from the excerpt, Qwen3.5-9B produced a value 6.3% of the time (11% for governing law); the same prompt with a JSON schema gave 6.7%, requiring a quoted source or null gave 6.0% while cutting recall on present fields by 6.7 points, and a literal substring check on the quote caught 0 of 19 inventions. Published: 2026-10-29 URL: https://porsync.com/research/ai-invents-values-for-missing-contract-fields ### The Check That Fails Without Telling You: A Required Phrase Missed 9% of the Time Part 4 of 6. A common way to wire an AI into code is to ask it to start its reason with a fixed phrase, then test for that phrase. It looks reliable and it is not: the model sometimes just doesn't write it, and your code concludes nothing happened. A separate yes/no field did not have the problem. Finding: When Qwen3.5-9B itself said an outage was reported, the code check that the reason starts with the required phrase silently failed on 5 of 55 cases (9.1%, 95% interval 1.8 to 18.2) at temperature 0, while a separate boolean field failed on 0 of 52; a tolerant matcher recovered none of the 5. Published: 2026-10-29 URL: https://porsync.com/research/the-phrase-check-that-silently-stops-matching ### Put the Answer Before the Explanation: 1.8 Seconds Down to 0.56, and What It Costs Part 5 of 6. If the model writes its explanation first, nothing is usable until it finishes. If the decision comes first, a screen or a downstream step can start right away. I tested both orders, with and without a cap on the explanation, on 500 tickets. The order roughly tripled how fast the answer appeared but barely changed total time, and the cap saved less than I predicted. Finding: With Qwen3.5-9B returning three decisions plus notes as JSON on 500 tickets, putting the decisions before the notes made the answer usable in a median 0.56 s instead of 1.80 s while total response time was 1.53 s against 1.85 s; capping the notes cut output by 1.6x rather than the predicted 2x, and cost 5.2 points of queue accuracy when the notes came first. Published: 2026-10-29 URL: https://porsync.com/research/answer-first-schema-faster-first-useful-token ### Does an AI Grader That Can See the Answer Just Agree With It? Not Quite. The Explanation Did. Part 6 of 6. A second AI checking the first one's work is a common safeguard. The worry is that a grader shown the proposed answer simply agrees with it. I tested two small graders on 734 answers, half of them wrong. Showing the answer did not make them agree more. Showing the first AI's reason did, and both graders were weak either way. Finding: Across 734 proposed answers (367 wrong), showing Gemma-4-E4B the proposed label did not change its catch rate of wrong answers (35.4% shown against 36.5% blind, gap +1.1 points [-4.4, +6.3]), but adding the classifier's one-line reason cut catch by 12.0 points and a stated 95% confidence by 6.0; the Qwen3.5-9B grader's blind arm simply reproduced its own labels (100% agreement), so it could not test anchoring. Published: 2026-10-29 URL: https://porsync.com/research/does-an-ai-grader-that-sees-the-answer-agree-with-it ### Seven Small Coding AIs, Four Benchmarks, One 16 GB Card: None Cleared the Bar My first coding test was too easy to rank anything, so I ran the same models up a ladder of public benchmarks, hardest last, with the pass marks and a time limit written down first. The best model solved 9 of 50 real open-source bug reports. Five of seven were stopped before the end, not for being wrong but for being too slow to wait for. The result is a floor, and the reasons each model stopped matter more than the scores. Finding: On one 16 GB RTX 5060 Ti, among seven local coding models that all passed a tool-calling gate (valid-call rate 0.960 to 1.000 on 400 BFCL items), none cleared a pre-registered ladder: the 9B reference was the only model to finish all four rungs inside an 8-hour generation budget (LiveCodeBench 0.406; Aider Polyglot 27.6%), and it resolved 9 of 50 SWE-bench Lite instances through OpenCode (18.0%, below its own 22.1% gate; the 95% interval at n=50 is roughly 10% to 31%); the only other model to finish rung 3, Qwen3-Coder-30B, scored 18.2% there, under the gate; five models were stopped on the time rule and carry no accuracy score at the rung where they stopped; one run (Gemma 4 12B, time rule waived) resolved 0 of 50. Single runs, one quantisation per model, public benchmarks with unknown training exposure. Published: 2026-10-28 URL: https://porsync.com/research/no-small-coding-model-cleared-the-bar-benchmark-ladder-16gb ### Same Test, Run Seven Times: How Much of a Local Coding AI's Result Is Luck? Readers pushed back on my coding-model test: three runs is too few, and where did the time actually go? So I ran the same ten jobs seven times on two models, then 17 new jobs, then a runaway case a third time. The 20-second speed gap I had reported turned out to be inside the run-to-run spread. The surprise was not speed: the faster model failed on writing new code, which my first ten jobs never tested. Finding: Across 7 repeats of the same 10 bug-fixing tasks on one 16 GB card, gpt-oss-20b took a median 151 s (range 144 to 244 s) and Qwen3.5-9B 190 s (160 to 208 s), so the 20 s gap reported earlier sits inside the run-to-run spread; both made 62 tool calls (median), and Qwen's median step was 1.9 s against 0.7 s for gpt-oss; gpt-oss failed 2 of 70 task-runs (a wrong Roman-numeral case and a deleted module), so "10 of 10 every time" did not hold at 7 repeats; on 17 new tasks (12 write-new-code, 5 fix) over 3 runs, Qwen passed 51 of 51 and gpt-oss 45 of 51, with every gpt-oss failure on write-new-code; and the 390-call runaway seen once with Qwen3-Coder-30B did not repeat in 10 further runs (max 8 calls). Small samples, tasks written by me. Published: 2026-10-28 URL: https://porsync.com/research/same-test-twice-run-to-run-noise-small-coding-models ### Which Small Coding AI Models Are Worth Running on a 16 GB Graphics Card? I put eight coding models on one 16 GB graphics card and gave them ten small bug fixes through the same coding assistant. Six fixed every job. The clearest differences were in runtime and tool use. One model wrote out its tool calls as text, so the assistant could never run them. I used these results to pick candidates for the harder benchmark that followed. Finding: On a 16 GB RTX 5060 Ti, six of eight models fixed all ten small Python jobs through OpenCode. Mellum 2 fixed seven and timed out on three. Qwen2.5-Coder 14B fixed zero after producing no executable tool calls. Three identical Bolt runs all passed, but wall time ranged from 234 to 338 seconds and tool errors from 2 to 24. The other timings are single runs. Published: 2026-10-25 URL: https://porsync.com/research/which-small-coding-ai-models-run-on-a-16gb-card ### Write Down Your Bet Before You Test the AI Most AI results are explained after the fact: run it, look at the numbers, then decide what they mean. I do it the other way round. Before a single test runs, I write down the question, the score that counts as a pass and how much wobble I'll treat as noise, then lock the page so it can't be quietly edited. This is why that habit matters for anyone buying or building with AI, with a live example where I had to change my method halfway and wrote the change down instead of hiding it. Finding: Fixing the question, the pass mark and the noise level in writing before any result exists makes an AI test hard to bend after the fact; in a small-model study the noise floor was set at 0.017 on a 0-1 score before scoring began, and a mid-study change of method was logged as a dated amendment before the affected results were read. Published: 2026-10-22 URL: https://porsync.com/research/write-down-your-bet-before-you-test-ai ### Can the Apprentice Replace the Master? A Small AI Learning From a Big One Big commercial AI models are good and cost money every time you use them. Small ones run on your own computer for the price of electricity. The tempting idea is to let the small one learn from the big one's answers and then retire the big one. I tested that on a real task, sorting customer banking questions into 77 categories, and the honest result is: the apprentice gets a lot better, but on this task it does not yet match the master. What it learns depends heavily on how good the master is. Finding: On a 77-category banking-question task, a small 4-billion-parameter model trained on the strongest teacher's answers scored 0.838 on a 0-1 scale against the teacher's 0.936 (below it, outside the pre-set noise bar of 0.017), while training on a better teacher's answers instead of a weak one's lifted the same small model by 0.14 (0.697 to 0.838), and even when the small model answered only the 80% of questions it was surest of it stayed 0.034 behind the teacher on those same questions (0.926 vs 0.960); validation data only, the held-out test set was never opened. Published: 2026-10-22 URL: https://porsync.com/research/can-the-small-ai-replace-the-big-one-apprentice ### I Broke an AI on Purpose. The Safety Check Never Said "Worse" Before you swap a working AI for a cheaper one, you want a check that says whether the new one is safe to ship. I tested that check the way you test a smoke alarm: I damaged the new model on purpose and waited for the alarm. It never went off. The damage was real and the check noticed it, but my own rule for what counts as an alarm was stricter than the damage I had caused. This is what that means for anyone relying on a "no worse than before" check, and what I did and did not conclude. Finding: A paired non-inferiority check on 965 banking questions downgraded its verdict as a small model was deliberately damaged, from "no worse than 3 points" at 20% scrambled training answers (accuracy 0.851 to 0.834) to "no worse than 5 points only" at 40% (to 0.821), but never returned "worse", because that verdict needs proof of a loss above 5 points; the registered test of the check therefore failed at both damage levels, and the plan's rule to stop before opening the held-out test set applied. Published: 2026-10-22 URL: https://porsync.com/research/i-broke-an-ai-on-purpose-the-safety-check-never-said-worse ### When Should a Task Run on a Local Model? Seven Rules With the Evidence Behind Each I ran five pre-registered experiments on small AI models over two months, most of them ending in a trade-off or a "no", plus one framework study. Read one at a time they look scattered; together they answer the question people actually ask: should this task run on my own hardware? This is the rulebook that came out of them — seven rules, each with the study that measured it, the conditions it was measured under, and where it stops applying. Six are supported by measurements, one is a method whose thresholds are conventions. A small command-line tool asks six questions and prints the rules that bear on your case, and every output says "evidence from 3 tasks on one GPU". Finding: A pre-registered decision guide for "should this run local?" can be written as seven rules (R1-R7) whose 89 published numbers are each bound by file hash to the write-up they were published in and to the raw aggregates they were computed from: check fit by measured peak memory rather than file size (R1, supported); test quantisation retention before training, >= 0.85 acceptable and < 0.70 collapse (R2, a convention rather than a measurement); serving setup is a speed lever, not an accuracy lever, and 4-bit flipped 8% of individual answers (R3, supported); local marginal cost is $0.0032 against $22.43 per 1,000 tickets in electricity (R4, supported); fine-tuning trades recall for precision at an unchanged overall score, 0.697 vs 0.698 [-0.043, +0.061] (R5, supported); the teacher's labels set the ceiling, +10 points then +2.7 not significant after the rubric fix (R6, supported); and small-model confidence does not route whole documents, with 1 of 28 setup-and-signal pairs passing the registered test at every escalation level in the second cascade study (R7, supported). Evidence covers 3 task families on one 16 GB GPU. Published: 2026-10-18 URL: https://porsync.com/research/when-should-this-run-local-rulebook ### Squeeze a Big AI Model Into Less Memory, or Run a Smaller One? A Pre-Registered Test on a 16 GB Card When your graphics card is the limit, you can run a small model at full quality or a much bigger one compressed to fit. I tested eight setups on 102 real contracts, with the rules written down first. The compressed big models won clearly, and the most compressed of them (13.5 GB) did as well as the least compressed (16 GB). The best of them landed within noise of a frontier model with no escalation at all. Deciding which contracts to hand up to the frontier model mostly still did not work: of 28 setup-and-signal pairs, 1 passed, 7 were flaky and 20 failed. Finding: On CUAD contract clause extraction (102 test contracts, 41 clause types), Qwen3.6-27B compressed to 2.7, 3.1 and 4.3 bits per weight scored 0.757, 0.769 and 0.762 macro-F1, statistically indistinguishable from each other and from the frontier model (Sonnet 5.5, 0.783; the paired differences from it were -0.026, -0.014 and -0.021, every 95% interval including zero), against 0.705 for a 4B small model. A 9B model at normal precision (0.703) and a 27B model squeezed to about 1 bit per weight (0.679) did not beat the 4B model. Across 28 pairs of setup and escalation signal, one passed the pre-registered routing test at every escalation level (contract length on the ternary 27B model), 7 passed at some levels and 20 at none; no confidence signal ever passed. Published: 2026-10-18 URL: https://porsync.com/research/big-model-squeezed-into-less-memory-local-cascade ### Can a Small AI Tell You When to Call the Big One? A Pre-Registered Test That Said No A popular cost-saving idea: let a cheap small AI handle everything, and escalate to an expensive frontier model only when the small one seems unsure. I tested it on 102 real contracts, with the rules written down before any results existed. No confidence signal reliably beat escalating at random. Even where escalating individual answers did help, it saved no money, because the frontier model charges for reading the contract, not for answering. Finding: On CUAD contract clause extraction (102 test contracts, 41 clause types), a 4B-parameter small model's own confidence did not reliably choose which contracts to escalate to a frontier model (Sonnet 5.5): no signal beat random escalation with a 95% confidence interval excluding zero at 20%, 30% and 40% escalation. Choosing individual answers instead of whole contracts did locate the frontier model's wins (+0.052 macro-F1 over random at 10% of answers), but saved no cost: the frontier bill is driven by reading the document (about $5 per million input tokens, roughly zero for output), so escalating 10% of answers touched 93 of 102 contracts and 95% of the always-frontier cost. Published: 2026-10-12 URL: https://porsync.com/research/small-model-confidence-cannot-route-to-frontier-cascade ### Running AI on Your Own Computer: What Makes It Faster, and What Quietly Changes the Answers I tested what happens when a small AI model sorts 500 customer-support tickets on one ordinary graphics card, changing one setup choice at a time. The best setup was 2.2x faster than the default and cost a fraction of a cent per thousand tickets, against about $22 per thousand through a paid AI service. Overall accuracy barely moved. But individual answers did: the faster, more compressed setups gave a different answer on about 1 ticket in 12. Finding: On one consumer graphics card (16 GB), sorting 500 support tickets with the open Qwen3.5-9B model, the best setup (llama-server, 4-bit compression, 4 tickets processed at once) ran 2.2x faster than the Ollama default (178 vs 80 tickets per minute) and cost $0.0032 per 1,000 tickets in electricity, against $22.43 per 1,000 for Claude on the same task. Accuracy did not measurably change, but the 4-bit model gave a different queue answer than the higher-quality 8-bit one on 8% of tickets. Published: 2026-10-06 URL: https://porsync.com/research/serving-choices-speed-not-accuracy-local-llm ### Teaching a Small AI to Read Contracts: It Got More Careful, Not More Thorough I tried to train a small AI model, small enough to run on one ordinary computer, to find 41 kinds of clause in real contracts and quote them word for word. Compared fairly, with both versions reading the contract the same way, training made it far more careful (fewer false claims, almost no invented text) and slightly less thorough, and left the overall score unchanged. The bar I set in advance was not cleared, so the main comparison against a larger model never ran. Finding: On 41-category contract-clause extraction, fine-tuning (LoRA) a roughly 4-billion-parameter model that reads contracts in 1,000-token chunks raised precision (the share of flagged clauses that were really there) from 0.78 to 0.93 and verbatim-quote fidelity from 87% to 97%, lowered recall from 0.87 to 0.82, and left the overall score (macro-F1) unchanged: 0.697 against 0.698 for the untuned model reading the same chunks (validation, 95% interval -0.043 to +0.061). No tuned version cleared the pre-registered bar, which was set against the untuned model reading the whole contract at once (0.727). Published: 2026-10-03 URL: https://porsync.com/research/tuning-a-small-model-precision-not-recall-cuad ### Copying a Bigger AI's Judgement Into a Smaller One: The Teacher Sets the Ceiling An experiment replacing a mid-size AI classifier with a small trained model. The small model's size, the amount of training data and the output format made no measurable difference. Changing who wrote the training answers made the largest one. And without real outcomes to check against, every score turned out to measure agreement with a judge, not correctness. Finding: When distilling an LLM classifier (training a small model on a larger model's labels), the label source sets the accuracy ceiling: a small student model matches its teacher regardless of size, data volume or output constraints, while retraining the identical recipe on a stronger teacher's labels raised balanced accuracy on the accept/reject decision by 10 points over the original student and 7.6 points over the teacher it replaces (both significant on 200 held-out items). After a later rubric fix the gain over the teacher shrank to 2.7 points and was no longer significant. Without outcome data, every gain is agreement with a better-specified judge, not proof of better decisions. Published: 2026-09-28 URL: https://porsync.com/research/the-teacher-sets-the-ceiling-slm-distillation ### Before You Build: A Way to Test Whether a Small AI Can Do the Job on Your Device Most projects that put AI on a phone, kiosk or other small device start by training a model and end by discovering it never fit the device. This is the reverse order: a measurement-first method that answers "can a small AI model do this job on this hardware at all?" before any training, using a simple pass/fail gate based on how much quality survives compression and whether it fits the device. Finding: Feasibility for running a small language model on an edge device can be decided without training anything: benchmark candidate models at several compression levels (quantisation) and treat retention (score at that compression ÷ score at 8-bit) as the gate. Below 0.85 is degradation, below 0.70 is collapse. If no candidate clears the gate inside the device's memory and speed limits, fine-tuning cannot save the project and should not be started. Published: 2026-07-05 URL: https://porsync.com/research/edge-slm-feasibility-framework --- ## Practical AI (practical guides adapted from research, not additional experiments) ### What Can AI Running on Your Own Computer Do for a Small Business? URL: https://porsync.com/ai-for-business/what-local-ai-can-do Author: Bart Porcincula Published: 2026-11-08 Updated: 2026-11-10 Local AI is worth testing on narrow jobs such as sorting enquiries, preparing document fields, and processing a reviewable backlog. The evidence here comes from controlled tasks on one computer, not measured savings in customer businesses. ### Can AI Work Through Your Paperwork Overnight? URL: https://porsync.com/ai-for-business/overnight-paperwork Author: Bart Porcincula Published: 2026-11-08 Updated: 2026-11-10 An overnight local-AI queue can prepare work for review, but the right model depends on the document mix. My synthetic batch test measured correct items per hour; it did not measure a complete paperwork workflow or staff time saved. ### Can You Trust AI to Pull Details From Documents? URL: https://porsync.com/ai-for-business/document-details Author: Bart Porcincula Published: 2026-11-08 Updated: 2026-11-10 AI-extracted document fields need source checking. In my missing-field contract test, the small model supplied a value in 6.3% of cases where the answer was absent from the excerpt; this is not an error rate for all document answers. ### Can an AI Assistant Use Your Company's Procedures? URL: https://porsync.com/ai-for-business/company-procedures Author: Bart Porcincula Published: 2026-11-08 Updated: 2026-11-10 A company assistant needs direct instructions, enough relevant source material, and reliable records of important details. Separate experiments on one small model show why supplying documents alone does not establish reliability. ### Is Running Your Own AI Really Cheaper? URL: https://porsync.com/ai-for-business/local-ai-cost Author: Bart Porcincula Published: 2026-11-08 Updated: 2026-11-10 Local AI can have a very small processing electricity cost, but the business comparison must include hardware, setup, maintenance, and review. The prices quoted below describe an October 2026 experiment, not current provider quotes. ### How to Test AI Before Spending Money on a Bigger System URL: https://porsync.com/ai-for-business/test-ai-before-spending Author: Bart Porcincula Published: 2026-11-08 Updated: 2026-11-10 Test one business job on representative examples, decide what success means in advance, and measure the time to an accepted result. Suggested trial sizes and acceptance rules below are examples, not thresholds established by the research. --- ## Projects (technical write-ups) ### Finding and Fixing Silent Data Loss in a Live Lead System A system that collects about 6,400 businesses twice a day, opened with a formal check of its own data quality. It found two serious problems that never raised an error: one quietly merged 545 different businesses into others, and another threw away 57.8% of one collector's results. I then rebuilt it with 197+ automated checks so the same kind of bug can't hide again. Published: 2026-09-07 URL: https://porsync.com/projects/silent-data-loss-lead-pipeline ### Getting Found and Used by AI Assistants: A Test on Porsync.com A May 2026 snapshot: porsync.com was tested against two companies that sell tools for exactly this. The same AI commerce assistant was run once on each of the three sites. It succeeded on porsync.com and failed on the other two. The site has changed since, and a rerun is pending. Published: 2026-05-28 URL: https://porsync.com/projects/porsync-ai-visibility ### Updating a Site's WebMCP Setup for the Current Browser API: What Changed and What I Changed WebMCP, the browser API that lets a page offer tools to AI agents, has moved since porsync.com first used it in May. I read Chrome's current documentation and the early adopters' write-ups, then updated this site: tools now register with a clean-up signal, carry read-only labels, and a second tool looks up a single research write-up. Tested against a production build with a stand-in for the browser API, not yet in Chrome with the real one. Published: 2026-11-02 URL: https://porsync.com/projects/webmcp-update-registertool-lifecycle --- ## Tools (read-only, WebMCP) The following tools are registered on https://porsync.com and can be called by AI agents via WebMCP. Catalogue: https://porsync.com/tools ### list_projects Input: {} Output: { projects: [{ slug, title, summary, tags, datePublished, page_url }], research: [{ slug, title, summary, datePublished, page_url }] } Endpoint: GET https://porsync.com/api/tools/list-projects Description: Read-only index of the projects and research write-ups. ### get_research Input: { slug: string } Output: { slug, title, summary, finding, tags, datePublished, page_url } Endpoint: GET https://porsync.com/api/tools/get-research/{slug} Description: Read-only lookup of one research write-up by slug (slugs come from list_projects). --- ## Contact Email: connect@porsync.com