# Lab detail — Trained by whom (you can't price the task without pricing the fine-tune) Raw lab detail for labs.layer2c.com/labs/trained-by-whom. A one-week campaign (2026-08-17 to 2026-08-23) on two owned NVIDIA DGX Sparks, one rented AMD MI300X, and four hosted model APIs, testing whether an owner-side fine-tune improves a per-task agentic code-repair process over the base model, with cost per gate-verified repair as the denominator. **What ships:** the decision procedure, the measured results, the cost model, the corrections, the failure-mode catalog, and the open-source instrument (loopcontrolbench, MIT). **What stays proprietary:** the specific 22-task set and its extracted tests, the teacher trajectory corpus, the tuning particulars, and any vendor-assessment consequences. Returns, not the algorithm. ## The question The corpus held two fine-tuning claims that had never met. An earlier lab measured an owner's LoRA moving accepted output from 23% to 70% on a bounded single-shot task and ruled: own the weights. A later lab measured the per-task agentic regime and found the separator is training, not size, and that the training was the vendor's. This lab ran the missing arm: does an owner's fine-tune improve a gate-verified agentic repair process? The measurement kept stalling on a cost the original framing had minimized, the cost of producing the model at all, and the deliverable became a decision procedure for when fine-tuning, and therefore self-hosting, is the right lever. ## Setup The eval instrument is a fixed pool of 22 localized repair tasks, each with an executable test, scored by a deterministic reproduce-or-drop gate unchanged across five prior labs. New apparatus: an 82-task disjoint training pool mined from a dozen fresh repositories, a teacher trajectory corpus, a synthetic precision-skill factory, and a self-generated harvest of gate-passed loop solutions. The tuning subject was one open model family. Comparators were tool-trained open peers on identical serving, a mini-class hosted model as the frontier reference, and an open frontier-scale model served on owned clustered hardware. Two agentic surfaces were tested. The **harness** drives the model through a tool channel with exact-match edits, pricing edit precision under long-context load. The **loop** regenerates whole-artifact edits judged by the gate, exercising diagnosis and generation in clean context every turn. ## Corrections (published before the conclusions that depend on them) **The harness "wall" was substantially a serving bug.** The first-phase harness numbers, a base edit-land rate near 37% against tool-trained peers near 96%, were measured with a tool-call parser that silently dropped the model's native emission format, scoring correctly-shaped tool calls as plain text. Served with the correct parser, the untuned base scored 15 of 22 where the wrong parser had it at six to eleven. A clean cross-model re-measurement was not finished before the rented card came down, so the execution-deficit reading is reopened, not overturned. The serving contract is part of the model. **The consolidation tune ran; an earlier draft said it would not.** The best-designed owner tune, a self-consolidation adapter trained on the model's own gate-passed loop solutions, was built and evaluated: 14 of 22 in the loop against a fair untimed loop base of 12. On 22 tasks that two-task delta sits inside sampling noise. **A harvest workspace bug inflated an early miss rate.** The solution harvester built prompts before resetting a workspace shared with harness runs, so on nodes that had run the harness the model was shown pre-edited files and its edits failed to apply. Corrected, the base first-shot keep rate on the training pool was roughly 48%, not the ~12% first observed. ## Measured results - **Owner fine-tunes, gate-verified gain.** Three reachable families (imitation from a teacher, isolated-skill precision) produced zero gate-verified improvement; the imitation tune was actively destructive, transferring the teacher's brevity without its precision. The self-consolidation adapter nudged the loop count by two tasks, inside noise. - **The frontier reference.** A mini-class hosted model solved all 22 tasks for about $3.14 at a ten-turn median. - **Apparatus fit.** The constrained loop clears the solvable set deterministically at a fraction of the harness token cost. The loop is the matched instrument for an untooled open checkpoint, not the lesser rung. - **The exception.** An open frontier-scale model, served on two clustered owned Sparks, swept the pool twice at a seven-minute median for electricity and hours, no fine-tune. Self-hosting succeeded where fine-tuning did not, on off-the-shelf open weights. - **Serving variance.** Back-to-back same-config hosted runs scored 6 and 11; cache warmth between replicates is the standing suspect. Single-run cell numbers from shared-cache venues are weather. ## The cost model (the part the pitch omits) Cost per task is unknowable until the fine-tune is priced, and the fine-tune is capital that amortizes, not an implementation detail. Fine-tuning is therefore volume economics: a fixed cost divided across every task it solves, catastrophic over one task, invisible over a million of one shape. The search itself was priced, and the unit matters more than the number. Metered compute was about $73 across roughly 90 owned GPU-hours and six days. The money measures electricity; the six days measure when the answer arrived. Compute is the floor, not the bill, and the bill is not attended labor either: nobody watches a progress bar, the runs go unattended, but the run that does not finish tonight is the answer that is not there tomorrow, which on a dependent question tree is the two or three experiments that cannot start. Slow hardware does not burn attended hours; it occupies the critical path. The instrument built to measure cost per solved task showed that cost per solved task was never the constraint. **Time to solved task was.** Price four things, not one: cost per solved task (what the answer cost), time to answer (how long before it exists), research throughput (how many questions close in a month), and capacity liquidity (whether you can get the compute at all). Faster rented hardware earns its price by removing delay, not by cutting compute cost. Serving cost is the last branch, below yield, apparatus fit, what the vendor trained, and whether training was needed at all. ## Key findings — the decision procedure Self-hosting and fine-tuning are different decisions the pitch fuses. **Compliance is an override, not a condition:** where data cannot leave the VPC or the workload is regulated or air-gapped, self-hosting is mandatory regardless of capability. Off-the-shelf open weights can win self-hosting with no tune. Fine-tuning is the narrowest door, and it splits into capability tuning and cost tuning (distillation), which do not share a rule. Capability fine-tuning pays only when five conditions hold together: 1. The foundation model fails your **gate**, not just your **budget**. Acceptable output at any price is a cost problem, not a capability problem. 2. The missing capability is **behavior, not knowledge**. Retrieval injects what a model knows; fine-tuning changes how it acts. If a corpus plus a frontier model clears the gate, it was a retrieval problem in a fine-tune costume. 3. A **deterministic gate** exists. No gate, no cost-per-verified-task, no economics. Most domains lack this. 4. **Value per task is high and volume is large** enough to amortize a fixed cost. 5. You have **priced the whole how**, including the deterministic enforcement the tune does not remove. A fine-tune moves behavior most of the way; a hard invariant still leaks the base prior, so a deterministic layer enforces the last mile, and that enforcement is a recurring cost that never amortizes. Owning the model does not retire the validator; it shifts work onto it. Code repair meets conditions 3 and 4 and fails condition 1: the frontier already clears the gate cheaply. Most enterprise domains that look like fine-tuning candidates are retrieval problems. ## Failure-mode catalog (operator error, now encoded as gates) Roughly a quarter of the campaign's spend was avoidable operator error, each fix now a mechanical gate in the instrument: - A proxy routing to the wrong API surface, and a served context cap below the reference envelope, each silently invalidating an arm. Fix: a venue preflight (envelope check, arm-scale prefill probe, end-to-end tool smoke, provenance stamp) that refuses a bad launch. - A tool-call parser mismatch that buried a working model. Fix: the parser is part of the serving contract and is verified before scoring. - A model alias that resolved to the base weights instead of the adapter. Fix: routing is verified by observing the adapter compute path, not by a provenance stamp that cannot tell them apart. - Two host reboots from pushing unified memory past its wall during training. Fix: chunk every long forward, and canary any new sequence length on one node before committing the pair. ## What did NOT get settled (honesty) - It did not prove owner fine-tuning never pays. It proved three reachable families bought nothing and the best fourth nudged the count within noise, on one domain that fails the decision test. The bounded single-shot regime where an earlier lab measured a 3x owner gain stands untouched. - The five-condition procedure is validated against a domain that fails it, not one that passes. Its predictive value on a passing domain awaits the next lab, and the hard part there is building the gate. - The parser correction reopens the phase-one harness numbers rather than cleanly replacing them. - The consolidation tune's harness arm was torn down before completion as immaterial to the conclusion. - The distillation branch and the compliance override are named and reasoned, not measured here.