Show HN: Freeze the Model, Train the Harness

The Show HN post Freeze the Model, Train the Harness frames a clear inversion of the usual fine-tuning stack: leave the LLM weights frozen and improve the system by training the surrounding harness. That harness is everything outside the model that shapes behavior, including prompts, context management, tools, and a repair loop. The claim is not a new model release. It is a training regime for the control plane around a fixed model.

Technically the harness is one editable Python file. Training runs in epochs. In each epoch an agent proposes a single diff to that file. The change is measured against a baseline on a task panel, and a strict Pareto criterion decides whether the commit is promoted. The optimizer step is not a gradient update. optimizer.step() is a git fast-forward when the candidate wins, or the unpromoted commit is kept as a git ref when it loses. Selection pressure is explicit, versioned, and reversible through ordinary git history.

For engineers building agent systems this matters because most production pain sits outside the base model. Tool schemas break, context windows fill with noise, recovery after tool failure is brittle, and prompt drift is hard to measure. Freezing the model and scoring only harness diffs turns those soft engineering choices into an optimization problem with a measurable baseline and a promotion gate. The unit of progress is a reviewed code change that either improves the task panel under Pareto rules or does not ship.

Market and competitive context sits between model training and hand-written agent frameworks. Fine-tuning and RLHF move weights and demand data, compute, and model access. Prompt and tool stacks are usually edited by humans without a closed evaluation loop. This approach keeps the model fixed and treats the harness as the trainable artifact, with git as the optimizer ledger. It competes less with a new foundation model and more with ad hoc agent scaffolding that never gets a strict accept or reject signal per change.

Evaluation results reported in the post include transfer learning across harness work, which is the practical signal that the loop is not only overfitting a single task panel. What to watch next is whether Pareto-gated harness commits keep transferring when the task panel, tool set, or frozen model changes, and whether a one-file harness remains enough as tools and repair policy grow. The near-term test for builders is simple: instrument a baseline, allow one harness diff per epoch, and only promote commits that strictly improve the panel under the same criterion.