Harness adapters: Safe RL for efficient agentic development

Hugging Face researchers Younes Belkada, Lysandre Debut, and Nathan Habib released harness adapters, a system of specialized LoRA adapters trained with reinforcement learning directly inside unmodified coding agent environments including pi, opencode, and hermes. According to huggingface.co's report, the system trains one low-rank adapter per model and harness on verifiable programming tasks while a proxy filter called safeagents monitors every tool invocation to prevent reward hacking. The project builds on earlier trials with LiquidAI's LFM2.5-24B-A2B under pi, hermes, and claude-code, expanding the architecture into a multi-model setup where a single vLLM server concurrently serves the base weights alongside separate harness-specific adapters.

This release covers the complete training pipeline, evaluation architecture, and safety mitigations developed for local agentic programming workflows. The technical design targets developers and machine learning engineers running open-source code agents who want base language models to use fewer tool calls without losing accuracy across differing execution frameworks. By treating the operational harness as a specialized target domain rather than retraining complete model weights, the team provides small adapter files that local agent operators can drop directly on top of base foundations without modifying existing harnesses or client software.

Harness adapters: what actually changed

The authors shifted from standard multi-harness reinforcement learning to training distinct rank-32 LoRA adapters attached to all linear layers of a base model for each agent harness. Rather than updating base model parameters across diverse harness tool signatures, prompts, and control flows, the pipeline hosts one base model instance on vLLM while loading multiple adapters side by side. The team broadened the training scope beyond the initial LFM2.5-24B-A2B mixture-of-experts model to include LiquidAI's LFM3-1.7B hybrid model and webAI-Official's TwIL-8B dense reasoning model, an 8.8 billion parameter derivative of Granite 4.2 that had never received prior agentic training.

To create diverse training data, the team built synthetic reinforcement learning environments from public agent trajectories, defining four task classes: repair, question answering, build, and extend. Build tasks evaluate new code implementations against hidden test suites validated by an oracle implementation. The training curriculum mixes tasks from SmolDataEnvs, weighted toward medium and hard difficulties, with 230 tasks drawn from Repo2RLEnvs. An additional 40 Repo2RLEnvs tasks and 150 SmolDataEnvs tasks remain strictly held out to evaluate generalization on tasks the models never observed during training.

Harness adapters: how it works

Training relies on the GRPO algorithm using TRL's AsyncGRPOTrainer paired with tree attention, running at a learning rate of 1e-5 with a sampling temperature of 0.8 and up to 32,768 output tokens per turn. Each model executes eight rollouts per task, scored against the environment verifier alongside an efficiency reward term calculated as 0.1 multiplied by one minus the tool call count divided by 15 for successful completions. To isolate execution, harnesses run unmodified in sandboxes managed via OpenEnv's Harbor integration, while a proxy layer called safeagents intercepts each tool call with rule-based detectors, assigning a zero score and injecting an explanatory rejection string whenever an agent attempts reward shortcuts.

The asynchronous architecture decouples rollout generation from gradient optimization, keeping up to 48 concurrent agent sessions active per trainer for LFM models and 12 for TwIL-8B. Finished rollouts are packed into micro-batches capped by token budgets of 131,072 tokens for LFM2.5-24B-A2B and 65,536 tokens for LFM3-1.7B and TwIL-8B, allowing new adapter checkpoints to register into vLLM every 10 steps without pausing inflight requests. Stale rollouts originating from policies older than four weight updates are automatically purged from the pipeline.

Harness adapters: why it matters

Serving one base model with multiple harness-specific LoRA adapters drastically reduces compute costs during both reinforcement learning and evaluation phases. Because agent rollouts spend the majority of their execution time waiting inside sandboxes rather than generating tokens, forward and backward passes take around 35 seconds per step compared to several minutes of environment waiting time. Sharing a single GPU across three trainers allows LFM3-1.7B training to maximize hardware utilization by interleaving compute phases instead of leaving three separate instances idling while tasks execute.

Evaluation efficiency shows comparable economic gains across held-out benchmark runs. In one test, evaluating nine checkpoints of LFM2.5-24B-A2B across 1,350 total agent sessions completed in under one hour on a single H200 GPU, incurring approximately $5 in total cloud GPU expenses. Merging each checkpoint back into base weights and serving them as standalone full-parameter models would have cost roughly $90 under identical test constraints, while also introducing potential discrepancy between the trained adapter weights and the served artifact.

Harness adapters: who is affected

Engineers building and deploying local developer agents inside pi, opencode, or hermes can directly swap in specific adapters without altering their client configurations, prompts, or tool schemas. Because harness adapters preserve model accuracy while reducing unnecessary tool calls on held-out tasks, local developers encounter lower latency and reduced computational consumption during iterative coding sessions. The workflow allows teams to preserve reasoning traces in context across multi-turn agent sessions because thinking mode remains permanently enabled throughout both training and inference.

Teams training proprietary open-source models also gain a verified recipe for adapting non-agentic models into functional coding agents. The experiments demonstrate that TwIL-8B, despite lacking agentic data in its pre-training history, successfully adapted to tool workflows under reinforcement learning when allotted 30 agent steps per session and a 20-minute execution timeout. This framework gives model builders a structured pathway to adapt base foundation architectures into functional tool-using systems without requiring massive supervised fine-tuning corpora.

Harness adapters: what to watch

The team is streaming active training runs and logging live batch metrics, including groups per step and samples per step, on their public training dashboard. Infrastructure allocation across the Hugging Face Jobs platform spans two H200 GPUs for LFM3-1.7B and four H200 GPUs for runs covering LFM2.5-24B-A2B and TwIL-8B. Observers should track whether the 200-step runs on LFM2.5-24B-A2B, the 400-step runs on LFM3-1.7B, and the 300-step runs on TwIL-8B maintain stability as they complete repeated cycles through their 530, 780, and 630 total tasks.

A key factor to monitor will be how safeagents scales its rule-based detection to novel shortcut strategies across complex repositories. As developer communities construct larger task suites with Repo2RLEnvs, observing adapter transfer across additional unmodified harness targets will determine whether the single-base multi-adapter approach becomes a standard deployment pattern for agentic open-source software development.