Claude Code self-improved on business workflows using conversation, not memories

I'll pull the source article so the paragraphs stick to real details rather than inventing numbers or claims.**Claude Code** (Opus 5, `xhigh` effort), instructed only to get better on tasks pulled from a simulated business-workflow MCP server (`curriculum`), raised held-out success from **34%** to **48%**. Tasks came from spreadsheets, CRM records, email, and ticketing; each required `get_task`, tool use, then `done` for scoring until the stream returned `{done: true}`. The run is documented by Andrew Jesson (6 Aug 2026) and sits on HN with minimal discussion so far.

On the technical side, the agent never wrote code, spawned subagents, or searched the web even though those tools were available and validated. It made **84** memory writes/edits and built a directed knowledge base of notes (frontmatter name/description/type, episode summaries and scores, `Related` links; **33** `feedback` notes with How to apply, **24** also with Why). Ablations on the same **120** held-out tasks (fresh session per task) showed the gain lived in the conversation: a cold start solved **41/120** (~$0.86, 29.8 turns); full conversation lifted success at every compaction boundary (**+10.0, +7.8, +7.5, +14.2** points); post-compaction summaries kept most of that for a few thousand tokens vs ~900k in the full trace for three of four boundaries (fourth boundary: summary lost **11.7** points, CI [−19.8, −3.6]). Memory files alone did not clear zero; pooled across comparisons they added nothing measurable, and under summary-only boots they even cost ~3 points in a post-hoc split. Thought content’s contribution fell across the run (**+4.2 → −0.8**). Across **1,080** memory-armed probes, notes were written in **545** runs but opened in only **195**.

For builders shipping agents on CRM/ticket/email loops, this separates **writing** memory from **using** it. Dense, immediate numeric grades plus a full graded history in context made external notes redundant; the harness’s compaction summary accidentally became the portable improvement artifact. Real production feedback is sparse and delayed, so a knowledge base may still pay off when context has already dropped what mattered—this experiment simply did not stress that regime. Auto-compaction (four times, agent cannot schedule or choose what survives) and fixed model/effort at session start also cap what self-improvement can touch.

In product and competitive terms, Claude Code exposes code execution, subagents, and web search, but under one-task-at-a-time delivery none of those paths fired; raising model/effort and explicit permission still did not. Skills need a session reload the agent cannot trigger; MCP config is read-only under `--strict-mcp-config`; model and effort are set at init (subagents can pick a model but not effort). Open harnesses such as Pi and Hermes can open those knobs and measure them. Jesson flags Fable 5 at max reasoning plus concurrent task dispense as the next place to look for code/subagent behavior. The substrate is **hgym**.

Practical takeaway: if you hand an improved agent to the next session after graded workflow practice, keep the **compaction summary** (and the scored task trace it compresses), not the self-authored memory graph. Watch whether agent-controlled compaction, concurrent multi-task servers, and denser production reward signals change when memory notes start moving held-out success—and whether cost shifts from “more correct answers” to “fewer steps to the same answer,” which is what the extra conversation dollars bought here.