Show HN: Do Codex skills save tokens? A six-run task-size benchmark

A Show HN post titled Do Codex skills save tokens? A six-run task-size benchmark is live on Hacker News, with the write-up hosted at https://codex-howto-benchmark.nguyenvantamdk2.chatgpt.site and discussion at https://news.ycombinator.com/item?id=49150997. At the time of this note it sat at 4 points with 0 comments. The piece frames a narrow question: whether Codex skills reduce token use across tasks of different sizes, and it answers that question with a six-run task-size benchmark rather than a single demo.

The technical design, as presented in the title and framing, is a controlled comparison over six runs that vary task size while holding the skills-related variable under test. That matters because agent token cost is dominated by how much context is stuffed into each turn—system prompts, tool schemas, skill instructions, and intermediate scratch—so a skills feature can either amortize guidance across runs or bloat every request with reusable boilerplate. A multi-run, size-stratified setup is the right shape for that claim: it can show whether skills help more on small tasks, large tasks, or both, without relying on one cherry-picked session.

For engineers shipping or buying coding agents, the useful output is not a slogan about skills in general but a measured answer to whether adding skill packs lowers billable tokens for the kinds of work they actually run. Token spend is now a first-class product and ops metric: it drives latency, rate limits, and unit cost for any team running Codex-class agents in CI, review bots, or interactive IDEs. A public six-run benchmark gives implementers a template for validating their own skill libraries against their own task mix instead of trusting vendor marketing.

The post lands in a crowded lane where Claude, Codex, and related agent stacks all compete on tool use, reusable instructions, and “skill” or “howto” packaging as differentiators. HN Claude/Codex/Fable sourcing and a Show HN launch put this next to the same audience that debates prompt libraries, MCP-style tool surfaces, and agent memory. Early traction is thin—4 points, no comments yet—so the work is still early signal rather than consensus, but the framing matches how that market currently argues: cost and reliability of multi-step agent runs, not feature checklists alone.

Practical takeaway: treat Codex skills as a measurable cost lever, not a default on switch. If you maintain skill packs, re-run a similar six-run, task-size split on your real workloads and keep the raw token totals; watch whether skills cut tokens on larger tasks (where reuse should win) or only pad small ones. Next to watch is whether follow-up discussion or replications on the Show HN thread challenge the methodology or reproduce the direction of the result once the post gets more eyes.