The Communication Bottleneck: A Round-Trip Study of Tree-Structured
When language models reason in chain-of-thought or exchange free-text intermediates, they serialize structured information into natural language.
By Dillip Chowdary • Oct 01, 2026 • Source: Apple Machine Learning Research

In this post
01What actually changed
60)
02How it works
No bold/bullets. Focus: What to watch (what builders should verify, future application of this evaluation protocol, model testing). Drafting Paragraph
No bold/bullets. Focus: What to watch (what builders should verify, future application of this evaluation protocol, model testing). Drafting Paragraph
Advertisement
Tech Pulse Daily
Get tomorrow's pulse first
Join engineers who read Tech Pulse before stand-up. Free, weekday mornings.
03Why it matters now
We propose a round-trip protocol that answers this question empirically for tree-structured expressions. A generator converts a procedurally generated arithmetic expression into a word problem, a separate extractor recovers the expression from the word problem alone, and symbolic equivalence provides an exact oracle.
04Who is affected
Evaluating all pairwise combinations of sixteen models yields a communication matrix whose marginals separate generation quality from extraction quality. First, the channel is lossy and asymmetric: swapping which model generates and which extracts shifts accuracy by up to 60.4 points, and the best pair reaches 92.9% by combining different models on each end rather than the same model on both.
05What to watch
Second, at least 73.6% of round-trip failures originate at generation, and difficulty is driven by tree structure (operator count, depth, right-branching) rather than model family. See the full write-up from Apple Machine Learning Research via the source link for quotes and complete context.
Author
Dillip Chowdary
Writes Tech Bytes coverage of AI, engineering, and the tools that actually ship. Editor of Tech Pulse Daily.
Related on Tech Bytes
Custom ChatGPTs push ClickFix attacks to deploy RAT malware
Read →
Bring near-Astra intelligence to everyday work with GPT-6.1 Sol on Amazon Bedrock
Read →
Accelerating agentic RL and evaluation research velocity with 45x faster GKE Agent Sandbox
Read →
Google is a technology partner for the launch of America.gov.
Read →
Today's Tech Pulse briefing
Full briefing →
Advertisement