Gemini 3.5 Pro introduces System 2 thinking, moving beyond token prediction to deliberate reasoning. Explore the architecture of Google
What System 2 Thinking Adds Beyond Token Prediction
Most large language models are trained to predict the next token well. That skill produces fluent answers, but fluency is not the same as careful reasoning. System 2 thinking refers to a deliberate mode of computation: the model spends extra internal work on a hard problem before committing to a final answer, rather than treating every prompt as a single forward pass of surface pattern matching.
In practice, that means the architecture must support more than a single “best next token” path. Intermediate steps, self-checks, and controlled exploration of alternatives become first-class parts of how the model produces output. The goal is not longer answers for their own sake; it is higher reliability when a problem needs planning, multi-step logic, or careful tradeoff analysis.
Architectural Building Blocks for Deliberate Reasoning
Architecting System 2 behavior usually involves several cooperating pieces. Training signals reward correct multi-step solutions, not only plausible wording. Inference-time controls decide when to think longer and when a short path is enough. Memory of intermediate conclusions must stay coherent so later steps do not contradict earlier ones. Safety and refusal layers still apply, because longer reasoning does not automatically make an answer safe or true.
- Routing: Classify whether a query needs shallow completion or deeper deliberation.
- Trace management: Keep intermediate reasoning structured enough to revise, prune, or verify.
- Stopping criteria: End the thinking phase when confidence or budget thresholds are met.
- Answer synthesis: Convert the internal trace into a clear final response the user can use.
Gemini 3.5 Pro sits in this design space: a model family entry that emphasizes deliberate reasoning over pure next-token fluency. The product claim is architectural—System 2 as a mode of operation—not merely a marketing label for longer chat replies.
Tradeoffs Teams Should Plan For
Deliberate reasoning costs latency and compute. Every extra internal step increases response time and token budget. Product designers need clear defaults: use deep thinking for code reviews, multi-hop analysis, and ambiguous specs; use a lighter path for greetings, lookups, and simple rewrites. Without that split, users pay for thinking they do not need, or worse, the system underthinks on tasks that require it.
Quality also depends on evaluation that matches the new mode. Classic fluency and short-form benchmarks understate gains (and failures) in multi-step work. Teams should test chain consistency, recovery from a wrong intermediate assumption, and whether the final answer still matches the original constraints after a long internal path. Hallucination risk does not disappear; it can move into intermediate steps that look rigorous but rest on a false premise.
How to Use a System 2 Model Effectively
Prompt for the job you want the deliberation to solve. State constraints, success criteria, and what “done” looks like. Ask for structured intermediate checkpoints when the task is complex: assumptions, options considered, risks, and the chosen path. Prefer tools and external verification for facts the model cannot observe—docs, tests, logs—so internal reasoning is grounded rather than invented.
Integrate the model where deliberate work has leverage: design reviews, incident postmortems, migration plans, and debugging sessions with incomplete information. Keep humans in the loop for high-stakes decisions. System 2 thinking improves how a model spends compute on hard problems; it does not replace domain judgment, measurement, or responsibility for what ships.