Veo 3 launched at Google I/O just a few weeks ago, and since then we’ve seen countless videos go viral, delighting millions of people and demonstratin...
What “thinking with images” actually means
Most multimodal systems still treat images as input to describe: caption the scene, answer a short question, extract text. “Thinking with images” points at a different loop. The model does not stop at a single pass over pixels. It can inspect a diagram, form intermediate conclusions, re-check regions that matter, and revise its answer when the visual evidence conflicts with an early guess.
That shift matters for work that already lives in screenshots, whiteboards, UI mockups, architecture diagrams, and photo evidence. Reasoning quality depends less on polished prose and more on whether the model can hold a visual claim, test it against another part of the image, and say when the picture does not support a conclusion.
Why the wider multimodal wave makes this timing real
OpenAI’s push on image-grounded reasoning arrives in a period when multimodal demos already move public attention fast. Veo 3 launched at Google I/O just a few weeks ago, and since then we’ve seen countless videos go viral, delighting millions of people and demonstrating how quickly people adopt tools that feel native to sight and motion. Video generation and image reasoning are not the same problem, but they share a product lesson: users expect AI to operate on what they can see, not only on text they type.
For builders, the practical takeaway is prioritization. If your product already ships screenshots, design files, or camera capture, pure chat is incomplete. Users will compare your tool to systems that appear to “look,” “check,” and “revise,” not merely describe.
Where multimodal reasoning helps—and where it still fails
Image-grounded reasoning is strongest when the hard part of the task is structure in the visual field: layout relationships, step sequences in a diagram, differences between two screenshots, or constraints that only appear when you compare regions. It is weaker when the image is ambiguous, cropped, low-resolution, or missing the legend that would make symbols mean something specific.
- Use it to draft diagnoses from UI bugs, board photos, and annotated sketches, then require a human confirm step for irreversible actions.
- Ask for region-level rationale (“which panel supports this?”) so you can audit the chain of thought against the pixels.
- Keep a text-only fallback path when images are private, degraded, or legally restricted from leaving the device.
- Treat visual answers as provisional when scale, units, or labels are incomplete in the source image.
Those constraints are product design, not footnotes. A model that reasons over images still needs clear failure modes, source attachment, and a way for users to correct a wrong visual interpretation without restarting the whole conversation.
How to evaluate systems in this phase
Skip marketing language and test workflows you already run. Give the system multi-panel diagrams, partial screenshots, and deliberately noisy photos. Score whether it notices contradictions, asks for the missing crop, and changes its mind when you supply a second image. Measure end-to-end usefulness: time to a correct next step, not eloquence of the caption.
Multimodal reasoning enters a useful phase when teams wire it into review, support, design, and documentation pipelines with those checks in place. The capability is real enough to change daily work; the risk is treating a confident visual narrative as proof. Keep the image, the claim, and the verification path together, and the new phase becomes operational rather than theatrical.