Giving Google’s mapping AI specialized capabilities produced larger score gains than subsequently adding planning and execution guidance, across both tested models on three benchmarks with room for improvement. That is the strongest finding from Superpower Daily’s comparison of Google’s published means—not evidence that planning generally fails, or that these differences are statistically significant.
Our original analysis asks what Google’s four agent configurations can tell builders separately about specialized skills and upfront planning. We compared Gemini 3.8 Flash and Gemini 3.1 Pro within each benchmark, using the supplied public-document snapshot through October 14, 2026, at 20:31 UTC. We subtracted published, rounded means; we did not run the agent or collect new experimental results. Google’s technical report says queries, decoding defaults and execution limits match across configurations within each model.
Two additions, not two isolated ingredients
Google’s experimental Geospatial Reasoning agent handles questions involving places, populations and environmental conditions. The evaluation starts with a standalone language model, then adds basic tools, then specialized geospatial skills and tools, and finally upfront planning plus execution guidance. Basic tooling includes search, maps and network-isolated Python, with a search cap. The specialized configuration reaches a much broader capability suite. That distinction matters: the comparison changes what the agent can access and do, not merely how identical tools are organized.
There is also a missing comparison: basic tools with planning enabled. Without it, the study cannot show what planning would add before specialized capabilities arrive, or whether planning and those capabilities strengthen each other. The standalone model does not fill that gap. The design supports comparisons between the tested packages, but not a complete separation of each ingredient’s contribution.
The larger increment appears in all six comparisons
Across GeoExpertQueries, GeoKTools and CBGB, both models show the same ordering: adding specialized capabilities produces the larger observed increment. The matrix below keeps every benchmark separate. Each row gives skills minus basic tools first, then full agent minus skills, in score points. These are descriptive contrasts between rounded means, not estimates of a universal effect or a ranking pooled across different tests.
Evidence matrix: specialized-capability increment / planning-plus-guidance increment
- Flash, GeoExpertQueries: +48.9 / +10.5 score points. Both additions raise the published mean; specialized capabilities add more.
- Flash, GeoKTools: +46.0 / −0.1 score points. The full configuration’s mean is slightly below the skills configuration.
- Flash, CBGB: +68.2 / −1.5 score points. The specialized-capability increment is positive; the subsequent increment is negative.
- Flash, UnivEARTH: +0.5 / −0.7 score points. Google treats this benchmark as saturated.
- Pro, GeoExpertQueries: +48.5 / −3.2 score points. Unlike Flash, Pro’s mean falls after planning and guidance are added.
- Pro, GeoKTools: +46.7 / −2.9 score points. The specialized-capability configuration has the higher published mean.
- Pro, CBGB: +71.2 / 0.0 score points. Adding planning and guidance leaves the rounded mean unchanged.
- Pro, UnivEARTH: +1.2 / −8.4 score points. This is another comparison on the benchmark Google considers saturated.
UnivEARTH deserves separate treatment rather than equal weight in a sweeping conclusion. Flash’s standalone model already scores 93.6 there. Google suggests that public availability of the source NASA articles and answers may explain the high baseline. With performance already near the top, the benchmark offers less room to distinguish the benefits of the added packages than the other three tests.
Planning’s strongest result is model-specific
Google’s October 14 research blog argues that upfront planning unlocks open-ended, real-world workflows, helping prevent shortcuts and unreliable assumptions. Flash’s GeoExpertQueries result supports a narrower version of that interpretation: adding planning and guidance raises its published mean by 10.5 points. But Pro moves in the opposite direction on the same benchmark. Across all eight model-and-benchmark rows, Flash’s open-ended test is the only positive full-minus-skills difference. The result is evidence for that configuration, not a consistent benefit across models.
Nor do the negative differences prove harm. Google reports means and standard deviations across three full agent runs, not confidence intervals for the component differences. Flash’s GeoExpertQueries scores are 52.2 ± 5.4 with skills and 62.7 ± 2.5 with the full agent. Those spreads describe variation in each configuration’s aggregate score. Without paired uncertainty for their difference, this comparison cannot establish statistical significance. Reading a small decline as a demonstrated regression would overstate the evidence just as surely as declaring planning a general winner.
A workflow score is not answer verification
GeoExpertQueries grades executed workflows against expert-authored reference workflows, not verified final answers. Its automated judge examines the agent’s execution record, including tool calls, code outputs and response. That makes the benchmark relevant to analytical method: whether an agent performs the steps an expert expects. It does not turn a higher workflow score into proof that the finished analysis is correct. Google accordingly describes the output as a strong first draft for expert review, rather than a final answer.
The other benchmarks measure different outcomes. GeoKTools awards full credit for exact matches, then uses a language-model judge to give partial credit for other responses. CBGB checks numerical correctness to two significant figures. UnivEARTH measures label accuracy. A point therefore does not represent the same achievement across these rows. Keeping the outcomes separate preserves the useful comparison—what changes within a benchmark—without inventing an overall score for geospatial intelligence.
The next checks need missing controls and test materials
One documentation inconsistency also remains unresolved. The report lists 76 GeoExpertQueries, 220 GeoKTools, 44 adapted CBGB and 140 UnivEARTH test queries per run. The benchmark repository instead lists 210 held-out GeoKTools queries. Our subtractions use the published means, not reconstructed counts, so they do not resolve that discrepancy. A clarified test denominator would make the underlying evaluation easier to audit.
Reader comments
Newest comments first. Replies stay oldest first.