ChatGPT Thinking mode boosts official sources over Reddit

A detailed analysis by Semrush and Kevin Indig reveals how ChatGPT's high-reasoning mode shifts citations from Reddit to official sources.

By Central
ChatGPT's Thinking mode prioritizes government and academic sources over user-generated content like Reddit.
Highlights
  • Only 25.6% of domains cited by ChatGPT's high-reasoning mode overlapped with minimal-reasoning mode.
  • Reddit's citation share dropped from 15% to 7% when ChatGPT used high-reasoning mode.
  • Consumer tech was an outlier where both reasoning modes cited the same top sources due to limited authoritative options.

When ChatGPT shifts into its high-reasoning or “Thinking” mode, it transforms into a fundamentally different search engine for brand visibility, according to a detailed analysis conducted by Semrush and SEO expert Kevin Indig. The study reveals that this advanced reasoning mode does not merely provide deeper answers; it cites a dramatically different set of sources, runs nearly five times as many web searches, and significantly elevates the prominence of official, government, and academic domains at the expense of user-generated content platforms like Reddit.

The Thinking Mode vs. Standard Response: A Tale of Two Search Surfaces

The core finding of the analysis is a stark divergence in citation behavior. For the exact same prompts, only 25.6% of the domains cited by the high-reasoning mode overlapped with those cited by the minimal-reasoning mode. This means that nearly three out of every four sources changed the moment a user asked for a more reasoned, step-by-step analysis rather than a quick, direct answer. The shift represents a fundamental change in what it means to be visible within the ChatGPT ecosystem, moving from surface-level brand mentions to a deeper integration within a complex research process.

Citation Behaviors Diverge Significantly Between Reasoning Levels

The statistical divergence between the two modes is pronounced. In minimal reasoning, akin to standard instant answers, ChatGPT cited sources in 50% of its responses. This rate jumped to 68% in high-reasoning mode. Furthermore, the depth of sourcing increased; the average number of citations per response rose from 2.6 to 4.5. This is directly correlated with the raw volume of research conducted. Across the test set, the high-reasoning mode executed 1,130 individual web searches, dwarfing the 245 searches run by its minimal-reasoning counterpart. The Thinking mode is not just thinking harder; it is reading far more widely to form its conclusions.

Reddit and User-Generated Content Lose Ground to Official Sources

One of the most significant shifts for digital marketers is the dramatic decline in citations from social and user-generated content platforms when ChatGPT engages its high-reasoning capabilities. Reddit’s share of citations was nearly halved, falling from 15% in minimal reasoning to just 7% in high reasoning. Similarly, citations from general user-generated content and review sites dropped from 14.3% to 6%. This suggests that for deeper, more analytical queries, the model’s reasoning process de-prioritizes anecdotal or community-based information in favor of more authoritative sources.

Government, Academic, and Official Documentation Surge

The sources that gained the most ground are precisely those associated with authority and verifiability. Government and academic sources saw their citation share rise from a mere 1.9% to a significant 8.8%. More impactful was the growth in official documentation and support pages, which increased their share from 12.4% to 17.5%. For brands, this is a clear signal: a Twitter thread or Reddit post may be enough for a surface-level query, but to be cited in a deeper analysis, a company needs vetted, official resources that satisfy a more rigorous selection process.

How ChatGPT’s Thinking Mode Reaches Its Answers

The mechanics of how high reasoning arrives at its answers explain the shift in sources. The mode does not simply search for one definitive answer; it deconstructs a complex question into a series of smaller, targeted sub-searches. The analysis found that for comparison-heavy prompts—such as comparing two software platforms—the high-reasoning mode averaged 24 sub-queries per prompt, compared to just 5.5 for minimal reasoning. The average number of citations for these comparison responses peaked at 9.8, against 5.8 for the simpler mode.

Examples of the Sub-Query Process

This process is highly granular. For a prompt asking for a comparison of two CRM platforms, the high-reasoning model might execute separate searches for each company’s pricing page, integration ecosystem, security certifications, customer support documentation, and independent analyst reviews before synthesizing its final, cited answer. This method of breaking down a query into its constituent parts is why the model favors official pages; it is seeking specific, factual information on discrete features rather than general opinions.

Why We Care: The Strategic Implications for Brand Visibility

The implications for content strategy and search engine optimization are profound. A brand might enjoy visibility in ChatGPT’s quick, minimal-reasoning responses, only to disappear when a user probes deeper with a more complex question. The key takeaway is that visibility is no longer a single state but is contingent on the depth of the user’s query. A brand’s presence in official documentation, robust support pages, and third-party analyst reports has become as important as its presence on social media platforms.

Brand Persistence and the Path to Purchase

The analysis also tracked “brand persistence”—the ability of a brand mentioned in an early, problem-defining stage of a journey to still be mentioned at the final selection stage. In four of the 20 buyer journeys tested, high reasoning successfully carried a brand from the initial problem stage to the final selection stage. Minimal reasoning showed no instances of this full-journey persistence. Additionally, high reasoning was more likely to reuse the same authoritative domains multiple times within a single answer. This domain re-use occurred in 51 of 100 high-reasoning responses, compared to only 26 of 100 minimal-reasoning responses, further reinforcing the value of becoming a “trusted source” for the model.

Vertical-Specific Impacts: Finance and Health See the Biggest Gains

The shift in citation behavior is not uniform across all industries. The data shows a clear vertical stratification. The finance category experienced the largest increase in citation rates, rising by 28 percentage points when shifting from minimal to high reasoning. Health and lifestyle categories saw a similar, significant jump of 24 points. In these high-stakes fields, the model’s reasoning engine aggressively seeks out authoritative, verifiable information. B2B SaaS also benefited, with a 16-point gain. Interestingly, consumer tech showed a minimal increase of only 4 points, suggesting that for less critical or more commonly discussed tech products, the sources cited by the simple and reasoning modes were largely the same.

Why Consumer Tech is an Outlier

The consumer tech result is particularly instructive. Even though the high-reasoning mode ran more sub-queries for consumer tech prompts than for any other category, it often concluded by citing the same major tech review sites and brand pages as the minimal-reasoning mode. This suggests that for well-established and widely reviewed product categories, the pool of top-tier, authoritative sources is already quite small and stable. The model’s deeper reasoning did not lead it to discover new sources; it simply confirmed the validity of the most prominent ones.

Analyzing the Data and Its Methodology

The study provides a robust foundation for these conclusions. Semrush and Kevin Indig tested 100 prompts across 20 distinct buyer journeys spanning B2B SaaS, finance, consumer tech, and health and lifestyle. Each prompt was run once with minimal reasoning and once with high reasoning, allowing for a direct, apples-to-apples comparison of citation rates, cited sources, and the number of background searches. The methodology provides a clear, empirical look at how the model’s internal processing changes its external output. As ChatGPT and similar large language models integrate deeper reasoning capabilities, the surface of the search is no longer the only surface that matters. The thinking layer behind the answer has become a distinct, more demanding gatekeeper for brand inclusion, prioritizing official documentation, academic rigor, and government resources over community sentiment and user reviews.

Share This Article