When searching for the "best" credit cards, consumers traditionally turn to Google for detailed comparisons, expert reviews, and community feedback. However, the landscape is changing rapidly as Large Language Models (LLMs) like ChatGPT, Bard, and others become popular sources of information. Unlike traditional search engines, LLMs synthesize information from multiple sources, providing streamlined answers directly within chat interfaces.
To understand how these LLMs source their information, we conducted an extensive analysis of the top 100 queries related to "best credit cards." Our goal was to discover where LLMs pull their data from, which sources dominate their answers, and how this contrasts with traditional Google search results. The findings were surprising and offer insights for marketers, financial institutions, and consumers alike.
Key Insights from Our Analysis
- Multiple Sources Inform LLM Responses
- On average, each query analyzed referenced approximately 4.8 distinct sources. This indicates that LLMs tend to aggregate information from diverse websites to generate balanced and comprehensive answers, rather than relying heavily on a single source.
- Affiliate Dominance: LLMs Love Affiliates
- A significant finding was the overwhelming dominance of affiliate sources in LLM responses
- 82% of all cited sources within LLM answers were affiliate websites.
- Affiliates already dominate traditional Google Search Engine Results Pages (SERPs), accounting for roughly 50% of the results. However, LLMs appear even more favorable toward affiliate content, highlighting an important consideration for brands when crafting their affiliate strategies.
- Aggregators Still Relevant but Losing Ground
- Aggregator websites, historically strong players in credit card searches, still hold noticeable visibility:
- Aggregators constitute 11% of the sources cited by LLMs.
- Despite experiencing declining traffic trends on Google searches over the past three years, aggregators remain significant, holding onto about 30% of Google SERP positions for "best credit card" queries.
- This suggests aggregators still provide valuable content but may need to innovate to maintain their prominence in the era of AI-generated answers.
- Community Sources Underutilized
- While consumers frequently turn to communities and user-generated content for unbiased opinions, LLMs show a surprising underrepresentation of these sources:
- Wikipedia and forum-style websites each accounted for just 4% of citations.
- Interestingly, Reddit and Quora, popular platforms known for authentic user-generated content, were completely absent from the LLM-sourced data, despite Reddit alone holding 16% of share-of-voice on Google searches.
- This highlights an opportunity for community-driven platforms to optimize content differently to gain traction with LLMs.
- Minimal Self-Citations from Brands
- Remarkably, direct brand sources (official bank or card issuer websites) made up only 1% of the citations. Brands rarely influence LLM-generated answers directly, a stark contrast to traditional SEO practices emphasizing direct site optimization.
- Brands might need to reconsider their digital strategy, shifting resources towards indirect channels and affiliates to effectively appear in LLM recommendations.
Brand Visibility & Consumer Sentiment
When analyzing brand mentions and sentiment, the following institutions were the most frequently referenced in LLM-generated responses:
- American Express (19.6%)
- BMO (14.6%)
- Scotiabank (14.4%)
- TD (10.6%)
- CIBC (7.9%)
- RBC (5.8%)
- Tangerine (5.0%)
- MBNA (4.2%)
- Neo Financial (2.7%)
- Other brands combined represented 15.2%
Across the top brands mentioned, "high annual fee" consistently emerged as the leading negative sentiment. Consumers and reviewers frequently cited this as a significant drawback, impacting brand perception.
Implications for Marketers and Brands
The shift towards LLM-driven information retrieval significantly affects how brands need to approach their online strategies. To effectively influence consumer perception, brands must:
- Prioritize strong affiliate relationships, given affiliates' substantial visibility within LLM results.
- Enhance content visibility on aggregator sites, recognizing their ongoing importance.
- Address community-driven content gaps, optimizing how user-generated content appears to LLMs.
- Reassess direct content strategies, focusing less on traditional SEO and more on third-party credibility.
As the digital landscape evolves, keeping pace with the dynamics of LLM-generated results will become crucial for success in highly competitive financial markets.
By aligning strategies to match these emerging patterns, brands can effectively position themselves for optimal visibility in this new information retrieval era.