We Searched 14 Same-Meaning Query Pairs to See How Semantic Search Works
Rare phrasings mostly got pages written in the common words. Same-meaning searches still got almost entirely different pages. Here's what both mean for SEO.
- Numbers in the post
- Query pairs in the post
- Embedding test: where the right sentence ranked
- Google’s own dates for its meaning-matching systems
Rare phrasings mostly got pages written in the common words. Same-meaning searches still got almost entirely different pages. Here’s what both mean for SEO.
Where the data comes from: our October 19, 2026 Google searches from the US of 14 pairs of queries that mean the same thing, the same-day ChatGPT and AI Overviews checks, and a test of three free embedding models.
- Medium post: We Searched 14 Same-Meaning Query Pairs to See How Semantic Search Works
- RankControl, an AI SEO and AEO platform
- Google Sites page: Same meaning, different results: what we found testing 14 query pairs
Numbers in the post
| Number | What the post says | Source | Date |
|---|---|---|---|
| 14 | On October 19, 2026, we searched Google from the US for 14 pairs of queries that mean the same thing. In most pairs one phrasing is common and the other is rare, going by Google Ads search volume. | Search Shift test, Google from the US | October 19, 2026 |
| 56 of 83 | Across the 11 pairs where one phrasing was clearly rarer, 56 of the 83 top results (about two in three) used only the common words in their titles. | Search Shift test, Google from the US | October 19, 2026 |
| 0 of 90 | For the common phrasings, 0 of 90 results were titled only in the rare words. | Search Shift test, Google from the US | October 19, 2026 |
| 7 of 207 | Across all 14 pairs, the two top 10 lists shared just 7 URLs, never more than one per pair, out of 207 different pages. That’s roughly 3% overlap for searches that mean the same thing. | Search Shift test, Google from the US | October 19, 2026 |
| 0 of 8 | “Car makes noise when braking” is the exception worth noticing. It had enough pages of its own that Google left it alone (0 of 8 swapped). | Search Shift test, Google from the US | October 19, 2026 |
| 9 of 29 | Same day, we sent all 29 queries to ChatGPT through a data provider’s ChatGPT feed. It went to the web for 9 of them and answered the other 20 from memory. | Search Shift test, ChatGPT through a data provider’s ChatGPT feed | October 19, 2026 (“Same day”) |
| 27 of 179 | Google’s AI Overviews turned up on 27 of our 28 pair searches. In the 13 pairs where both phrasings got one, the two overviews shared 27 of their 179 sources, about 15%. | Search Shift test, Google’s AI Overviews | October 19, 2026 (“Same day”) |
| 11 to 18 | Our rough estimate for doing this by hand is 11 to 18 hours a month. | Search Shift estimate | Not dated in the post |
Query pairs in the post
Top results for the rare version that were titled only in the common words.
| Rare phrasing | Common phrasing | Results titled only in the common words | Source | Date |
|---|---|---|---|---|
| “inexpensive airfare to london” | cheap flights to london | 10 of 10 | Search Shift test, Google from the US | October 19, 2026 |
| “customer management software for small companies” | best crm for small business | 8 of 8 | Search Shift test, Google from the US | October 19, 2026 |
| “egg cooking time” | how long to boil an egg | 7 of 7 | Search Shift test, Google from the US | October 19, 2026 |
| “sneakers for fallen arches” | running shoes for flat feet | 6 of 7 | Search Shift test, Google from the US | October 19, 2026 |
| “notebook computer not powering up” | laptop won’t turn on | 5 of 7 | Search Shift test, Google from the US | October 19, 2026 |
| “car makes noise when braking” | squeaky brakes | 0 of 8 | Search Shift test, Google from the US | October 19, 2026 |
Embedding test: where the right sentence ranked
| Query | What the post says | Source | Date |
|---|---|---|---|
| “how to fix a dripping tap” | the real answer about a leaky faucet came 3rd | Search Shift test, bge-small | Not dated in the post |
| “best crm for small business” | the right sentence came 4th, behind a CRM for enterprise banks | Search Shift test, bge-small | Not dated in the post |
| “cheap flights to london” | 4th again, behind cheap London hotels | Search Shift test, bge-small | Not dated in the post |
| “how to fix a dripping tap” | On the faucet query, bge-large scored the decoy 0.730 and the real answer 0.715. | Search Shift test, bge-large | Not dated in the post |
| “best crm for small business” | mxbai-large lost the CRM one by 0.004 | Search Shift test, mxbai-large | Not dated in the post |
Google’s own dates for its meaning-matching systems
| System | What the post says | Source | Date |
|---|---|---|---|
| RankBrain | it called RankBrain, launched in 2015, the first deep learning system it deployed in Search | Google’s 2022 post on how AI powers its results | 2015 launch, in a 2022 post |
| Neural matching | Neural matching followed in 2018. | Google’s 2022 post on how AI powers its results | 2018 |
| BERT | When BERT arrived in 2019, Google said it would help with one in 10 English searches in the US. | Google’s post on BERT | 2019 |
- Reference: https://medium.com/@thesearchshift/we-searched-14-same-meaning-query-pairs-to-see-how-semantic-search-works-f89d06d90763
- Reference: https://rctrl.com
- Reference: https://sites.google.com/view/search-shift-notes/semantic-search-test
- Reference: https://blog.google/products/search/how-ai-powers-great-search-results/
- Reference: https://blog.google/products/search/how-ai-powers-great-search-results/
- Reference: https://blog.google/products/search/search-language-understanding-bert/
Write a comment