Why AI shopping agents skip good products, and how to diagnose it SKU by SKU

A 4.9-star product, zero AI picks

On November 2, Search Engine Journal published one of the most detailed public tests yet of how AI shopping agents choose products. Slobodan Manic gave three Claude models (Haiku 4.5, Sonnet 5 and Opus 5) the results Shopify's catalog returns for a search and asked each to pick one product to buy. Across all runs, the models made 1,980 picks.

One result stands out for consumer brands. When the request was "mineral sunscreen SPF 50, under $25, for oily skin," the models made 300 picks. The best-rated sunscreen in the group, at 4.9 stars, received none of them. A 4.2-star lotion with 105 reviews won 33 of Opus's picks. Its listing was the only one that said "without clogging pores," and Opus quoted that phrase in 31 of its 33 reasons.

The better product lost on wording, not on quality. Most brands would never see this happen, because AI answers send very little traffic to measure. OpenAI says ChatGPT now handles about a fifth of Google's search volume. Yet Ahrefs found ChatGPT sent only 0.36% of visits to 118,671 tracked sites in October. Shoppers get their answers, and the brand often never sees the visit.

Below, we walk through what the data shows at each stage of AI visibility work (Monitor, Diagnose, Create and Sync) and what a brand can do at each one.

Monitor: is the product invisible everywhere, or only in some answers?

The test's first finding is that there is no single AI ranking. Across six everyday searches (backpack, coffee beans, running shoes, water bottle, wireless earbuds and yoga mat), the three models' favorite products matched in none. Each model stayed consistent with itself, though.

Source: Slobodan Manic via Search Engine Journal, Nov 2, 2026. 50 picks per model per search.

This means a brand-level average can hide real problems. A brand can look healthy overall while a specific product is missing from one engine's answers. Tracking has to happen per product and per engine. Searchpanel's Visibility Score is built that way: a 0 to 100 score for each SKU, refreshed daily across ChatGPT, Gemini, Perplexity, Copilot and Google AI Overviews, broken down by mention rate, position, citations and sentiment.

The second finding is that the wording of the question changes the winner. For "mineral sunscreen SPF 50," Haiku picked the catalog's $48 top product 100 times out of 100. Adding "under $25, for oily skin" took that product to zero picks across all three models. Category-level monitoring would miss this entirely. Prompt Tracking breaks a category into the question types buyers actually use (category searches, comparisons, attribute filters, occasions and problems) and tracks each one separately.

The third finding concerns what competitors win on. Two of the three models leaned heavily on review counts. Shopify's catalog listed one water bottle at 447,221 reviews, about 68 times the 6,537 shown on the brand's own page, and the models picked it most often without any way to verify it. When a competitor is recommended instead of you, it helps to know what that recommendation is based on. Competitors & Citations scores rival products on the same scale as yours and records the sources each answer cites.

Diagnose: what is the product missing?

In the sunscreen example, the gap was four words. The winning listing stated the shopper's need ("without clogging pores") in plain language. The highest-rated product described its ingredients ("organic plant oils") instead. The agent matched need to listing, and rating came second.

Gaps like this are common and easy to miss, because the language often already exists in a brand's own reviews but never reaches the listing. Content Gaps compares each listing with the answers it loses, its own review language and the rival listings that win, then identifies the specific attribute, claim or phrase that is missing.

Some gaps are technical rather than editorial. Adobe scored grocery pages at 55% for how much of a page AI systems can read, the lowest of any category it measured. A SISTRIX study of 48,527 AI-cited pages found that pages with FAQPage schema were cited 1.5 times as often. If an engine cannot parse the page, the copy does not matter. Technical & Schema checks each product page the way an engine reads it, including product schema, review markup, canonical tags and llms.txt.

A full catalog will surface dozens of issues at once, so order matters. The Fix Queue ranks every content and technical issue by the visibility it is expected to recover against the effort required, with the supporting evidence attached. For a product in the 4.9-star sunscreen's position, the top item would be to state its fit for oily skin in the title and first bullet, provided reviews or testing support the claim.

Create and Sync: fix the source, then publish it everywhere agents read

Agents repeat what listings say. In Manic's test, "clinically proven" appeared in 63 of the models' reasons, and 61 of those came directly from the chosen product's listing. The agent does not verify claims; it restates them to the shopper. When the product does not match, the cost shows up later. A Blue Yonder survey reported by Retail Dive found that 46% of shoppers who bought on an AI recommendation returned an item in the past year, most often (36%) because it was different than described.

That makes accuracy the starting point for any new copy. Before rewriting a listing, a brand needs one confirmed set of facts per product: specs, claims, ratings and sizes, each with a source. The review-count gap above (447,221 in the catalog vs 6,537 on the brand's page) is the kind of conflict that builds up when product data lives in separate systems. Searchpanel's Knowledge Base keeps each claim, spec and FAQ in one record with its source, and flags where a brand's own systems disagree.

With the facts settled, the copy can be rewritten around the gaps. Listing Generation drafts titles, bullets and descriptions from the specific gap behind each fix, grounded in the brand's verified claims and specs, and checks that no source detail was dropped. Nothing is published until a person approves it.

The last step is distribution. Agents read from different places: Shopify's catalog, retailer listings, brand sites and product feeds. A fix that lands in only one of them leaves the others out of date. Integrations syncs products from a brand's store or marketplace, publishes approved updates back, and measures the traffic AI answers send. Measurement needs care: Gemini stopped adding utm_source tags to its links around September 28, and its iOS app sends no referrer, so those visits appear in GA4 as Direct traffic.

A seven-step check for your top product

Run this for one hero product before Black Friday on November 27. It can be done manually or with a tool.

☐ Score the SKU, not the brand. Ask ChatGPT, Gemini, Perplexity and Google the same buyer question and write down whether your product is named, in what position and with what source. Repeat weekly.

☐ Add the clauses. Re-ask with budget, skin or hair type, diet, size or occasion. If you disappear when the question gets specific, that's your gap.

☐ Read the winner's listing. Note the exact phrase the AI quotes when it picks the competitor. Check whether your own reviews already say the same thing.

☐ Move proof from reviews into the listing. Title, first bullet and summary first, because those are what agents read.

☐ Line up your facts. Rating, review count, size and claims should match on your site, your Shopify catalog record and every retailer.

☐ Cut claims you can't back up. The agent will repeat them as its own reasons to buy.

☐ Check the page is readable. Valid Product and FAQPage schema, a correct canonical and no blocked AI crawlers.

Done by hand, this takes an afternoon for one product. Repeating it daily across a full catalog is the problem Searchpanel was built to solve.

See how AI engines treat one of your products. Run a free SKU audit for a visibility score across ChatGPT, Gemini, Perplexity and Google's AI answers. It takes about 60 seconds, with no account or credit card required. Or book a demo for a walkthrough of a live Visibility Score.

FAQ

Why does AI recommend my competitor instead of my product? Often because the competitor's listing states the shopper's exact need in plain words. In one test, a 4.2-star sunscreen that said "without clogging pores" beat a 4.9-star one that never mentioned pores.

Do all AI models recommend the same products? No. In a 1,980-pick test, three models' favorite products matched in none of six searches. You need to track each engine separately.

Why don't I see AI traffic in my analytics? AI answers send few clicks. ChatGPT sent 0.36% of tracked site visits in October, and Gemini's iOS app sends no referrer, so those visits land as Direct.

What is a SKU-level AI visibility score? A score for each individual product, not the brand, based on how often and how well AI engines name it across the questions buyers ask.