The assistant keeps recommending products you stopped selling

Start by ruling out the explanation everyone reaches for. A model inventing a product it has never seen is rare in a working assistant; naming a product you genuinely sold until March is not invention, it is retrieval doing its job against data that no longer matches the shop. There are 4 mechanisms that produce that behaviour, they need different fixes, and 1 observation separates them.

This is the failure mode most likely to reach a customer, because a discontinued recommendation reads as confident and correct right up to the moment somebody tries to buy it. It is also the one that undermines the case for the assistant internally faster than any latency number.

The check that splits it in 10 minutes

1. Reproduce the answer with the exact wording the customer used, and capture the full trace: the messages, any tool calls, the arguments sent, and the raw tool responses. If your assistant does not log tool calls with arguments, stop here and add that — every question below is unanswerable without it.
2. Call the tool yourself with the same product identifier or query, outside the assistant, and record what comes back.
3. Fetch the indexed document for that product directly from the retrieval store, and read its fields — not just the text, the metadata.
4. Look up the product in the catalogue of record and note its lifecycle state, its published state per region, and when it last changed.
5. Compare all 4. The pattern of agreement tells you which layer is lying.

Row 4 is the one that changes the meeting. It is not an AI problem at all: the assistant is faithfully reporting what your own systems believe, and the withdrawal never propagated past the merchandising screen where somebody clicked it.

Cause 1: the index lags the catalogue, and nobody set the cadence

Most retrieval indexes are rebuilt on a schedule inherited from the first proof of concept — nightly, because that was easy in week 2. A catalogue that changes hourly against an index that changes daily produces exactly this symptom, and it gets worse in the weeks when merchandising is most active.

• Distinguish addition lag from deletion lag. Many pipelines upsert on change and never delete, so a withdrawn product stays retrievable forever while new products appear promptly.
• Check that the identifier used for upsert is the same one used for delete. A pipeline keyed on a product id that writes and a pipeline keyed on a URL that deletes will leave orphans indefinitely.
• Measure the lag rather than guessing: pick 20 products changed in the last week and check how long each took to appear correctly in the index.
• Two-speed retrieval is the durable answer — a slow semantic index for finding candidates, a live authoritative call before anything is quoted or committed, as set out in grounding an assistant in a catalogue that moves.

Cause 2: the indexed document has no field that says the product is gone

Retrieval can only filter on fields that exist in the document. A pipeline that indexes title, description and category — the 3 fields that make embeddings look good in a demo — leaves the retriever with nothing to exclude on. The document then competes on relevance forever, and the product sounds appealing because the copy was written when it was appealing.

The fields worth carrying on every indexed document are unglamorous: lifecycle state, published state per market, availability at index time, the last-updated timestamp, and the identifier that joins back to the catalogue of record. That list is a subset of the readiness gate in what has to be true before an assistant may quote price and stock, and skipping it is why so many assistant projects stall at the demo.

A related variant hits configurable products specifically: the parent is live, 1 variant is withdrawn, and the indexed document describes the parent. The assistant recommends a combination that cannot be bought, which is the same defect that produces orders arriving for combinations that do not exist. If your catalogue has variants, the index has to hold their state, not the parent's summary.

Cause 3: the model answered from memory because nothing made it call the tool

If the trace shows no tool call, retrieval is innocent. The model produced a plausible answer from training data, from a summary sitting in the system prompt, or from earlier turns in the conversation where a product was mentioned while it was still live. Long conversations make this worse: a product named 12 turns ago is in context, and context is cheaper to use than a tool.

1. Make product-specific claims structurally impossible without a tool call — no name, price, availability or policy statement unless it came from a response in this turn.
2. Give the assistant a way to say it does not know, and treat refusal as a passing outcome in evaluation rather than a failure, as an evaluation loop for a shopping assistant argues.
3. Expire product facts from context. A recommendation made 10 turns ago is not evidence, and re-checking before a second mention costs 1 call.
4. Add withdrawn products to the frozen evaluation set. This regression only appears when something has been removed, so a test set built from live products cannot catch it.

Cause 4: the tool returns rows the storefront would never show

This is the cause that appears when an internal API gets wrapped in a hurry. Admin and PIM endpoints exist to show merchandisers everything — drafts, scheduled products, regional delistings, items live in 1 market and withdrawn in another — and the filters that hide them are applied by the storefront, not by the data layer. Wrap the endpoint and the assistant inherits the merchandiser's view of the catalogue.

• Set the read boundary explicitly: published catalogue, live availability, published price, scoped to the requesting market. The full argument is in what a catalogue tool server may expose.
• Make the market a required argument rather than a default, because "available" without a market is a claim that is true somewhere and wrong here.
• Shape responses so the assistant cannot misread them: a status field the model has to acknowledge beats a silently absent one, which is the response-shaping point in designing the tools a shopping agent may call.
• Test the tool independently of the assistant, with withdrawn products as fixtures. Most teams only ever test it through the assistant, where a wrong row and a good answer can coexist.

What a correct fix still will not do

Fixing all 4 causes fixes the assistant you control. It does nothing about the copy of your catalogue sitting in a third-party surface, where an agent may hold a feed it fetched last week and answer from that. If discontinued products keep appearing outside your own assistant, the question is which of your published surfaces is stale, and the shape of that investigation is different — agents fetch your manifest and still skip your store covers the declaration side.

Two other symptoms look related and are not. If agents cannot reach your data at all, the cause is usually your own edge rules rather than your catalogue — bot protection blocking the buyers you want. And if the products are right but the numbers disagree at checkout, that is an authority problem between the agent's rendered state and your session, handled in the agent shows one total and your checkout session returns another.

An assistant that recommends what you stopped selling is usually telling the truth about your data. The uncomfortable part is that a customer asked before anyone internal did.

Run the check before opening the prompt file — the answer is almost always in the trace, and the fix is almost always in the data layer rather than the model. Building that layer properly, with a read boundary and a schema somebody owns, is product build work. The rest of this silo sits under agentic commerce and AI shopping assistants, inside our retail and ecommerce practice.

• agentic commerce
• retrieval
• catalogue data
• diagnostics