Available in search, gone at checkout: locating the stale layer

A buyer sees a listing as bookable, gets to checkout, and is refused. There are four layers that can be holding the wrong answer, and one comparison tells you which family you are in: at the moment of the failure, ask the source of truth the same availability question the search result claimed to answer, and put the two answers side by side. If they agree, nothing is stale and your read path and write path are applying different rules. If they disagree, a copy is out of date and the interesting question is which copy and for how long.

Almost every team starts by making the copy fresher. That is the right emergency measure and the wrong fix, because the version of this bug that survives a year is the one where two pieces of code both believe they know what "available" means.

Ask the database the question while the failure is still warm

1. Instrument the refusal. When checkout rejects, log the listing id, the exact requested interval, the rule that refused it, the request timestamp and the search request id that produced the impression.
2. Replay the authoritative predicate. Run the write path's own availability check against the primary database for that listing and interval, as close to the failure time as you can get.
3. Fetch the indexed document for the same listing and read its availability fields and its indexing timestamp.
4. Record three values per incident: authoritative answer, indexed answer, and the age of the indexed document. Twenty incidents is enough to see the pattern; one is a story.
5. Group by listing, by hour and by the age figure. Layers separate cleanly under that grouping, and they rarely separate any other way.

Answering that reliably means being able to say what availability was at 14:32, not just what it is now. A platform whose orders carry only a current status cannot reconstruct it, which is one of the practical arguments in an append-only event log or a status column. If you cannot reconstruct the moment, add the logging first and accept that this week's incidents are lost.

Four layers, and the timing signature that identifies each

The fourth row is the one worth hoping for, because it is deterministic and therefore fixable. The first three are races, and races are fixed by narrowing windows until the remaining failure rate is acceptable — which is a maintenance commitment rather than a repair.

Freshness is a schedule, not a promise

A search engine does not make a write visible the instant it accepts it. In Elasticsearch the refresh interval defaults to 1 second on the Elastic Stack and 5 seconds on serverless, where 5 seconds is also the floor. More surprising is what happens to quiet indices: shards that have seen no search traffic for the search-idle period — 30 seconds by default — stop receiving background refreshes until a search arrives. A low-traffic marketplace can therefore have an index that is nominally near-real-time and in practice refreshes only when someone looks.

• Measure lag as a number, not as a setting. Emit the current time minus the newest indexing timestamp in the index, per minute, and chart it against the failure incidents.
• Watch the queue, not the engine. Most staleness in practice is a backlog in whatever ships changes into the index, and it shows up as lag that grows through the day and drains overnight.
• Treat bulk reindexing as an outage class. A full rebuild that swaps an alias at the end is safe; one that updates documents in place leaves a moving window of listings answering from a half-built state.
• Keep availability out of the index where you can. Indexing what a listing is and asking the database what it can do costs one query and removes an entire layer from this table.

The hold that took capacity and never gave it back

This one has a distinctive shape: one listing refuses every buyer, no order exists to explain it, and it fixes itself later on its own. Something is consuming capacity that nobody can see — a checkout hold from an abandoned session, a payment authorisation still open, a reservation created by an admin action that took a different code path. Search does not count holds, checkout does, and the two disagree until the hold dies.

Holds that depend on a scheduled job to expire fail in exactly this way when the job is late, dead, or running in a timezone nobody checked. Making expiry a property of the row rather than of a cron run is the subject of expiring a hold without trusting a cron, and the fields a hold needs before any of that works are in the hold row that stops the second buyer.

The rule that is written twice

When both answers agree and checkout still refuses, nothing is stale. Two implementations of "available" exist, and the simpler one is the one buyers see. The read path was built for speed over a denormalised document and quietly dropped the conditions that are awkward to precompute.

• Buffers between bookings. Turnaround, cleaning and travel time consume capacity that is not part of the booked interval, so search sees a free window that checkout knows is unusable.
• Lead time and cutoffs. A same-day booking made at 19:00 against a 12-hour notice rule is available in the index and rejected by the rule.
• Minimum and maximum duration. A 1-night request against a 2-night minimum is a valid interval and an invalid order.
• Seller-level state. Pauses, blackout dates, capacity caps and verification status usually live outside the listing document.
• Timezone and daylight-saving handling. Two implementations of local time diverge twice a year, and the failures land on exactly the dates nobody is testing.

How many of these exist at all is decided by the granularity chosen early — the trade in fixed slots or open intervals — and by whether constraints like travel time between jobs were modelled as part of availability or bolted on afterwards, which is the reasoning in forty cleaners, one city, Saturday morning. Both are on the list of modelling decisions that stop being reversible once real supply arrives.

Search is allowed to be optimistic. Only the write path is allowed to be right — and this bug is what happens when someone lets search be right as well.

Fresher copy, or one rule with two callers

1. Answers agree: extract the predicate. Move the write path's check into one function, have search call it for the candidates it is about to show, and delete the second implementation rather than aligning it.
2. Answers disagree and converge within seconds: fix invalidation on write, and add the release paths that were missed. Measure lag afterwards to confirm the window actually narrowed.
3. Answers disagree persistently on one listing: hunt the hold. Expect an expiry mechanism that depends on something running on time.
4. Answers disagree only around specific dates or hours: this is rule divergence wearing a staleness costume. Go back to the first branch.
5. Whatever the branch, make the refusal legible to the buyer. "This slot was taken while you were deciding" holds a session that a generic error loses.

Then keep it fixed with a canary rather than a memory. Sample a few hundred live listings an hour, ask both paths the same question, and alert on any divergence above a threshold you have chosen deliberately. It runs in a few seconds, it catches the next regression before a buyer does, and it is the kind of unglamorous instrumentation we build into an MVP or product build because it pays for itself the first week it fires. The rest of the transaction model — listings, holds, orders and the transitions between them — sits under marketplace architecture and the transaction data model, part of our marketplaces and platforms practice.

• availability
• search index
• caching
• marketplace architecture