Signal, score and sort order: three layers people keep collapsing

A ranker has 3 layers and each fails differently. A signal is a measurement taken from your own data — travel minutes, median reply seconds, cancellations in the last 30 days. A score is policy: the arithmetic turning several measurements into 1 comparable number. A sort is what the surface does with that number, including ignoring it when a buyer taps price, low to high.

Changing a signal changes what the system knows. Changing the score changes what it prefers. Changing the sort changes what a buyer sees regardless of either. Three fixes, 3 owners, 3 different ways to tell whether the change worked.

Signals are measurements, and they inherit the bugs of whatever emits them

A signal has a unit, a window and a source system. "Responds quickly" is not a signal; "median seconds to first reply over the last 20 offers, from the messaging service" is. Written that way you can test the measurement independently of the ranking, and say out loud what happens when the source is wrong.

• Every signal has a source that can lie. Response time built from call records is only as honest as the masking layer attributing those calls, and a masked call that connects to the wrong person poisons it before ranking sees it.
• Some signals are conditional on your own behaviour. Acceptance rate as a dispatch input has a denominator you choose, so it partly measures your targeting.
• A signal with no window has an infinite one. A lifetime average moves so slowly that a seller who fixed their behaviour in March still pays for January.
• Missing is not zero. A listing with no reviews has no evidence, not bad evidence, and encoding that as 0 is a score decision wearing data clothing.

Which measurements are worth taking first is a different question, answered in the first ranking function for a thin market. This layer only claims what is true. It decides nothing.

The score is policy, written as arithmetic

The moment 2 signals are combined into 1 number, somebody has stated a preference: this much distance is worth that much rating. That is commercial policy, and it belongs in writing with its reason beside it rather than discovered later by reading code. A weight is the most reviewable artefact in the pipeline and the least reviewed.

Everything that trades 1 quality against another lives here. A temporary boost for new supply is a score decision with a known cost — how a listing with no history earns its first impressions spends an exploration budget inside the score. Whether the weights are hand-set or learned is a mechanism question, argued in hand-tuned weights versus a learned ranker; the layer is the same either way.

The sort is where product rules overrule the arithmetic

The order on the page is not the score order. Between them sit rules nobody counts as ranking: the shipped default, the buyer-chosen sort, page size, dedup so 1 seller does not hold 4 of the top 5 slots, and reserved positions. Paid placement has its own argument — where sponsored results belong in the ranker — and it belongs in this layer, not the score.

In dispatch this layer is an assignment rather than an ordering. Holding requests briefly and solving them together, as in batched dispatch versus nearest-first, changes who gets the job without touching 1 signal or 1 weight. So does a volume cap — capping how much one seller can win constrains the result set, it does not state a preference.

One Saturday cleaning request, traced through all 3 layers

Producing that table for a live query is the point. A console that replays 1 request and prints the candidate count, the signal values, the score components and the constraints applied turns the next ranking argument into a 5-minute lookup — the sort of thing we build under internal tools and ops.

Three complaints, and the layer each one actually belongs to

Row 1 deserves care, because the answer is empirical: whether a handful of sellers taking every job is the ranker or the market is settled by re-ranking history without the suspected feedback loop.

A signal says what is true, a score says what you prefer, and a sort says what the buyer is allowed to see. Arguing about the third while editing the second is how ranking work goes in circles.

The discipline is to name the layer before proposing the change, and to log all 3 per request so the naming is evidence rather than opinion. The rest of this silo sits under matching, ranking and dispatch, inside our marketplace and platform practice.

• ranking
• search
• marketplaces
• matching