Renewals that never reached the access system

A renewal charged successfully and the reader lost access anyway, which means the money moved and the message about it did not arrive, or arrived and changed nothing. Before theorising, measure: pull the provider's own list of events for a 24-hour window, pull the events your system recorded as processed for the same window, and count the difference by event type. That single comparison splits the problem in two, and the two halves have almost nothing in common — one is a delivery problem outside your process, the other is a bug inside it.

What happens to access while a payment is failing is a different subject with its own states, set out in dunning as a state machine. This page is about events that should have restored or extended access and did not.

One day of provider events against one day of your own

1. List every event the provider generated in a 24-hour window, using its API rather than your inbox of received payloads. That list is the denominator and nothing else can be.
2. List every event your system recorded, with its provider event id. If you have no such table, stop and build it: without event ids stored, the rest of this is guesswork.
3. Anti-join on the event id and group the misses by type and by hour. Record the count, not just the examples, because the shape is the diagnosis.
4. Separately, count events you received but whose effect never landed — received rows with no corresponding entitlement change. These are the expensive ones, because retries will not fix them.
5. Repeat over 7 days. A one-off cluster and a steady daily trickle are different failures with different owners, and one window cannot tell them apart.

Five ways a delivered event changes nothing

• The handler is not idempotent, so retries are dropped. Providers redeliver: Stripe's documentation states that endpoints might receive the same event more than once, and recommends logging processed event ids and skipping repeats. A handler that crashes halfway and refuses the retry as a duplicate has lost the event permanently.
• An older state is applied last. Delivery order is not promised — Stripe's documentation says events are not guaranteed to arrive in the order they were generated, and gives the subscription example where the created, invoice and charge events can land in any sequence. Apply an older snapshot after a newer one and you overwrite a renewal with the state that preceded it.
• The endpoint failed during a deploy or a secret rotation. Signature verification returning an error, or a 500 from a restarting process, for the 4 minutes of a rollout. Stripe retries for up to 3 days in live mode with exponential backoff, so a short outage self-heals and a long one does not.
• The handler throws and the error is swallowed. A 200 was already returned, or the exception is caught and logged at debug level, so the provider believes delivery succeeded and never retries. This is the failure mode that produces the steady trickle in the table above.
• The event refers to a product the code has never heard of. A new plan, a campaign price or a gift product is created on the commercial side, the handler's mapping has no entry, and it falls through to a default that does nothing. Redemption flows are especially prone to it — the orphan-account problems in gift and redemption flows usually start here.

Order is not promised, so the write has to survive being wrong

Handlers are usually written as if events arrive in the order things happened. They do not, and the fix is not to enforce ordering but to make each write safe under any ordering. Every subscription state you receive carries something monotonic — a period end, an updated timestamp, a version counter — and the rule is to apply an incoming state only when its marker is newer than the one already stored, discarding it otherwise.

1. Store the marker you compared against, alongside the entitlement. A stale-event rejection you cannot explain later is indistinguishable from a bug.
2. Make the event id a unique constraint in your ledger, so a duplicate delivery fails to insert rather than running twice.
3. Separate receipt from effect. One row records that the event arrived; another records what it changed. When those two counts diverge you have found processing loss without any further investigation.
4. Where the event does not carry enough state to decide, fetch the current object from the provider's API rather than reconstructing it from the payload. The provider's documentation recommends exactly this for out-of-order arrivals, and it removes a whole class of ordering bugs.
5. Log a reason code on every entitlement change: which event, which handler, which marker. Six months later, that is the only thing that answers why a reader's access ended on a particular Tuesday.

Webhooks are a latency optimisation. Treating them as the source of truth is how a subscription business ends up with an access state nobody can reconstruct.

The provider holds the record; your database holds a projection

No amount of handler hardening removes the possibility of a missed event, because the delivery path crosses networks, deploys and rate limits you do not control. The durable design accepts that and adds a second path: a scheduled job that reads current subscription state from the provider and makes your entitlements agree with it. Events keep access fast; reconciliation keeps it correct. Deciding which system owns a record and which merely reflects it is the same argument made about editorial data in CMS fields or a rights service, and it resolves the same way: one owner, everything else a projection that can be rebuilt.

• Run it over a rolling window — the last 7 days of subscriptions with any change — every hour, and over the full active base nightly.
• Give it authority to grant, extend and correct, and to revoke only with a reason code recorded. A job that silently removes access is worse than the bug it was built to fix.
• Alert on the volume it repairs, not on whether it ran. Zero corrections is healthy; 40 in an hour is an incident in the event pipeline that nobody would otherwise see.
• Write its output through the same access resolver every surface reads, so a repaired grant is visible everywhere at once — the design in one access check every surface can call.
• Give support a screen showing, for one reader, every event received, every entitlement change and every reconciliation touch, in time order. That console is the difference between a five-minute answer and a ticket that escalates to engineering, and it is the sort of thing we build as internal tools and ops software.

Why the complaint arrives days after the failure

A subscriber whose renewal never landed does not hit a wall immediately. On most sites they quietly drop back to the metered population and read 3 or 4 articles first, so the report arrives detached from the event that caused it — the behaviour described in the meter resets in a private window. That delay is why the reconciliation count matters more than the ticket count: by the time readers complain, the pipeline has been dropping events for days, and the number of affected subscribers is always larger than the number who bothered to tell you.

The rest of this silo sits under paywalls, subscriptions and entitlements, inside our media and publishing practice.

• webhooks
• subscriptions
• entitlements
• reconciliation