The Cron Job That Admits Your Webhooks Failed
Open the codebase of any team that has run webhooks in production for more than a year. Somewhere in the scheduler there is a nightly job that pulls the provider's list endpoint, walks every record, and reconciles it against the local copy. Nobody is proud of it. It exists because the webhooks it backs up cannot be trusted to keep two systems in agreement, and everyone who wrote it knew that when they shipped it.
That job is the subject of an argument that got a lot of attention in August. A post titled "The Valley of Webhooks," published on 5 August 2026 and pushed to 247 points on Hacker News, calls the reconciliation cron a written confession. You built an entire delivery pipeline to receive changes, and then you built a second pipeline that assumes the first one lost some. The author's point is sharper than the usual webhook complaint: the problem is not your retry logic. The problem is the shape.
What "The Valley" Actually Means
The framing borrows from evolutionary biology. Picture a fitness landscape where every design sits at some height, and better designs sit higher. Webhooks-for-replication sit in a valley: a local optimum you can only leave by first getting worse. Around that valley is a ring of mitigation infrastructure that keeps you alive but never lets you climb. The better peak across the gap is a pull-based log, and almost nobody crosses to it because crossing means throwing away the pipeline you already paid for.
I find the metaphor more useful than it has any right to be. It explains why smart teams keep building the same broken thing. You are not choosing webhooks for replication because they are good at it. You chose them because the first event you ever integrated was a notification, the SDK handed you a POST handler, and replication got bolted onto the same channel by accident. The valley is where you land when you never questioned the direction of the arrow.
The Stack You Built to Survive Push
Look at what receiving pushed events for replication actually costs you. You verify a signature on every request because the endpoint is public and anyone can hit it. You keep a dedup table because delivery is at-least-once and the same event arrives twice during a retry storm. You buffer handlers because events land out of order and a deleted can beat the created it depends on. You write a bootstrap importer with a locking scheme so the initial backfill does not race live traffic. And you run the reconciliation cron on top of all of it, because none of the previous layers can prove you did not silently drop an event that never retried.
That is five distinct pieces of infrastructure, each with its own failure mode, all keeping a replica honest. We have written about most of them as if they were unavoidable: the reconciliation backstop, the state-sync gap that opens when a delete never arrives, the ordering buffers you need when sequence matters. The valley argument is that they are unavoidable only inside the push model. Change the direction and most of them lose their reason to exist.
SCROLL, in One GET Request
The author did not stop at a critique. They drafted a protocol, draft-scroll-protocol-00, published at welidev.github.io/scroll. SCROLL stands for Synchronized Change Replication Over Line Logs, and the whole idea fits in a single request. Instead of the provider pushing to you, you read an ordered feed:
GET /scroll/feed/customers?scroll-cursor=01J9XQ4R
Prefer: stream
The response is newline-delimited JSON, one change per line, streamed or paged:
Content-Type: application/x-ndjson
Preference-Applied: stream
{"cursor":"01J9XQ4S","operation":"upsert","object":{"id":"cus_42","email":"a@b.co","plan":"pro"}}
{"cursor":"01J9XQ4T","operation":"delete","object":{"id":"cus_17"}}
{"cursor":"01J9XQ4U","operation":"tombstone","object":{"id":"cus_09"}}
Every line carries three fields: a cursor that durably marks your position, an operation that is one of upsert, delete, or tombstone, and the full object state. The Prefer: stream header asks for a persistent connection; drop it and the same endpoint pages. There is no callback URL to register, no secret to rotate, no separate webhook API. The provider reuses the same credential you already use for the rest of its API.
Why a Pull Feed Deletes Your Infrastructure
Walk the same five-piece stack again against this shape and watch it fall away. Bootstrap is not a special import anymore; you read the feed from cursor zero and you have the whole dataset, then you keep reading and you have every change since. There is no race between backfill and live traffic because they are the same stream. Ordering buffers disappear because a log is ordered by definition. Dedup tables disappear because every line carries full state, so a blind upsert is idempotent whether you apply it once or five times, which is exactly the property we argue for when we talk about idempotent consumers generally.
The reconciliation cron disappears too, and this is the part I like most. The feed ends with a count and a checksum computed at the cursor you now hold. You compare your replica's checksum to the provider's. If they match, you are provably in sync, not hopefully in sync. Silent deletes stop being silent because a tombstone is a logged event with a cursor, not the absence of one. The whole category of "an event that never arrived and never alerted" is closed by making absence impossible to express.
The Endpoint Problem It Quietly Dodges
There is a second, less obvious win. Because the consumer opens the connection, there is no public ingress to defend. No inbound endpoint means no SSRF surface, no signature verification, no allowlist of sender IP ranges that changes without warning. The credential is the one you already manage for the provider's REST API. For teams behind NAT or a corporate proxy, where standing up a reachable webhook receiver is its own project, a pull feed sidesteps the networking problem. You make outbound calls, which your infrastructure already permits.
That is not nothing. A meaningful share of webhook incidents are really ingress incidents: a firewall change, an expired certificate, a load balancer that returns 503 under load and gets the endpoint disabled by the sender. Remove the endpoint and that failure family goes with it.
Where the Change Feed Still Leaves You Exposed
Now the honest part, because SCROLL is not a free lunch. A pull feed is worse than a webhook at exactly one thing, and it is the thing webhooks were invented for: telling you something happened the instant it happens so you can trigger a side effect. If a payment succeeds and you need to send a receipt within a second, polling a feed on a five-second interval is a regression, and streaming mode means holding a persistent connection open per resource type per consumer, which pushes load back onto the provider in a new shape.
There is also the matter of who pays for the read. Push spends effort at the moment work actually occurs. Pull means someone asks "anything new?" on a schedule even when nothing changed. At low change rates that is wasted work. At high rates the streaming variant helps, but now you run long-lived connections and you are back to caring about reconnection, cursor persistence across restarts, and backpressure when your consumer falls behind. The infrastructure does not vanish. Some of it moves.
Draft-00 Is Not a Dependency
Here is where I get blunt about adoption. SCROLL is at draft-scroll-protocol-00. That is the earliest possible version number, authored by one person, implemented by zero providers you actually integrate with. Stripe is not shipping a SCROLL feed this quarter. Neither is GitHub, Shopify, or Twilio. Rewrite your ingestion layer around a draft-00 spec because a good blog post convinced you, and you have traded a known-imperfect system for a bet on a protocol that may never leave draft. That is not crossing the valley. That is walking into a different one with worse maps.
Treat the draft as what it is: a clear articulation of a direction, not an API you can call. Its value right now is that it names the shape precisely enough for you to recognize where you already have that shape available and are not using it.
What You Can Adopt Today Without SCROLL
Because the shape is not new, and several providers already expose most of it under a different name. Stripe retains 30 days of events and offers /v1/events as an ordered log you can page with a cursor. WorkOS ships a cursor-paginated Events API built for exactly this. These are not full SCROLL feeds, but they give you the two properties that matter: an ordered stream and a durable cursor. That is enough to replace the reconciliation cron with a resumable catch-up read, and enough to bootstrap a replica without a bespoke import job.
The migration path the author suggests is a shim, and it is the pragmatic move. Keep your existing webhook receiver as a low-latency nudge for side effects, and point replication at the ordered events log behind it. The webhook says "something changed, go look"; the feed is the source of truth you reconcile against. That extends the fetch-on-notify pattern we already recommend from a single object fetch to a cursor-addressed catch-up. You do not need a new protocol to start. You need to stop treating the push channel as your replica's source of truth.
The Honest Split: Notifications Versus Replication
Strip the argument down and it is about conflating two jobs. Webhooks are excellent notifications. They are a mediocre replication transport that we reached for because it was already there. The valley post, and the SCROLL draft under it, are worth your time. Not because you should switch protocols next sprint, but because they force the question you skipped: is this event a nudge to do something, or a claim about state I need to keep true over time? For the first, keep your webhook. For the second, the arrow should point the other way, and you can start pulling today with the Events API your provider already ships.
Frequently Asked Questions
Should I replace my webhook receivers with SCROLL right now? No. SCROLL is at draft-00 with no production providers implementing it, so there is nothing to call. Adopt the shape instead: point state replication at your provider's existing ordered events log and keep webhooks for low-latency side effects. Rewriting ingestion around a single-author draft is a bet, not a migration.
Does a pull feed mean I can delete my reconciliation cron? Eventually, yes, and that is the point of the design. A feed that ends with a count and a checksum at your current cursor lets you prove your replica matches the source rather than hoping it does. Until your providers expose that, a cursor-paginated Events API still lets you replace the nightly full-scan cron with a cheaper resumable catch-up read.
Why is at-least-once delivery not a problem for a change feed? Because every line carries the full object state, applying the same change twice produces the same result as applying it once. A blind upsert is naturally idempotent, so you can drop the dedup table you needed to protect against duplicate webhook deliveries. Reprocessing from an old cursor after a crash is safe for the same reason.
What does a change feed do worse than webhooks? Latency and provider load. Pushing tells you the instant something happens; polling adds interval delay, and streaming means holding persistent connections open that cost the provider resources. For side effects that must fire within a second, a webhook is still the better tool, which is why the practical answer is to run both rather than pick one.
Which providers already give me an ordered events log to pull from? Stripe retains 30 days of events and exposes a cursor-pageable list at its events endpoint, and WorkOS ships a cursor-paginated Events API designed for replication. Neither is full SCROLL, but both give you the ordered stream and durable cursor you need to reconcile state and bootstrap a replica without a custom import job.
