Your GitHub App's tokens just grew thirteenfold
A pull_request webhook arrives. Your handler reads installation.id from the payload, mints an installation token, caches it, and calls the API to post a check run. That flow has worked for years. This week it might be throwing value too long for type character varying(40) in a worker nobody has opened since it shipped.
On October 2, 2026, GitHub announced that the staged rollout of stateless installation tokens, which started on April 27, is complete. Every newly minted GitHub App installation token now uses the ghs_APPID_JWT format. The prefix is still ghs_. The length went from 40 characters to roughly 520.
GitHub's position is that nothing you should have depended on has changed. Permissions, repository scoping, the one-hour expiry and the REST endpoint that mints the token are all the same. Fair enough. But plenty of code depended on the length anyway, and a lot of that code lives in webhook handlers.
What GitHub actually changed
The old installation token was a random 40-character string: ghs_ followed by 36 alphanumeric characters. GitHub had to look it up on every request to know what it meant. The new one carries its own claims. It's a signed token with the app ID embedded, and GitHub says that makes issuance and validation faster and the API more reliable. Good trade for them. For you, it's a string thirteen times longer, with characters your old assumptions never allowed.
Three details from the changelog matter in practice.
Tokens minted before the switch keep working until they expire. With a one-hour lifetime, that grace period ended long ago. Every token your app holds right now is the new format.
The rollout took five months. Seen intermittent failures on some installations and not others since late April? This is a likely cause, and the staggered switch is why it was so easy to misdiagnose.
GitHub added a temporary request header, X-GitHub-Stateless-S2S-Token, so integrators could test their code against either format on demand. GitHub stops respecting it on November 30, 2026. More on that below.
Why the webhook handler is where it breaks
Most code that touches installation tokens runs because a webhook arrived. GitHub Apps work that way: an event comes in, the payload tells you which installation it belongs to, and you mint a token scoped to that installation before you can do anything useful.
So token handling sits on the hot path of event processing. If minting succeeds but storing or validating the token fails, the event fails with it. Token caching also tends to get written once, early, by whoever set up the app. Nobody rereads that code when GitHub publishes a changelog entry that says "unchanged" four times.
And the failure shows up somewhere you won't look first. Delivery succeeded and the signature checked out. The 500 or the dead-lettered job comes from a database column or a regex, and the stack trace points at your token cache. Nothing in it says GitHub.
The column sized for a token you no longer get
This is the most common break. Someone created a table to cache tokens per installation and sized the column to what they saw:
CREATE TABLE installation_tokens (
installation_id BIGINT PRIMARY KEY,
token VARCHAR(40) NOT NULL,
expires_at TIMESTAMPTZ NOT NULL
);
On Postgres the insert fails loudly. That's the good outcome. On MySQL without strict mode, the value gets truncated to 40 characters without complaint and the insert succeeds. The next API call presents a token that's 480 characters short and GitHub answers 401 Bad credentials. Now you're debugging an authentication failure that has nothing to do with authentication.
VARCHAR(255) won't save you either. It just moves the cliff. GitHub says to treat the token as an opaque string, so give it an unbounded type:
ALTER TABLE installation_tokens
ALTER COLUMN token TYPE TEXT;
Then check the other stores too. Redis values are fine, and so is a DynamoDB cache. A token written into a fixed-width field in an audit log, a session row, or a job payload column that someone sized "generously" at 128 will break.
Validation that encodes the old shape
The second break is defensive code that was correct for years. People validate inputs, and a token is an input:
// Legacy shape. Rejects every token GitHub now issues.
const TOKEN_RE = /^ghs_[A-Za-z0-9]{36}$/
if (!TOKEN_RE.test(token)) {
throw new Error('Invalid installation token')
}
A JWT is base64url segments joined by dots, and the new format also puts an underscore between the app ID and the JWT. That pattern allows neither. The guard rejects a perfectly valid token, the handler throws, and the event is gone.
Delete the check. If you want a sanity guard, check the prefix and stop there. Length limits and character classes on a credential you don't control are a bet that the issuer never changes it. GitHub just showed you how that bet ends.
Don't overcorrect by parsing the JWT. That it happens to be decodable is an implementation detail. The mint response gives you expires_at, and that's the only expiry you should read.
Your log redaction stopped redacting
This one breaks nothing, and that's why it's the worst of the three.
Plenty of teams scrub credentials from logs with a pattern list, and the installation token pattern is usually the legacy shape: ghs_ plus 36 alphanumerics. Now look at the new layout. After ghs_ comes the numeric app ID, then an underscore. The old pattern finds no run of 36 alphanumerics where it expects one, so the redactor walks straight past and the full token goes to your log pipeline.
Webhook handlers leak the most because that's where people log aggressively: request context, outbound API calls, the error object from a failed Octokit request with its headers still attached. If you followed our logging advice, you log a lot around each delivery. Make sure the scrubber covers what you emit now:
// Any ghs_ token, whatever comes after the prefix.
const TOKEN_RE = /\bghs_[A-Za-z0-9_.\-]+/g
// Belt and braces: strip Authorization values outright.
const AUTH_HEADER_RE = /(authorization:\s*(?:token|bearer)\s+)\S+/gi
export function scrub(line: string): string {
return line
.replace(TOKEN_RE, '[REDACTED]')
.replace(AUTH_HEADER_RE, '$1[REDACTED]')
}
Then search your log store for ghs_ back to late April. Anything you find has expired by now, since tokens last an hour. Delete them anyway. The hits also tell you exactly which code paths still write credentials to logs.
Headers, proxies and other places 520 characters can hurt
GitHub's checklist also mentions proxies that truncate long Authorization headers. A 520-character header is nowhere near the 8 KB defaults of common reverse proxies, so this is rarely the cause. If it bites, it bites in odd corners: an egress proxy with a tight header limit, an HTTP client wrapper that copies headers into a fixed buffer, a corporate gateway someone configured years ago.
Environment variables and CI secrets are a more likely place to trip. Passing a minted token to a subprocess or container as an env var works fine. Putting it in a git clone URL, as in https://x-access-token:TOKEN@github.com/..., also fits, but now a long credential sits in a URL that tools love to echo into logs. So the redaction point applies twice here.
The failure GitHub won't retry for you
This is why the change deserves more than a migration ticket: GitHub doesn't automatically redeliver failed webhook deliveries. Stripe retries for days. GitHub records the failure and moves on. If your endpoint returned a 500 because the token insert blew up, or missed GitHub's ten-second window because the worker kept retrying a doomed database write, that event won't come back on its own.
If you process asynchronously, which you should (see queue-based processing), it looks different but it's no better. You returned 202 the moment the event hit your queue, so GitHub's delivery log shows green. Then the worker fails on the token and the job lands in a dead-letter queue, or gets dropped after a few attempts. That's the 200-but-nothing-happened pattern: success on the wire, nothing in the business logic.
Either way, the evidence is on your side. Look for token-related errors since April 27, grouped by installation. Installations didn't all switch at once, so expect a slow drip of failures across a growing set of customers.
Recovering the deliveries you dropped
GitHub keeps the last three days of deliveries for an app's webhook and lets you redeliver them through the REST API. You authenticate as the app itself with its JWT. An installation token won't work here. Anything older than three days is out of reach, and for those you reconcile against the API instead, which is the case for an Events API backstop.
Once your fix is deployed, a short Octokit script covers the three-day window:
import { App } from 'octokit'
const app = new App({ appId: process.env.APP_ID!, privateKey: process.env.APP_PRIVATE_KEY! })
const since = Date.now() - 3 * 24 * 60 * 60 * 1000
const deliveries = await app.octokit.paginate('GET /app/hook/deliveries', { per_page: 100 })
const failed = deliveries.filter(
(d) => d.status_code >= 400 && !d.redelivery && Date.parse(d.delivered_at) >= since,
)
for (const d of failed) {
await app.octokit.request('POST /app/hook/deliveries/{delivery_id}/attempts', {
delivery_id: d.id,
})
console.log(`redelivered ${d.guid} (${d.event})`)
}
That only catches deliveries GitHub saw fail. If you acknowledge first and process later, GitHub thinks everything went fine, and your list of what to replay comes from your dead-letter queue. Your handler also has to be idempotent on X-GitHub-Delivery before you replay anything. A redelivery carries the same GUID, and some of those events may have half-completed the first time.
Treat the token as opaque, then prove it
The fix itself is boring. Store the token in an unbounded field. Skip shape validation and parsing, read expiry from expires_at, and scrub by prefix. The harder part is confirming nothing else in your stack still assumes 40.
Grep is the cheapest audit you'll ever run. Search for ghs_, for {36}, for 40 near anything named token, and for VARCHAR in migrations that mention tokens. Check the GitHub client wrapper and any fixture files with a hardcoded legacy token. Tests using a fake 40-character token will happily pass against code that breaks on a real one. Swap those fixtures for something long, full of dots and underscores.
The header that disappears on November 30
If you used X-GitHub-Stateless-S2S-Token to test against a specific token format, take it out before November 30, 2026. After that date GitHub ignores it. Requests won't start failing. The header will just quietly do nothing, so any test that forced the legacy format stops testing what you think it tests.
Same story as the rest of this change. A provider can keep every documented contract and still break you, because your code encoded something it observed. Token length, header casing, field order, retry timing. Write down which of those your webhook handlers lean on, because that list is your exposure the next time a changelog says nothing changed.
Frequently asked questions
Does this change the GITHUB_TOKEN in Actions workflows?
The October 2 changelog doesn't say. It covers GitHub App installation tokens minted through the installation access token endpoint. If your workflows pass GITHUB_TOKEN into scripts that validate or store it, apply the same rules anyway: unbounded storage, no shape checks, redaction by prefix. It costs nothing and you stop depending on a detail GitHub never promised.
Should I decode the JWT to read the expiry instead of storing expires_at?
No. The mint response already gives you expires_at, and GitHub tells you to treat the token as an opaque string. Parsing the token's internals is the same kind of dependency that broke the 40-character assumption, and it will break the same way when the format moves again.
Can I still redeliver a GitHub webhook that failed a week ago? No. API redelivery only covers deliveries from the last three days. For anything older, rebuild state by calling the REST API for the affected resources, such as open pull requests or recent check runs per installation, and reconcile that against what your system recorded.
Is a leaked new-format token more dangerous than a leaked old one? It grants the same permissions and expires after the same hour, so the blast radius is unchanged. What changed is how likely it is to leak, because scrubbers written for the old shape miss it. Treat any token you find in logs as an incident even if it has expired, and fix the code path that put it there.
