CI credentials and expiry

Constellation's CI workflows authenticate to Project Tracker and the wiki with Directory API keys stored as GitHub Actions secrets. Keys issued to an admin role default to a 14-day lifetime and are capped at 30 days by default — API_KEY_TTL_MAX_ADMIN_DAYS can raise that cap, up to the platform's absolute ceiling of 365. Non-admin keys are capped at 365. The CI credentials here are admin-issued, so 30 days is the working assumption, but it is a configurable default and not a law.

This page is the operator's reference for what breaks when one expires, how you are told, and how to rotate.

Why this page exists

On 2026-07-26 20:47 UTC the key behind WIKI_SERVICE_TOKEN reached its expiry. The next Spec → Wiki Sync run failed, and so did every run for the next five days. Nothing had noticed, for two reasons:

1. The sync reported only 401 UNAUTHORIZED — while holding a token whose own exp claim said, in plain text, that it had died two hours earlier.
2. Nothing read that claim ahead of time. Expiry is a function of the calendar, not of a diff, so no PR gate can catch it.

Because that sync is the GitHub → wiki edge of the spec → wiki → knowledge-base funnel, the effect was not cosmetic: no spec mirror advanced to approved, so nothing was nominated for the KB, so the next release's curation pass found an empty ingestion queue. Tracked as INF-357.

The watched credentials

Secret	What stops working when it expires
WIKI_SERVICE_TOKEN	the spec → wiki sync (PLT-302), and with it the KB ingestion-candidate funnel (PLT-269)
PT_AUTH_TOKEN	the multi-LLM PR reviewer and every CI step that reads Project Tracker

The list is defined in WATCHED_TOKENS in scripts/check-ci-token-expiry.ts. Adding a new CI credential means adding a row — a secret missing from that list is simply unwatched, which is the exact failure mode above.

How you are warned

Two mechanisms, deliberately different in strength:

• CI Credential Expiry — a scheduled workflow (Mondays 07:00 UTC) that fails when any watched token is expired, is inside the warning floor, or is unreadable, and opens a tracking issue labelled ci-credential-expiry carrying the report and the run link. Set the CI_CREDENTIAL_OWNER repository variable to a GitHub login (or an org/team slug) so the issue is assigned and @-mentioned to a person — GitHub has no way to watch a label, and an unassigned bot-created issue notifies nobody at the default "Participating and @mentions" level. The scheduled workflow fails until that variable is set — deliberately, and on every run including passing ones, so the misconfiguration surfaces on a quiet Monday rather than in the middle of the first outage. The alert issue is still created regardless — fail-closed on configuration, never on delivery. It is not auto-closed while the recipient is unusable, and that asymmetry is deliberate: an alert whose subject is "nobody would be told" must not close itself before anyone is reachable to read it. Fix the recipient and the next passing run closes it.

gh variable set CI_CREDENTIAL_OWNER --repo B2B-Online/constellation --body "<github-login>"

A user login is positively confirmed before it is accepted: the workflow calls GET /repos/{owner}/{repo}/assignees/{login}, which answers exactly "can this user be assigned here" and needs only the issues permission the job already holds. A pass proves access, because GitHub refuses to assign a user without it. Anything else — nonexistent account, no repository access, a rate limit, any API error — fails the job, because access was never established.

A team is not accepted by default. It cannot be assigned, and its existence cannot be confirmed with a repository-scoped token (GET /orgs/{org}/teams/{slug} needs organisation Members-read). An unverifiable recipient is indistinguishable from a mistyped one, so it fails closed. If you want a team anyway, check the slug by hand and set the acknowledgement explicitly:

gh variable set CI_CREDENTIAL_OWNER_TEAM_UNVERIFIED_OK --repo B2B-Online/constellation --body "<org>/<team>"

Set it to the team slug itself, not true. The acknowledgement is bound to the slug it acknowledges, so changing CI_CREDENTIAL_OWNER later re-opens the question instead of inheriting a stale yes for a team nobody checked. It is a logged decision rather than a silent default, and every accepted run warns that the slug is unverified.

It comments on the existing issue rather than opening a new one each week, and closes it automatically once the check passes — so an open issue always means "act now". Alerts are found by their label, and also by a hidden marker in the body, so an alert opened while the label was momentarily uncreatable is still updated and still auto-closed (it is re-labelled as soon as label access recovers). Do not delete that marker comment from an issue — an alert without it can only be closed by hand. The marker is not a credential and grants nothing: every lookup additionally requires the issue to have been opened by this workflow's own bot, so pasting the marker into an unrelated issue does not hand it to the automation. The issue exists because a red scheduled run is not a delivered signal: that is precisely what went unnoticed for five days.

• A preflight on each consuming workflow — warn-only, at a 14-day floor, so the notice appears in a workflow people already read rather than only in one they might not. Both are continue-on-error and can never stop the real work from running:
  • Spec → Wiki Sync checks WIKI_SERVICE_TOKEN.
  • Multi-LLM review checks PT_AUTH_TOKEN. This one matters more than it looks: an expired PT token does not fail the review — the reviewer degrades to flat-file grounding and posts anyway, losing its PT-ticket and KB grounding while still going green. Nothing else would tell you.

The warning floor (14 days) is deliberately longer than the weekly schedule, and shorter than the 30-day key lifetime you should be issuing — those two constraints are what fix it at 14, and a key minted for 14 days satisfies neither. With a 7-day floor and a weekly cron, a token expiring just after next Monday's run sits outside the floor today, so this Monday's run does not flag it — and the next one catches it minutes before it dies. If the cron is ever slowed, the floor must go up with it.

Scheduled workflows are evaluated from the repository's default branch, which here is develop — so the monitor activates on merge to develop and does not wait for a release to main.

It also runs on a merge to develop that touches its own workflow file. That is the bootstrap: a schedule-only workflow does not exist until its first tick, so a monitor merged after a Monday 07:00 UTC cron would sit inert until the following Monday with no earlier run to re-run — which for INF-357 would have been six days past the expiry it was built for.

There is deliberately no manual trigger. (The bootstrap above is not one: a push runs develop's own post-review workflow and code, the same trust level as schedule, with no ref for a caller to choose.) A workflow_dispatch lets the operator choose any ref, and this job hands both production secrets to that ref's copy of the script — so an unreviewed branch could exfiltrate them, and an in-workflow ref guard cannot prevent it because the chosen branch supplies the workflow file too. To check on demand, re-run the most recent scheduled run instead; a re-run replays the original event and ref:

gh run list --workflow ci-token-expiry.yml --limit 1

gh run rerun <id>

You can also run it locally against secrets already in your environment (the default 14-day floor is the right one — 30 days is the DEFAULT cap on an admin-issued key, not a ceiling: API_KEY_TTL_MAX_ADMIN_DAYS can raise it, clamped to the platform's absolute 365, and a non-admin key defaults to 365 already. A token cannot reveal the configured value, so the check names the default rather than claiming no longer key exists):

npm run check:ci-token-expiry

Rotating a key

Rotation is a human action — agents never source credentials.

0. Read which secret the warning names. WIKI_SERVICE_TOKEN and PT_AUTH_TOKEN are separate credentials with separate consumers; every step below depends on which one you are rotating.

1. In Directory, go to API keys and issue a new key. Give it a label that says where it is used (e.g. Github, Multi-LLM Github) so the next person can match a secret to a row in identity.api_keys.

   Set Expiry explicitly — do not leave it blank. Left blank, an admin-issued key defaults to 14 days (ttlCapsForRoles in apps/directory/src/server/services/api-key.service.ts), which is exactly the warning floor below — so the replacement is flagged the moment it is created, and rotating again the same way produces another one. Choose the longest expiry the form offers you. The maximums are 30 days for an admin role and 365 otherwise — those are caps, not defaults, and the default is the 14 days named above. Both caps are overridable server-side (API_KEY_TTL_MAX_ADMIN_DAYS, API_KEY_TTL_MAX_USER_DAYS, each clamped to an absolute 365), so trust the form over any number written here. The expiry check names this case explicitly rather than repeating "rotate it", so you are not sent round the loop.

2. Update the GitHub Actions secret named in the warning — there are two watched credentials and they are not interchangeable. Use the prompting form so the value stays out of your shell history:

   gh secret set WIKI_SERVICE_TOKEN --repo B2B-Online/constellation

   gh secret set PT_AUTH_TOKEN --repo B2B-Online/constellation

3. Re-run the workflow that consumes that secret and confirm it goes green:

   Rotated secret	Verify with
   WIKI_SERVICE_TOKEN	Spec → Wiki Sync. It only triggers on a push touching .ai/specs/**, so use the Actions Re-run button on the most recent failed run rather than waiting for the next merge.
   PT_AUTH_TOKEN	Multi-LLM review on any open PR. Note it does not go red on an expired PT token — the reviewer degrades to flat-file grounding and still posts. Confirm from the run log that PT-ticket and KB grounding ran, not just that it is green.

4. If WIKI_SERVICE_TOKEN was the one that expired and the sync was down for a while, back-sync the mirrors that were missed — the sync only publishes specs changed by the triggering push, so specs changed during the outage never get picked up by the next merge on their own.

Reading a failure

A 401 or 403 from the wiki now diagnoses itself, naming whichever status actually came back. The error names whether the token is expired and by how long, its jti (which identifies the identity.api_keys row to replace), and the secret to update:

GET .../api/pages/by-path/default/specs → 401 {"error":{"code":"UNAUTHORIZED",...}}
    ↳ WIKI_SERVICE_TOKEN EXPIRED at 2026-07-26T20:47:14.000Z (3 day(s) ago), jti ef459a75-… —
      that is almost certainly this 401. Rotate it: issue a fresh Directory API key, then update
      the WIKI_SERVICE_TOKEN GitHub Actions secret.

A 403 the wiki produced is not a credential failure and the diagnosis says so: every bearer problem — missing header, malformed token, expiry — returns 401 from withAuth, so a 403 from the wiki is only reachable after authentication has already succeeded. That guarantee is about the wiki's 403, and the diagnosis checks: a 403 whose body carries none of the platform's FORBIDDEN envelope may have come from Deployment Protection, a WAF, a CDN or a proxy, which answer before any bearer is examined — there it establishes nothing about the credential, and any token defect is reported as a second problem. Two things produce one, in the order they are evaluated:

1. Tenant context — withValidatedTenant denies when the key's user has no active membership, targets the wrong tenant, or the tenant is disabled.
2. Permissions — the policy layer denies when the roles lack what the route needs (e.g. wiki:page:write).

First check the 403 actually came from the wiki. Vercel Deployment Protection, a WAF, a CDN or any reverse proxy in front of it answers 403 before a bearer token is examined — so a body naming neither wiki denial proves nothing about the credential, and the diagnosis says so rather than certifying authentication. Verify WIKI_ZONE_URL points at the intended deployment and that it is not access-protected.

When the body IS one of the wiki's own denials, read it — it is printed with the error. A permission denial names the missing permission outright; only the tenant-context denial is deliberately generic, and it is that case where you check membership before roles.

A 403 never hides a broken credential from you. Whatever the body says, the diagnosis also reports what the stored token itself looks like — expired, carrying no exp, or unparseable — because a 403 you cannot attribute is precisely the case where nothing has examined the bearer yet. There the credential defect is a second problem: fix the access protection and the next request fails with a 401 instead. Re-issue the key as well as fixing the routing.

If the 403 came from the wiki itself and the token nonetheless looks unparseable or has no exp, do not re-issue on that basis. The wiki answers 403 only after authentication succeeded, and auth-core refuses a token without exp, so the two facts cannot describe the same request — the value being inspected is not the value that was sent (a stale local copy, or the wrong secret). Reconcile that first.

Whether you must re-issue the key depends on which of the two it was:

• Membership or tenant status — checked live by validateTenant. Re-activate the membership or re-enable the tenant and the existing key works again. Do not rotate.
• Wrong tenant or insufficient roles — signApiKeyToken embeds tenant_id and roles as a snapshot at signing time, and the policy reads them from the token, so no Directory-side change reaches the key you already hold. Fix it, then re-issue the key and update the secret.

If instead it says the token is unexpired and well-formed, expiry is not the cause. Note that wording carefully: this check never verifies signatures, so "not expired" is not "accepted" — a corrupted key, one signed with a different secret, or a server whose API_KEY_JWT_SECRET is unset or rotated all produce a 401 that lands there. The message names those causes first, then the one configuration check that applies: that the zone URL points where you think it does. It deliberately does not send you to check the tenant — validateJwt requires tenant_id to be present but never compares it, and tenant matching happens afterwards in withValidatedTenant, which denies with 403. A wrong tenant cannot produce a 401.

A 401 is attributed before expiry is blamed. Only a body carrying the wiki's own UNAUTHORIZED envelope means the wiki rejected the token. Vercel deployment protection answers 401 before any bearer is examined — the -planet-b2-b wiki aliases are protected that way — so against one of those hosts an expired key is a second problem and the first thing to check is that WIKI_ZONE_URL points at the intended, unprotected deployment. Rotating first there leaves the sync failing on the same 401.

Revocation is offered only where the consumer enforces it. For the wiki it is not — see Known limits below — so the wiki diagnosis says so explicitly instead of sending you to a table that cannot explain the failure.

Known limits

• Exit codes. 0 every watched credential is unexpired and structurally well-formed — not a certificate that it is accepted, because this monitor never verifies signatures · 1 a credential NEEDS ATTENTION — read its per-token remediation rather than assuming rotation, since exit 1 also covers a truncated or unreadable secret, a wrong credential type and missing required claims, and re-storing a fresh key the same broken way reproduces the fault · 2 the check is INCOMPLETE — at least one watched secret never reached the job · 3 both at once · 4 the monitor did NOT COMPLETE — it crashed before finishing, so its exit code is not a finding about any credential: nothing here says a token is broken, and nothing says it is fine. Fix the monitor and re-run; do not rotate on the strength of a 4. Note what 2 does not mean: a credential that WAS present is still conclusively checked and reported, so read the attached report for which ones were covered rather than discarding the run. What you must not do is treat an unchecked credential as healthy. The alert issue carries a different headline for each, because they need different actions.
• Revocation is not detected, and for the wiki it is not even enforced. revoked_at lives in the database, not in the token, so the expiry check cannot see it. It is consulted only by a consumer that passes a principalCheck to validateJwt — and the wiki wires none (apps/wiki/src/middleware.ts), so a revoked-but-unexpired key still authenticates there. The 401 diagnosis says so rather than sending you to inspect revoked_at for a wiki failure it cannot explain.
• The check reads tokens without verifying their signatures. It is a diagnostic, never an authorization input. Real verification is validateJwt in @constellation-platform/auth-core.
• last_used_at is not a reliable usage signal for wiki-facing keys — it stays NULL on a key the wiki has been accepting for a month, because the wiki app does not appear to wire the liveness hook that Project Tracker does. Do not conclude a key is unused from a NULL there.