6 · Rotation playbooks

Question this chapter answers: How do you replace a live credential without taking the product down?

Rotation is a deploy problem wearing a security costume. Every playbook below follows one pattern; only the console paths and constraints differ.

The zero-downtime dual-credential pattern

1. Create a second credential alongside the old one in the provider console. Nothing breaks; two valid credentials coexist.
2. Deploy consumers onto the new credential, one service at a time where possible. Watch error rates as each goes.
3. Verify traffic moved: the old credential's request volume trends to zero while the new one carries load.
4. Revoke or expire the old credential only after its request count has been flat at zero for a watch period (hours, not minutes — scheduled jobs are quiet for a reason).
5. Confirm dead: an authenticated call with the old value fails. Record revocation time in the rotation log (Appendix D).

Decision points:

• If the provider caps concurrent credentials (AWS allows two access keys per user — exactly enough for this pattern) → follow its cap.
• If the provider offers native rotation with a grace window → use it; it automates steps 1–4.
• If the credential is single-slot by design → schedule a brief swap: deploy new-value config, restart consumers, revoke old within the same window.
• If active abuse is underway → invert everything: revoke now, redeploy fast, accept downtime.

Stripe

Path: Dashboard → API keys (docs.stripe.com/keys).

• Prefer restricted keys (rk_live_) over unrestricted secret keys; Stripe's guidance is to migrate new use cases to restricted keys and reserve secret keys for cases that truly require full access.
• Native rotation: API keys page → overflow menu on the key → Rotate key. Both keys work during a grace window of up to 7 days; choose "now" to delete the old immediately, or set a later expiration. Before expiry, check the old key's request logs and expire only after request volume sits at zero.
• Webhook signing secrets rotate separately: Workbench → Webhooks → endpoint → overflow menu → Roll secret. You can delay the previous secret's expiration by up to 24 hours so your receiver can learn both secrets; Stripe signs each delivery once per active secret during the overlap (docs.stripe.com/webhooks).
• Access policies (IP allowlists, ASN and country rules) attach to keys and block requests from outside your infrastructure — worth configuring after any incident.

AWS

Path: IAM → Users → Security credentials (IAM user guide).

1. Create second access key (the two-key cap exists for exactly this).
2. Update applications and CI onto the new pair.
3. Set the old key to inactive — a reversible test of "did we miss a consumer."
4. Watch CloudTrail for denied attempts naming the old key; fix stragglers.
5. Delete the old key. Inactive is not deleted; deletion ends the incident.
6. CLI equivalents: aws iam create-access-key, aws iam update-access-key --status Inactive, aws iam delete-access-key.

Structural follow-up: AWS's own best-practice line is to prefer temporary credentials and roles over long-lived keys. OIDC federation from your CI straight into cloud roles removes stored cloud keys from the pipeline entirely (Chapter 9, stage 4).

Supabase

Path: Project Settings → API Keys (API keys guide).

• New-format keys rotate individually: create a fresh sb_secret_…, deploy consumers onto it, then delete the leaked one. Deletion is permanent — do it last.
• Legacy service_role JWTs cannot be rotated one-by-one; they derive from the project's JWT secret. The documented migration path is to replace legacy usage with new-format secret keys — which also matters because Supabase states the legacy JWT-based anon and service_role keys "will be deprecated by the end of 2026" (API keys guide).
• A leaked management token (sbp_…, account-level) rotates in account settings; treat as critical since it manages projects, not just data.
• Database passwords (the postgresql:// connection string) reset separately in database settings; restarting poolers and serverless connections afterward is part of the swap.

OpenAI and LLM providers

Path: platform.openai.com/api-keys; Anthropic: Console → Settings → API Keys.

OpenAI's help center states the rule directly: if a key is lost or exposed, create a new one and update your application (help.openai.com). Playbook: create a project-scoped key, deploy consumers, delete the compromised key, then set or tighten spend limits on the project so the next leak has a ceiling. Admin keys deserve extra speed — their scope is organization-wide. Keys sharing the legacy sk- shape across compatible providers (DeepSeek, Groq, Together, Fireworks) rotate at each provider's own console; the pattern is identical even when the menus differ.

GitHub

Paths: Settings → Developer settings → Personal access tokens (fine-grained and classic lists) (managing PATs).

1. Revoke the leaked token immediately — GitHub tokens protect source, and revocation costs nothing because replacements mint instantly.
2. Audit what the token touched via org/repo audit logs before revoking if possible; after revoking, work from logs you already export.
3. Replace with a fine-grained token limited to selected repositories and minimal permissions, or better, a GitHub App for long-lived automation.
4. In Actions workflows, prefer the built-in GITHUB_TOKEN over personal tokens entirely.
5. Note the safety net: GitHub's secret scanning detects certain token formats in public repos and works with partners toward automatic revocation (about secret scanning). It covers GitHub-format tokens, not your AWS or OpenAI keys — do not wait for anyone else to revoke those.

Email, SMS, and messaging

Databases

Connection strings embed passwords, so rotation means changing the password and every copy of the string.

• Postgres: ALTER ROLE app WITH PASSWORD '…'; (ALTER ROLE reference). Zero-downtime variant: create a second role with identical grants, migrate the app to it, drop the first.
• MongoDB Atlas: Database Access → edit user → edit password (Atlas users), then update connection strings and restart pools and serverless connection caches.
• Always restart or recycle connection poolers after a password change; they hold authenticated sessions longer than you expect.

After every rotation

Update the rotation log (Appendix D): credential, fingerprint, created, deployed, revoked, operator, ticket. Rescan the deployed surface. And close the loop structurally — Chapter 7 places detection so that the next instance of this class is caught in your pipeline rather than by a stranger.

Do this now

• [ ] Walk one harmless rotation end-to-end this week using Appendix D as the log; the first real rotation should not be anyone's first rotation.
• [ ] Migrate one unrestricted payment or LLM key to a scoped equivalent with spend limits.
• [ ] If you run Supabase on legacy JWT keys, start the migration to new-format keys ahead of the end-of-2026 deprecation.
• [ ] Configure access policies or IP allowlists on your highest-severity credentials where the provider supports them.