KeyDrift
Free scan
The Secret Leak Field Manual

10 · Drills

The Secret Leak Field Manual · 5 min read

Question this chapter answers: When the real incident arrives, will your response hold together?

Runbooks read calmly are not runbooks executed under pressure. A tabletop drill takes forty-five minutes, costs nothing, and converts this book from prose into muscle memory. The three scenarios below are fictional exercises; any resemblance to real events is the point of the exercise, not a claim about anyone. Run each with the actual people who would respond, against Chapters 3–6.

Drill format

One facilitator reads the situation card aloud. Responders act using only what they would really have: their consoles, their logs, this book. The facilitator injects complications at marked points. Every drill ends with two questions: what surprised you? and what will we change?

Drill A — Payment key in a production bundle

Situation card (fictional): 4:12 p.m., Friday. A customer posts on a public forum that they found "a Stripe key" while viewing source on your marketing site. They include a screenshot: sk_live_FAKE000000000000000000000 (synthetic example). Your support team forwards it to engineering chat.

Expected path through the book:

  1. Classify (Ch. 2): live-mode secret-shaped key — but confirm attribution first; Clerk issues keys in exactly Stripe's shape. Check your Stripe dashboard for a matching key.
  2. Rate (Ch. 3): if it is a real unrestricted sk_live_, critical. Fill the worksheet in parallel with response.
  3. First hour (Ch. 4): start the record; rotate via Dashboard → API keys → Rotate key with the grace window (Ch. 6); decide the inversion question explicitly — is there evidence of active use justifying immediate revocation and an outage?
  4. Complication: the marketing site is a static build deployed by a contractor's pipeline nobody in the room owns. Who rebuilds and purges the CDN, and how fast? (Ch. 4 step 6.)
  5. First day (Ch. 5): audit per-key request logs across the full exposure window; notify your payment processor of the exposure as a matter of course; assess disclosure inputs with counsel if personal data was reachable.
  6. Structural fix: checkout session creation moves server-side; only pk_ keys remain client-facing.

Lessons this drill usually produces: attribution before rotation; the deploy-path gap; processor notification as routine rather than crisis.

Drill B — Cloud key in CI logs

Situation card (fictional): During a routine log review, an engineer finds that an open-source project's public CI job printed its full environment — including AKIAFAKEEXAMPLE000000 and a matching secret (synthetic example) belonging to your AWS account, plus a temporary ASIA… pair from last week's experiment. The job has been public for months.

Expected path:

  1. Classify and rate (Ch. 2–3): long-lived pair = critical regardless of the temporary key; ASIA pairs are time-boxed but full-role while valid — and session tokens can be refreshed by holders, so expiry is never the response (Ch. 4 step 3).
  2. First hour (Ch. 4): rotate the long-lived pair by the two-key pattern — create second, migrate, set old to inactive, watch CloudTrail denials, delete (Ch. 6). Revoke the temporary pair's session immediately rather than waiting.
  3. Audit window (Ch. 5): CloudTrail filtered by both access key IDs from first-public-commit to revocation; look for reconnaissance shapes (listing followed by reads).
  4. Surface sweep (Ch. 4 step 7): the same values may sit in CI context variables, local .env files, and the repository history of the open-source project.
  5. Complication: the open-source repo cannot take force-pushes without breaking contributors. Decision: rotate-and-leave-history vs. coordinated history rewrite — the book's answer is rotation always, history rewrite only as planned hygiene (Ch. 4 mistakes table).
  6. Structural fix: OIDC federation from CI into scoped roles removes stored cloud keys from the pipeline entirely (Ch. 9 stage 4); add log-redaction and artifact scanning so the next printout dies at pre-deploy (Ch. 7).

Lessons: expiry ≠ containment; CloudTrail fluency; the structural fix is federation, not better log hygiene alone.

Drill C — LLM key in a public repository

Situation card (fictional): A security researcher emails: an OpenAI project key (sk-proj-FAKE000000000000000000000000, synthetic example) appears in a commit pushed to your public repository four days ago. They have not published it but note that automated crawlers index new commits continuously.

Expected path:

  1. Assume exposed from push-time, not discovery-time (Ch. 3 clock #2). Harvest scripts watch for key-shaped strings; assume minutes matter.
  2. First hour (Ch. 4): rotate now — create replacement project key, redeploy, delete compromised key (Ch. 6); check whether spend limits exist on the project and set them if not.
  3. Usage audit (Ch. 5): per-project activity over the full window — models called, hours, token volumes inconsistent with your traffic.
  4. Nuance worth saying aloud: GitHub's secret scanning can automatically revoke certain detected tokens in public repos (about secret scanning) — but that program covers GitHub-format tokens, not third-party keys like OpenAI's. Nobody revokes your LLM key for you.
  5. History: same decision as Drill B — rotation first; rewrite optional and planned.
  6. Structural fix: the key was committed because an agent echoed it into a fixture during a test-writing session (Ch. 8 movement table). Apply least-privilege setups and two-pass review; verify monitoring would have caught this as a bundle or repo finding within a day even without the email.

Lessons: third-party keys have no auto-revocation net; spend caps bound the damage; reintroduction monitoring closes the loop the commit opened.

After each drill

Update one page: the runbook changes, the role assignments, the console paths that moved. Then schedule the next drill before leaving the room — drills decay without a calendar behind them.

One request

That is the whole manual: where secrets escape, what each class exposes, how to contain, rotate, audit, and prevent. It works on paper. It works better verified — point a scanner at something you own and see what ships today: keydrift.dev/scan — free, no account.

Do this now

  • [ ] Put Drill C on the calendar first; it is the shortest and most common.
  • [ ] Assign the facilitator role permanently and give it the situation cards.
  • [ ] Add "what surprised us" notes directly into Appendix C's template so drills feed the record format.
  • [ ] Verify once, today, that every responder can reach provider consoles with real permissions — drills fail on access, not knowledge.