7 · Detection in the pipeline

The Secret Leak Field Manual · 4 min read

Question this chapter answers: Where should scanning live so leaks are found by you first?

Detection is not a tool purchase; it is a placement decision. The same scanner produces completely different value depending on which of five stages it runs in, because each stage sees a different surface from Chapter 1.

The five stages

Stage	Catches	Misses	Friction
Pre-commit (developer machine)	Keys typed or generated into source, seconds after creation	Anything entering later; trivially skipped locally	Lowest — but only when fast
CI gate on the repository	Everything committed, enforced for every contributor regardless of local setup	Build-time inlining — the repo may be clean while the build is not	Merge-blocking; false-positive tuning matters
Pre-deploy on the artifact	Secrets inlined at build time into bundles and artifacts — surface 4 before it ships	What hosting layers add afterward	Small build-time cost; needs artifact access
Post-deploy against the live URL	What visitors actually download: chunks, streamed server data, published source maps, CDN edge behavior	Code behind logins (paste modes cover that manually)	None to your developers; requires a scanning endpoint
Scheduled monitoring	Drift over time: new findings, regressions ("it came back"), resolutions confirmed	Nothing structural — it is the safety net under all four above	None; alerts only on change

The gap this table exposes is pre-deploy on the artifact. Repo scanners answer "did a secret enter version control?" — and stay green while the build publishes a secret that never touched git. This is the "clean repo, leaky build" gap, and only an artifact-level check closes it.

Coverage versus friction

Friction is not free; every gate spends developer trust. Spend it where coverage is unique:

Pre-commit earns its keep by being instant. If it adds seconds, people disable it, and a disabled control is worse than none because it feels like coverage.
CI gates are the enforcement point — the one place skipping is visible. Budget tuning time for false positives here.
Pre-deploy is cheap because it runs where artifacts already exist.
Post-deploy and scheduled scans cost nothing per commit and are the only stages that see the deployed truth, including changes made outside your pipeline.

Two engineering details decide whether any of this survives its first bad week:

Separate exit codes for clean / findings / could-not-scan. A pipeline that treats "the scanner broke" as "the app is leaking" gets switched off within a month. A pipeline that cannot tell them apart will silently pass broken builds too.
Allowlists with owners and expiry dates. A documented demo key in a tutorial fixture is a known pattern — some scanners reject documented sample keys by name rather than alerting on them. For everything else, an allowlist entry without an owner and expiry is just approved debt.

The layered standard

A defensible baseline, better, best:

Baseline: CI repo gate + scheduled post-deploy monitoring with alerts on change.
Better: add a pre-deploy artifact scan so inlined secrets die before shipping.
Best: add fast pre-commit feedback, plus preview-environment scanning so branch builds get the same scrutiny as production.

The pipeline itself can leak

Two primary accounts show why detection must partly live outside the pipeline. In 2021, Codecov's Bash Uploader was modified to export data from users' CI environments; affected teams were told to re-roll credentials held in CI variables (Codecov security update). In January 2023, CircleCI disclosed that customer environment variables were taken from its platform and asked customers to rotate secrets stored there (incident report). In both cases, the pipeline's own integrity failed — and no check running inside that pipeline could have flagged the problem, because the checker and the failure shared a root. An external, scheduled scan of your deployed app keeps one witness independent of your tooling.

Honest limits

Static detection — patterns, entropy, context rules — cannot see keys assembled at runtime from concatenated fragments, and URL-based tools cannot reach code behind authentication. Say both out loud in your program documents. A detection story with stated boundaries is trusted longer than one claiming totality.

Do this now

[ ] Map your current checks onto the five-stage table; name the empty columns honestly.
[ ] Add the missing baseline first: if you have repo scanning but no post-deploy scan, close that gap this week.
[ ] Verify your scanner's exit-code semantics (clean vs. findings vs. could-not-scan) before wiring CI gates.
[ ] Write the allowlist policy: owner + expiry required, reviewed quarterly.
[ ] Schedule one external check of your production bundle that does not depend on any pipeline component.