KeyHog: a Rust secret scanner

KeyHog scans repositories, Git history, CI workspaces, hosted Git collections, cloud object inventories, archives, and local systems for leaked credentials. You can start with one command, then choose a source boundary, detection policy, and execution route that match the job.

The sample below comes from a portable,simd build. The default crates.io install reports the pure-Rust CPU route instead; host labels and backend lines are evidence from the running binary, not universal defaults.

$ keyhog scan . --progress
    K E Y H O G
    ───────────
    v0.5.81 · secret scanner · 934 detectors
    by santh

  16 cores | SIMD: AVX-512 | Hyperscan | 934 detectors (5820 patterns) io_uring | backend=simd-regex | gpu=none

  ┌    CRITICAL ─── Stripe Secret Key
  │ Secret:     sk_l...p7dc
  │ Location:   src/config/.env.staging:14
  │ Evidence:   likely/vendor-pattern  ■■■■■■ 100%
  │ Action:     Roll the exposed Stripe secret key in the Dashboard, update production consumers, then delete the old key.
  │ Docs:       https://docs.stripe.com/keys#roll-api-key
  └─────────────────────────────────────────────

  ━━━ Results ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
  1 secret found · 1 unverified

Scan complete. Found 1 secret in 1.42s.

What it does

KeyHog walks working trees, Git history, Docker images, Git provider collections, cloud buckets, URL lists, and local systems. Each finding has:

• a detector that fired (stripe-secret-key, aws-access-key, …)
• a location (file, line, offset, optionally commit hash and author)
• an exact evidence tier and reason code, plus entropy and evidence score when measured
• an optional live verification result if you pass --verify

KeyHog also supports Git provider inventories, S3, GCS, and Azure Blob objects, Docker images, whole-system audits, CPU, Hyperscan, CUDA, Metal, and WGPU execution, and an optional Unix daemon for repeated eligible inputs. Choose a scanning workflow starts with the operator task and links each capability to the chapter that owns its contract.

The detector corpus ships as TOML files under detectors/. Run keyhog detectors --format json to inspect the exact corpus embedded in the installed binary. A custom --detectors <DIR> remains an explicit replacement by default, so there is never a hidden merge with embedded rules. Reviewed extensions can opt into --detectors-mode overlay; detector-ID collisions then fail closed rather than shadowing shipped rules.

What it doesn’t do

• No telemetry. Findings stay local. The scanner never phones home.
• No hosted scanning agent. Findings are not uploaded to a KeyHog service. A local daemon exists for eligible stdin and single-file requests on Unix. Starting it is explicit and it stays on your machine; after you start a compatible daemon, the ordinary Unix scan default (--daemon=auto) uses it for eligible requests. Use --daemon=off to force the in-process path.
• No remote “AI-powered” detection. Service detectors use TOML regexes and structural validators; generic detectors compose assignment shape, entropy, BPE token efficiency, context, and local confidence policy. The small on-device MoE scores ambiguous candidates without sending content away. Verification is optional and is the only detection-adjacent step that calls a service endpoint.

Why another scanner

Three things, in order of how much they matter:

1. Precision. A scanner that emits one false positive per ten findings teaches developers to ignore it. KeyHog suppresses example credentials (the Stripe docs key, the AWS sample key, the RFC 7519 specimen JWT), vendored bundles (minified jQuery, node_modules), and CI workflow ${{ secrets.NAME }} references by default. Repository dogfood and detector-specific negative twins keep those decisions exercised through the same scanner path users run.

2. Recall. The detector corpus is built service-by-service. For every detector, the test suite carries positive shapes (env-var, JSON, YAML, header, URL), negative shapes (placeholder, EXAMPLE marker), and adversarial evasions (split across lines, hex/base64-encoded, reversed via Caesar cipher). If a shape isn’t in the suite, the detector isn’t shipped.

3. Speed. Hyperscan SIMD prefilter, vectorized entropy, and a GPU region-presence route can accelerate different workloads. The winning route depends on the binary, detector/config digest, source shape, candidate density, cache state, CPU, GPU, driver, and storage. KeyHog records fastest-correct calibration for the installed host instead of treating a benchmark from another machine as a routing threshold.

   Published benchmark panels separate full-process startup, warm daemon requests, detection policy, cache state, backend diagnostics, scan workers, filesystem readers, corpus size, storage class, and concurrent partitions. Each panel labels development-host evidence that cannot support a clean-source release routing claim. See Backends and routing and the reproducible benchmark harness.

Get going

Install the current release from crates.io:

cargo install --locked keyhog
keyhog --version
keyhog scan .

KeyHog requires Rust 1.89 or newer. The Install guide shows exact-version, portable, CI-only, and checked-out source builds. Your first scan gives you a safe synthetic finding to confirm output, redaction, and exit status before you scan a repository.

Where things live

• Source: github.com/santhreal/keyhog
• Issues: github.com/santhreal/keyhog/issues
• Published crates: crates.io/crates/keyhog
• Security: use GitHub private vulnerability reporting first. If the form is unavailable, email security@santh.dev; PGP is not required. See the security policy.

License: MIT OR Apache-2.0. Read the MIT terms and Apache-2.0 terms.

Choose a scanning workflow

Start with the source boundary you need. Then choose a detection policy and an execution route. These are separate decisions: a backend changes how KeyHog executes, while a preset changes what detection work it performs.

Choose in 30 seconds

Then make three independent choices:

1. Select the source boundary. This determines which bytes are eligible.
2. Keep the default detection policy unless you accept a documented fast, deep, or precision tradeoff.
3. Use calibrated automatic routing for normal scans. Select an explicit backend only for diagnosis, measurement, or a required-accelerator gate.

If any source reports incomplete coverage, preserve that status with the findings and report. A partial scan is not a clean scan.

Choose the source boundary

Read Your first scan for a local repository, CI secret scanning for direct CI jobs, and Mass repository and cloud scanning for inventory partitioning and coverage.

Choose a scan policy

The three presets are mutually exclusive bases. Compatible explicit options refine them; a precision confidence override may raise but never lower 0.85. Lockdown refuses fast and other completeness-reducing switches, and always runs in process. See Configuration and Hardening.

The default decode cap can hide an encoded credential

Decide this one before you trust a clean result on a repository with large files. A Base64-encoded credential in a chunk above --decode-size-limit is never decoded, so it is never reported. The scan exits 0:

file size    keyhog scan <file>                       with --deep
400K         1 finding, exit 1, status success        1 finding, exit 1
510K         1 finding, exit 1, status success        1 finding, exit 1
520K         0 findings, exit 0, status partial       1 finding, exit 1
600K         0 findings, exit 0, status partial       1 finding, exit 1

The miss is reported, but only in the envelope. coverage_gap_summary carries scanner decode-through declined by --decode-size-limit, and scan_status becomes partial. The exit code stays 0, so a CI gate that branches on the exit code alone passes over a real credential. Gate on the gap reason, not the exit code.

Position in the file governs this, not size. The cap applies per chunk, and a file is read in 1 MiB windows, so only the short tail window of a large file can fall under the 512K limit. An encoded credential in the interior of any file above about 1 MiB is never decoded, at any file size:

2000K file, payload at end of file       1 finding, exit 1
2000K file, payload in the middle        0 findings, exit 0
3000K file, payload at end of file       1 finding, exit 1
3000K file, payload in the middle        0 findings, exit 0

That is also why the size table above looks erratic: planting at the end of the file tests the one position that can still succeed, and the tail window’s size rises and falls as the file grows. Do not infer a safe file size from a fixture that passed, and do not build a regression fixture that plants at the end.

Either preset choice restores it. --deep raises the ceiling as part of its policy, and --decode-size-limit 4M raises it without changing anything else:

keyhog scan . --decode-size-limit 4M

Choose an execution route

Normal scans use auto. An explicit backend is a diagnostic or benchmark contract, not a recommendation for routine routing.

Use keyhog --version --full to inspect compiled capability, keyhog backend --self-test --json to prove backend health, and keyhog backend --autoroute --json to inspect the measured route. These commands answer different questions: discovery, correctness, and comparative selection.

Choose a detector corpus mode

--detectors <DIR> selects a custom detector directory. Choose how it participates in the corpus:

The selected corpus owns matching, validation, entropy, suppression, ML, and declared decode-transform policy. Replace mode does not inherit detector-local policy from the embedded corpus. See Detectors.

What KeyHog can scan

The default and official release builds support the sources below. Reduced source builds can omit feature-gated Git, web, cloud, container, and verifier support. Every enabled source feeds the same compiled detector pipeline.

How KeyHog decides what is real

Precision is the product. A finding survives several independent stages before it reaches your terminal.

How KeyHog stays fast

The generated scaling matrix measures these controls instead of prescribing a fixed thread count. Run make -C benchmarks readme-scaling on the target host. The result binds the binary, detector corpus, exact workload bytes, effective CPU limit, filesystem identity, page-cache policy, raw trials, and process exit status.

What KeyHog emits

How KeyHog protects the secrets it reads

A scanner holds credentials in memory by design, so KeyHog hardens the process that does it.

Every subcommand

The full flag surface for every command is in the CLI reference.

Update or repair with cargo install --locked --force keyhog, then run keyhog doctor to verify the replacement.

Pick your input shape

KeyHog scans more than one kind of thing. A repository working tree, a 4 GiB log, a minified bundle, a container image, and a Git history are five different workloads. Each has its own command, its own limits, and its own way of going quiet when it fails.

Find your shape in the table, run the command, then run the coverage check for that shape. The default command is right for exactly one shape.

The small ci profile intentionally omits Git, cloud, web, container, binary, and verification flags. Add only the named source feature you need, or use the default portable profile. keyhog scan --help is authoritative for the installed binary.

Why the shape matters

The three defaults that decide whether a scan covers your input are set for a repository working tree.

--max-file-size is 100 MiB. A single larger file is skipped.

The default exclusion policy removes .git/, lockfiles, vendored trees, and minified bundles. On a repository that is what you want. On a directory of build output it removes everything.

--stdin accepts 10 MiB. A larger stream fails closed.

None of the three is wrong. Each is wrong for at least one of the shapes above.

Check coverage the same way every time

Every page in this section ends with the same check, because the question is always the same. Did KeyHog read my input?

rm -f keyhog.json
rc=0
keyhog scan <target> --format json-envelope -o keyhog.json || rc=$?
jq '{bytes: .metadata.source_bytes_scanned, status: .scan_status,
     gaps: .coverage_gap_summary, findings: (.findings | length)}' keyhog.json
printf 'keyhog exit=%s\n' "$rc"

Tell a real clean from a skipped input explains each field, lists every coverage-gap reason, and names the cases where the current build reports a clean scan over input it did not read.

Combining shapes in one run

You can pass several roots to one keyhog scan. You cannot mix source kinds:

keyhog scan src/ config/ vendor-drop/

Nested or duplicate roots fold into their covering parent. One report and one exit code cover all of them, so a gap in any root makes the whole run partial.

A container image, a bucket, a Git history, and a working tree each need their own run. Give each one its own output file so a failure in one does not hide behind a success in another:

keyhog scan . --format json-envelope -o worktree.json
keyhog scan --git-history . --format json-envelope -o history.json
keyhog scan --docker-image registry/app:v1 --format json-envelope -o image.json

Check the exit code and envelope of each run. A failing source or coverage gap returns 13 when no finding outcome takes precedence. Advisory skips can leave scan_status: partial with exit 0, so automation must also inspect the gap reasons it treats as unacceptable.

Install

KeyHog releases are Rust packages on crates.io. Install the latest published version with Cargo:

cargo install --locked keyhog
keyhog --version
keyhog doctor

cargo install builds KeyHog for your host and places the binary in Cargo’s binary directory. This is usually $HOME/.cargo/bin on Linux and macOS, or %USERPROFILE%\.cargo\bin on Windows. Add that directory to PATH if your shell cannot find keyhog.

Platform support

cargo install is the current distribution path and builds KeyHog from source for the host that runs it. Hosted release CI proves this matrix:

Other Rust host targets are not part of the hosted release contract. Cargo may build them when KeyHog’s dependencies support the target, but a successful local build is the evidence for that host. In particular, Linux arm64 and Windows arm64 do not have hosted release jobs.

Installing a bundle you already hold

There is no binary download channel. No workflow builds, signs, or uploads release binaries, and KeyHog has no self-update command.

install.sh and install.ps1 install a binary you already have, with --from-file. Use them for an air-gapped host or to place a locally built binary on PATH with the same layout, permission, and PATH handling that a packaged install would use. Neither script contacts the network. Both refuse to install without --from-file and print the Cargo command instead.

Both scripts run keyhog doctor, publish an execution-pack generation, calibrate autoroute, and then scan a throwaway two-file directory with no backend override. That last scan is the install’s pass criterion for routing: a calibrated cache that cannot serve an ordinary scan fails the install and the previous binary is restored.

Update with cargo install --locked --force keyhog. That is also the repair path: it rebuilds and replaces the installed binary.

Install Rust

KeyHog requires Rust 1.89 or newer. Install Rust with rustup when cargo --version is unavailable. Then open a new terminal and run the install command again.

The default build includes filesystem, Git, web, cloud, container, archive, and native binary sources plus live verification. It uses the pure-Rust CPU scanner and has no Hyperscan, GPU-driver, CUDA-toolkit, or Ghidra build prerequisite. Binary string and object scanning works without Ghidra. If you install Ghidra separately, KeyHog can also enrich supported binaries with decompiled content.

Pin an exact version

Use an exact Cargo version requirement when a build or CI job must stay on one release:

cargo install --locked --version '=0.5.81' keyhog

The leading equals sign prevents Cargo from selecting another compatible version. KeyHog publishes canonical X.Y.Z versions. Do not include a leading v in the Cargo version requirement.

To update to the latest release, run:

cargo install --locked --force keyhog
keyhog doctor

Every successful main CI run publishes the next patch version. KeyHog does not publish binary release assets or installer bundles.

Update or roll back

Stop a running daemon before replacing the executable:

keyhog daemon stop
cargo install --locked --force keyhog
keyhog doctor

Cargo builds the replacement before it changes the installed binary. A compile or download failure leaves the previous executable in place.

To roll back, choose a version from the crates.io version list, replace MAJOR.MINOR.PATCH below, and install that exact package:

cargo install --locked --force --version '=MAJOR.MINOR.PATCH' keyhog
keyhog doctor

The commands are identical in Bash, Zsh, and PowerShell. If PowerShell reports that keyhog.exe is in use, stop the daemon and close other KeyHog processes, then retry. If Cargo reports that libhs is missing, remove an unintended simd feature or install the Hyperscan/Vectorscan development package. The default install does not require libhs.

Choose installation features

The profiles below serve different products. ci is the small user-facing CI build; ci-lean is a broad maintainer test closure and is not the lightweight edition.

Install the default portable build:

cargo install --locked keyhog

Enable CUDA, native Metal, and WGPU:

cargo install --locked keyhog --no-default-features --features portable,gpu

Enable Hyperscan or Vectorscan:

cargo install --locked keyhog --no-default-features --features portable,simd

Install the small checkout-only CI scanner:

cargo install --locked keyhog \
  --no-default-features \
  --features ci

Cargo does not execute the binary after installation. After installing a multi-backend portable,gpu or portable,simd build, acquire and calibrate the eligible peers explicitly:

keyhog backend --self-test
keyhog calibrate-autoroute
keyhog backend --autoroute

A scalar-only portable or ci build reports autoroute health as direct because it has no backend choice to calibrate.

Which build your workload needs

Source providers are compile-time features. A flag that its feature did not build is not hidden or ignored: it is absent from the command line, so the command exits 2 with error: unexpected argument. That is loud, and it is the reason to pick the right build before you script against it.

The ci build covers the filesystem and standard-input workloads and nothing else. That is the point of it: a checked-out tree is what a CI job scans, and dropping the rest removes the network and native dependencies. Add back only what you need:

cargo install --locked keyhog --no-default-features --features ci,git

Check what your installed build has before you script against it:

keyhog scan --help

A workload whose flag is missing from that output is not available in your build. Reinstall with its feature rather than working around the error.

Build the checked-out source

From the repository root:

cargo install --locked --path crates/cli

Use this path when you are testing an unreleased checkout. A tagged GitHub Action ref installs its exact crates.io version with the lean ci feature. A branch or commit Action ref builds its checked-out source.

Confirm the installation

Inspect the compiled capabilities and health before your first scan:

keyhog --version --full
keyhog scan --help
keyhog doctor
keyhog backend --self-test
keyhog backend --autoroute
keyhog scan .

scan --help is the authoritative list of source flags compiled into this binary. backend --self-test executes available accelerator diagnostics and reports a successful SKIP when no physical GPU is present. backend --autoroute reports direct for a scalar-only build and ready for a valid multi-backend calibration.

keyhog doctor exits 0 when the installed binary is healthy and 4 when a health check fails. keyhog scan . exits 0 when no finding blocks the active evidence policy and 1 when at least one finding blocks. Continue with Your first scan to exercise a safe synthetic finding.

Your first scan

Start with a synthetic token whose checksum is valid for the detector but which is not a live credential. This confirms detection and redaction without putting a real secret in your shell history.

On Linux or macOS:

demo_dir=$(mktemp -d)
token='ghp_'
token="${token}aBcD1234EFgh5678ijkl9012MNop343hK7n2"
printf 'GH_TOKEN="%s"\n' "$token" > "$demo_dir/demo.env"
if keyhog scan "$demo_dir/demo.env"; then status=0; else status=$?; fi
printf 'keyhog exit code: %s\n' "$status"
rm -rf "$demo_dir"
test "$status" -eq 1

On Windows PowerShell:

$Demo = Join-Path ([IO.Path]::GetTempPath()) "keyhog-first-scan-$PID.env"
$Token = 'ghp_' + 'aBcD1234EFgh5678ijkl9012MNop343hK7n2' # keyhog:ignore detector=github-classic-pat
Set-Content -Path $Demo -Value "GH_TOKEN=`"$Token`""
keyhog scan $Demo
$Status = $LASTEXITCODE
Write-Output "keyhog exit code: $Status"
Remove-Item $Demo
if ($Status -ne 1) { throw "expected finding exit 1, got $Status" }

You should see a GitHub Classic PAT finding with the credential rendered as ghp_...K7n2, followed by keyhog exit code: 1. File paths, timing, host capabilities, and detector counts vary by installation.

KeyHog redacts credential values by default in every output format, including the --output file, not only the terminal. --show-secrets deliberately prints plaintext and can leak it into logs, artifacts, or scrollback. Do not use that flag for routine scans, and never in CI.

Now scan your repository:

cd /path/to/your/repository
keyhog scan .

That walks the current directory and reports findings. A successful scan returns exactly one process exit code:

keyhog scan --help prints the same canonical table. CI does not need grep, jq, or exit-code arithmetic. When several conditions apply, a scanner panic takes precedence over findings, a confirmed-live finding takes precedence over other findings, and a blocking finding takes precedence over a later cache or source-coverage failure. Read the coverage warning and structured scan_status before treating partial output as complete.

What you get out of it

Findings go to stdout as redacted boxes followed by a summary:

┌    CRITICAL ─── GitHub Classic PAT
│ Secret:     ghp_...K7n2
│ Location:   /tmp/keyhog-first-scan/demo.env:1
│ Evidence:   likely/vendor-pattern  ■■■■■■ 100%
└─────────────────────────────────────────────

━━━ Results ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
1 secret found · 1 unverified

Each finding gives you the detector, the redacted credential, the location, the exact evidence tier and reason code, an optional evidence score, and remediation guidance. Use --output to write the report to a file instead of stdout.

Add --progress when you need to see which engine ran the scan. It writes a banner to stderr before the findings:

    K E Y H O G
    ───────────
    v0.5.81 · secret scanner · 934 detectors
    by santh

  16 cores | SIMD: AVX-512 | Hyperscan | 934 detectors (5820 patterns) io_uring | backend=simd-regex | gpu=none

The banner reports this host’s CPU and GPU labels, the scanner engine, the compiled pattern count, the selected backend, and the GPU engagement result. KeyHog writes it only when stderr is a terminal, so a redirected log or a CI capture never contains it.

Default suppressions

KeyHog ships with a Tier-B suppression list of publicly documented test fixtures - credentials that appear in vendor docs as examples. Findings on these are suppressed by default. Examples:

• Stripe’s sk_live_4eC39HqLyjWDarjtT1zdp7dc (docs sample)
• AWS’s AKIAIOSFODNN7EXAMPLE (docs sample)
• The RFC 7519 specimen JWT
• GitHub’s ghp_aBcDeFgHiJ… placeholder

To see what was suppressed, pass --no-suppress-test-fixtures. The list lives at crates/cli/data/suppressions/test-fixtures.toml inside the source tree and is baked into the binary at build time. It is one visible suppression layer; detector-owned examples, structural/context gates, default path policy, .keyhogignore, and .keyhogignore.toml have distinct documented ownership. See Suppressions for the full order.

JSON output

keyhog scan . --format json-envelope

The compatibility --format json form remains a top-level findings array; choose json-envelope when a durable schema identity and scan metadata are needed.

The output is a versioned envelope. schema_version.major selects the incompatible schema generation; consumers must reject an unsupported major. Minor revisions are additive, so a reader that understands major 2 may accept a newer minor and ignore fields it does not know. The optional metadata object identifies the scan; coverage_gap_summary preserves any source or scanner coverage gaps; findings contains the redacted finding objects. Every finding has a canonical evidence object with exact tier and reason_code. entropy and evidence_score are included only when measured. A present entropy value is Shannon bits-per-byte evidence, not an evidence score and not a claim that entropy alone caused the finding. The optional correlations array is emitted only when the scan ran with --correlate; the key is absent, not empty, otherwise. See Output formats. The source_bytes_scanned and source_chunks_scanned counters are the exact workload consumed by the scanner, so an importer can calculate throughput from the artifact without scraping console progress.

{
  "schema_version": {"major": 2, "minor": 0},
  "scan_status": "success",
  "metadata": {
    "scan_id": "0123456789abcdef0123456789abcdef",
    "scan_status": "success",
    "keyhog_version": "0.5.81",
    "git_hash": "<build-commit>",
    "detector_digest": "934-<digest>",
    "config_digest": "<effective-config-digest>",
    "resolved_scan": {
      "schema_version": 1,
      "preset": "default",
      "effective": {"max_decode_depth": "10", "entropy_enabled": "true"},
      "overrides": []
    },
    "generated_at": "2026-07-14T00:00:01",
    "scan_started_at": "2026-07-14T00:00:00",
    "scan_finished_at": "2026-07-14T00:00:01",
    "duration_ms": 1000,
    "targets": ["."],
    "source_chunks_scanned": 1,
    "source_bytes_scanned": 128,
    "detector_count": 934
  },
  "coverage_gap_summary": [],
  "findings": [
    {
      "detector_id":        "stripe-secret-key",
      "detector_name":      "Stripe Secret Key",
      "service":            "stripe",
      "severity":           "critical",
      "credential_redacted": "sk_l...p7dc",
      "credential_hash":     "sha256-hex",
      "companions_redacted": {},
      "location": {
        "source":    "filesystem",
        "file_path": "src/config/.env.staging",
        "line":      14,
        "offset":    12,
        "commit":    null,
        "author":    null,
        "date":      null
      },
      "verification": "skipped",
      "metadata": {},
      "additional_locations": [],
      "evidence": {
        "tier": "likely",
        "reason_code": "vendor-pattern",
        "provenance": {
          "schema_version": 1,
          "detector_digest": "0123456789abcdef",
          "pattern_index": 0,
          "candidate_channel": "pattern",
          "source_role": "environment-assignment-value",
          "context_class": "vendor-pattern"
        }
      },
      "entropy": 4.5,
      "evidence_score": 1.0,
      "remediation": {
        "action":     "Roll the exposed Stripe secret key in the Dashboard, update production consumers, then delete the old key.",
        "revoke_url":  "https://docs.stripe.com/keys#roll-api-key",
        "docs_url":    "https://docs.stripe.com/keys"
      }
    }
  ]
}

Pipe .findings into jq, into a SARIF converter for the GitHub Security tab, or into your own dedup / triage tooling.

Limiting scope

keyhog scan src/                        # one subdirectory
keyhog scan src/config/staging.env      # one file
keyhog scan --stdin < staging.env       # from stdin (CI: cat | keyhog)
keyhog scan . --exclude-paths 'docs/*'  # exclude a glob

The default walk skips a file when any segment of its path is one of these directory names, at any depth:

.git  node_modules  target  .cache  __pycache__  .venv  venv  .tox
dist  build  out  .next  .nuxt  vendor  swagger  swagger-ui

It also skips lock files, editor backups, and filenames containing .min. or .bundle.. A skipped file produces a WARN line on stderr and no finding, and the scan still exits 0. Check that list against your repository before you trust a clean result: build/, dist/, out/, and vendor/ hold real source in some projects. Scan them with keyhog scan . --no-default-excludes, which also stops the scanner discarding findings inside minified and vendored bundles, or name one tree directly with keyhog scan vendor/. See files the walker never reads.

Going further

A first scan of a real repository usually reports credentials that were already there. Decide what to do with them before you wire KeyHog into anything:

• Rotate what you can. A leaked credential in the working tree is also in Git history.
• Record the rest once with --create-baseline, then gate on new findings only. See Fail only on new secrets.

Then continue with:

• Suppressions and baselines - allowlists, inline directives, per-detector floors, and what a baseline does and does not match.
• Output formats - JSON, SARIF, plain text.
• Verification - --verify makes API calls to confirm credentials are live; a dead credential is downgraded one severity tier (critical → high, …), never collapsed to a fixed level.
• Pre-commit hook - block leaked creds before they hit the repo.
• CI integration - GitHub Actions, GitLab CI, CircleCI patterns.

Output formats

KeyHog’s --format flag takes one of eleven values: text (default), json, json-envelope, jsonl, jsonl-envelope, sarif, csv, github-annotations, gitlab-sast, html, and junit. Pick the one that fits the consumer. csv emits a spreadsheet-importable row per finding, github-annotations emits GitHub Actions workflow-command annotations, gitlab-sast emits a GitLab SAST security report, html emits a self-contained report page, and junit emits a JUnit XML test-report (one <testcase> per finding) for CI systems that ingest JUnit.

Every renderer receives the same completed scan report. Its common metadata (version, timestamps, duration, targets, source bytes, source chunks, and detector count) is owned by the core ScanReport model, so an output format cannot accidentally invent a second scan clock or target list. Each format retains its owned projection: HTML displays the full metadata panel, GitLab SAST projects the scan times required by its schema, and finding-only formats omit scan-wide state. JSON-envelope, JSONL-envelope, and HTML artifacts also include a versioned resolved_scan object with the selected preset, sorted effective detection values, and an overrides list. This is the authoritative machine-diffable record of what default, fast, deep, or precision meant for that run; it includes compatible refinements such as --deep --decode-depth 3, rather than requiring consumers to infer behavior from CLI text or stderr.

Metadata-bearing formats expose scan_status as success, complete_after_recovery, partial, cancelled, or failed. complete_after_recovery is a successful complete scan, but it proves that a visible fault in an authenticated selected backend occurred and every affected byte was recovered. Invalid autoroute state selects no backend and records partial. Any source or scanner coverage gap overrides recovery; incomplete input never reports clean.

The composite Action output named scan-status is a different, normalized wrapper receipt: success, partial, cancelled, or failed. complete_after_recovery remains success there because the process completed with ordinary clean/findings semantics. Consumers that must distinguish healthy completion from recovery must inspect a metadata-bearing report (for the Action’s SARIF default, the KeyHog run properties), not the wrapper output.

After a selected accelerated backend faults, the recovery backend is the confidence-separated fastest remaining measured-correct peer for the same workload and runtime class. When no trustworthy route can be selected, no recovery backend is substituted: the affected input remains unscanned and the report records partial coverage.

Authenticated-backend fault recovery is structured in every metadata-bearing artifact. JSON-envelope, JSONL-envelope, HTML, and the CSV preamble carry backend_recoveries; SARIF uses runs[].properties["keyhog.backend.recoveries"]; GitLab SAST uses scan.keyhog_backend_recoveries; JUnit adds keyhog.backend.recovery suite properties; GitHub annotations emit a warning with recovered bytes and the repair command. Plain json and jsonl remain finding-only and receive the same recovery warning on stderr as text. Each metadata projection retains the failed backend, recovery backend, recovered byte count, and keyhog calibrate-autoroute remediation.

Every finding also carries companions_redacted, a sorted JSON object of nearby credential or context values captured by the detector. Companion values are redacted at the same boundary as the primary credential, so plaintext never enters verification results or reports. JSON, JSONL, and HTML expose the object directly; SARIF exposes companions_redacted.<name> result properties; CSV, JUnit, GitLab SAST, and GitHub annotations use a deterministic redacted summary. An empty object means the detector did not produce companion evidence, not that companion matching was disabled.

Every finding format exposes the exact canonical evidence tier and reason code. confirmed identifies intrinsic or live proof, likely identifies strong vendor-specific shape in a credential-bearing role, and review identifies a candidate that needs human judgment. The optional evidence_score supplements that verdict when the detection path measured a score; it never replaces the tier or reason.

entropy is an optional Shannon bits-per-byte measurement. It is present only when the detection path measured entropy; an omitted field means that path did not produce entropy evidence. JSON, JSONL, and HTML expose it as a numeric field; SARIF exposes it as a result property; text, JUnit, GitLab SAST, and GitHub annotations render it only when measured. It is independent of the optional evidence_score.

--format text (default)

Human-readable boxes. Best for terminal use, pre-commit hook output, and screenshots. Colors auto-detect TTY; pipe through cat (or set NO_COLOR=1) to disable.

  ┌    CRITICAL ─── Stripe Secret Key
  │ Secret:     sk_l...p7dc
  │ Location:   src/config/.env.staging:14
  │ Evidence:   likely/vendor-pattern  ■■■■■■ 100%
  │ Action:     Roll the exposed Stripe secret key in the Dashboard, update production consumers, then delete the old key.
  │ Docs:       https://docs.stripe.com/keys#roll-api-key
  └─────────────────────────────────────────────

  ━━━ Results ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
  1 secret found · 1 unverified

Each finding is a severity-colored box: the header carries the severity and detector name, then Secret: (the credential redacted to its first and last few characters), Location: (file:line:offset), Evidence: with the exact tier/reason and optional score bar, and an Action:/Docs: remediation hint. Verified runs add the liveness state and commit/author rows when known. The Results footer joins the counts with ·.

--format json

Bare JSON array for simple pipelines. Every finding has all required documented fields present; optional fields are omitted only when their value is unavailable. Use --format json-envelope for a versioned root object with schema identity and scan metadata.

The following is one complete finding object. The values are synthetic. The credential is already redacted, and the hash is a non-secret example value.

{
  "detector_id": "stripe-secret-key",
  "detector_name": "Stripe Secret Key",
  "service": "stripe",
  "severity": "critical",
  "credential_redacted": "sk_l...p7dc",
  "credential_hash": "0000000000000000000000000000000000000000000000000000000000000000",
  "companions_redacted": {},
  "location": {
    "source": "filesystem",
    "file_path": "src/config/.env.staging",
    "line": 14,
    "offset": 218,
    "commit": null,
    "author": null,
    "date": null
  },
  "verification": "skipped",
  "metadata": {},
  "additional_locations": [],
  "evidence": {
    "tier": "likely",
    "reason_code": "vendor-pattern",
    "provenance": {
      "schema_version": 1,
      "detector_digest": "0123456789abcdef",
      "pattern_index": 0,
      "candidate_channel": "pattern",
      "source_role": "environment-assignment-value",
      "context_class": "vendor-pattern"
    }
  },
  "evidence_score": 1.0,
  "remediation": {
    "action": "Roll the exposed Stripe secret key in the Dashboard, update production consumers, then delete the old key.",
    "revoke_url": "https://docs.stripe.com/keys#roll-api-key",
    "docs_url": "https://docs.stripe.com/keys"
  }
}

Optional fields such as entropy are absent when they were not measured. Location members are present and use null when the value is unknown. A verification transport failure is encoded as an externally tagged object, for example "verification":{"error":"timeout: the endpoint did not respond within the verification deadline. Fix: raise the verification timeout with --timeout, or check network egress / proxy reachability to the credential's host"}.

evidence.provenance is secret-safe. The detector-corpus digest and pattern ordinal bind the exact detector pattern. The candidate channel, source role, and context class bind the evidence used before optional live verification. Unsupported scanner context retains its producer identity and uses the unsupported-context context class. Caller-created findings use the unattributed channel with no detector digest or pattern ordinal. Cross-detector grouping retains the strongest evidence reason while keeping provenance owned by the detector named in detector_id; folded detectors remain listed in cross_detector.* companions. See Pattern provenance and secret-safe evidence and Triage and feedback interchange. Do not enable --show-secrets when stdout or --output is retained by CI, uploaded as an artifact, or sent to another process. That option deliberately replaces credential_redacted with plaintext. credential_hash is safe from accidental plaintext disclosure, but it is a stable SHA-256 correlation value. Treat every report as security-sensitive data.

keyhog scan . --format json | jq '.[].detector_id' | sort | uniq -c

That sample command dedups findings by detector, which is the most common “what kinds of leaks do I have” question.

--format json-envelope

Versioned JSON envelope. The root object contains schema_version and findings, plus optional scan-wide metadata and the coverage_gap_summary array. Each gap preserves the canonical reason and count used by SARIF/HTML, including when there are no findings. A reader must reject an unsupported schema_version.major; a newer minor under a supported major is additive and may be accepted. See Your first scan for the complete schema. Metadata includes the binary Git identity, detector-set digest, effective-config digest when available, a stable non-secret scan_id, targets, timing, and counters including the exact source bytes and chunks consumed by the scanner. backend_recoveries records bounded, non-secret failed-backend, recovery-backend, range, chunk, byte, reason, and repair-command aggregates whenever an automatic route completes exact recovery. The top-level scan_status is one of success, complete_after_recovery, partial, cancelled, or failed; readers must preserve the explicit terminal state in detached artifacts. The scan_id lets independently stored JSON-envelope, JSONL-envelope, and HTML projections be joined without exposing secrets. Reports from older KeyHog versions may omit it; the HTML projection displays that state as not recorded rather than inventing an identifier. resolved_scan is omitted only for library-created reports that have no resolved CLI scan policy.

Cross-file correlation

Report schema 2.0 carries an optional correlations array after findings. It is present only when the scan ran with --correlate; without the flag the key is absent, not empty, and the rest of the report is unchanged. Correlation never adds, drops, reorders, or edits a finding, so a --correlate run and a default run produce the same findings array.

Each entry joins several findings into one credential risk:

value_reuse is not the same as additional_locations. Per-detector dedup folds repeats of ONE detector into a finding’s additional_locations; it never crosses detectors, so one value matched as two different detector ids in two files stays two unrelated findings until correlation joins them.

Which providers have composite halves is Tier-B data in crates/core/data/credential-correlation.toml, not a hardcoded list. A composite is reported only when a directory holds exactly one candidate credential for each required part and no single file holds them all: an ambiguous directory yields nothing rather than a guess, and a pair that already shares a file is left to the detector’s own companion match.

keyhog scan . --correlate --format json-envelope \
  | jq '.correlations[] | {kind, severity, evidence_score, title, files: .file_count}'

--format text renders the same groups as a Correlated credentials block above the results summary. Every other format is untouched by the flag.

Status and process exit are separate contracts

Machine consumers must read the status carried by a metadata-bearing artifact. Do not derive scan completeness from the process exit code:

A scanner panic exits 11, a required or explicitly selected GPU that is unavailable exits 12, and Ctrl-C exits 130. Blocking and live findings take precedence over an input-coverage failure in process-exit selection. A partial scan with such findings can therefore exit 1 or 10. Its envelope still says "scan_status":"partial". This is why detached consumers must inspect scan_status and coverage_gap_summary.

Legacy json and jsonl contain findings only. They cannot distinguish a complete zero-finding scan from an incomplete one. Use json-envelope, jsonl-envelope, SARIF, CSV with its CLI preamble, GitLab SAST, or JUnit when that distinction controls a gate.

scan_status alone is not a gate

Scanning any ordinary repository reports "scan_status":"partial", because the default walker prunes .git/ and node_modules/ during discovery and counts each pruned directory once as a coverage gap. Branch on the coverage_gap_summary reasons rather than on the status.

A scan that read nothing is a third case. --exclude-paths '**', a .keyhogignore containing path:**, and an empty stdin stream all read zero source bytes. That now exits 13 and carries a scan covered nothing gap row, and the text report says so instead of No secrets detected. Assert it anyway in any gate whose input path can change, because the assertion is cheap and it names the problem in the job log:

keyhog scan . --format json-envelope --output keyhog.json
jq -e '.metadata.source_bytes_scanned > 0' keyhog.json

That command exits 1 when the scan read nothing.

Tell a real clean from a skipped input owns the complete rule, including what each counter means and the shipped cases where a clean scan is wrong.

--format csv

CSV emits one row per finding. CLI scan output begins with one schema-2 metadata comment (# keyhog.scan.metadata=<JSON>) before the header. It records a schema version, terminal scan_status, backend_recoveries, and the complete coverage_gap_summary, so a zero-finding partial scan cannot be mistaken for a clean scan. CSV consumers should ignore comment lines before parsing the RFC 4180 header and data rows. The library-compatible ReportFormat::Csv renderer omits this preamble; the write_csv_coverage_report entrypoint emits it explicitly.

The companions_redacted, remediation, metadata, and additional_locations columns contain deterministic JSON objects or arrays. Metadata keys are sorted before serialization, and duplicate locations retain their complete source, path, line, offset, commit, author, and date fields. evidence_tier and evidence_reason_code are required textual columns. evidence_score and entropy are numeric columns that remain empty when the detection path did not measure them. Every textual cell is escaped with RFC 4180 quoting plus spreadsheet-formula neutralization; remediation is still emitted so a CSV artifact never loses the canonical action guidance.

Finding-field losslessness

Use the versioned envelope formats when a downstream system needs the complete finding model. The other formats are deliberate projections:

Fields not listed for a projection are intentionally unavailable in that format; they must not be inferred from stderr or the process exit code.

--format sarif

SARIF 2.1.0 is the preferred format for GitHub Code Scanning and SARIF-aware IDEs.

keyhog scan . --format sarif --output keyhog-results.sarif
status=$?
test "$status" -eq 0 -o "$status" -eq 1 -o "$status" -eq 10

The file remains available when blocking or live findings make KeyHog exit 1 or 10. Do not write a command chain that uploads the file only after an exit-zero scan.

The important machine fields have this shape. The values are synthetic and the message contains only the redacted credential:

{
  "version": "2.1.0",
  "runs": [{
    "results": [{
      "ruleId": "stripe-secret-key",
      "level": "error",
      "message": {"text": "stripe secret detected: sk_l...p7dc"},
      "locations": [{
        "physicalLocation": {
          "artifactLocation": {"uri": "src/config/.env.staging"},
          "region": {"startLine": 14, "charOffset": 218}
        }
      }],
      "properties": {
        "verification": "skipped",
        "evidence_tier": "likely",
        "evidence_reason_code": "vendor-pattern",
        "evidence_provenance": {
          "schema_version": 1,
          "detector_digest": "0123456789abcdef",
          "pattern_index": 0,
          "candidate_channel": "pattern",
          "source_role": "environment-assignment-value",
          "context_class": "vendor-pattern"
        },
        "evidence_score": 1.0,
        "cwe": "CWE-798",
        "owasp": "A07:2021",
        "remediation.action": "Roll the exposed Stripe secret key in the Dashboard, update production consumers, then delete the old key."
      }
    }],
    "properties": {
      "keyhog.scan.status": "success",
      "keyhog.backend.recoveries": []
    }
  }]
}

The full document also contains $schema, tool.driver, rules, taxonomies, optional fixes, and partial fingerprints. Consume those fields from the file rather than treating the abbreviated example as a complete SARIF document. runs[0].properties["keyhog.scan.status"] carries the terminal state. When coverage gaps exist, SARIF includes invocations[0], executionSuccessful is false, and the exact reasons appear in toolExecutionNotifications. Consumers must still read keyhog.scan.status because a cancelled or failed artifact is allowed to have no coverage notification.

Upload to GitHub even when the scan found credentials:

- name: Scan
  id: keyhog
  continue-on-error: true
  run: keyhog scan . --format sarif --output keyhog-results.sarif

- uses: github/codeql-action/upload-sarif@dd903d2e4f5405488e5ef1422510ee31c8b32357 # v3
  if: always() && hashFiles('keyhog-results.sarif') != ''
  with:
    sarif_file: keyhog-results.sarif

- name: Enforce KeyHog result
  if: always()
  env:
    KEYHOG_OUTCOME: ${{ steps.keyhog.outcome }}
  run: test "$KEYHOG_OUTCOME" = success

The final step fails for any nonzero KeyHog exit. If your policy permits unverified findings but rejects live credentials, capture the numeric exit in a wrapper instead of using the GitHub step outcome.

--format github-annotations

GitHub Actions workflow commands emit one annotation line per finding. Use this when you want findings to appear inline in the Actions log without uploading SARIF:

keyhog scan . --format github-annotations

Critical and high findings render as error annotations, medium and low as warning, and info as notice. Each annotation carries the file, line, title, detector, service, redacted credential, verification state, exact evidence tier and reason code, and optional evidence score. The plaintext credential is not emitted. When source coverage is incomplete, the formatter also emits one terminal ::warning notice with deterministic reason/count pairs, so the GitHub job log shows the incomplete state even when there are no findings. CLI output always also emits ::notice title=keyhog scan::scan status: success|partial|cancelled|failed; the legacy library-only ReportFormat::GithubAnnotations variant remains finding- only for compatibility.

SARIF carries the same terminal state in runs[0].properties["keyhog.scan.status"]; coverage gaps remain detailed in invocations[].toolExecutionNotifications.

--format gitlab-sast

GitLab SAST security-report JSON. Use it with artifacts:reports:sast when GitLab should ingest KeyHog findings into the pipeline Security tab:

keyhog:
  script:
    - keyhog scan . --format gitlab-sast --output gl-sast-report.json
  artifacts:
    reports:
      sast: gl-sast-report.json

GitLab SAST reports require every finding to have a file path and a one-based line number. If a non-file source cannot be represented in that schema, KeyHog fails the report with an error instead of fabricating a location. Use json or sarif for mixed file and non-file sources.

The scan.start_time and scan.end_time values come from the same report metadata used by HTML. This keeps CI artifacts and the human report aligned when a daemon or a long-running scan finishes at a different time than the reporting step began. If source coverage gaps occur, KeyHog emits the schema-supported scan.status: "failure"; a complete scan emits scan.status: "success". Because GitLab’s schema has no distinct cancelled or failed values, the nested scan.keyhog_scan_status extension preserves KeyHog’s exact success|partial|cancelled|failed state for detached-artifact consumers.

--format html

HTML is a self-contained interactive report. In addition to findings and coverage gaps, its metadata panel shows the terminal scan status, producing KeyHog version, scan interval, duration, redacted targets, source bytes and chunks, and detector count. The metadata is descriptive only; it never changes finding or exit-code semantics.

--format junit

JUnit XML contains one failing testcase per finding. The suite always contains keyhog.scan.status (success, partial, cancelled, or failed), and partial scans add one keyhog.coverage_gap property per reason/count pair. CI consumers can reject a partial artifact without scraping stderr.

--format jsonl

Legacy newline-delimited JSON retained for compatibility: one finding object per line and no header. Use --format jsonl-envelope when the stream needs a schema identity and explicit concatenation boundaries.

--format jsonl-envelope

Versioned newline-delimited JSON. The first line is a record_type: "header" object carrying the same schema_version major contract as --format json-envelope (JSONL has its own additive minor revision) and optional scan metadata; every following line is one finding object. The final line is a record_type: "summary" object with transport status: "complete", a scan_status of success, complete_after_recovery, partial, cancelled, or failed, the exact finding count, and the coverage-gap summary. An empty scan still emits both header and summary. A stream without the final summary is interrupted and must not be treated as complete; concatenated streams are split at the next header. Importers must validate both records before accepting the stream. This is better than --format json-envelope for streaming consumers that want to start processing before the scan finishes.

Retain the stream while you consume it, so you can check the summary the rule above requires:

keyhog scan /huge/monorepo --format jsonl-envelope \
  | tee keyhog.jsonl \
  | jq -r 'select(.record_type == null) | .location.file_path'

jq -e 'select(.record_type == "summary")
       | .scan_status == "success" or .scan_status == "complete_after_recovery"' \
  keyhog.jsonl

The second command exits nonzero when the summary is missing or reports an incomplete scan: jq -e returns 4 when the summary record is absent entirely, and 1 when it is present but the status is not one of those two. A consumer that reads only the finding lines and stops cannot tell a finished scan from a truncated one, because the finding records look identical in both.

Combining with --verify

--verify sends eligible findings to the detector’s declared verification endpoint. A live result keeps its severity. A dead or revoked result downgrades it by one tier. The machine value is one of "live", "dead", "revoked", "rate_limited", "unverifiable", "skipped", or an {"error":"..."} object.

set +e
keyhog scan . --verify --format json-envelope --output keyhog-results.json
status=$?
set -e

jq -e '.scan_status == "success" or
       .scan_status == "complete_after_recovery"' keyhog-results.json
jq '.findings[] | select(.verification == "live")' keyhog-results.json
test "$status" -ne 10

The first jq rejects incomplete input. The second emits only live findings. The final command enforces the documented live-credential exit while permitting exit 1. Select default or paranoid evidence policy to control which non-live finding tiers produce exit 1.

Findings-only output

On an interactive terminal keyhog scan shows a banner, a live progress ticker, and a completion summary on stderr. Most of the time you do not need to silence it: the banner and ticker are printed only when stderr is a TTY (they never appear in a pipe, a file, or CI logs), and the structured formats (json, json-envelope, jsonl, jsonl-envelope, sarif, csv, github-annotations, gitlab-sast, junit) carry structured findings and format-specific coverage metadata, with no banner or footer prose. So a CI script that wants machine output just selects a structured format:

keyhog scan . --format json

The text format does print a footer summary (counts + any skip summary) to stdout alongside the findings; if you want findings only, choose json/json-envelope/jsonl/jsonl-envelope/sarif/csv/github-annotations/gitlab-sast instead. The interactive banner is TTY-gated and never reaches a pipe or a file. Exit code semantics are unchanged by the format choice (see exit codes).

When you do want to silence the interactive chrome on a TTY (for example a local run whose stderr you are capturing), pass --quiet. It suppresses the banner, the progress ticker, and the “Scan complete” summary, but still prints coverage FAIL/WARN lines and fatal errors so a quiet scan can never be mistaken for a clean one. Use --no-color to drop ANSI styling regardless of whether output is a TTY (the NO_COLOR convention is also honored).

Recipes

Every recipe is a copy-paste command. Find your goal, paste the line, done. Put provider tokens in the documented environment variables, never on the command line. See environment variables and exit codes.

Find the right recipe

Each command scans one explicit source boundary. Run several recipes when your review spans several boundaries, and retain each json-envelope report with its raw exit code.

Scan code you have locally

keyhog scan .                              # canonical default policy
keyhog scan path/to/file.env              # one file; may use a ready Unix daemon
keyhog scan . --fast                      # pattern-only: no decode, entropy, or ML
keyhog scan . --deep                      # bounded highest-recall preset
keyhog scan . --precision                 # 0.85 floor, no entropy/relaxed keyword bridge
keyhog scan . --lockdown                  # Linux; requires sufficient memlock

Gate commits and pull requests

keyhog scan --git-staged                  # pre-commit: staged blobs (uses guard daemon if live)
keyhog scan --git-diff main               # only files changed since a base ref
keyhog scan --git-history .               # added lines from reachable commits, bounded by max_commits
keyhog scan --git-history . --max-commits 500

Guard a repository for instant pre-commit scans

# 1. Start daemon in background (reconciles durable roots)
keyhog guard up
# 2. Register repository (indexes baseline into memory once)
keyhog guard add /path/to/repo --mode repo

# 3. Inspect in-memory status and attestation metrics
keyhog guard status /path/to/repo

# 4. Staged commits now execute with sub-millisecond in-memory attestation caching
cd /path/to/repo && keyhog scan --git-staged

# 5. List all active guarded repositories
keyhog guard list

# 6. Free daemon memory whenever you finish working on a repository
keyhog guard remove /path/to/repo

Pre-commit framework: keyhog ships a hook, so a .pre-commit-config.yaml repo: https://github.com/santhreal/keyhog entry wires keyhog scan --git-staged into every commit. See perpetual guard and pre-commit.

Add it to CI (one workflow file)

# .github/workflows/keyhog.yml
name: keyhog
on: [push, pull_request]
permissions: { contents: read, security-events: write }
jobs:
  scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
      - uses: santhreal/keyhog@v0
        with: { path: ., severity: high, format: sarif, preset: default, lockdown: 'false' }

Findings upload to the GitHub Security tab as SARIF. Commit a baseline first so CI fails only on new secrets. See Adopt on a noisy repo, the GitHub Action guide, and the direct CI guide.

Scan an entire GitHub organization

export KEYHOG_GITHUB_TOKEN="$GH_PAT"
keyhog scan --github-org acme --format json-envelope --output acme.json

The command traverses the organization until the configured page, repository, and byte limits bind. The envelope records source identity and any remaining inventory as coverage gaps. See mass scanning.

Scan a single repo’s collaboration surfaces

Issues, pull requests, discussions, wikis, and gists carry secrets that never land in the tree:

export KEYHOG_GITHUB_TOKEN="$GH_PAT"
keyhog scan --github-collaboration acme/service --github-all

See GitHub collaboration scans.

Scan a GitLab group or Bitbucket workspace

KEYHOG_GITLAB_TOKEN="$GL_PAT" keyhog scan --gitlab-group acme      # incl. subgroups
KEYHOG_BITBUCKET_USERNAME="$BB_USER" KEYHOG_BITBUCKET_TOKEN="$BB_APP_PASSWORD" \
  keyhog scan --bitbucket-workspace acme

Scan a Docker image before you ship it

keyhog scan --docker-image registry/app:v1                # unpacks image layers

Audit a cloud bucket

keyhog scan --s3-bucket logs-prod --s3-prefix config/     # --s3-endpoint for non-AWS
keyhog scan --gcs-bucket logs-prod --gcs-prefix config/
keyhog scan --azure-container-url "$AZURE_CONTAINER_URL" --azure-prefix config/

Scan a URL, endpoint response, or HAR capture

keyhog scan --url https://api.example.com/config          # one or more URLs

See HTTP and wire scanning.

Pipe arbitrary text through

echo "$SOME_BLOB" | keyhog scan --stdin
kubectl get secret app -o yaml | keyhog scan --stdin

A producer that fails writes nothing to stdout. The scan then reads zero bytes and exits 13 with a scan covered nothing gap row, which is honest but blames the scanner rather than the producer. Make the pipeline carry the real failure:

set -o pipefail
kubectl get secret app -o yaml | keyhog scan --stdin

With pipefail, a missing kubectl or a denied request surfaces the producer’s own exit code. See tell a real clean from a skipped input.

Sweep an entire machine

keyhog scan-system --space 50G            # eligible mounts and discovered Git history, bounded at 50 GiB

See system-wide triage.

Confirm a finding is a live credential

keyhog scan . --verify                    # validate against provider APIs (exit 10 if live)
keyhog scan . --verify --verify-oob       # out-of-band verification server

See verification.

Adopt on a legacy or noisy repo

keyhog scan . --create-baseline .keyhog-baseline.json     # snapshot existing findings once
keyhog scan . --baseline .keyhog-baseline.json            # then report only NEW findings

Commit the first file. An entry matches on the detector and the credential value, not on the path, so moving a baselined secret does not fail the gate but rotating it does. The complete CI path, including monorepo partitions, is Fail only on new secrets.

Approve one exact fixture finding

Append a detector, path, and credential hash to the same rule:

cat >> .keyhogignore.toml <<'EOF'
[[suppress]]
detector = "aws-access-key"
path_eq = "fixtures/aws.env"
credential_hash = "5e884898da28047151d0e56f8dc6292773603d0d6aabbdd62a11ef721d1542d8"
EOF
keyhog scan .

All three fields must match. A different value in the fixture, or the same value in another path, still reports and keeps the findings exit. Invalid TOML stops the scan with exit 2; KeyHog does not ignore a broken policy. See suppressions.

Ignore one generated tree

cat >> .keyhogignore <<'EOF'
path:generated/**
EOF
keyhog scan .

The rooted pattern matches generated/app.js, not packages/web/generated/app.js. Use path:**/generated/** only if every generated directory is reviewed and safe to exclude. .keyhogignore has no negation or last-rule-wins override. An invalid entry stops the scan with exit 2.

Scan third-party archives without a false clean

rc=0
keyhog scan incoming/ --format json-envelope -o keyhog-archives.json || rc=$?
jq '{scan_status, coverage_gap_summary, findings: (.findings | length)}' \
  keyhog-archives.json
printf 'keyhog exit=%s\n' "$rc"

Corrupt, encrypted, unsafe, oversized, or truncated members produce coverage gaps. With no blocking finding, incomplete coverage exits 13, not 0. Blocking findings in the covered portion take exit 1, or 10 when verification confirms a live credential, while scan_status remains partial. See source archives.

Make the CI loop fast

keyhog scan . --incremental               # BLAKE3 Merkle skip of unchanged inputs
keyhog scan . --incremental --incremental-cache .keyhog-cache

Emit for any pipeline or SIEM

One engine, every dialect. Pick with --format:

keyhog scan . --format sarif -o keyhog.sarif          # GitHub / GitLab code scanning
keyhog scan . --format github-annotations             # inline PR annotations
keyhog scan . --format gitlab-sast -o gl-sast.json    # GitLab SAST report
keyhog scan . --format junit -o keyhog.xml            # JUnit for any CI dashboard
keyhog scan . --format jsonl-envelope                 # streaming machine contract
keyhog scan . --format csv -o findings.csv

Available formats: text · json · json-envelope · jsonl · jsonl-envelope · sarif · csv · html · junit · github-annotations · gitlab-sast.

Filter and set the gate

keyhog scan . --severity high             # info | client-safe | low | medium | high | critical
keyhog scan . --min-confidence 0.5        # raise the reporting confidence floor
keyhog scan . --exclude-paths vendor,node_modules

Exit 0 means no finding blocks the active evidence policy and no failing source gap occurred. It can still accompany advisory skip gaps and scan_status: partial, so it is not proof that skipped content was clean. Exit 1 means a finding blocks the selected evidence policy; 10 means at least one live credential under --verify; and 13 means failing source or coverage gaps when no blocking finding took precedence. A blocking or live finding can therefore exit 1 or 10 while scan_status remains partial. See the full precedence table in exit codes.

Perpetual repository and filesystem guard

The guard is a daemon-resident runtime that registers Git repositories and filesystem trees as guarded roots. It maintains an in-memory clean Git object attestation index and filesystem event tracking, enabling sub-second pre-commit scanning on staged changes without cold-start detector compilation or redundant file I/O.

Guard requires the Unix-domain daemon transport. On Windows, keyhog guard exits with an unsupported-platform error; use keyhog scan <path> in process.

The guard supplements staged and working-tree scans. It does not replace them. A commit is allowed only after the exact staged-object transaction proves the staged content is clean.

Three-step quickstart

1. Register the repository:

   keyhog guard add /path/to/repo
   

   Registers the repository with the guard daemon and installs the managed pre-commit hook at .git/hooks/pre-commit.

2. Start the daemon (if not already running):

   keyhog guard up
   

   Ensures the background daemon process is active and ready to handle scan requests. One daemon process serves all registered repositories.

3. Stage changes and commit:

   git add <files>
   git commit -m "commit message"
   

   The pre-commit hook runs keyhog scan --git-staged against the daemon, verifies staged object IDs against cached attestations, and blocks the commit if credentials are detected.

Core mental model

1. One-command registration (keyhog guard add <path>): Indexes the target repository into daemon memory once and establishes clean baseline attestations.
2. Fast staged commit gating (keyhog scan --git-staged): Pre-commit hooks query the active guard daemon. The daemon verifies only changed staged blob OIDs against in-memory attestations, skipping unchanged clean payloads.
3. Full lifecycle control: List active roots with keyhog guard list, check memory and attestation metrics with keyhog guard status <path>, and free daemon memory immediately with keyhog guard remove <path> when finished working on a project.

Lifecycle commands

Detailed walkthrough

1. Start the daemon

keyhog guard up

guard up ensures the daemon is active in the background, compiles the active 934-detector corpus once, and stays resident in memory. One daemon process serves all guarded repositories and scan requests.

2. Register a repository

keyhog guard add /path/to/repo --mode repo

• --mode repo (default): Uses Git object IDs (OIDs) for exact immutable staged-content identification, and automatically installs the managed pre-commit hook at .git/hooks/pre-commit (best-effort; skipped if a foreign hook already exists, or if --no-hook is passed).
• --mode filesystem: Uses file content hashes without Git OIDs.

The command waits for initial baseline reconciliation to complete before returning:

OK guard: root /path/to/repo registered (state stopped, sequence 1)
OK guard: reconciliation complete, root is current

3. Check guarded status

Inspect in-memory metrics, cache efficiency, and policy binding:

keyhog guard status /path/to/repo

Human-readable output:

root:           /path/to/repo
mode:           repo
state:          current
sequence:       2
accepted seq:   2
completed seq:  2
pending events: 0
files scanned:  142
bytes scanned:  1849204
cache hits:     0
cache misses:   142
findings:       0
coverage gaps:  0
initial recon:  2026-08-17T00:15:00Z
last recon:     2026-08-17T00:15:00Z
residency:      resident
backend route:  gpu-cuda-region-presence
build digest:   1a2b3c4d5e6f7a8b
detector:       934-dc43f6629978321b
suppression:    0000000000000000
config:         18cc6ed841bf6dfe
autoroute:      calibrated
store schema:   1

Structured JSON output for monitoring and scripts:

keyhog guard status /path/to/repo --format json

4. Run instant staged commit scans

Inside the guarded repository, run:

keyhog scan --git-staged

The command connects to the guard daemon via Unix domain socket, checks the staged Git blob OIDs against in-memory attestations, and returns in milliseconds.

• Clean commit outcome:

  No secrets detected in the scanned files.

• Blocked commit outcome:

  ┌    CRITICAL ─── OpenAI API Key
  │ Secret:     sk-9...M8vZ
  │ Location:   client.ts:4
  │ Evidence:   likely/vendor-pattern  ■■■■■■ 100%
  │ Entropy:    5.383 bits/byte
  │ Action:     Revoke immediately at the provider, rotate dependent credentials, and audit recent usage.
  └─────────────────────────────────────────────

  ━━━ Results ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
  1 secret found · 1 unverified
  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

5. List all guarded repositories

Check which repositories are currently guarded:

keyhog guard list

Output:

OK 2 guard roots registered
  /path/to/repo-a  current  seq=14
  /path/to/repo-b  current  seq=3

6. Free resources when finished

When you finish working on a project, remove it from the guard to immediately reclaim daemon memory and watcher resources:

keyhog guard remove /path/to/repo

guard remove unregisters the root from the daemon and removes any KeyHog-owned pre-commit hook by default (pass --keep-hook to preserve the hook). You can re-add the repository at any time with keyhog guard add /path/to/repo.

Pre-commit hook integration

Standalone Git hook

Create or update .git/hooks/pre-commit in your repository:

#!/usr/bin/env bash
set -euo pipefail

# Run staged scan against guard daemon (falls back to in-process if daemon is off)
keyhog scan --git-staged

Make the script executable:

chmod +x .git/hooks/pre-commit

pre-commit framework

Add the following to .pre-commit-config.yaml:

repos:
  - repo: https://github.com/santhreal/keyhog
    rev: v0.5.81
    hooks:
      - id: keyhog
        stages: [pre-commit]

Bypassing checks and suppressing test fixtures

Emergency single-commit bypass

To bypass the pre-commit hook for a single urgent commit:

git commit --no-verify -m "urgent fix"

This bypasses local Git hooks. Use it sparingly.

Suppressing intentional test fixtures and false positives

When committing intentional test fixtures, mock data, or vendor example keys:

1. Suppress by credential hash (.keyhogignore): Add the SHA-256 hash of the value to .keyhogignore:

   hash:5e884898da28047151d0e56f8dc6292773603d0d6aabbdd62a11ef721d1542d8
   

2. Scoped suppression (.keyhogignore.toml): Target the exact detector and file path in .keyhogignore.toml:

   [[suppress]]
   detector = "stripe-secret-key"
   path_eq = "tests/fixtures/mock_stripe.env"
   reason = "reviewed synthetic test fixture"
   

3. Inline source directive: Append // keyhog:ignore detector=<detector-id> or # keyhog:ignore detector=<detector-id> directly to the source line. Without detector=, the directive suppresses every finding on that line. After committing the suppression rule, run keyhog guard reconcile <path> if you need to update the in-memory baseline immediately.

Guard state machine

Every guarded root operates within a 7-state machine owned by GuardRootState:

Process exit codes

keyhog guard status and keyhog scan --git-staged enforce strict exit semantics:

How clean attestations work

When KeyHog scans a staged Git blob and detects zero unsuppressed secrets, it records a clean attestation record keyed by four immutable elements:

1. Blob Git OID: The SHA-1 or SHA-256 object hash of the staged blob.
2. Byte Length: The exact byte length of the blob payload.
3. Policy Identity Digest: The 32-byte digest of the active detector corpus, suppression rules, and scanner configuration.
4. Sorted Source-Path Set: The hashed set of all sorted staged source paths mapped to that blob.

Future commit transactions matching all four elements skip payload re-scanning and return an instant cache hit. If a file is renamed, moved across source roles, or an alias is added, the attestation is invalidated. Persisted policy identity records contain no plaintext staged paths.

Guard configuration

The guard runtime resolves settings from the [guard] table in .keyhog.toml:

[guard]
# Periodic scrub interval for `current` roots (e.g. "5m", "24h").
# Default: disabled.
# scrub_interval = "5m"

# Durable redb state database path (default: disabled, ephemeral memory).
# state_path = "~/.local/state/keyhog/guard.redb"

# Memory budget ceiling for hot clean attestation index (default: "64MB").
# hot_index_memory = "64MB"

# Maximum queued filesystem events per root before degraded status (default: 8192).
# max_pending_events_per_root = 8192

# Maximum total queued filesystem events across all roots (default: 65536).
# max_pending_events_total = 65536

# Event coalescing window before applying state transitions (default: "100ms").
# coalesce_window = "100ms"

# Scanner residency mode: "warm" (keep loaded) or "idle-unload" (unload after idle timeout).
# scanner_residency = "warm"

# Scanner idle timeout before reporting `idle-unload` residency (default: "5m").
# scanner_idle_timeout = "5m"
# Maximum files scanned during one subtree reconciliation (default: 10000).
# subtree_max_files = 10000

# Maximum directory depth during subtree reconciliation (default: 64).
# subtree_max_depth = 64

Durable state persistence

By default, guard state is held in daemon memory. To persist root registrations and clean attestations across daemon restarts, configure state_path in .keyhog.toml:

[guard]
state_path = "~/.local/state/keyhog/guard.redb"

The durable store uses a high-performance redb database with owner-only (0600) file permissions, enforces 0700 permissions on its parent directory, and rejects symlinked state paths. On daemon restart, persisted roots load in the stopped state. Running keyhog guard reconcile /path/to/repo re-verifies the repository and transitions it back to current.

In lockdown mode ([lockdown] require = true), durable persistence is disabled and the guard operates strictly in ephemeral memory.

Periodic scrubbing

Configure scrub_interval in .keyhog.toml to periodically re-verify current repositories:

[guard]
scrub_interval = "24h"

Scrubbing detects modifications made outside standard kernel filesystem events (such as NFS mounts, container volume mutations, or out-of-band Git object manipulation).

Unauthoritative filesystems

When registering a root, KeyHog automatically probes the backing filesystem type. Local filesystems (such as ext4, btrfs, xfs, apfs, and ntfs) generate kernel change events reliably and are classified as authoritative.

Network filesystems (nfs, cifs/smb, 9p, afs, ceph), userspace/virtual filesystems (fuse, overlay), and unrecognized filesystem types do not reliably generate real-time local kernel notifications. When an unauthoritative filesystem is registered and no operator scrub_interval is configured, KeyHog enforces a default 60-second periodic scrub interval to guarantee that remote modifications are caught.

When the scrub interval elapses, each current root automatically transitions to dirty for re-reconciliation.

Recovering corrupted roots

If a repository’s durable state becomes corrupt or desynchronized, rebuild it:

keyhog guard rebuild /path/to/repo

rebuild clears the root’s durable database entries, re-registers the root, and triggers a clean baseline reconciliation.

Pre-commit hook

A pre-commit hook stops credentials before they enter repository history. It scans staged content and blocks the commit on findings. It also blocks when the scan cannot complete, so an unavailable scanner cannot look clean.

Install in one command

From inside a git repo:

keyhog hook install

If a non-KeyHog pre-commit hook already exists, installation refuses to replace it. Pass keyhog hook install --force only when replacement is intentional; keyhog hook uninstall removes only the KeyHog-owned hook.

That writes a .git/hooks/pre-commit script that calls keyhog scan --fast --git-staged --backend cpu (the same command .pre-commit-hooks.yaml exposes for the pre-commit framework). The next git commit invokes the hook.

If keyhog is missing from PATH, the hook blocks the commit because the security scan did not run. Install KeyHog, fix PATH, or remove .git/hooks/pre-commit if the repository should not be protected.

What the installed hook does not catch

Read this before you rely on the hook. --fast keeps named and multiline pattern matching and drops decode, entropy, and ML work. A credential written in plain text is blocked. A credential that is only Base64-encoded is not. Stage a file whose sole credential is Base64-encoded and the commit succeeds:

$ git commit -m "add config"
No secrets detected in the scanned files.
[main 8f56830] add config
 1 file changed, 1 insertion(+)

The default policy decodes that file and reports the credential. Check any staged change the hook passed:

keyhog scan --git-staged

Treat the hook as a fast first gate, not as the control that protects the branch. Keep a default-policy scan in CI, where the cost is paid once per push rather than once per commit. See Fail only on new secrets.

Fast pre-commit scanning with Perpetual Guard

For fast pre-commit gating with the full default policy (including complete decoding, entropy analysis, and all 934 detectors), use the perpetual KeyHog daemon:

1. Ensure the guard daemon is active in the background:

   keyhog guard up
   

2. Register the repository and install the pre-commit hook in one step:

   keyhog guard add . --mode repo
   

keyhog guard add registers the repository in daemon memory, performs the initial baseline reconciliation, and automatically installs the managed pre-commit hook at .git/hooks/pre-commit (pass --no-hook to skip hook installation).

Because the daemon maintains an in-memory clean Git blob attestation index, keyhog scan --git-staged checks only changed staged blobs against the daemon’s in-memory index, skipping unchanged clean payloads rather than re-scanning them.

See the perpetual guard guide for full lifecycle management.

pre-commit framework

This repository’s hook uses language: system. Follow the exact-version install, then confirm that KeyHog is on PATH:

keyhog --version

Add the following to .pre-commit-config.yaml:

repos:
  - repo: https://github.com/santhreal/keyhog
    rev: v0.5.81
    hooks:
      - id: keyhog
        stages: [pre-commit]

Run pre-commit install once. The hook then runs on every commit. The rev pin selects the hook definition. It does not install or pin the keyhog binary. Keep the binary and hook definition on compatible release versions.

What gets scanned

keyhog scan --git-staged walks the index (the set of files git is about to commit), not the working tree. Why this matters:

• A file you’ve modified but not git added is NOT scanned. You’re free to keep credentials in scratch files as long as you don’t stage them.
• A file you’ve staged then modified gets scanned in the staged form, not the working-tree form. The scanner sees what git commit would commit.

The walk only includes files that are part of this commit. Runtime depends on the staged bytes, detector corpus, binary, and host; use the command’s reported duration to characterize a repository.

What happens on a finding

Stderr:

$ git commit -m "add staging config"
  ┌    CRITICAL ─── Stripe Secret Key
  │ Secret:     sk_l...p7dc
  │ Location:   src/config/.env.staging:14
  │ Evidence:   likely/vendor-pattern  ■■■■■■ 100%
  │ Action:     Roll the exposed Stripe secret key in the Dashboard, update production consumers, then delete the old key.
  │ Docs:       https://docs.stripe.com/keys#roll-api-key
  └─────────────────────────────────────────────

  ━━━ Results ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
  1 secret found · 1 unverified

  1. Revoke active secrets in the provider's dashboard.

The hook runs exec keyhog scan --fast --git-staged --backend cpu, so this is the ordinary scan report over the staged blobs. Exit 1 means a finding blocks the default evidence policy and aborts the commit. Exit 10 would mean a confirmed live credential, although the shipped hook does not enable verification. Every operational nonzero exit also aborts the commit because the scan did not complete. Your staged work remains intact. You can then:

1. Remove the credential, stage the fix, and commit again.
2. Replace it with a placeholder and load the value from the environment.
3. For a false positive, add its hash to .keyhogignore or add a narrowly scoped predicate rule to .keyhogignore.toml. Record the reason and owner beside the exception.

When you really need to commit anyway

git commit --no-verify

That bypasses the hook. KeyHog logs nothing about it; that’s your prerogative. Use it sparingly. A team norm of --no-verify for “trust me” commits defeats the point of the hook.

A better pattern when a legitimate-looking credential needs to ship (e.g. a public OAuth client_id that vendor docs say to commit):

1. Add its hash to .keyhogignore as hash: + the bare 64-character SHA-256 hex digest (no sha256: prefix; that spelling is baseline-file-only):

   hash:5e884898da28047151d0e56f8dc6292773603d0d6aabbdd62a11ef721d1542d8
   

2. Commit the suppression file alongside the credential.
3. The next commit sees the hash and skips it.

This way the next contributor doesn’t have to learn the trick.

Performance

Pre-commit scans operate in two modes:

1. In-process staged scan: When no daemon is running, keyhog scan --git-staged evaluates staged Git blobs in process with the CPU backend.
2. Guard daemon commit transaction: When a KeyHog daemon is active with guarded roots (keyhog guard up), keyhog scan --git-staged connects over the Unix socket. The daemon checks its in-memory Git OID clean attestation index, skips unchanged clean blobs, and scans only modified payloads.

The installed pre-commit hook runs keyhog scan --fast --git-staged --backend cpu, which uses the guard daemon automatically when reachable and falls back to in-process execution when absent.

Uninstall

keyhog hook uninstall

This removes .git/hooks/pre-commit only when it carries the generated KeyHog marker. If you edited the hook, keyhog hook uninstall refuses to touch it. Remove that hook by hand. For the pre-commit framework, delete the KeyHog stanza from .pre-commit-config.yaml and run pre-commit clean.

GitHub Action

Use the KeyHog Action when you want to scan one checked-out repository path in a GitHub Actions job. A published Action ref installs its exact KeyHog crate with the lean ci feature. A reviewed branch or commit ref builds the checked-out portable source profile and requires backend: cpu. Both paths run the scan, retain the report as a workflow artifact, and upload SARIF to GitHub Code Scanning by default.

For GitLab, CircleCI, Jenkins, and other CI systems, use the CI integration guide. For organization-wide repository or cloud inventory scans, use the mass-scanning guide.

Scan a repository

Create .github/workflows/keyhog.yml:

name: keyhog

on:
  push:
    branches: [main]
  pull_request:

permissions:
  contents: read
  security-events: write

jobs:
  scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
      - uses: santhreal/keyhog@v0
        with:
          path: .
          severity: high

This workflow scans the checked-out working tree. It fails when KeyHog reports a finding at high or critical severity. It also fails on configuration, coverage, backend, installation, and report-publication errors.

The default sarif report appears in two places:

• Security > Code scanning contains the uploaded SARIF results.
• The workflow run contains a keyhog-report-* artifact for later review.

The Action retries one failed Code Scanning upload on trusted pushes and same-repository pull requests. It fails the job if both attempts fail. GitHub removes security-events: write from fork pull requests, so the upload is advisory for that event. The workflow artifact is still retained, and findings still fail the scan.

Choose the workflow for the source

The Action scans one file or directory already present in the runner workspace. Use the CLI directly when the scan source needs additional flags.

Do not use a single repository Action job as an organization-wide inventory scanner. Inventory scans need explicit partitions, independent reports, source limits, and retry boundaries.

Adopt KeyHog without blocking existing findings

Create a baseline from a reviewed local scan:

keyhog scan . --create-baseline .keyhog-baseline.json
git add .keyhog-baseline.json
git commit -m "chore: add KeyHog baseline"

Then pass the committed file to the Action:

- uses: santhreal/keyhog@v0
  with:
    path: .
    baseline: .keyhog-baseline.json

A baseline suppresses findings it already contains. New likely and confirmed findings fail under the default evidence policy; new review findings remain visible. Set evidence-policy: paranoid to make review-tier findings block. The equivalent repository setting is [scan].evidence_policy = "paranoid" in .keyhog.toml. Review baseline changes like source changes. Do not regenerate the baseline inside CI.

An entry matches on the detector and the credential value, never on the file path. Moving or renaming a baselined file keeps it suppressed, and copying the same credential into a new file keeps it suppressed too. See Baselines for the complete matching rules and how to retire an entry after rotation.

For an advisory rollout, set fail-on-findings: 'false'. Non-live findings that block the active evidence policy then remain visible without blocking the job. A verified-live credential and every operational failure still fail.

- uses: santhreal/keyhog@v0
  with:
    path: .
    fail-on-findings: 'false'

Scan a monorepo

Give each partition a stable, unique analysis-category. GitHub uses this value to keep Code Scanning result sets separate across commits.

strategy:
  matrix:
    include:
      - path: services/api
        category: services-api
      - path: services/web
        category: services-web

steps:
  - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
  - uses: santhreal/keyhog@v0
    with:
      path: ${{ matrix.path }}
      analysis-category: ${{ matrix.category }}

Keep a category unchanged when the partition remains the same. Use a different category for every KeyHog scan in the same job. A category contains 1 to 64 lowercase letters, digits, dots, underscores, or dashes, and it starts and ends with a letter or digit.

Add baseline per partition when each team reviews its own exceptions:

  - uses: santhreal/keyhog@v0
    with:
      path: ${{ matrix.path }}
      analysis-category: ${{ matrix.category }}
      baseline: ${{ matrix.path }}/.keyhog-baseline.json

One root baseline works just as well when a single team reviews every exception, because matching ignores the path. Choose by ownership. The CI guide compares both layouts.

Select detection policy

The preset input selects one detection policy:

default passes no preset flag. A discovered .keyhog.toml may therefore select fast = true, deep = true, or precision = true. An explicit non-default Action preset takes normal CLI precedence over the file.

lockdown: 'true' is separate from the preset. It can be used with default, deep, or precision. KeyHog rejects fast with lockdown instead of weakening either request. Lockdown requires Linux and enough locked-memory capacity for the scanner process. Standard GitHub-hosted Linux runners do not currently provide that capacity. Use a provisioned self-hosted runner when lockdown is a required control.

Control credential verification

Verification is off by default:

- uses: santhreal/keyhog@v0
  with:
    verify: 'false'

The Action always passes either --verify or --no-verify, so the input overrides a committed verify setting. When verification is enabled, eligible detectors may send credential or companion material to provider endpoints in a URL, query, header, or request body. Review the detector corpus and your outbound trust boundary before setting verify: 'true'.

A confirmed live credential exits with code 10 and always fails the Action, even when fail-on-findings is false.

Inputs

Boolean inputs are strings in GitHub Actions. Use quoted 'true' and 'false'. Invalid values fail before scanning.

The Action wrapper supports four report formats because it validates the report and finding count before publication. Use the CLI directly for CSV, HTML, JUnit, GitLab SAST, and envelope formats.

Outputs

Give the step an id before reading its outputs:

- id: keyhog
  uses: santhreal/keyhog@v0
  with:
    fail-on-findings: 'false'

- name: Record the finding count
  env:
    KEYHOG_FINDINGS: ${{ steps.keyhog.outputs.findings }}
  run: printf 'KeyHog findings: %s\n' "$KEYHOG_FINDINGS"

Check report-present before consuming report. The report path is private to the job and disappears during runner cleanup. Do not reconstruct its parent path or treat it as a persistent artifact. Use the uploaded workflow artifact for retention across jobs or runs.

Pin Action code and scanner releases

The floating major ref follows the latest published v0 release:

- uses: santhreal/keyhog@v0

Use an exact Action ref when workflow code must change only through review:

- uses: santhreal/keyhog@v0.5.81

The optional version input pins the scanner crate. It does not pin the Action implementation. Pin both when both contracts must remain fixed.

Release refs install the exact scanner version from crates.io. A missing version fails the job. A reviewed branch or commit ref builds the portable source profile with the repository’s pinned Rust toolchain and requires backend: cpu. It does not silently substitute source for a missing crate.

Failure behavior

Treat these states separately:

• findings > 0 means the scan completed and reported credentials at the chosen severity floor. Review-tier findings can coexist with exit 0.
• scan-status: partial means the report contains a coverage gap. Inspect the report and raw exit-code; do not treat it as clean.
• scan-status: failed or report-present: false means the Action did not publish a trusted report.
• Exit 10 means verification confirmed a live credential. It always blocks.
• Installation, configuration, source, backend, and report validation failures remain failures even in advisory mode.

See the exit-code reference for the complete CLI contract and the output-format guide for report fields.

CI integration

Add KeyHog in two stages: make findings visible with a durable report, then turn new findings into a merge gate. Most repositories already contain credentials, so start at Fail only on new secrets for the complete path from a first scan to a passing gate. The provider recipes below keep scanning, enforcement, and report retention explicit so a missing upload or unsupported source cannot look like a clean run.

The shell recipes use an Ubuntu worker and install the full default portable crate profile. The GitHub Action has a different installation contract: published refs install the lean ci feature, while branch and commit refs build the portable profile from checked-out source and require backend: cpu.

Fail only on new secrets

This is the usual adoption path for a repository that already holds credentials you cannot rotate today. The findings that exist now stay visible in a committed baseline. Only findings added after that point fail the build.

1. Create and commit the baseline

Run this once, on a clean local checkout of the branch CI scans:

keyhog scan . --create-baseline .keyhog-baseline.json

The command writes the file, prints no findings, and exits 0. Read the file before committing it. Every entry is a credential you are choosing to accept.

git add .keyhog-baseline.json
git commit -m "chore: add KeyHog baseline"

2. Gate the build

keyhog scan . --baseline .keyhog-baseline.json --format json-envelope --output keyhog.json

Exit 0 means no new finding blocks the active evidence policy. Exit 1 means a blocking credential is present that the baseline does not list. Review-tier findings remain visible under the default policy; select paranoid to block them. Exit 13 means the scan could not cover its input; a baseline never suppresses that, so do not read it as complete. Keep keyhog.json on every outcome. The generic shell wrapper is the portable way to retain the report and the exact exit code together.

A scan that reads zero source bytes exits 13 with a scan covered nothing gap row, so an --exclude-paths glob that matches everything, a container mount that landed empty, or a partition path that no longer exists fails the job rather than passing it. Assert the byte count anyway when the input path can change, because the assertion names the problem in the job log instead of leaving a reader to decode an exit code:

jq -e '.metadata.source_bytes_scanned > 0' keyhog.json

See tell a real clean from a skipped input.

3. Respond when the gate fires

A failing gate means someone added a credential. Remove it from the code and rotate it at the provider. When the finding is a reviewed exception instead, pick the narrowest surface:

• One exact value in one path: add a [[suppress]] rule to .keyhogignore.toml.
• A credential the team accepts everywhere: run keyhog scan . --update-baseline .keyhog-baseline.json locally, review the diff, and commit it.

Never run --update-baseline inside CI. A job that rewrites its own baseline accepts every secret it finds. The flag also still reports the new findings and still exits 1, so it cannot turn a red job green.

Monorepos: one baseline or several

A baseline entry matches on the detector and the credential value, never on the path. One root baseline therefore covers every partition, and moving code between partitions never fires the gate. Choose by who reviews exceptions, not by how matching works:

Per-partition baselines have one consequence worth knowing. A credential accepted in services/api is reported again when the same value appears in services/web, because that job loads a different file. That is usually what you want.

Run each partition as its own job so a failure names an owner:

strategy:
  fail-fast: false
  matrix:
    partition: [services/api, services/web]
steps:
  - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
  - name: Scan partition
    env:
      PARTITION: ${{ matrix.partition }}
    shell: bash
    run: |
      keyhog scan "$PARTITION" \
        --baseline "$PARTITION/.keyhog-baseline.json" \
        --format json-envelope \
        --output "keyhog-${PARTITION//\//-}.json"

fail-fast: false keeps the other partitions running after one fails, so a single leak does not hide the rest. Give each partition its own --incremental-cache path and cache key if you enable incremental scanning.

For what a baseline matches, how to retire an entry after rotation, and how to compare two baselines, see Baselines.

CI speed and concurrency

One KeyHog process uses the available CPU cores by default. Leave --threads unset on a dedicated runner. When a matrix runs several KeyHog jobs on one shared worker, divide the worker’s CPU budget across them with --threads <N> so every process does not claim the full host. Set --reader-threads only after --profile shows a storage-reader bottleneck.

Use --incremental only when the CI cache is bound to the same trusted repository and partition. Give each monorepo partition a separate --incremental-cache path and cache key. A cache hit changes work reuse, not the selected source boundary or detection policy.

Do not use --fast as the only merge or release gate. It intentionally omits decode, entropy, and ML work. It is suitable for an additional short feedback job when the default policy still runs before merge. Directory and Git jobs run in process; a warm daemon does not accelerate them.

Live verification has a separate network budget. Use --verify-concurrency, --verify-rate, or --verify-batch based on provider limits rather than CPU count.

GitHub Actions

Use the GitHub Action guide for the maintained composite Action, its inputs and outputs, monorepo categories, baseline adoption, report retention, and failure semantics.

Use the CLI directly in GitHub Actions when you need a source option that the Action does not expose. For example, fetch complete ancestry before scanning reachable commit additions:

- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
  with:
    fetch-depth: 0
- name: Scan reachable history
  run: keyhog scan --git-history . --format sarif --output keyhog.sarif

Install KeyHog before these steps. Capture the exact scan status, upload the report, then restore that status after the upload:

- name: Scan reachable history
  id: keyhog
  shell: bash
  run: |
    scan_status=0
    keyhog scan --git-history . --format sarif --output keyhog.sarif \
      || scan_status=$?
    printf 'exit-code=%s\n' "$scan_status" >> "$GITHUB_OUTPUT"
- name: Upload KeyHog SARIF
  if: always()
  uses: github/codeql-action/upload-sarif@dd903d2e4f5405488e5ef1422510ee31c8b32357 # v3
  with:
    sarif_file: keyhog.sarif
- name: Enforce scan result
  if: always()
  env:
    KEYHOG_EXIT: ${{ steps.keyhog.outputs.exit-code }}
  shell: bash
  run: exit "$KEYHOG_EXIT"

The capture step exits successfully so the upload can run. The enforcement step then restores every KeyHog finding, live-credential, panic, backend, system, and coverage status without translating it to a generic failure.

Exclusions and adoption policy

Use exclusions for content that should not be scanned, and a baseline for known findings that should remain visible but not block adoption:

• Put generated trees, vendored fixtures, and intentionally synthetic corpora in .keyhogignore as path: rules. Keep a short comment explaining each exclusion; broad globs can hide real coverage.
• Put finding-specific exceptions in .keyhogignore or .keyhogignore.toml, preferably with reason, expiry, and approval metadata.
• Never convert a source failure or coverage gap into an exclusion. KeyHog uses distinct nonzero exit semantics for invalid configuration, system failures, unavailable required GPU execution, and incomplete sources.

An exclusion decides which bytes are read. A baseline decides which findings count as new. Reach for an exclusion only when the content should not be scanned at all; otherwise use Fail only on new secrets. In a monorepo, never hide one team’s paths behind another team’s ignore file. Give each team its own subdirectory job with its own report.

GitLab CI

# .gitlab-ci.yml
keyhog:
  stage: test
  image: rust:1.89-bookworm
  before_script:
    - cargo install --locked --version '=0.5.81' keyhog
  script:
    # Exits non-zero on findings, which fails the job and gates the MR.
    - keyhog scan . --format gitlab-sast --output gl-sast-report.json
  artifacts:
    when: always           # keep the report even when the scan fails the job
    reports:
      sast: gl-sast-report.json
    paths:
      - gl-sast-report.json

The job’s exit status gates the merge request. KeyHog emits GitLab’s SAST JSON schema directly, so artifacts:reports:sast publishes findings to the merge request security widget without a converter. The same report remains a downloadable artifact when the scan fails.

CircleCI

# .circleci/config.yml
version: 2.1

jobs:
  keyhog:
    docker:
      - image: cimg/rust:1.89
    steps:
      - checkout
      - run:
          name: Install keyhog
          command: |
            cargo install --locked --version '=0.5.81' keyhog
            echo 'export PATH="$HOME/.cargo/bin:$PATH"' >> $BASH_ENV
      - run:
          name: Scan repo
          command: keyhog scan . --format sarif --output keyhog.sarif
      - store_artifacts:
          path: keyhog.sarif
          destination: keyhog.sarif

workflows:
  build:
    jobs:
      - keyhog

Drone CI

# .drone.yml
kind: pipeline
type: docker
name: default

steps:
  - name: keyhog
    image: rust:1.89-bookworm
    commands:
      - cargo install --locked --version '=0.5.81' keyhog
      - |
        scan_status=0
        keyhog scan . --format json-envelope --output keyhog.json \
          2>keyhog.stderr || scan_status=$?
        printf '%s\n' "$scan_status" > keyhog.exit-code
        cat keyhog.stderr >&2 || true
        exit "$scan_status"

  - name: publish-keyhog-report
    image: plugins/s3
    settings:
      endpoint:
        from_secret: keyhog_artifacts_endpoint
      bucket:
        from_secret: keyhog_artifacts_bucket
      access_key:
        from_secret: keyhog_artifacts_access_key
      secret_key:
        from_secret: keyhog_artifacts_secret_key
      source: keyhog.*
      target: keyhog/${DRONE_REPO}/${DRONE_BUILD_NUMBER}
    when:
      status:
        - success
        - failure

The S3-compatible publisher runs after clean scans, findings, and operational errors. Configure its four keyhog_artifacts_* secrets for your artifact store. The scan step exits with KeyHog’s exact status after writing keyhog.exit-code and replaying keyhog.stderr to the job log.

Generic shell

Use the same scan wrapper in Jenkins, Buildkite, Woodpecker, Concourse, or any CI that can run a POSIX shell:

#!/bin/sh
set -eu

printf '%s\n' '{"schema_version":{"major":2,"minor":0},"scan_status":"failed","coverage_gap_summary":[],"findings":[]}' > keyhog.json
scan_status=0
keyhog scan . --format json-envelope --output keyhog.json \
  2>keyhog.stderr || scan_status=$?
printf '%s\n' "$scan_status" > keyhog.exit-code
cat keyhog.stderr >&2 || true
exit "$scan_status"

Configure the CI artifact publisher to retain keyhog.json, keyhog.stderr, and keyhog.exit-code on both success and failure. When the output path is writable, KeyHog writes the report even when every source failed to read; that report then carries scan covered nothing and the reason each source failed. An output-path failure exits 2 and cannot produce that report. Evaluate a present report together with keyhog.exit-code.

Buildkite

Use a dedicated artifact path so the report survives a finding exit:

# .buildkite/pipeline.yml
steps:
  - label: ":mag: keyhog secret scan"
    command: |
      cargo install --locked --version '=0.5.81' keyhog
      keyhog scan . --severity high --format json-envelope --output keyhog.json
    artifact_paths:
      - keyhog.json

Jenkins

Archive the report in post so it remains available when the scan blocks the stage:

// Jenkinsfile
pipeline {
    agent any
    stages {
        stage('keyhog') {
            steps {
                sh '''
                    cargo install --locked --version '=0.5.81' keyhog
                    keyhog scan . --severity high --format json-envelope --output keyhog.json
                '''
            }
            post {
                always {
                    archiveArtifacts artifacts: 'keyhog.json', allowEmptyArchive: true
                }
            }
        }
    }
}

Pin the scanner version

Manual CI installation can pin one exact crates.io version:

cargo install --locked --version '=0.5.81' keyhog

Review the release before changing the version. GitHub Action code and scanner crate pinning are separate contracts; see Pin Action code and scanner releases.

Scan commit additions on main and release, not per PR

An added-line history scan is useful on main post-merge and on release tags, but it is overkill for every PR. Add --git-blobs . when the policy must also cover reflogs, stashes, tag messages, and unreachable objects that remain in the repository object database. A typical setup:

Duration depends on history size, changed bytes, verification endpoints, rate limits, runner hardware, and cache state. Record it from the actual job.

The PR scan keeps the dev feedback loop fast. The post-merge history scan catches anything that slipped through pre-commit + PR review. The release scan verifies what’s live, useful for the changelog (“rotated these N credentials before shipping”).

Mass scanning

Use the mass-scanning guide for repository organizations, hosted Git groups, cloud buckets, local partitions, source limits, report aggregation, and retry boundaries. A mass scan is an inventory workflow, not a larger version of a pull-request job.

Run each partition as an independent CI job. Retain its machine-readable report, raw exit code, source inventory, and coverage state before aggregating results.

Failure modes worth knowing

• Forked PR + secret credentials: GitHub Actions doesn’t expose org secrets to forked-PR runners, so a verifier endpoint that needs authentication won’t run. Findings still get reported as unverified; that’s correct behavior.
• Advisory findings: preserve the raw KeyHog exit separately from report publication, then decide explicitly whether exit 1 blocks the job. A verified-live credential exits 10 and should remain blocking.
• Shallow clones fail the job: actions/checkout defaults to fetch-depth: 1, which exposes only the checked-out HEAD commit. A --git-history or --git-blobs scan of that clone now exits 13 with "scan_status":"partial" and a Git object unreadable or wrong object kind gap, because the graft boundary names parent commits the clone does not contain. Set fetch-depth: 0 on any job that scans history. A depth-1 clone of a single-commit repository stays clean and correct, because its boundary is the root commit and hides no parent.
• --git-history misses branches you did not check out: it covers the ancestry present in the current checkout only. A credential on another local ref, in a reflog or stash, or left behind by git commit --amend can be reported by --git-blobs and missed by --git-history with no coverage gap. Use --git-blobs when the policy has to cover the repository object database, not only reachable added-line history. Objects already pruned from the clone cannot be scanned.
• A base ref that resolves to HEAD: keyhog scan --git-diff <BASE> exits 0 after scanning zero bytes when <BASE> and HEAD are the same commit, which is what a shallow clone gives you for origin/main. The gate passes without examining anything. A base ref that is missing from the clone is safe by comparison: that exits 13 and refuses to report clean.
• LFS files: keyhog reads the LFS pointer file, not the contents. To scan LFS-stored binaries, enable LFS in checkout (lfs: true) and let the scanner pull the real file.

Deep recovery

Git history keeps a credential after you delete it from the working tree. Scan that history explicitly. A filesystem scan never includes it.

keyhog scan --git-history . --format json-envelope --output history.json

--git-history scans the lines added by commits reachable from the current checkout, bounded by --max-commits and by the ancestry the clone contains.

A shallow clone reports the gap instead of a clean scan. git clone --depth N records a graft boundary, and the parent commits named there were never fetched, so their blobs cannot be scanned. Both --git-history and --git-blobs count those absent parents, exit 13, and report "scan_status":"partial" with a Git object unreadable entry in coverage_gap_summary. Read that as “this clone does not contain the history you asked me to search”, not as a scanner failure. The fix is on the checkout side:

git fetch --unshallow && keyhog scan --git-history .

actions/checkout clones one commit by default, so set fetch-depth: 0 on any job that scans history. See failure modes.

Choose the Git boundary before changing anything else:

The two boundaries differ by more than cost. A credential committed on a branch you never checked out, or left behind by git commit --amend, is reachable through --git-blobs and invisible to --git-history.

A committed archive is a known gap in both. --git-blobs refuses a blob that decodes as binary, so bundles/payload.tar.gz is reported as Git object unreadable and the scan exits 13 rather than reporting the archive clean. Scan the checked-out working tree as well when the repository carries archives: the filesystem source does extract them.

Neither boundary reaches sibling repositories, provider organizations, cloud buckets, or mounted filesystems. Run those boundaries as separate jobs and preserve their reports and statuses.

Deep Git scans run in process. A warm daemon does not accelerate history or blob traversal.

Add the deep preset

Use --deep when recall matters more than routine scan cost, such as an incident review or a release gate:

keyhog scan --deep --git-history . --git-blobs . --daemon=off \
  --format json-envelope --output deep-history.json

For a multi-backend installation, calibrate the deep policy before relying on automatic routing:

keyhog calibrate-autoroute --policy deep
keyhog config --effective --deep

A normal installation already calibrates every preset. Re-run the first command after changing the binary, host, driver, or routing-relevant configuration. The second command prints the resolved policy. Record it with benchmark or incident results.

When a report is written as json-envelope, jsonl-envelope, or html, its metadata contains a resolved_scan manifest. The manifest records the selected preset, every effective detection value, and the keys that differ from that preset’s base. This makes a deep run with compatible overrides directly comparable to default, fast, and precision artifacts:

{
  "schema_version": 1,
  "preset": "deep",
  "effective": {"max_decode_depth": "3", "entropy_enabled": "true"},
  "overrides": ["max_decode_depth"]
}

Values are strings by contract so the manifest remains stable as new typed settings are added; maps are serialized in key order. It contains no paths, credentials, or host-specific routing decisions. The benchmark runner should store this object alongside timing and accuracy so results are never compared across silently different detection policies.

What changes

--deep is a bounded preset, not an unbounded evaluator.

ML remains enabled. Deep retains its score as evidence but does not let the model alone discard an entropy candidate. Explicit compatible flags apply on top of the preset, such as --deep --decode-depth 3.

The decode input ceiling is the row with teeth, because the default value sits BELOW the 1 MiB window the reader hands to the scanner. A window larger than --decode-size-limit is not decoded at all, so nothing Base64, hex, or URL-encoded inside it is recovered.

Measured on release-fast v0.5.81, one Base64-wrapped AWS key pair at the end of an otherwise plain text file, default preset:

The result looks random in file size and is exact in WINDOW size: the payload in that table sits at the end of the file, so it lands in the last window, and it is recovered when that window fits under the limit. Windows are 1 MiB with 128 KiB of overlap, so the last window’s size cycles as the file grows, which is why 1100 KiB succeeds and 1500 KiB does not.

Position, not size, is what governs. Same 2000 KiB file, same bytes, only the offset of the payload differs: at the end of the file the default preset reports the credential, and in the middle of the file it reports nothing. A full-size window is always over the default limit, so the tail is the only place an encoded payload is reachable at all, and every size in that table is really measuring how big the tail happened to be. Read the table as a demonstration, not as a rule you can plan around, and write any regression fixture with the payload mid-file, where the outcome is the same at every size.

Two remedies, in order of preference:

keyhog scan . --decode-size-limit 4M
keyhog scan . --deep

Raise the limit when you want only this behavior; --deep raises it to 1 MiB as part of a wider preset. Prefer the explicit flag when you are pinning a policy, because it changes one setting instead of five.

Do not gate on the exit code here. The skip is reported as a coverage gap naming --decode-size-limit, and that gap is the signal, whether or not the run also reported findings. A run that reports findings AND the gap is telling you the truth: some windows were expanded and others were not. See file shapes for the same measurement from the input-shape side.

Backend routing

Deep is a detection preset. It does not select a backend. The default --backend auto looks up evidence calibrated for the deep preset and the exact resolved overrides:

keyhog scan . --deep
keyhog scan . --deep --decode-depth 3

Those two scans have different resolved configuration identities. If the second scan reports an uncovered workload, measure that exact diagnostic shape once, then return to normal automatic routing:

keyhog scan . --deep --decode-depth 3 \
  --autoroute-calibrate --autoroute-gpu
keyhog scan . --deep --decode-depth 3

Use an explicit backend only to isolate an engine problem:

keyhog scan . --deep --backend cpu
keyhog scan . --deep --backend simd
keyhog scan . --deep --backend gpu-cuda
keyhog scan . --deep --backend gpu-wgpu

An explicit backend bypasses autoroute evidence. It does not repair or calibrate the deep route. It is a hard contract, so an unavailable SIMD runtime, GPU driver, or selected GPU peer fails the scan. KeyHog does not silently continue with another backend. Use keyhog --version --full for discovery and keyhog backend --self-test --require-gpu to execute the GPU diagnostic paths.

Recovery mechanisms

Deep runs the normal detector corpus and expands bounded recovery around it:

• recursive Base64, hex, URL, Unicode escape, and supported transport decoding;
• source-file entropy discovery for unknown opaque values;
• comment scanning without the normal comment penalty;
• static JavaScript recovery for recognized cyclic XOR expressions;
• static AES-256-CBC recovery when the key, IV, ciphertext, and bindings are literal and internally consistent;
• static CryptoJS passphrase recovery for the exact immutable wrapper dialect, with strict OpenSSL Salted__, EVP_BytesToKey MD5, AES-256-CBC, PKCS#7, and UTF-8 validation.

Static program recovery does not execute JavaScript or invoke Node.js. It accepts a small side-effect-free grammar and rejects dynamic operands. The implementation lives in crates/scanner/src/decode/javascript_static.rs and its aes.rs and cryptojs.rs submodules.

The grammar reads spellings, not one canonical style. Within the XOR and Node AES rules these are all the same program and all recover:

• const, let, or var for a binding;
• String.fromCharCode or String.fromCodePoint;
• Buffer.from(literal, encoding) or new Buffer(literal, encoding);
• 'aes-256-cbc' in any case, and hex, base64, utf8, or utf-8 in any case, in single quotes, double quotes, or backticks;
• .toString() with no argument, which is Node’s UTF-8 default;
• a plain string literal wherever a [...].join('') chain is accepted.

What still fails closed is anything the grammar cannot prove is constant: a binding written to after its declaration, including through an index or a mutating array method; a template literal containing ${...} or a backslash escape; new Buffer(size); any algorithm other than AES-256-CBC; and any operand that is not a literal. A refused program produces no plaintext, and the original source stays in the ordinary scan path.

The CryptoJS rule is deliberately stricter and stays at const or let. It resolves names through scope analysis rather than the whole-source occurrence count the other rules use, and var hoisting could let a sibling-scope binding win.

The static evaluator caps source size, literal arrays, binding count, and expression count. Decode recursion also enforces depth, output-size, expansion, and total-work budgets. A rejected transform still leaves the original source available to ordinary detection.

Read recovery receipts

Deep static recovery and backend recovery are separate. Inspect both from the metadata-bearing report:

jq '{
  preset: .metadata.resolved_scan.preset,
  static_recovery: .metadata.static_recovery,
  scan_status,
  backend_recoveries: (.metadata.backend_recoveries // [])
}' deep.json

static_recovery counts supported, unsupported, and erroneous bounded program transforms. These counters describe the JavaScript, AES, and CryptoJS evaluator. They do not mean that a scan backend failed.

backend_recoveries records recovery after an authenticated automatic backend faults. Each row names the failed backend, the backend that completed the stable bytes, recovered range, chunk, and byte counts, a non-secret reason, and a repair command. scan_status: "complete_after_recovery" means byte coverage is complete, but the route still needs repair. Missing or invalid autoroute evidence creates no recovery row; it leaves input unscanned and reports scan_status: "partial".

Use the receipt to remediate the route:

• Run keyhog calibrate-autoroute --policy deep for the standard deep ladder.
• Re-run an exact deep scan once with --autoroute-calibrate --autoroute-gpu when compatible overrides created an uncovered configuration.
• Run installer calibration for Git, Docker, or web source workloads.
• If a GPU backend faulted, run keyhog backend --self-test --require-gpu, repair the driver or runtime, and recalibrate. Inspect the quarantine with keyhog backend --autoroute.
• If SIMD faulted, confirm the running build and Hyperscan/Vectorscan runtime with keyhog --version --full, repair it, and recalibrate.

Do not replace these repairs with a permanent --backend cpu setting. That would bypass the invalid autoroute state rather than restore measured routing.

Non-LLM recovery benchmark

The repository’s ioc-recovery corpus contains 4,368 labeled fixtures across 13 JavaScript concealment phases. It has exact expected credentials and is scored by the same benchmark runner used for other corpora.

The paper authors publish 13 demonstration files in the pinned llm-ioc-detection repository. Their 336-program evaluation corpus is not present there. KeyHog’s 4,368 fixtures are deterministic synthetic adaptations of the phase taxonomy, not copies of the paper’s evaluation files.

Reproduce the checked benchmark matrix:

make -C benchmarks ioc-recovery-corpus
make -C benchmarks ioc-recovery

The committed deep target requires 4,368 true positives, zero false negatives, and zero false positives for the pinned corpus and scanner identity. The fast comparison reports 1,344 true positives and 3,024 false negatives. These are corpus results, not a claim that every possible program transform is supported.

• Executable deep target
• Corpus and scorer contract

The reproduction commands write local artifacts under benchmarks/results-ioc-recovery/. Those artifacts are intentionally ignored because timings and hardware identity are host-specific. Each artifact records the mode, backend, cache and daemon state, scanner version, corpus size, exact detection totals, wall time, and peak RSS. Compare results only when those identities match.

File shapes and sizes

Three file shapes need a different command from the repository default. Many small files. One very large file. A minified or single-line file.

This page gets you a correct scan of each, names the limit that applies, and shows the check that separates a real clean from a skipped input.

Many small files

Scan the root. There is no special flag:

keyhog scan <root> --format json-envelope -o keyhog.json

Confirm the file count reached the scanner:

jq '.metadata.source_chunks_scanned' keyhog.json

A file under 1 MiB is one chunk. For a tree of small files, that count should be close to the number of files you expect. If it is far lower, something removed files before the scanner saw them.

The usual cause is the default exclusion policy. On a repository it removes .git/, lockfiles, vendored trees, and minified bundles, which is correct. On a directory of generated or vendored content it can remove almost everything. Check what it removed:

jq '.coverage_gap_summary' keyhog.json

An exclusion policy gap with a count near your file count means the default skips ate the input. Pass --no-default-excludes when you deliberately want vendored and minified paths walked, and read Minified and single-line files first, because that flag turns off two separate rules and you should know both.

Limits that bind on this shape:

• --threads <N> caps parallel scanner workers. Unset uses the available cores.
• --max-file-size still applies per file, so one oversize file in the tree produces a gap while the rest of the tree scans normally.

Cost on a 5,000-file tree of about 240 KiB: median peak resident memory 154 MiB over five runs. Peak memory on this shape is dominated by the compiled detector corpus, not by the file count, so it barely moves as the tree grows. Plan against memory, and measure wall time on your own runner: the same five runs took a median of 0.84 s on an idle-ish host and 7.73 s on a heavily loaded one, so a wall time quoted here would tell you nothing about yours.

One very large file

The default --max-file-size is 100 MiB. A larger file is not scanned.

Passing a 256 MiB file with the default cap fails loudly:

keyhog scan bigfile.log

WARN skipping file: size exceeds --max-file-size cap path=bigfile.log
     size_bytes=268436133 max_size=104857600
WARN source: failed to read source: ... file was not scanned.
error: a requested scan source failed to read and produced no data (see the
warnings above). Not reporting "clean": that scan did not run.

Exit code 13.

Raise the cap to scan it. The value requires a unit:

keyhog scan bigfile.log --max-file-size 300M --format json-envelope -o keyhog.json

A bare number is rejected before the scan starts, with exit 2:

error: invalid value '300000000' for '--max-file-size <SIZE>': byte size
'300000000' is missing a unit. Use `B`, `K`/`KB`, `M`/`MB`, `G`/`GB`, or `T`/`TB`.

keyhog watch takes the same option as a bare byte count, not a size string. That asymmetry is real. See Watch mode.

Files above the 1 MiB window size are read in overlapping windows, so a credential that straddles a window boundary is still found. The 256 MiB file above, with a credential on its last line, reports one finding and 293 chunks.

Cost on the same tree: median peak resident memory 315 MiB over five runs, for a 256 MiB file. Peak memory does not track file size one-for-one, because windows stream to the scan pool as they are decoded rather than being collected first. It is still the number to check before raising the cap on a shared runner.

The dangerous case is a large file inside a normal tree

When the oversize file is one member of a directory, the rest of the directory scans and the run does not stop:

keyhog scan mixed-tree/ --format json-envelope -o keyhog.json

{
  "scan_status": "partial",
  "coverage_gap_summary": [
    {"reason": "source emitted error rows (requested input was not fully scanned)", "count": 1},
    {"reason": "exceeded --max-file-size", "count": 1}
  ]
}

Exit code is 13 when nothing was found in the covered part, and 1 when something was. In the 1 case the report still carries the gap. Read coverage_gap_summary, not the exit code alone.

Do not raise --max-file-size just to clear exit 13. Raise it when you trust the input and the runner has the memory. Otherwise scan the large file as its own job so its cost and its result are separate.

Minified and single-line files

A minified file is one whose whole content sits on one or two very long lines. A single-line file is any file with no newline until the end.

Line length is not a problem. A 32 MiB single-line JavaScript bundle named bundle.js scans normally: 37 chunks, one finding for a credential near the end, and median peak resident memory 144 MiB over five runs, the same as an ordinary tree. Windowing is by bytes, not by lines.

The problem is the filename and the directory.

What the default policy does

A directory containing only app.min.js reports no findings and zero source_bytes_scanned. The file never reaches the scanner. The report carries two gap rows and exits 13:

{
  "scan_status": "partial",
  "coverage_gap_summary": [
    {"reason": "scan covered nothing (zero source bytes read; every candidate was skipped by exclusion or skip policy, so nothing was examined)", "count": 1},
    {"reason": "exclusion policy (default excludes such as lock files, minified/bundled assets, vendored and build-output trees; --git-staged also counts repository `.keyhogignore` matches here)", "count": 1}
  ]
}

A scan that reads zero bytes is a loud failure, not a clean result. The second row tells you which policy did it.

What --no-default-excludes now does

--no-default-excludes turns off both layers: the walker reads the file, and the post-match drop is disabled, so the credential is reported. The same directory with the flag reads 1441 bytes and reports the finding.

Without the flag, a match is still dropped after the fact when its path ends in .min.js, .bundle.js, or .min.css, or sits under node_modules/, bower_components/, jspm_packages/, site-packages/, wp-includes/, wp-content/plugins/, wp-content/themes/, public/plugins/, public/static/, public/vendor/, static/vendor/, dist/vendor/, dist/assets/, or vendor/assets/. Each drop is counted, and the total appears in coverage_gap_summary as its own row, so the suppression is visible in the report rather than silent.

Measured: a directory holding only app.min.js with a planted credential reports zero findings by default, and reports the credential with --no-default-excludes. --dogfood shows the individual suppressed matches and their reason, which is more detail than the gap-row count:

keyhog scan dist/ --no-default-excludes --dogfood --format json-envelope

The stderr trace names the reason vendored_minified_path.

Scan a bundle you own

Pass one first-party bundle as an explicit file, or disable default exclusions for the directory that owns it:

keyhog scan dist/app.min.js --format json-envelope -o keyhog.json
keyhog scan dist/ --no-default-excludes --format json-envelope -o keyhog-dist.json

An explicit file request is not removed by the directory walker’s default path policy. Use --no-default-excludes when the scan must cover several minified or vendored-shaped paths. Review that broader scope first: it also enables third-party bundles that the repository default intentionally omits.

This matters most for a first-party bundle you ship. A vendored third-party bundle you did not write is what the suppression is for, and leaving it suppressed is usually right.

Encoded payloads: position decides

In any file over 1 MiB, a credential inside an encoded payload is only found when it sits in the last part of the file. Everywhere else it is missed, on the default preset, with no warning that fails your build. This is the sharpest edge on this page.

--decode-size-limit defaults to 512K and bounds decoding per WINDOW, not per file. Files over 1 MiB are read as 1 MiB windows, so every window except the tail is over the limit and is never decode-expanded.

Measured on one file whose only credential is a Base64 payload on the LAST line, at the default preset:

The table looks random in file size and is exactly predictable in something else. A credential at the END of a file lands in the LAST window. Windows are 1 MiB with 128K of overlap, so they advance 896K at a time and the last window holds whatever is left over. Decoding happens per window, so:

A payload is decoded when the window holding it is at or under --decode-size-limit.

At 1100K the last window holds only 204K and the payload is found. At 1500K it holds 604K and is not. At 2000K it holds 208K and is found again.

Position matters more than size

Every row above puts the credential at the end of the file, which is the only position that can succeed. Hold the size fixed and move the payload instead:

Same size, same bytes, opposite result. A payload anywhere but the tail sits in a full 1 MiB window, and a full window is always over the 512K limit.

So the honest rule is not about file size at all. In any file larger than 1 MiB, only the tail is decode-reachable, and the whole interior is not. The bigger the file, the smaller that reachable fraction: a 12 MB file is roughly 1.6% decode-reachable.

Measured on a 2.2 GB Rust registry checkout, 377 of 110,846 files are over 512K and hold 595 MB between them. Of that, 574 MB is decode-unreachable and 20 MB is reachable through tail windows.

The underlying cause is that two defaults are ordered wrongly. The window size, 1 MiB, is larger than the decode limit, 512K, so a full-size window can never be decode-expanded no matter what it contains. That ordering alone accounts for 173 MB of the 574 MB, in files between 512K and 1 MiB whose single window merely exceeds the cap. The remaining 422 MB is genuine interior of files over 1 MiB, which only a subdividing decode path can reach.

The report does say so, in the one place worth reading:

{
  "reason": "scanner decode-through declined by --decode-size-limit (chunk larger than the limit; raw bytes scanned, nothing encoded inside it was recovered)",
  "count": 1
}

scan_status is partial and the exit code is 0. This is a skip, not an error, so nothing fails your build. Gate on the gap reason, not the exit code.

The gap also appears on the rows that DID report findings, at 1100K and 2000K above. That is correct, not a false positive: those files still hold oversize chunks whose encoded content was never expanded, and a differently-split chunk happened to recover this particular payload. Findings plus this gap means partial decode coverage, not complete coverage.

Raise the limit when your inputs carry encoded payloads:

keyhog scan . --decode-size-limit 4M --format json-envelope -o keyhog.json

That recovers the credential at every size and every position tested above.

--deep also recovers them, but do not rely on it for this. Deep raises the decode ceiling to exactly 1 MiB, which is exactly the window size, so it clears the limit with no margin at all. Any increase to the window, or a ceiling compared as strictly-less-than rather than at-most, silently reopens the hole for deep too. Use --decode-size-limit when the decode budget is what you need, and --deep when you want the rest of its policy as well.

Check coverage on every shape

jq '{bytes: .metadata.source_bytes_scanned, chunks: .metadata.source_chunks_scanned,
     status: .scan_status, gaps: .coverage_gap_summary}' keyhog.json

Zero bytes means nothing was scanned, whatever the findings list says. Tell a real clean from a skipped input explains each field and each gap reason.

Container images and OCI layers

Scan an image by reference:

keyhog scan --docker-image registry/app:v1 --format json-envelope -o image.json

KeyHog runs docker image save for that reference, then streams each layer tarball through the shared in-memory archive scanner. Layer members are not materialized onto disk before scanning. A credential baked into a layer is found even when a later layer deletes the file, because every layer is scanned independently: whiteout and opaque-dir markers are ordinary members, not a reason to hide earlier-layer content.

Nested members keep the same coverage as an unpack-then-walk scan: gzip/zip/tar/compressed payloads descend in memory, .7z/.rar use the shared path extractors (staged from the already-buffered member), and layer .har files expand at the Docker boundary with wire:har labels. Nested .har inside ordinary zip/tar/7z/RAR keep the historical filesystem/archive leaf identity. Large already-UTF-8 plain layer members stream in ~1 MiB windows from the tar entry; UTF-16 and other encodings keep the whole-member decode path. Windowed members keep the filesystem/archive source identity.

What you need

--docker-image shells out to the docker CLI and needs a reachable Docker daemon. It does not talk to a registry itself, so the image must already be present locally or pullable by that daemon.

Pull first when the image is remote:

docker pull registry/app:v1
keyhog scan --docker-image registry/app:v1

A reference the daemon cannot resolve fails loudly:

keyhog scan --docker-image registry/app:no-such-tag

WARN source: failed to read source: failed to export docker image:
registry/app:no-such-tag: Error response from daemon: No such image: ...
error: a requested scan source failed to read and produced no data (see the
warnings above). Not reporting "clean": that scan did not run.

Exit code 13. That is the behavior you want. A typo in a tag never reads as a clean image.

Scanning a saved tarball

docker image save writes an OCI layout whose layer payloads are blobs/sha256/<digest> files with no extension. Scanning that tarball as a path works:

docker image save registry/app:v1 -o app.tar
keyhog scan app.tar --format json-envelope

For a one-layer image holding a live-shaped credential, that reports the finding. A container member is admitted by its own leading bytes when its name carries no recognized extension, so an extensionless gzip or tar layer is descended into rather than treated as opaque.

Extracting the tarball first works too:

mkdir -p /tmp/image-audit
tar xf app.tar -C /tmp/image-audit
keyhog scan /tmp/image-audit --format json-envelope -o image.json

Prefer --docker-image when you have a daemon. It takes a reference rather than asking you to produce and manage a tarball, and a bad reference fails loudly instead of scanning a file that is not the image you meant.

If you want the layer contents as ordinary files, for example to map a finding back to a path inside the image, unpack the layers yourself:

mkdir -p /tmp/image-audit/layers
for blob in /tmp/image-audit/blobs/sha256/*; do
  if file -b "$blob" | grep -q gzip; then
    mkdir -p "/tmp/image-audit/layers/$(basename "$blob")"
    tar xzf "$blob" -C "/tmp/image-audit/layers/$(basename "$blob")"
  fi
done
keyhog scan /tmp/image-audit/layers --format json-envelope -o layers.json

An image in a repository or a bucket is not covered

Container handling applies to files on disk and to the sources that expand them. It does not apply to Git objects or to cloud object bodies. An image tarball committed to a repository, or sitting in an S3 bucket, is not descended into.

The same bytes behave differently depending on how you reach them:

keyhog scan repo/ --no-default-excludes      # descends into the tarball
keyhog scan --git-history repo               # does not

The working-tree scan reports the credential inside the archive. The Git-history scan reports a binary (extension or content sniff) gap for that blob and exits 0 with a partial status. Running --git-blobs instead reports the same gap and exits 13.

That gap row is easy to misread. A binary gap usually means an image or a compiled object you did not want scanned. Here it means an archive whose contents were never examined. If your repository holds committed archives and you scan history, unpack them and scan the result as a separate job.

Limits

Three caps bound image expansion. Each one fails loudly when it binds.

A cumulative cap that binds stops the whole export:

keyhog scan --docker-image registry/app:v1 --limit-docker-tar-total-bytes 100B

WARN source: failed to read source: docker archive cumulative size exceeds 100
bytes at entry 'blobs/sha256/0d8eec63...' (likely zip-bomb).
error: a requested scan source failed to read and produced no data ...

Exit code 13.

A per-entry cap that binds skips that entry and keeps going, with the skip recorded:

keyhog scan --docker-image registry/app:v1 --limit-docker-tar-entry-bytes 10B \
  --format json-envelope

{
  "scan_status": "partial",
  "coverage_gap_summary": [
    {"reason": "source emitted error rows (requested input was not fully scanned)", "count": 1},
    {"reason": "exceeded --max-file-size", "count": 1}
  ]
}

Exit code 13, because nothing was found in the part that was covered.

The caps exist because an image layer is attacker-controllable compressed data. Raise them for an image you trust, on a runner with the memory to hold the expansion. Do not raise them to make exit 13 disappear on an image you pulled from somewhere you do not control.

Check coverage

jq '{bytes: .metadata.source_bytes_scanned, chunks: .metadata.source_chunks_scanned,
     status: .scan_status, gaps: .coverage_gap_summary,
     findings: (.findings | length)}' image.json

For an image scan, sanity-check bytes against the image size you expect. A multi-hundred-megabyte application image that reports a few kilobytes was not unpacked.

Prove your pipeline can see into a layer before you trust it. Build a canary image, scan it, and confirm the finding:

mkdir -p /tmp/canary-image
printf 'STRIPE_SECRET_KEY=sk_live_%s\n' \
  "$(head -c 32 /dev/urandom | base64 | tr -dc 'A-Za-z0-9' | head -c 24)" \
  > /tmp/canary-image/creds.env
printf 'FROM scratch\nCOPY creds.env /etc/creds.env\n' > /tmp/canary-image/Dockerfile
docker build -t keyhog-canary:v1 /tmp/canary-image
keyhog scan --docker-image keyhog-canary:v1 --format json-envelope \
  | jq '.findings | length'

Expect 1. A pipeline that reports 0 there is not scanning layers, and no real image will tell you so.

Tell a real clean from a skipped input covers the report fields and every gap reason.

Standard input and pipelines

Scan bytes that never touch disk:

kubectl get secret app -o yaml | keyhog scan --stdin

keyhog scan - is the same as keyhog scan --stdin, following the grep and wc convention:

terraform output -json | keyhog scan -

Findings from stdin carry the source stdin and no file path. The file_path field in the report is null, because there is no file.

The result depends on your working directory

A stdin scan has no filesystem path, so KeyHog resolves its allowlist from the current directory. A .keyhogignore in the directory you happen to be standing in is applied to bytes that have nothing to do with that repository.

Reproduce it. The same credential on stdin, the same binary, two directories:

mkdir -p /tmp/allowlist-demo
VALUE="sk_live_$(head -c 32 /dev/urandom | base64 | tr -dc 'A-Za-z0-9' | head -c 24)"
HASH="$(printf '%s' "$VALUE" | sha256sum | cut -d' ' -f1)"
printf 'hash:%s; reason="demo"; approved_by="you"\n' "$HASH" \
  > /tmp/allowlist-demo/.keyhogignore

cd /tmp          && printf 'STRIPE_SECRET_KEY=%s\n' "$VALUE" | keyhog scan --stdin
cd /tmp/allowlist-demo && printf 'STRIPE_SECRET_KEY=%s\n' "$VALUE" | keyhog scan --stdin

The first exits 1 with one finding. The second exits 0 with none, over identical bytes. There is no coverage gap, source_bytes_scanned is the same 51 in both runs, and scan_status is complete_after_recovery either way. Every field that normally distinguishes a skipped input from a scanned one says this was a real scan that found nothing.

This matters because the recommended pipeline above is usually run from a checkout.

Only hash: rules can affect a pipe. A path: rule has no path to match against, so it does nothing here: piping the same credential from a directory whose .keyhogignore contains path:** still reports the finding. Judge your exposure by the number of live hash: entries, not by the length of the file.

Two ways to avoid it:

cd / && kubectl get secret app -o yaml | keyhog scan --stdin

Run the pipe from a directory with no .keyhogignore, or write the input to a file outside the checkout and scan the path, which resolves the allowlist from the input’s own location:

kubectl get secret app -o yaml > /tmp/secret.yaml
keyhog scan /tmp/secret.yaml

Check which allowlist file is in play before you trust a clean stdin scan:

keyhog config --effective

The allowlist_file line names it.

Empty input fails the scan

The command in front of the pipe is the part that usually fails. A kubectl call against the wrong context, a vault read without a token, or a curl that 404s all produce empty output. KeyHog treats that as a failed scan rather than a clean one:

printf '' | keyhog scan --stdin

Exit code 13. The envelope reports scan_status partial, one chunk, zero bytes, and one coverage gap:

{
  "reason": "scan covered nothing (zero source bytes reached the scanner and no skip was counted; nothing was examined, so this result is not a clean bill of health)",
  "count": 1
}

This is a change in behavior. An empty stream used to exit 0 with scan_status success and no gaps, which read as a clean scan. If you have a pipeline that legitimately feeds an empty stream, for example a matrix job whose partition is sometimes empty, it will now fail. Guard the producer instead of suppressing the exit code:

if [ -s changed.diff ]; then
  keyhog scan --stdin < changed.diff
fi

There is deliberately no flag that turns the failure off.

Check the byte count, always:

kubectl get secret app -o yaml \
  | keyhog scan --stdin --format json-envelope -o keyhog.json
jq -e '.metadata.source_bytes_scanned > 0' keyhog.json

jq -e exits non-zero when the expression is false, so that line fails the pipeline when nothing was scanned.

Set pipefail as well, so the producer’s own failure is not swallowed:

set -o pipefail
kubectl get secret app -o yaml \
  | keyhog scan --stdin --format json-envelope -o keyhog.json

Without pipefail the shell reports only KeyHog’s exit code, and KeyHog succeeded at scanning nothing.

pipefail is a Bash and Zsh feature, not a POSIX one. Under dash, which is /bin/sh on Debian and Ubuntu, set -o pipefail fails with set: Illegal option -o pipefail and the guard you thought you had is not there. Give the script a #!/usr/bin/env bash shebang, or run it with bash, or capture the report and branch on the exit code as shown below, which needs no shell option at all.

Piping KeyHog into jq throws its exit code away

The reverse direction has the same hazard and is easier to get wrong, because it looks like a summary rather than a gate:

keyhog scan . --format json | jq '.findings | length'

A shell reports the exit status of the LAST command in a pipeline. That line reports jq’s status, not KeyHog’s. A bad flag, an unreadable config, or a detector corpus that will not compile all exit 2 and print nothing on stdout, and the pipeline still succeeds. Add || echo 0 and you have converted every one of those into the answer you were hoping for.

Two ways to keep the exit code. Turn on pipefail:

set -o pipefail
keyhog scan . --format json | jq '.findings | length'

That reports 2 when the scan failed. Or capture first and parse second, which also lets you inspect what was written:

rm -f keyhog.json
rc=0
keyhog scan . --format json-envelope -o keyhog.json || rc=$?
[ "$rc" -le 1 ] || { echo "keyhog failed: exit $rc" >&2; exit 1; }
jq '.findings | length' keyhog.json

Capturing is the more robust of the two, because a scan that fails before it can report writes no file at all, so the missing file is a second independent signal that something went wrong.

The size limit

--stdin accepts 10 MiB by default. A larger stream fails closed:

keyhog scan --stdin < big.json

WARN source: failed to read source: stdin exceeds 10485760 byte limit.
error: a requested scan source failed to read and produced no data (see the
warnings above). Not reporting "clean": that scan did not run.

Exit code 13. Nothing is scanned, including the first 10 MiB. The limit is not a truncation point.

Raise it when a larger stream is intentional:

keyhog scan --stdin --limit-stdin-bytes 20M < big.json

An 11 MB input is first spooled to an anonymous temporary file so the limit is validated before any partial result can escape. KeyHog then scans overlapping 1 MiB windows with 128 KiB of boundary coverage. Memory stays bounded, findings retain absolute offsets and line numbers, and independent windows can use the configured scan workers. The anonymous file is removed automatically when the source closes.

--limit-stdin-bytes applies only to --stdin. It does not bound a directory scan.

Warm routing

An eligible stdin request can be served by a running daemon, which avoids recompiling the detector corpus per invocation. That is the case worth a daemon: a pipeline that runs keyhog scan --stdin many times.

keyhog daemon start
kubectl get secret app -o yaml | keyhog scan --stdin

--daemon=auto is the default and is safe to leave on unattended. A daemon failure degrades to an in-process scan rather than losing the run, and for the default configuration the findings are the same.

Do not read that as byte-identical. Coverage can differ across the fallback boundary: scanner-side gaps such as the decode-cap skip below do not cross the daemon wire, so the two routes can report a different coverage_gap_summary for the same input even when the findings match.

What the pipeline cannot tell you is whether the daemon was actually used. Omitting the flag prints nothing on either path, so a daemon you started but are not reaching looks exactly like one that is working. Pass --daemon=auto explicitly when that matters, and it says so on stderr when it fell back:

keyhog: daemon route not used (no daemon is listening on <socket>);
running in-process scanner

That notice is on stderr, so a pipeline that redirects stderr and pipes stdout into jq can be silently in-process and silently failing at the same time.

Force in-process execution when you want the scan isolated from daemon state:

kubectl get secret app -o yaml | keyhog scan --stdin --daemon=off

See GPU-backed daemon file queues for eligibility and lifecycle.

A pipeline that cannot report a false clean

set -o pipefail
rm -f keyhog.json
rc=0
vault kv get -format=json secret/app \
  | keyhog scan --stdin --format json-envelope -o keyhog.json || rc=$?

[ -f keyhog.json ] \
  || { echo "keyhog wrote no report; the scan did not run" >&2; exit 1; }
jq -e '.metadata.source_bytes_scanned > 0' keyhog.json \
  || { echo "keyhog scanned nothing; the producer failed" >&2; exit 1; }

case "$rc" in
  0)  echo "no policy-blocking finding over $(jq '.metadata.source_bytes_scanned' keyhog.json) bytes" ;;
  1)  echo "blocking findings"; exit 1 ;;
  10) echo "live credential"; exit 1 ;;
  13) echo "coverage incomplete"; exit 1 ;;
  *)  echo "keyhog failed: exit $rc"; exit 1 ;;
esac

The byte check runs before the exit-code branch on purpose. Exit 0 only means something when bytes were scanned.

Tell a real clean from a skipped input covers the report fields. Exit codes lists every code.

Watch mode

keyhog watch monitors directories and scans files as they change.

keyhog watch src/ config/

It runs in the foreground, compiles one scanner at startup, scans what is already in the tree, and then prints each finding as it happens:

🔍 stripe-secret-key /home/you/project/src/b.env:1 CRITICAL (1.00)  sk_l...EaNn sha256:<credential-digest>
👁  keyhog watch (☰ 934 detectors compiled)
    workers: 16
    watching: /home/you/project/src
    Ctrl-C to exit

Readiness means the tree is covered

The startup block is printed last, on purpose. By the time you see watching:, the filesystem watches are registered and every file already in the tree has been scanned. Findings from that first pass appear above the block, which is why the example shows one there.

That ordering is the whole guarantee. A file written while the scanner was compiling, or between compilation and watch registration, is still reported, because the startup pass sees it on disk. It is reported once: the same deduplication that collapses an editor’s event burst also collapses a file caught by both the startup pass and an event.

A large tree makes startup slower for the same reason. The watcher is scanning it. If a root holds more than 10000 files, the startup pass stops at that limit and says so, naming the keyhog scan command that covers the rest.

Watch still reports changes, not state. Run a full scan again after the watcher exits for any reason, because every change while it was down is unobserved:

keyhog scan src/ config/ --format json-envelope -o baseline.json

A scan also gives you the machine-readable coverage the watcher cannot.

What triggers a scan

A new file in a watched root is scanned. A new file in a subdirectory created after the watcher started is scanned, because roots are watched recursively.

A whole directory that appears in a watched tree is scanned, including the files it already contained. This covers mv ~/Downloads/config-dump src/, cp -r, tar -xf, and a dependency vendoring step. The kernel reports one event for the directory and none for its contents, so the watcher walks the new subtree itself rather than waiting for events that never come. The walk stops at 10000 files or 64 levels and says so; it does not follow a symlinked directory out of the watched root.

A modified file is rescanned in full. Every finding in that file is printed again, including ones you already saw. One editor save can produce more than one event, so the same finding can appear several times in a row. The output is a stream of events, not a deduplicated report.

An editor that saves by writing a temporary file and renaming it over the original is handled. The rename is a change to the destination path and the destination is rescanned. Some editors leave the temporary file visible long enough to be scanned too, so you may see the same finding reported once against a .swp-style name and once against the real one.

Nested or duplicate roots fold into their covering parent, the same as keyhog scan. Each root must be a directory.

Paths watch skips, and the one that stops it

A symlink is not followed, and neither is a FIFO, socket, or device node. keyhog scan declines the same paths, so this is shared policy rather than a watch limitation. Each one prints a line saying it was not scanned and that keyhog scan skips it too. These do not count toward --max-consecutive-failures, so a tree full of symlinks does not stop the watcher. A file that exists, is in policy, and still cannot be read, such as one you lack permission for, does count.

Deleting a watched root removes its watch. The kernel discards the watch along with the directory, so there is nothing left to observe and no way to tell that apart from a quiet tree. The watcher says so and then waits up to 30 seconds for the directory to come back, because a root is more often replaced than deleted:

WARN keyhog watch: watched root /home/you/project/build was removed; its
     filesystem watch is gone and changes under that path are NOT being
     observed. Waiting up to 30s for it to return.
OK   keyhog watch: /home/you/project/build is being watched again; rescanned
     12 file(s) to cover the gap while it was missing.

The rescan is the point. Nothing was observed between the two messages, so the whole subtree is scanned again on return rather than assumed unchanged. A file written while the root was missing is reported.

If the directory does not come back within that window, keyhog watch exits non-zero naming the root, rather than continuing to report a clean tree that nobody is watching.

What watch cannot do

keyhog watch prints text only. It has no --format and no --output. There is no JSON envelope, so there is no source_bytes_scanned, no coverage_gap_summary, and no machine-readable coverage signal. When you need a report you can check, run keyhog scan.

Suppression that runs after a match applies, so a change to app.min.js produces no output, for the same reason a scan of it produces no findings. See Minified and single-line files.

Exclusion by filename does not. keyhog scan skips lock files before reading them; keyhog watch only skips excluded DIRECTORY names, so it reports findings in files a scan of the same tree leaves out. Measured on one tree with a credential planted in each file, scan reports 4 and watch reports 9; the five extra are Cargo.lock, package-lock.json, pnpm-lock.yaml, yarn.lock, and go.sum. Watch is noisier here, never blinder: nothing that scan reports is missing from watch.

Watch is not the daemon. It compiles its own scanner, does not use the daemon socket, and does not appear in keyhog daemon status. Watch also does not scan Git history.

Limits

watch --max-file-size takes a bare byte count. scan --max-file-size requires a unit. These two are not interchangeable:

keyhog watch ~/projects --max-file-size 104857600
keyhog scan ~/projects --max-file-size 100M

Passing 104857600 to keyhog scan exits 2 with a message about the missing unit. Passing 100M to keyhog watch is not a byte count.

--max-consecutive-failures exists so a wedged scanner cannot keep silently dropping changed files. It counts scanner faults only. A path skipped by shared policy, such as a symlink or a file over --max-file-size, does not count, so ordinary repository layout cannot exhaust the budget. When the watcher does exit for that reason, treat every change since the first failure as unscanned and run a full scan.

Warm routing

The watcher warms its routes at startup and reuses that evidence for later file events, so it does not pay a cold backend start per change. Without valid autoroute calibration the watcher fails startup without scanning changes. Run keyhog calibrate-autoroute once on the host before starting the watcher.

A working loop

keyhog scan ~/projects --format json-envelope -o ~/keyhog-baseline.json
jq '{bytes: .metadata.source_bytes_scanned, status: .scan_status,
     gaps: .coverage_gap_summary}' ~/keyhog-baseline.json

keyhog watch ~/projects \
  --max-file-size 104857600 \
  --max-consecutive-failures 8

Read the baseline coverage before you trust the watcher. If the baseline scan did not reach your files, Tell a real clean from a skipped input explains why, and the watcher will have the same blind spots.

Mass scanning

A mass scan covers an inventory that is too large or too independent for one repository gate. Partition it by ownership and retry boundary, then preserve one report and raw exit code per partition. Use the GitHub Action guide for one checked-out repository and the CI integration guide for provider-specific job setup.

One command, whole account

For a single organization, group, workspace, or bucket, keyhog does the inventory traversal itself. No loop, no clone script:

export KEYHOG_GITHUB_TOKEN="$GH_PAT"
keyhog scan --daemon=off --github-org acme \
  --format json-envelope --output acme.json

KEYHOG_GITLAB_TOKEN="$GL_PAT" \
  keyhog scan --daemon=off --gitlab-group acme \
  --format json-envelope --output gitlab.json

KEYHOG_BITBUCKET_USERNAME="$U" KEYHOG_BITBUCKET_TOKEN="$P" \
  keyhog scan --daemon=off --bitbucket-workspace acme \
  --format json-envelope --output bitbucket.json

keyhog scan --daemon=off \
  --s3-bucket logs-prod --s3-prefix config/ \
  --format json-envelope --output s3.json

Each run traverses repositories or objects until a configured page, object, byte, or source limit binds. The envelope records source identity and names any remaining inventory as a coverage gap. That is the complete setup for one bounded provider target. The rest of this guide covers scanning across many targets (multiple orgs, mixed local and cloud sources, or thousands of repositories) where you partition and aggregate.

Use one bounded report and one exit status per partition when scanning many repositories, buckets, or files. Keep the partition manifest outside the scan tree so a scanner never treats its own answer key as input.

Local partitions

This shell pattern preserves every report and status without turning a partial partition into a clean result:

#!/usr/bin/env bash
set -u

out="${1:-keyhog-results}"
mkdir -p "$out"
overall=0

while IFS= read -r -d '' partition; do
  name="$(basename -- "$partition")"
  report="$out/$name.json"
  set +e
  keyhog scan --daemon=off "$partition" \
    --max-file-size 100MiB \
    --format json-envelope --output "$report"
  rc=$?
  set -e
  printf '%s\t%s\t%s\n' "$partition" "$rc" "$report" \
    >> "$out/status.tsv"
  # Keep each raw status in status.tsv. The wrapper only needs one nonzero
  # terminal status to make the aggregate CI job fail.
  (( rc != 0 )) && overall=1
done < <(find ./partitions -mindepth 1 -maxdepth 1 -type d -print0)

exit "$overall"

The envelope records scan-wide coverage and the resolved policy. Keep status.tsv with the reports; an aggregator must not discard a nonzero status just because another partition was clean. If a partition is retried, replace its report atomically and append a new attempt column or manifest row rather than overwriting the only evidence.

--max-file-size bounds each regular file. The default is 100 MiB. A larger file is skipped and recorded as a coverage gap. --limit-stdin-bytes does not bound a directory partition. It applies only to --stdin.

For CI, upload the whole output directory as an artifact and make the job fail on any status that the policy treats as actionable. Exit 13 means the scan completed with coverage gaps, not that it found nothing; inspect the envelope before deciding whether a retry is safe. Exit 2 or 3 is an input or system failure and needs operator attention. See exit codes.

Hosted Git and cloud inventories

The source flags keep inventory traversal inside KeyHog so source identity and coverage remain in the report:

keyhog scan --daemon=off --github-org "$ORG" \
  --limit-hosted-git-pages 100 \
  --format json-envelope --output github.json

keyhog scan --daemon=off --gitlab-group "$GROUP" \
  --limit-hosted-git-pages 100 \
  --format json-envelope --output gitlab.json

keyhog scan --daemon=off \
  --s3-bucket "$BUCKET" --s3-prefix "$PREFIX" \
  --limit-cloud-max-objects 10000 --limit-s3-object-bytes 100MiB \
  --format json-envelope --output s3.json

keyhog scan --daemon=off \
  --gcs-bucket "$BUCKET" --gcs-prefix "$PREFIX" \
  --limit-cloud-max-objects 10000 --limit-gcs-object-bytes 100MiB \
  --format json-envelope --output gcs.json

Use the credential environment variables documented by keyhog scan --help and environment variables; do not put provider tokens in the command line. Azure Blob uses --azure-container-url and its matching prefix/object limits. A page or object cap is deliberate bounded coverage: the report names the limit and exits 13 when more inventory remains.

Hosted APIs and cloud listings can return transient transport or rate-limit errors. Retry only the failed source with bounded exponential backoff, keep the original partial envelope, and preserve the provider request diagnostics. Do not increase object/page caps automatically, and do not classify a rate-limit failure as a clean scan. Respect each provider’s pagination and retry headers.

Daemon and corpus semantics at scale

Use --daemon=off when a scan needs baseline state, live verification, lockdown, a preset, a detector overlay, a custom allowlist, or another per-scan engine policy. The mass route accepts spec-bound incremental state for daemon-local filesystem roots; the other contracts remain in process.

Use the explicit mass service for standard-policy directory trees, Git history, hosted inventories, cloud buckets, archives, binaries, and remote endpoints:

keyhog calibrate-autoroute --policy default
keyhog daemon start --mass --socket /run/user/$UID/keyhog-mass.sock

keyhog scan --daemon=mass \
  --daemon-socket /run/user/$UID/keyhog-mass.sock \
  ./partitions/team-a \
  --format json-envelope --output team-a.json

For warm unchanged-tree scans, add --incremental --incremental-cache /absolute/path/merkle.idx. The daemon loads and publishes that spec-bound generation without rebuilding its scanner. Files with findings are forgotten before publication and remain visible on every scan. Trusted metadata and content-confirmed Merkle skips remain complete coverage when no source bytes need scanner dispatch.

--daemon=mass is required routing. A missing, warm-only, stale, or incompatible service is an error. KeyHog does not fall back to an in-process scan. Policy incompatibility is checked before source acquisition.

GPU-backed daemon worker

For a local filesystem root, the client sends only canonical path and source-policy metadata. The daemon reads and batches the files without copying payload bytes through IPC. Sources that require client-side credentials, such as hosted Git and cloud inventories, use protected wire frames. Both paths keep each batch at no more than 8 MiB of raw payload and 1,024 chunks. The daemon processes each batch with its persisted autoroute decision while retaining one exclusive fragment-state lease for the transaction. It clears fragment state when the transaction ends or the client disconnects. This bounds batch memory independently of total inventory size. Response JSON is written directly into its bounded transport frame, so the daemon does not retain a second complete serialized response body. Daemon-local acquisition uses one drain request. The daemon streams one bounded result response per batch followed by the terminal completion response. Socket backpressure remains the memory bound; the client does not pause each batch to send another request.

The completion receipt contains exact total and GPU batches, chunks, bytes, and daemon execution time. Protected wire mode compares total chunks and bytes with the sent stream. Daemon-local path mode uses the daemon receipt as source-byte authority. Stderr reports the transport, GPU byte share, whether GPU processed more than half of all bytes, and throughput. Invalid receipt invariants fail. Source acquisition gaps remain visible in the envelope and exit 13.

Check keyhog daemon status --socket /run/user/$UID/keyhog-mass.sock before admitting jobs. The daemon serializes fragment-sensitive engine work, so additional concurrent clients do not create extra GPU lanes. Scale across separately budgeted worker hosts, not unbounded clients on one socket.

Routine workers use persisted autoroute evidence. Add --mass-gpu-primary when the worker must prove that GPU processed more than half of all non-empty payload bytes. The client rejects a CPU-majority receipt before producing the final report. A forced --backend gpu-cuda-region-presence, gpu-metal-region-presence, or gpu-wgpu-region-presence service is a diagnostic GPU-only contract. It exits 12 when required GPU startup fails and returns a request error instead of substituting CPU after a runtime fault. It does not prove that GPU is the fastest route for the workload.

The daemon and client must use the same replacement detector corpus. Overlay composition is unsupported. Start the service with the reviewed replacement corpus, then select that same corpus on the client:

keyhog daemon start --mass --detectors ./reviewed-detectors
keyhog scan --daemon=mass \
  --detectors ./reviewed-detectors --detectors-mode=replace \
  ./partitions/team-a

A replacement identity mismatch is a handshake error. See daemon and warm scans for lifecycle, socket, eligibility, and retry behavior.

For a large inventory, partition at the provider or repository boundary. Calibrate autoroute on the actual worker class and retain the per-partition resolved policy, coverage envelope, and execution receipt. Missing or stale autoroute evidence leaves the affected batch unscanned and records incomplete coverage. It does not silently claim calibrated CPU, Hyperscan, or GPU execution.

Concurrency and worker sizing

Each KeyHog process uses the available CPU cores by default. This is the right default for one dedicated partition. It can oversubscribe a worker when your CI or scheduler starts several partitions on the same host.

Allocate the host CPU budget across concurrent processes. For example, four partition jobs on a 16-vCPU worker can each start with --threads 4. This is a resource allocation example, not a universal optimum. Leave --reader-threads unset until profiling shows that storage readers are the bottleneck; its default derives from the scanner worker pool.

Keep these boundaries when increasing concurrency:

• Give every partition its own json-envelope report, raw exit code, incremental cache, and retry identity.
• Keep one incremental cache bound to one trusted repository or partition. Sharing it across unrelated jobs turns reuse into cross-workspace state.
• Bound provider jobs by API quotas and pagination limits as well as CPU. Live verification has separate --verify-concurrency, --verify-rate, and --verify-batch controls.
• Use --daemon=mass for standard-policy directory, history, archive, remote, or cloud streams. Use --daemon=off when the partition needs policy state that the mass service rejects.
• Aggregate only after every concurrent partition has reached a terminal state. One successful job cannot erase another job’s coverage gap or error.

Use --profile on representative partitions before changing advanced --reader-threads, --fused-batch, or --fused-depth values. Keep those controls unset when a repeatable target-host measurement does not show a benefit.

Report aggregation

Aggregate only after every partition has a terminal envelope. One clean partition never cancels another partition’s coverage gap or error.

Read every report at once:

cat keyhog-results/*.json | jq -s '{
  partitions: length,
  findings: (map(.findings | length) | add),
  coverage_gaps: (map(.coverage_gap_summary | length) | add),
  incomplete: [ .[] | select(.scan_status != "success") | .metadata.targets[0] ]
}'

{
  "partitions": 2,
  "findings": 1,
  "coverage_gaps": 0,
  "incomplete": []
}

Act on incomplete first. A partition listed there did not cover its input, so its finding count proves nothing. Fix the cause, rerun that partition alone, and replace only its report. The findings total is trustworthy once incomplete is empty.

The envelope carries coverage state, not the process exit code, so keep status.tsv beside the reports. The wrapper in Local partitions already exits nonzero when any partition did; status.tsv is what tells a reviewer which partition produced which raw code, because one aggregate exit code cannot express two different failures.

Preserve the partition identity, source inventory, resolved policy, coverage state, finding count, and exit code. JSON and JSONL legacy formats contain findings only; json-envelope and jsonl-envelope are the recommended machine contracts for mass scans because they carry terminal coverage and identity. Never concatenate JSON arrays or merge findings before deduplicating with the partition and location identity.

Daemon and warm scans

The Unix daemon keeps one compiled scanner and its backend state warm. The default service handles repeated standard-policy scans of stdin or one regular file. Starting it with --mass also accepts bounded streams acquired from directories, repositories, archives, binaries, remote endpoints, and cloud inventories. Watch and system-wide scans remain in process.

For the repository map and bytes-to-finding pipeline, see Architecture. For backend selection and autoroute, see Backends and routing and Autoroute calibration.

Starting the daemon is an explicit operational step. KeyHog never starts one for you. Run the server in one terminal or under a service manager:

# Terminal 1. This stays in the foreground until stop or a fatal service error.
keyhog daemon start

Wait for the ready line before sending required requests. Use a second terminal for scans and administration:

# Terminal 2. Omitted --daemon means --daemon=auto on Unix.
keyhog scan --stdin < changed-file.txt
keyhog scan path/to/one-file.txt

# Require daemon execution. This fails rather than changing execution mode.
keyhog scan --daemon=on path/to/one-file.txt

# Guarantee in-process execution even while the daemon is ready.
keyhog scan --daemon=off path/to/one-file.txt

keyhog daemon status
keyhog daemon stop

Background guard daemon lifecycle

When using Perpetual Guard (keyhog guard), you can manage the daemon in the background directly with keyhog guard up and keyhog guard down:

# Start the background daemon if not running and reconcile durable roots
keyhog guard up

# Stop the daemon cleanly while preserving durable registrations and indexes
keyhog guard down

Start an opt-in mass service when one worker should process a large source stream:

# Terminal 1.
keyhog daemon start --mass

# Terminal 2. This required route never retries in process.
keyhog scan --daemon=mass /srv/inventory/team-a \
  --format json-envelope --output team-a.json

The client sends at most 8 MiB and 1,024 chunks per batch. The terminal receipt reports exact total and GPU batches, chunks, bytes, GPU share, and throughput. The client rejects a receipt that does not match the bytes it sent.

keyhog watch is separate. It is a foreground filesystem watcher with its own compiled scanner. It does not use the daemon socket and does not appear in daemon status.

The watch and scan-system subcommands do not accept --daemon. Each compiles and owns an in-process scanner. A running daemon neither serves nor accelerates those commands.

Lifecycle and readiness

daemon start prints a compilation message first. The service can accept requests only after this line appears:

keyhog daemon ready on <socket> (<count> detectors, wire=<version>)

The ready line follows detector loading, scanner compilation, backend validation and warmup, socket binding, and socket permission checks. Startup fails instead of announcing readiness when any required step fails. With a valid decision table, an autorouted daemon warms only peers selected by at least one persisted warm-daemon route. An acquired but unused peer cannot block readiness, while every selected peer must initialize and warm successfully. Missing, stale, or invalid autoroute state prevents daemon readiness. A forced --backend gpu-cuda|gpu-metal|gpu-wgpu|simd|cpu is a diagnostic startup choice and must be usable as requested. See Autoroute calibration.

GPU startup failures retain their stage and exit 12. This covers required GPU preflight, scanner compilation, an unavailable or incompatible backend, and degradation during the readiness warmup. The diagnostic tells the operator to run keyhog backend --self-test, repair the driver/runtime, or start with --backend simd or --backend cpu. An invalid backend value or unrelated daemon configuration error remains exit 2.

After readiness, an automatically routed accelerated-backend fault does not kill the service or drop the request. The daemon warns and replays that request’s stable text or file input through the fastest remaining measured-correct peer. GPU recovery replays only exact unprocessed ranges. The daemon records recovered ranges and bytes, quarantines that workload route, and keeps unrelated requests alive. Later requests for the quarantined workload fail closed: no backend is selected, the affected batch remains unscanned, and the response names the required recalibration. Runtime route health is persisted separately from timing evidence, so restarting the daemon cannot erase quarantine; successful recalibration clears only the repaired workload identity. A forced GPU daemon remains an explicit contract and returns a request error instead of substituting another backend.

daemon status connects to an existing service. It reports uptime, completed scan attempts, active scans, detector count, backend policy, and identity staleness. scans served includes attempts that returned a daemon error, so it is an activity counter rather than a success counter. Status never starts a daemon. It also prints whether the service accepts warm stdin and single-file requests or mass source batches. Warm stdin and single-file requests return before baseline, verification, lockdown, and per-request scanner-policy post-steps. The mass route additionally accepts Merkle state for daemon-local filesystem roots. active scans counts accepted scan attempts until their blocking task finishes, including attempts queued behind the scanner’s fragment-state lock. Backend health reports the number of recovered authenticated-route requests and the last failed and recovery backend with recovered byte count. After a restart with persisted runtime quarantine, status prints backend policy: autoroute degraded; healthy workload routes remain usable and affected routes fail closed without scanning until recalibration clears them. The daemon can frame multiple client connections concurrently, but production scanner execution is serialized so fragment state cannot cross requests.

daemon stop sends a shutdown request and succeeds after receiving the acknowledgement. The server then stops accepting connections and removes the socket. The current implementation does not wait for other active scan handlers to finish. Check that daemon status reports 0 active before stopping when in-flight requests must complete. An abrupt process exit can leave a socket entry. The next start removes it only after the stale-socket trust checks pass.

Use the service manager as the process owner when it started the daemon. Before restarting, run daemon status against the same socket and wait for the status line to show 0 active when requests must finish. Then use daemon stop, or stop the service-manager unit. Start the replacement process and wait for its ready line. Do not delete a live socket to restart the service. Removing the path does not stop the listener and can leave an unreachable daemon running.

What the daemon actually buys

The warm single-file asset is the compiled scanner and its backend state. There is no per-connection or per-path result cache on that route, so a repeated single-file scan costs the same as the first request. Mass daemon filesystem scans can also persist a spec-bound Merkle generation. An unchanged clean file then bypasses its read and scanner dispatch. A file that produced a finding is excluded from the generation and remains visible on every transaction.

Measured on one 16-core host, medians of five runs, machine load average 51-56 on 32 logical cores, so read the ratios rather than the absolute seconds:

The saving is startup, not scanning. Measured on the socket with no client process in the way, the daemon spends 0.8 ms on the 13 KB file, 30 ms on the 2.9 MB one and 103 ms on the 29.8 MB one. Everything above that is process start plus scanner compilation, which the daemon pays once at daemon start instead of once per scan. That is why the speedup shrinks as the file grows: the fixed cost you avoid stays the same while the work you still do goes up.

The memory difference is the larger effect and it is what makes the daemon worth running for many small scans. A daemon-served client holds about 39 MB. An in-process client holds about 550 MB, because it builds its own scanner. Eight concurrent scans of a 13 KB file finished in 0.22 s through the daemon against 1.80 s in process, and peaked at about 0.3 GB across all eight clients against about 4.3 GB.

A daemon that will only ever serve small files still pays the full engine, so daemon start is worth it when scans are frequent, and not worth it for one scan.

Footprint over a long session

There is no idle shutdown and no TTL. A daemon runs until daemon stop, a fatal service error, or the service manager stops it. It holds its scanner the whole time, so an idle daemon is not free.

Measured with the embedded corpus on the same host: about 538 MB resident at the ready line, unchanged after 15 seconds idle. After 1,000 warm scans of a 2.9 MB file it reached about 606 MB, with the second five hundred of those scans adding 1.2 MB. Descriptors stayed at 10 and threads at 35. The growth is allocator and scratch warm-up that plateaus, not accumulation per request.

Concurrency and queueing

The daemon frames many connections at once but runs one scan at a time, so fragment-reassembly state cannot cross requests. Plan for that: a large scan delays every small scan behind it for its whole duration.

Measured on the socket, same host: eight concurrent 29.8 MB scans finished in 0.74 s against 0.70 s for the same eight run one after another, and one 13 KB scan that arrived behind a 29.8 MB scan took 60 ms instead of its solo 0.8 ms. Under heavier load the ratio holds and the penalty grows: at load average 178 the same queued 13 KB scan took 403 ms against a 2.8 ms solo.

A queued client sees nothing until its result arrives. The protocol has no queued notice, no position, and no estimate, so a client waiting on a busy daemon is indistinguishable from a hung one. Use daemon status from another terminal to tell them apart: it reports active scans, and the control plane answers while the data plane is busy.

Socket selection and trust

The server, scan client, status command, and stop command use the same default socket resolver:

1. $XDG_RUNTIME_DIR/keyhog.sock when XDG_RUNTIME_DIR is set.
2. The OS user-cache directory plus keyhog/server.sock.
3. The OS temporary directory plus keyhog/server.sock.

The usual cache paths are ~/.cache/keyhog/server.sock on Linux and ~/Library/Caches/keyhog/server.sock on macOS. There is no KeyHog socket environment variable. For a fixed location, pass the same path at both ends:

keyhog daemon start --socket /private/path/keyhog.sock
keyhog scan --daemon=on --daemon-socket /private/path/keyhog.sock one-file.txt
keyhog daemon status --socket /private/path/keyhog.sock
keyhog daemon stop --socket /private/path/keyhog.sock

The socket carries unredacted matches between same-user processes. The server requires an owned, non-symlinked socket path, tightens a created parent to mode 0700, and requires the socket itself to be mode 0600. Both client and server verify the connected peer UID. A stale entry is removed only when it is an owned 0600 Unix socket in a trusted directory and no listener accepts a connection. KeyHog refuses ordinary files, symlinks, foreign owners, loose permissions, and a live socket rather than replacing them. An untrusted stale entry is not removed automatically. Correct or remove it after verifying the path and owner, then start the daemon again.

Windows ships no daemon transport. An absent daemon flag or --daemon=off runs in process. Explicit --daemon=auto, --daemon=on, and all daemon subcommands fail with the Unix-only error.

Routing contract

On Unix, omitting --daemon is equivalent to --daemon=auto. Bare --daemon is equivalent to --daemon=on.

--daemon=on, bare --daemon, and --daemon=mass require the daemon route. An unavailable service is an error. A daemon that cannot honor the source or policy is also an error. No in-process retry occurs. --daemon=auto is the opportunistic warm route. It can use a reachable daemon only when it can honor the request. It otherwise keeps compatible one-file or bounded-stdin requests in process, and retries them in process after a daemon execution failure.

--daemon-socket cannot be combined with --daemon=off.

The socket state and daemon state are separate signals. Use this matrix when diagnosing an automatic route:

Some compatibility failures can be known only after connecting. This includes a detector identity or wire mismatch. Under auto, KeyHog reports that failure and retries the same eligible input in process. Under on, it returns the failure without rescanning. A request that is known to be unsupported from its source or policy does not connect in either mode.

The automatic retry boundary is a fully decoded and validated ScanResults response. Failures before that boundary, including incompatible required wire fields, retry in process under auto. Allowlist loading, finalization, output creation, serialization, and report writes occur after that boundary. Those client-side failures return directly and never rescan. This prevents duplicate or mixed output after a partial write.

stdin is single-consumer, so the client acquires it into one bounded replay buffer before sending ScanText. If an automatic daemon request fails before the validated result boundary, the in-process retry scans that same buffer as the stdin source. It does not read the pipe again, and it preserves the configured byte limit, source metadata, and lossy UTF-8 decoding. A successful daemon response releases the buffer with the rest of the request.

An automatic in-process retry uses the normal one-shot autoroute contract. It does not pin CPU to make the retry succeed. Missing or stale one-shot evidence therefore remains a visible calibration error.

GPU-backed mass worker

Calibrate the worker host, then start an opt-in mass service under a service manager:

keyhog calibrate-autoroute --policy default
keyhog daemon start --mass --socket /run/user/$UID/keyhog-mass.sock

The persisted decision table may select CPU, Hyperscan, CUDA, Metal, or WGPU for each batch. Confirm the ready identity before admitting jobs:

keyhog daemon status --socket /run/user/$UID/keyhog-mass.sock
keyhog scan --daemon=mass \
  --daemon-socket /run/user/$UID/keyhog-mass.sock \
  /srv/inventory/team-a --format json-envelope --output team-a.json

Each batch contains no more than 8 MiB of raw payload and 1,024 chunks. The daemon holds an exclusive fragment-state lease across the transaction and clears it on completion or disconnect. Additional concurrent clients do not create extra GPU lanes. Partition across separately budgeted hosts when one worker is saturated. Response JSON is written directly into the bounded transport frame. The 64 MiB frame ceiling is enforced during serialization, and a failed response rolls back without leaving partial bytes for the next frame.

The completion receipt records exact total and GPU batches, chunks, bytes, and daemon execution time. The client verifies total chunks and bytes against its sent stream. Stderr reports GPU byte share, whether GPU handled more than half of all bytes, and throughput. Acquisition gaps remain visible in the report and use exit 13.

Add --mass-gpu-primary when each completed transaction must prove that GPU processed more than half of all non-empty payload bytes. The client rejects a CPU-majority receipt before reporting. You may force --backend gpu-cuda-region-presence, gpu-metal-region-presence, or gpu-wgpu-region-presence at daemon startup for diagnostics. A forced GPU service exits 12 when required startup fails and returns a request error instead of substituting CPU after a runtime fault. Routine workers use persisted autoroute evidence. A forced backend is not proof that the route is fastest for the exact workload.

Request eligibility

The warm route accepts exactly one primary input:

• --stdin, subject to the configured stdin byte limit
• one path whose metadata identifies it as a regular file

The mass route accepts the source classes supported by keyhog scan, including directories, Git modes, archives, binaries, remote endpoints, hosted Git, and cloud inventories. Local filesystem roots send only canonical path and policy metadata; payload bytes remain in the daemon process. Sources that require client-side credentials stream protected chunks to the daemon. Eligible requests may still use client-owned reporting and finalization such as output formats, output files, deduplication, bundled test-fixture suppression, local default allowlists, inline suppression, and --dogfood. Dogfood detail is request-scoped and bounded; exact aggregate counters are carried separately.

The daemon can use an explicit replacement corpus. Start it and scan with the same reviewed directory:

keyhog daemon start --detectors ./reviewed-detectors
keyhog scan --daemon=on \
  --detectors ./reviewed-detectors \
  --detectors-mode=replace \
  one-file.txt

The daemon startup flag always selects a complete replacement corpus. It does not compose that directory over the embedded corpus. The scan-side --detectors-mode=replace spelling is optional, but makes the contract visible. The client derives the expected rules identity from its selected directory. The warm route is accepted only when the daemon compiled the exact same rules. The report records the replacement count, digest, source, and mode.

Start the daemon from a directory you chose on purpose. --detectors defaults to the relative path detectors, and when that path does not exist KeyHog searches the installed locations and then the directory holding the keyhog executable. A daemon started inside a checkout that happens to contain detectors/ therefore compiles that corpus, while a client that passes no --detectors uses the embedded one. Their rules digests differ, so every default client is refused with a detector rules identity mismatch until the daemon is restarted somewhere else. The ready line names the corpus the daemon actually compiled, so read the count before you send work:

keyhog daemon ready on <socket> (<count> detectors, wire=12, warm generation=...)

--detectors-mode=overlay is not daemon-compatible. The daemon cannot rebuild its warm scanner for a per-request overlay. With --daemon=auto, an overlay scan stays in process without opening the socket. With --daemon=on, it fails before scanning. Use --daemon=off to make the supported execution mode explicit:

keyhog scan --daemon=off \
  --detectors ./site-detectors \
  --detectors-mode=overlay \
  source-tree/

If the client and daemon select different replacement corpora, the handshake fails. auto prints the identity error and retries in process. on prints the identity error and exits. It never substitutes the daemon’s corpus.

The warm route requires the in-process orchestrator for directories, multiple roots, Git modes, remote, cloud, container, binary, dynamic, or mixed sources. The mass route accepts those source classes, but it requires the daemon-owned standard scanner policy. Both routes reject these per-scan contracts:

• baseline filtering or live verification
• Merkle/incremental source state on the warm route; the mass route accepts it only for daemon-local filesystem roots
• --fast, --deep, --precision, benchmark mode, or changes to decode, entropy, ML, Unicode normalization, comment scanning, scanner limits, detector vocabulary, or detector overlay composition
• a replacement corpus whose rules identity does not exactly match the daemon
• per-request backend, GPU, batch-pipeline, autoroute, cache, or calibration controls
• path-exclusion changes
• lockdown, secret display, client-safe hiding, confidence or severity floors, custom AWS canaries, detector disable or confidence policy, allowlist governance, or a malformed effective configuration

--daemon=auto keeps a source or policy rejected by the warm route in process. A replacement identity mismatch is learned during the handshake, then auto reports it and retries in process. --daemon=on and --daemon=mass fail without an in-process scan. The mass route checks incompatible policy before source acquisition. Daemon availability cannot weaken the selected policy.

These examples show the routing boundary:

# Standard-policy directory stream through an opt-in mass service.
keyhog scan --daemon=mass source-tree/

# The same directory with baseline state stays in process.
keyhog scan --daemon=off --baseline .keyhog-baseline.json source-tree/

# Overlay composition remains in process.
keyhog scan --daemon=off \
  --detectors ./site-detectors --detectors-mode=overlay one-file.txt

Identity, wire data, and coverage

Every connection begins with a versioned handshake. Scan clients require all of these values to match the current client:

• wire version
• KeyHog package version
• Git build hash
• canonical detector-rules digest

For the default corpus, the detector digest is compared with the client’s embedded rules. For an explicit replacement corpus, the client derives the expected digest from its own selected directory and accepts only an exact match. Overlay composition is not daemon-eligible. The handshake also carries the daemon-owned backend policy. It must be autoroute or a canonical forced backend label. A scan client rejects an unknown label. daemon status and daemon stop tolerate package, build, and detector identity staleness so you can inspect and terminate an old service. They still require a compatible wire protocol. Stale status prints the exact mismatch and exits successfully because the health request succeeded.

Current scan results require matches, example-suppression count, dogfood detail, exact static-recovery rejection aggregates, dropped-detail count, and source coverage gaps. Missing fields are malformed protocol data. The client never invents zero values for absent coverage or telemetry.

Coverage gaps include oversized or binary input, unreadable data or Git objects, archive truncation, unresolved binary section names, source truncation, structured-source parse failures, unavailable archive duplicate scans, and Git LFS pointers. The client prints a warning whenever any count is nonzero. Current exit behavior is:

A scan with both findings and coverage gaps exits 1 and prints the incomplete coverage warning. It is not reported as clean.

Administrative and routing errors

Daemon availability, eligibility, trust, handshake, and ordinary operator-correctable path errors normally exit 2. This includes forced --daemon=on without a usable service, status or stop without a service, an incompatible forced request, and invalid startup configuration. Low-level operating-system I/O failures outside the operator-input classes exit 3. Daemon GPU validation, initialization, and warmup failures exit 12. A forced GPU dispatch failure after readiness returns a request error. An autorouted dispatch fault completes against the same stable request through the visible recovery contract when full coverage is possible. If an auto request fails inside the daemon, KeyHog reports the error and retries in process; the retry then owns its normal exit semantics, including automatic backend recovery or 12 when GPU was explicitly required.

A fatal listener accept or connection-handler spawn error prints a failure, stops the service, removes the daemon socket, and makes daemon start exit 3. The typed service failure remains distinct from requested daemon stop, which cleans up the same socket and leaves daemon start at exit 0.

daemon status against an identity-stale but wire-compatible service exits 0 and prints a warning. daemon stop can stop that service. A wire-incompatible service cannot be inspected or stopped through the current protocol. Stop it with the matching KeyHog binary or the service manager that owns it.

daemon start --request-timeout-secs <N> limits how long a connected client may take to deliver one complete request frame. The default is 300 seconds. On timeout, the daemon closes that connection and reclaims its concurrency slot. This is a request-read deadline, not a scan execution deadline.

After a request is sent, the client applies a response deadline by request kind. Hello, Health, and Shutdown use 5 seconds. ScanText uses 60 seconds. ScanPath uses 300 seconds. A timeout fails with restart guidance. Automatic scan routing may then use its documented in-process recovery path, while --daemon=on, daemon status, and daemon stop return an error.

Reading the in-process notice

When you pass --daemon explicitly and the scan runs in process anyway, KeyHog prints the reason on stderr:

keyhog: daemon route not used (the daemon only supports exactly one source: ...); running in-process scanner
keyhog: daemon route not used (no daemon is listening on /run/user/1000/keyhog.sock); running in-process scanner

Omitting --daemon prints nothing, because the flag defaults to auto and most scans can never use the warm route. --daemon=off prints nothing either, since it already asked for the in-process path. Use --daemon=auto explicitly whenever you are comparing timings: without it you cannot tell a daemon-served scan from an in-process one, and a daemon you started but never reached looks exactly like one that is working.

GitHub collaboration scans

Repository clones do not contain every place a credential can be pasted. Select collaboration surfaces explicitly when they are part of the review boundary:

export KEYHOG_GITHUB_TOKEN='read-only-token'

keyhog scan \
  --github-collaboration acme/payments \
  --github-all

No collaboration surface is implied by --github-org. A repository-only scan makes no issue, pull request, discussion, wiki, or gist request.

Surface contract

Pass --github-all for the complete set, or select individual flags for the surfaces the token is allowed to read. Use KEYHOG_GITHUB_TOKEN instead of --github-token so the credential does not enter the process argument list.

Limit a fine-grained token to the target repository and grant read-only access for the selected Issues, Pull requests, Discussions, and Contents resources. A classic token may need repo for private repository surfaces. Public access still depends on the repository and organization policy. Gist scanning does not claim private or secret gists because repository ownership does not identify the authenticated token owner.

Bounds and coverage

All API calls share one request budget. --limit-hosted-git-pages controls that budget for a collaboration source, including item pages, comment pages, gist revisions, and GraphQL pages. API responses use --limit-web-response-bytes. Collaboration chunks also honor the Git aggregate byte and chunk limits.

An inaccessible selected surface produces a typed inaccessible source coverage error. A request, response, byte, or chunk cap produces a typed truncated source coverage error. These errors make coverage incomplete. KeyHog does not report the selected surface as clean.

GitHub response bodies and credentials are never included in diagnostics. Redirects are disabled while an authorization header is present.

Provenance and edits

Findings use credential-free github:// paths. Issue, pull request, discussion, and comment revisions combine the immutable GitHub node identity with its updated_at value. Wiki and gist revisions use Git object IDs. Repeated objects with the same immutable revision identity are scanned once. Edited content gets a new reproducible revision identity.

System-wide credential triage

keyhog scan-system performs a one-shot audit across the mounted filesystems that KeyHog can enumerate and read. It scans working-tree files first. It also discovers Git repositories and scans their history unless you disable that step.

Start with a bounded local sweep:

sudo keyhog scan-system --space 10G 2>scan-system.log

Keep the exit status and stderr log. Stderr contains the selected mounts, coverage warnings, byte count, and terminal summary. Add --output findings.json when you also need a redacted JSON findings array.

Run with enough privilege to read the trees in scope. An unreadable path is a coverage gap. KeyHog reports the gap and does not describe the run as complete. Use elevated privilege only on a host and scope you are authorized to audit.

Choose the host boundary explicitly:

The quick command is not a substitute for the full recovery command. Keep its report labeled as a working-file-only system check.

Execution mode

scan-system always builds and runs its scanner in process. It does not accept --daemon, and a running daemon does not change its behavior. This is distinct from keyhog scan --daemon=auto, which can route an eligible single file or stdin request through a warm daemon.

Use watch after the one-shot sweep when you need to inspect future file changes:

keyhog watch ~/projects \
  --max-file-size 104857600 \
  --max-consecutive-failures 8

watch is also an in-process foreground command. It compiles one scanner and scans changed regular files. It does not perform the initial directory sweep or Git-history scan for you. Run keyhog scan --daemon=off ~/projects for a full rescan after a watch failure. The watcher exits after eight consecutive scan engine failures by default so a broken scanner does not keep dropping changes.

What the system sweep walks

The command discovers targets instead of accepting a path. It enumerates mounted filesystems and skips pseudo-filesystems such as /proc, /sys, tmpfs, and nsfs. Network mounts such as NFS, SMB, and sshfs are skipped by default. Add --include-network only when that remote scope is authorized:

sudo keyhog scan-system --space 1T --include-network

During discovery, KeyHog finds ordinary worktrees, bare repositories, and submodules. After the filesystem walk, it scans their Git histories. A credential committed and later deleted can therefore surface. Use --no-git-history when only current files are in scope:

sudo keyhog scan-system --space 10G --no-git-history

A binary built without the git feature cannot scan discovered history. KeyHog reports each skipped history as a coverage gap. Reinstall a build with Git support, or use --no-git-history to state the reduced scope.

Space and findings limits

--space <SIZE> is a hard aggregate byte ceiling. The default is 50G, which means 50 GiB. Sizes require a unit and accept B, K, M, G, or T and their KB or KiB forms. Fractional values such as 1.5G are accepted.

KeyHog stops before a chunk that would cross the ceiling. Filesystem data is scanned before Git history, so a small ceiling can leave later mounts and histories untouched. Reaching the ceiling records a coverage gap. Raise the limit and rerun when complete host coverage is required.

The command retains at most 1,000,000 redacted findings in memory and continues counting additional findings. A warning says when detail was dropped. Preserve the stderr summary with the JSON output.

The terminal states are:

Exit 13 is not a clean result. Correct permissions, raise --space, restore Git support, or isolate the affected path with a normal in-process scan.

Detector corpus

Both system scans and watchers accept an explicit detector directory:

sudo keyhog scan-system --space 10G \
  --detectors /opt/keyhog/reviewed-detectors

keyhog watch ~/projects \
  --detectors /opt/keyhog/reviewed-detectors

An explicitly selected directory is a replacement corpus by default. It does not silently fall back to embedded detectors when the path is missing or invalid. These subcommands do not expose a --detectors-mode flag. They use the shared scan configuration, so a valid detectors_mode = "overlay" in the resolved .keyhog.toml explicitly requests composition instead.

This corpus is compiled by the command itself. It need not match a running daemon because neither scan-system nor watch uses the daemon socket.

Ignore and lockdown policy

Unlike a normal filesystem scan, scan-system ignores .gitignore and .keyhogignore by default. This prevents a local ignore rule from hiding the files being triaged. Use --respect-gitignore only when that narrower scope is intentional.

Every system sweep disables core dumps and ptrace when the host permits it. --lockdown also requires the stronger protections to succeed and refuses network mounts. It cannot be combined with --include-network.

For recovery, address every warning before treating the host as covered. Fix permissions for unreadable paths, raise the byte ceiling, repair corrupt Git objects, or rerun a deliberately reduced scope. See the CLI reference for the full flag list and exit codes for shared failures.

Source archives

KeyHog recognizes archive and compressed formats while scanning a file or directory. You do not need a separate archive flag.

keyhog scan incoming/ --format json-envelope -o keyhog-archives.json

ZIP-family containers include ZIP, JAR, WAR, APK, IPA, CRX, Python and NuGet packages, and common office formats. KeyHog also reads tar, 7z, RAR, and tar or single streams compressed with gzip, zstd, LZ4, Snappy, bzip2, or xz. Supported archives nested inside those formats are scanned within bounded depth and byte budgets.

Each eligible readable regular member enters the normal detector pipeline. Binary members use printable-string extraction. Finding paths retain every container and member, such as submission.tar//paper/main.tex.

How a member is admitted

A member’s extension is consulted first, so a named member always resolves to the format it has always resolved to. A member whose name carries no recognized extension is admitted on its own leading bytes: gzip, zstd, xz, LZ4, Snappy, bzip2, ZIP, and tar all have a signature at a fixed offset. An extensionless gzip layer inside a tarball, a .zip renamed .dat, and a .dat that is really a tar are therefore all descended into.

The probe reads a bounded prefix at offset zero. KeyHog never speculatively runs an extractor to find out what a member is.

Some families have no signature and are not covered: cab, iso, xar, lzip, .Z, and raw deflate or raw LZMA streams, which have no magic in principle. Those fall through to printable-string extraction with no container coverage gap. A 7z, RAR, or ar/deb member is different: it produces an explicit error row naming the family, because those formats need a seekable file and have no in-memory extractor.

Streaming nested archives

KeyHog streams compressed tarballs (.tar.gz, .tgz, and nested compressed tar members) into the member scanner. It does not keep a full decompressed tarball resident while walking entries. Peak resident memory for archive extraction is bounded by the compressed input, decoder window state, and the largest single member under the active size caps, not by the sum of every decompressed layer.

TeX role annotations need member source bytes up front. Uncompressed .tar and ZIP-family packages still run the buffered provenance pass when header/central directory names include TeX sources. Compressed tarballs stay on the single-pass streaming path so nested layers do not pay a second full inflate; members remain fully scannable, without TeX role annotations on that compressed-tar path.

Archives reached through Git or a bucket

Container expansion applies to files on disk and to the sources that expand them. It does not apply to Git objects or to cloud object bodies. An archive committed to a repository is descended into by keyhog scan <path> and is not descended into by --git-history or --git-blobs, which report it as a binary (extension or content sniff) gap. An object in an S3, GCS, or Azure container is not descended into either.

Download or check out the archive and scan it as a file when its contents are in scope.

Scan untrusted archives

Keep the envelope report when the input may be corrupt or hostile:

rc=0
keyhog scan incoming/ --format json-envelope -o keyhog-archives.json || rc=$?
jq '{scan_status, coverage_gap_summary, findings: (.findings | length)}' \
  keyhog-archives.json
printf 'keyhog exit=%s\n' "$rc"

KeyHog does not extract members into the filesystem or execute their contents. Compressed tarballs are streamed member-by-member so nested layers do not each retain a full decompressed image in resident memory. It rejects archive paths that are absolute, contain traversal, use ambiguous encoding, or name a special file. It also refuses a top-level archive reached through a symbolic link. Decoded bytes per member, nested depth, and aggregate output are bounded. Formats with compression-ratio metadata also apply a ratio guard.

Rejected, encrypted, corrupt, unreadable, oversized, or budget-truncated input is not treated as complete. KeyHog reports the uncovered member or archive on stderr and records a coverage gap in the envelope. With no blocking finding, incomplete coverage exits 13. A blocking finding in the covered portion takes exit 1, or a confirmed live credential takes exit 10. The coverage warning and scan_status = "partial" remain in the report.

Do not raise --max-file-size merely to make exit 13 disappear. Raise it only when you trust the input and the runner has enough memory. Otherwise isolate the failed archive, repair or unpack it with a trusted tool, and scan the recovered files as a new input.

Suppress one reviewed archive fixture

Use the full member path in .keyhogignore.toml. Include the detector and hash so another credential in the same member still reports:

[[suppress]]
detector = "aws-access-key"
path_eq = "vendor-bundle.zip//examples/demo.env"
credential_hash = "5e884898da28047151d0e56f8dc6292773603d0d6aabbdd62a11ef721d1542d8"

The rule does not match the same member in a differently named container. It also does not match a different value in demo.env. Inline directives are not loaded from archive member text. See Suppressions for rule precedence and failure behavior.

TeX source packages

Scan a TeX source package like any other archive:

keyhog scan submission.tar

TeX packages receive an additional bounded dependency pass. It recognizes input, include, subfile, includegraphics, bibliography, and addbibresource commands. Paths are resolved relative to the referring member. Absolute paths and references that escape the archive root do not enter the dependency graph. After an exact member-name lookup, unresolved references try the command’s standard .tex, .bib, or graphics extensions.

Archive chunks use these source labels:

• filesystem/archive/tex-root for document roots
• filesystem/archive/tex-referenced for members reachable from a root
• filesystem/archive/tex-orphaned for inventory members not reachable from a root
• filesystem/archive/tex-comment/<role> for exact unescaped TeX comment spans

Binary members with printable strings use the corresponding filesystem/archive-binary/tex-<role> label.

Comments are scanned with their original member path and byte offset. The full member is also scanned unchanged. Expansion cycles terminate through a visited member set. Malformed commands do not stop ordinary member scanning.

The dependency pass caps member count, per-source bytes, total source bytes, references per member, and group nesting. If the analysis exceeds a cap, role annotations are unavailable and every readable member still takes the ordinary archive path. The scan reports the incomplete semantic expansion as a coverage gap rather than claiming complete TeX analysis.

Android packages

keyhog scan app-release.apk --format json-envelope -o app-release.keyhog.json

APK scans decode resources.arsc value tables and compiled Android XML before running the normal detector pipeline. Resource findings preserve package, type, entry name, resource ID, and configuration qualifier. XML findings preserve the element path, attribute name, framework resource ID, and inline or referenced value.

Decoded chunks use filesystem/archive/android-resource or filesystem/archive/android-xml. KeyHog also scans each original member through the ordinary archive path. It does not execute Dalvik code or resolve runtime resource selection.

Input bytes, chunk count, string pools, resource entries, XML depth, emitted items, and emitted bytes are bounded. A malformed compiled resource records an inaccessible coverage gap. A cap records a truncated coverage gap. Semantic decoding is incomplete in either case, while the original member scan continues.

HTTP and wire scanning

Real credentials don’t always sit on disk. They flow through:

• Live web bundles that ship from production at a public URL.
• HAR files that browsers (Chrome / Firefox / Safari DevTools) produce when you click “Save all as HAR with content.”
• mitmproxy / Burp captures of an authenticated session.
• curl / httpie / Postman exports of one specific request you want to verify.

KeyHog scans every one of these, but the surface is split across a few flags and sources. This page is the map.

TL;DR

The --url flag (Web Source)

keyhog scan --url https://app.example.com/static/main.js
keyhog scan --url https://app.example.com/static/main.js \
            https://app.example.com/static/runtime.js \
            https://app.example.com/static/vendor.js

Each URL is fetched with the shared HTTP client policy (see Proxy and TLS below). The response is routed by extension:

• .js → one chunk per file, scanned as plain text.
• .map → JSON parsed, each sourcesContent[i] becomes its own chunk tagged with the original filename. This is how a Webpack build with devtool: 'source-map' accidentally exposes server- side env vars baked into the bundle at build time.
• .wasm → linear-memory + import section dumped as strings (best- effort; native WASM symbol extraction lives behind the binary feature).
• Everything else (HTML, JSON that is not a source map, extensionless, …) → one chunk of text, scanned as-is.

Findings are tagged source: "web:js", web:sourcemap, web:sourcemap:raw, or web:wasm. Anything scanned as plain text (including the “everything else” case above) carries web:js; there is no separate web:other tag. The original URL is the file_path.

Use URLs without credentials, signed query strings, or secret fragments. Scan metadata redacts a target’s query and fragment, but a finding’s file_path identifies the fetched URL. Store web-source reports as sensitive artifacts. Do not add --show-secrets to a retained report.

SSRF defense

--url refuses to fetch:

• Private RFC1918 ranges (10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16).
• Loopback (127.0.0.0/8, ::1).
• Link-local (169.254.0.0/16, fe80::/10).
• Cloud metadata endpoints (169.254.169.254, the GCP / Azure / AWS / DigitalOcean / Hetzner variants).

This protection is not configurable. It prevents a URL scan from reaching a metadata service or another private endpoint.

Proxy and TLS

Remote sources and verification share the same explicit proxy and TLS policy.

Precedence is the CLI, then .keyhog.toml, then no proxy and strict TLS. HTTPS_PROXY, HTTP_PROXY, ALL_PROXY, and NO_PROXY cannot change it. TLS environment toggles are also ignored. --insecure applies to every remote-source and verification client in that scan, so use it only with an interception proxy or endpoint you control.

KeyHog sets User-Agent: keyhog/<version>.

Web fetching follows at most five redirects. Every redirect target is parsed, DNS-screened, and pinned again before the request. Verification requests follow no redirects. A verifier redirect becomes an error finding instead.

An invalid proxy value prevents the HTTP client from starting. An unreachable proxy does not fall back to a direct connection.

HAR auto-expansion

Any file with a .har extension is recognised by the filesystem source and expanded into one chunk per request and one chunk per response. Each chunk carries a source-type that tells you which side of the exchange it came from:

Finding file_path becomes <har-path>#<request-url>, so the same HAR with five different requests produces five distinct paths. Editors that jump-to-file on path:line URIs land on the HAR but the URL tail makes the location unambiguous.

The finding remains a normal redacted finding. This JSON fragment shows the exact location shape with synthetic values:

{
  "credential_redacted": "ghp_...4f2a",
  "location": {
    "source": "wire:har:request",
    "file_path": "capture.har#https://api.example.test/v1/me",
    "line": 2,
    "offset": 41,
    "commit": null,
    "author": null,
    "date": null
  }
}

The HAR request URL is part of file_path. A query credential in the captured URL can therefore appear in the report’s location even though the detected credential itself is redacted. Remove or sanitize sensitive query strings before sharing a HAR artifact or its report.

keyhog scan capture.har --format json-envelope | \
  jq '.findings[] | select(.location.source == "wire:har:request")'

This filters down to outbound credentials for the bug-bounty “what did I send” view. Select wire:har:response instead to see what the upstream reflected back at you.

A HAR that fails to parse is still scanned as plain text. KeyHog also records a structured_source_parse_failure coverage gap because request and response expansion did not happen. A metadata-bearing report is therefore partial, even if the raw fallback found no credentials.

The expander applies two bounds:

• Cumulative rendered request and response bytes are limited to four times --max-file-size.
• KeyHog decodes the text, checks for the required log and entries markers, and only then invokes the JSON parser. The marker check covers the full decoded text, so valid HAR metadata may precede entries.

Scanning a single HTTP exchange (stdin)

The most common ad-hoc workflow:

curl -s https://api.example.com/v1/me \
     -H "Authorization: Bearer $TOKEN" \
| keyhog scan --stdin

Or just pipe a saved response:

keyhog scan --stdin < response.txt

keyhog scan - (bare dash) is the same as --stdin (grep / wc convention).

--stdin reads up to 10 MiB by default. Raise the explicit --limit-stdin-bytes <SIZE> ceiling when a larger stream is intentional, or write the input to a file and scan the path. Findings from stdin carry the stdin source. To get the richer wire:har:request / wire:har:response provenance tags, save the exchange as a .har file and scan that instead (see HAR auto-expansion).

Use --format json-envelope for automation. Legacy json cannot say whether the source was complete:

set +e
keyhog scan capture.har \
  --format json-envelope \
  --output keyhog-results.json
status=$?
set -e

jq -e '.scan_status == "success" or
       .scan_status == "complete_after_recovery"' keyhog-results.json
test "$status" -eq 0 -o "$status" -eq 1 -o "$status" -eq 10

The jq check rejects malformed-HAR fallback and other coverage gaps. Exit 0 means no finding blocks the active evidence policy. Exit 1 means a finding blocks without a live verification result. Exit 10 means at least one live finding. A source failure with no blocking finding exits 13. Blocking findings take precedence, so a partial report can still exit 1 or 10.

Headers, bodies, and URL parameters

The detector engine matches the bytes supplied by each source adapter. A plain text capture is scanned as-is. The HAR adapter renders each request and response into separate chunks before scanning. A synthetic Bearer ghp_...4f2a in an HTTP header is therefore checked by the same detector policy as a synthetic "token":"ghp_...4f2a" in a JSON body or ?token=ghp_...4f2a in a URL.

The finding location gives the byte offset in the rendered chunk. It does not identify the exact header, JSON path, or query field. HAR findings identify only the request or response side.

Unsupported behavior:

• Parse the HTTP wire format and emit header:Authorization vs body:json:$.token provenance fields.
• Attach field-level provenance such as header:Authorization, body, or query to a finding. HAR findings do distinguish the request and response sides through source_type.

Fetch and parse errors

A non-success HTTP status, timeout, DNS failure, response above the configured size limit, invalid content encoding, invalid WASM response, blocked destination, or redirect-policy failure means some requested bytes were not scanned. KeyHog records the reason as a coverage gap and makes metadata-bearing artifacts partial.

When no finding blocks the active evidence policy, incomplete source coverage exits 13 and stderr says KeyHog is not reporting the scan as complete. A blocking or live finding takes exit 1 or 10 precedence. Consume the artifact’s scan_status and coverage_gap_summary; never use a finding count or process exit alone as a completeness signal.

Unsupported Wire Features

The wire-scanning surface is intentionally narrow. These features are not part of the shipped HTTP-wire contract:

1. mitmproxy .mitm flow-dump support. The binary-framed format is not decoded. Export HAR when request/response provenance matters, or export text and scan it as an ordinary file.

2. Header / body / URL-param provenance. HAR expansion emits one chunk per request and one chunk per response. It does not attach wire_location: header:<name> | body | query to each finding, so the JSON consumer cannot filter wire_location == "header:Authorization" for the highest-signal subset (intentional auth tokens vs accidental body leaks vs URL-logged secrets).

3. Live proxy mode. KeyHog does not ship keyhog proxy --listen :8080 or an inline HTTP proxy that scans flows while forwarding them.

4. WebSocket frame scanning. HAR files do not include WebSocket payloads, and KeyHog does not parse mitmproxy frame dumps as a WebSocket source.

Why this matters for bug bounties

A modern SPA bundle on a typical SaaS app can ship 200+ npm dependencies and a sourcemap that exposes every server-side env var the build process touched. Manual code review of one main.js.map against the full detector corpus is hours; running keyhog scan --url https://app.target.com/static/main.js.map takes seconds.

Pair it with --hide-client-safe (see CLI reference) to filter out keys that the vendor designed to ship in client bundles (Sentry DSN, Stripe pk_*, Mapbox pk., PostHog phc_, etc.) and you’re left with the keys that actually represent an exfiltration boundary.

Backends and routing

KeyHog has several execution engines for the same compiled detector policy. Changing a backend may change performance, startup cost, and hardware use; it must not change findings, locations, internal confidence, evidence verdicts, suppression, verification, or output ordering.

For the repository map, dependency direction, and bytes-to-finding pipeline, see Architecture.

The backend choices

Use automatic routing for normal scans

Start with a calibrated automatic route:

keyhog calibrate-autoroute
keyhog backend --autoroute
keyhog scan .

The last command is equivalent to keyhog scan . --backend auto. Cargo cannot execute a binary after cargo install, so run the first command once after installing a multi-backend Cargo build and again after any binary, driver, hardware, or routing-relevant configuration change. A scalar-only build needs no calibration and reports health: direct. auto performs an exact cache lookup. It does not try backends in order and does not benchmark during the scan.

For a healthy multi-backend installation, keyhog backend --autoroute reports health: ready. A scalar-only build reports health: direct because it has no backend choice to calibrate.

--backend is an explicit diagnostic or benchmark override. It bypasses autoroute and its calibration cache:

keyhog scan . --backend cpu
keyhog scan . --backend simd
keyhog scan . --backend gpu-cuda
keyhog scan . --backend gpu-metal
keyhog scan . --backend gpu-wgpu

Use these commands to compare engines or isolate a driver problem. Do not put an explicit backend in a routine scan configuration. It does not prove that the chosen engine is fastest for the input, repair autoroute evidence, or publish a route decision.

An explicit backend is a hard execution contract. If it was not compiled, its runtime is unavailable, initialization fails, or dispatch fails, the scan returns an error. KeyHog does not substitute CPU, SIMD, or another GPU peer. gpu-cuda, gpu-metal, or gpu-wgpu are separate choices. There is no generic gpu override.

Check SIMD and GPU availability

Inspect discovery before forcing an accelerator:

keyhog --version --full
keyhog backend --self-test
keyhog backend --self-test --require-gpu

--version --full reports whether the running binary can see Hyperscan/Vectorscan and a physical GPU. This is a discovery report, not an execution test. backend --self-test executes the GPU diagnostic and production dispatch paths. It reports SKIP and exits successfully when no physical GPU is available. Add --require-gpu to make that condition fail. After calibration, keyhog backend --autoroute --json records the exact eligible_backends set that was measured.

The scalar cpu-fallback backend is always the portable reference. The simd-regex backend requires a build with scanner SIMD support and a usable, identifiable Hyperscan/Vectorscan runtime. A CPU with AVX2 or AVX-512 alone does not provide that runtime. GPU candidates require scanner GPU support, a physical device, and a usable driver path. CUDA, native Metal, and WGPU are acquired and measured independently, so one available peer does not imply that another is available. Diagnose exact engines with --backend simd (simd-regex), gpu-cuda (gpu-cuda-region-presence), gpu-metal (gpu-metal-region-presence), or gpu-wgpu (gpu-wgpu-region-presence).

Library backend contract

The Rust library deliberately has a different default contract. Calling CompiledScanner::scan or scan_coalesced without a backend uses the portable cpu-fallback reference, so identical library code does not change execution with host hardware or local calibration files. Library callers that want acceleration choose scan_with_backend/scan_coalesced_with_backend; the CLI is the owner of persisted automatic routing.

Those explicit-backend methods return typed Result values. Unavailable selected SIMD or GPU backends and later runtime failures return ScanError; they never terminate an embedding process and never return findings from another backend. warm_backend probes startup eligibility in-band. The keyhog CLI separately maps terminal scanner errors to its documented exit statuses. The no-backend portable CPU methods do not acquire an accelerator.

The GPU literal matcher keeps its immutable VYRE tables resident after the first successful batch. One dispatch returns both region presence and complete positions for the shared confirmed-anchor and generic-keyword localizers. Backend-shaped phase-two DFA catalogs are also immutable for the compiled detector set and are reused across scans. Haystack and region capacity grow in bounded bands from the actual workload. KeyHog serializes each resident session so concurrent requests cannot interleave uploads against the same device buffers. Preparation, growth, match-output overflow, dispatch, and readback errors remain selected-GPU failures. Teardown cleanup errors are logged. There is no per-batch pipeline or CPU substitution. Each physical dispatch accepts at most 65,536 positioned literal matches, which bounds resident readback to 768 KiB. Exceeding that cap returns no partial evidence: automatic routing visibly replays the stable bytes, while an explicit or required GPU route fails its backend contract.

A coalesced request above the smaller of the live VRAM/config budget and the selected backend’s hard ceiling is split between source chunks. An individually oversized chunk is scanned through physical windows whose overlap covers the longest compiled GPU literal. Window presence rows are OR-reduced and position rows are offset-adjusted and deduplicated into one logical source row before phase-two evidence is consumed. A complete region-presence request above 4,096 physical dispatches fails visibly before execution instead of amplifying chunk count or custom-detector overlap without bound. Prefixless phase-two GPU regex admission stays on whole chunks because regex width may be unbounded. Oversized rows retain the authoritative CPU no-hit admission path instead of accepting an unsafe GPU negative. Readback words are consumed through a scoped borrow while the resident session is locked, then zeroized without discarding the warmed host allocation.

What “same results” means

Calibration compares the complete RawMatch identity: chunk index; detector id, name, service, and severity; exact credential, stored-hash, and companion identity; source, file, line, offset, commit, author, and date; entropy, internal confidence, evidence tier, and evidence reason. A candidate is rejected if any field or finding multiplicity differs from the Hyperscan reference, if repeated reference trials are inconsistent, or if required GPU timing evidence is invalid. Diagnostics name only the differing fields and occurrence counts. They never emit raw values or deterministic value fingerprints. Normal automatic scans do not benchmark or silently replace a rejected backend.

Each backend is measured with phase-two plain-pattern localization disabled and enabled. The persisted route owns both choices, so concurrent scans never mutate scanner-global tuning and decode or recovery replays retain the selected route.

Among parity-correct candidates, routing uses complete trial distributions, never a lucky fastest trial. The selected route’s 95% confidence interval must lie below every other eligible execution route. Phase-two localization plans are distinct routes even when they use the same backend, so overlapping same-backend timings are inconclusive rather than permission to choose the lowest median. Overlap that separates nothing at all resolves to the lowest-complexity backend inside the fastest route’s own 95% upper bound, reported as a dead heat rather than as a proved win. Autoroute inspection prints this selection basis.

scan_coalesced_with_backend already includes extraction, decode, built-in suppression, confidence, and scanner postprocessing. Autoroute parity therefore compares the complete RawMatch values returned by that production scanner path. CLI allowlists and rules, severity and confidence floors, cross-source deduplication, optional verification, and reporting run after backend selection. The same detector TOML corpus and resolved configuration digest identify every backend and localization plan.

Why size alone is insufficient

Two inputs with the same byte count can have different winners. Autoroute also keys evidence by logarithmic buckets for bytes, chunk count, largest source size, and detector pattern count, plus one boolean recording whether any decoder was admitted. Source family, resolved configuration, build features, and host identity also participate.

A measured key covers only the values grouped into that key. A size band nobody measured is served only when at least two measured bands of the same source class and decode state reconcile to one route: a backend measured at every one of them and proved slower at none, on the compiled default plan when they split on the plan. Never for a GPU route. See Autoroute calibration.

Runtime lifetime matters too. A one-shot process includes GPU first-dispatch cost. A ready daemon has already initialized accelerator state and uses the warm GPU trials from the same calibration evidence. See Daemon and warm scans.

The 8 MiB Hyperscan crossover

The July 10 RTX 5090 artifact is retained for regression history, but it is not release or routing evidence. Its SIMD timing used the generic per-chunk entry point instead of the faster production coalesced Hyperscan path. The artifact is marked production_comparable = false and must not support a crossover claim.

The checked benchmark now sends identical 1 MiB windows with 128 KiB overlap through the explicit production execution-route entry point for Hyperscan and every acquired CUDA, Metal, or WGPU peer. It measures all four combinations of plain-pattern and keyword-anchor localization and every resident pipeline depth the peer declares eligible. Synchronous peers expose depth one. Asynchronous peers expose depths one through four. It requires sorted full-match parity from every route, rejects GPU degradation, and rotates candidate order during selection. Selection-only samples choose one measured-correct GPU route and one measured-correct Hyperscan route. Both routes then run in 300 fresh rotating held-out trials. Every other parity-correct Hyperscan route also runs in those trials and remains visible in the artifact, but no held-out observation can change either selected route.

The gate passes only when the selected GPU to selected Hyperscan paired ratio’s 95% confidence upper bound is below 1.0 at 8 MiB. A slower CPU tuning choice cannot make the GPU result look favorable because independent selection compares all eligible localization plans before the held-out phase. A per-trial minimum across several CPU plans is not an eligible backend and is not used as a hindsight oracle. A forced plain or keyword localizer filter, profiling, or perf tracing retains parity and degradation checks but cannot pass the release speed gate.

Schema 10 records both selected backends, localization choices, and the GPU pipeline depth, every route-selection sample, and a separate held-out confidence interval for each Hyperscan plan. crossover_passed is based only on the independently selected GPU and Hyperscan routes. Use --diagnostic for an unprofiled 8 MiB measurement from a dirty development tree. That mode retains exact parity and degradation checks but records diagnostic = true, production_comparable = false, and cannot pass the release gate.

Diagnostic runs may isolate either localization dimension with KH_BENCH_PHASE2_PLAIN_LOCALIZER=0|1 or KH_BENCH_PHASE2_KEYWORD_LOCALIZER=0|1. Setting either variable makes the run ineligible for release evidence; an unrestricted run measures all four plans.

Use --profile to attribute scanner stages to exact routes. Candidate selection and held-out trials remain unprofiled; after timing, the benchmark runs one isolated scan for each Hyperscan localization plan and the selected GPU route. Profile labels include the backend, both localization values, and the resident pipeline depth, so costs from different execution plans are never merged into one report. Profile runs cannot pass the release gate.

The checked artifact at benchmarks/baselines/gpu_8mib_crossover_rtx5090.toml retains the last measured timing and parity distribution, but it is historical rather than release evidence because that run did not attest a clean source tree. It recorded 143 identical findings with no degradation, a 24.5886 ms VYRE CUDA median versus 69.5641 ms for Hyperscan, and a paired ratio confidence interval of 0.3482 to 0.3579 across 100 held-out pairs. Those measurements cannot prove the current release binary is reproducible from the recorded commit. A new crossover claim requires build_source_tree_state = "clean", source_tree_state = "clean", and production_comparable = true from the corrected route with exact binary, detector, configuration, host, runtime, workload, result count, peer, and trial identity. The build script watches the tracked and non-ignored source inventory, so cleaning a tree after compiling dirty source forces a rebuild before the artifact can qualify.

Run the crossover benchmark when you change backend performance or routing. Release automation does not run it. The benchmark remains the evidence for route comparisons: it records the candidate commit, detector digests, feature set, GPU identity, finding parity, held-out pairs, and confidence interval. Autoroute still requires calibration on the deployment host for the exact workload class.

What the GPU does for a whole-tree scan

The crossover above measures one 8 MiB window through the matching kernel. A repository scan is a different workload, and the answer there is different, so measure before you choose a backend for one.

Scan a tree with each backend and compare. These are median wall times over five runs on an RTX 5090 with a Ryzen 9 9950X, scanning copies of this repository:

The GPU route is slower, by roughly the same percentage at both sizes, so this is not a startup cost that a larger tree amortizes away.

The reason is which stage the GPU accelerates. A scan runs phase one, which finds candidate regions, and phase two, which confirms them against the full detector patterns. The GPU runs phase one. Phase two runs on the CPU in both cases, and phase two is the larger cost: a --perf-trace of one 4,096 chunk batch shows dispatch=0.04s for the GPU kernel against phase2=0.39s for confirmation. The GPU shortens the smaller half and adds its own dispatch and transfer on top of an unchanged larger half.

You can see the same thing in the trace field phase2_gpu_ascii_patterns. It reads 0 on this detector corpus, meaning no pattern is eligible for the GPU phase-two path, so none of the confirmation work moves off the CPU.

What to do with that:

• For a repository, container, or history scan, leave routing automatic. The calibrated router measures both and will pick the CPU route when it is faster.
• Use an explicit --backend gpu-cuda for diagnostics, parity checks, and kernel-level benchmarking, not because you expect a whole-tree scan to finish sooner.
• Findings do not depend on the choice. Every route reports the same secrets, which is what the parity contract above guarantees.

Memory footprint and zero-allocation execution

KeyHog is designed to scan multi-gigabyte repositories and large disk images with a bounded memory footprint.

Streaming windowing and rendezvous channels

Files larger than 1 MiB are divided into overlapping 1 MiB windows with 128 KiB overlap. The filesystem reader uses a rendezvous queue (fused_depth = 0), so windows are handed directly to the scanner workers without accumulating resident memory in crossbeam channels.

Zero-allocation source semantic indexing

Structured configuration and source AST indexing in StructuredSourceIndex uses compact byte offsets (SourceSpan) and fixed-size stack arrays ([SourceSpan; 12]) rather than allocating heap strings or nested hash maps. Tokens are referenced as borrowed slices of the original window.

Bounded GPU scratch and readback buffers

The GPU execution engine bounds device allocations and host transfers:

• Resident literal tables are compiled once and reused across dispatches.
• Match readback buffers are capped at 65,536 entries (768 KiB), preventing high-candidate inputs from triggering host memory allocation spikes.
• Oversized chunks are sliced into physical windows, uploaded within the configured --gpu-batch-input-limit, and reduced on-device before transfer.

Memory tuning controls

Automatic routing failures and recovery

Automatic routing has two visible failure states. Neither one changes an explicit backend contract.

The route state is invalid

Missing, stale, malformed, disabled, incomplete, or quarantined evidence cannot authorize an automatic route. KeyHog prints the missing workload identity and a repair command. No backend is selected. The affected batch remains unscanned, metadata-bearing output records partial coverage, and the process exits non-success.

Run keyhog backend --autoroute to distinguish calibration_required, stale, invalid, disabled, and quarantined. Run keyhog calibrate-autoroute for the core ladder. For Git, Docker, or web workloads, run the exact scan --autoroute-calibrate --autoroute-gpu repair command printed in the routing diagnostic.

Use an explicit backend only when you intentionally want a diagnostic override. It bypasses the invalid route state but does not repair it.

A selected automatic backend faults

During a normal automatic scan, an accelerated backend fault is warned and the same stable bytes are replayed through the confidence-separated fastest remaining peer. GPU recovery replays only exact unprocessed intervals. A backend that fails before scanning replays the full stable batch. Completed dispatches remain owned by their original backend.

This recovery is not silent. KeyHog reports the failed and recovery backends, recovered ranges, chunks, and bytes, and records complete_after_recovery. The exact workload route is quarantined in a bounded runtime-health artifact. That artifact is separate from immutable timing evidence, survives restart, and clears the repaired workload only after successful recalibration. If recovery cannot prove full coverage, the result is incomplete rather than clean.

Calibration candidates, explicit backend overrides, and --require-gpu remain hard execution contracts. They fail instead of recovering through another backend.

For cache identity, inspection commands, calibration coverage, and recovery, see Autoroute calibration.

Performance evidence

Treat a timing as comparable only when the executable, detector corpus, configuration, workload, host, and route are recorded. A table without those identities is historical context, not release evidence.

Canonical receipts

The repository owns two canonical generated evidence surfaces:

• readme-matrix.json records scanner configuration, executable SHA-256, detector-corpus SHA-256, workload SHA-256, host resources, findings, precision, recall, wall time, throughput, and peak RSS. It also binds the generated README tables to the workload-catalog digest. make -C benchmarks readme-matrix-check rejects prose or table bytes that do not match the receipt.
• readme-scaling.json records thread and process scaling with every trial, effective core count, page-cache state, storage class, workload size, findings, wall time, and peak RSS.

The benchmark index in benchmarks/README.md owns focused receipts for Bloom filtering, autoroute, daemon routing, recovery, and competitive accuracy. Read the linked JSON receipt before quoting a generated Markdown report.

Comparison boundary

Match all of these fields before comparing two rows:

1. executable digest and stamped commit;
2. detector-corpus digest and detector count;
3. resolved configuration, including backend, cache, daemon, verification, decode, and confidence policy;
4. workload digest, bytes, file count, and input shape;
5. host CPU, GPU, memory, operating system, affinity, and cgroup quota;
6. warm or cold page-cache and process state;
7. trial count, aggregation rule, exit code, scan status, findings, and coverage gaps.

A backend override is diagnostic. It does not prove automatic routing. Autoroute evidence is valid only when calibration authenticated the exact workload class, binary, detector/config state, host, accelerator state, and selected route.

Reproduce the generated tables

Run:

make -C benchmarks readme-matrix-check
make -C benchmarks readme-scaling-check

To collect new measurements, follow the command in the focused report or benchmark index. Keep raw host-local results outside tracked reports until the run records immutable executable and workload identities and exact finding parity.

Interpret older reports

benchmarks/reports/perf.md, cross-device.md, and workload-matrix.json predate the complete canonical receipt contract. They remain useful for investigation, but their rows must not replace the generated README matrix or scaling receipt. In particular, an absolute path, a mutable binary path, a missing executable digest, or an unattested working tree prevents a release claim.

Daemon measurements have a separate boundary. One-shot process time includes scanner construction; warm daemon request time does not. Resident daemon RSS belongs to the server. Compare daemon rows only with the same request class and lifecycle.

Multi-corpus benchmark evaluation

Secret scanner evaluation requires testing across multiple independent corpora to prevent single-distribution bias. Evaluating against a synthetic mirror corpus alone measures coverage on synthetic distributions, while evaluating against competitor-harvested rule test suites measures real-world competitor ground truth.

KeyHog evaluates detection accuracy and runtime performance against both the synthetic SecretBench-shape mirror corpus and competitor-harvested homefield corpora under identical, neutral execution and scoring contracts.

Multi-corpus methodology

Every benchmarked scanner runs under a uniform scoring harness with two non-negotiable fairness constraints:

1. Answer-key isolation. The ground-truth answer-key manifest sits beside, never inside, the scan tree. Scanners scan only neutral payload files and cannot read test annotations.
2. Neutral scan directory layout. Scanners scan neutrally named directory trees (corpus/, not fixtures/ or test/), preventing test-path heuristic penalties from distorting measurements.

Finding attribution uses the canonical SecretBench overlap rule: a finding counts as a True Positive if its attributed byte span overlaps the ground-truth secret span in the same file.

Benchmark results

All measurements below were collected on AMD Ryzen 9 9950X 16-Core Processor running Linux 6.17.0-19-generic with 32 logical cores.

Synthetic mirror corpus

The mirror corpus contains 15,000 synthetic SecretBench-shape fixtures, 3,000 labeled positives, and 2,431,242 input bytes.

Competitor homefield corpus

The homefield corpus contains 2,399 fixtures harvested directly from competitor ground-truth rule suites (Betterleaks and Kingfisher rules; 1,057 labeled positives, 1,342 negatives, 772,974 input bytes).

Provenance and reproducibility

Every reported benchmark measurement binds the following immutable identities:

• Scanner executable digest (SHA-256) and stamped git commit hash.
• Detector set count and detector corpus digest.
• Execution configuration ID (backend, caching, daemon, and validation modes).
• Host CPU, memory, GPU, kernel, and operating system identity.
• Workload byte count, fixture count, and labeled positive count.

To reproduce measurements locally, see benchmarks/README.md and docs/src/performance-evidence.md.

Autoroute calibration

Not the same as confidence calibration. This page is about backend selection: measuring which engine (SIMD, scalar CPU, GPU) is fastest and proven-correct for your workload. For the per-detector Bayesian confidence counters (keyhog calibrate --tp/--fp), see Confidence calibration.

KeyHog uses measured evidence to select an execution route for a calibrated workload key: Hyperscan/SIMD, scalar CPU, CUDA, native Metal, or WGPU, each measured with all four combinations of phase-two plain-pattern and keyword-anchor localization. GPU routes also measure every resident pipeline depth the acquired peer declares eligible. Synchronous peers expose depth one; asynchronous peers expose depths one through four. It does not guess from a device name or a hard-coded size threshold. Autoroute is not a fallback hierarchy. During calibration KeyHog measures every eligible execution class exposed by that scanner, rejects candidates whose complete redacted raw-match identity differs from the independent scalar reference, and records the fastest survivor for the measured representative. Optional SIMD, CUDA, Metal, and WGPU engines are candidates, never correctness oracles. Every executable GPU path is acquired and measured independently. One driver never substitutes for another. The parity identity covers chunk membership; detector id/name/service/severity; exact credential, stored-hash, and companion identity; full source/history location; entropy; confidence; and finding multiplicity. Mismatch diagnostics expose only field names and occurrence counts. They never expose credentials, companions, history values, or deterministic value fingerprints. Normal scans then do a direct table lookup; they never benchmark mid-scan.

Calibration, in-process batches, and daemon requests call the same explicit backend-dispatch boundary. Hyperscan uses its coalesced multi-chunk path. Scalar CPU and GPU use their normal batch paths, including the measured GPU resident pipeline depth. A timing row therefore measures the implementation that the persisted route authorizes.

Ordered multi-device GPU routes

When one GPU API exposes two or more distinct physical adapters, calibration also measures the complete ordered device set as a peer route. The route binds each adapter’s stable topology, driver/runtime identity, capacity, and measured integer throughput weight. Cross-API aliases for one physical adapter are deduplicated before the set is formed.

Acquisition is all-or-nothing. A missing, reordered, reset, or identity-changed member invalidates the complete route before any batch is scanned. KeyHog allocates each member’s bounded resident slots before dispatch, checks the aggregate process ceiling, assigns one contiguous weighted source range to each member, dispatches concurrently, and retires results in source order. An error, panic, incomplete receipt, or missing shard on any required member invalidates the whole route; sibling results are not reported as complete.

The device-set identity is stable across workload-specific throughput weights. One scanner therefore acquires one resident set for that physical topology, while each workload decision retains its own authenticated weights, budgets, pipeline depths, detector/config digests, and correctness receipt. Normal scans never retime or rebalance the set.

A route class must be something calibration can enumerate ahead of any scan. The workload key is therefore the shape of the work, not a measurement of the bytes: logarithmic byte, chunk, maximum-file, and pattern bands, one boolean recording whether any decoder was admitted, and the canonical set of source execution classes with each class’s size provenance. Reordering chunks keeps the same key. Changing the proportion between two source classes, which decoder families ran, the phase-one admission outcome, the phase-two keyword density, or the number of decode candidates does not: those are properties of the input, and a key that contained them made every scan an uncalibrated class.

Calibration still observes those statistics. It logs the phase-2 keyword trigger counts for each measured decision on the keyhog::routing tracing target, and every persisted point records its exact sample byte count, chunk count, and measurement shape digest. They describe a measurement; they do not select one.

Noncanonical, duplicate, inconsistent, or oversized persisted mixtures invalidate the cache instead of being normalized silently. Each persisted decision also carries a digest of the complete workload key. Changing or relabeling any keyed field invalidates the row before routing.

Filesystem producers keep each path’s chunks contiguous. KeyHog uses that contract to end a batch when the source execution class or full-size provenance changes, unless the next chunk belongs to the same path dependency. Ordinary, windowed, PDF, archive, web-script, source-map, and other preprocessing classes therefore use independently measured homogeneous routes. Dynamic ELF, PE, and Mach-O section names collapse to their binary-format class because the label does not change execution. Sources without a contiguous-path contract retain their exact mixed key instead of being split on an unsafe assumption.

Git diff producers make the same ordering guarantee. Tracked diff hunks and full-size untracked files therefore calibrate as separate route classes during installer calibration, even when one --git-diff scan contains both.

Performance selection uses the complete recorded distribution, not the single fastest sample. All candidates are measured in the same rotated rounds, so KeyHog compares paired per-round differences at 95% Student-t confidence. If those differences prove one exact plan faster than every peer plan, selection_basis is exact-plan-paired-95pct-confidence. If same-backend plans remain tied, KeyHog chooses the compiled default when it is among the tied leaders. Otherwise, it chooses a stable typed plan from that tied set. The chosen plan’s interval must remain below every plan on each peer backend. Inspection reports peer-separated-compiled-default-plan or peer-separated-statistically-tied-plan.

When nothing separates, the measurement is resolved rather than discarded. A route stays in contention unless some peer is proved faster, meaning that peer’s whole 95% interval lies below the route’s own. Among the routes still in contention, only those whose median falls inside the fastest route’s own 95% upper bound are eligible, so a route can never win on a wide error bar while its central tendency is measurably worse. The eligible set is then ordered by backend complexity, cpu-fallback before simd-regex before the GPU peers, because when nothing is proved faster the backend that needs no accelerator bring-up and always runs is the honest choice, and it is the same choice on every rerun of the same evidence. Inspection reports unseparated-dead-heat-lowest-complexity-backend and confidence_separated stays false, so a permitted decision is never presented as a proved one.

This matters on real trees. Calibrating benchmarks/corpora/homefield measured cpu-fallback at 4.507 s [3.08, 11.49] against gpu-wgpu at 4.462 s [4.40, 4.92], with every interval overlapping every other. Refusing to decide left the workload with no persisted route, so every later automatic scan failed closed without scanning even though the measurement showed the backends were indistinguishable.

Calibration records 7 normalized timing trials per route. A warm trial repeats short candidate executions until their combined timing reaches 10 ms. It stops after 1,024 executions and records the per-execution average. This keeps scheduler resolution from dominating small workloads without extending large workloads. Accelerator evidence retains one real cold dispatch. Steady and warm rounds rotate route order so host drift is shared across peers. Overlap that survives the rotation is resolved as a dead heat rather than spending unbounded install time or guessing.

Because the decision is measured, it must be recorded before --backend auto (the default) can claim a fastest route. A fresh install has no decisions yet, so an automatic scan selects no backend for each unproved batch, records incomplete coverage, and reports autoroute calibration required with a repair command.

Calibrate, inspect, then scan

For a multi-backend build, use this sequence:

keyhog calibrate-autoroute
keyhog backend --autoroute
keyhog scan .

The scan uses auto by default. It reads the persisted table and never benchmarks during the scan. Use an explicit backend only for a deliberate diagnostic or benchmark.

Before calibration:

• Run the same KeyHog binary that will perform the scans. Build identity and scanner feature identity are part of every decision.
• Use a writable persistent cache path. --autoroute-cache off is rejected because calibration must publish durable evidence.
• Keep the host reasonably idle. Route trials are interleaved across peers so common drift is shared. Overlapping intervals resolve deterministically to a non-inferior low-complexity route; unusable evidence or backend disagreement across retained points exits without publication.
• Make every source prerequisite available. The subcommand covers the core stdin and filesystem ladder. Git, Docker, and web fixtures use the low-level scan --autoroute-calibrate probe on the exact source.

A build with only cpu-fallback reports health: direct and does not require calibration.

Calibrate core and source-specific workloads

Cargo installation does not benchmark your host. Calibrate the installed binary before its first automatic scan:

keyhog calibrate-autoroute

Run the command again after a binary, detector, configuration, driver, or hardware change. The command calibrates the core stdin and filesystem workload ladder. A routing diagnostic for an unproved Git, Docker, or web workload prints the exact low-level scan --autoroute-calibrate --autoroute-gpu command.

The default command calibrates the ordinary policy and every documented preset:

keyhog calibrate-autoroute                 # all policies
keyhog calibrate-autoroute --policy default
keyhog calibrate-autoroute --policy fast
keyhog calibrate-autoroute --policy deep
keyhog calibrate-autoroute --policy precision

The policy names correspond to no preset flag, --fast, --deep, and --precision. Each policy has its own resolved configuration digest and route decisions. A focused run keeps valid evidence for other configurations. It publishes only after every workload in the selected policy succeeds. Use the default all-policy sweep for a new installation. Use a focused policy to repair or refresh only the preset you run.

Which configuration the run measures

Route decisions are stored under the resolved scan configuration, and a .keyhog.toml found on the walk-up from the working directory is part of it. Run keyhog calibrate-autoroute from the repository whose scans it serves. Running it elsewhere primes the compiled-in defaults, and a scan inside a repository that carries a .keyhog.toml then reports none matching config digest and exits 2.

keyhog calibrate-autoroute                 # measures ./.keyhog.toml if present
keyhog calibrate-autoroute --no-config     # measures the compiled-in defaults

install.sh and install.ps1 pass --no-config. An install runs from an arbitrary directory, so it primes the host baseline; calibrate again inside a repository that overrides scan policy. The all-policy sweep measures in four isolated child processes, and each child inherits the mode: asking for the baseline measures the baseline in all four.

This drives the core stdin + filesystem workload ladder across every scan preset. Plain single-file probes cover every power-of-two size band from 1 byte through 32 MiB, with additional 4 MiB + 1, 8 MiB - 1, 8 MiB + 1, and 16 MiB - 1 probes retaining raw evidence on both sides of the required 8 MiB crossover. A coarse size class holds its points together when they agree, and also when they disagree without proving anything: the class keeps the lowest-complexity backend that is measured at every point and measurably slower at none. A disagreement that measurement does prove, where one point’s whole 95% interval for the selected backend sits above a peer’s, is a real crossover; it rejects calibration and requires the class to be split. Such a class reports confidence_separated: false, because the route is permitted by the evidence rather than proved by it. File-tree probes cover every chunk-count band through the default 32-chunk fused batch. Tar-member probes cover the same count ladder for payload-derived extracted filesystem chunks. Decode-heavy probes cover the decoder path. Empty input has no routing work and is not counted as a calibrated workload; daemon and watch paths return the exact empty result without consulting the cache. Each preset uses one compiled production scanner. Immutable detector, GPU literal, and GPU phase-two program artifacts are reused. Workload-shaped resident GPU state is reset before each representative. The measured shared literal and backend-shaped phase-two preparation costs are added to every matching one-shot GPU observation. Candidate order rotates across workload bands rather than giving one backend the same thermal position in every probe. The final count is the number of probes run, not the number of unique persisted route classes. Multiple representatives can share one logarithmic workload key. The summary separately reports unique route classes, exact measurement points retained by this sweep, and the total route decisions in the cache. The required readback checks every measured shape digest, not only the shared workload key. A missing representative prevents publication. The cache total can include valid decisions from prior calibration runs. The command also prints a cache route summary showing how many one-shot and daemon rows select a VYRE GPU route, plus the number of GPU candidate receipts measured. The command does not cover Git, Docker, or web source probes. Those workloads need a real external fixture such as a repository, running daemon, or served URL. The low-level scan --autoroute-calibrate probe measures one caller-supplied workload. It does not synthesize or sweep external fixtures. If one of these sources reports autoroute calibration required, run the reported repair_command. Decisions are written, parity-checked, to the autoroute cache ($XDG_CACHE_HOME/keyhog/autoroute.json by default; override with --autoroute-cache <path> or [system].autoroute_cache).

Canonical calibration admits every eligible execution class. The low-level scan --no-autoroute-gpu --autoroute-calibrate diagnostic measures a CPU-only candidate set, and that evidence cannot overwrite or be replayed by a normal all-candidate decision. The isolation lives in the persisted host generation, not in the config digest. A host generation records the eligible backend census plus the complete GPU device, runtime, driver, and batch-limit identity, and a scan replays a row only when that whole profile compares equal, so a CPU-only measurement is invisible to a scan that admits a GPU.

The resolved config digest deliberately does not record whether a calibration excluded a GPU. Recording it there was a guaranteed miss on every host and build with no GPU candidate, because the exclusion is vacuous but the digest still differed: calibration wrote decisions under a key no scan would ever request, and the immediately following identical scan reported a config mismatch and left the batch unscanned.

Startup reports every available GPU peer without creating execution devices or pipelines. Calibration acquires each peer when its candidate is measured and reports the exact acquisition failure. The autoroute cache stores separate CUDA, Metal, and WGPU cold and warm timing vectors, and keyhog backend --autoroute prints each eligible peer. A failed driver is ineligible until it is repaired and calibration is rerun.

Low-level calibration saves take an exclusive sibling-file lock across the complete read/merge/atomic-write cycle. The canonical calibrate-autoroute command adds a generation transaction around the full workload and preset sweep: every probe writes to an isolated cache, completed evidence is read back and validated there, and the live cache is replaced once only after the full sweep succeeds. A failed probe leaves the live cache byte-identical. Publication also compares both the live cache and its runtime-health artifact captured at sweep start while holding their canonical locks. If another process changed either one, KeyHog preserves the concurrent update and asks the operator to rerun instead of overwriting evidence or clearing a new route fault. A successful publication clears only the exact route faults remeasured by this sweep. The operating system releases a held lock if a writer exits or crashes.

Only identity-compatible, structurally valid rows are preserved. A storage or permission error while reading an existing cache aborts without replacing it. A readable cache with an incompatible schema, invalid JSON, invalid structure, or a different build/corpus identity emits an unconditional stderr warning with the cache path and replacement reason, then starts a fresh staged generation; unrelated rows from that invalid artifact are not merged.

One cache can be shared across hosts. Each route generation is keyed by the exact resolved config digest and host profile. Calibrating the same config on a second host preserves the first host’s evidence, and recalibrating either host merges only that host’s workload rows. A scan replays only the generation whose complete host identity matches the live machine. JSON inspection exposes the stable host_identity digest used to distinguish those generations.

Reading the cache hit rate

Every automatic scan prints one line on stderr saying what the cache did:

INFO autoroute cache: 100.0% hit (2 hit / 2 lookup(s))

One lookup is one batch asking the cache for its route. A hit means the batch ran on a persisted, measured-correct backend without benchmarking anything. A miss means no backend was selected, the batch remained unscanned, and the run records incomplete coverage.

The line prints in every output mode, including --format json -o <file>. That is the shape CI and calibration harnesses use, and it used to suppress the whole routing summary.

A scan with any miss names the cause and the repair:

WARN autoroute cache: 0.0% hit (0 hit / 2 lookup(s)); every byte was still
scanned, this costs speed not coverage; miss causes: cache-rejected=2;
2 distinct uncalibrated bucket(s); repair: the cache belongs to a different
build, host, detector corpus or scan config; recalibrate this exact
configuration (recalibrating one bucket will not help)

Read the miss cause before you recalibrate. The causes call for different actions:

A miss costs speed, never coverage. Every byte is still scanned. This line is not a coverage gap, and a 0% hit rate does not mean the scan was incomplete; see Coverage truth for the signals that do mean that.

Run with -v to list every distinct uncalibrated bucket under the keyhog::routing target, most expensive first. One recalibration can then be planned to cover all of them, rather than learning about one bucket per run.

Under --profile, the same outcomes appear in the standard cache family as autoroute-decision (a scan reusing a persisted route) and autoroute-calibration (a calibration reusing evidence instead of measuring again), with hits, misses, and hit_rate_ppm in the --profile-out JSON.

Cache schema compatibility

The cache has one strict schema version. KeyHog reads the small version field before decoding any version-specific payload, so an older or newer cache cannot be mistaken for a partially valid one. There is no silent in-place migration: an unsupported version is reported as unsupported autoroute cache version with the version found, the version expected by the binary, and the command to regenerate it. The scan loader, calibration merge path, and backend --autoroute inspection use this same diagnostic. Re-run calibration after upgrading KeyHog or changing the cache format; a replacement save never merges rows from an incompatible schema.

Any decision containing a GPU one-shot route, persistent route, parity receipt, or measured candidate also binds the installer-owned GPU matcher manifest. KeyHog verifies every named .bin member against its SHA-256 digest before accepting the cache. Missing, malformed, duplicate, symlinked, oversized, or changed members reject the cache. Unrelated lazy runtime-cache files do not change this identity.

Each timing point stores a content-addressed measurement receipt: the canonical receipt generator, a digest of the complete payload multiset, and a digest of the exact source, offset, and decode shape. It stores no source text or paths. Same-sized representatives with different candidate density therefore remain distinct points, while the same chunks in a different producer order reuse one receipt. keyhog backend --autoroute --json exposes all three fields so a crossover can be tied to its exact probe.

What a decision covers

A decision is tied to its recorded build identity, host profile, detector corpus, and routing-relevant resolved scan configuration. Options that change that identity get their own calibration, even when they do not change which backend is fastest:

• Build identity records the exact running executable SHA-256, package version, Git hash, and the CLI and dependency feature sets. GPU and SIMD support are read from the scanner library that actually owns and compiled those backends, not inferred from similarly named CLI features. Source capability identity separately records each compiled filesystem, archive, forge, cloud, container, and web source feature (including GitHub, GitLab, and Bitbucket), while verifier identity records whether live verification is compiled. A different artifact or recorded capability set cannot reuse the evidence, including dirty/profile/native-link builds that happen to share a package version and Git hash.
• Host identity includes OS/architecture, CPU model and topology, memory, CPU instruction support, the live linked Hyperscan/Vectorscan runtime version when SIMD is eligible and, when the scanner can use a physical GPU, the GPU device, every available runtime backend and version, driver/runtime identity, resolved batch-input byte cap, and the exact sorted eligible-backend census for that resolved config. A missing or changed required field invalidates the evidence and requires recalibration.
• SIMD is admitted when the scanner produced a nonempty backend-neutral plan and the linked Hyperscan/Vectorscan runtime has a reproducible identity. Scanner construction does not build its databases. Calibration or a selected SIMD route materializes the plan exactly once; failure aborts calibration or the selected scan with the initialization reason instead of removing SIMD from the census or substituting scalar CPU.
• Backend identity covers the complete scan tail, not only phase one. The always-active phase-two Hyperscan prefilter executes only for the SIMD candidate. Scalar and GPU candidates use their own measured trigger path and the portable host residual, so their timing cannot borrow hidden SIMD work.
• Each scan preset (default, --fast, --deep, --precision) is calibrated separately.
• Flags hashed into the scan config (for example --threads, --min-confidence, --profile, or --perf-trace) fork the decision; instrumentation cannot reuse timings measured without its hot-path overhead. keyhog calibrate-autoroute sweeps the documented presets so the common combinations are covered.
• Candidate-shape knobs (--min-secret-len, --entropy-threshold, decode depth, entropy/ML/keyword floors) fork the decision, because they change what reaches scan-phase output and can therefore change backend crossover.
• Pipeline knobs (--threads, --reader-threads, --fused-batch, --fused-depth) and [tuning] settings fork the decision because they change work partitioning and backend warm-up behavior.
• One calibration process may reuse a KeyHog-owned Rayon pool only at the same worker width. An external pool is rejected because its stack, naming, and ownership settings cannot be attested. An incompatible preset or live width fails before measurement, and the actual count is part of the resolved config identity.
• Source policy (--limit-*, --max-file-size, --no-default-excludes) and detector floors fork the decision for real stdin/directory buckets that feed different cache/chunk geometry.
• Workload shape matters: a single file, a directory, and a piped stdin stream are distinct buckets, and stdin is content-sensitive.

The host profile is deliberately checked, but it is not a complete performance- environment fingerprint: for example, CPU governor, system load, and every accelerator limit are not all identity fields. Inspection reports each decision’s persisted calibration timestamp and current age. Decisions do not expire by age. A timestamp later than the inspecting system clock is invalid evidence, so cache loading and inspection fail closed with clock and recalibration guidance. Recalibrate after driver, firmware, power-policy, or material workload changes even when the stored identity still parses as compatible.

keyhog config --effective prints the resolved scan settings. Pair it with keyhog backend --autoroute --json to verify that a routing-relevant setting change produced a new config_digest row.

A lookup first tries the complete workload key. Size, chunk-count, and maximum-file dimensions use one-power-of-two logarithmic ranges. A decision proves correctness and timing for the representative measured under that key. It does not prove that the same backend is fastest for every individual byte length inside the numeric range.

A size band nobody measured is served only by measured invariance. KeyHog collects every calibrated decision that shares this workload’s pattern band, decode state, and source-class set. Two such bands are the minimum: one band says nothing about whether the winner depends on size. Their measurements are then reconciled by the rule that reconciles the repeated points inside a single band. The served backend is the lowest-complexity backend measured at every one of those bands and proved slower at none. Bands that agree on the backend and disagree on the phase-2 localizer plan resolve to the compiled default plan, which each of them must have measured. A band whose own evidence resolves no route, a backend crossover where one band proves a peer faster and another proves the reverse, and any GPU route all withdraw the reuse. Nothing is benchmarked, guessed, or substituted at scan time; the served route is one calibration measured, repeatedly, for this exact class.

GPU routes are never reused for an unmeasured band. GPU correctness, not only GPU speed, varies with input size: batch input caps and per-slot capacities bind to the measured shape, and a parity receipt proves that shape and no other.

When neither an exact key nor an invariant family covers the batch, a normal scan selects no backend for it, leaves it unscanned, records incomplete coverage, and exits nonzero with recalibration guidance. Calibration and explicit backend contracts also fail when their requested evidence or route cannot be produced.

Large directory and multi-source scans run in process and produce multiple real batches. The core calibration command includes file-tree probes, while Git, Docker, and web fixtures require installer calibration.

One-shot scans and the daemon

Runtime lifetime changes accelerator cost, so it is part of routing semantics. Calibration records the scalar CPU distribution directly. For SIMD and each GPU peer it records the real first materialization/dispatch followed by warm trials:

Every candidate contributes exactly seven positive trial durations. Route comparisons pair the same rounds. Missing, extra, zero, or unpaired trials invalidate the decision instead of being trimmed or substituted.

• An in-process one-shot scan includes cold Hyperscan or GPU cost when choosing a backend.
• A ready daemon initializes accelerator state before accepting requests and chooses from warm accelerator trials. Startup derives its required peer set from the validated decision table. It does not warm unrelated eligible peers, and it refuses readiness if any selected peer cannot be warmed.
• keyhog watch is also a compile-once persistent runtime. It warms every selected route before announcing readiness and uses warm evidence for later file events; it does not repeatedly price the same cold backend startup.

Decoded derived buffers are part of the measured route rather than a hidden runtime choice. Scalar and SIMD candidates keep their own backend for decoded rescans. GPU candidates explicitly compose with scalar for those small buffers, so neither scalar nor GPU timing can silently borrow Hyperscan work.

The current in-process router applies that cold-aware decision to each workload lookup. It does not infer request-wide GPU startup amortization across a large number of batches. This is why the cache and inspection output describe a measured workload key and runtime class rather than promising one universal crossover size.

Both routes consume the same parity-checked primary evidence; they derive the appropriate decision for their runtime instead of sharing one misleading “GPU time.” keyhog backend --autoroute prints both routes. CPU, Hyperscan/SIMD, and GPU remain peers in both cases. See Daemon and warm scans for request eligibility, in-process retry policy, socket, and timeout semantics.

Calibration never accepts a candidate that needs recovery. During an ordinary automatic scan, an accelerated-backend fault warns and replays the same stable snapshot through the fastest remaining measured-correct peer. GPU recovery replays only exact unprocessed ranges and retains completed GPU shards. Recovered work is counted separately, the affected workload route is quarantined, and the backend fault is written to a bounded <cache>.runtime-health.json artifact. Runtime health is separate from immutable timing evidence and survives restart. A successful calibration commit clears only the workload identities remeasured in that command. Missing health state means no runtime fault has been observed; malformed, oversized, unknown-backend, or calibration-inconsistent health state invalidates automatic routing. No backend is selected for an affected batch; it remains unscanned and receives repair guidance. An explicit GPU override or --require-gpu remains a hard backend contract and is not substituted. keyhog backend --autoroute reports quarantined readiness, aggregate and per-config fault counts, and the failed backend/reason on each affected workload; keyhog doctor reports the same repair state.

Diagnose invalid state and authenticated recovery receipts

Capture a metadata-bearing report, then inspect routing health:

keyhog scan . --format json-envelope --output keyhog.json
jq '{scan_status, backend_recoveries: (.metadata.backend_recoveries // [])}' keyhog.json
keyhog backend --autoroute
keyhog doctor

When automatic route state is unusable, the scan warning names the missing workload bucket and the dimensions that differ from the nearest measured class. No backend is selected, the affected batch remains unscanned, and the report uses scan_status: "partial" with a coverage gap. metadata.backend_recoveries contains only completed recovery from a faulting authenticated backend; invalid route state never creates a scalar recovery receipt.

Use the reported state to choose the repair:

• For one uncovered core workload, rerun the same scan once with --autoroute-calibrate --autoroute-gpu. This measures its actual source, resolved configuration, and workload class.
• For a standard preset ladder, run keyhog calibrate-autoroute. Add --policy default, fast, deep, or precision for a focused repair.
• For Git, Docker, or web source classes, run the exact repair_command reported for that source.
• For stale, recalibrate with the new binary after an upgrade.
• For quarantined, repair the named SIMD or GPU runtime first. Use keyhog backend --self-test --require-gpu for a GPU path, then recalibrate so the exact route fault can clear.
• For disabled or a storage error, fix the cache path or permissions and run the repair_command shown by JSON inspection.

An explicit diagnostic such as keyhog scan . --backend simd bypasses autoroute. It does not clear an invalid state. It is a hard contract and fails if that backend cannot initialize or execute. It is never substituted with another backend.

Inspect what is calibrated

keyhog backend --autoroute          # concise human-readable summary
keyhog backend --autoroute --verbose # every workload receipt
keyhog backend --autoroute --json    # machine-readable
keyhog backend --autoroute --autoroute-cache /absolute/custom/autoroute.json
keyhog doctor                       # reports the same readiness and repair action

The inspection command is also a health gate. A single-backend build reports health: direct and exits 0 even when its unused cache is absent or stale. For a multi-backend build, health: ready exits 0; quarantined, calibration_required, disabled, stale, and invalid exit 4 so automation cannot mistake an unusable autoroute state for a healthy host. JSON includes the same health value plus repair_command: null for direct or ready, the canonical calibration command for quarantined, absent, stale, or invalid evidence, and an explicit cache-path command when persistence is disabled. Scan reports expose recovered chunks, ranges, and bytes only after a fault in an authenticated selected backend. Invalid route state records an unscanned coverage gap; inspection remains unhealthy until calibration produces confidence-separated evidence.

Pass --autoroute-cache when the scan uses a non-default cache path through the matching scan flag or [system].autoroute_cache.

These show every persisted config and host generation, its workload buckets, representative route times, whether confidence was separated, the selection basis, and the resolved one-shot and daemon backends. The JSON view is lossless: each route includes its ordered nanosecond trials, cold observation, exact one-shot and warm projections, and 95 percent confidence bounds, so the result can be reproduced without parsing the private cache file. Each generation’s eligible_backends array defines the complete backend set. Every decision must contain all four localization plans for every eligible backend and every eligible resident depth for each GPU peer, and prove each route correct. Removing a candidate timing and its receipt together still invalidates the cache because validation compares the full Cartesian route set with this live config identity. The inspection shows exactly what is covered and how each existing decision was made. An invalid decision makes the inspection report the cache as unusable; inspection never omits a malformed row and presents the remainder as healthy.

Inspection validates build compatibility and the complete persisted cache structure. It does not have the live scan’s host, detector, rule, and resolved config inputs; those identities are checked when a real scan loads its decision. Therefore a readable, build-matched inspection is evidence that the cache can be examined, not a guarantee that the next workload has a usable row.

The top-level calibration_required field is true only when this build has multiple compiled scan backends. When false, direct_backend names the only possible route and a disabled or absent cache does not make automatic scans unhealthy. inspected_at_unix_ms is the clock value used for timestamp validation and age derivation. The per-decision JSON fields have these exact meanings:

Single-backend builds

A build that compiled only one backend has nothing to route. The portable build, for example, ships only the scalar CPU backend, so it skips autoroute entirely and never reports calibration required. Calibration applies only to builds that compiled a real backend choice (Hyperscan/SIMD and/or GPU).

How detection works

A KeyHog scan is a pipeline. Files come in one side, findings go out the other. In between, four stages:

files → [chunker] → [prefilter] → [detector match] → [post-process] → findings

Most chunks that fail the cheap prefilter stop there, which keeps full regex evaluation focused on plausible inputs. This is not an unconditional hard drop: a rejected chunk that looks encoded can enter a bounded decode-only recovery pass (recursively decoding up to a max_decode_depth, defaulting to 10), so an encoded secret is not lost merely because its plaintext anchor is absent from the original bytes.

Detection mechanisms

KeyHog does not use one universal test for “secret-like.” It composes several mechanisms, and their roles are deliberately different:

Regex, generic extraction, entropy, and decode-through therefore find different candidate classes. Named regexes and generic assignment extraction create candidates; companions, validators, BPE, English bigram evidence, shape/context policy, and confidence then confirm, reject, or score them. Verification runs only after a candidate survives detection and reporting policy.

Structured extraction accepts balanced Helm actions as render-time syntax. It replaces each action with an inert YAML value while retaining every literal source byte. A Jupyter notebook truncated at end of file is repaired only by closing its open string and container delimiters. Any other syntax error remains a counted coverage gap.

ML participation is detector-owned through [detector.ml]. lift can raise a structural score but cannot veto a match, blend combines model and structural evidence, and authoritative lets the model decide an otherwise ambiguous channel. In the shipped corpus every regex-pattern channel is currently lift; 929 of 934 entropy channels disable ML. The five that do not are all generic owners: generic-api-key, generic-high-entropy-string, generic-keyword-secret, and generic-secret use authoritative, and generic-password uses lift. The model is therefore not a general regex false-positive veto. Its current feature record identifies detector and pattern-versus-entropy channel, but not the exact matched pattern. Pattern-local weak-anchor policy remains outside model conditioning until that provenance is carried end to end. The retrainer refuses mismatched detector/channel records and now requires positive and negative held-out support for every blend or authoritative channel before writing another model.

Before model inference, a cheap probabilistic gate may assign 0.1 confidence to an unaccompanied generic candidate that lacks secret-like randomness. It cannot short-circuit a service-regex match or a candidate backed by a matched companion, including a weakly anchored service pattern: those candidates continue into their detector-owned ML mode. This distinction prevents a corpus-wide randomness shortcut from overriding stronger detector-local evidence.

Static program recovery is a decode mechanism, not arbitrary code execution. KeyHog does not invoke Node.js or evaluate source. It recognizes a bounded grammar for cyclic byte-array XOR and Node-style AES-256-CBC decryption, resolves only literal numeric arrays, Base64-encoded JSON arrays, buffer literals, and empty-separator string joins, then checks binding consistency, UTF-8, AES block shape, and PKCS#7 padding before rescanning the recovered plaintext. Recovered XOR calls and Node AES ciphertext bindings are spliced back into bounded parent context, preserving assignment evidence and absolute source offsets. The CryptoJS dialect additionally requires an exact immutable require("crypto-js") alias, decrypt wrapper, literal passphrase and ciphertext bindings, an OpenSSL Salted__ envelope, and EVP_BytesToKey MD5 derivation. Dynamic values or unsupported syntax produce no derived candidate. The original source still follows the normal detector pipeline. The mechanism is disabled with decode recursion, including under --fast.

BPE is not a replacement name for entropy: it is an independent post-candidate signal. BetterLeaks calls the approach Token Efficiency; KeyHog uses the same broad BPE idea while keeping its own detector schema, thresholds, pipeline, and behavioral evidence.

Terminology matters here: BetterLeaks’ public documentation names the feature Token Efficiency and describes BPE tokenization as a natural-language false positive filter; it does not present “BPD” as a separate score. KeyHog names its related mechanism BPE token efficiency, uses cl100k_base, measures UTF-8 bytes per token, and resolves the ceiling per detector. If “BPD” is being used informally to mean a bits/byte or bytes/token density, do not treat it as a third byte-density score. KeyHog also uses a separate English letter-bigram discriminator, but it does not measure bits per byte or bytes per token.

The English bigram discriminator evaluates lowercase ASCII alphabetic runs against an embedded 26 by 26 log-probability model. Digits and symbols end a run. Fewer than six alphabetic characters produce no randomness verdict. A random-token verdict requires a mean score at or below -6.85 and at least three distinct letters. Depending on the calling gate, that evidence can keep an otherwise identifier-shaped random credential or reject a confidently English placeholder. The model does not model arbitrary bytes, numeric keys, hexadecimal or Base64 alphabets as such, or short values. Its current thresholds are scanner-wide constants, not detector TOML fields.

Detector-owned tuning: what each setting changes

Detection policy belongs in the detector TOML whenever the choice is specific to a credential type. Scan-wide CLI/TOML values are operational overrides for controlled comparisons or a corpus-wide policy; they are not a second hidden detector definition. keyhog explain <detector-id> shows the policy declared by that detector TOML and its provenance; keyhog config --effective shows the resolved scan-wide policy.

Practical ownership rule: any numeric value that changes one secret family’s recall, precision, shape admission, or scoring must be a named detector-TOML field. If the schema cannot express it, extend the typed schema and its explain/contract surfaces rather than adding a detector-specific literal in scanner code. Only true shared invariants, such as parser safety caps or a model’s fixed vocabulary, remain global.

Global structural non-secrets are typed Tier-B data in rules/entropy-universal-rejections.toml. Plain prefixes and explicit prefix-plus-length rules, including the Ag Sealed Secrets ciphertext boundary, apply uniformly before detector plausibility. They are corpus-global structural invariants, not a second source of per-detector tuning.

Resolution rules

These settings do not all use one generic “last value wins” rule:

• Generic keyword ownership: the highest entropy_policy_priority among detectors claiming the normalized assignment keyword owns entropy and BPE policy. Equal priorities use stable detector identity, independent of corpus order. Custom detector policy keywords join entropy discovery directly; they do not need to be repeated in [scan].secret_keywords.
• Final match resolution: the active compiled plan classifies named, phase-2 generic, entropy, and enclosing private-key findings. The reporting service string and detector-ID length do not change specificity. Unknown finding identities fail checked resolution instead of inheriting embedded or service-name behavior.
• Entropy entry roles: entropy_roles selects the detector that owns each corpus-level entry path. A compiled corpus may have at most one owner for each role. Missing roles remain disabled, and duplicate owners fail scanner construction. Role selection never depends on a detector ID spelling.
• Weak anchors: detector-level weak_anchor = true applies to every pattern, while the same field inside [[detector.patterns]] governs that exact regex. Each such detector owns entropy_high and length-bucketed entropy_floor. Scanner construction rejects an explicit weak anchor without that local policy. KeyHog never guesses this semantic choice from regex text, and min_confidence does not disable it. The compiled hot path uses a primitive detector-indexed floor program.
• BPE ceiling: every active entropy owner declares either bpe_max_bytes_per_token or bpe_enabled = false; omission fails detector validation and scanner construction. An explicitly supplied [scan].entropy_bpe_max_bytes_per_token or --entropy-bpe-max-bytes-per-token replaces every BPE-enabled entropy/generic detector ceiling; the CLI wins over the config file. bpe_enabled = false still disables the gate for that detector.
• Confidence floor: the scan floor defaults to 0.40. A detector TOML min_confidence replaces the scan floor for that detector, and an operator [detector.<id>].min_confidence replaces the detector-declared floor. Under --precision, the resolved global and per-detector floors are clamped to at least 0.85; neither source can weaken the precision preset.
• Entropy policy: every active entropy owner must declare its high, low, very-high, sensitive-path, mixed-alphanumeric, symbolic, tail-entropy, length, isolated-shape, and BPE policy. Scanner construction compiles these into concrete detector-indexed values; a missing field is an error, not a runtime default. Schema defaults remain only for non-owning programmatic detector values that never supply entropy policy. The scan-wide entropy_threshold is deliberately not a blanket replacement for all four bands. On the phase-2 generic bridge it tightens only when it exceeds the owning detector’s high band. On the entropy scanner, a value above that high band tightens keyword and isolated candidates; a value below the keyword detector’s low band loosens that keyword path, while values between the low and high bands leave its low floor in place. The isolated path keeps its mixed-alphanumeric floor unless the scan threshold exceeds the high band. Named-detector heuristic confidence uses the resolved scan threshold as its partial entropy tier and the scoring margin above it as its full tier; changing the setting can therefore change a named finding’s confidence without changing whether its regex matched. These rules preserve the different evidence carried by an assignment key, an isolated opaque token, and an unanchored generic value. The owning entropy_high and entropy_very_high values also define the partial and full heuristic-confidence tiers for emitted entropy fallbacks. Detector ML policy then composes with that heuristic; an authoritative ML mode may replace it, while disabled, lift, and blend modes retain the documented heuristic semantics.
• Sensitive paths: sensitive_path_entropy_very_high is a required detector-local threshold for active entropy owners. Equaling entropy_very_high means no sensitive-path relaxation; a lower declared value is an explicit detector-owned recall choice.
• Credential plausibility: the required detector plausibility block owns its entropy floors, length carve-out, alphabetic admission, and repeated, identifier, and dash-segment rejection choices. There is no production-path fallback for an active entropy owner.
• Synthetic entropy identity: every active entropy owner declares [detector.entropy_fallback] with a semantic class (generic, password, token, or api-key), an entropy-* id, display name, and service. The compiled scanner uses the complete metadata from the active detector corpus for entropy-only findings. Omitting the block is a visible compile error; no scanner-global keyword classifier or compatibility identity can relabel a custom candidate.
• Isolated entropy shapes: generic entropy owners declare one data-driven shape with its character set, entropy floor, optional fixed-width grouping, diversity requirements, and special minimum length. For the shipped lower-alphanumeric app-password policy, candidate length is derived from four groups of four plus three separators; special_min_length controls the short-candidate revisit and must not exceed that derived length. The shape is used for anchorless synthetic entropy recovery; the anchored bluesky-app-password regex remains the source of the named Bluesky finding. A custom corpus without the shape has no isolated exception, rather than inheriting an embedded detector policy.
• Isolated symbolic credentials: the detector’s plausibility.isolated_symbolic_min_len, plausibility.isolated_symbolic_min_symbols, and plausibility.isolated_symbolic_requires_non_underscore fields control the shorter symbol-rich exception. Contiguous and underscore-delimited mixed tokens stay under plausibility.isolated_mixed_entropy_floor when the owner requires a non-underscore symbol, and an exact declared lower-dash layout must satisfy its entropy_shapes policy instead of bypassing it as symbolic.

Token efficiency can carry more of the precision burden for a detector whose assignment key or regex already creates the candidate. That is the practical per-detector alternative to making Shannon entropy the decisive signal: use a permissive detector-owned entropy floor appropriate to the credential family, then let its BPE, shape, context, and confidence policy reject word-like noise. It is not equivalent to blindly replacing entropy with one global BPE number, and bpe_enabled alone never creates a candidate. Both configured gates still execute; the current pipeline has no entropy-or-BPE branch.

Detector-owned canonical_hex_key_material is the deliberate exception to the BPE and generic low-diversity/decode-as-data gates. Hexadecimal key bytes tokenize efficiently and use a small alphabet for the same mechanical reasons hexadecimal digests do, so the exact detector-owned contract supplies the discriminator: assignment scope plus length for generic detectors, or matched regex plus length for named detectors. Placeholder, degenerate-repeat, entropy, context, and reporting gates remain active. When ML is enabled, this exact TOML match is structural positive evidence and therefore preserves the detector heuristic floor; the model may raise its score but cannot erase a policy-proven key as if it were an unowned entropy candidate.

Scan-wide settings remain operational controls, but they do not all compose the same way. The operator-layer order and working TOML/CLI examples are in Configuration. Stable per-detector tuning belongs in the owning detector TOML and should be proved with that detector’s positive, negative, evasion, backend-parity, and corpus contracts.

Settings, active corpus, and exact identity

KeyHog keeps detector content, resolved scan policy, and corpus provenance separate:

• The reported detector corpus digest binds the normalized corpus schema and the active detector specifications after composition and [detector.<id>] enabled = false removal. A matching disable therefore changes the digest. An unknown disabled ID warns and leaves this digest unchanged.
• The autoroute rules identity also describes the active detector specifications. Operator confidence-floor overrides are composed later so different scan presets can coexist in one calibration cache. It is the canonical corpus identity an execution pack carries, so a scan that compiles the corpus and a scan that hydrates an installed generation of that same corpus read the same calibrated table. Self-test fixtures and declaration order are excluded.
• The autoroute configuration identity binds the resolved scanner and operator policy. It includes the selected fast, deep, or precision preset, scan-wide and per-detector floors, the configured disabled-ID set, detector tuning inputs, worker and pipeline settings, backend/GPU policy, and profiling instrumentation.
• The corpus path and embedded, replace, or overlay label are provenance. They are reported in versioned output, but the path spelling is not detector content. Copying the same normalized corpus to another directory does not create a different content digest. A mode change changes the digest only when it changes the resulting active specifications.

The preset definitions and their override rules are in Configuration. --profile is performance instrumentation, not a named scan-policy profile.

Hardware changes execution, not detection policy. CPU, SIMD/Hyperscan, and GPU routes consume the same resolved detector and configuration identities. Autoroute accepts a candidate only when its canonical detection identities match the reference: chunk membership, detector id/name/service/severity, exact credential, stored hash, companion identity, source, file, line, byte offset, commit, author, date, entropy, confidence, and multiplicity. Mismatch diagnostics name only the differing fields and occurrence counts. They never expose raw values or deterministic value fingerprints.

Built-in suppression, confidence, decode, and scanner postprocessing are already part of those backend results. CLI allowlists and rules, policy floors, cross-source deduplication, verification, and output formatting run after selection. Missing or stale exact evidence is an error. Calibration never relaxes a detector to make a backend look faster.

Two kinds of change exist, and keeping them apart is the whole point of the parity model. A policy change is allowed to change findings. An execution change is not. A finding-set difference across an execution change is a parity failure, which KeyHog treats as a defect rather than a result.

Policy changes: findings may change

Execution changes: findings must not change

A difference in the first table is a decision you made. A difference in the second table is a bug. Report it with the effective config, detector digest, input identity, backend, host identity, and the complete finding sets from both runs.

Strict Backend Parity

KeyHog exposes three search-backend classes: pure Rust CPU, SIMD/Hyperscan (simd-regex), and GPU/VYRE region presence. Autoroute measures five concrete runtime peers when eligible: scalar CPU, Hyperscan CPU, CUDA, native Metal, and WGPU. Portable builds retain the pure-Rust trigger path without Hyperscan. keyhog calibrate-autoroute rejects any peer whose canonical match identity differs from the reference. It records the first real GPU dispatch plus warm trials: an ordinary process resolves against the cold-aware GPU cost, while a daemon that initialized its engines before readiness resolves against the warm GPU evidence. A missing or invalid decision is not autoroute evidence: KeyHog warns, selects no backend for the affected batch, records incomplete coverage, and prints the exact repair command.

When comparing settings, record the effective config, detector digest, input identity, backend, host/accelerator identity, and complete findings, not only elapsed time or finding count. A faster run with a different result set is a detection change or parity failure, not a routing win.

Stage 1 - chunker

A file becomes one or more chunks. A chunk is {data: str, metadata: {source_type, path, line_offsets, …}}. The chunker:

• Skips obvious binaries via magic-byte sniffing (PDF, PNG, zip, …).
• Skips files matching is_default_excluded_path (node_modules, .min.js, build/, etc.).
• Splits files larger than the 1 MiB window size into overlapping ~1 MiB windows so a single giant log file doesn’t blow scratch memory. Each window carries its absolute base byte offset and base line so findings report the real file offset/line, not the per-window one. Cross-window secrets are reassembled in stage 4.
• Decodes UTF-16 BOM files into UTF-8 (PowerShell / .NET configs).

Specialized chunkers run too:

• Git history → one chunk per (commit × file × diff line)
• Docker images → one chunk per layer × file
• Web URLs → one chunk per response body / sourcemap / WASM strings
• S3 buckets → one chunk per object body
• GCS buckets → one chunk per object body
• Azure Blob containers → one chunk per blob body

Stage 2 - prefilter (the cheap pass)

Three gates, in order, each cheaper than the next:

1. Alphabet screen. A 256-bit mask of which bytes the corpus’s detectors care about. A chunk with no relevant byte becomes a prefilter miss.

2. Bigram bloom. A 4096-bit bloom filter of 2-byte sequences from detector keyword prefixes. A chunk with no overlapping bigram becomes a prefilter miss. This cheaply recognizes source that carries no relevant anchor vocabulary.

After these screens, ordinary misses stop. Decode-shaped misses instead take the bounded decode-only path described above; transformed plaintext is then attributed back to the original source.

3. Backend trigger pass. The simd-regex backend compiles the detector corpus into Hyperscan databases when the simd feature is present; cpu-fallback uses the pure-Rust trigger path. One pass returns which detector IDs have a candidate match.

   GPU-capable builds add VYRE’s resident fused literal-evidence backend. Its single dispatch returns region presence plus detector-derived localization positions; the shared host regexes still decide every finding. There is no universal model-name or byte threshold at which KeyHog silently switches to it. --backend auto requires an exact persisted calibration decision for the current binary, detector/config digest, host/device/driver, workload class, and size bucket. Calibration keeps a GPU route only when its canonical match identities equal the reference and it is the fastest eligible backend for that key.

Stage 3 - detector match

For each pattern-backed detector that the prefilter flagged, the full regex evaluates. The regex is detector.patterns[].regex in that detector’s TOML, and its configured capture group becomes the candidate credential. Generic phase-2 detector TOMLs use keyword, length, entropy, token-efficiency, and shape policy for shapeless assignments or isolated opaque values. They may also carry explicit patterns for strongly structured envelopes such as JSON "secret", "token", or "apiKey" fields; both mechanisms remain owned by the same detector TOML instead of a central compatibility detector.

A detector’s .toml carries:

• id, name, service, severity, keywords
• zero or more patterns, each with regex + group + optional description (required for service-anchored detectors; optional structured-envelope anchors for phase2-generic)
• optional typed companions; required entries gate acceptance, reinforcing entries add evidence, and forbidden entries suppress
• optional bounded detector_relations with requires, conflicts, or subsumes semantics across findings in the same source, file, and revision
• optional verify block: HTTP method, URL template, auth scheme, success status

Detectors fall into two camps:

• Service-anchored. Regex requires a service-specific keyword (AWS_SECRET_ACCESS_KEY=, stripe.com/v1/, dn_ Deepnote prefix). These have HIGH precision: the keyword itself is positive evidence, not just a hint.

• Generic / entropy discovery (generic-password, entropy-api-key, entropy-token). Triggered by entropy + assignment shape only - password = "...", secret: "...", JSON { "token": "..." }. Lower precision; suppression filters do most of the work.

  Surviving candidates also pass a BPE token-efficiency gate. Shannon entropy asks how evenly bytes are distributed; token efficiency asks how readily a fixed subword vocabulary compresses the value. Dotted API names and prose can have high Shannon entropy but tokenize into a few common pieces, while opaque secrets usually require many short tokens. The mechanisms are complementary, and generic detector TOMLs may own their token-efficiency ceiling through bpe_max_bytes_per_token. Opaque API-key/secret policies use their detector-owned ceiling, falling back to the scan-wide default of 2.2 UTF-8 bytes per token when they do not declare one; password/passphrase policies set bpe_enabled = false because human-chosen credentials may intentionally be word-like. Disabled policies skip tokenizer work entirely rather than using a magic oversized ceiling.

The entropy-generic, entropy-password, entropy-token, and entropy-api-key IDs are output classifications for entropy-discovered findings, not four additional detector TOML files. Their candidate policy is owned by the corresponding phase-2 TOMLs selected from the assignment context: generic-secret, generic-password, generic-keyword-secret, or generic-api-key. Use keyhog explain on those owning detector IDs when tuning entropy, BPE, length, or canonical-key policy.

The split matters for the post-process stage.

Stage 4 - post-process

Even a regex match isn’t always a credential. Stage 4 filters:

• Known example fixtures (Stripe docs key, AWS docs key, RFC 7519 specimen JWT).
• Placeholder language - credentials containing YOUR_, INSERT, EXAMPLE, PLACEHOLDER, TODO, FIXME, etc.
• Shape gates.
  • Universal: punctuation_decorated_identifier - credentials starting with --, &, @, !, /, $ (CLI flags, pointers, SQL vars, shell vars, GraphQL refs) or ending in : / ! (UI labels, TypeScript non-null assertions).
  • Generic / entropy only: pure_identifier, word_separated_identifier, scheme_prefixed_uri, url_or_path_segment, contains_uuid_v4_substring. These shapes can be real credentials when paired with a service or protocol anchor, so named detector TOMLs and structural authorization detectors own those cases. A generic token=<uuid> remains an identifier; an Authorization: Bearer <uuid> value is a credential because the Bearer envelope supplies the missing evidence. Public salts and nonces are not generic secrets. A detector for a product whose field is genuinely secret despite that name must own the product syntax explicitly.
• Path-based suppressions - vendored bundles (node_modules/, wp-includes/, bower_components/), CI workflow files (where ${{ secrets.NAME }} references are syntactic, not credentials), i18n translation files, secret-scanner source files (the file IS a scanner; its regex literals shouldn’t fire on itself).
• Cross-chunk reassembly. A secret split across window boundaries gets reassembled from the tail of chunk N + the head of chunk N+1.

A finding that survives stage 4 makes it to output.

Semantic source roles and structured parsing

KeyHog analyzes the syntactic role of matched text to distinguish genuine credential assignments from comments, documentation, and mock data.

Each candidate is classified into a SemanticSourceRole:

Structured parser scoping and abstention

Structured configuration parsers (dotenv, JSON, YAML, TOML) enforce strict file-extension scoping:

1. Extension-matching paths: Files with recognized extensions (.env, .json, .yaml, .yml, .toml) parse according to their format grammar.
2. Non-matching extensions: Files with non-matching extensions (for example config.unknown or data.txt) abstain to SemanticSourceRole::Unknown. They do not guess format syntax from arbitrary file extensions.
3. Unnamed memory buffers: Unnamed streams (path: None), such as standard input or in-memory chunk buffers, use content-based structural sniffing to identify dotenv or JSON payloads.

Decoded sub-chunk semantic scoping

When an input contains encoded strings (such as Base64, Hexadecimal, or URL-encoded payloads), KeyHog’s decode-through engine extracts the decoded bytes into a sub-chunk.

A decoded sub-chunk clears its inherited file path to None during semantic indexing. This ensures that the decoded payload is parsed based on its own syntactic structure rather than inheriting the outer file’s extension. For example, a Base64-encoded JSON object inside a .txt file is parsed as structured JSON rather than plain text.

A credential found in both the container bytes and the decoded payload is reported once, at the coordinate in the file you can open. Its evidence is the stronger of the two. A Kubernetes Secret whose base64 data: value decodes to AWS_ACCESS_KEY_ID=... therefore reports likely with the assignment role the decoded text proves, at the offset of the encoded value.

Pattern provenance and secret-safe evidence

Every finding emitted by KeyHog carries a structured provenance record inside its evidence block. This metadata identifies the exact pattern and context that produced the match without disclosing secret material:

{
  "schema_version": 1,
  "detector_digest": "0123456789abcdef",
  "pattern_index": 0,
  "candidate_channel": "pattern",
  "source_role": "environment-assignment-value",
  "context_class": "vendor-pattern"
}

Provenance fields

• schema_version: Version of the provenance schema (currently 1).
• detector_digest: 16-character lowercase hexadecimal hash of the active compiled detector specification.
• pattern_index: 0-indexed ordinal of the matched regex pattern in the detector TOML, or null for entropy-discovered candidates.
• candidate_channel: Pipeline stage that generated the candidate: pattern (regex match), entropy (entropy discovery), companion (companion match), static-recovery (bounded JavaScript XOR/AES evaluation), or unattributed.
• source_role: The SemanticSourceRole where the match was located.
• context_class: Surrounding context category (vendor-pattern, weak-anchor, generic-assignment, standalone-token, or unsupported-context).

Provenance records are deterministic, portable, and safe to share in public CI logs and triage envelopes.

Where the speed comes from

The alphabet screen and bigram bloom reject irrelevant chunks before regex confirmation. Literal triggers narrow the active detector set, and the scanner shares confirmation, suppression, and reporting tails across CPU and GPU backends. Windowing bounds scratch space for large inputs; caches avoid repeated compiler and index work.

End-to-end throughput depends on the detector/config digest, source shape, candidate density, decoding and verification policy, cache state, CPU, GPU, driver, and storage. Use keyhog calibrate-autoroute for routing evidence on the installed host and the repository benchmark harness for reproducible cross-version measurements; do not treat a throughput number from another machine or detector corpus as a routing threshold.

Where the precision comes from

Each filter has a known-FP-cluster it was built to defuse. The Suppressions page enumerates them with examples.

What this looks like for one finding

file.env contains: AWS_SECRET_ACCESS_KEY=ev0BsFtSD7S/4VWYObxiEhME3hJBXeYzR43jgiB1

stage 1 - chunker:        emit chunk{ path: "file.env", data: "AWS_SECRET..." }
stage 2 - alphabet:       PASS (chunk has `=`, alphanumerics from the corpus)
stage 2 - bigram bloom:   PASS (`AW`, `WS`, `_S` are in the bloom)
stage 2 - simd-regex:     MATCH → triggers `aws-secret-access-key` + `generic-password`
stage 3 - regex eval:
  `aws-secret-access-key` detector pattern captures the 40-byte value
    captures `ev0BsFtSD7S/4VWYObxiEhME3hJBXeYzR43jgiB1`
  `generic-password` regex doesn't match (no `_password`/`_pwd` substring)
stage 4 - post-process:
  known-example check: no
  `looks_like_pure_identifier`: false (has digits + /)
  `looks_like_punctuation_decorated_identifier`: false
  → EMIT

That’s one finding’s life. Multiply by 10⁶ files and the throughput math is why each stage matters.

Detectors

A detector is a single TOML file that teaches KeyHog one shape of credential. The embedded corpus is generated from detectors/*.toml; query the running binary for its exact corpus size rather than relying on a number copied into documentation.

Pattern counts

KeyHog counts detectors and patterns separately. A detector is one TOML file; each file may define one or more [[detector.patterns]] rows. The startup banner’s parenthesized pattern total is the compiled scanner count after the engine expands those rows (and related trigger keywords) into the literal and regex slots it actually runs, so it is always larger than the raw TOML row count. Use keyhog detectors --format json | jq length for the embedded detector count; the banner line shows the live compiled total for your binary.

Anatomy of a detector

# detectors/stripe-secret-key.toml

[detector]
id = "stripe-secret-key"
name = "Stripe Secret Key"
service = "stripe"
severity = "critical"
ml = { match_mode = "lift", entropy_mode = "disabled", weight = 1.0, context_radius_lines = 5 }
match_confidence = { literal_prefix_weight = 0.35, context_anchor_weight = 0.20, entropy_weight = 0.20, high_entropy_partial_weight = 0.12, moderate_entropy_threshold = 3.0, moderate_entropy_weight = 0.05, low_entropy_penalty_floor = 2.0, low_entropy_min_match_length = 10, low_entropy_penalty_multiplier = 0.60, keyword_nearby_weight = 0.10, sensitive_file_weight = 0.10, companion_weight = 0.05, very_high_entropy_margin = 1.2999999999999998, named_anchor_floor = 0.55, assignment_context_multiplier = 1.0, string_literal_context_multiplier = 0.9, unknown_context_multiplier = 0.8, documentation_context_multiplier = 0.3, comment_context_multiplier = 0.4, test_context_multiplier = 0.3, encrypted_context_multiplier = 0.05, soft_context_suppression_threshold = 0.5, encrypted_context_suppression_threshold = 0.8, post_match = { placeholder_multiplier = 0.05, minimum_byte_diversity = 0.1, low_diversity_multiplier = 0.1, maximum_repeat_ratio = 0.8, degenerate_run_min_length = 10, degenerate_repeat_multiplier = 0.1, fixture_path_multiplier = 0.5, ml_context_reapply_below = 0.95 } }
validators = [{ type = "pattern-shape", prefixes = ["sk_live_", "sk_test_", "rk_live_", "rk_test_"], allow_overlong = false }]
keywords = ["sk_live_", "sk_test_", "rk_live_", "rk_test_", "stripe"]
simdsieve_prefixes = ["sk_live_", "sk_test_", "rk_live_", "rk_test_"]

[[detector.patterns]]
regex = 'sk_live_[a-zA-Z0-9]{24,}'
description = "Stripe live secret key"

[[detector.patterns]]
regex = 'sk_test_[a-zA-Z0-9]{24,}'
description = "Stripe test secret key"


[detector.verify]
method = "GET"
url = "https://api.stripe.com/v1/charges?limit=1"
allowed_domains = ["api.stripe.com"]

[detector.verify.auth]
type = "basic"
username = "match"
password = ""

[detector.verify.success]
status = 200
policy = "status_with_error_backstop"

That’s the whole contract for one service. Every other detector follows the same shape.

Verification success and metadata json_path fields use the single rooted response-selector grammar documented in Verification.

Each shipped detector owns at least one canonical positive/negative truth pair:

[[detector.tests]]
pattern_index = 0
test_positive = "SERVICE_PUBLIC_ID=example-production-fixture"
test_negative = "SERVICE_PUBLIC_ID=short"
negative_class = "boundary"

[[detector.tests]]
pattern_index = 1
test_positive = "SERVICE_LEGACY_ID=example-legacy-fixture"
test_negative = "SERVICE_LEGACY_ID=short"
negative_class = "boundary"

Detector test records are executable production-path fixtures, not prose examples. pattern_index is the zero-based pattern ordinal. negative_class is one of boundary, identifier, prose, regex-literal, or sibling-prefix. The positive must surface that exact detector id and the negative must leave that detector silent. An enforcement-capable policy requires one indexed positive and named negative per pattern. Compatibility-mode detectors retain their detector-level pairs; the deterministic corpus gate supplies an exact regex witness for every pattern. Larger adversarial, evasion, performance, and scale corpora remain separate because one compact pair cannot prove those contracts.

Fields

detector.id - kebab-case, globally unique. Shows up in JSON output as detector_id and in CLI output as the third column.

detector.kind - optional execution class. Omission or "regex" selects the normal anchored-regex contract. "phase2-generic" selects the shared generic discovery engine with every detector-specific decision owned by this TOML; it may have no patterns, but must declare the mechanism-specific fields validation requires. service is report taxonomy and never selects either execution class.

detector.ml - model policy compiled with the detector. match_mode controls regex and generic-assignment candidates; entropy_mode controls synthetic entropy candidates owned by this detector. Each mode is disabled, lift, blend, or authoritative. lift applies the declared fraction of positive model evidence without letting an uncalibrated model veto structural evidence; blend is the arithmetic mixture and can raise or lower confidence; authoritative uses the model score directly. weight controls lift and blend, and context_radius_lines is the bounded source window supplied to feature extraction. Normal scans use these detector values. An explicitly supplied --ml-weight is a scan-wide diagnostic or benchmark override. Detectors that do not own entropy policy set entropy_mode = "disabled". The model input also carries detector facts from this same TOML: exact service context, entropy-policy/phase-2 ownership, weak-anchor and structural-password-slot classification, verification, required companions, and the pattern or entropy candidate channel. Entropy candidates additionally carry a one-hot family read from this detector’s entropy_fallback.class, so API keys, passwords, tokens, and generic entropy do not collapse into one model input. The shared scorer does not infer a detector family from its id or apply one unconditioned probability to every secret type. Every detector TOML must declare detector.ml; omission fails parsing instead of silently applying an embedded or scanner-side model policy. Programmatic DetectorSpec::default() disables both model paths until the caller opts in.

detector.match_confidence is the complete pre-model scoring policy for regex matches. The six signal weights define the maximum normalized evidence. high_entropy_partial_weight, moderate_entropy_threshold, and moderate_entropy_weight define the lower entropy tiers. very_high_entropy_margin is added to the resolved operational entropy threshold for the full entropy weight. The shipped value is the exact binary64 difference between the historical 5.8 and 4.5 tiers. The low-entropy fields define the long-value penalty. The seven context multipliers define how assignment, string-literal, unknown, documentation, comment, test, and encrypted source contexts affect this detector before and after model scoring. The two context-suppression thresholds decide when a comment, test, documentation, or encrypted candidate is too weak to report. The nested post_match policy owns the placeholder, byte-diversity, repeated-run, decoded-envelope, fixture-path, and post-model context adjustments. data_envelope_multiplier is present only when decoded payload evidence applies to that detector. A named detector declares named_anchor_floor and omits low_promise_confidence. A phase-two generic owner does the reverse. This lets the cheap promise gate reject only unaccompanied generic candidates. Missing, misplaced, non-finite, or non-monotonic policy fails detector validation. KeyHog precomputes the normalization reciprocal once in the detector execution plan. DetectorSpec::default() leaves this policy unset, so a programmatic detector must declare it before scanner compilation.

detector.validators - optional typed offline validation programs compiled with this detector. crc32-base62 declares prefixes, entropy_len, checksum_len, reject_overlong, and the confidence floor earned by a valid checksum. github-fine-grained-crc32 declares both segment lengths and checksum width. base64-payload declares the exact base64 alphabet (standard, standard-no-pad, url-safe, or url-safe-no-pad) plus encoded and decoded length bounds, so the hot path performs one direct decode without guessing a dialect from candidate bytes. pattern-shape reuses this detector’s patterns as its structural contract and does not claim checksum proof or raise confidence. Prefixes, widths, bounds, and floors belong here, never in a scanner-side service table. Named matches dispatch directly through their compiled detector plan; generic candidates use a compiled first-byte prefix index. One verdict follows the candidate through suppression, ML batching, and final confidence, so validation is not repeated after inference.

detector.decode_transforms declares admission for asymmetric evasion recovery. Use plaintext prefixes. KeyHog compiles the reversed and rotated spellings once from the active corpus:

decode_transforms = { reverse_prefixes = ["dapi"], caesar_prefixes = ["dapi"] }

An empty list disables that transform for this detector. A custom corpus does not inherit prefixes from the embedded corpus. Reverse prefixes must contain at least three ASCII bytes. Caesar prefixes must contain an ASCII letter. These fields control only reverse and Caesar admission. Base64, hex, URL, JSON, Unicode, MIME, quoted-printable, and bounded static-program recovery use shared representation grammars because their eligibility is not specific to a secret type.

detector.name - human-readable name. Shows up in keyhog detectors listing and IDE plugins.

detector.service - the upstream service slug. Used for grouping findings (e.g. “you leaked 3 stripe credentials”); a single service can have multiple detectors (stripe-secret-key, stripe-restricted-key, stripe-publishable-key).

detector.simdsieve_prefixes - optional literal prefixes for the first-pass AVX-512/AVX2/NEON accelerator. This is detector-owned Tier-B policy: each value must be non-empty ASCII, unique in the loaded corpus, and must be an actual literal prefix of one of the same detector’s regex patterns. The loaded corpus may declare at most 16 total (the backend ABI limit); duplicate ownership, unbacked prefixes, and over-capacity corpora fail scanner construction instead of silently disabling acceleration. Most detectors leave this empty.

detector.severity - one of critical | high | medium | low | client-safe | info. The CLI exits non-zero when any finding clears the active gate; under --verify, confirmed live credentials escalate that outcome to exit 10. SARIF / GitHub Code Scanning surface severity prominently.

client-safe is the bug-bounty tier for keys public by design (Sentry DSN, Stripe pk_*, Mapbox pk., PostHog phc_, Firebase Web API key, Google Maps browser key, Mixpanel project token, Algolia search-only, Datadog browser RUM, Bugsnag, Segment write key). The detector still fires (a token grep is a token grep), but the finding renders below low and --hide-client-safe filters it out entirely. Set per-pattern via the client_safe = true field on a [[detector.patterns]] block - detectors that fire on both the public and the secret prefix (Stripe pk_* vs sk_*, Mapbox pk. vs sk.) tag only the public pattern so a misused secret key still surfaces at its nominal severity.

detector.keywords - optional prefilter and context signals. Regexes with an extractable leading literal use that prefix automatically. A prefixless regex uses only its declared required_literals; without either route it uses the keyword-gated or always-active phase-2 path. kind = "phase2-generic" detectors require keywords because their assignment/context bridge is the candidate source.

detector.capture_role declares what the pattern captures. Values are unknown, assignment-value, token, credential-envelope, private-key-block, connection-string, url-userinfo, header-value, and command-argument-value.

detector.anchor_role declares the surrounding anchor strength. Values are unknown, exact-key, distinctive-prefix, structured-envelope, companion-bound, weak-context, and unanchored.

detector.allowed_source_roles is a list of candidate-bounded source roles. Values are structured-assignment-value, environment-assignment-value, string-literal, command-argument-value, command-option-declaration, header-value, url-authority-userinfo, connection-string, standalone-token, pem-block, regex-rule-definition, identifier-type-member-name, prose-documentation, test-fixture, and generated-vendor-material. The observed source-role enum also contains unknown, but it is invalid in this declaration. Omit the field when no source-role restriction is declared.

Structured source-role extraction runs only for emitted candidates. JSON, JSONL, TOML, YAML, dotenv, and INI inputs build at most one 64 KiB source index per bounded chunk. Every candidate uses exact candidate/value span lookup against that reused index and retains bounded key-path spans. Malformed or truncated syntax and unsupported, over-nested, or over-budget input yields unknown with abstaining parser confidence. Parser failure never suppresses a finding. These roles are retained as adjudication evidence without changing public RawMatch output.

Rust, JavaScript/TypeScript, and Python source-role extraction runs only for emitted candidates and accepts at most 64 KiB of source. Exact lexical spans distinguish string literals, identifiers, regex definitions, test fixtures, command arguments, and command-option declarations. Inline test scopes use balanced syntax; test-file ownership uses the scanner’s Tier-B path rules. Malformed, truncated, unsupported, or over-budget code yields unknown with abstaining confidence and never suppresses a finding.

Markdown, roff/man, shell script, Dockerfile, and Containerfile extraction is candidate-triggered and accepts at most 64 KiB of source. Markdown prose and inline code receive prose-documentation; shell-language fences use shell token roles. Roff option declarations remain distinct from prose. Shell tokenization distinguishes environment assignments from command argument values. Structured files under detector and rule paths derive regex-definition, test-fixture, and prose roles from rules/structured-source-role-markers.toml. Malformed, truncated, unsupported, or over-budget input yields unknown with abstaining confidence and never suppresses a finding.

detector.required_evidence is a list containing checksum, required-companion, private-key-companion, structural-grammar, or live-verification. Omitted semantic fields carry no proof and preserve the schema-3 finding policy. Unknown enum spellings, unknown source-role entries, and duplicate list entries fail corpus validation.

detector.patterns[] - one or more regexes. Each carries:

• regex - the pattern. Every regex is compiled case_insensitive, so it matches both cases without explicit alternation. To make a single pattern case-SENSITIVE (AWS AKIA is uppercase; some GCP/Snowflake ids are lowercase), prefix its regex with the inline flag (?-i) in the TOML - no schema field needed. The loaded expression is byte-for-byte the authored TOML value: KeyHog never widens separator classes or quantifiers at load. When an anchor intentionally accepts joined, spaced, underscored, and hyphenated words, write [_\-\s]* explicitly in that detector. A narrower class remains narrow and changes only that detector’s digest and behavior.

  Anchor command-line options before the leading dash so a detector cannot restart inside a longer option such as --add-password. When providers share a token prefix, separate their issued length grammars and use provider-owned context for the overlapping shape. Record each decision as a sibling-prefix or regex-literal hard negative.

• group - which capture group is the credential. 0 = whole match, 1 = first captured group, etc.

• description - what shape this captures (env var, header, URL, …).

• required_literals - optional detector-owned routing literals. Every regex match must contain at least one listed ASCII literal. Corpus loading proves that OR-condition from the regex AST, then scalar, Hyperscan, CUDA, Metal, and WGPU compile the same literals into their candidate plan. Invalid, optional, or branch-incomplete declarations reject the detector instead of risking recall. KeyHog never selects a non-prefix literal from the regex implicitly.

• client_safe - optional bool, default false. When true, any match against this pattern collapses to Severity::ClientSafe regardless of the detector’s nominal severity. Use for patterns that capture keys the vendor expects to ship in client bundles (Sentry DSN, Stripe pk_*, etc.). Per-pattern (not per-detector) so a detector that covers both the public and the secret prefix can tag only the public one.

Multiple patterns means “any of these shapes”. A typical detector has 1-3 patterns covering env-var, JSON, and inline forms.

detector.companions[] defines typed evidence near a primary match. Each entry sets name, regex, within_lines, optional within_bytes, direction, scope, requirement, optional capture_group, and value_relation. requirement = "required" gates the primary. reinforcing evidence enriches confidence or verification, while forbidden evidence suppresses the primary. The scopes are window, same-line, same-record, and same-object.

detector.detector_relations[] resolves findings produced by different detectors in the same source, file, and revision. Each relation names a detector_id, a kind (requires, conflicts, or subsumes), bounded line and byte distances, and the target direction. requires removes the owner when the target is absent. conflicts removes the owner when the target is present. subsumes keeps the owner and removes the bounded target. Unknown targets, self-relations, contradictory pairs, and dependency cycles reject the compiled corpus.

[[detector.detector_relations]]
detector_id = "notion-integration-token"
kind = "subsumes"
within_lines = 0
within_bytes = 0
direction = "either"

detector.source_admission restricts a detector to positively selected sources. path_patterns contains file-path regexes, source_types contains exact source labels, and file_extensions contains lowercase extensions without a dot. Lists are alternatives within one field. Every non-empty field must match. Missing path or source metadata fails closed when that selector is declared.

detector.verify - optional. If present, keyhog scan --verify makes the documented API call with the captured credential and:

• live + valid -> keep severity, mark verification: "live"
• live + invalid -> downgrade severity one tier, mark verification: "dead"

Every shipped HTTP verifier declares allowed_domains beside its URL. Literal hosts must match that list when the TOML loads, and the resolved host must match again before a request is sent. Use the narrowest service-owned host. Do not add localhost, a generic decoder site, or an unresolved tenant placeholder to make a non-verifiable detector appear live-capable.

detector.verify.metadata[] maps provider responses to report evidence. Each entry owns three fields in this detector TOML:

• name: a reviewed provider-neutral semantic role. Unknown and duplicate canonical roles fail detector validation.
• json_path: the rooted response selector.
• sensitivity: public emits a scalar value up to 256 bytes, hashed emits only its SHA-256 digest, and secret never enters findings. Omission defaults to hashed for compatibility with older custom detectors.

Multi-step extract entries use arbitrary flow-local names because later request templates consume them. They are transport state and never become report metadata.

Per-detector recall/precision knobs

Credential-family policy belongs in the individual detector TOML whenever the schema provides a detector field. This is where stable entropy bands, length bounds, BPE behavior, confidence floors, allowlists, and shape classifications are tuned for one secret type. Scan-wide CLI and [scan] settings remain explicit operational overrides for corpus-wide policy and controlled comparisons; they are not hidden detector definitions.

This follows the design precedent established by min_confidence (the per-detector confidence floor) and entropy_floor (the low-entropy suppression floor).

Active entropy owners must declare their complete policy. Missing tuning data fails corpus validation or scanner construction instead of inheriting scanner-side detector policy. An explicitly supplied scan-wide override has final authority only where the field documents that precedence, such as BPE or the ML weight diagnostic override.

The available per-detector tuning fields are:

Entropy Thresholds

For example, an entropy owner can declare its complete confidence mapping:

entropy_fallback_confidence = { low_entropy_max = 0.55, high_entropy = 0.65, very_high_entropy = 0.75, keyword_lift = 0.1, max_confidence = 0.9 }

KeyHog compiles this mapping with the detector. A custom corpus can tune one secret family without changing another family or inheriting scanner literals.

The same owner declares how a generic assignment becomes a confidence score:

[detector.generic_assignment_confidence]
ordinary_base = 0.60
test_base = 0.25
documentation_base = 0.30
comment_base = 0.30
scanned_comment_base = 0.60
entropy_reference = 3.5
entropy_gain_per_bit = 0.10
entropy_lift_max = 0.25
length_reference = 16
length_gain_per_byte = 0.005
length_lift_max = 0.15
max_confidence = 0.95

For example, a 20-byte value gains four times length_gain_per_byte. Entropy above entropy_reference gains entropy_gain_per_bit for each additional bit. Each lift stops at its declared maximum, and the final score stops at max_confidence.

• entropy_high (float, required for active entropy owners): Per-detector high-entropy threshold (bits/byte) for keyword-independent detection and the partial entropy-fallback heuristic-confidence tier.
• entropy_low (float, required for active entropy owners): Per-detector keyword-context entropy threshold.
• entropy_very_high (float, required for active entropy owners): Per-detector very-high threshold for keyword-free or isolated tokens and the full entropy-fallback heuristic-confidence tier. Compiled policy requires entropy_low <= entropy_high <= entropy_very_high.
• sensitive_path_entropy_very_high (float, required for active entropy owners): Per-detector keyword-free threshold for clearly sensitive paths. It must not exceed entropy_very_high; omission is invalid rather than an undocumented scanner-wide discount.
• plausibility.keyword_free_operator_margin (float, required only for the keyword-free role owner): Margin added to the resolved Tier-A entropy_threshold before keyword-free admission. The effective floor is max(path-specific entropy_very_high, entropy_threshold + keyword_free_operator_margin); no scanner-owned margin is applied.
• entropy_fallback (table, required for active entropy owners): Identity metadata for synthetic entropy findings owned by this detector. class is one of generic, password, token, or api-key; id must use the entropy- namespace; and name/service must be non-empty. Both regex-kind detectors that set entropy_policy_priority and phase-2 generic owners must declare this block. The primary detector’s reporting service may name any taxonomy and does not grant or deny entropy ownership. Omitting the block is a compile error, never a compatibility identity.
• entropy_fallback_confidence (inline table, required for active entropy owners): Maps this detector’s Shannon evidence to report confidence. low_entropy_max caps candidates below entropy_high; high_entropy and very_high_entropy are the base scores for those detector thresholds; keyword_lift applies only when a configured keyword owns the candidate; and max_confidence caps the result. Every value must be a finite probability, and the three base tiers must be monotonic. The scanner does not supply confidence tiers for an omitted policy.
• generic_assignment_confidence (table, required for active entropy owners): Maps generic assignment evidence to confidence for the detector that owns the assignment keyword. The five context fields choose the base score. entropy_reference, entropy_gain_per_bit, and entropy_lift_max define the entropy lift. length_reference, length_gain_per_byte, and length_lift_max define the byte-length lift. Every base, gain, lift cap, and maximum must be a finite value from 0 to 1. entropy_reference must be from 0 to 8. The scanner does not supply a hidden assignment-confidence policy.
• entropy_roles (string array): Corpus-level entry roles owned by this detector. keyword-free owns anchor-free high-entropy candidates, isolated-bare owns detector-shaped bare candidates, and unclaimed-keyword owns configured credential keywords not claimed by another detector. A role may have only one owner in a compiled corpus. Omission disables that entry path for a focused custom corpus; the scanner never substitutes a built-in detector ID or global policy.
• entropy_shapes (one array-table entry required for active entropy owners): Declarative isolated-shape policy owned by this detector. The entry declares a charset, entropy_floor, special_min_length, optional fixed-width grouping, and explicit diversity requirements such as require_group_alpha_digit or require_non_hex_alpha. A grouped candidate’s exact length is derived from its group count, group width, and separator, so adding a shape family does not require a scanner enum variant. Multiple entries are rejected rather than silently ignored or ambiguously combined.
• plausibility (inline table, required for active entropy owners): Complete strict candidate-shape policy. Its entropy floors, length boundaries, diversity requirements, isolated-token shapes, and rejection switches are all required detector data. second_half_min_len, unique_chars_min_len, min_unique_chars, unanchored_hex_max_len, identical_char_max_len, structured_dotted_min_len, and leading_slash_base64_min_len own boundaries that were formerly scanner literals. isolated_mixed_entropy_floor governs contiguous or underscore-delimited mixed tokens. The three isolated-symbolic fields govern the shorter symbol-rich exception’s byte length, minimum symbol count, and whether one symbol must differ from underscore. reject_program_identifiers covers pure source-language names, while reject_source_symbol_identifiers independently controls digit-bearing mixed alphanumeric names. An exact lower-dash layout declared by entropy_shapes cannot bypass its shape-specific rules. The keyword-free role owner additionally declares keyword_free_operator_margin; no detector inherits an invisible family decision from scanner code.
• entropy_floor (array of tables, required for active entropy owners and detectors or patterns using weak_anchor): Length-bucketed low-entropy suppression floor mapping maximum lengths to minimum entropy scores. Weak-anchor regex findings read their own table; they never borrow another detector’s calibration.
  • max_len (integer, optional): Inclusive maximum length for this bucket.
  • floor (float): Shannon entropy floor.
• entropy_policy_priority (integer, optional): Resolves overlapping generic keyword claims. Higher values own entropy, length, canonical-shape, and BPE policy for the shared keyword. Phase-2 generic detectors participate at priority zero when omitted. Regex detectors do not participate unless they set this field. This makes the primary precedence explicit in detector TOML without overloading reporting service. Equal priorities use stable detector identity, so corpus order cannot change the owner.

BPE token efficiency

• bpe_enabled (bool, optional): Detector-local token-efficiency switch. Omission inherits the enabled default. Set false for families such as human-chosen passwords where word-like values are legitimate; the scanner then skips BPE tokenization for that detector. Do not combine false with a bpe_max_bytes_per_token ceiling; detector validation rejects the conflict.
• bpe_max_bytes_per_token (float, optional): Per-detector cl100k_base UTF-8-bytes-per-token ceiling. Values above the ceiling use fewer common subword tokens per byte, which makes them more likely to be word-like; they are suppressed after the cheaper shape and entropy gates. The detector field takes precedence over the compiled scan fallback. An explicitly configured [scan].entropy_bpe_max_bytes_per_token or CLI flag is the final Tier-A override for all eligible detectors. Lower ceilings favor precision and higher ceilings favor recall. This field tunes the precision gate after a detector or phase-2 discovery path has produced a candidate; it never creates one. The runtime relationship between BPE, Shannon entropy, and BetterLeaks’ Token Efficiency terminology is defined in How detection works.

Decoded key material

• decoded_hex_key_material_lengths (integer array, optional; kind = "phase2-generic" only): Exact printable-hex character counts this detector may retain after transport decoding. Each width must be even and at least 16, with no duplicates. generic-api-key.toml declares [32, 48]; broad token/secret detectors declare none, so decoded 40-hex SHA-1 and 64-hex SHA-256 shapes remain digest-suppressed. Structured decoders preserve transport provenance, so direct secret_key=<64hex> policy cannot silently reclassify a base64-wrapped digest.
• canonical_hex_key_material (array of tables, optional): Declares exact pure-hex character counts this detector may treat as key material instead of a digest. A kind = "phase2-generic" table must include keywords or suffixes; exact keywords must also appear in the detector’s top-level keywords, suffixes admit only vendor-prefixed names, and excluded_keywords removes ambiguous names such as license_key. A regex detector instead declares a length-only table. Its matched pattern supplies the scope, so assignment scopes are rejected on that path rather than silently ignored. Matching assignment keys ignores case and _, -, or . separators. Direct assignments and structured assignment extraction (including XML) resolve the same policy; there is no format-specific override. For example, generic-api-key.toml admits 64-hex only for its explicit cryptographic roles such as signing_key, encryption_key, and hmac_secret, while generic-secret.toml owns private_key, signing_secret, and its declared vendor suffixes. Neither turns a broad api_key=<sha256> assignment into a finding. Canonical hex admitted by this policy skips BPE token efficiency and the generic low-diversity/decode-as-data confidence penalties because those mechanisms inherently classify pure hexadecimal as non-secret. The entropy, placeholder, degenerate-repeat, context, and reporting gates still apply. The owning detector’s ml.match_mode governs structurally proven key material, while ml.entropy_mode governs its weaker entropy fallback.

The fields live beside the detector’s other top-level policy, not in a scan-wide suppression table. A phase-2 example is:

decoded_hex_key_material_lengths = [32, 48]
canonical_hex_key_material = [
  { lengths = [32, 48], keywords = ["api_key"], suffixes = ["key", "secret"], excluded_keywords = ["license_key"] },
  { lengths = [64], keywords = ["encryption_key"] },
]

A weak-anchor named detector declares only the widths its own regex captures:

canonical_hex_key_material = [{ lengths = [32] }]

Omitting that declaration leaves a pure-hex capture digest-suppressed. KeyHog does not infer widths from the service name, confidence floor, or a global list.

keyhog explain <detector-id> prints both declarations in the human-readable policy view. keyhog detectors --format json exposes them under each detector’s policy object, so automation can inspect the same loaded TOML contract the scanner uses.

Candidate Lengths

• keyword_free_min_len (integer): Per-detector minimum length for an anchor-free (keyword-free or isolated) candidate. Active entropy owners must declare it; omission fails compilation instead of selecting a scanner constant. The backend-neutral no-hit router combines the active role owner’s value with a conservative necessary length derived from that owner’s effective Shannon floor, so replacement detector corpora keep their own boundary without widening the shipped hot path to candidates that cannot reach its entropy threshold.
• min_len (integer, optional): Per-detector minimum candidate length in UTF-8 bytes for any candidate this detector emits. Falls back to no detector-specific floor beyond the path-wide default if unset.
• max_len (integer, required for every entropy-policy owner): Inclusive maximum byte length for every candidate owned by the detector. Generic assignment, entropy fallback, and explicit regex envelopes use one compiled bound before entropy or BPE. An overlength value is rejected whole with value_too_long; it is never reported as a truncated prefix. The generic candidate generator uses the largest ceiling in the loaded corpus so the resolved owner can apply its exact value. max_len must be at least 8 and no smaller than min_len. Omission fails scanner construction. Regex patterns can use narrower repetition bounds.

The generic assignment bridge exists only when the loaded corpus contains at least one phase2-generic detector. A focused custom corpus without one compiles without that bridge; KeyHog does not silently inject the bundled generic rules.

Allowlists & Exclusions

• allowlist_paths (array of strings, optional): Per-detector path-exclusion regexes (BetterLeaks-style allowlist). Any candidate match whose file path matches any of these regexes is suppressed.
• allowlist_values (array of strings, optional): Per-detector value-exclusion regexes. Any candidate secret value matching any of these regexes is suppressed (useful for filtering out test, example, or placeholder values).
• stopwords (array of strings, optional): Per-detector literal stopwords. A matched value equal to or containing any of these strings (case-insensitive) is suppressed.
• public_identifier_assignment_markers (array of strings, optional): Detector-local canonical-uppercase assignment-key fragments for public IDs such as wallet, contract, address, or peer identifiers. Boundary bytes are significant; KeyHog performs allocation-free ASCII-insensitive matching against the source line. An empty list disables this suppression for that detector.

Classification and shape policy

These fields are detector facts, not operator preferences. They therefore live only in the individual detector TOML and have no CLI or global-config override:

• structural_password_slot (bool, default false): The pattern proves a syntactic password slot, such as URL userinfo, IDENTIFIED BY, a password CLI flag, or an authorization scheme. The scanner keeps the dedicated placeholder checks but does not reject a legitimate free-form password with the generic randomness floor.
• weak_anchor (bool, default false): At detector level it applies to every pattern. Inside one [[detector.patterns]] table it applies only to that regex, so a strong sibling does not inherit its gates. Use it when the service context is useful but the captured value still collides with broad hex/base64/identifier shapes. Generic shape and randomness safeguards remain active, and the detector must declare entropy_high and entropy_floor. KeyHog does not infer this field from regex syntax.
• private_key_block (bool, default false): The match spans an enclosing PEM/OpenSSH private-key block. Resolution suppresses lower-specificity child findings inside that span instead of reporting the key body repeatedly.
• generic_vendor_suffixes (string array, default empty): A phase-two generic detector can own structural <vendor>_<suffix> assignments that no exact keyword claims. Only one detector may declare this list. Entries are lowercase ASCII alphanumeric tokens.
• generic_assignment_tail_suffixes (string array, default empty): The same owner can admit suffix segments after an exact keyword, such as secret_key_base. KeyHog compiles this list into the assignment matcher; omission disables the extra tail grammar.
• resolution_priority (integer, default 0): When two detectors claim overlapping credentials, the higher value wins before generic class and confidence tie-breakers. Use it only for demonstrably more specific attribution, such as a GitHub App key over a generic PEM block. Equal values keep the normal deterministic resolution order.
• [detector.credential_shape] (table, optional): A fail-closed byte-shape contract. It can declare exact_length, prefix, body_min_length, and body_max_length; candidates outside the declared shape are suppressed.

Because these values are loaded from the active detector corpus, custom corpora carry their classifications with them. There is no separate detector-id list or hidden Rust-side family table to keep synchronized.

Confidence Floors

• min_confidence (float, optional): Per-detector minimum confidence floor. Overrides the global scan confidence floor.

Listing the active corpus

With no --detectors flag, KeyHog first uses ./detectors when that directory exists in the caller’s working directory. Otherwise it checks the platform user and system data directories and the executable directory for an installed detector corpus. The first directory found is the complete active corpus. If none exists, KeyHog uses the embedded corpus.

keyhog detectors
keyhog detectors --format json
keyhog detectors --format json | jq length
keyhog detectors --detectors "$PWD/.keyhog/detectors" --format json

The last command names one corpus explicitly. KeyHog does not search another location or fall back to the embedded corpus when that path is explicit.

Structured listings include a policy object for every detector. It carries the loaded detector-local kind, entropy/BPE/length thresholds, stopwords, allowlists, classifications, and credential shape; absent optional fields are null, not silently filled with an undocumented value.

Filter by service:

keyhog detectors --format json \
  | jq '.[] | select(.service == "stripe")'

Which mechanisms a detector actually uses

KeyHog matches patterns, validates structure, scores entropy, gates on BPE token efficiency, recovers encoded values, confirms with companions, verifies live, and suppresses with detector-owned allowlists. A given detector uses some of those and not others. Ask which:

keyhog detectors --mechanisms

Mechanism manifest: 934 detectors from detectors

  regex                    932  phase-1 pattern anchors
  keywords                 934  phase-2 keyword triggers for shapeless candidates
  structure                 47  offline structural proof: checksum, payload decode, or declared shape
  entropy                   58  detector-owned Shannon entropy floors
  bpe                        5  BPE token-efficiency precision gate
  byte_pair_likelihood     n/a  fixed-point byte-pair log-likelihood scoring [UNAVAILABLE: see --format json for the reason]
  decode                    80  detector-declared evasion and transport decode recovery
  companions               174  secondary patterns that confirm a match
  detector_relations         5  relations to findings from other detectors
  verification             347  live verification against the provider
  suppression               10  detector-owned allowlists, stopwords, and public-identifier markers
  source_admission           1  positive source selectors gating where this detector fires

Every detector declares at least one mechanism.

This does not scan. It reads the same corpus a scan would load.

Scope it with --search, and take the machine-readable document with --format json:

keyhog detectors --mechanisms --search aws --format json \
  | jq '.detectors[] | select(.id == "aws-access-key")'

{
  "id": "aws-access-key",
  "service": "aws",
  "kind": "regex",
  "mechanisms": [
    { "id": "regex", "evidence": ["patterns"] },
    { "id": "keywords", "evidence": ["keywords"] },
    { "id": "structure", "evidence": ["credential_shape"] },
    { "id": "decode", "evidence": ["decode_transforms.reverse_prefixes", "decode_transforms.caesar_prefixes"] },
    { "id": "companions", "evidence": ["companions"] },
    { "id": "verification", "evidence": ["verify"] }
  ]
}

evidence names the detector TOML field that made each mechanism active, so the claim is checkable against the data file rather than taken on trust. There is no per-detector table in the binary: every answer is derived from the corpus, so scoping the manifest changes every count in it, and a TOML edit changes the manifest with no release.

A mechanism KeyHog cannot express yet is reported rather than omitted, because a missing row cannot be told apart from “no detector uses this”:

keyhog detectors --mechanisms --format json \
  | jq '.summary[] | select(.available == false)'

{
  "id": "byte_pair_likelihood",
  "description": "fixed-point byte-pair log-likelihood scoring",
  "available": false,
  "unavailable_reason": "no detector field expresses this yet; the fixed-point byte-pair model is unbuilt (BACKLOG KH-850), so no detector can declare it and this row is structurally empty rather than measured",
  "detectors": 0
}

Two questions the manifest answers that nothing else does. Which detectors can verify a credential against the provider:

keyhog detectors --mechanisms --format json \
  | jq -r '.detectors[] | select([.mechanisms[].id] | index("verification")) | .id'

And which detector has no regex anchor at all, and therefore reaches candidates only through phase 2:

keyhog detectors --mechanisms --format json \
  | jq -r '.detectors[] | select([.mechanisms[].id] | index("regex") | not) | .id'

On the shipped corpus that second query returns exactly one line, generic-secret.

Explaining one detector

keyhog explain stripe-secret-key

Prints the loaded detector’s keywords, patterns, companions, verification endpoint, and detector-local admission policy. For generic detectors that policy includes Shannon-entropy floors, BPE UTF-8 bytes/token ceilings, length bounds, stopwords, and allowlists exactly as declared by the detector TOML:

keyhog explain generic-secret

Add --compiled-plan to inspect the evidence rules the scanner executes. The output includes resolved companion capture groups, direction, structural scope, requirements, value relations, and cross-detector operations:

keyhog explain notion-oauth-secret --compiled-plan

This is the first place to look when debugging why a detector did or did not fire. It makes detector-owned tuning visible without searching for a Rust-side override table.

Custom detector corpora

Start from a detector that already satisfies the current schema. This example creates a focused corpus from the repository’s shipped Stripe detector:

mkdir -p "$PWD/.keyhog/detectors"
cp detectors/stripe-secret-key.toml "$PWD/.keyhog/detectors/"
cat > "$PWD/.keyhog/detectors/corpus.toml" <<'EOF'
schema_version = 5
EOF
keyhog detectors --detectors "$PWD/.keyhog/detectors" --audit

Edit the copied TOML only after the audit succeeds. A new detector must declare all required policy blocks. Copying only the short [detector] and [[detector.patterns]] example from this chapter does not create a valid schema-5 detector.

Declare the current corpus schema beside the detector files:

# my-detectors/corpus.toml
schema_version = 5

Schema 5 adds zero-based pattern_index ownership and typed negative_class values to [[detector.tests]]. Declaring either field under a schema-1 through schema-4 manifest fails the complete corpus load. The complete per-pattern evidence gate applies to schema-5 corpora. Schema-4 semantic policies continue to load without ownership fields that their manifest cannot declare.

Schema 4 adds typed capture_role, anchor_role, allowed_source_roles, and required_evidence declarations. Omission preserves the schema-3 finding policy and carries no semantic proof. Declaring any of these fields under a schema-1, schema-2, or schema-3 manifest fails the complete corpus load.

Schema 3 adds typed companion semantics and cross-detector relations. It also keeps the schema-2 requirement that every [detector.verify.success] and per-step success table classify its evidence explicitly with policy = "body_positive", "status_with_error_backstop", or "status_authoritative". An omitted policy is a validation error that names the affected success table.

For compatibility, a directory without corpus.toml is schema 1, as is a manifest that explicitly declares schema_version = 1. Schema-1 success tables written before policy classification are normalized to status_with_error_backstop: an accepted status is necessary, but a known error-shaped response still prevents a live verdict. This is deliberately not the more permissive status_authoritative policy. New corpora should declare schema 5 and serialize every policy rather than relying on legacy normalization.

Manifest typos, unsupported schema versions, and schema-2 through schema-5 success tables with missing policies fail closed. A bounded newer schema declaration may be parsed only to produce compatibility diagnostics; a gated load refuses the complete corpus rather than skipping fields or detector files it cannot interpret. The effective corpus digest binds the normalized schema and manifest identity, so legacy and schema-2 through schema-5 corpora cannot share an identity merely because their detector fields otherwise match.

Audit a custom corpus directly before scanning with it:

keyhog detectors --detectors "$PWD/.keyhog/detectors" --audit

Then select exactly one composition mode:

Overlay never shadows a shipped detector. An ID collision fails before scanning. Replace never fills missing detector families from the embedded corpus.

The path must name an existing, non-empty directory. A missing path, a regular file, an empty directory, invalid TOML, an unsupported manifest, or an invalid detector fails before findings are written. Supplying --detectors-mode without a detector path from either the CLI or .keyhog.toml also fails. KeyHog does not turn any of these errors into an embedded scan.

The CLI and configuration-file precedence for the path and mode is documented in Configuration. Versioned JSON envelopes report embedded, replace, or overlay, the source and detector counts, and the effective corpus digest under metadata.resolved_scan.effective.

Disabling specific detectors