HTTP and wire scanning

Real credentials don’t always sit on disk. They flow through:

• Live web bundles that ship from production at a public URL.
• HAR files that browsers (Chrome / Firefox / Safari DevTools) produce when you click “Save all as HAR with content.”
• mitmproxy / Burp captures of an authenticated session.
• curl / httpie / Postman exports of one specific request you want to verify.

KeyHog scans every one of these, but the surface is split across a few flags and sources. This page is the map.

TL;DR

The --url flag (Web Source)

keyhog scan --url https://app.example.com/static/main.js
keyhog scan --url https://app.example.com/static/main.js \
            https://app.example.com/static/runtime.js \
            https://app.example.com/static/vendor.js

Each URL is fetched with the shared HTTP client policy (see Proxy and TLS below). The response is routed by extension:

• .js → one chunk per file, scanned as plain text.
• .map → JSON parsed, each sourcesContent[i] becomes its own chunk tagged with the original filename. This is how a Webpack build with devtool: 'source-map' accidentally exposes server- side env vars baked into the bundle at build time.
• .wasm → linear-memory + import section dumped as strings (best- effort; native WASM symbol extraction lives behind the binary feature).
• Everything else (HTML, JSON that is not a source map, extensionless, …) → one chunk of text, scanned as-is.

Findings are tagged source: "web:js", web:sourcemap, web:sourcemap:raw, or web:wasm. Anything scanned as plain text (including the “everything else” case above) carries web:js; there is no separate web:other tag. The original URL is the file_path.

Use URLs without credentials, signed query strings, or secret fragments. Scan metadata redacts a target’s query and fragment, but a finding’s file_path identifies the fetched URL. Store web-source reports as sensitive artifacts. Do not add --show-secrets to a retained report.

SSRF defense

--url refuses to fetch:

• Private RFC1918 ranges (10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16).
• Loopback (127.0.0.0/8, ::1).
• Link-local (169.254.0.0/16, fe80::/10).
• Cloud metadata endpoints (169.254.169.254, the GCP / Azure / AWS / DigitalOcean / Hetzner variants).

This protection is not configurable. It prevents a URL scan from reaching a metadata service or another private endpoint.

Proxy and TLS

Remote sources and verification share the same explicit proxy and TLS policy.

Precedence is the CLI, then .keyhog.toml, then no proxy and strict TLS. HTTPS_PROXY, HTTP_PROXY, ALL_PROXY, and NO_PROXY cannot change it. TLS environment toggles are also ignored. --insecure applies to every remote-source and verification client in that scan, so use it only with an interception proxy or endpoint you control.

KeyHog sets User-Agent: keyhog/<version>.

Web fetching follows at most five redirects. Every redirect target is parsed, DNS-screened, and pinned again before the request. Verification requests follow no redirects. A verifier redirect becomes an error finding instead.

An invalid proxy value prevents the HTTP client from starting. An unreachable proxy does not fall back to a direct connection.

HAR auto-expansion

Any file with a .har extension is recognised by the filesystem source and expanded into one chunk per request and one chunk per response. Each chunk carries a source-type that tells you which side of the exchange it came from:

Finding file_path becomes <har-path>#<request-url>, so the same HAR with five different requests produces five distinct paths. Editors that jump-to-file on path:line URIs land on the HAR but the URL tail makes the location unambiguous.

The finding remains a normal redacted finding. This JSON fragment shows the exact location shape with synthetic values:

{
  "credential_redacted": "ghp_...4f2a",
  "location": {
    "source": "wire:har:request",
    "file_path": "capture.har#https://api.example.test/v1/me",
    "line": 2,
    "offset": 41,
    "commit": null,
    "author": null,
    "date": null
  }
}

The HAR request URL is part of file_path. A query credential in the captured URL can therefore appear in the report’s location even though the detected credential itself is redacted. Remove or sanitize sensitive query strings before sharing a HAR artifact or its report.

keyhog scan capture.har --format json-envelope | \
  jq '.findings[] | select(.location.source == "wire:har:request")'

This filters down to outbound credentials for the bug-bounty “what did I send” view. Select wire:har:response instead to see what the upstream reflected back at you.

A HAR that fails to parse is still scanned as plain text. KeyHog also records a structured_source_parse_failure coverage gap because request and response expansion did not happen. A metadata-bearing report is therefore partial, even if the raw fallback found no credentials.

The expander applies two bounds:

• Cumulative rendered request and response bytes are limited to four times --max-file-size.
• KeyHog decodes the text, checks for the required log and entries markers, and only then invokes the JSON parser. The marker check covers the full decoded text, so valid HAR metadata may precede entries.

Scanning a single HTTP exchange (stdin)

The most common ad-hoc workflow:

curl -s https://api.example.com/v1/me \
     -H "Authorization: Bearer $TOKEN" \
| keyhog scan --stdin

Or just pipe a saved response:

keyhog scan --stdin < response.txt

keyhog scan - (bare dash) is the same as --stdin (grep / wc convention).

--stdin reads up to 10 MiB by default. Raise the explicit --limit-stdin-bytes <SIZE> ceiling when a larger stream is intentional, or write the input to a file and scan the path. Findings from stdin carry the stdin source. To get the richer wire:har:request / wire:har:response provenance tags, save the exchange as a .har file and scan that instead (see HAR auto-expansion).

Use --format json-envelope for automation. Legacy json cannot say whether the source was complete:

set +e
keyhog scan capture.har \
  --format json-envelope \
  --output keyhog-results.json
status=$?
set -e

jq -e '.scan_status == "success" or
       .scan_status == "complete_after_recovery"' keyhog-results.json
test "$status" -eq 0 -o "$status" -eq 1 -o "$status" -eq 10

The jq check rejects malformed-HAR fallback and other coverage gaps. Exit 0 means no finding blocks the active evidence policy. Exit 1 means a finding blocks without a live verification result. Exit 10 means at least one live finding. A source failure with no blocking finding exits 13. Blocking findings take precedence, so a partial report can still exit 1 or 10.

Headers, bodies, and URL parameters

The detector engine matches the bytes supplied by each source adapter. A plain text capture is scanned as-is. The HAR adapter renders each request and response into separate chunks before scanning. A synthetic Bearer ghp_...4f2a in an HTTP header is therefore checked by the same detector policy as a synthetic "token":"ghp_...4f2a" in a JSON body or ?token=ghp_...4f2a in a URL.

The finding location gives the byte offset in the rendered chunk. It does not identify the exact header, JSON path, or query field. HAR findings identify only the request or response side.

Unsupported behavior:

• Parse the HTTP wire format and emit header:Authorization vs body:json:$.token provenance fields.
• Attach field-level provenance such as header:Authorization, body, or query to a finding. HAR findings do distinguish the request and response sides through source_type.

Fetch and parse errors

A non-success HTTP status, timeout, DNS failure, response above the configured size limit, invalid content encoding, invalid WASM response, blocked destination, or redirect-policy failure means some requested bytes were not scanned. KeyHog records the reason as a coverage gap and makes metadata-bearing artifacts partial.

When no finding blocks the active evidence policy, incomplete source coverage exits 13 and stderr says KeyHog is not reporting the scan as complete. A blocking or live finding takes exit 1 or 10 precedence. Consume the artifact’s scan_status and coverage_gap_summary; never use a finding count or process exit alone as a completeness signal.

Unsupported Wire Features

The wire-scanning surface is intentionally narrow. These features are not part of the shipped HTTP-wire contract:

1. mitmproxy .mitm flow-dump support. The binary-framed format is not decoded. Export HAR when request/response provenance matters, or export text and scan it as an ordinary file.

2. Header / body / URL-param provenance. HAR expansion emits one chunk per request and one chunk per response. It does not attach wire_location: header:<name> | body | query to each finding, so the JSON consumer cannot filter wire_location == "header:Authorization" for the highest-signal subset (intentional auth tokens vs accidental body leaks vs URL-logged secrets).

3. Live proxy mode. KeyHog does not ship keyhog proxy --listen :8080 or an inline HTTP proxy that scans flows while forwarding them.

4. WebSocket frame scanning. HAR files do not include WebSocket payloads, and KeyHog does not parse mitmproxy frame dumps as a WebSocket source.

Why this matters for bug bounties

A modern SPA bundle on a typical SaaS app can ship 200+ npm dependencies and a sourcemap that exposes every server-side env var the build process touched. Manual code review of one main.js.map against the full detector corpus is hours; running keyhog scan --url https://app.target.com/static/main.js.map takes seconds.

Pair it with --hide-client-safe (see CLI reference) to filter out keys that the vendor designed to ship in client bundles (Sentry DSN, Stripe pk_*, Mapbox pk., PostHog phc_, etc.) and you’re left with the keys that actually represent an exfiltration boundary.