fix: install hints resolving to unpublished PyPI names, plus four silent scanner defects

[spierenburg]

Summary

• Install hints pointed at PyPI names the upstream projects never publish. Eight return HTTP 404, so pip install <name> resolves to whatever a third party registers under them; two (graphw00f, gqlmap) are already occupied by third-party placeholder packages. Now install from upstream source. Separately, knockpy on PyPI is an unrelated statistics library, so that hint installed the wrong software entirely.
• Three silent scanner defects fixed, each of which fabricated or suppressed findings without any signal in the output: catch-all detection never took effect (subshell discarded its state), SAML endpoint discovery had no baseline control, and four curl calls dropped credentials on authenticated runs.
• A GraphQL phase that could never run was gated on a non-existent binary while reporting itself "skipped"; the built-in probe underneath it was doing the work all along.
• The credential-attack installer reported success for tools it skipped.

Measured on a host that returns 200 for every path (as React/Vue/Angular front ends do): phantom [UPLOAD-CANDIDATE] 8 → 0, phantom [SAML-ENDPOINT] 9 → 0.

Type

• Bug fix
• New feature / scanner module
• Methodology improvement
• Documentation
• False positive reduction

Test Plan

• pytest tests/ passes locally — 1124 passed, 4 failed. Measured against a clean worktree of main at 40b03ee, which gives 1118 passed, 4 failed — the same four, so this branch adds 6 passing tests and no failures. Leaving the box unchecked rather than claiming a green suite. The four are pre-existing and unrelated:

  • test_benchmark.py::test_verifier_benchmark_recall_and_no_false_positives — benchmark targets failed to start
  • test_benchmark.py::test_shipped_demo_is_hardened — demo app failed to start
  • test_brain_auto_detect.py::test_openai_and_grok_keys_both_front_nothing_available
  • test_brain_auto_detect.py::test_no_keys_falls_back_to_default_priority

• No hardcoded targets, API keys, or real domain names — no keys or secrets added. The only domain this diff introduces is target.com, in one SKILL.md usage example, matching the placeholder convention already used ~70 times across 19 files on main (including 40 in that same file). Flagging in case you'd rather standardise on example.com.

• Scanner additions use [CONFIRMED] / [POSSIBLE] / [INFORMATIONAL] confidence states — no new finding types are introduced; existing states are preserved, and the changes reduce false [INFORMATIONAL] output.

• New functionality has at least one regression test — tests/test_vuln_scanner_catchall_guards.py, 6 assertions in the style of the existing test_vuln_scanner_review_fixes.py. Nothing covered any of these constructs before.

  Each defect here was silent — the scanner emitted or suppressed findings with nothing in its output to say coverage had changed — so a regression would be invisible in a manual run. Every assertion was mutation-tested: reintroducing the pipe, un-anchoring the match, reverting the awk field to 2, removing a splat, and wrongly adding one to the signature-stripping probe each make the corresponding test fail. A test that passes under both the fixed and the broken code is not a regression guard.

  One of the six is the inverse of the others: it asserts the MFA workflow-skip and SAML signature-stripping probes still do not carry $BB_AUTH_ARGS. Both look like missing-auth bugs to anyone auditing for them, and "fixing" either silently destroys the check — so that now fails loudly instead.

tests/test_vuln_scanner_review_fixes.py passes, including test_saml_signature_stripping_is_opt_in_and_policy_gated — which is why the two deliberately-anonymous probes below were left alone rather than "fixed". bash -n clean on all four shell files.

Related Issue

Closes #

(No existing issue — found while using the toolkit. The supply-chain half was also reported privately per SECURITY.md, since §1 names registry entries that are currently unclaimed.)

---

Detail

1. Install hints resolve to unpublished PyPI names

bughunter/tools/external_arsenal.sh instructed pip/pipx install <name> for tools whose upstreams have no PyPI release. Verified against the PyPI JSON API — each returns 404:

ghauri  jwt-tool  dnsreaper  whatwaf  cloud-enum  sublert  linkfinder  graphql-cop

for n in ghauri jwt-tool dnsreaper whatwaf cloud-enum sublert linkfinder graphql-cop; do
  printf '%-13s %s\n' "$n" "$(curl -s -o /dev/null -w '%{http_code}' https://pypi.org/pypi/$n/json)"
done

Python packages execute arbitrary code at install time, so whoever registers one of these gets execution on anyone who follows the hint — without them ever running the tool. Two names are already taken by third-party placeholder packages (graphw00f, gqlmap, both v0.0.1) whose own description says they were registered defensively; following those hints today installs something that is not the tool.

Fixed by installing from upstream source: pipx install git+<repo> where the project ships setup.py/pyproject.toml (tested working for ghauri 1.4.3 and cloud_enum 0.8), else git clone.

Also in this area:

• knockpy installed unrelated software. PyPI knockpy is "Knockoffs for variable selection" v1.3.5, a statistics library — not the subdomain scanner. Changed to knock-subdomains v9.0.0, which provides a knockpy binary. Tested.
• gqlmap has no upstream. The cited repo nicola-inchingolo/gqlmap returns 404. Repointed at swisskyrepo/GraphQLmap, noting it is an interactive REPL. Known gap: the registry key is still gqlmap while GraphQLmap's binary is graphqlmap, so _have will keep reporting it missing. Tell me which name you want and I'll align the key.
• aquatone marked archived upstream (suggests gowitness); bbot/x8 note the Rust toolchain requirement; mobsf pinned to python3.11.
• scoutsuite key → scout, its real binary name — it was reporting a false MISSING while installed. Note: commands/arsenal.md:26 still says scoutsuite, so --install-hint scoutsuite now answers "Unknown tool". Can fix in this PR if you prefer.
• clairvoyance deliberately left unchanged and marked verified — it is published by its upstream author.

2. A GraphQL phase that could never run, reported as "skipped"

bughunter/tools/graphql_audit.sh gated phase 5 on _have gqlmap, a binary that cannot exist (see above). The gate was never true, the else branch always ran, and summary.txt always reported injection_scan: skipped (gqlmap not installed) — while the built-in SQLi probe inside that branch did the actual work. A reader of the summary would conclude injection was not tested.

Removed the conditional and promoted the built-in probe to be the phase. Deliberately not repointed at GraphQLmap: its entrypoint is while True: input("GraphQLmap > "), a REPL with no batch mode, so driving it here would block on stdin or die on EOF.

3. Catch-all detection never took effect → fabricated findings

head -10 "$ORDERED_SCAN" | while read -r host; do   # subshell
    CATCHALL_HOSTS="${CATCHALL_HOSTS},${host}"      # discarded on exit
done
[[ "$CATCHALL_HOSTS" == *"$host"* ]] && continue    # never matched

Repro, no target needed:

C=""; printf 'a\n' | while read -r h; do C="$C,$h"; done; echo "[$C]"   # prints []

The skip could never fire — even on hosts where "Catchall detected" was correctly logged. Replaced both pipes with process substitution (done < <(head -N …), already used at line 398 of the same file).

Separately, the SAML endpoint check accepted 200|301|302|403 as "endpoint found" with no baseline at all. Added a per-host control probe against a random path; if it returns one of those codes the check is skipped with a logged reason. Since 403 was already accepted, a host returning 403 for unknown paths previously reported the entire SAML path list.

4. Four curl calls dropped credentials on authenticated runs

The catch-all baseline, the upload probe loop, SAML endpoint discovery and SAML metadata retrieval omitted $BB_AUTH_ARGS while neighbouring calls include it. The upload one mattered most: behind a login every probe got a 401, found nothing, and the run reported no findings — which reads as "no upload surface" rather than "not tested".

Left deliberately unauthenticated, now with comments saying so, because adding the session destroys them: the MFA workflow-skip test (asks whether a page is reachable without auth) and the SAML signature-stripping probe (forges a login; a valid session returns 200/302 regardless, producing a false CRITICAL). recon_engine.sh's three uncredentialed calls are also correct as-is — they query crt.sh and web.archive.org, which must never receive a target session cookie. Not touched.

5. Three defects in my own first pass (fixed in the last commit)

I reviewed this branch before submitting and found three problems with it. Flagging them because two change how an earlier commit reads.

(a) The catch-all fix activated a latent bug. Making CATCHALL_HOSTS survive the loop promoted [[ "$CATCHALL_HOSTS" == *"$host"* ]] from dead code to live logic — and that match is substring, not host equality, so a recorded https://api.t.com:8443 would suppress a probed https://api.t.com. Entries are now stored ,host, and matched with a comma-anchored case.

(b) The SAML metadata and ACS legs read the wrong field, and my commit message overclaimed because of it. Rows are [INFORMATIONAL] [SAML-ENDPOINT] <url> | HTTP <code>, so the URL is field 3 — but both consumers read field 2, the literal tag [SAML-ENDPOINT], which curl rejects at URL-parse time. Those legs have never reached the network. Pre-existing, but it means the $BB_AUTH_ARGS I added to the metadata call was inert and commit 84cc1e4's subject ("auth propagation in … SAML checks") was wrong as written. Fixed to field 3.

Please review this consequence deliberately. Fixing the field index activates two legs that previously could not fire. The metadata check is a GET. The signature-stripping probe POSTs a forged SAMLResponse — it stays behind unsafe_method_guard / ALLOW_UNSAFE_HTTP_TESTS, which tests/test_vuln_scanner_review_fixes.py already asserts, so the policy gate is the control. But it will now actually execute where policy allows, for the first time. If you would rather it stay dormant, drop the ACS_URL half and keep the metadata half.

(c) Install hints chained commands without entering the clone. git clone X && pip install -r requirements.txt leaves the shell in the parent directory, so in a recon directory holding cloned target source, pasting that hint installs the target's requirements and runs its build hooks. Six hints fixed to cd into the clone, matching the form skills/graphql-audit/SKILL.md already used.

6. Installer reported success for tools it skipped

BREW_CRED_TOOLS=("hashcat" "theharvester")
for tool in "${BREW_CRED_TOOLS[@]}"; do
    if command -v "$tool" &>/dev/null || command -v "theHarvester" &>/dev/null; then
        log_ok "$tool already installed"

The || command -v "theHarvester" applied on every iteration. The alias is needed — theHarvester's binary is capitalised and never matches its formula name — but as written, anyone who already had theHarvester got [+] hashcat already installed and no install. Reproduced on macOS 15 / arm64: --with-credential-attack finished reporting 15/16 with hashcat: NOT FOUND in its own verification block, contradicting the earlier success line. Scoped the alias check to theharvester.

---

Relationship to #157

§3 and §4 sit upstream of the report gate in #157. That PR's description states it is report-time only and does not affect scanner detection or probing — so a phantom upload path is still probed, and a phantom SAML endpoint still written to saml/endpoints.txt, regardless of whether the resulting report is later suppressed. Complementary, not overlapping. Also worth noting both §3 and §4 emit at [INFORMATIONAL], and if that tier isn't gated, these false positives bypass the verification spine entirely.

Not included

54 invocations across commands/ and skills/ still reference tools/* paths that no longer exist since the bughunter/ move, and .gitignore:53's memory/leads/ is anchored to the repo root so it no longer matches bughunter/memory/leads/ (running the test suite leaves untracked files). Both are restructure fallout rather than scanner bugs — happy to open a separate issue or PR.

Environment

macOS 15.x arm64, bash 3.2.57, branched from 40b03ee. PyPI status via the PyPI JSON API, repo status via the GitHub API.