BAD CODE™: stream machine-local path scans with bounded memory

[flyingrobots]

Classification

BAD CODE™ — bounded-memory tooling debt discovered while hardening #838.

Problem

GitMachineLocalPathGuard.findTreePaths() sends every unique tree blob to one git cat-file --batch process and materializes the complete response in one buffer. MAX_INSPECTED_BLOB_BYTES caps that aggregate response at 512 MiB.

The guard fails closed, which is correct, but its memory and success envelope scale with the total bytes in the committed tree. A clean tree can therefore be rejected solely because the aggregate batch crosses the fixed ceiling. That is inconsistent with the repository direction toward streaming and bounded materialization.

Required correction

• Consume git cat-file --batch incrementally or inspect deterministic bounded object batches.
• Bound memory independently of total committed-tree size.
• Preserve exact object identity, binary-blob inspection, symlink-target inspection, and deterministic offender ordering.
• Detect policy tokens that cross internal read-chunk boundaries.
• Continue to fail closed on malformed or truncated Git output.
• Never print matching content; report only safe repo-relative paths or object identifiers.

Acceptance

• A test forces a deliberately small inspection window and proves a clean aggregate larger than that window completes.
• A test detects a forbidden token split across internal chunks.
• Malformed and truncated batch streams remain hard failures.
• Existing working-tree, staged-index, outgoing-object, and exact-tree guard tests pass.

Related: #838.

[coderabbitai]

🔗 Related PRs

#85 - release: v17.0.0 follow-up substrate closeout [merged]
#93 - Fix recursive tree OID read fanout and release v17.0.1 [merged]
#753 - fix(storage): retain trie roots through git-cas [merged]
#763 - perf: establish Git object access strategy [merged]
#825 - Make v18-to-v19 migration safe and observable [merged]

---

📝 Issue Planner

_{Check the box below or use the @coderabbitai plan command to generate an implementation plan and prompts that you can use with your favorite coding assistant.}

• Create Plan

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!