Skip to content

Nested memory files: show when a file cannot collect evidence under the all-paths attribution rule #191

Closed

Description

@Pantoria

Problem

A nested file trains only on sessions whose every in-repo work path is under its directory (workedPaths and workedUnder, src/nested.js). Reads count as work paths.

In a repo where most sessions cross directories, a nested file never reaches minGapEvidence sessions, and nothing tells the user. The file is listed as configured, runs finish, and it gets no edits. The user cannot tell "no gaps" apart from "no evidence can reach this file".

Measured

One repo, Claude Code sessions, last 30 days, minUserTurns: 2: 78 sessions. I replayed the attribution offline over the transcripts (structured tool-call paths, repo-relative, worktree prefixes removed):

attribution rule most sessions for any one top-level subdirectory subdirectories with 2 or more sessions
current: every path under the directory, reads included 2 2
same, but reads of root-level files ignored 3 6
only edit/write paths counted 3 4

So in this repo a looser rule helps only a little. The main gap is that the user does not see the count. (Here the root memory file tells agents to read a root-level map file first, so one read makes many sessions cross-cutting.)

Proposal

  1. backpass status and the run summary print, for each nested file, the number of sampled sessions attributed to it, with a warning below minGapEvidence. For example: apps/api/AGENTS.md: 1 of 78 sessions worked only under apps/api/, below minGapEvidence (2); this file cannot receive edits.
  2. Optional and opt-in, still deterministic: do not count reads of the configured memory files themselves (the root file and named ancestors) as work paths. The agent reads them as instructions, not as work. In the table above, the effect of this is at most the "reads of root-level files ignored" row.

backpass 0.1.30 (main d1e7550).

Activity

  1. kunchenguid commented on Sep 30, 2026

    @kunchenguid
    Owner

    Speaking as Kun's firstmate:

    Outcome: ready-for-pr — real silence on main (d8b2c8a). No open PR. Nested attribution (workedPaths / workedUnder, src/nested.js) can leave a nested file with fewer than minGapEvidence sessions while status/run summary still look healthy, so the user cannot tell "no gaps" from "no evidence can reach this file".

    contract-class: proposal (1) status/summary attribution counts + below-minGapEvidence warning = restore (Failure is loud). Proposal (2) opt-in ignore of configured memory-file reads as work paths = opt-in (changes deterministic attribution; must stay explicit). A PR may ship (1) alone; (2) only behind config.

    VISION (per rule):

    1. Evidence is the only currency — aligns (show whether evidence can accumulate).
    2. The human owns the weights — aligns (human named the nested file; they need to see it cannot train).
    3. Nothing the model says is taken on faith — aligns (counts from attribution, not model prose).
    4. The budget is the constraint — aligns (avoid silent zero-edit runs that look like success).
    5. It reads what you already have and owns nothing — aligns; (2) is accuracy-over-coverage if kept opt-in.
    6. Failure is loud and named — aligns strongly for (1).
    7. Scope — aligns.

    ready-for-pr. I will not open a competing PR.

  2. kunchenguid commented on Oct 14, 2026

    @kunchenguid
    Owner

    Speaking as Kun's firstmate:

    Still-real pass (14-day restamp): ready-for-pr. Rechecked latest main (e6a030a): nested attribution is still all-paths (workedPaths / workedUnder in src/nested.js), and neither status nor the run summary prints a per-nested-file attributed-session count or a below-minGapEvidence warning. No open PR. (#205 changes where partially attributed additions go. It doesn't surface this count.)

    contract-class: part (1), the count plus warning, is restore (failure is loud). Part (2), ignoring reads of configured memory files, is opt-in only. A PR can ship (1) on its own.

    VISION (per rule):

    1. Evidence is the only currency: aligns (shows whether evidence can reach the file at all).
    2. The human owns the weights: aligns (they named the file and need to see it can't train).
    3. Nothing the model says is taken on faith: aligns (counts come from deterministic attribution).
    4. The budget is the constraint: aligns.
    5. It reads what you already have and owns nothing: aligns. Keeping (2) opt-in keeps accuracy ahead of coverage.
    6. Failure is loud and named: aligns strongly for (1).
    7. Scope / align-resist: aligns (helps a person tell "no gaps" from "no evidence"). Making (2) the default would be the resist case.

    Leaving ready-for-pr. I won't open a competing PR.

  3. Pantoria commented on Oct 15, 2026

    @Pantoria
    ContributorAuthor

    PR #209 is up for review (raised through no-mistakes). It implements part (1) only: status (text and --json) and the run summary show each nested file's attributed sampled-session count, with a report-only warning below minGapEvidence. Part (2), the opt-in to ignore reads of configured memory files, is left for a follow-up.

  4. kunchenguid commented on Oct 15, 2026

    @kunchenguid
    Owner

    Speaking as Kun's firstmate:

    existing-pr: #209 (thanks @Pantoria). It covers part (1): the attributed sampled-session count per nested file in status (text and --json) and the run summary, with a report-only warning below minGapEvidence. That's the restore part. Part (2), the opt-in, is still open for a follow-up. Heads-up: #209 currently shows as conflicting with main after today's merges, so it'll need a rebase through no-mistakes. Keeping ready-for-pr.

    VISION: evidence, human-owns-weights, nothing-on-faith, budget, owns-nothing, loud failure (strongly for part 1) and scope all align. Making part (2) a default would be the resist case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    ready-for-prIssue is in scope and ready for a PR

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions