Skip to content

App data (W7): a data delete is not undone by a re-score running around it - #50

Open
ayiskakov wants to merge 3 commits into
mainfrom
review/w07-delete-hold
Open

ayiskakov wants to merge 3 commits into
mainfrom
review/w07-delete-hold

Conversation

@ayiskakov

Copy link
Copy Markdown
Owner

Phase 2 fix for W7: a data delete that lands while a re-score runs must not be undone by it. Covers W07-017 and W07-033 (S3), plus W07-048, found and fixed by the independent review. One commit per fix, plus the status-board update. Each row in docs/review/workstreams/W07-app-data.md holds its V0, fix, V1 and V2.

The defect

A scoring pass reads its raw data first and writes at the end. A delete that landed between the two had its rows written back from what the pass had already read. This covered the Devices delete-data and Forget actions and the Apple Health delete. With no raw data left, no later pass could evict those rows (W07-017, W07-033).

The fix

Every delete runs through IntelligenceEngine.withScoringHeld. It does four things:

  • Holds off new passes. A forced trigger that arrives meanwhile is folded into the re-score after the delete.
  • Raises the running pass's RescorePassAbandon. The pass checks it at each day and once more, synchronously, before its first score write. If it is raised, the pass ends without writing scores (re-score: dropped after …). The Re-score needs 10s of work but takes 25-70min backgrounded, so the score lands ~an hour after the data (#1005) ryanbr/noop#1538 owed mark stays, for the next pass to settle.
  • Waits for the lock, then runs the delete. A pass that is already writing finishes first, and the delete then removes what it wrote.
  • Launches one forced re-score (trigger=data-delete).

W07-048 (found by the V2 review): the delete's re-score covered max(21, held trigger) and dropped the cut-short pass's own window. A 4,000-day effort re-score or timestamp heal cut short by a delete therefore came back as 21 days, and its caller saw the lock free and set its done flag. The pass now folds its width into the held window when a delete is in flight and the pass was dropped or owed a re-pass.

Evidence

  • V0/V1: DataDeleteDuringRescoreTests.
    • On the unfixed engine, a delete landed between the scan and the writes left three computed days written back.
    • Removing the wait fails deleteRanDuringPass, and removing the hold fails the folded window. Removing the drop fails only the drop's log line: the wait alone already keeps the rows out.
    • testADroppedWidePassKeepsItsWindowInTheDeletesRescore fails with W07-048's fold removed.
  • Independent V2: holds with caveats.
    • It built a standalone model of the hold and the pass and landed a delete at every phase: before the gate, mid-scan, between the scan and the writes, and during the writes. It also tried two deletes at once, a delete that throws, and a forced trigger arriving during the hold.
    • In every case no rows survive, the delete never runs while computing, the hold returns to 0, and exactly one re-score fires.
    • Every delete caller in Strand/, StrandiOS*, NOOPWatch and Packages/ goes through the hold.
    • It found W07-048 (fixed here) and corrected a drop line that claimed "writing nothing" (the edited-stage heal writes before the second drop check).
    • It also found W07-049 and W07-050, both S4, both recorded:
      • W07-049: the manual Fitness Age refresh runs outside the hold.
      • W07-050: a delete re-scores only 21 days. This predates the fix.
  • Builds and tests, run locally on this branch rebased on main:
    • xcodebuild test, scheme Strand (macOS): 2,151 tests, 0 failures, 2 skipped.
    • NOOPiOS builds. That build was done without the watch app, which needs a watchOS runtime this machine lacks.
  • App-target Swift. No default CI job compiles it, which is why the builds and tests above were run locally.

… (W07-017, W07-033)

A pass reads its raw first and writes at the end. A strap delete, a
forget or the Apple Health delete landing in between had its rows
written back from what the pass read before it, and with no raw left no
later pass could evict them.

Every delete now runs through IntelligenceEngine.withScoringHeld: new
passes are held off (a forced trigger is folded into the re-score after
the delete), the running pass's RescorePassAbandon is raised and the pass
ends without writing (checked at each day and synchronously before its
first write; the ryanbr#1538 owed mark stays for the next pass), the delete
waits for the lock, and one forced re-score (trigger=data-delete) follows.

DataDeleteDuringRescoreTests failed on the unfixed engine (three computed
days written back) and pass now; removing the wait, the drop or the hold
each fails its own assertion. StrandTests 2,150, NOOPiOS (no watch) builds.
…lete's re-score (W07-048)

W07-017's V2 found that the delete's re-score covered max(21, held
trigger) and folded the dropped pass's own maxDays nowhere. A 4,000-day
effort re-score or timestamp heal dropped for a delete came back as 21
days, and its caller, finding the lock free, set its done flag.

The pass's defer now folds its maxDays into the held window when a delete
is in flight and the pass was dropped or owed a re-pass. The drop line
says "writing no scores" (the edited-stage heal writes before the reads
drop site), and the ryanbr#1538 comment names the drop as its one early return.

testADroppedWidePassKeepsItsWindowInTheDeletesRescore passes, and fails
with the fold removed. W07-049 and W07-050 recorded.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant