Repository navigation
Conversation
… (W07-017, W07-033) A pass reads its raw first and writes at the end. A strap delete, a forget or the Apple Health delete landing in between had its rows written back from what the pass read before it, and with no raw left no later pass could evict them. Every delete now runs through IntelligenceEngine.withScoringHeld: new passes are held off (a forced trigger is folded into the re-score after the delete), the running pass's RescorePassAbandon is raised and the pass ends without writing (checked at each day and synchronously before its first write; the ryanbr#1538 owed mark stays for the next pass), the delete waits for the lock, and one forced re-score (trigger=data-delete) follows. DataDeleteDuringRescoreTests failed on the unfixed engine (three computed days written back) and pass now; removing the wait, the drop or the hold each fails its own assertion. StrandTests 2,150, NOOPiOS (no watch) builds.
…lete's re-score (W07-048) W07-017's V2 found that the delete's re-score covered max(21, held trigger) and folded the dropped pass's own maxDays nowhere. A 4,000-day effort re-score or timestamp heal dropped for a delete came back as 21 days, and its caller, finding the lock free, set its done flag. The pass's defer now folds its maxDays into the held window when a delete is in flight and the pass was dropped or owed a re-pass. The drop line says "writing no scores" (the edited-stage heal writes before the reads drop site), and the ryanbr#1538 comment names the drop as its one early return. testADroppedWidePassKeepsItsWindowInTheDeletesRescore passes, and fails with the fold removed. W07-049 and W07-050 recorded.
… the status board
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Phase 2 fix for W7: a data delete that lands while a re-score runs must not be undone by it. Covers W07-017 and W07-033 (S3), plus W07-048, found and fixed by the independent review. One commit per fix, plus the status-board update. Each row in
docs/review/workstreams/W07-app-data.mdholds its V0, fix, V1 and V2.The defect
A scoring pass reads its raw data first and writes at the end. A delete that landed between the two had its rows written back from what the pass had already read. This covered the Devices delete-data and Forget actions and the Apple Health delete. With no raw data left, no later pass could evict those rows (W07-017, W07-033).
The fix
Every delete runs through
IntelligenceEngine.withScoringHeld. It does four things:RescorePassAbandon. The pass checks it at each day and once more, synchronously, before its first score write. If it is raised, the pass ends without writing scores (re-score: dropped after …). The Re-score needs 10s of work but takes 25-70min backgrounded, so the score lands ~an hour after the data (#1005) ryanbr/noop#1538 owed mark stays, for the next pass to settle.trigger=data-delete).W07-048 (found by the V2 review): the delete's re-score covered
max(21, held trigger)and dropped the cut-short pass's own window. A 4,000-day effort re-score or timestamp heal cut short by a delete therefore came back as 21 days, and its caller saw the lock free and set its done flag. The pass now folds its width into the held window when a delete is in flight and the pass was dropped or owed a re-pass.Evidence
DataDeleteDuringRescoreTests.deleteRanDuringPass, and removing the hold fails the folded window. Removing the drop fails only the drop's log line: the wait alone already keeps the rows out.testADroppedWidePassKeepsItsWindowInTheDeletesRescorefails with W07-048's fold removed.computing, the hold returns to 0, and exactly one re-score fires.Strand/,StrandiOS*,NOOPWatchandPackages/goes through the hold.main:xcodebuild test, schemeStrand(macOS): 2,151 tests, 0 failures, 2 skipped.