Context
From the 2026-07-27 concurrency-fix series (patch 0003). Generated upsert had a check-then-act race: pk_map.get(&pk) then update()/insert(), so a concurrent delete between check and update made upsert return NotFound (observed ~5% of full-suite runs via tests/worktable/vacuum.rs::vacuum_parallel_with_upserts).
Current state (correct, but not the ideal design)
The fix in gen_table_upsert_fn (codegen/src/generators/{persist,in_memory}/table/impls.rs) is an optimistic retry loop:
update() hits NotFound → retry as insert.insert() hits WorkTableError::PrimaryAlreadyExists (typed variant added for this) → retry as update.- Secondary unique-index conflicts propagate (retrying can never succeed).
Convergence argument: a retry is only taken when a concurrent delete/insert flipped the key's existence between the check and the operation, so the loop cannot spin without the key changing state each iteration. Measured 0/100 full-suite failures. But it has no linearization point, and a review flagged theoretical starvation under adversarial same-key churn.
The better design
Now that lock acquisition is atomic (LockMap::get_or_insert_with, patch 0001), the lock map can hand out a row lock for a pk whether or not the row exists. The clean upsert:
- Acquire the full row lock for
pk once.
- Check
pk_map under that serialization.
- Perform insert-or-update while holding it.
Blocker: update() and insert() internally acquire the same row lock, so a lock-holding upsert deadlocks against its own callees. The prerequisite is factoring non-locking inner variants (e.g. update_locked/insert_locked) that the public methods and upsert both wrap. delete() also takes the row lock, which is exactly why the held-lock upsert excludes the racing delete.
Verification hints
vacuum_parallel_with_upserts (3000 upserts racing a delete task) is the stress test; it must stay 0/N. Also re-check PrimaryAlreadyExists remains reachable from plain insert() (tests match on it).
Context
From the 2026-07-27 concurrency-fix series (patch 0003). Generated
upserthad a check-then-act race:pk_map.get(&pk)thenupdate()/insert(), so a concurrentdeletebetween check and update made upsert returnNotFound(observed ~5% of full-suite runs viatests/worktable/vacuum.rs::vacuum_parallel_with_upserts).Current state (correct, but not the ideal design)
The fix in
gen_table_upsert_fn(codegen/src/generators/{persist,in_memory}/table/impls.rs) is an optimistic retry loop:update()hitsNotFound→ retry as insert.insert()hitsWorkTableError::PrimaryAlreadyExists(typed variant added for this) → retry as update.Convergence argument: a retry is only taken when a concurrent delete/insert flipped the key's existence between the check and the operation, so the loop cannot spin without the key changing state each iteration. Measured 0/100 full-suite failures. But it has no linearization point, and a review flagged theoretical starvation under adversarial same-key churn.
The better design
Now that lock acquisition is atomic (
LockMap::get_or_insert_with, patch 0001), the lock map can hand out a row lock for a pk whether or not the row exists. The clean upsert:pkonce.pk_mapunder that serialization.Blocker:
update()andinsert()internally acquire the same row lock, so a lock-holding upsert deadlocks against its own callees. The prerequisite is factoring non-locking inner variants (e.g.update_locked/insert_locked) that the public methods and upsert both wrap.delete()also takes the row lock, which is exactly why the held-lock upsert excludes the racing delete.Verification hints
vacuum_parallel_with_upserts(3000 upserts racing a delete task) is the stress test; it must stay 0/N. Also re-checkPrimaryAlreadyExistsremains reachable from plaininsert()(tests match on it).