[Due for payment 2026-04-14] [$250] [Sentry] fix The database connection is closing issue

@fabioh8010

Sentry issue https://expensify.sentry.io/issues/7092239780

Thread https://expensify.slack.com/archives/C05LX9D6E07/p1770496102994319

Please re-state the problem that we are trying to solve in this issue.

When Onyx operations attempt to create an IndexedDB transaction, the browser may have already closed the database connection, causing:
InvalidStateError: Failed to execute 'transaction' on 'IDBDatabase': The database connection is closing.

This affects ~58,700 occurrences across 430 users on all browsers (Chrome: 31K, Safari: 18K, Firefox: 5K, Mobile Safari: 3K, Edge: 1K). The error is an unhandled promise rejection.

What is the root cause of that problem?

The existing db.onclose handler (line 22) sets dbp = undefined when the browser closes the connection, which protects future calls to getDB(). However, any operations that have already resolved the dbp promise still hold a stale db reference and proceed to call db.transaction() at line 59 — which throws InvalidStateError on the now-closing connection.

This can be triggered by:

• The browser closing idle IDB connections (Safari is especially aggressive, but Chrome/Firefox do this too under memory pressure or tab suspension)
• Another tab triggering a version upgrade (versionchange event — currently unhandled)
• The verifyStoreExists path closing the DB (line 39) while concurrent operations still hold the old reference

What changes do you think we should make in order to solve the problem?

All changes are in lib/storage/providers/IDBKeyValProvider/createStore.ts:

Retry mechanism with a single retry on InvalidStateError
   — Extract the transaction chain (getDB → verifyStoreExists → db.transaction) into openTransactionAndExecute()
   — Wrap the call in .catch(): if the error is InvalidStateError, reset dbp = undefined and retry once with a fresh connection
   — If the retry also fails, the error is thrown normally and still surfaces in Sentry exactly as it does today
Diagnostic logging on every retry (does NOT hide the issue)
   — Log at alert level with: dbName, storeName, txMode, and closedBy
   — closedBy tracks why the connection was closed: 'browser' (user-agent closed it), 'versionchange' (cross-tab upgrade), 'verifyStoreExists' (missing store path), or 'unknown'
   — This gives us more visibility than today (where we only get a crash count with no context)
onversionchange handler
— When another tab triggers a DB version upgrade, proactively close the connection and reset dbp
— Without this, cross-tab upgrades can block indefinitely and in-flight operations crash on the stale connection
onclose handler now logs
— The existing onclose handler now logs when the browser unexpectedly closes the connection, giving us data on browser-initiated closures even when no retry is needed

Key safety guarantees:

• Data is never lost — the transaction fails before any data is written; the retry completes successfully with a valid connection
• Only one retry is attempted — no infinite loops
• If the retry fails, the error propagates normally and Sentry captures it as before
• Normal operations (no InvalidStateError) are completely unaffected — zero performance impact
• The closedBy logging gives us instrumentation to continue investigating the root cause of the spike

cc @fabioh8010

Issue Owner

Current Issue Owner: @mallenexpensify

Upwork Automation - Do Not Edit

Upwork Job URL: https://www.upwork.com/jobs/~022029317821194916452
Upwork Job ID: 2029317821194916452
Last Price Increase: 2026-03-04

Proposal

Please re-state the problem that we are trying to solve in this issue.

When Onyx operations attempt to create an IndexedDB transaction, the browser may have already closed the database connection, causing InvalidStateError: Failed to execute 'transaction' on 'IDBDatabase': The database connection is closing. This affects ~58,700 occurrences across 430 users on all browsers (Chrome: 31K, Safari: 18K, Firefox: 5K, Mobile Safari: 3K, Edge: 1K). The error is an unhandled promise rejection that currently has zero error handling.

What is the root cause of that problem?

There are two race conditions in react-native-onyx's createStore.js, both converging on the same unprotected db.transaction() call at the return statement:

return (txMode, callback) => getDB()
    .then(verifyStoreExists)
    .then((db) => callback(db.transaction(storeName, txMode).objectStore(storeName)));

Race Condition A (primary — ~58K occurrences): getDB() returns the cached dbp promise which resolves to a db object. Between the promise resolving and db.transaction() executing, the browser closes the connection (Safari does this aggressively for idle connections; Chrome/Firefox do it under memory pressure or tab suspension). The db.onclose handler fires and sets dbp = undefined, which helps future calls to getDB() re-open the connection — but the in-flight .then() chain already has the stale db reference. db.transaction() throws InvalidStateError synchronously inside the .then() callback.

Race Condition B (secondary): The verifyStoreExists function (added in PR #684) calls db.close() when it detects the store is missing and needs to recreate it with a version bump. This immediately puts the connection into "closing" state. Any concurrent operation that already resolved getDB() with this same db object will hit InvalidStateError when it reaches db.transaction().

Why the error is unhandled: All 9 storage operations (7 via idb-keyval, 2 direct calls in multiMerge/multiSet) flow through the same single choke point. None of them have any try/catch or error handling around db.transaction(). The tryOrDegradePerformance wrapper in storage/index.js only catches synchronous throws — since db.transaction() is called inside an async .then() chain, the InvalidStateError becomes a rejected promise that bypasses the try/catch entirely. Additionally, tryOrDegradePerformance only checks for "Internal error opening backing store", not InvalidStateError.

There is also no onversionchange handler, meaning cross-tab database upgrades can forcibly close connections and trigger the same error.

What changes do you think we should make in order to solve the problem?

All changes in react-native-onyx's lib/storage/providers/IDBKeyValProvider/createStore.ts:

Wrap db.transaction() with retry on InvalidStateError — Extract the transaction chain into a helper. If db.transaction() throws InvalidStateError, reset dbp = undefined and retry once with a fresh connection. If the retry also fails, propagate the error normally (Sentry still captures it). This is the standard approach used by other libraries facing the same issue (localForage, Firebase, Workbox).
Add onversionchange handler — When another tab triggers a DB version upgrade, proactively close the connection and reset dbp before the browser force-closes it. This prevents the race window where in-flight operations use a connection that's about to be killed.
Add diagnostic logging on retry — Log dbName, storeName, txMode, and the reason the connection was closed (browser, versionchange, verifyStoreExists, or unknown). This gives visibility into which trigger is most common without hiding the error.
Enhance onclose handler with logging — The existing onclose handler silently resets dbp. Adding a log line gives production visibility into browser-initiated closures.

What alternative solutions did you explore? (Optional)

Degrade to MemoryOnlyProvider: Too aggressive — the IDB connection can be re-established with a single retry. Degrading to memory-only loses all persistent state.
Serialize all operations through a queue: Prevents Race Condition B but doesn't help with Race Condition A (browser-initiated closures), and adds significant overhead to all IDB operations.
Check db.closePending before db.transaction(): IndexedDB doesn't expose a reliable connection state property — the only way to know the connection is closing is to catch the error.

Detailed Investigation

Evidence Chain

The single choke point — createStore.js line 83: db.transaction(storeName, txMode) is called inside a .then() with zero error handling. Every storage operation flows through this line.
Stale reference problem — db.onclose (line 54) sets dbp = undefined for future callers, but the promise .then() chain already has the resolved db object. The stale db reference is used to call db.transaction(), which throws.
No upstream protection — idb-keyval v6.2.1 has the identical unprotected pattern. Its functions (get, set, del, getMany, delMany, clear, keys) simply call customStore(txMode, callback) with no try/catch.
tryOrDegradePerformance doesn't help — It wraps fn() in try/catch, but fn() returns a promise. The InvalidStateError is thrown inside an async .then() chain, becoming a rejected promise that the synchronous try/catch cannot intercept. The error message also doesn't match either of the two checked patterns.
verifyStoreExists secondary race — Added in PR #684 (merged Oct 2025) to handle corrupted databases from third-party scripts. When the store is missing, it calls db.close() and reopens with a version bump. Any concurrent operation holding the old db reference will fail.
Missing onversionchange — No handler exists anywhere in the codebase. Cross-tab upgrades (including those triggered by verifyStoreExists) can force-close connections on other tabs.

Relevant Code

createStore.js:81-83 — The unprotected db.transaction() call (single choke point)
createStore.js:50-54 — db.onclose handler that only helps future calls
createStore.js:62-79 — verifyStoreExists with db.close() race
storage/index.js:62-85 — tryOrDegradePerformance that doesn't catch this error
idb-keyval/dist/index.cjs:32-46 — Upstream functions with zero error handling

Open Questions

What percentage of the ~58,700 errors come from Safari vs other browsers? The issue reports Chrome: 31K, Safari: 18K — confirming this isn't Safari-only (despite the original code comment suggesting Safari is the culprit).
How often does verifyStoreExists actually find the store missing? If rarely, Race Condition A (browser-initiated closures) dominates.
Does the app commonly run in multiple tabs? If so, the onversionchange handler is critical.

Next Steps for Contributor+ team: Reply with @MelvinBot implement this to create a draft PR, or @MelvinBot <your feedback> to refine this analysis.

@Krishna2323

Current assignee @Krishna2323 is eligible for the External assigner, not assigning anyone new.

Job added to Upwork: https://www.upwork.com/jobs/~022029317821194916452

@Krishna2323

@Krishna2323 Huh... This is 4 days overdue. Who can take care of this?

Not overdue, draft PR is up. Will open for review shortly.

@Krishna2323

Payment Summary

Upwork Job

Reviewer: @Krishna2323 owed $250 via NewDot
Reviewer: @mkhutornyi owed $250 via NewDot

BugZero Checklist (@mallenexpensify)

(if NewFeature) I have created a PR for any necessary HelpDot updates, or I confirmed no updates are necessary.
I have verified the correct assignees and roles are listed above and updated the necessary manual offers
I have verified that there are no duplicate or incorrect contracts on Upwork for this job (https://www.upwork.com/ab/applicants/2029317821194916452/hired)
I have verified the PR was not reverted
I have applied any discounts due to bugs/regressions introduced by this PR
I have paid out the Upwork contracts or cancelled the ones that are incorrect
I have verified the payment summary above is correct

Is this due payment or part of a project? Plz tag me if it's due.

@mallenexpensify

@mallenexpensify it's due payment. Checklist not needed.

@Krishna2323

Payment Summary

Contributor+: @Krishna2323 due $250 via NewDot
Contributor+: @mkhutornyi due $250 via NewDot

Requested in ND.

@Krishna2323

@Krishna2323 $250 approved in New Expensify

@mkhutornyi

approved $250 for @mkhutornyi

Navigation Menu

Repository navigation

[Due for payment 2026-04-14] [$250] [Sentry] fix The database connection is closing issue #84192

Please re-state the problem that we are trying to solve in this issue.

What is the root cause of that problem?

What changes do you think we should make in order to solve the problem?

Key safety guarantees:

Proposal

Please re-state the problem that we are trying to solve in this issue.

What is the root cause of that problem?

What changes do you think we should make in order to solve the problem?

What alternative solutions did you explore? (Optional)

Evidence Chain

Relevant Code

Open Questions

41 remaining items

Payment Summary

BugZero Checklist (@mallenexpensify)

Payment Summary

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[Due for payment 2026-04-14] [$250] [Sentry] fix The database connection is closing issue #84192

Description

Please re-state the problem that we are trying to solve in this issue.

What is the root cause of that problem?

What changes do you think we should make in order to solve the problem?

Key safety guarantees:

Activity

MelvinBot commented on Mar 4, 2026

Proposal

Please re-state the problem that we are trying to solve in this issue.

What is the root cause of that problem?

What changes do you think we should make in order to solve the problem?

What alternative solutions did you explore? (Optional)

Evidence Chain

Relevant Code

Open Questions

melvin-bot commented on Mar 4, 2026

melvin-bot commented on Mar 4, 2026

melvin-bot commented on Mar 10, 2026

Krishna2323 commented on Mar 10, 2026

41 remaining items

melvin-bot commented on Apr 14, 2026

Payment Summary

BugZero Checklist (@mallenexpensify)

mallenexpensify commented on Apr 15, 2026

Krishna2323 commented on Apr 15, 2026

mallenexpensify commented on Apr 16, 2026

Payment Summary

Krishna2323 commented on Apr 17, 2026

laurenreidexpensify commented on Apr 20, 2026

trjExpensify commented on May 6, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions