[Onyx] Investigate NitroSQLiteError: [NativeNitroSQLiteException][SqlExecutionError] database or disk is full storage error

Coming from #87844.

Summary

Let's investigate the following Onyx storage error: NitroSQLiteError: [NativeNitroSQLiteException][SqlExecutionError] database or disk is full, which accounts for 89,174 (2.7%) of the 3,364,645 analysed errors during a time span of 1 month. Full details about the investigation and methodology are in #87844.

Platforms	Top Methods
Other/Native (89K), Android Native (153)	`setWithRetry` (89K)

Retries Exhausted	Evictions	No Eviction	Quota Checks	>= 95% Full	Mean % Full	Quota Check Failed
10	88,998	106	60,468	0	2.4%	0

60,468 quota checks with mean 2.4% of device disk used — the SQLite database itself is relatively small compared to overall device storage, but the device's free space might have been exhausted by other apps/data
Native-only (NitroSQLite) — very few requests exhausted retries (10), suggesting eviction generally resolves the issue

Action Items

Investigate with Sentry, VictoriaLogs and Fullstory to understand user scenarios — are these devices with very low storage, or are there specific app states that cause the database to grow unusually large?
106 requests hit the "no evictable keys" dead end — review whether more Onyx keys could be added to the evictableKeys list

106 requests hit the "no evictable keys" dead end — review whether more Onyx keys could be added to the evictableKeys list

For last 30 days we have total 187 events but these are logged in bursts. VL query

_time:30d "Storage operation failed after 5 retries" "database or disk is full"
  | stats by (email, _time:60m) count() as failures_in_burst
  | stats
      count() as incidents,
      count_uniq(email) as affected_users,
      sum(failures_in_burst) as total_failures

shows that we have 13 unique users and 15 separate incidents where eviction didn't help. It means that 13 users encountered this situation only once

@Julesssss

I've checked the user with the most of disk is full error. There is one storm of these storage errors which cooccurs with massive amounts of OnyxDerived recomputations .

This is the same loop I've found in #87873 investigation and my proposal from https://expensify.slack.com/archives/C08CZDJFJ77/p1781601596606719 will also help here.

Having this in mind and the previous comment that affected users are affected just once for most case it seems like users have temporary storage problem on their devices.

I recommend to not investigate this further and treat as duplicate of #87873 cc: @Julesssss

I recommend to not investigate this further and treat as duplicate

👍

Navigation Menu

Repository navigation

[Onyx] Investigate `NitroSQLiteError: [NativeNitroSQLiteException][SqlExecutionError] database or disk is full` storage error #87868

Summary

Action Items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[Onyx] Investigate NitroSQLiteError: [NativeNitroSQLiteException][SqlExecutionError] database or disk is full storage error #87868

Description

Summary

Action Items

Activity

sosek108 commented on Jun 16, 2026

sosek108 commented on Jun 16, 2026

Julesssss commented on Jun 17, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

[Onyx] Investigate `NitroSQLiteError: [NativeNitroSQLiteException][SqlExecutionError] database or disk is full` storage error #87868