Repository navigation
build: add a wa-sqlite rebuild recipe with a 125-database attach limit - #3
Draft
whygee-dev wants to merge 2 commits into
Draft
whygee-dev wants to merge 2 commits into
whygee-dev wants to merge 2 commits into
Conversation
The wa-sqlite.wasm published in @journeyapps/wa-sqlite 1.7.0 keeps SQLite's default limit of 10 attached databases per connection, and nothing in this repository showed how to raise it. wasm/Dockerfile rebuilds the same wa-sqlite commit (e4bb5532c8, the one published as 1.7.0: SQLite 3.53.0, PowerSync core v0.4.13) with SQLITE_MAX_ATTACHED=125 and the upstream flags. It pins the emsdk 4.0.9 image by digest and checks the SHA-256 of the downloaded PowerSync core library and of the generated SQLite amalgamation. `pnpm build:wasm` writes wasm/dist and checks both files against wasm/SHA256SUMS. Two builds with --no-cache on one host gave byte-identical files. Other hosts were not tried. The pinned emsdk image is amd64 only, so the build runs emulated on arm64 hosts and takes about nine minutes there.
`pnpm benchmark:wasm` runs the same SQL through Memory Write-Ahead VFS with the wa-sqlite.wasm from @journeyapps/wa-sqlite 1.7.0 and with the one built by wasm/Dockerfile: four reads and one insert over a seeded synthetic database of 30,000 rows, 16 runs per arm in alternating blocks, each in a fresh browser context and worker. Every run must return the same row counts and result hashes. The published run shows the rebuilt build is not faster: its medians are 0.4% to 5.0% above the npm build's on the SQL workloads, with overlapping ranges, and instantiation is 0.67 ms shorter. The README now says to rebuild for the attach limit, not for speed. The machine was not idle during the run; the load average is in the result file.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The
wa-sqlite.wasmpublished in@journeyapps/wa-sqlite1.7.0 lets one connection attach at most 10 databases, and this repository did not show how to raise that. An application that attaches many databases through this VFS had to work out its own build.pnpm build:wasmrunswasm/Dockerfile, which rebuilds the wa-sqlite commit published as 1.7.0 (SQLite 3.53.0, PowerSync core v0.4.13) withSQLITE_MAX_ATTACHED=125and the upstream flags, writeswasm/dist/wa-sqlite.mjsandwasm/dist/wa-sqlite.wasm, and checks both againstwasm/SHA256SUMS. The emsdk 4.0.9 image is pinned by digest, and the build checks the SHA-256 of the downloaded PowerSync core library and of the generated SQLite amalgamation. Another limit is one--build-arg.pnpm benchmark:wasmruns four reads and one insert over a seeded synthetic database of 30,000 rows through this VFS with each build, 16 runs per arm in alternating blocks, and fails unless every run returns the same row counts and result hashes.Nothing in
src/changes, and the built files are not committed.Validation
pnpm build:wasm: two builds with--no-cacheon one host produced byte-identical files, matchingwasm/SHA256SUMS; both input hash checks passed in each.pnpm benchmark:wasm: passed twice. The published result isbenchmarks/wasm-build-comparison/results/npm-1.7.0-vs-rebuilt.json. The machine was not idle (1-minute load average 8.43 at the start, 7.56 at the end, 18 logical CPUs); the second run, under a higher load, also put every SQL workload within 5%.pnpm format:check,pnpm check,pnpm build,pnpm test:benchmark-publisher(8 passed),pnpm test:browser(27 passed): pass.Not run: a build on another host, so reproducibility across machines is not shown. The pinned emsdk image is amd64 only; on an arm64 host the build runs emulated and took about nine minutes. Not run: the benchmark in WebKit or Firefox, on an idle machine, or with attached databases (the workload uses one database).