diff --git a/README.md b/README.md index b125bbc..fab3bd0 100644 --- a/README.md +++ b/README.md @@ -160,6 +160,26 @@ registerMemoryWriteAheadSharedFiles({ Structured cloning shares the underlying `SharedArrayBuffer`s; it does not copy the database bytes. +## Raising the attached-database limit + +The `wa-sqlite.wasm` published in `@journeyapps/wa-sqlite` 1.7.0 keeps SQLite's default `SQLITE_MAX_ATTACHED=10`, so one connection can attach at most 10 databases. [`wasm/Dockerfile`](https://github.com/whygee-dev/memory-write-ahead-vfs/blob/main/wasm/Dockerfile) rebuilds the same wa-sqlite source with a higher limit, 125 by default, which is SQLite's ceiling: + +```bash +pnpm build:wasm +``` + +This writes `wasm/dist/wa-sqlite.mjs` and `wasm/dist/wa-sqlite.wasm`, then checks both against [`wasm/SHA256SUMS`](https://github.com/whygee-dev/memory-write-ahead-vfs/blob/main/wasm/SHA256SUMS). For another limit, run Docker directly; the hash check only applies to the default: + +```bash +docker build --build-arg SQLITE_MAX_ATTACHED=32 --output wasm/dist wasm +``` + +Copy both files into your app and import the rebuilt `wa-sqlite.mjs` in place of `@journeyapps/wa-sqlite/dist/wa-sqlite.mjs`. Keep `@journeyapps/wa-sqlite` installed for `SQLite.Factory` and the VFS base classes. The glue loads `wa-sqlite.wasm` from its own directory unless you pass `locateFile` or `wasmBinary` to the factory. + +The recipe pins the inputs that decide the output bytes: the `emscripten/emsdk:4.0.9` image by digest, [powersync-ja/wa-sqlite](https://github.com/powersync-ja/wa-sqlite) at `e4bb5532c8`, the commit published as `@journeyapps/wa-sqlite` 1.7.0 (SQLite 3.53.0, PowerSync core v0.4.13), and the SHA-256 of the downloaded PowerSync core library and of the generated SQLite amalgamation. It builds only the synchronous `wa-sqlite` variant this VFS uses, with the upstream flags (`-Oz -flto`, `WASM_BIGINT=0`); the attach limit is the only configuration change. Two builds with `--no-cache` on the same host produced byte-identical files; other hosts were not tried. + +The npm artifact is not the same build with one flag changed: its compile options report clang 23, while the recipe uses Emscripten 4.0.9 (clang 21), the version the fork's CI pins. Rebuild for the attach limit, not for speed: in [the comparison benchmark](https://github.com/whygee-dev/memory-write-ahead-vfs/blob/main/benchmarks/wasm-build-comparison/README.md), the rebuilt build ran within 5% of the npm build on every SQL workload, slightly slower on each. + ## Capacity options ```ts diff --git a/benchmarks/wasm-build-comparison/README.md b/benchmarks/wasm-build-comparison/README.md new file mode 100644 index 0000000..11a56e5 --- /dev/null +++ b/benchmarks/wasm-build-comparison/README.md @@ -0,0 +1,77 @@ +# wa-sqlite build comparison + +This benchmark compares the `wa-sqlite.wasm` published in `@journeyapps/wa-sqlite` 1.7.0 ("stock") with the one built by [`wasm/Dockerfile`](../../wasm/Dockerfile) ("rebuilt"). Both arms run the same SQL through Memory Write-Ahead VFS in a dedicated worker, with the same wa-sqlite JavaScript API; only the Emscripten glue and the wasm binary change. + +## What differs between the two builds + +| | stock | rebuilt | +| ------------------------------------------ | ---------------------------------- | ------------------------------------------ | +| Source | npm `@journeyapps/wa-sqlite` 1.7.0 | [`wasm/Dockerfile`](../../wasm/Dockerfile) | +| SQLite | 3.53.0 | 3.53.0 | +| Compiler in `PRAGMA compile_options` | clang-23.0.0 | clang-21.0.0 | +| `MAX_ATTACHED` in `PRAGMA compile_options` | 10 | 125 | +| `wa-sqlite.wasm` size | 1,128,032 bytes | 1,129,991 bytes | +| `wa-sqlite.wasm` SHA-256 | `d167becf5a69…` | `91defe9a35e0…` | + +A difference between the arms can come from the compiler or from the attach limit; this benchmark does not separate the two. + +## Result + +Run on 2026-10-07 with Chromium 148.0.7778.96 on an Apple M5 Pro (18 logical CPUs, darwin arm64), 16 runs per arm. The machine was not idle: the 1-minute load average was 8.43 when the benchmark started and 7.56 when it ended. + +| Workload | stock, ms: median (min–max) | rebuilt, ms: median (min–max) | rebuilt against stock | +| ----------------- | --------------------------- | ----------------------------- | --------------------- | +| `filterScan` | 46.72 (43.83–52.36) | 46.89 (45.01–50.15) | +0.4% | +| `join` | 11.86 (11.50–15.02) | 12.10 (11.46–15.28) | +2.0% | +| `groupedLeftJoin` | 12.80 (11.05–14.01) | 13.44 (12.57–14.94) | +5.0% | +| `orderByText` | 45.63 (41.88–54.92) | 47.01 (44.01–50.61) | +3.0% | +| `insert` | 23.89 (22.78–25.95) | 24.48 (23.31–26.06) | +2.5% | +| `seed` | 164.15 (155.18–184.40) | 167.60 (158.62–172.60) | +2.1% | +| `instantiate` | 3.93 (3.62–4.43) | 3.26 (3.00–3.51) | −17.0% | + +The rebuilt build is not faster. On the five SQL workloads and the seed, its median is 0.4% to 5.0% above the stock build's, and the two ranges overlap on every one of them. Instantiation is 0.67 ms shorter. Every run of both arms returned the same row counts and the same result hashes. + +## Workload + +A seeded generator in [`dataset.ts`](./dataset.ts) builds the same database in every run: 1,000 categories and 30,000 items with 20 columns mixing text, integers, reals, and nullable values, plus an index on `item.category_id`. + +| Workload | What runs | +| ----------------- | ------------------------------------------------------------------------------------------------------- | +| `filterScan` | `SELECT *` with a filter on two unindexed columns, returning 17,900 of the 30,000 rows | +| `join` | items joined to their category, filtered on three of the 20 category families, ordered by item id | +| `groupedLeftJoin` | every category left-joined to its active items, grouped with `COUNT`, `SUM`, `AVG`, `MAX` | +| `orderByText` | 4 columns of all 30,000 items ordered by a text column, without an index | +| `insert` | 5,000 rows with all 20 columns in one transaction through one prepared statement, into an indexed table | +| `instantiate` | the glue's module factory compiling and instantiating the wasm bytes, once per run | +| `seed` | creating the schema and inserting the 31,000 rows and the index in one transaction, once per run | + +Each read is timed from `prepare` until every row has been read into JavaScript values with `sqlite3.row`, so the time includes wa-sqlite's per-row JavaScript overhead, which is the same in both arms. + +## Method + +- One run opens a new browser context and page, starts a dedicated worker, loads one arm's glue, fetches its wasm bytes, and passes them as `wasmBinary`. The worker opens a Memory Write-Ahead VFS database in WAL mode with a 64 MiB page cache, which holds the whole database, so warm reads do not go through the VFS read path. +- Each run executes 2 warmup rounds, which are discarded, then 5 measured rounds. A round runs the four reads, then the insert. A run's value for a workload is the median of its 5 measured rounds; the table reports the median, minimum, and maximum of those values across runs. +- Arms alternate in ABBA and BAAB blocks so that each arm runs equally often early and late. One priming run per arm, before the first block, is discarded. +- Every run hashes each read result and the inserted rows; all runs of both arms must produce the same hashes. The worker also hashes the wasm bytes it loaded and the test checks them against the files on disk. +- [Raw runs](./results/npm-1.7.0-vs-rebuilt.json) include every measured round and the 1-minute load average when each run started. + +## What this does not measure + +- Attached databases: the workload uses one database, so it does not exercise the code paths that the higher attach limit changes. +- Other wa-sqlite variants (asynchronous, JSPI, multiple ciphers), other VFS implementations, other browsers, or other machines. +- Memory use, download size, page-load time, or the cost of fetching the wasm over a network. +- Concurrency between workers or connections; every run uses one connection. +- Prepared-statement reuse for reads; each read prepares its statement again. + +## Reproduce + +Requirements: Docker, Node.js 20+, pnpm 10.27.0, and Playwright Chromium. + +```bash +pnpm install --frozen-lockfile +pnpm exec playwright install chromium +pnpm build:wasm +pnpm benchmark:wasm +``` + +The raw result is written to `benchmark-results/wasm-build-comparison.json`. `BENCHMARK_RUNS_PER_ARM` (even, default 16) changes the number of measured runs, and `WASM_BENCHMARK_CANDIDATE_DIR` points the rebuilt arm at another directory inside the repository, for example a build with a different `SQLITE_MAX_ATTACHED`. diff --git a/benchmarks/wasm-build-comparison/benchmark.spec.ts b/benchmarks/wasm-build-comparison/benchmark.spec.ts new file mode 100644 index 0000000..2ca436e --- /dev/null +++ b/benchmarks/wasm-build-comparison/benchmark.spec.ts @@ -0,0 +1,246 @@ +import { createHash } from 'node:crypto'; +import { existsSync, readFileSync } from 'node:fs'; +import { mkdir, writeFile } from 'node:fs/promises'; +import { cpus, loadavg, platform, arch } from 'node:os'; +import { dirname, isAbsolute, join, relative, resolve } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { expect, test } from '@playwright/test'; +import memoryWriteAheadPackage from '../../package.json' with { type: 'json' }; + +import { DATASET, WORKLOAD } from './dataset.js'; +import type { WasmBuildRunRequest, WasmBuildRunResult, WorkloadName } from './worker.js'; + +type ArmName = 'stock' | 'rebuilt'; + +type Arm = { + directory: string; + name: ArmName; + source: string; +}; + +type Summary = { + max: number; + median: number; + min: number; + runs: number; +}; + +const REPOSITORY_ROOT = resolve(dirname(fileURLToPath(import.meta.url)), '../..'); + +const readPositiveEvenInteger = (name: string, fallback: number) => { + const value = Number(process.env[name] ?? fallback); + if (!Number.isInteger(value) || value < 2 || value % 2 !== 0) { + throw new Error(`${name} must be a positive even integer; received ${process.env[name]}`); + } + return value; +}; + +const readPackageVersion = (packageName: string) => { + const packageJson = JSON.parse( + readFileSync(join(REPOSITORY_ROOT, 'node_modules', packageName, 'package.json'), 'utf8') + ) as { version: string }; + return packageJson.version; +}; + +const sha256File = (path: string) => createHash('sha256').update(readFileSync(path)).digest('hex'); + +const round = (value: number) => Math.round(value * 100) / 100; + +const median = (values: readonly number[]) => { + const sorted = [...values].sort((left, right) => left - right); + const middle = Math.floor(sorted.length / 2); + const upper = sorted[middle] ?? Number.NaN; + return sorted.length % 2 === 0 ? ((sorted[middle - 1] ?? Number.NaN) + upper) / 2 : upper; +}; + +const summarize = (values: readonly number[]): Summary => ({ + max: round(Math.max(...values)), + median: round(median(values)), + min: round(Math.min(...values)), + runs: values.length, +}); + +const resolveCandidateDirectory = () => { + const directory = resolve(REPOSITORY_ROOT, process.env.WASM_BENCHMARK_CANDIDATE_DIR ?? 'wasm/dist'); + const relativeDirectory = relative(REPOSITORY_ROOT, directory); + if (relativeDirectory.startsWith('..') || isAbsolute(relativeDirectory)) { + throw new Error('WASM_BENCHMARK_CANDIDATE_DIR must be inside the repository so Vite can serve it'); + } + if (!existsSync(join(directory, 'wa-sqlite.mjs')) || !existsSync(join(directory, 'wa-sqlite.wasm'))) { + throw new Error(`${relativeDirectory} has no wa-sqlite.mjs and wa-sqlite.wasm; run pnpm build:wasm first`); + } + return relativeDirectory.split('\\').join('/'); +}; + +const createSchedule = (runsPerArm: number): ArmName[] => + Array.from({ length: runsPerArm / 2 }, (_, block): ArmName[] => + block % 2 === 0 ? ['stock', 'rebuilt', 'rebuilt', 'stock'] : ['rebuilt', 'stock', 'stock', 'rebuilt'] + ).flat(); + +test('compares the stock and rebuilt wa-sqlite builds through Memory Write-Ahead VFS', async ({ browser, baseURL }) => { + test.setTimeout(60 * 60_000); + + const runsPerArm = readPositiveEvenInteger('BENCHMARK_RUNS_PER_ARM', 16); + const arms: readonly Arm[] = [ + { + directory: 'node_modules/@journeyapps/wa-sqlite/dist', + name: 'stock', + source: `npm @journeyapps/wa-sqlite ${readPackageVersion('@journeyapps/wa-sqlite')}`, + }, + { directory: resolveCandidateDirectory(), name: 'rebuilt', source: 'wasm/Dockerfile' }, + ]; + const armByName = new Map(arms.map((arm) => [arm.name, arm])); + const artifacts = arms.map((arm) => ({ + directory: arm.directory, + glueSha256: sha256File(join(REPOSITORY_ROOT, arm.directory, 'wa-sqlite.mjs')), + name: arm.name, + source: arm.source, + wasmSha256: sha256File(join(REPOSITORY_ROOT, arm.directory, 'wa-sqlite.wasm')), + })); + + const runArm = async (armName: ArmName) => { + const arm = armByName.get(armName); + if (!arm) { + throw new Error(`Unknown arm ${armName}`); + } + const context = await browser.newContext({ baseURL: baseURL ?? 'http://127.0.0.1:5175' }); + try { + const page = await context.newPage(); + await page.goto('/test-host.html'); + const request: WasmBuildRunRequest = { + glueUrl: `/${arm.directory}/wa-sqlite.mjs`, + wasmUrl: `/${arm.directory}/wa-sqlite.wasm`, + }; + return await page.evaluate( + (runRequest) => + new Promise((resolveRun, rejectRun) => { + const worker = new Worker('/benchmarks/wasm-build-comparison/worker.ts', { type: 'module' }); + worker.onmessage = ( + event: MessageEvent< + { ok: true; result: WasmBuildRunResult } | { ok: false; message: string } + > + ) => { + worker.terminate(); + if (event.data.ok) { + resolveRun(event.data.result); + } else { + rejectRun(new Error(event.data.message)); + } + }; + worker.onerror = (event) => { + worker.terminate(); + rejectRun(new Error(event.message || 'Benchmark worker failed to load')); + }; + worker.postMessage(runRequest); + }), + request + ); + } finally { + await context.close(); + } + }; + + const loadAverageAtStart = loadavg().map(round); + for (const arm of arms) { + console.info(`[wasm-build-benchmark] priming ${arm.name}`); + await runArm(arm.name); + } + + const schedule = createSchedule(runsPerArm); + const runs: Array<{ arm: ArmName; loadAverage1m: number; result: WasmBuildRunResult; run: number }> = []; + for (const [index, armName] of schedule.entries()) { + const loadAverage1m = round(loadavg()[0] ?? 0); + console.info(`[wasm-build-benchmark] run ${index + 1}/${schedule.length} ${armName}`); + runs.push({ arm: armName, loadAverage1m, result: await runArm(armName), run: index + 1 }); + } + const loadAverageAtEnd = loadavg().map(round); + + const firstRun = runs[0]; + if (!firstRun) { + throw new Error('The benchmark recorded no runs'); + } + for (const run of runs) { + expect(run.result.crossOriginIsolated).toBe(true); + expect(run.result.fingerprints).toEqual(firstRun.result.fingerprints); + expect(run.result.rowCounts).toEqual(firstRun.result.rowCounts); + expect(run.result.build.wasmSha256).toBe(artifacts.find((artifact) => artifact.name === run.arm)?.wasmSha256); + } + + const workloadNames = Object.keys(firstRun.result.measuredMs) as WorkloadName[]; + const summaries = Object.fromEntries( + arms.map((arm) => { + const armRuns = runs.filter((run) => run.arm === arm.name).map((run) => run.result); + return [ + arm.name, + { + ...Object.fromEntries( + workloadNames.map((name) => [ + name, + summarize(armRuns.map((run) => median(run.measuredMs[name]))), + ]) + ), + instantiate: summarize(armRuns.map((run) => run.instantiateMs)), + seed: summarize(armRuns.map((run) => run.seedMs)), + }, + ]; + }) + ); + + const result = { + resultFormat: 'wasm-build-comparison-v1', + runDate: new Date().toISOString().slice(0, 10), + environment: { + browser: `Chromium ${browser.version()}`, + crossOriginIsolated: firstRun.result.crossOriginIsolated, + hardwareConcurrency: firstRun.result.hardwareConcurrency, + host: { + arch: arch(), + cpuModel: cpus()[0]?.model ?? 'unknown', + logicalCpus: cpus().length, + platform: platform(), + }, + loadAverageAtEnd, + loadAverageAtStart, + }, + runtimeVersions: { + memoryWriteAheadVfs: memoryWriteAheadPackage.version, + playwright: readPackageVersion('@playwright/test'), + vite: readPackageVersion('vite'), + waSqlite: readPackageVersion('@journeyapps/wa-sqlite'), + }, + configuration: { + dataset: DATASET, + primingRunsPerArm: 1, + runsPerArm, + schedule: 'ABBA/BAAB blocks, fresh browser context, page and worker per run', + summaryUnit: 'per-run median of measured rounds; instantiate and seed are one sample per run', + vfs: 'MemoryWriteAheadVFS in a dedicated worker', + workload: WORKLOAD, + }, + arms: artifacts.map((artifact) => { + const build = runs.find((run) => run.arm === artifact.name)?.result.build; + return { ...artifact, ...build }; + }), + fingerprints: firstRun.result.fingerprints, + rowCounts: firstRun.result.rowCounts, + summaries, + runs: runs.map(({ arm, loadAverage1m, result: { instantiateMs, measuredMs, seedMs }, run }) => ({ + arm, + instantiateMs: round(instantiateMs), + loadAverage1m, + measuredMs: Object.fromEntries( + Object.entries(measuredMs).map(([name, values]) => [name, values.map(round)]) + ), + run, + seedMs: round(seedMs), + })), + }; + + const outputPath = resolve( + REPOSITORY_ROOT, + process.env.BENCHMARK_OUTPUT ?? 'benchmark-results/wasm-build-comparison.json' + ); + await mkdir(dirname(outputPath), { recursive: true }); + await writeFile(outputPath, `${JSON.stringify(result, null, 2)}\n`, 'utf8'); + console.info(`[wasm-build-benchmark] ${JSON.stringify(summaries)}`); +}); diff --git a/benchmarks/wasm-build-comparison/dataset.ts b/benchmarks/wasm-build-comparison/dataset.ts new file mode 100644 index 0000000..d8e850b --- /dev/null +++ b/benchmarks/wasm-build-comparison/dataset.ts @@ -0,0 +1,212 @@ +export type SqlValue = number | string | null; + +export const DATASET = { + categories: 1_000, + categoryFamilies: 20, + insertRows: 5_000, + items: 30_000, + seed: 20_260_407, +} as const; + +export const WORKLOAD = { + cacheSizeKiB: 64 * 1024, + measuredRounds: 5, + warmupRounds: 2, +} as const; + +const SYLLABLES = [ + 'ka', + 'lo', + 'mi', + 'ren', + 'tus', + 'va', + 'dor', + 'pel', + 'qui', + 'sa', + 'ne', + 'bri', + 'zo', + 'fa', + 'gel', + 'hu', +]; +const STATUSES = ['active', 'active', 'active', 'pending', 'draft', 'retired', 'archived']; +const TAGS = Array.from({ length: 30 }, (_, index) => `tag-${String(index + 1).padStart(2, '0')}`); + +const itemTableSql = (table: string) => `CREATE TABLE ${table} ( + id INTEGER PRIMARY KEY, + code TEXT NOT NULL, + name TEXT NOT NULL, + description TEXT, + category_id INTEGER NOT NULL, + status TEXT NOT NULL, + priority INTEGER NOT NULL, + quantity INTEGER NOT NULL, + unit_price REAL NOT NULL, + weight_kg REAL, + width_mm REAL, + height_mm REAL, + created_at INTEGER NOT NULL, + updated_at INTEGER, + owner TEXT, + location TEXT NOT NULL, + tag TEXT, + is_active INTEGER NOT NULL, + score REAL NOT NULL, + note TEXT +)`; + +export const SCHEMA = [ + `CREATE TABLE category ( + id INTEGER PRIMARY KEY, + name TEXT NOT NULL, + family TEXT NOT NULL, + weight REAL NOT NULL, + parent_id INTEGER + )`, + itemTableSql('item'), + itemTableSql('item_insert'), +] as const; + +export const INDEXES = [ + 'CREATE INDEX idx_item_category ON item(category_id)', + 'CREATE INDEX idx_item_insert_category ON item_insert(category_id)', +] as const; + +const ITEM_COLUMNS = [ + 'id', + 'code', + 'name', + 'description', + 'category_id', + 'status', + 'priority', + 'quantity', + 'unit_price', + 'weight_kg', + 'width_mm', + 'height_mm', + 'created_at', + 'updated_at', + 'owner', + 'location', + 'tag', + 'is_active', + 'score', + 'note', +] as const; + +const placeholders = (count: number) => Array.from({ length: count }, () => '?').join(', '); + +export const INSERT_CATEGORY_SQL = `INSERT INTO category (id, name, family, weight, parent_id) VALUES (${placeholders(5)})`; +export const INSERT_ITEM_SQL = `INSERT INTO item (${ITEM_COLUMNS.join(', ')}) VALUES (${placeholders(ITEM_COLUMNS.length)})`; +export const INSERT_WORKLOAD_SQL = `INSERT INTO item_insert (${ITEM_COLUMNS.join(', ')}) VALUES (${placeholders( + ITEM_COLUMNS.length +)})`; + +export const READ_QUERIES = { + filterScan: ` + SELECT * + FROM item + WHERE is_active = 1 AND quantity >= 2500 + `, + join: ` + SELECT i.id, i.code, i.name, i.unit_price, i.quantity, c.name AS category_name, c.family + FROM item i + JOIN category c ON c.id = i.category_id + WHERE c.family IN ('family-03', 'family-07', 'family-11') + ORDER BY i.id + `, + groupedLeftJoin: ` + SELECT + c.family, + c.id, + COUNT(i.id) AS item_count, + SUM(i.quantity) AS total_quantity, + AVG(i.unit_price) AS average_price, + MAX(i.weight_kg) AS max_weight + FROM category c + LEFT JOIN item i ON i.category_id = c.id AND i.status = 'active' + GROUP BY c.id + ORDER BY c.family, c.id + `, + orderByText: ` + SELECT id, name, code, location + FROM item + ORDER BY name, id + `, +} as const; + +export type ReadQueryName = keyof typeof READ_QUERIES; + +const createRandom = (seed: number) => { + let state = seed >>> 0; + return () => { + state = (state + 0x6d2b79f5) >>> 0; + let value = state; + value = Math.imul(value ^ (value >>> 15), value | 1); + value ^= value + Math.imul(value ^ (value >>> 7), value | 61); + return ((value ^ (value >>> 14)) >>> 0) / 4_294_967_296; + }; +}; + +type Random = ReturnType; + +const pick = (random: Random, values: readonly T[]): T => { + const value = values[Math.floor(random() * values.length)]; + if (value === undefined) { + throw new Error('Cannot pick from an empty list'); + } + return value; +}; +const integer = (random: Random, min: number, max: number) => min + Math.floor(random() * (max - min + 1)); +const decimal = (random: Random, max: number) => Math.round(random() * max * 100) / 100; +const nullable = (random: Random, nullRate: number, value: () => T) => (random() < nullRate ? null : value()); +const word = (random: Random) => Array.from({ length: integer(random, 2, 4) }, () => pick(random, SYLLABLES)).join(''); +const sentence = (random: Random, minWords: number, maxWords: number) => + Array.from({ length: integer(random, minWords, maxWords) }, () => word(random)).join(' '); +const title = (random: Random) => + `${word(random)} ${word(random)}`.replace(/(^|\s)\S/g, (letter) => letter.toUpperCase()); + +export const generateCategories = (): SqlValue[][] => { + const random = createRandom(DATASET.seed); + return Array.from({ length: DATASET.categories }, (_, index) => [ + index + 1, + title(random), + `family-${String((index % DATASET.categoryFamilies) + 1).padStart(2, '0')}`, + decimal(random, 10), + index < DATASET.categoryFamilies ? null : integer(random, 1, DATASET.categoryFamilies), + ]); +}; + +export const generateItems = (count: number, seed: number): SqlValue[][] => { + const random = createRandom(seed); + return Array.from({ length: count }, (_, index) => { + const id = index + 1; + const createdAt = 1_700_000_000_000 + integer(random, 0, 30_000_000_000); + return [ + id, + `ITM-${String(id).padStart(6, '0')}`, + title(random), + nullable(random, 0.2, () => sentence(random, 6, 16)), + integer(random, 1, DATASET.categories), + pick(random, STATUSES), + integer(random, 1, 5), + integer(random, 0, 9_999), + decimal(random, 1_000), + nullable(random, 0.1, () => decimal(random, 50)), + nullable(random, 0.3, () => decimal(random, 2_000)), + nullable(random, 0.3, () => decimal(random, 2_000)), + createdAt, + nullable(random, 0.4, () => createdAt + integer(random, 0, 5_000_000_000)), + nullable(random, 0.25, () => `user-${integer(random, 1, 400)}`), + `site-${integer(random, 1, 12)}/floor-${integer(random, 0, 9)}/room-${integer(random, 1, 60)}`, + nullable(random, 0.5, () => pick(random, TAGS)), + random() < 0.8 ? 1 : 0, + decimal(random, 100), + nullable(random, 0.7, () => sentence(random, 2, 6)), + ]; + }); +}; diff --git a/benchmarks/wasm-build-comparison/results/npm-1.7.0-vs-rebuilt.json b/benchmarks/wasm-build-comparison/results/npm-1.7.0-vs-rebuilt.json new file mode 100644 index 0000000..672f16a --- /dev/null +++ b/benchmarks/wasm-build-comparison/results/npm-1.7.0-vs-rebuilt.json @@ -0,0 +1,619 @@ +{ + "resultFormat": "wasm-build-comparison-v1", + "runDate": "2026-10-07", + "environment": { + "browser": "Chromium 148.0.7778.96", + "crossOriginIsolated": true, + "hardwareConcurrency": 18, + "host": { + "arch": "arm64", + "cpuModel": "Apple M5 Pro", + "logicalCpus": 18, + "platform": "darwin" + }, + "loadAverageAtEnd": [7.56, 8.98, 8.84], + "loadAverageAtStart": [8.43, 9.38, 8.97] + }, + "runtimeVersions": { + "memoryWriteAheadVfs": "0.2.0", + "playwright": "1.60.0", + "vite": "8.0.16", + "waSqlite": "1.7.0" + }, + "configuration": { + "dataset": { + "categories": 1000, + "categoryFamilies": 20, + "insertRows": 5000, + "items": 30000, + "seed": 20260407 + }, + "primingRunsPerArm": 1, + "runsPerArm": 16, + "schedule": "ABBA/BAAB blocks, fresh browser context, page and worker per run", + "summaryUnit": "per-run median of measured rounds; instantiate and seed are one sample per run", + "vfs": "MemoryWriteAheadVFS in a dedicated worker", + "workload": { + "cacheSizeKiB": 65536, + "measuredRounds": 5, + "warmupRounds": 2 + } + }, + "arms": [ + { + "directory": "node_modules/@journeyapps/wa-sqlite/dist", + "glueSha256": "80b4f2bd9b3434d51cb799f274ab6e3cd4d5c53f466c75267ed6d20868a1bbce", + "name": "stock", + "source": "npm @journeyapps/wa-sqlite 1.7.0", + "wasmSha256": "d167becf5a6949b1f31b1d026dc15e138b2b34b12fc44b71c56164392900b3b8", + "compiler": "clang-23.0.0", + "maxAttached": 10, + "sqliteVersion": "3.53.0", + "wasmBytes": 1128032 + }, + { + "directory": "wasm/dist", + "glueSha256": "443f99db6d628fbf518a340de01d377187bed4e0619377c9a1ce1bae46b1b375", + "name": "rebuilt", + "source": "wasm/Dockerfile", + "wasmSha256": "91defe9a35e0662fbb343e1b54cb37567a51e8385300a40dc0797a04fd792422", + "compiler": "clang-21.0.0", + "maxAttached": 125, + "sqliteVersion": "3.53.0", + "wasmBytes": 1129991 + } + ], + "fingerprints": { + "filterScan": "7f7a2d32561d2d4eb9fe8d2af12749fe585d505fa5f497a3bda966a44ba8fc46", + "join": "f015fa1fad3cc5bef9f54d8c298127be9c8c60736ab9153c1858d8ab4431a7de", + "groupedLeftJoin": "8774e2429d7e9032ede762eaef5437bfb455addd0d9b4d7db2d36c42b317ee46", + "orderByText": "88f71a491506fcefc0d8d517e74858be19376c1c63e817e4fadd9c20e359c9d8", + "insert": "edc8eea99dba6b38988e1b817ceb30b5c977dacc2f9fa32f5518dd5677414797" + }, + "rowCounts": { + "filterScan": 17900, + "join": 4462, + "groupedLeftJoin": 1000, + "orderByText": 30000 + }, + "summaries": { + "stock": { + "filterScan": { + "max": 52.36, + "median": 46.72, + "min": 43.83, + "runs": 16 + }, + "join": { + "max": 15.02, + "median": 11.86, + "min": 11.5, + "runs": 16 + }, + "groupedLeftJoin": { + "max": 14.01, + "median": 12.8, + "min": 11.05, + "runs": 16 + }, + "orderByText": { + "max": 54.92, + "median": 45.63, + "min": 41.88, + "runs": 16 + }, + "insert": { + "max": 25.95, + "median": 23.89, + "min": 22.78, + "runs": 16 + }, + "instantiate": { + "max": 4.43, + "median": 3.93, + "min": 3.62, + "runs": 16 + }, + "seed": { + "max": 184.4, + "median": 164.15, + "min": 155.18, + "runs": 16 + } + }, + "rebuilt": { + "filterScan": { + "max": 50.15, + "median": 46.89, + "min": 45.01, + "runs": 16 + }, + "join": { + "max": 15.28, + "median": 12.1, + "min": 11.46, + "runs": 16 + }, + "groupedLeftJoin": { + "max": 14.94, + "median": 13.44, + "min": 12.57, + "runs": 16 + }, + "orderByText": { + "max": 50.61, + "median": 47.01, + "min": 44.01, + "runs": 16 + }, + "insert": { + "max": 26.06, + "median": 24.48, + "min": 23.31, + "runs": 16 + }, + "instantiate": { + "max": 3.51, + "median": 3.26, + "min": 3, + "runs": 16 + }, + "seed": { + "max": 172.6, + "median": 167.6, + "min": 158.62, + "runs": 16 + } + } + }, + "runs": [ + { + "arm": "stock", + "instantiateMs": 3.94, + "loadAverage1m": 8.24, + "measuredMs": { + "filterScan": [68.53, 72.5, 52.36, 45.65, 41.84], + "join": [15.02, 24.4, 12.03, 11.16, 15.37], + "groupedLeftJoin": [12.81, 21.15, 14.35, 11.06, 12.08], + "orderByText": [59.51, 67.96, 45, 42.33, 42.07], + "insert": [32.98, 25.77, 23.28, 25.56, 23.81] + }, + "run": 1, + "seedMs": 161.65 + }, + { + "arm": "rebuilt", + "instantiateMs": 3.03, + "loadAverage1m": 8.24, + "measuredMs": { + "filterScan": [45.01, 49.88, 43.76, 44.81, 46.13], + "join": [12.1, 11.9, 12.83, 11.46, 11.34], + "groupedLeftJoin": [14.54, 12.38, 17.05, 12.94, 11.07], + "orderByText": [65, 42.19, 48.85, 47, 43.05], + "insert": [22.94, 24.38, 22.94, 26.06, 24.81] + }, + "run": 2, + "seedMs": 158.62 + }, + { + "arm": "rebuilt", + "instantiateMs": 3.1, + "loadAverage1m": 8.24, + "measuredMs": { + "filterScan": [50.15, 51.24, 44.2, 48.41, 57.46], + "join": [12.23, 11.73, 14.95, 12.06, 11.95], + "groupedLeftJoin": [13.26, 13.23, 13.94, 13.85, 13.26], + "orderByText": [47.51, 47.34, 46.15, 46.66, 51.47], + "insert": [23.11, 25.17, 24.57, 37.85, 23.59] + }, + "run": 3, + "seedMs": 162.67 + }, + { + "arm": "stock", + "instantiateMs": 3.8, + "loadAverage1m": 7.98, + "measuredMs": { + "filterScan": [44.59, 43.4, 48.94, 50.91, 43.95], + "join": [11.63, 17.65, 11.54, 11.89, 15.53], + "groupedLeftJoin": [11.23, 14.23, 12.2, 13.58, 13.19], + "orderByText": [44.19, 44.94, 46.52, 45.57, 47.4], + "insert": [22.77, 23.74, 24.18, 27.17, 25.13] + }, + "run": 4, + "seedMs": 160.97 + }, + { + "arm": "rebuilt", + "instantiateMs": 3.27, + "loadAverage1m": 7.98, + "measuredMs": { + "filterScan": [46.75, 43.3, 56.36, 47.3, 46.98], + "join": [11.92, 18.13, 13.58, 11.69, 17.35], + "groupedLeftJoin": [12.9, 12.89, 15.35, 13.09, 14.31], + "orderByText": [46.95, 43.36, 51.71, 46.94, 44.27], + "insert": [23.42, 24.29, 24.7, 27.71, 25.32] + }, + "run": 5, + "seedMs": 161.15 + }, + { + "arm": "stock", + "instantiateMs": 3.93, + "loadAverage1m": 7.98, + "measuredMs": { + "filterScan": [47.72, 55.66, 44.83, 46.64, 51.2], + "join": [11.8, 11.91, 15.22, 12.15, 11.9], + "groupedLeftJoin": [15.56, 14.02, 12.79, 13.5, 13.54], + "orderByText": [55.68, 49.25, 48.05, 51.66, 52.76], + "insert": [24.22, 24.76, 25.99, 28.08, 24.44] + }, + "run": 6, + "seedMs": 166.22 + }, + { + "arm": "stock", + "instantiateMs": 4.43, + "loadAverage1m": 7.98, + "measuredMs": { + "filterScan": [50.89, 52.47, 47.76, 50.52, 56.79], + "join": [13.75, 11.91, 15.3, 14.31, 12.77], + "groupedLeftJoin": [17.01, 14.18, 13.21, 14.01, 13.88], + "orderByText": [49.23, 51.77, 45.42, 54, 52.2], + "insert": [22.97, 28.64, 25.95, 34.55, 23.81] + }, + "run": 7, + "seedMs": 178.25 + }, + { + "arm": "rebuilt", + "instantiateMs": 3.26, + "loadAverage1m": 7.9, + "measuredMs": { + "filterScan": [45.73, 50.32, 42.47, 46.67, 48.27], + "join": [11.82, 11.73, 11.1, 11.46, 11.43], + "groupedLeftJoin": [13.8, 12.41, 13, 12.41, 13.11], + "orderByText": [52.71, 44.28, 43.98, 46.25, 42.75], + "insert": [23.04, 24.93, 23.42, 26.6, 24.29] + }, + "run": 8, + "seedMs": 165.02 + }, + { + "arm": "stock", + "instantiateMs": 3.87, + "loadAverage1m": 7.9, + "measuredMs": { + "filterScan": [45.99, 49.17, 53.31, 51.32, 54.94], + "join": [11.7, 17.53, 12.37, 12.18, 13.54], + "groupedLeftJoin": [12.66, 13.97, 13.74, 15.65, 15.9], + "orderByText": [65.37, 51.16, 49.64, 56.66, 54.92], + "insert": [23.78, 25.7, 26.32, 28.1, 25.43] + }, + "run": 9, + "seedMs": 161.74 + }, + { + "arm": "rebuilt", + "instantiateMs": 3.25, + "loadAverage1m": 7.9, + "measuredMs": { + "filterScan": [48.41, 55.79, 47.04, 47.34, 49.31], + "join": [12.14, 13.09, 13.63, 12.84, 11.51], + "groupedLeftJoin": [14.66, 14.82, 14.4, 13.19, 13.32], + "orderByText": [52.44, 50.97, 44.89, 46.48, 47.49], + "insert": [23.57, 25.42, 23.88, 26.67, 25.65] + }, + "run": 10, + "seedMs": 172.6 + }, + { + "arm": "rebuilt", + "instantiateMs": 3.5, + "loadAverage1m": 7.91, + "measuredMs": { + "filterScan": [47.08, 62.51, 47.79, 43.2, 48.44], + "join": [12.14, 13, 11.97, 14.15, 11.81], + "groupedLeftJoin": [15.38, 16.47, 13.1, 13.08, 13.61], + "orderByText": [50.44, 50.12, 45.22, 46.41, 50.06], + "insert": [23.51, 23.7, 25.67, 26.75, 23.44] + }, + "run": 11, + "seedMs": 169.42 + }, + { + "arm": "stock", + "instantiateMs": 4.01, + "loadAverage1m": 7.91, + "measuredMs": { + "filterScan": [46.7, 49.18, 42.69, 44.89, 48.29], + "join": [11.67, 11.54, 11.36, 11.19, 11.59], + "groupedLeftJoin": [12.33, 11.26, 14.76, 12.02, 13.33], + "orderByText": [46.54, 39.53, 46.14, 43.43, 43.91], + "insert": [22.9, 24.5, 23.33, 25.27, 23.81] + }, + "run": 12, + "seedMs": 164.98 + }, + { + "arm": "rebuilt", + "instantiateMs": 3.27, + "loadAverage1m": 7.91, + "measuredMs": { + "filterScan": [46.88, 46.8, 59.31, 45.44, 44.04], + "join": [11.84, 17.19, 11.81, 11.58, 15.12], + "groupedLeftJoin": [13.72, 12.31, 12.87, 12.57, 12.42], + "orderByText": [47.21, 46.41, 47.75, 43.52, 42.53], + "insert": [23.01, 24.13, 23.56, 28.12, 23.2] + }, + "run": 13, + "seedMs": 163.55 + }, + { + "arm": "stock", + "instantiateMs": 3.92, + "loadAverage1m": 8, + "measuredMs": { + "filterScan": [44.65, 47.56, 46.69, 48.99, 46.73], + "join": [11.47, 16.87, 11.53, 11.71, 15.22], + "groupedLeftJoin": [11.49, 12.36, 12.58, 14.93, 12.87], + "orderByText": [48.03, 48.75, 46.88, 52.44, 48.64], + "insert": [22.69, 23, 23.46, 28.53, 23.38] + }, + "run": 14, + "seedMs": 164 + }, + { + "arm": "stock", + "instantiateMs": 4.28, + "loadAverage1m": 8, + "measuredMs": { + "filterScan": [46.5, 48.55, 41.98, 47.25, 50.27], + "join": [11.61, 11.49, 12.57, 11.83, 12.15], + "groupedLeftJoin": [12.07, 11.97, 12.8, 13.03, 13.37], + "orderByText": [46.56, 44.06, 44.59, 49.91, 47.76], + "insert": [22.76, 24.57, 23.17, 26.88, 23.97] + }, + "run": 15, + "seedMs": 170.23 + }, + { + "arm": "rebuilt", + "instantiateMs": 3.5, + "loadAverage1m": 8, + "measuredMs": { + "filterScan": [49.5, 51.01, 46.3, 45.75, 44.26], + "join": [12.32, 17.38, 11.46, 13.16, 15.84], + "groupedLeftJoin": [15.25, 12.79, 12.97, 15.06, 12.02], + "orderByText": [51.62, 42.47, 45.84, 48.49, 44.08], + "insert": [23.79, 23.33, 23.32, 27.67, 24.81] + }, + "run": 16, + "seedMs": 167.11 + }, + { + "arm": "stock", + "instantiateMs": 3.88, + "loadAverage1m": 7.67, + "measuredMs": { + "filterScan": [45.47, 41.71, 45.74, 45.1, 45.57], + "join": [12.14, 17.1, 11.46, 11.25, 15.98], + "groupedLeftJoin": [13.25, 10.86, 12.39, 10.65, 13.2], + "orderByText": [45.17, 41.28, 44.89, 44.06, 43.84], + "insert": [22.55, 22.73, 23.39, 25.6, 23.94] + }, + "run": 17, + "seedMs": 158.18 + }, + { + "arm": "rebuilt", + "instantiateMs": 3.24, + "loadAverage1m": 7.67, + "measuredMs": { + "filterScan": [47.11, 64.27, 47.51, 52.09, 45.99], + "join": [11.95, 19.99, 11.81, 15.28, 17.54], + "groupedLeftJoin": [12.78, 12.91, 15.35, 15.25, 14.94], + "orderByText": [49.81, 43.51, 53.78, 52, 50.61], + "insert": [22.75, 23.02, 24.05, 29.69, 26.08] + }, + "run": 18, + "seedMs": 161.47 + }, + { + "arm": "rebuilt", + "instantiateMs": 3.51, + "loadAverage1m": 7.67, + "measuredMs": { + "filterScan": [47.42, 45.6, 46.34, 44.88, 48.81], + "join": [11.95, 16.79, 11.52, 11.39, 11.76], + "groupedLeftJoin": [12.9, 12.56, 13.44, 12.1, 13.81], + "orderByText": [47.37, 44.98, 43.48, 45.77, 45.06], + "insert": [23.31, 23.07, 23.26, 25.92, 24.7] + }, + "run": 19, + "seedMs": 170.67 + }, + { + "arm": "stock", + "instantiateMs": 4.06, + "loadAverage1m": 7.67, + "measuredMs": { + "filterScan": [44.05, 44.23, 46.19, 44.8, 49.58], + "join": [11.27, 17.04, 11.5, 11.39, 15.7], + "groupedLeftJoin": [11.7, 13.16, 13.08, 11.8, 13.82], + "orderByText": [44.12, 43.14, 46.4, 45.69, 48.92], + "insert": [22.22, 22.92, 22.76, 26.96, 26.01] + }, + "run": 20, + "seedMs": 165.19 + }, + { + "arm": "rebuilt", + "instantiateMs": 3.08, + "loadAverage1m": 7.62, + "measuredMs": { + "filterScan": [48.29, 55.32, 47.66, 47.81, 46.19], + "join": [13.64, 11.97, 15.37, 11.58, 11.18], + "groupedLeftJoin": [14.63, 14.18, 13.95, 11.51, 11.38], + "orderByText": [49.34, 47.03, 47.21, 43.25, 42.19], + "insert": [24.63, 24.98, 23.98, 25.59, 22.79] + }, + "run": 21, + "seedMs": 169 + }, + { + "arm": "stock", + "instantiateMs": 3.69, + "loadAverage1m": 7.62, + "measuredMs": { + "filterScan": [43.04, 48.72, 48.41, 48.8, 42.78], + "join": [11.54, 16.73, 12.98, 11.64, 15.26], + "groupedLeftJoin": [12.16, 12.94, 13.66, 12.55, 11.57], + "orderByText": [44.5, 48.69, 50.18, 43.42, 41.2], + "insert": [22.97, 27.68, 23.88, 27.06, 24.22] + }, + "run": 22, + "seedMs": 155.79 + }, + { + "arm": "stock", + "instantiateMs": 3.62, + "loadAverage1m": 7.62, + "measuredMs": { + "filterScan": [43.83, 45.65, 45.94, 43.67, 42.85], + "join": [11.69, 16.56, 11.35, 11.34, 14.86], + "groupedLeftJoin": [11.05, 11.59, 12.25, 10.77, 10.78], + "orderByText": [41.19, 42.82, 44.28, 40.1, 41.95], + "insert": [22.39, 22.78, 22.67, 26.73, 23.7] + }, + "run": 23, + "seedMs": 155.18 + }, + { + "arm": "rebuilt", + "instantiateMs": 3, + "loadAverage1m": 7.33, + "measuredMs": { + "filterScan": [47.91, 45.09, 45.83, 49.1, 44.92], + "join": [11.82, 17.31, 11.51, 11.63, 16.16], + "groupedLeftJoin": [13.38, 13.03, 13.22, 12.44, 12.75], + "orderByText": [47.46, 45.67, 52.7, 46.69, 46.72], + "insert": [23.66, 23.31, 29, 26.06, 26.19] + }, + "run": 24, + "seedMs": 161.9 + }, + { + "arm": "stock", + "instantiateMs": 4.14, + "loadAverage1m": 7.33, + "measuredMs": { + "filterScan": [44.4, 45.35, 46.69, 45.28, 43.66], + "join": [11.79, 16.91, 11.45, 11.63, 15.67], + "groupedLeftJoin": [13.63, 13.1, 12.7, 12.74, 13.7], + "orderByText": [46.94, 44.15, 46.77, 47.85, 47.66], + "insert": [22.87, 23.54, 23.07, 27.8, 23.57] + }, + "run": 25, + "seedMs": 161.94 + }, + { + "arm": "rebuilt", + "instantiateMs": 3.34, + "loadAverage1m": 7.33, + "measuredMs": { + "filterScan": [46.66, 52.63, 47.26, 49, 51.41], + "join": [11.96, 14.52, 12.23, 13.14, 11.88], + "groupedLeftJoin": [14.13, 14.66, 16.44, 14.75, 13.81], + "orderByText": [49.84, 50.68, 50.12, 51.82, 50.26], + "insert": [23.21, 26.77, 24.47, 26.97, 25.1] + }, + "run": 26, + "seedMs": 169.69 + }, + { + "arm": "rebuilt", + "instantiateMs": 3.25, + "loadAverage1m": 7.78, + "measuredMs": { + "filterScan": [46.13, 46.69, 49.25, 51.28, 46.35], + "join": [11.94, 16.96, 12.04, 12.35, 15.96], + "groupedLeftJoin": [13.52, 13.31, 14.53, 14.38, 14.18], + "orderByText": [46.73, 54.6, 48.31, 48.44, 48.06], + "insert": [22.86, 24.38, 24.22, 28.89, 25.17] + }, + "run": 27, + "seedMs": 168.09 + }, + { + "arm": "stock", + "instantiateMs": 4.06, + "loadAverage1m": 7.78, + "measuredMs": { + "filterScan": [51.69, 51.69, 43.41, 45, 50.45], + "join": [12.33, 11.68, 16.45, 11.35, 12.01], + "groupedLeftJoin": [14.17, 14.01, 14.19, 13.51, 13.52], + "orderByText": [50.39, 49.25, 45.09, 48.93, 50.3], + "insert": [24, 23.13, 25.19, 30.04, 24.49] + }, + "run": 28, + "seedMs": 184.4 + }, + { + "arm": "rebuilt", + "instantiateMs": 3.33, + "loadAverage1m": 7.78, + "measuredMs": { + "filterScan": [45.84, 41.52, 45.81, 46.14, 44.4], + "join": [11.94, 17.3, 11.54, 11.53, 16.03], + "groupedLeftJoin": [14.15, 11.81, 13.6, 13.92, 13.81], + "orderByText": [44.01, 42.59, 51.51, 42.15, 54.05], + "insert": [22.79, 23.55, 24.88, 25.63, 32.95] + }, + "run": 29, + "seedMs": 169.61 + }, + { + "arm": "stock", + "instantiateMs": 4.27, + "loadAverage1m": 7.56, + "measuredMs": { + "filterScan": [44.11, 42.89, 47.7, 45.44, 42.62], + "join": [11.51, 17.43, 11.45, 10.95, 16.09], + "groupedLeftJoin": [11.05, 11.13, 11.64, 11.82, 11.01], + "orderByText": [43.43, 41.32, 45.96, 41.88, 40.59], + "insert": [22.76, 22.79, 23.72, 25.93, 23.51] + }, + "run": 30, + "seedMs": 164.31 + }, + { + "arm": "stock", + "instantiateMs": 3.81, + "loadAverage1m": 7.56, + "measuredMs": { + "filterScan": [43.69, 46.53, 46.28, 45.34, 44.23], + "join": [11.35, 17, 11.13, 11.52, 15.29], + "groupedLeftJoin": [10.89, 11.89, 11.1, 11.79, 12.38], + "orderByText": [45.81, 41.64, 42.99, 45.4, 44.38], + "insert": [23.26, 22.66, 22.72, 27.33, 25.48] + }, + "run": 31, + "seedMs": 180.04 + }, + { + "arm": "rebuilt", + "instantiateMs": 3.37, + "loadAverage1m": 7.56, + "measuredMs": { + "filterScan": [49.04, 55.36, 48.29, 48.42, 45.56], + "join": [11.9, 18.01, 11.66, 14.08, 15.87], + "groupedLeftJoin": [13.21, 17.5, 14.01, 14.13, 13.74], + "orderByText": [49.73, 51.59, 48.17, 48.19, 46.45], + "insert": [23.47, 23.75, 25.95, 28.43, 25.3] + }, + "run": 32, + "seedMs": 171.84 + } + ] +} diff --git a/benchmarks/wasm-build-comparison/worker.ts b/benchmarks/wasm-build-comparison/worker.ts new file mode 100644 index 0000000..48ca638 --- /dev/null +++ b/benchmarks/wasm-build-comparison/worker.ts @@ -0,0 +1,191 @@ +import * as SQLite from '@journeyapps/wa-sqlite'; + +import { MEMORY_WRITE_AHEAD_VFS, MemoryWriteAheadVFS, createMemoryWriteAheadSharedRuntime } from '../../src/index.js'; +import { + DATASET, + INDEXES, + INSERT_CATEGORY_SQL, + INSERT_ITEM_SQL, + INSERT_WORKLOAD_SQL, + READ_QUERIES, + SCHEMA, + WORKLOAD, + generateCategories, + generateItems, + type ReadQueryName, + type SqlValue, +} from './dataset.js'; + +export type WorkloadName = ReadQueryName | 'insert'; + +export type WasmBuildRunRequest = { + glueUrl: string; + wasmUrl: string; +}; + +export type WasmBuildRunResult = { + build: { + compiler: string | null; + maxAttached: number | null; + sqliteVersion: string; + wasmBytes: number; + wasmSha256: string; + }; + crossOriginIsolated: boolean; + fingerprints: Record; + hardwareConcurrency: number; + instantiateMs: number; + measuredMs: Record; + rowCounts: Record; + seedMs: number; +}; + +type WorkerResponse = { ok: true; result: WasmBuildRunResult } | { ok: false; message: string; stack?: string }; + +type WaSqliteModuleFactory = (moduleArg?: { wasmBinary?: ArrayBuffer }) => Promise; + +const toHex = (buffer: ArrayBuffer) => + Array.from(new Uint8Array(buffer), (byte) => byte.toString(16).padStart(2, '0')).join(''); + +const sha256 = async (data: ArrayBuffer | string) => + toHex(await crypto.subtle.digest('SHA-256', typeof data === 'string' ? new TextEncoder().encode(data) : data)); + +const runWorkload = async ({ glueUrl, wasmUrl }: WasmBuildRunRequest): Promise => { + const { default: factory } = (await import(/* @vite-ignore */ glueUrl)) as { default: WaSqliteModuleFactory }; + const wasmBinary = await (await fetch(wasmUrl)).arrayBuffer(); + const wasmSha256 = await sha256(wasmBinary); + + const instantiateStartedAt = performance.now(); + const module = await factory({ wasmBinary }); + const instantiateMs = performance.now() - instantiateStartedAt; + + const sqlite3 = SQLite.Factory(module); + const dbFilename = '/wasm-build-comparison.sqlite'; + const runtime = createMemoryWriteAheadSharedRuntime(dbFilename); + const vfs = await MemoryWriteAheadVFS.create(MEMORY_WRITE_AHEAD_VFS, module, runtime); + sqlite3.vfs_register(vfs as unknown as SQLiteVFS, true); + const db = await sqlite3.open_v2(dbFilename, undefined, MEMORY_WRITE_AHEAD_VFS); + + const selectRows = async (sql: string) => { + const rows: SqlValue[][] = []; + for await (const stmt of sqlite3.statements(db, sql)) { + while ((await sqlite3.step(stmt)) === SQLite.SQLITE_ROW) { + rows.push(sqlite3.row(stmt) as SqlValue[]); + } + } + return rows; + }; + + const insertRows = async (sql: string, rows: readonly SqlValue[][]) => { + for await (const stmt of sqlite3.statements(db, sql)) { + for (const row of rows) { + sqlite3.bind_collection(stmt, row); + await sqlite3.step(stmt); + await sqlite3.reset(stmt); + } + } + }; + + try { + await sqlite3.exec(db, 'PRAGMA journal_mode = WAL'); + await sqlite3.exec(db, `PRAGMA cache_size = -${WORKLOAD.cacheSizeKiB}`); + await sqlite3.exec(db, 'PRAGMA temp_store = memory'); + + const compileOptions = (await selectRows('PRAGMA compile_options')).map((row) => String(row[0])); + const readOption = (prefix: string) => + compileOptions.find((option) => option.startsWith(prefix))?.slice(prefix.length) ?? null; + const maxAttached = readOption('MAX_ATTACHED='); + const sqliteVersion = String((await selectRows('SELECT sqlite_version()'))[0]?.[0]); + + const categories = generateCategories(); + const items = generateItems(DATASET.items, DATASET.seed); + const insertWorkloadRows = generateItems(DATASET.insertRows, DATASET.seed + 1); + + const seedStartedAt = performance.now(); + await sqlite3.exec(db, 'BEGIN'); + for (const statement of SCHEMA) { + await sqlite3.exec(db, statement); + } + await insertRows(INSERT_CATEGORY_SQL, categories); + await insertRows(INSERT_ITEM_SQL, items); + for (const statement of INDEXES) { + await sqlite3.exec(db, statement); + } + await sqlite3.exec(db, 'COMMIT'); + const seedMs = performance.now() - seedStartedAt; + + const readQueryNames = Object.keys(READ_QUERIES) as ReadQueryName[]; + const measuredMs = Object.fromEntries( + [...readQueryNames, 'insert'].map((name) => [name, [] as number[]]) + ) as Record; + const fingerprints = {} as Record; + const rowCounts = {} as Record; + + for (let round = 0; round < WORKLOAD.warmupRounds + WORKLOAD.measuredRounds; round += 1) { + const measured = round >= WORKLOAD.warmupRounds; + + for (const name of readQueryNames) { + const startedAt = performance.now(); + const rows = await selectRows(READ_QUERIES[name]); + const elapsedMs = performance.now() - startedAt; + if (measured) { + measuredMs[name].push(elapsedMs); + } + if (round === WORKLOAD.warmupRounds) { + fingerprints[name] = await sha256(JSON.stringify(rows)); + rowCounts[name] = rows.length; + } + } + + await sqlite3.exec(db, 'DELETE FROM item_insert'); + const insertStartedAt = performance.now(); + await sqlite3.exec(db, 'BEGIN'); + await insertRows(INSERT_WORKLOAD_SQL, insertWorkloadRows); + await sqlite3.exec(db, 'COMMIT'); + const insertMs = performance.now() - insertStartedAt; + if (measured) { + measuredMs.insert.push(insertMs); + } + if (round === WORKLOAD.warmupRounds) { + fingerprints.insert = await sha256( + JSON.stringify( + await selectRows( + 'SELECT COUNT(*), SUM(quantity), SUM(unit_price), COUNT(description) FROM item_insert' + ) + ) + ); + } + } + + return { + build: { + compiler: readOption('COMPILER='), + maxAttached: maxAttached === null ? null : Number(maxAttached), + sqliteVersion, + wasmBytes: wasmBinary.byteLength, + wasmSha256, + }, + crossOriginIsolated, + fingerprints, + hardwareConcurrency: navigator.hardwareConcurrency, + instantiateMs, + measuredMs, + rowCounts, + seedMs, + }; + } finally { + await sqlite3.close(db); + await vfs.close(); + } +}; + +self.onmessage = async (event: MessageEvent) => { + const response: WorkerResponse = await runWorkload(event.data).then( + (result) => ({ ok: true as const, result }), + (error: unknown) => { + const err = error instanceof Error ? error : new Error(String(error)); + return { ok: false as const, message: err.message, ...(err.stack ? { stack: err.stack } : {}) }; + } + ); + self.postMessage(response); +}; diff --git a/package.json b/package.json index 3e531fd..bf40af0 100644 --- a/package.json +++ b/package.json @@ -26,7 +26,9 @@ "types": "./dist/index.d.ts", "scripts": { "build": "tsc -p tsconfig.build.json && node scripts/prepare-dist.mjs", + "build:wasm": "docker build --output wasm/dist wasm && cd wasm/dist && sha256sum -c ../SHA256SUMS", "benchmark": "playwright test --config playwright.benchmark.config.ts", + "benchmark:wasm": "playwright test --config playwright.wasm-benchmark.config.ts", "benchmark:publish-result": "node scripts/publish-benchmark-result.mjs", "check": "tsc -p tsconfig.json --noEmit", "format": "prettier --write .", diff --git a/playwright.wasm-benchmark.config.ts b/playwright.wasm-benchmark.config.ts new file mode 100644 index 0000000..de95701 --- /dev/null +++ b/playwright.wasm-benchmark.config.ts @@ -0,0 +1,28 @@ +import { defineConfig, devices } from '@playwright/test'; + +export default defineConfig({ + testDir: './benchmarks/wasm-build-comparison', + testMatch: 'benchmark.spec.ts', + timeout: 60 * 60_000, + expect: { + timeout: 5_000, + }, + fullyParallel: false, + workers: 1, + use: { + baseURL: 'http://127.0.0.1:5175', + trace: 'retain-on-failure', + }, + webServer: { + command: 'pnpm exec vite --host 127.0.0.1 --port 5175', + url: 'http://127.0.0.1:5175/test-host.html', + reuseExistingServer: !process.env.CI, + timeout: 20_000, + }, + projects: [ + { + name: 'chromium', + use: { ...devices['Desktop Chrome'] }, + }, + ], +}); diff --git a/tsconfig.json b/tsconfig.json index 4358fbe..8e5b63a 100644 --- a/tsconfig.json +++ b/tsconfig.json @@ -22,6 +22,7 @@ "benchmarks/**/*.ts", "playwright.config.ts", "playwright.benchmark.config.ts", + "playwright.wasm-benchmark.config.ts", "vite.config.ts" ] } diff --git a/wasm/Dockerfile b/wasm/Dockerfile new file mode 100644 index 0000000..e8bf766 --- /dev/null +++ b/wasm/Dockerfile @@ -0,0 +1,25 @@ +# check=skip=FromPlatformFlagConstDisallowed + +FROM --platform=linux/amd64 emscripten/emsdk:4.0.9@sha256:3c853ef9c3b4c2708da1adac2fdfdba49c775fdc4144ceef4989423963e96811 AS build + +RUN apt-get update \ + && apt-get install -y --no-install-recommends curl jq make openssl \ + && rm -rf /var/lib/apt/lists/* + +ARG WA_SQLITE_COMMIT=e4bb5532c8cfa9fa80bb26e62cc22814d16440be +WORKDIR /wa-sqlite +RUN git init -q \ + && git fetch -q --depth 1 https://github.com/powersync-ja/wa-sqlite.git "${WA_SQLITE_COMMIT}" \ + && git checkout -q FETCH_HEAD \ + && node scripts/download-core-build.js \ + && make deps/version-3.53.0/sqlite3.c \ + && printf '%s\n' \ + '4898210a5d532d374cba71682f8030fc9538fa57961e1686b2d7a0d1bb78370c powersync-libs/libpowersync-wasm.a' \ + 'c94657c2a08ed5f3f61a25f0018eb2f64ec42fd053da26d741db9e52e6ffaa77 deps/version-3.53.0/sqlite3.c' \ + | sha256sum -c - + +ARG SQLITE_MAX_ATTACHED=125 +RUN make dist/wa-sqlite.mjs WASQLITE_EXTRA_DEFINES=-DSQLITE_MAX_ATTACHED=${SQLITE_MAX_ATTACHED} + +FROM scratch +COPY --chmod=0644 --from=build /wa-sqlite/dist/wa-sqlite.mjs /wa-sqlite/dist/wa-sqlite.wasm / diff --git a/wasm/SHA256SUMS b/wasm/SHA256SUMS new file mode 100644 index 0000000..546b945 --- /dev/null +++ b/wasm/SHA256SUMS @@ -0,0 +1,2 @@ +443f99db6d628fbf518a340de01d377187bed4e0619377c9a1ce1bae46b1b375 wa-sqlite.mjs +91defe9a35e0662fbb343e1b54cb37567a51e8385300a40dc0797a04fd792422 wa-sqlite.wasm