Lingji

A browser-local data sonification studio: CSV → explicit mappings → deterministic notes → audible playback.

Lingji turns data sonification into a shared creative instrument: the human decides what the data should communicate, while their browser agent operates and refines the live composition without erasing protected human decisions. Inspect a dataset, choose mappings, edit the score, lock deliberate decisions, and ask a WebMCP-capable browser agent to refine the rest. React and TypeScript provide the interface; Web Audio synthesizes the sound locally. There is no embedded chatbot, audio-generation API, backend, or account system.

Creating intentional sonification requires coordinating mappings, musical structure, and repeated listening-based refinement. Lingji's contribution is an agent-operable, collaboratively editable live artifact—not the invention of sonification. TwoTone and Highcharts Sonification Studio are established browser-sonification prior art. Better authoring efficiency and listener comprehension remain hypotheses, not measured product claims.

Run locally

Use Node.js 22.12+ on the 22.x line, or Node.js 24+, with npm. From this standalone repository's root:

npm ci
npm run dev

Open the local URL printed by Vite. Press Play to enable browser audio; a user gesture is required. No API key or account is needed. On Windows, use npm.cmd if PowerShell's execution policy blocks npm.

For agent collaboration, open the same page in a WebMCP-capable browser. The activity panel reports whether all nine tools registered. Unsupported browsers retain the human studio; there is no mock WebMCP fallback. The implementation follows the current imperative API, using document.modelContext.registerTool and AbortSignal cleanup.

Try the checkpoint

1. The 64 exoplanets sample loads automatically with readable labels and units. Press Play to hear its starting mappings—no setup is needed. The numbered Data → Listen → Refine links jump to the relevant sections; optional agent tools are below the human studio.
2. Select a generated note to follow its source. To explore the dataset, open Inspect 8 columns under Your data. Selecting a column shows its readable name, original archive field, detected type, counts, missing values, numeric range or category lookup and chart; it does not change the music.
3. In Connections, change a Data column, Sound property, or Track, then press Generate. A visible glossary explains pitch, velocity, density, brightness and instruments. The initial Melody maps equilibrium temperature → pitch, planet radius → velocity, and stellar temperature → brightness. Pulse maps orbital period → rhythm; Bass maps system distance → pitch. Inspect a rule for exact normalization and transformation. Full-dataset statistics remain authoritative; pending changes do not affect the score until Generate.
4. Choose Playback mode, then Play, Pause, or Restart. Repeat this 16-step pattern retains speed and 1–16 pattern repeats (bars). Play every CSV row once generates every row and plays it over 4–120 whole seconds, in original file order (not inferred chronological order). The screen follows 16-row pages while playing. Missing values and rhythm gates can produce rests; manually edited/locked notes remain declared exceptions. Switching to pattern mode retains hidden decisions for a later scan. Choose timing before locking a track. Use Track sound & mix for piano, marimba or the existing synths, exact volume (0–1), and pitch-mapping intensity (0–2). Intensity 1 preserves the rule; other values apply clamp(0.5 + (normalized - 0.5) × intensity, 0, 1) before pitch quantization. Other mappings remain unchanged. Voice overrides apply to generated notes; explicit note edits and frozen snapshots take precedence. Live mute, solo and volume do not restart the clock. Now Playing shows the active source entity, values, mappings and effective notes independently of your selected-note inspector.
5. Select a generated note to see raw mapped values immediately below the score, with a disclosure for its full source row. The source strip follows its pitch-mapping column. A connected agent can call get_selected_note when you ask “Why did this note become A4?” and read the same effective note, source row and provenance without guessing from screen position. Full note inspector links to exact normalization, rules and editing controls. Change pitch, velocity, position, duration, waveform, or brightness. Click an empty step to add a note, or delete a selected note. Repeatedly selecting a cell with overlapping notes cycles through them. All available notes retain native keyboard access; optional arrow/Home/End shortcuts move between cells without selecting them. The links before the grid skip directly to Connections or the inspector. Edits apply immediately; there is no Undo, and original generated values remain in provenance.
6. Lock a mapping, track, or individual note, then regenerate. Protected content stays intact until unlocked. Manual edits and note deletions also survive regeneration within the current session; original mapping evidence remains distinct from manual overrides.
7. Choose Import CSV to use your own data. Invalid input shows an error without replacing the active project. Dataset replacement asks for confirmation when the current session has changes.
8. Choose Export WAV, then Download WAV when rendering finishes. The 48 kHz stereo PCM16 file uses the same playback plan and synthesizer as the live transport, including mute/solo, manual notes and protected snapshots. Changing the composition invalidates a prepared download; generate pending mapping drafts first. Rendering happens locally, without recording your microphone or uploading data. A WAV is audio, not an editable project backup.

Import mappings are editable starting suggestions, not scientific interpretations. Recognized headers retain explicit presets; unfamiliar numeric measurements are distributed across controls before reuse, preferring varying measurements over constant columns and known ID/time fields. NASA exoplanet headers use separate temperature, radius, star-temperature, orbital-period and distance measurements, with discovery method as the instrument category. Without a suitable category, Bass keeps its default waveform and no instrument rule is added. This only affects newly created/imported projects; existing mappings, edits and locks are not automatically rewritten. See import-default verification and NASA sound references.

An ordinary import should be usable, not deliberately made worse to sell collaboration. Both the human and browser agent have the same controls and sound engine. The agent can translate a listening brief into explicit mapping and mix changes; it has no hidden higher-quality renderer. Choose timing before locking your bass, then ask it to refine unlocked layers. The studio's From first import to intentional sound disclosure provides a starting prompt. See the scan and live refinement verification for a reproducible example and its limits.

Datasets and privacy

The automatic first-run sample is the 64-row exoplanet snapshot documented below. Known archive headers such as pl_eqt and pl_rade are shown as readable labels with units while their original CSV field names remain available as provenance.

The repository also retains a 64-row, five-column fictional delivery sample as a privacy-safe test and fallback asset. It is not copied, sampled, anonymized, or reduced from the previously supplied food-delivery dataset. No original food-delivery data is bundled. See sample data and reproduction rules.

The default public/data/lingji-exoplanets-64.csv is a NASA Exoplanet Archive-derived snapshot for the flagship space-data demo. The file was recovered from Lingji's complete inspect_dataset output on 2026-09-03; the exact original TAP query and archive retrieval date were not retained. It is not the complete, current, or representative exoplanet catalogue. See source attribution, fields, recovery provenance, and scientific limits.

Uploaded CSV contents, mappings, and notes live in browser memory and are not sent to a Lingji backend. A connected browser agent can receive dataset contents and composition state through WebMCP; the agent provider's data policy applies. Do not use sensitive data without understanding that boundary. Reloading or closing the page loses the editable session: WAV export saves audio only; there is no project save/load or recovery. WAV filenames and metadata contain no CSV identifiers or raw rows, although the sound itself encodes the chosen data mappings. The application and its bundled fonts/sample are ordinary static assets served by Vite or a static host.

Determinism and architecture

• src/domain/: pure CSV parsing, column profiling, normalization, mapping evaluation, sequence generation, provenance, and immutable project commands. It has no React or Web Audio dependency.
• src/audio/: fixed-timeline repeat/full-dataset scan planning, shared oscillator/sampler/filter/envelope voices, offline rendering and PCM WAV encoding. Track gain gates support live mute/solo/volume without restarting notes. Piano and marimba use seven locally served CC0 recordings, with verified MIDI roots and nearest-sample transposition. See sample attribution and processing.
• src/studio/session.ts: the single live store for the project, human mapping draft, and revision. React subscribes to this store; both UI and agent mutations dispatch the same domain commands.
• src/webmcp/: semantic tools, runtime argument validation, registration lifecycle, note-plan inspection, fidelity checks, and visible activity. No agent-only composition or alternate generator exists.
• src/App.tsx, src/studio/, and src/components/: the human interface, shared controls, and one audio player used by UI and preview tools.

The same dataset, source window, and mapping configuration produce the same generated event sequence. Numeric columns use min/max normalization; constant numeric columns use 0.5. Categorical columns use an inspectable, stable sorted lookup, not a claim that categories have an inherent numeric order. Missing values skip affected notes. Pitch is quantized to a selected musical scale, and rhythm uses an explicit threshold gate rather than random sampling. Manual changes and locks are a separate deterministic layer.

Verify and build

npm test
npm run check:design
npm run build
npm run preview

Tests cover parsing, normalization, provenance, deterministic generation, manual edits, locks, audio transport, shared controls, stale agent requests, draft preservation, malformed tool inputs, registration cleanup and indirect lock bypasses. check:design compares the colors, font families, radii, and spacing in DESIGN.md with the baseline CSS variables; forced-colors accessibility overrides are intentionally separate. It checks token consistency, not visual quality or accessibility by itself. test-fixtures/ contains small synthetic CSV inputs for upload checks; tests/fixtures/satisfaction.csv is a separate, entirely fictional declining-satisfaction scenario.

The production build is written to dist/; preview serves that build locally. Live browser verification evidence is kept separately from unit-test and build results.

See the usability audit for the current first-use, keyboard, mobile, and form-recovery checks, including screenshots and the limits of that testing.

Current limits

• Session-only editable state; no save/load project files, persistence, recovery or undo. WAV export is not a project backup.
• Three tracks and a 16-cell editor viewport: repeat the visible pattern for 1–16 bars, or scan all rows once over 4–120 seconds. No multi-pattern song sections. Playback stops at the selected duration. Track locks preserve the events captured when locked; a previously locked 16-row pattern is not silently expanded into 64 newly generated bass notes.
• WAV export requires browser OfflineAudioContext support and is bounded to 130 seconds / 10,000 scheduled notes. Output is synthesized instrumental audio, not an AI-produced vocal song.
• CSV uploads up to 5 MB. Live playback rejects scores above 10,000 scheduled notes instead of silently truncating rows. Instruments: sampled piano and marimba plus sine, triangle, square and sawtooth synths. Seven single-velocity recordings are a small instrument set, not a production sound library; extreme transpositions may sound unnatural. No vocals, music-generation API or new backend.
• No embedded AI model, audio-generation service, backend, authentication, or multi-user synchronization. The agent is supplied by the browser, not Lingji.

WebMCP collaboration checkpoint

Nine tools: inspect_dataset, get_project_state, get_selected_note, get_mapping_graph, inspect_locks, modify_mapping, modify_arrangement, render_preview, validate_data_fidelity.

get_selected_note turns the human's live note selection into shared pointing context. After a person clicks a note, the agent can read that exact note's effective sound, source row and complete mapping provenance without guessing from screen position or asking the person to copy an event ID. Empty, manual and stale selections are explicit, and reading selection never changes the composition revision.

Mutations require the current expectedRevision. Human drafts and pending confirmations/imports block agent writes. Only the human can unlock. Agent operations cannot change protected-track audibility indirectly through Solo, change protected volume, or retime locked notes/tracks. modify_arrangement accepts set-playback (scan, 4–120 seconds; or pattern), set-track-mix (muted/solo/volume), and set-track-sound (instrument/mappingIntensity), as well as the existing operations. Locked instrument/pitch mappings block indirect track overrides. Individually frozen notes retain their exact voice/pitch when other unlocked notes change. In scan mode, steps are absolute zero-based row slots; sourceOffset only pages the editor. State, preview, provenance and export use the same effective sequence. See checkpoint verification. Audio quality and listener comprehension remain listening-test hypotheses, not measured improvements.

render_preview operates the same audio player and returns deterministic note-plan statistics, plus an instantaneous output RMS when available. It is not offline acoustic analysis. The agent does not receive audio from Lingji and must ask the human for taste judgments. validate_data_fidelity checks internal source/mapping consistency and declares manual/locked exceptions; it does not certify real-world data truth, musical quality or listener comprehension.

See the reproducible collaboration test and evidence. The live trace shows the last 50 actual calls with arguments, UTC timestamps, outcomes, and before/after revisions. It is session-only, not an immutable audit service.

Design and third-party assets

The working-score interface pairs neutral surfaces and graphite controls with amber, blue and green musical tracks. Source, Connections, and Composition are sections of one workspace. Pitch height is a visual contour scaled separately within each track; exact MIDI values remain available in the inspector. The interface's tokens and Dribbble references are documented in DESIGN.md. No third-party reference artwork is bundled. Space Grotesk headings, Manrope controls and IBM Plex Mono data are self-hosted with their original SIL Open Font License notices. See font sources and licenses.

License

Lingji's original source code, documentation and synthetic delivery dataset are licensed under the MIT License. The NASA Exoplanet Archive-derived snapshot is third-party source data and is not relicensed under MIT; retain its provenance, acknowledgement, and citation guidance. Instrument recordings retain their CC0 dedication and attribution. The bundled fonts retain their separate SIL Open Font Licenses and copyright notices; they are not relicensed under MIT. See font sources and licenses.