الأثير · Aether

A browser theremin you play with your hands in the air.

▶ Play it · no install, works on phones · How it works

A theremin is played without being touched: you shape pitch and volume by moving your hands near two antennas. Aether puts that in a browser tab. Move a hand toward the pitch antenna to raise the note, lift the other away from the volume loop to get louder. Nothing to press.

Two ways to play: the pointer works instantly, and camera mode tracks both hands through a webcam. Sound is synthesized live with the Web Audio API. Camera frames are processed on your device and are never uploaded.

How it works

The interesting part is not calling a hand-tracking model. It is keeping a gesture-to-sound loop responsive on hardware you do not control.

Hand tracking. Camera mode runs MediaPipe Hand Landmarker on-device via WebAssembly, returning 21 landmarks per hand for up to two hands. The model is pre-trained; everything below is the part that makes it usable as an instrument.

Latency is the whole product. A theremin that lags is not an instrument, so the gesture-to-sound path is attacked at four points:

• Overlapping the two cold starts. The model download begins while the camera permission prompt is still open, so the two waits happen at once instead of back to back (app.js, startCamera).
• An adaptive inference cadence. The median frame cost over a rolling 24-frame window drives the target rate down through 30 → 24 → 20 fps when a device cannot keep up, so tracking degrades smoothly instead of stuttering (app.js, adaptTrackingCadence). The median rather than the mean, so one long frame does not swing the whole loop.
• Following the video clock. Tracking runs off decoded video frames rather than requestAnimationFrame, so it never processes the same frame twice.
• An interactive-latency audio path, with a Measure response tool that estimates the real display, audio, tracking, and glide budget on the current machine and says when a lower glide setting would feel more immediate.

Degrading instead of breaking. The GPU delegate falls back to CPU; capture resolution drops to 480×360 based on reported device memory and core count; and if the camera is unavailable, blocked, or unsupported, the pointer instrument is always there. No path ends in a dead screen.

One instrument, many inputs. Pointer, touch, and both hands all normalize into the same pitch/volume state driving a single Web Audio graph, so a new input type never means a second copy of the instrument.

Offline and installable. A service worker precaches the app and its fonts, so after one visit the instrument, tutorial, practice tools, and local portfolio work with no network. It installs to a home screen as a PWA.

Privacy by construction. Camera frames never leave the browser. Recordings live in IndexedDB on the device until you explicitly export or share them.

Tests

Pitch, scale quantisation, and the tuning maths are pure functions in lib/pitch.js, covered by test/pitch.test.js with the built-in Node test runner and no dependencies:

npm test

The suite asserts the properties that matter rather than fixed outputs: free mode never quantises, every scale mode emits only in-scale degrees, pitch rises monotonically with hand position in every mode, the field spans exactly three octaves, and the displayed note name always agrees with the frequency being played.

Design

The interface is one warm palette used semantically, not decoratively: brass means pitch and teal means volume, and that mapping holds from the logo through the on-stage antennas to the numeric readouts, so the instrument can be read before it is explained.

The identity pairs the Arabic wordmark الأثير (al-athir, "the aether") with a field emblem: the tilted orbit is the invisible field, and the vertical rod and horizontal loop are the pitch and volume antennas in their matching colours. The wordmark ships as outlined vector paths, so it renders identically everywhere and never depends on a font loading.

Run it

From the repository root:

python3 -m http.server 8080

Then open http://localhost:8080/.

Pointer mode works immediately: move horizontally for pitch and vertically for volume. Clicking the performance area starts the audio. Camera mode loads MediaPipe hand tracking from a CDN and therefore needs an internet connection on first use.

After the first visit, the pointer instrument, tutorial, practice tools, and local portfolio are available offline. Use your browser's Install app or Add to Home Screen action to keep Aether as a standalone app. Camera tracking may still need a connection if its MediaPipe files have not already been cached by the browser.

The responsive interface includes safe-area support for notched phones, touch-sized controls, pointer capture for finger playing, portrait and landscape full-camera layouts, dynamic mobile viewport sizing, and PNG/SVG install icons for iOS, Android, and desktop browsers.

Camera permission

Use http://localhost:8080/ in Chrome, Safari, or Firefox and allow camera access when prompted. Some embedded or in-app browsers block camera hardware even for local pages; in that case, open the same URL in a regular browser. Camera frames stay in the browser and are not uploaded.

Browser support

• Current Chrome and Edge on desktop and Android: instrument, camera tracking, recording, sharing, and PWA installation.
• Current Safari on macOS, iPhone, and iPad: instrument, camera tracking, recording, Home Screen installation, and the in-page full-camera fallback where element fullscreen is restricted.
• Current Firefox: instrument, camera tracking, recording, and local storage; installation and native sharing depend on the operating system.
• Camera mode requires HTTPS when hosted publicly (or localhost during development), user permission, WebGL/Wasm support, and an internet connection the first time the MediaPipe model is loaded. Pointer mode remains the fallback.

Controls

• Pointer: horizontal position controls pitch; vertical position controls volume.
• Camera: move the rightmost visible hand horizontally toward the pitch antenna for higher notes. Raise the left hand away from the volume loop for louder sound and lower it toward the loop for silence.
• Space: start or mute audio.
• C: start or stop the camera.
• Full camera: expands the playable stage, camera preview, note display, antennas, and transport controls to fill the screen.
• Camera permission and hand-tracking setup run in parallel, so the preview becomes useful sooner.
• Camera tracking requests 640×360 at 30fps, or 480×360 at 24fps on constrained devices. It follows decoded video frames when the browser supports them, adapts its inference cadence to the device, and uses an interactive-latency audio path for a quicker hand-to-sound response.
• The four-part guided lesson teaches pitch, breath, stillness, and a short C–E–G phrase with live visual feedback. Progress is stored locally on the device.
• R: start or stop a recording.
• Calibrate: choose a right- or left-handed layout and fit the pitch and volume fields to your reach. Safety mute fades the sound when both hands leave the camera.
• Measure response: estimates the current display, audio, tracking, and glide path, then suggests when a lower Glide setting would feel more immediate.
• Practice: opens a five-note pitch exercise. Land within 25 cents of each target and hold it steady for one second. Completed sessions receive a private control score, average pitch error, duration, best score, and daily streak. The player can clear this local history at any time.

Audio is generated locally with the Web Audio API. Camera frames are processed in the browser and are not uploaded by this application. Choose Free pitch mode for continuous, traditional theremin behavior; the scale modes are learning aids.

Community platform

Portfolio takes can be prepared as public or unlisted device previews without uploading the local original. A prepared take can be exported as a self-contained listening page and sent directly or put on any static host. The interaction model around it favours specific, useful responses over counters.

Nothing here simulates an audience. There are no seeded posts, no invented players, no reaction counts, and no weekly prompt. Sharing is described as what it is, a local workflow with hosting not yet connected, and the shelf where prepared takes appear stays out of the page until you have actually prepared one.

The production product, privacy, publishing, and moderation design is in ARCHITECTURE.md. Deployment boundaries and security checks are in HOSTING.md. A Supabase-ready schema with row-level security and private Storage policies is in supabase/schema.sql.

Immersive direction

The interface uses a restrained vintage electronic-instrument theme. Its optional Listening Chamber adds depth around the same controls without requiring a headset or 3D download. The next layers are a desktop/mobile 3D room and optional VR/AR modes. See IMMERSIVE.md for the longer-term direction.

License

Code is MIT licensed; see LICENSE. The bundled typefaces in fonts/ are licensed separately under the SIL Open Font License 1.1; see fonts/README.md.