Latent Gloves

Latent Gloves is a wearable Digital Musical Instrument (DMI) designed for the embodied exploration of neural audio latent spaces through hand gestures. The instrument combines a DIY glove-based controller, embedded sensor processing, and a neural audio synthesis engine based on the nn~ Max package to transform a vocal audio recording using a RAVE timbre transfer model.

This instrument was used in performances at:

• Interactive Digital Multimedia Techniques (IDMT) module concert from the Sound and Music Computing MSc at Queen Mary University of London, Dec 2025.
• AI Playground at Sónar+D, Barcelona, Jun 2026.
• NIME LATAM Gathering, at NIME 2026, London, Jun 2026.

Build your own glove

Read the building guide

TODO: Update with new prototype and lessons learned.

Software Requirements

• Max 8 or later
• Arduino IDE
• nn~ Max Package
• Download pretrained RAVE model models by Shuoyang Zheng
• Place downloaded model in nn~ models folder

Instructions for playing the instrument

Setup

1. Put the glove on and connect it via to the PC via USB

2. Connect MIDI Interface to PC (if additional control is wanted)

3. Open Arduino/Latent_gloves/Latent_gloves.ino on Arduino IDE, select the Ardino Nano ESP32 board and click Upload.

4. (Calibration) While holding the hand in a neutral, horizontal position with fingers extended and the palm facing the floor, run the code. The LED will start bliking. Wait until the blinking stops.

5. Open the Serial Monitor to check if the calibration ran without errors, you should be seeing the values of flexion in each finger and the pitch and roll angles updating.

6. Close Serial Monitor.

Play

For playing, assuming everything is working correctly, you will only need to plug the glove, as the program is already saved in the board. Remember that everytime the board is turned on the calibration is triggered, so act accordingly, or manually trigger the calibration by pressing the reset button on the board.

1. Connect Glove to PC

2. Wait for Calibration to finish (wait until the blinking stops)

3. Ensuring the Serial Port is closed, open Max/Latent Gloves.maxpat Max Patch in Max and follow instructions on screen.

Acknowledgments

• Max Patch is Max/Latent Gloves.maxpat based on examples of acid-ircam's nn~ Max Package
• Pre-trained RAVE model downloaded from Shuoyang Zheng's pre-trained RAVE models
• Max Patch test Max/Tests/Latent Gloves - regression test.maxpat is based on FluCoMa's tutorial, Controlling a Synth using a Neural Network
• Arduino MPU6050 code in Arduino/Latent_gloves/Latent_gloves.ino based on examples of Adafruit's MPU6050 library
• Arduino Kalman filter code in Arduino/Latent_gloves/Latent_gloves.ino based on Kristian Lauszus (TKJ Electronics) implementation