M3L: Multi-Modal Machine Listening Toolkit

M3L is a research toolkit for multi-modal machine listening, including audio representation learning, sound event detection, and reproducible experiment recipes.

Key features

• Reproducible recipes – End-to-end pipelines document every stage (data prep, training, inference, evaluation) for reproducible studies.
• HPC ready – Recipes and Makefiles are tuned for ABCI, TSUBAME, and other clusters, yet remain runnable on a single workstation.
• Highly modular – Lightning tasks, common datasets, models, and utilities are built to swap components and run ablations with minimal friction.

Getting started

For inference or use as a Python dependency, install M3L from GitHub:

pip install m3l-toolkit

For development, use Pixi to manage the project environment.

git clone git@github.com:b-sigpro/m3l.git
cd m3l
pixi install
pixi run pre-commit install

Note

The Pixi workspace currently supports Linux (linux-64), with a GPU-oriented default environment and a CPU environment for CI.

Recipes

Example experiment recipes are available under recipes/:

• DESED – Sound event detection using the DESED dataset.
• URBAN-SED – Sound event detection using synthetic urban soundscapes.
• AudioSet – Audio tagging, sound event detection, and audio-visual learning.

Dataset preparation and experiment execution require the corresponding datasets and recipe-specific configuration.

Note

These recipes are currently under maintenance.

License

M3L is licensed under the MIT License, except for the following files and directories, which are subject to their own licenses and copyright notices:

• m3l/common/nn/transformer.py – TransformerEncoderLayer and TransformerDecoderLayer adapted from PyTorch, licensed under MIT AND BSD-3-Clause.
• m3l/third/beats/ – BEATs, licensed under its own MIT.

Please refer to the linked license files before reusing or redistributing these components.

Acknowledgement

• Part of this software was developed in a project commissioned by the New Energy and Industrial Technology Development Organization (NEDO).
• Part of this software was developed by using ABCI 3.0 provided by AIST and AIST Solutions.
• Part of this software was developed by using the TSUBAME4.0 supercomputer at Institute of Science Tokyo.