Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,10 @@

# Apollo: Band-sequence Modeling for High-Quality Audio Restoration

<p align="center">
<a href="README.md">English</a> | <a href="README.zh-TW.md">Traditional Chinese guide</a>
</p>

## 📖 Abstract

Audio restoration has become increasingly significant in modern society, not only due to the demand for high-quality auditory experiences enabled by advanced playback devices, but also because the growing capabilities of generative audio models necessitate high-fidelity audio. Typically, audio restoration is defined as a task of predicting undistorted audio from damaged input, often trained using a GAN framework to balance perception and distortion. Since audio degradation is primarily concentrated in mid- and high-frequency ranges, especially due to codecs, a key challenge lies in designing a generator capable of preserving low-frequency information while accurately reconstructing high-quality mid- and high-frequency content. Inspired by recent advancements in high-sample-rate music separation, speech enhancement, and audio codec models, we propose Apollo, a generative model designed for high-sample-rate audio restoration. Apollo employs an explicit **frequency band split module** to model the relationships between different frequency bands, allowing for **more coherent and higher-quality** restored audio. Evaluated on the MUSDB18-HQ and MoisesDB datasets, Apollo consistently outperforms existing SR-GAN models across various bit rates and music genres, particularly excelling in complex scenarios involving mixtures of multiple instruments and vocals. Apollo significantly improves music restoration quality while maintaining computational efficiency.
Expand Down
80 changes: 80 additions & 0 deletions README.zh-TW.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
<p align="center">
<img src="asserts/apollo-logo.png" alt="Apollo logo" width="150"/>
</p>

# Apollo 繁體中文導讀

<p align="center">
<a href="README.md">English README</a> |
<a href="https://arxiv.org/abs/2409.08514">Paper</a> |
<a href="https://cslikai.cn/Apollo/">Demo</a>
</p>

> 為方便臺灣及其他使用繁體中文的讀者快速認識 Apollo,也希望讓更多人有機會接觸這個 open-source project,我們冒昧補上一份簡短導讀。若這項補充與專案目前的文件規劃不一致,敬請維護者包涵,也歡迎隨時指正。完整技術內容與最新資訊仍以 [English README](README.md) 及原始 paper 為準。

## Project overview

Apollo 是一個用於 high-quality audio restoration 的 generative model,目標是從 compressed audio 重建較完整的音訊內容。模型透過 explicit **frequency band split module** 建立不同 frequency bands 之間的關係,在保留 low-frequency information 的同時重建 mid- and high-frequency content。

研究使用 MUSDB18-HQ 與 MoisesDB 進行評估。模型架構、實驗設定與完整結果請參閱 [paper](https://arxiv.org/abs/2409.08514);此導讀不取代原始研究說明。

## Quick links

- [English README](README.md)
- [Apple Silicon macOS notes](MACOS_ARM64.md)
- [Apollo-data-preprocess](https://github.com/JusperLee/Apollo-data-preprocess)
- [Hugging Face checkpoint](https://huggingface.co/JusperLee/Apollo)
- [Demo](https://cslikai.cn/Apollo/)

## Installation

官方環境安裝方式:

```bash
git clone https://github.com/JusperLee/Apollo.git && cd Apollo
conda create --name look2hear --file look2hear.yml
conda activate look2hear
```

Apple Silicon 使用者可另參閱 [MACOS_ARM64.md](MACOS_ARM64.md),其中記錄 native arm64 Python、PyTorch MPS 與目前已知限制。

## Inference

以 repository 內的 public fixture 執行:

```bash
python inference.py \
--in_wav=asserts/input_wav.wav \
--out_wav=output.wav \
--device=auto
```

`--device=auto` 依序選擇 CUDA、Apple Silicon MPS、CPU。預設 checkpoint 來自官方 `JusperLee/Apollo` Hugging Face repository;自訂 `--checkpoint` 只接受明確存在且使用者信任的 local file。

可明確指定 device:

```bash
python inference.py --in_wav=input.wav --out_wav=output.wav --device=mps
python inference.py --in_wav=input.wav --out_wav=output.wav --device=cpu
```

請勿以技術測試通過推論 audio quality 已完成主觀驗收;輸出結果仍應由使用者自行聆聽與判斷。

## Dataset and training

Apollo 使用 MUSDB18-HQ 與 MoisesDB。dataset 取得方式、Source Activity Detection、data augmentation、MP3 codec simulation、rescaling 與 HDF5 preprocessing 的完整說明,請以 [English README](README.md) 和 [Apollo-data-preprocess](https://github.com/JusperLee/Apollo-data-preprocess) 為準。

Training command:

```bash
python train.py --conf_dir=configs/apollo.yml
```

## Results, license, and citation

- Results figures: [bitrates](asserts/bitrates.png), [music types](asserts/types.png)
- License: [CC BY-SA 4.0](http://creativecommons.org/licenses/by-sa/4.0/)
- Citation information: [English README](README.md#citation)
- Contact: `tsinghua.kaili@gmail.com`

感謝 Apollo maintainers 開放原始碼與研究成果。這份導讀僅協助繁體中文讀者找到正確入口;若內容與 upstream 有差異,請一律以上游英文文件為準。
Loading