Add local audio peak BPM detection to synchronize dance animations

[pongsathorncha]

Modification Report: BPM Dance Synchronization (V1.2 Stable)

This document summarizes all the changes made to the original state of the project to implement the local BPM synchronization feature.

Note

All modifications were strictly confined to a single script to minimize the footprint and maintain the project's original structure.

Modified File

Assets/MATE ENGINE - Scripts/AvatarHandlers/AvatarAnimatorController.cs

Summary of Code Changes

1. New State Variables Added

We introduced tracking variables specifically for the Energy Flow / BPM logic.

[Header("BPM Sync")]
private AudioSessionControl activeAudioSession;
private List<float> bpmHistory = new List<float>();
private float lastBeatTime = 0f;
private float dynamicBeatThreshold = 0.05f;
private float currentEstimatedBPM = 120f;
private float lastValidSoundTime = 0f; // Used for Dance Stop Grace Period

2. Updated IsValidAppPlaying()

Original: Scanned the system for allowed audio apps (from the user's UI list) and immediately returned true to trigger the dance animation if one was making sound.
New: Before returning true, it now captures and stores the exact audio source into activeAudioSession. Because the original creator already built a brilliant UI for "App Isolation", storing this session allows our new BPM detector to read the volume peak of only that specific app (e.g., Spotify). This ensures our beat detection doesn't accidentally trigger from system sounds like Discord notifications!

- if (pname.StartsWith(allowedApps[j], System.StringComparison.OrdinalIgnoreCase)) return true;
+ if (pname.StartsWith(allowedApps[j], System.StringComparison.OrdinalIgnoreCase))
+ {
+     activeAudioSession = s; // Track the session
+     return true;
+ }

3. Updated SetDancing(bool value)

Original: Simply turned the animator parameter on/off.
New: When the avatar stops dancing (value == false), we actively wipe our BPM trackers and force the animator's speed back to standard (1.0f). This safely guarantees the idle state remains completely unaffected by the dance logic.

- if (!value && danceTransitionCoroutine != null)
+ if (!value)
+ {
+     animator.speed = 1f; // Reset speed when not dancing
+     bpmHistory.Clear();
+     activeAudioSession = null;
...

4. Added ProcessBpmSync() Method

Original: Did not exist.
New: This entirely new method acts as an "Energy Flow" detector. It:

• Applies a slow decay to a dynamicBeatThreshold to naturally filter out weak background noises.
• Halves the detected BPM if it spikes over 135 BPM (forcing a realistic, bouncy dance for very fast tracks instead of vibrating rapidly).
• Averages a rolling history of 16 beats to act as a smooth low-pass energy filter.
• Directly updates animator.speed dynamically between 0.5x and 1.5x for hyper-responsive animation scaling.

5. Added "Dance Stop" Grace Period

Original: During songs that feature sparse rhythms (like the isolated kick drum intro of Ghosts 'n' Stuff), the avatar would constantly start dancing, stop, and start again because the music momentarily dipped below the threshold.
New: Added a simple lastValidSoundTime cooldown check. The avatar now features a "Grace Period" that forces her to continue dancing smoothly through brief gaps in the music, completely fixing the stuttering issue.

+ // If we are currently dancing, and heard a peak recently, ignore this silence.
+ if (isDancing && Time.time - lastValidSoundTime < 1.5f) {
+     return true;
+ }

6. Updated Update()

Original: Managed dragging inputs and timers.
New: Injects the execution of ProcessBpmSync() so that the beat detector runs every single frame only when the avatar is actively dancing.

[pongsathorncha]

Hi, @shinyflvre! I've been testing this BPM sync feature and it feels great. I know you're busy, so take your time reviewing! In the meantime, I've compiled a test release on my fork for anyone who wants to try it out and provide feedback before it gets merged: https://github.com/pongsathorncha/Mate-Engine/releases/tag/x3.3.0-bpm

📝 Note on Fast Songs: To keep the avatar looking realistic and human, I coded it so that if a song is extremely fast (over 135 BPM), the avatar will naturally drop into a "half-time" groove. This prevents the animations from looking like a fast-forwarded, glitchy robot!

[pongsathorncha]

Hi @shinyflvre!

Just a quick update: I have pushed a new V1.2 Stable version of the code to this pull request. After a lot of my own testing on many style of songs, I completely refined the algorithm to focus on "Energy Flow" rather than strict metronome math.

Key improvements in this update:

16-Beat Rolling Average: Acts as a natural low-pass filter so the avatar smoothly glides her dancing speed up and down to match the vibe of the song without snapping.
"Dance Stop" Grace Period: Fixed a stuttering bug where songs with sparse kick-drum intros (like Ghosts 'n' Stuff) would cause the avatar to rapidly start and stop dancing.
I have updated the main PR description above with the new code diffs. For anyone who wants to test this final stable version before it gets merged, you can download the release directly from my fork here: https://github.com/pongsathorncha/Mate-Engine/releases/tag/x3.3.0-bpm-v1.2

Thanks again for reviewing when you have the time!