Synthesia can't light up LEDs when second input

[gabeluci]

I have Sythesia running on an Android tablet that I have also plugged into the Raspberry Pi. On the tablet, enabled Developer settings, and in the dev settings, under "Networking" set the USB configuration to MIDI. It shows up as another MIDI device in the Visualizer app, where I set it as the second input.

Here's what my setup looks like. "USB func for MIDI" is my piano, and "B3-A40" is my tablet with Synthesia.

When Synthesia plays a song, the piano will play, but the LEDs will not light up.

aconnect -l looks like this:

client 0: 'System' [type=kernel]
    0 'Timer           '
    1 'Announce        '
        Connecting To: 128:1
client 14: 'Midi Through' [type=kernel]
    0 'Midi Through Port-0'
client 16: 'B3-A40' [type=kernel,card=0]
    0 'B3-A40 MIDI 1   '
        Connecting To: 20:0
        Connected From: 20:0
client 20: 'USB func for MIDI' [type=kernel,card=1]
    0 'USB func for MIDI MIDI OUT 1'
        Connecting To: 131:0, 16:0, 128:0
        Connected From: 129:0[real:0], 16:0, 128:0
client 128: 'rtpmidid' [type=user,pid=580]
    0 'Network Export  '
        Connecting To: 20:0
        Connected From: 20:0
client 129: 'RtMidiOut Client' [type=user,pid=1180]
    0 'RtMidi output   '
        Connecting To: 20:0[real:0]
client 131: 'RtMidiIn Client' [type=user,pid=1180]
    0 'RtMidi input    '
        Connected From: 20:0

The issue seems to be that the tablet ("client 16: 'B3-A40'") is not connected to "RtMidiIn Client". If I run:

aconnect 16:0 131:0

then it works. The output of aconnect -l now looks like this:

client 0: 'System' [type=kernel]
    0 'Timer           '
    1 'Announce        '
        Connecting To: 128:1
client 14: 'Midi Through' [type=kernel]
    0 'Midi Through Port-0'
client 16: 'B3-A40' [type=kernel,card=0]
    0 'B3-A40 MIDI 1   '
        Connecting To: 20:0, 131:0
        Connected From: 20:0
client 20: 'USB func for MIDI' [type=kernel,card=1]
    0 'USB func for MIDI MIDI OUT 1'
        Connecting To: 131:0, 16:0, 128:0
        Connected From: 129:0[real:0], 16:0, 128:0
client 128: 'rtpmidid' [type=user,pid=580]
    0 'Network Export  '
        Connecting To: 20:0
        Connected From: 20:0
client 129: 'RtMidiOut Client' [type=user,pid=1180]
    0 'RtMidi output   '
        Connecting To: 20:0[real:0]
client 131: 'RtMidiIn Client' [type=user,pid=1180]
    0 'RtMidi input    '
        Connected From: 20:0, 16:0

Your code automatically connects the primary input to "RtMidiIn Client", so maybe you need to also connect the secondary to it as well? But maybe there's a reason you haven't, since I'm not entirely sure the purpose of having a primary and secondary. If you end up connecting both as I suggest, then it's not really a primary and secondary but just two equal inputs.

Or maybe you have a different idea about how Synthesia is supposed to be connected?

[onlaj]

Does Synthesia light up the LEDs if you set B3-A40 as an active port?

[gabeluci]

Yes, but then the LEDs don't light up when pressing the keys.

[onlaj]

That's intended, it's either a lightshow or learning companion :D

[gabeluci]

Ok. I guess I understand your intent there. My thought is about how my kids use it. One reason I set this up is to keep my kids interested, since kids gravitate to blinking lights as much as cats gravitate to lasers 😄 They'll go back and forth between playing themselves and using Synthesia. If it's up to them to change the input to go back and forth, it will mean that they just won't do it (because kids nowadays have short attention spans).

I noticed this when I first set it up, so I updated my own copy to make that extra connection. I had to revert it to merge in your most recent code, but I'll probably make that change again for myself anyway.

[gabeluci]

I wanted to revisit this after your latest changes. I like the addition of the "Light show" vs "Learning". Something I'd like to see, and I may be able to do the work for it, is the ability to automatically switch between light show and learning when signalled by Synthesia.

Synthesia sends control messages when it starts playing a song, which is described in the old forums here: https://web.archive.org/web/20210803011834/https://www.synthesiagame.com/forum/viewtopic.php?p=43585

For example:

User-played right-hand: B0 10 05
User-played left-hand: B0 10 06
User-played both hands: B0 10 07
Watch and listen only: B0 10 00

I can take a stab at doing the work myself. From what I understand from your new documentation, the code now decides which MIDI messages to pass on rather than directly connecting them with aconnect. If I understand that correctly, could you point me to where in the code those decisions are made?

[gabeluci]

I've been looking through the code and I found what I need. I found a few bugs too. I'll submit those separately.