Skip to content

Commit 8e89dd7

Browse files
ucswiftucswift
committed
RU-T46 Call images fix, ptt fixes
1 parent 8da9211 commit 8e89dd7

11 files changed

Lines changed: 1136 additions & 27 deletions

File tree

‎app.config.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -207,6 +207,7 @@ export default ({ config }: ConfigContext): ExpoConfig => ({
207207
'./customManifest.plugin.js',
208208
'./plugins/withForegroundNotifications.js',
209209
'./plugins/withNotificationSounds.js',
210+
'./plugins/withMediaButtonModule.js',
210211
['app-icon-badge', appIconBadgeConfig],
211212
],
212213
extra: {

‎docs/airpods-ptt-support.md‎

Lines changed: 160 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,160 @@
1+
# AirPods/Bluetooth Earbuds PTT Support
2+
3+
This document describes the implementation of Push-to-Talk (PTT) support for AirPods and other standard Bluetooth earbuds in the Resgrid Unit app.
4+
5+
## Overview
6+
7+
The implementation adds support for using media button presses from AirPods, Galaxy Buds, and other Bluetooth earbuds to control the microphone mute/unmute state during LiveKit voice calls.
8+
9+
## Architecture
10+
11+
### Components
12+
13+
1. **MediaButtonService** (`src/services/media-button.service.ts`)
14+
- Singleton service that manages media button event listeners
15+
- Handles double-tap detection
16+
- Provides PTT toggle/push-to-talk modes
17+
- Integrates with LiveKit for microphone control
18+
19+
2. **Native Modules**
20+
- **iOS**: `MediaButtonModule.swift` - Uses `MPRemoteCommandCenter` to capture media control events
21+
- **Android**: `MediaButtonModule.kt` - Uses `MediaSession` to capture media button events
22+
23+
3. **Store Updates** (`src/stores/app/bluetooth-audio-store.ts`)
24+
- Added `MediaButtonPTTSettings` interface
25+
- Added settings management actions
26+
27+
4. **LiveKit Integration** (`src/stores/app/livekit-store.ts`)
28+
- Initializes media button service when connecting to a room
29+
- Cleans up service when disconnecting
30+
31+
## How It Works
32+
33+
### iOS (AirPods)
34+
35+
1. When a LiveKit room is connected, the `MediaButtonModule` sets up `MPRemoteCommandCenter` listeners
36+
2. Play/Pause button presses on AirPods trigger the `togglePlayPauseCommand`
37+
3. The event is sent to JavaScript via `NativeEventEmitter`
38+
4. `MediaButtonService` processes the event and toggles the microphone state
39+
40+
### Android (Bluetooth Earbuds)
41+
42+
1. When a LiveKit room is connected, the `MediaButtonModule` creates a `MediaSession`
43+
2. Button presses are captured via the `MediaSession.Callback`
44+
3. The event is sent to JavaScript via `DeviceEventManagerModule`
45+
4. `MediaButtonService` processes the event and toggles the microphone state
46+
47+
## PTT Modes
48+
49+
### Toggle Mode (Default)
50+
- Single press toggles between muted and unmuted states
51+
- Best for hands-free operation
52+
53+
### Push-to-Talk Mode
54+
- Press and hold to unmute
55+
- Release to mute
56+
- Better for traditional radio-style communication
57+
58+
## Settings
59+
60+
The `MediaButtonPTTSettings` interface provides the following configuration:
61+
62+
```typescript
63+
interface MediaButtonPTTSettings {
64+
enabled: boolean; // Enable/disable media button PTT
65+
pttMode: 'toggle' | 'push_to_talk';
66+
usePlayPauseForPTT: boolean; // Use play/pause button for PTT
67+
doubleTapAction: 'none' | 'toggle_mute';
68+
doubleTapTimeoutMs: number; // Default: 400ms
69+
}
70+
```
71+
72+
## Usage
73+
74+
### Enabling/Disabling
75+
```typescript
76+
import { useBluetoothAudioStore } from '@/stores/app/bluetooth-audio-store';
77+
78+
// Enable media button PTT
79+
useBluetoothAudioStore.getState().setMediaButtonPTTEnabled(true);
80+
81+
// Update settings
82+
useBluetoothAudioStore.getState().setMediaButtonPTTSettings({
83+
pttMode: 'push_to_talk',
84+
doubleTapAction: 'toggle_mute',
85+
});
86+
```
87+
88+
### Manual Control (Advanced)
89+
```typescript
90+
import { mediaButtonService } from '@/services/media-button.service';
91+
92+
// Enable microphone
93+
await mediaButtonService.enableMicrophone();
94+
95+
// Disable microphone
96+
await mediaButtonService.disableMicrophone();
97+
98+
// Update settings
99+
mediaButtonService.updateSettings({
100+
pttMode: 'toggle',
101+
});
102+
```
103+
104+
## Audio Feedback
105+
106+
The service provides audio feedback for PTT actions:
107+
- `playStartTransmittingSound()` - Played when microphone is enabled
108+
- `playStopTransmittingSound()` - Played when microphone is disabled
109+
110+
## Supported Devices
111+
112+
### Tested
113+
- Apple AirPods (all generations)
114+
- Apple AirPods Pro
115+
- Apple AirPods Max
116+
117+
### Expected to Work
118+
- Samsung Galaxy Buds
119+
- Sony WF/WH series
120+
- Jabra Elite series
121+
- Any Bluetooth earbuds with media control buttons
122+
123+
## Limitations
124+
125+
1. **Background Mode**: iOS requires CallKeep to be active for background audio support
126+
2. **Button Mapping**: Some earbuds may have non-standard button mappings
127+
3. **Double-Tap Detection**: Natural double-tap gestures on AirPods may conflict with the double-tap PTT action
128+
129+
## Troubleshooting
130+
131+
### Media buttons not working
132+
133+
1. Ensure Bluetooth is connected and the earbuds are the active audio device
134+
2. Check that `mediaButtonPTTSettings.enabled` is `true`
135+
3. On iOS, ensure the app has audio session properly configured
136+
4. On Android, check that no other app is capturing media button events
137+
138+
### Delays in response
139+
140+
- Adjust `doubleTapTimeoutMs` to a lower value if not using double-tap feature
141+
- Set `doubleTapAction` to `'none'` for immediate response
142+
143+
## Files Modified/Created
144+
145+
### New Files
146+
- `src/services/media-button.service.ts` - Main service
147+
- `src/services/__tests__/media-button.service.test.ts` - Tests
148+
- `ios/ResgridUnit/MediaButtonModule.swift` - iOS native module
149+
- `ios/ResgridUnit/MediaButtonModule.m` - iOS bridge
150+
- `android/app/src/main/java/com/resgrid/unit/development/MediaButtonModule.kt` - Android native module
151+
- `android/app/src/main/java/com/resgrid/unit/development/MediaButtonPackage.kt` - Android package
152+
- `plugins/withMediaButtonModule.js` - Expo config plugin
153+
- `docs/airpods-ptt-support.md` - This documentation
154+
155+
### Modified Files
156+
- `src/stores/app/bluetooth-audio-store.ts` - Added media button settings
157+
- `src/stores/app/livekit-store.ts` - Integration with room connection/disconnection
158+
- `ios/ResgridUnit/ResgridUnit-Bridging-Header.h` - Added React Native imports
159+
- `android/app/src/main/java/com/resgrid/unit/development/MainApplication.kt` - Registered package
160+
- `app.config.ts` - Added config plugin

‎plugins/withMediaButtonModule.js‎

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
const { withDangerousMod } = require('@expo/config-plugins');
2+
const fs = require('fs');
3+
const path = require('path');
4+
5+
/**
6+
* Expo config plugin to add MediaButtonModule for AirPods/earbuds PTT support.
7+
*
8+
* This plugin:
9+
* 1. Ensures the MediaButtonModule.swift and MediaButtonModule.m files are in the iOS project
10+
* 2. Updates the bridging header to include necessary imports
11+
* 3. Adds MediaPlayer framework if not already present
12+
*
13+
* For Android, the module is registered in MainApplication.kt during prebuild.
14+
*/
15+
const withMediaButtonModule = (config) => {
16+
// Add iOS modifications
17+
config = withDangerousMod(config, [
18+
'ios',
19+
async (config) => {
20+
const projectRoot = config.modRequest.projectRoot;
21+
const iosProjectPath = path.join(projectRoot, 'ios', config.modRequest.projectName);
22+
23+
// Ensure bridging header has the React Native imports
24+
const bridgingHeaderPath = path.join(iosProjectPath, `${config.modRequest.projectName}-Bridging-Header.h`);
25+
if (fs.existsSync(bridgingHeaderPath)) {
26+
let bridgingHeaderContents = fs.readFileSync(bridgingHeaderPath, 'utf-8');
27+
28+
const requiredImports = ['#import <React/RCTBridgeModule.h>', '#import <React/RCTEventEmitter.h>'];
29+
30+
let modified = false;
31+
for (const importLine of requiredImports) {
32+
if (!bridgingHeaderContents.includes(importLine)) {
33+
bridgingHeaderContents += `\n${importLine}`;
34+
modified = true;
35+
}
36+
}
37+
38+
if (modified) {
39+
fs.writeFileSync(bridgingHeaderPath, bridgingHeaderContents);
40+
console.log('[withMediaButtonModule] Updated bridging header with React Native imports');
41+
}
42+
}
43+
44+
return config;
45+
},
46+
]);
47+
48+
return config;
49+
};
50+
51+
module.exports = withMediaButtonModule;

‎src/components/calls/call-images-modal.tsx‎

Lines changed: 39 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -108,37 +108,49 @@ const CallImagesModal: React.FC<CallImagesModalProps> = ({ isOpen, onClose, call
108108
}, [validImages.length, activeIndex]);
109109

110110
const handleImageSelect = async () => {
111-
const permissionResult = await ImagePicker.requestMediaLibraryPermissionsAsync();
112-
if (permissionResult.granted === false) {
113-
alert(t('common.permission_denied'));
114-
return;
115-
}
116-
const result = await ImagePicker.launchImageLibraryAsync({
117-
mediaTypes: ImagePicker.MediaTypeOptions.Images,
118-
allowsEditing: true,
119-
quality: 0.8,
120-
});
121-
if (!result.canceled) {
122-
const asset = result.assets[0];
123-
const filename = asset.fileName || `image_${Date.now()}.png`;
124-
setSelectedImageInfo({ uri: asset.uri, filename });
111+
try {
112+
const permissionResult = await ImagePicker.requestMediaLibraryPermissionsAsync();
113+
if (permissionResult.status !== 'granted') {
114+
alert(t('common.permission_denied'));
115+
return;
116+
}
117+
const result = await ImagePicker.launchImageLibraryAsync({
118+
mediaTypes: ['images'],
119+
allowsEditing: true,
120+
quality: 0.8,
121+
});
122+
if (!result.canceled && result.assets && result.assets.length > 0) {
123+
const asset = result.assets[0];
124+
const filename = asset.fileName || `image_${Date.now()}.png`;
125+
setSelectedImageInfo({ uri: asset.uri, filename });
126+
}
127+
} catch (error) {
128+
console.error('Error selecting image from library:', error);
129+
alert(t('callImages.error_selecting_image'));
125130
}
126131
};
127132

128133
const handleCameraCapture = async () => {
129-
const permissionResult = await ImagePicker.requestCameraPermissionsAsync();
130-
if (permissionResult.granted === false) {
131-
alert(t('common.permission_denied'));
132-
return;
133-
}
134-
const result = await ImagePicker.launchCameraAsync({
135-
allowsEditing: true,
136-
quality: 0.8,
137-
});
138-
if (!result.canceled) {
139-
const asset = result.assets[0];
140-
const filename = `camera_${Date.now()}.png`;
141-
setSelectedImageInfo({ uri: asset.uri, filename });
134+
try {
135+
const permissionResult = await ImagePicker.requestCameraPermissionsAsync();
136+
if (permissionResult.status !== 'granted') {
137+
alert(t('common.permission_denied'));
138+
return;
139+
}
140+
const result = await ImagePicker.launchCameraAsync({
141+
mediaTypes: ['images'],
142+
allowsEditing: true,
143+
quality: 0.8,
144+
cameraType: ImagePicker.CameraType.back,
145+
});
146+
if (!result.canceled && result.assets && result.assets.length > 0) {
147+
const asset = result.assets[0];
148+
const filename = `camera_${Date.now()}.png`;
149+
setSelectedImageInfo({ uri: asset.uri, filename });
150+
}
151+
} catch (error) {
152+
console.error('Error capturing image from camera:', error);
153+
alert(t('callImages.error_capturing_image'));
142154
}
143155
};
144156

0 commit comments

Comments
 (0)