Skip to content

feat(app): camera for iOS simulator apps (webcam, image, video) - #76

Merged
dungngminh merged 1 commit into
mainfrom
feat/ios-simulator-camera
Oct 21, 2026
Merged

dungngminh merged 1 commit into
mainfrom
feat/ios-simulator-camera

Conversation

@dungngminh

@dungngminh dungngminh commented Oct 21, 2026 •

Copy link
Copy Markdown
Owner

Description

iOS Simulator has no camera. This gives simulator apps one in the macOS desktop app, ported from serve-sim (Apache-2.0):

  • Injector: libSimCameraInjector.dylib swizzles AVFoundation / UIImagePickerController inside an app launched with SIMCTL_CHILD_DYLD_INSERT_LIBRARIES. Sources are vendored in app/macos/SimCameraInjector/ with LICENSE + NOTICE. Only the fake camera names and the fallback image text were changed, to say SimUtil. A Runner script phase builds it (arm64 + x86_64) into the app's Resources.
  • Feed: serve-sim's host helper runs in-process as SimCameraFeed (ObjC: a POSIX shm header and a ring of global IOSurfaces, one per simulator), behind the simutil/sim_camera method channel. Sources: Mac webcam, image, looping video.
  • UI: the device menu and the camera button on a running simulator's stream both get a Camera menu (webcams / Image… / Video…, Open app with camera, Turn off camera). Switching the source updates running apps live. Closing the stream or shutting the simulator down turns the camera (and webcam) off.
  • MCP: launch_app takes camera: webcam, webcam:<name>, image:<path> or video:<path>.
  • simutil_apple 1.4.0: IosAutomation.launchWithCamera (camera privacy grant + env-prefixed simctl launch through CommandExec) and installedApps.
  • Release: release-app.yaml signs the dylib before the --deep app sign, because --deep skips Mach-O files in Resources.

Apps get the camera through AVCaptureDevice.default(_:for:position:), discovery sessions and the image picker. A known gap from upstream: AVCaptureDevice.default(for: .video) is not hooked.

Screenshots

A test app with an AVCaptureVideoPreviewLayer, launched from SimUtil (iPhone 17 Pro, iOS 27):

Image Video

The webcam source was checked too, through SimUtil's MCP launch_app + screenshot.

Verification

  • flutter analyze --fatal-infos and flutter test (114) in app/; dart analyze --fatal-infos and the tests in simutil_apple.
  • Image, video and webcam feeds end to end on a simulator.
  • Not verified: notarizing a release build with the simulator dylib inside it.

Type of Change

  • 🔥 New feature (non-breaking change which adds functionality)
  • ✨ Enhancement (non-breaking change which improves existing functionality)
  • 🛠️ Bug fix (non-breaking change which fixes an issue)
  • 🧹 Code refactor
  • 📝 Documentation
  • 🗑️ Chore
  • 📦 Build configuration change
  • ✅ Test

Summary by CodeRabbit

  • New Features

    • Added macOS iOS Simulator camera support.
    • Select a webcam, image, or looping video from device controls or the stream action.
    • Switch camera sources while apps are running; camera feeds stop when simulators shut down.
    • Added optional camera input to app launches, including MCP-driven launches.
    • Added a list of installed simulator apps for launch workflows.
  • Bug Fixes

    • Improved camera feed cleanup when simulator sessions end.
  • Documentation

    • Added release notes and package changelog entries for simulator camera support.

Port serve-sim's simulator camera: libSimCameraInjector.dylib (vendored
verbatim in app/macos/SimCameraInjector with LICENSE/NOTICE, built by a
Runner script phase into Resources) swizzles AVFoundation inside an app
launched with SIMCTL_CHILD_DYLD_INSERT_LIBRARIES. Its host helper runs
in-process as SimCameraFeed (ObjC, one shm + IOSurface ring per
simulator) behind the simutil/sim_camera channel.

Device menu: Camera > webcam / Image / Video, Open app with camera,
Turn off camera. MCP launch_app takes camera. simutil_apple 1.4.0 adds
IosAutomation.launchWithCamera and installedApps. Release signing signs
the dylib before the --deep app sign.
@dungngminh dungngminh added feature Features ✨ macos app Desktop app only: shows in app-v* release notes labels Oct 21, 2026
@coderabbitai

coderabbitai Bot commented Oct 21, 2026

Copy link
Copy Markdown

Review in Change Stack →

Note

Currently processing new changes in this PR. This may take a few minutes, please wait...

⚙️ Run configuration
  • Configuration used: Repository UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 59c9b478-9b28-4f12-9cc2-1f08a0422bc0
📥 Commits

Reviewing files that changed from the base of the PR and between 444037f and 3406a43.

📒 Files selected for processing: 34
  • .github/workflows/release-app.yaml
  • app/CHANGELOG.md
  • app/lib/main.dart
  • app/lib/src/mcp/tools/input_tools.dart
  • app/lib/src/stream/ios/ios_sim_session.dart
  • app/lib/src/stream/ios/sim_camera.dart
  • app/lib/src/ui/shared/device_context_menu.dart
  • app/lib/src/ui/streams/stream_tile.dart
  • app/macos/Runner.xcodeproj/project.pbxproj
  • app/macos/Runner/Info.plist
  • app/macos/Runner/MainFlutterWindow.swift
  • app/macos/Runner/Runner-Bridging-Header.h
  • app/macos/Runner/SimCamera.swift
  • app/macos/Runner/SimCameraFeed.h
  • app/macos/Runner/SimCameraFeed.m
  • app/macos/SimCameraInjector/LICENSE
  • app/macos/SimCameraInjector/NOTICE
  • app/macos/SimCameraInjector/SimCamFakes.h
  • app/macos/SimCameraInjector/SimCamFakes.m
  • app/macos/SimCameraInjector/SimCamFrameSource.h
  • app/macos/SimCameraInjector/SimCamFrameSource.m
  • app/macos/SimCameraInjector/SimCamLog.h
  • app/macos/SimCameraInjector/SimCamLog.m
  • app/macos/SimCameraInjector/SimCamSwizzles.h
  • app/macos/SimCameraInjector/SimCamSwizzles.m
  • app/macos/SimCameraInjector/SimCameraInjector.m
  • app/macos/SimCameraInjector/build.sh
  • app/macos/SimCameraInjector/include/SimCamShared.h
  • app/pubspec.yaml
  • app/test/sim_camera_test.dart
  • packages/simutil_apple/CHANGELOG.md
  • packages/simutil_apple/lib/src/ios_automation.dart
  • packages/simutil_apple/pubspec.yaml
  • packages/simutil_apple/test/services/ios_automation_test.dart
+-----------------------------------------------------------------+
| You are pretty high on the list of people I want to help today. |
+-----------------------------------------------------------------+
  \
   \   (\__/)
       (•ㅅ•)
       /   づ
✨ Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@dungngminh
dungngminh merged commit ec1dfc2 into main Oct 21, 2026
5 of 7 checks passed
@dungngminh
dungngminh deleted the feat/ios-simulator-camera branch October 21, 2026 18:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

app Desktop app only: shows in app-v* release notes feature Features ✨ macos

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant