Skip to content

Volume HID, bounded simctl, and interaction-paced simulator capture - #9

Merged
jkelleyrtp merged 5 commits into
mainfrom
devin/1787353340-ios-streaming-patch
Aug 22, 2026
Merged

Volume HID, bounded simctl, and interaction-paced simulator capture#9
jkelleyrtp merged 5 commits into
mainfrom
devin/1787353340-ios-streaming-patch

Conversation

@jkelleyrtp

Copy link
Copy Markdown
Member

Summary

Three iOS Simulator changes, applied from a patch on top of the ADB socket work:

  1. Volume/mute buttons. SimulatorHID gains press_hid_button(page, usage, hold_ms), backed by IndigoHIDMessageForHIDArbitrary, which is dlsym'd optionally — volume up/down/mute are consumer-page usages (0x0c / 0xe9, 0xea, 0xe2) rather than Indigo button IDs. Because the symbol may be absent, the worker reports what it resolved:

    pub fn spawn_input_worker(udid: &str) -> Result<(Sender<InputCommand>, InputCapabilities)>
    //                                              ^ arbitrary_hid: bool

    SimSession::input_capabilities() exposes it so a client can hide the buttons instead of firing commands that fail on the HID thread.

  2. xcrun simctl can no longer hang the caller. settings::simctl now spawns and reaps with a 2s deadline (kill + wait on timeout) instead of Command::output(), which had no bound.

  3. Capture stays at the configured fps while the user is interacting. The framebuffer idle thread previously always slept IDLE_INTERVAL (200ms), so a screen whose only change came from an in-flight gesture or its animation was re-emitted at ~5fps. SimSession::send_input now calls capture.note_interaction(), which pushes active_until = now + 1s and wakes the thread through a Condvar; while active the tick interval is 1s / config.fps. EncodedFrame/EncodedChunk also carry captured_at from the framebuffer callback through the encoder, so latency can be measured against capture time rather than emit time.

Massaging on top of the patch

  • The patch kept a spawn_input_worker wrapper that discarded the capabilities and had no callers; folded it into the single tuple-returning spawn_input_worker.
  • The idle loop computed the active/idle interval twice with duplicated bodies; that is now CaptureState::tick_interval(active_until), called once before the wait_timeout and once after it (an interaction can arrive while the thread is parked).
  • Restored the first line of the REWIRE_INTERVAL doc comment, which the patch's hunk had eaten.

Testing

cargo fmt --all, cargo clippy --workspace --all-targets -- -D warnings, cargo test --workspace --lib --bins, cargo test -p accessibility-cli --test cli_smoke — all green on Linux.

Most of the diff is macOS-only and does not compile on this box; the x86_64-apple-darwin / aarch64-apple-darwin build jobs are the first real compile of it, and nothing here has run against a booted simulator.

Link to Devin session: https://dioxus.staging.devinenterprise.com/sessions/d88fc07ea8064b38a4b4e5a7481ffebc
Open in Devin Desktop: https://dioxus.staging.devinenterprise.com/desktop/session/d88fc07ea8064b38a4b4e5a7481ffebc?variant=devin-insiders
Requested by: @jkelleyrtp

Fold spawn_input_worker_with_capabilities back into spawn_input_worker,
and share the framebuffer tick interval between the wait and the capture
check.
@staging-devin-ai-integration

Copy link
Copy Markdown
Contributor

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR that start with 'DevinAI' or '@devin'.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

Move iOS simulator settings from spawning `xcrun simctl ui` subprocesses to
calling CoreSimulator's `SimDevice` getters and setters directly. Run the
synchronous SimDevice calls on a dedicated worker thread and expose async
methods on SimulatorControl. Add timeout handling and proper error propagation.
@jkelleyrtp
jkelleyrtp merged commit 0cea53f into main Aug 22, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant