本文へスキップ

Docs Remote Control Remote Control

Phones and Tablets

VMPal shows an iPhone's, iPad's or Android device's screen in a window on your Mac, and you use it with your mouse and keyboard. It works by cable or over Wi-Fi, and with iOS simulators too. Your AI agent can use the device as well: it reads the screen, taps through your app, installs builds, and reads their logs and crash reports.

Phones and tablets need VMPal 0.119 or later. There's nothing to install from the App Store or Google Play.

iPhone and iPad

Apple lets only its testing tools control an iPhone, so VMPal uses Xcode's. On your Mac you need:

  • Xcode, from the App Store.
  • An Apple account signed in to Xcode › Settings › Accounts. A free account works, but what it signs lasts 7 days.

On the iPhone or iPad:

  1. Connect it to your Mac with a cable once, unlock it, and choose Trust. After that, it also works over Wi-Fi, on the same network as your Mac.
  2. Turn on Developer Mode in Settings › Privacy & Security › Developer Mode. It restarts.
  3. Keep it unlocked while VMPal connects.

The first time you connect, Xcode builds a small test app, signs it with your account and starts it on the device, which takes a minute or two. The device then shows Automation Running. If iOS says it doesn't trust the developer, go to Settings › General › VPN & Device Management, choose your account, then Trust.

iOS simulators

The simulators Xcode has show in the same list. They need no Apple account. VMPal starts a simulator without Simulator's window, and shows its screen in VMPal's.

Android phones and tablets

  1. On the device, turn on Developer options: in Settings › About phone, tap Build number 7 times.
  2. In Settings › System › Developer options, turn on USB debugging.
  3. Connect it to your Mac with a cable, unlock it, and choose Allow when it asks about USB debugging. For Wi-Fi, turn on Wireless debugging and connect it with adb pair and adb connect in Terminal.

Your Mac needs Android's adb, which comes with Android Studio, or with Homebrew: brew install android-platform-tools. The first time you connect, VMPal downloads the server of scrcpy, the open source screen mirroring tool, checks it, and starts it on the device.

Add it to VMPal

  1. Choose File › Add Phone or Tablet…, or click + in the Remote section's toolbar, then Add Phone or Tablet….
  2. Choose the device. VMPal says whether it's ready, and what to do on it if it isn't. Click Refresh after you change something.
  3. For an iPhone or iPad, choose the Apple account to sign VMPal's test app with, under Sign with.
  4. Click Add.

The device then shows in the Remote section, and its window opens.

The window

On your Mac iPhone and iPad Android
Click Tap Tap
Drag Swipe, played when you let go Swipe, as you move
Right-click Long press Back
Middle click Home
Scroll Short swipe Scroll
Type Types into the field in focus Types into the field in focus

An Android device's screen comes as video. An iPhone's comes as pictures, up to 20 a second, sent only when the screen changes. Quality in the toolbar works as for a Mac.

Test your app with your AI agent

Your AI agent uses a phone with the same tools as a Mac or a VM: it reads the screen as a list of buttons and text, taps them, types and takes screenshots. To test your app, it can also install a build, launch it, and read what it printed and how it crashed. Ask it, for example: "Install the build in ./build on my iPhone, open it, sign in with the test account, and tell me what's wrong."

The same works from Terminal with vmpal:

vmpal device "My iPhone" install ./build/MyApp.app
vmpal device "My iPhone" launch com.example.MyApp
vmpal ui "My iPhone" tree
vmpal device "My iPhone" logs com.example.MyApp
vmpal device "My iPhone" crashes com.example.MyApp

vmpal help device lists every action: install, launch, terminate, uninstall, apps, logs, crashes, crash, rotate, pinch, button, open-url and release.

  • What to install: an iPhone or iPad takes an .app or .ipa signed for it, a simulator an .app built for the simulator, Android an .apk. Build it with Xcode or Gradle first.
  • Logs: launch keeps what the app prints from then on, which logs reads. On Android, logs reads the app's logcat.
  • Your own UI tests: an iPhone runs one test session at a time. vmpal device "My iPhone" release ends VMPal's, so Xcode can run yours. Opening the device's window starts VMPal's again.

Good to know

  • iPhone drags are whole gestures. Xcode's testing tools play a swipe from start to end, so on an iPhone or iPad, a drag happens when you let go of the button. Android follows your mouse as it moves.
  • Home on an iPhone: vmpal device "My iPhone" button home, or ask your AI agent.
  • Locked devices: an iPhone or iPad has to be unlocked when VMPal connects. After that, it stays usable.