esp32-iphone-agent

An open-source project that lets an AI model operate a real iPhone using a $7 ESP32 board, seeing the screen over AirPlay and tapping through USB input.

Why it matters

Lets an AI vision model operate a real, unmodified iPhone using about $7 of hardware. It's for makers and testers who want phone automation without a jailbreak or a developer account.

Highlights

• Eyes: the ESP32-S3 board acts as an AirPlay receiver on its own hotspot and forwards the screen stream to the app over BLE.
• Hands: the board is a USB HID mouse and keyboard. With AssistiveTouch on, clicks become taps and drags become swipes.
• Brain: an iPhone app sends screenshots to any OpenAI-compatible vision model and runs a planner and actor, with pointer re-homing and miss correction.
• Every run is kept on the phone for step-by-step replay, and traces can be exported for review by another agent.
• The original computer-based version is kept as tag v0.1.0 and a legacy branch.

Taken from the project's own README, docs or launch page. Claims about speed or accuracy are the maker's unless stated otherwise.

Worth knowing: The repo tagline says "No Mac", but the README requires a Mac with Xcode once to flash the board and build the app. The README also warns to ignore install commands posted in issues or comments.

Quick facts

Links

• Repository: https://github.com/onehumanbeing/esp32-iphone-agent
• Release notes: https://github.com/onehumanbeing/esp32-iphone-agent/releases
• Docs: https://github.com/onehumanbeing/esp32-iphone-agent/blob/main/docs/legacy.md

Related ranking

See how established tools compare in Best AI Agents (General & Computer-Use). New releases aren't scored until we review them against the rubric.

Checked 2026-10-17 from the tool's own pages. Prices and star counts change, so confirm on the official site. Listing here is not a ranking or an endorsement.