A phone agent that taps screens and calls app tools

An open mobile agent nearly doubled its real-device score, and part of the trick was letting apps expose tools as well as screens.

OpenMobile-2 is a phone-using agent whose environments, data and training recipes are all open. The authors built MobileGym++, a controllable set of 35 realistic commercial-style apps, and combined it with Android emulator apps for over 110 apps in total. They release about 12K interaction trajectories and what they describe as the largest open set of verifiable RL tasks for mobile agents, with over 2K executable tasks and automatic rewards.

The interesting idea is hybrid control. Normally a mobile agent taps and swipes through a GUI. Here, apps can also expose selected functions as native tools, and the agent can mix the two inside one task. The paper implements 300+ tools across 50+ apps.

Results they report: 79.9 on AndroidWorld, 50.4 on MobileWorld, and on real devices a jump from 31.9 to 59.6 on SPA-Bench.

Why it matters beyond phones: GUI clicking is slow and brittle. When a clean function exists, calling it is faster and safer. The best agents will probably use both, and pick per step.

Would you rather your agent tap through your app, or call an API you control?