A research team from six institutions, including Nanjing University and SenseTime, has released the model, training data, and training environment for OpenMobile-2, an AI that operates smartphones. According to an arXiv preprint published on October 9, 2026, the 27-billion-parameter model achieved a 79.9±3.0% success rate on AndroidWorld, a benchmark that measures the ability to operate Android apps. OpenMobile-2 recognizes the screen and carries out tasks given by the user through actions such as tapping and typing text. It matched commercial models in some evaluations, but results differ between the Android emulator and real devices, and training also uses simulated apps that run in a browser. What matters in this research is not only the model's performance, but also that the team has released a training environment in which many different apps can be operated repeatedly, along with a method that combines screen operation and tool calls.
Performance rivaling commercial AI, but results vary by evaluation environment
To assess OpenMobile-2's performance, it is necessary to distinguish the execution environment and measurement conditions of each benchmark. AndroidWorld covers 116 tasks across 20 apps, and MobileWorld covers 201 tasks across 20 apps; both run on an Android emulator. SPA-Bench, by contrast, is an evaluation that uses actual smartphones. Extracting the results for OpenMobile-2 and Google's Gemini-3.1-pro-preview from Table 2 of the paper published on October 9, 2026 (v1) gives the following. Pass@1 on AndroidWorld is the rate at which a task is completed in a single attempt, the GUI column for MobileWorld represents results using screen operation, and SPA-Bench figures are limited to difficulty level L3 tasks.
| Model | AndroidWorld Pass@1 | MobileWorld GUI | SPA-Bench L3 (real device) |
|---|---|---|---|
| OpenMobile-2-27B +RL | 79.9±3.0% | 50.4% | 59.6% |
| Gemini-3.1-pro-preview | 70.7% | 58.1% | 70.2% |
In this comparison, OpenMobile-2-27B outperforms Gemini on AndroidWorld but falls short on MobileWorld's screen-operation evaluation and on SPA-Bench L3, which uses real devices. The ± on AndroidWorld indicates the standard deviation across random seeds used during evaluation, and the SPA-Bench figures are not the overall results for all 340 tasks. Because each benchmark differs in its target tasks and evaluation conditions, the three results cannot simply be added together to judge which model is better.
On real-device SPA-Bench L3, GPT-5.6-Sol, which the research team re-measured, also scored 68.1%, ahead of OpenMobile-2-27B's 59.6%. However, the commercial-model results in the paper include previously reported figures, and even the re-measured models were evaluated using different execution frameworks. This table is therefore a comparison compiled by the research team, not a ranking in which an independent third party measured all models under identical conditions.
Reinforcement learning also did not improve every evaluation result. At the supervised fine-tuning stage that combined screen operation and tool calls, the 27B model's AndroidWorld success rate was 80.8±3.5%, which became 79.9±3.0% after reinforcement learning. Because evaluation results vary, this difference alone does not justify concluding that reinforcement learning worsened performance. The result is best understood as the release of a research model that approaches commercial models in specific evaluation environments, together with a foundation that lets third parties verify its training methods.
An environment covering more than 110 apps, with repeated training from the same state
"MobileGym++," developed by the research team, is a mechanism for expanding the training environment for smartphone-operating AI. The team newly reimplemented 35 commercial apps, and including existing ones, 63 app clients can now run in the browser. Together with the apps that run on the Android emulator, the training environment as a whole handles more than 110 apps. However, this does not mean it connects directly to 35 commercial services; it means research apps were built to imitate their functions and operations.
Training on an Android emulator has the problem of heavy load from initializing the environment and running multiple instances in parallel. In addition, operating real apps changes login states and triggers processes such as orders and reservations, making it difficult to train repeatedly under the same conditions. In MobileGym++, details such as order contents, budgets, and itineraries are managed as data inside the app, and that state can be saved and restored. The environment also supports a shared clipboard, sharing functions, navigation to payment screens, and media selection, so it can reproduce sequences of operations spanning multiple apps. Its distinguishing feature is that it builds an environment in which operations change internal data, rather than merely reproducing how screens look.
To create the training data, the team used Gemini-3.1-Pro-Preview as the teacher model. The researchers had this model generate task instructions and operation trajectories, then selected the successful cases to build the dataset. According to the public data card, the collected trajectories number 11,643, with a total of 118,582 operation steps. The breakdown is 7,725 trajectories collected on the Android emulator, 2,144 in the simulated environment using screen operation only, and 1,774 combining screen operation and tool calls. For reinforcement learning, 2,475 tasks equipped with goals, initial states, and success-checking programs have also been released. However, only 512 of these were actually used for reinforcement learning.
While releasing the data and environment makes the training method easier to reproduce and verify, the stage that generates trajectories depends on a commercial teacher model. This point matters when judging whether the released models and datasets alone can independently reproduce the entire training pipeline.
The experiments also tested how the diversity of apps in the training environment affects performance. In Section 5.3 of the paper, the team held the base model, optimization conditions, and the total of 2,000 emulator-collected trajectories fixed, and compared increasing the number of training apps from 15 to 53. As a result, Pass@1 on apps not used in training rose from 60.7% to 67.5%. Meanwhile, performance on the original 15 apps went from 73.7% to 71.4%, which the team judged to be broadly stable.
What changed in this experiment was not the total amount of training data but the types and allocation of apps from which operating experience was collected. In other words, even with the same number of trajectories, collecting them from more diverse apps may improve the ability to handle unfamiliar apps. However, these results were obtained in a research evaluation environment, and the same effect will not necessarily appear in every real service. The release of MobileGym++, which can save and restore state, is also meaningful for developers who want to examine such allocation of training data and diversity of environments.
Combining screen operation and tool calls to make tasks more efficient
Another feature of MobileGym++ is that the AI can switch between operating the screen directly and calling app functions as tools within the same task. The team prepared tools for calling more than 300 in-app functions across more than 50 apps. For example, running add_to_cart, which adds a product to the cart, updates the same cart contents as when the product is opened through screen operation. This is because screen operation and tool calls do not run in separate environments but share the same internal app data. The official OpenMobile-2 site also explains this design built on shared state management.
The model is shown only the tools available in the app it is currently operating. Tools have defined argument formats, and executing one returns a result or an error. Screen operation, on the other hand, uses screenshot image information, and the model does not directly read the accessibility tree or the app's internal state. This allows the method of looking at the screen to tap and type and the method of directly executing app functions with tools to be combined within a single task.
To compare the two, the team prepared 215 tasks that can use the same initial state and success conditions whether screen operation alone or tools in combination are used. Of the 215 tasks, 109 are completed within a single app, and the rest require operations across multiple apps. Whether a task succeeds is not left to the AI's own judgment; success-checking programs inspect changes in the app's internal data and similar signals. This makes it possible to compare whether the same goal was achieved even when the method of operation differs.
However, simply providing available tools does not make the AI able to use them appropriately. The team reports that unless the model learns both screen operation and tool calls, it cannot adequately judge when to use tools. In fact, when the same 27B model trained on both methods was evaluated on MobileGym++, its success rate was 55.3% with screen operation only, but rose to 65.6% when tools were also used. The difference in success rate was 10.3 points, and the number of operation steps to complete a task also fell by an average of 5.8.
This result does not compare two models built with different training methods; it comes from an experiment that changed the operating method for the same trained model. It shows that combining screen operation and tool calls can improve task success rate and execution efficiency in a simulated environment. At the same time, real smartphone apps do not necessarily offer the same tools. Introducing such a mechanism into real services would require building integration features individually so that both screen operation and tool calls can handle the same app state.
Note the different license terms for the model and the training environment
The released "OpenMobile-2-27B" is a model based on Qwen3.6-27B. The Hugging Face model card lists Apache License 2.0 as the license and also describes how to start an inference server using vLLM. However, what has been released is an AI model intended for smartphone operation, not a consumer app that runs the entire 27-billion-parameter model on an ordinary smartphone.
License terms also differ depending on the asset used, between the model and the training environment. The code, documentation, and build scripts of MobileGym++ fall under Apache License 2.0, while LICENSE-DATA treats data snapshots such as apps/*/data/, apps/*/assets/, and public/*.json separately. Among these, compilations and generated materials created by the maintainers fall under CC BY-NC 4.0, which includes non-commercial conditions. Furthermore, for data provided by third parties, the original rights and platform terms of use remain, so the fact that MobileGym++'s code is released under Apache License 2.0 does not mean all the data it contains can be used under the same terms. This license document alone also cannot determine the terms of use for OpenMobile-Data-v2 as a whole.
With this release of OpenMobile-2, third parties can more easily verify not only the performance of smartphone-operating AI but also which apps training data should be collected from and how screen operation and tool calls should be used together. In particular, an environment that can save and restore app state and a mechanism that can judge operation results programmatically are important foundations for comparing training methods. However, before adopting it in actual development, it will be necessary to unify the initial states and success conditions of the target apps, confirm the license terms of the models and data used, and verify performance on real devices. Going forward, the key to judging OpenMobile-2's practicality will be not high scores on research benchmarks, but how reliably it can complete user instructions in real apps.