Browse Teach Curate

Dynamic In-the-Wild Evaluation Platform for Mobile GUI Agents

An interactive benchmarking environment that evaluates mobile GUI agents by running them live inside real third-party Android apps against multi-step, goal-oriented tasks, rather than scoring single frozen screenshots. It provides a dataset-agnostic action space so agents trained on any data can be tested, and uses an automated large-language-model judge to grade task completion with minimal human labelling.

Data Text

A3: Android Agent Arena for Mobile GUI Agents Yuxiang Chai

This paper presents Android Agent Arena (A3), an evaluation platform for mobile GUI agents: AI systems, typically driven by multimodal large language models, that autonomously operate smartphone apps…