v1v2 (latest)

Android in the Wild: A Large-Scale Dataset for Android Device Control

19 July 2023

ArXiv (abs)PDF HTML Github (35636★)

Papers citing "Android in the Wild: A Large-Scale Dataset for Android Device Control"

41 / 41 papers shown

Title
Think or Not? Selective Reasoning via Reinforcement Learning for Vision-Language Models Jiaqi Wang Kevin Qinghong Lin James Cheng Mike Zheng Shou OffRL ReLM LRM 94 0 0 22 May 2025
From Assistants to Adversaries: Exploring the Security Risks of Mobile LLM Agents Liangxuan Wu Chao Wang Tianming Liu Yanjie Zhao Haoyu Wang AAML 62 0 0 19 May 2025
Imperative MPC: An End-to-End Self-Supervised Learning with Differentiable MPC for UAV Attitude Control Haonan He Yuheng Qiu Junyi Geng 132 0 0 17 Apr 2025
TongUI: Building Generalized GUI Agents by Learning from Multimodal Web Tutorials Bofei Zhang Zirui Shang Zhi Gao Wang Zhang Rui Xie Xiaojian Ma Tao Yuan Xinxiao Wu Song-Chun Zhu Qing Li LLMAG 109 3 0 17 Apr 2025
ViMo: A Generative Visual GUI World Model for App Agents Dezhao Luo Bohan Tang Kang Li Georgios Papoudakis Jifei Song S. Gong Haifeng Zhang Jun Wang Kun Shao LM&Ro VGen 137 1 0 15 Apr 2025
A Survey of WebAgents: Towards Next-Generation AI Agents for Web Automation with Large Foundation Models Liangbo Ning Ziran Liang Zhuohang Jiang Haohao Qu Yujuan Ding ... Xiao Wei Shanru Lin Hui Liu Philip S. Yu Qing Li LLMAG LM&Ro 140 9 0 30 Mar 2025
UI-Vision: A Desktop-centric GUI Benchmark for Visual Perception and Interaction Shravan Nayak Xiangru Jian Kevin Qinghong Lin Juan A. Rodriguez Montek Kalsi ... David Vazquez Christopher Pal Perouz Taslakian Spandana Gella Sai Rajeswar 473 2 0 19 Mar 2025
Plan-and-Act: Improving Planning of Agents for Long-Horizon Tasks Lutfi Eren Erdogan Nicholas Lee Sehoon Kim Suhong Moon Hiroki Furuta Gopala Anumanchipalli Kemal Kurniawan Amir Gholami LLMAG LM&Ro AIFin 156 3 0 12 Mar 2025
SpiritSight Agent: Advanced GUI Agent with One Look Zhiyuan Huang Ziming Cheng Junting Pan Zhaohui Hou Mingjie Zhan LLMAG 147 4 0 05 Mar 2025
DistRL: An Asynchronous Distributed Reinforcement Learning Framework for On-Device Control Agents Taiyi Wang Zhihao Wu Jianheng Liu Jianye Hao Jun Wang Kun Shao OffRL 92 25 0 24 Feb 2025
MobileSteward: Integrating Multiple App-Oriented Agents with Self-Evolution to Automate Cross-App Instructions Yuxuan Liu Hongda Sun Wei Liu Jian Luan Bo Du Rui Yan 122 4 0 24 Feb 2025
AgentStudio: A Toolkit for Building General Virtual Agents Longtao Zheng Zhiyuan Huang Zhenghai Xue Xinrun Wang Bo An Shuicheng Yan 191 19 0 17 Feb 2025
InSTA: Towards Internet-Scale Training For Agents Brandon Trabucco Gunnar Sigurdsson Robinson Piramuthu Ruslan Salakhutdinov ALM 166 4 0 10 Feb 2025
AutoGUI: Scaling GUI Grounding with Automatic Functionality Annotations from LLMs Hongxin Li Jingfan Chen Jingran Su Yuntao Chen Qing Li Zhaoxiang Zhang 433 1 0 04 Feb 2025
WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learning Zehan Qi Xiao-Chang Liu Iat Long Iong Hanyu Lai Xingwu Sun ... Shuntian Yao Tianjie Zhang Wei Xu J. Tang Yuxiao Dong 154 36 0 28 Jan 2025
The BrowserGym Ecosystem for Web Agent Research Thibault Le Sellier De Chezelles Maxime Gasse Alexandre Lacoste Alexandre Drouin Massimo Caccia ... Siva Reddy Quentin Cappart Graham Neubig Ruslan Salakhutdinov Nicolas Chapados LLMAG 146 15 0 06 Dec 2024
GUI Agents with Foundation Models: A Comprehensive Survey Shuai Wang Wen Liu Jingxuan Chen Weinan Gan Xingshan Zeng ... Bin Wang Chuhan Wu Yasheng Wang Ruiming Tang Jianye Hao LLMAG 105 25 0 07 Nov 2024
Ferret-UI 2: Mastering Universal User Interface Understanding Across Platforms Zhangheng Li Keen You Hao Zhang Di Feng Harsh Agrawal Xiujun Li Mohana Prasad Sathya Moorthy Jeff Nichols Yue Yang Zhe Gan MLLM 98 20 0 24 Oct 2024
Lightweight Neural App Control Filippos Christianos Georgios Papoudakis Thomas Coste Jianye Hao Jun Wang Kun Shao LM&Ro 88 5 0 23 Oct 2024
Beyond Browsing: API-Based Web Agents Yueqi Song Frank F. Xu Shuyan Zhou Graham Neubig 108 22 0 21 Oct 2024
SPA-Bench: A Comprehensive Benchmark for SmartPhone Agent Evaluation Jingxuan Chen Derek Yuen Bin Xie Yue Yang Gongwei Chen ... Liqiang Nie Yasheng Wang Jianye Hao Jun Wang Kun Shao LLMAG 117 14 0 19 Oct 2024
TinyClick: Single-Turn Agent for Empowering GUI Automation Pawel Pawlowski Krystian Zawistowski Wojciech Lapacz Marcin Skorupa Adam Wiacek Sebastien Postansque Jakub Hoscilowicz LRM LLMAG MLLM 95 7 0 09 Oct 2024
Audio-Agent: Leveraging LLMs For Audio Generation, Editing and Composition Zixuan Wang Chi-Keung Tang Chi-Keung Tang DiffM VGen LLMAG 91 4 0 04 Oct 2024
AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agents Christopher Rawles Sarah Clinckemaillie Yifan Chang Jonathan Waltz Gabrielle Lau ... Daniel Toyama Robert Berry Divya Tyamagundlu Timothy Lillicrap Oriana Riva LLMAG 119 68 0 23 May 2024
Draw-and-Understand: Leveraging Visual Prompts to Enable MLLMs to Comprehend What You Want Weifeng Lin Xinyu Wei Ruichuan An Peng Gao Bocheng Zou Yulin Luo Siyuan Huang Shanghang Zhang Hongsheng Li VLM 134 39 0 29 Mar 2024
Gorilla: Large Language Model Connected with Massive APIs Shishir G. Patil Tianjun Zhang Xin Wang Joseph E. Gonzalez ELM CLL ALM SyDa 84 556 0 24 May 2023
UGIF: UI Grounded Instruction Following S. Venkatesh Partha P. Talukdar S. Narayanan 99 12 0 14 Nov 2022
Interactive Language: Talking to Robots in Real Time Corey Lynch Ayzaan Wahid Jonathan Tompson Tianli Ding James Betker Robert Baruch Travis Armstrong Peter R. Florence LM&Ro 91 228 0 12 Oct 2022
Understanding HTML with Large Language Models Izzeddin Gur Ofir Nachum Yingjie Miao Mustafa Safdari Austin Huang Aakanksha Chowdhery Sharan Narang Noah Fiedel Aleksandra Faust AI4CE 188 71 0 08 Oct 2022
Towards Better Semantic Understanding of Mobile Interfaces Srinivas Sunkara Maria Wang Lijuan Liu Gilles Baechler Yu-Chung Hsiao Jindong Chen Chen Abhanshu Sharma James Stout 64 24 0 06 Oct 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 817 9,387 0 28 Jan 2022
UIBert: Learning Generic Multimodal Representations for UI Understanding Chongyang Bai Xiaoxue Zang Ying Xu Srinivas Sunkara Abhinav Rastogi Jindong Chen Blaise Agüera y Arcas 63 94 0 29 Jul 2021
AppBuddy: Learning to Accomplish Tasks in Mobile Apps via Reinforcement Learning Maayan Shvo Zhiming Hu Rodrigo Toro Icarte Iqbal Mohomed A. Jepson Sheila A. McIlraith 50 14 0 31 May 2021
AndroidEnv: A Reinforcement Learning Platform for Android Daniel Toyama P. Hamel Anita Gergely Gheorghe Comanici Amelia Glaese Zafarali Ahmed Tyler Jackson Shibl Mourad Doina Precup VLM SSeg 64 75 0 27 May 2021
Screen Recognition: Creating Accessibility Metadata for Mobile Applications from Pixels Xiaoyi Zhang Lilian de Greef Amanda Swearngin Samuel White Kyle I. Murray ... Jeffrey Nichols Jason Wu Chris Fleizach Aaron Everitt Jeffrey P. Bigham 343 171 0 13 Jan 2021
ActionBert: Leveraging User Actions for Semantic Understanding of User Interfaces Zecheng He Srinivas Sunkara Xiaoxue Zang Ying Xu Lijuan Liu Nevan Wichers Gabriel Schubiner Ruby B. Lee Jindong Chen Blaise Agüera y Arcas 73 79 0 22 Dec 2020
Object Detection for Graphical User Interface: Old Fashioned or Deep Learning or a Combination? Jieshan Chen Mulong Xie Zhenchang Xing Chunyang Chen Xiwei Xu Liming Zhu Guoqiang Li OOD 49 148 0 12 Aug 2020
Acme: A Research Framework for Distributed Reinforcement Learning Matthew W. Hoffman Bobak Shahriari John Aslanides Gabriel Barth-Maron Nikola Momchev ... Srivatsan Srinivasan A. Cowie Ziyun Wang Bilal Piot Nando de Freitas 110 226 0 01 Jun 2020
Language Conditioned Imitation Learning over Unstructured Data Corey Lynch P. Sermanet LM&Ro 82 251 0 15 May 2020
Unblind Your Apps: Predicting Natural-Language Labels for Mobile GUI Components by Deep Learning Jieshan Chen Chunyang Chen Zhenchang Xing Xiwei Xu Liming Zhu Guoqiang Li Jinshui Wang 49 139 0 01 Mar 2020
Weakly Supervised Action Learning with RNN based Fine-to-coarse Modeling Alexander Richard Hilde Kuehne Juergen Gall 50 197 0 23 Mar 2017