UGIF: UI Grounded Instruction Following

14 November 2022

Papers citing "UGIF: UI Grounded Instruction Following"

19 / 19 papers shown

Title
MobileSteward: Integrating Multiple App-Oriented Agents with Self-Evolution to Automate Cross-App Instructions Yuxuan Liu Hongda Sun Wei Liu Jian Luan Bo Du Rui Yan 99 4 0 24 Feb 2025
AMEX: Android Multi-annotation Expo Dataset for Mobile GUI Agents Yuxiang Chai Siyuan Huang Yazhe Niu Han Xiao Liang Liu Dingyu Zhang Shuai Ren Hongsheng Li LLMAG 67 32 0 03 Jul 2024
UL2: Unifying Language Learning Paradigms Yi Tay Mostafa Dehghani Vinh Q. Tran Xavier Garcia Jason W. Wei ... Tal Schuster H. Zheng Denny Zhou N. Houlsby Donald Metzler AI4CE 100 305 0 10 May 2022
PaLM: Scaling Language Modeling with Pathways Aakanksha Chowdhery Sharan Narang Jacob Devlin Maarten Bosma Gaurav Mishra ... Kathy Meier-Hellstern Douglas Eck J. Dean Slav Petrov Noah Fiedel PILM LRM 353 6,132 0 05 Apr 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 616 9,009 0 28 Jan 2022
Learning UI Navigation through Demonstrations composed of Macro Actions Wei Li LLMAG 31 9 0 16 Oct 2021
HelpViz: Automatic Generation of Contextual Visual MobileTutorials from Text-Based Instructions Mingyuan Zhong Gang Li Peggy Chi Yang Li 25 24 0 07 Aug 2021
UIBert: Learning Generic Multimodal Representations for UI Understanding Chongyang Bai Xiaoxue Zang Ying Xu Srinivas Sunkara Abhinav Rastogi Jindong Chen Blaise Agüera y Arcas 54 92 0 29 Jul 2021
AndroidEnv: A Reinforcement Learning Platform for Android Daniel Toyama P. Hamel Anita Gergely Gheorghe Comanici Amelia Glaese Zafarali Ahmed Tyler Jackson Shibl Mourad Doina Precup VLM SSeg 34 71 0 27 May 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 480 3,952 0 18 Apr 2021
Grounding Open-Domain Instructions to Automate Web Support Tasks N. Xu Sam Masling Michael Du Giovanni Campagna Larry Heck James A. Landay M. Lam LLMAG AI4TS 30 41 0 30 Mar 2021
Spatial Reasoning from Natural Language Instructions for Robot Manipulation S. Gubbi Anirban Biswas Raviteja Upadrashta V. Srinivasan Partha P. Talukdar B. Amrutur LM&Ro LRM 68 29 0 26 Dec 2020
ActionBert: Leveraging User Actions for Semantic Understanding of User Interfaces Zecheng He Srinivas Sunkara Xiaoxue Zang Ying Xu Lijuan Liu Nevan Wichers Gabriel Schubiner Ruby B. Lee Jindong Chen Blaise Agüera y Arcas 59 79 0 22 Dec 2020
Widget Captioning: Generating Natural Language Description for Mobile User Interface Elements Yongqian Li Gang Li Luheng He Jingjie Zheng Hong Li Zhiwei Guan 32 107 0 08 Oct 2020
Language-agnostic BERT Sentence Embedding Fangxiaoyu Feng Yinfei Yang Daniel Cer N. Arivazhagan Wei Wang 118 896 0 03 Jul 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 544 41,106 0 28 May 2020
Language Conditioned Imitation Learning over Unstructured Data Corey Lynch P. Sermanet LM&Ro 60 245 0 15 May 2020
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 335 19,824 0 23 Oct 2019
Reinforcement Learning on Web Interfaces Using Workflow-Guided Exploration Emmy Liu Kelvin Guu Panupong Pasupat Tianlin Shi Percy Liang OnRL 43 211 0 24 Feb 2018