Never-ending Learning of User Interfaces

Never-ending Learning of User Interfaces

17 August 2023

Rebecca Krosnick

Amanda Swearngin

Jeffrey P. Bigham

Jeffrey Nichols

Papers citing "Never-ending Learning of User Interfaces"

14 / 14 papers shown

Title
DreamStruct: Understanding Slides and User Interfaces via Synthetic Data Generation Yi-Hao Peng Faria Huq Yue Jiang Jason Wu Amanda Li Jeffrey P. Bigham Amy Pavel DiffM 32 4 0 30 Sep 2024
Misty: UI Prototyping Through Interactive Conceptual Blending Yuwen Lu Alan Leung Amanda Swearngin Jeffrey Nichols Titus Barik 26 3 0 20 Sep 2024
GUing: A Mobile GUI Search Engine using a Vision-Language Model Jialiang Wei A. Courbis Thomas Lambolais Binbin Xu P. Bernard Gérard Dray Walid Maalej DiffM CLIP 32 6 0 30 Apr 2024
UIClip: A Data-driven Model for Assessing User Interface Design Jason Wu Yi-Hao Peng Amanda Li Amanda Swearngin Jeffrey P. Bigham Jeffrey Nichols HAI 26 6 0 18 Apr 2024
Unblind Text Inputs: Predicting Hint-text of Text Input in Mobile Apps via LLM Zhe Liu Chunyang Chen Junjie Wang Mengzhuo Chen Boyu Wu Yuekai Huang Jun Hu Qing Wang 29 10 0 03 Apr 2024
Tappy: Predicting Tap Accuracy of User-Interface Elements by Reverse-Engineering Webpage Structures Hiroki Usuba Junichi Sato Naomi Sasaya Shota Yamanaka Fumiya Yamashita 21 2 0 05 Mar 2024
AppAgent: Multimodal Agents as Smartphone Users C. Zhang Zhao Yang Jiaxuan Liu Yucheng Han Xin Chen Zebiao Huang Bin-Bin Fu Gang Yu LM&Ro LLMAG 24 77 0 21 Dec 2023
Towards Automated Accessibility Report Generation for Mobile Apps Amanda Swearngin Jason Wu Xiaoyi Zhang Esteban Gomez Jen Coughenour ... Bhavya Garg Greg Hughes Adriana Hilliard Jeffrey P. Bigham Jeffrey Nichols 23 10 0 29 Sep 2023
Screen Parsing: Towards Reverse Engineering of UI Models from Screenshots Jason Wu Xiaoyi Zhang Jeffrey Nichols Jeffrey P. Bigham 3DV 163 71 0 17 Sep 2021
Screen2Words: Automatic Mobile UI Summarization with Multimodal Learning Bryan Wang Gang Li Xin Zhou Zhourong Chen Tovi Grossman Yang Li 167 152 0 07 Aug 2021
Understanding Mobile GUI: from Pixel-Words to Screen-Sentences Jingwen Fu Xiaoyi Zhang Yuwang Wang Wenjun Zeng Sam Yang Grayson Hilliard 23 14 0 25 May 2021
Screen Recognition: Creating Accessibility Metadata for Mobile Applications from Pixels Xiaoyi Zhang Lilian de Greef Amanda Swearngin Samuel White Kyle I. Murray ... Jeffrey Nichols Jason Wu Chris Fleizach Aaron Everitt Jeffrey P. Bigham 205 167 0 13 Jan 2021
Augmented SBERT: Data Augmentation Method for Improving Bi-Encoders for Pairwise Sentence Scoring Tasks Nandan Thakur Nils Reimers Johannes Daxenberger Iryna Gurevych 205 241 0 16 Oct 2020
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications Andrew G. Howard Menglong Zhu Bo Chen Dmitry Kalenichenko Weijun Wang Tobias Weyand M. Andreetto Hartwig Adam 3DH 950 20,567 0 17 Apr 2017