ScreenQA: Large-Scale Question-Answer Pairs over Mobile App Screenshots

16 September 2022

Victor Carbune

Jason Lin

Maria Wang

Yun Zhu

Jindong Chen

RALM

ArXiv PDF HTML

Papers citing "ScreenQA: Large-Scale Question-Answer Pairs over Mobile App Screenshots"

24 / 24 papers shown

Title
Data Metabolism: An Efficient Data Design Schema For Vision Language Model Jingyuan Zhang Hongzhi Zhang Zhou Haonan Chenxi Sun Xingguang Ji Jiakang Wang Fanheng Kong Y. Liu Qi Wang Fuzheng Zhang VLM 58 1 0 10 Apr 2025
Capybara-OMNI: An Efficient Paradigm for Building Omni-Modal Language Models Xingguang Ji Jiakang Wang Hongzhi Zhang Jingyuan Zhang Haonan Zhou Chenxi Sun Y. Liu Qi Wang Fuzheng Zhang MLLM VLM 58 0 0 10 Apr 2025
MP-GUI: Modality Perception with MLLMs for GUI Understanding Ziwei Wang Weizhi Chen Leyang Yang Sheng Zhou Shengchu Zhao Hanbei Zhan Jiongchao Jin Liangcheng Li Zirui Shao Jiajun Bu 65 1 0 18 Mar 2025
PP-DocBee: Improving Multimodal Document Understanding Through a Bag of Tricks Feng Ni Kui Huang Yao Lu Wenyu Lv Guanzhong Wang Zeyu Chen Y. Liu VLM 48 0 0 06 Mar 2025
SpiritSight Agent: Advanced GUI Agent with One Look Zhiyuan Huang Ziming Cheng Junting Pan Zhaohui Hou Mingjie Zhan LLMAG 101 2 0 05 Mar 2025
Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generation Chengyue Wu Xiaokang Chen Z. F. Wu Yiyang Ma Xingchao Liu ... Wen Liu Zhenda Xie Xingkai Yu Chong Ruan Ping Luo AI4TS 57 74 0 17 Oct 2024
Harnessing Webpage UIs for Text-Rich Visual Understanding Junpeng Liu Tianyue Ou Yifan Song Yuxiao Qu Wai Lam Chenyan Xiong Wenhu Chen Graham Neubig Xiang Yue 74 5 0 17 Oct 2024
Leopard: A Vision Language Model For Text-Rich Multi-Image Tasks Mengzhao Jia Wenhao Yu Kaixin Ma Tianqing Fang Zhihan Zhang Siru Ouyang Hongming Zhang Meng-Long Jiang Dong Yu VLM 29 5 0 02 Oct 2024
MM1.5: Methods, Analysis & Insights from Multimodal LLM Fine-tuning Haotian Zhang Mingfei Gao Zhe Gan Philipp Dufter Nina Wenzel ... Haoxuan You Zirui Wang Afshin Dehghan Peter Grasch Yinfei Yang VLM MLLM 40 32 1 30 Sep 2024
MobileVLM: A Vision-Language Model for Better Intra- and Inter-UI Understanding Qinzhuo Wu Weikai Xu Wei Liu Tao Tan Jianfeng Liu Ang Li Jian Luan Bin Wang Shuo Shang VLM 32 10 0 23 Sep 2024
MobileViews: A Large-Scale Mobile GUI Dataset Longxi Gao Li Zhang Shihe Wang Shangguang Wang Yuanchun Li Mengwei Xu 28 5 0 22 Sep 2024
POINTS: Improving Your Vision-language Model with Affordable Strategies Yuan Liu Zhongyin Zhao Ziyuan Zhuang Le Tian Xiao Zhou Jie Zhou VLM 35 5 0 07 Sep 2024
WebQuest: A Benchmark for Multimodal QA on Web Page Sequences Maria Wang Srinivas Sunkara Gilles Baechler Jason Lin Yun Zhu Fedir Zubach Lei Shu Jindong Chen LRM LLMAG 24 1 0 06 Sep 2024
Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMs Shengbang Tong Ellis L Brown Penghao Wu Sanghyun Woo Manoj Middepogu ... Xichen Pan Austin Wang Rob Fergus Yann LeCun Saining Xie 3DV MLLM 48 279 0 24 Jun 2024
On Efficient Language and Vision Assistants for Visually-Situated Natural Language Understanding: What Matters in Reading and Reasoning Geewook Kim Minjoon Seo VLM 36 2 0 17 Jun 2024
MLLMGuard: A Multi-dimensional Safety Evaluation Suite for Multimodal Large Language Models Tianle Gu Zeyang Zhou Kexin Huang Dandan Liang Yixu Wang ... Keqing Wang Yujiu Yang Yan Teng Yu Qiao Yingchun Wang ELM 47 12 0 11 Jun 2024
DeepSeek-VL: Towards Real-World Vision-Language Understanding Haoyu Lu Wen Liu Bo Zhang Bing-Li Wang Kai Dong ... Yaofeng Sun Chengqi Deng Hanwei Xu Zhenda Xie Chong Ruan VLM 36 292 0 08 Mar 2024
ScreenAI: A Vision-Language Model for UI and Infographics Understanding Gilles Baechler Srinivas Sunkara Maria Wang Fedir Zubach Hassan Mansoor Vincent Etter Victor Carbune Jason Lin Jindong Chen Abhanshu Sharma 115 47 0 07 Feb 2024
WebVLN: Vision-and-Language Navigation on Websites Qi Chen D. Pitawela Chongyang Zhao Gengze Zhou Hsiang-Ting Chen Qi Wu 34 8 0 25 Dec 2023
CogAgent: A Visual Language Model for GUI Agents Wenyi Hong Weihan Wang Qingsong Lv Jiazheng Xu Wenmeng Yu ... Juanzi Li Bin Xu Yuxiao Dong Ming Ding Jie Tang MLLM 142 319 0 14 Dec 2023
Referring to Screen Texts with Voice Assistants Shruti Bhargava Anand Dhoot I. Jonsson Hoang Long Nguyen Alkesh Patel Hong-ye Yu Vincent Renkens 21 2 0 10 Jun 2023
Enabling Conversational Interaction with Mobile UI using Large Language Models Bryan Wang Gang Li Yang Li 175 132 0 18 Sep 2022
Screen2Words: Automatic Mobile UI Summarization with Multimodal Learning Bryan Wang Gang Li Xin Zhou Zhourong Chen Tovi Grossman Yang Li 164 152 0 07 Aug 2021
FUNSD: A Dataset for Form Understanding in Noisy Scanned Documents Guillaume Jaume H. K. Ekenel Jean-Philippe Thiran 134 355 0 27 May 2019