Title
DocVXQA: Context-Aware Visual Explanations for Document Question Answering Mohamed Ali Souibgui Changkyu Choi Andrey Barsky Kangsoo Jung Ernest Valveny Dimosthenis Karatzas 25 0 0 12 May 2025
CM1 - A Dataset for Evaluating Few-Shot Information Extraction with Large Vision Language Models Fabian Wolf Oliver Tüselmann Arthur Matei Lukas Hennies Christoph Rass Gernot A. Fink 53 0 0 07 May 2025
QID: Efficient Query-Informed ViTs in Data-Scarce Regimes for OCR-free Visual Document Understanding Binh M. Le Shaoyuan Xu Jinmiao Fu Zhishen Huang Moyan Li Yanhui Guo Hongdong Li Sameera Ramasinghe Bryan Wang 33 0 0 03 Apr 2025
UniHDSA: A Unified Relation Prediction Approach for Hierarchical Document Structure Analysis Jiawei Wang Kai Hu Qiang Huo 55 0 0 20 Mar 2025
UI-Vision: A Desktop-centric GUI Benchmark for Visual Perception and Interaction Shravan Nayak Xiangru Jian Kevin Qinghong Lin Juan A. Rodriguez Montek Kalsi ... David Vazquez Christopher Pal Perouz Taslakian Spandana Gella Sai Rajeswar 189 0 0 19 Mar 2025
KIEval: Evaluation Metric for Document Key Information Extraction Minsoo Khang Sang Chul Jung Sungrae Park Teakgyu Hong 47 0 0 07 Mar 2025
Chart-HQA: A Benchmark for Hypothetical Question Answering in Charts Xiangnan Chen Yuancheng Fang Qian Xiao Juncheng Billy Li J. Lin Siliang Tang Yi Yang Yueting Zhuang 70 0 0 06 Mar 2025
SpiritSight Agent: Advanced GUI Agent with One Look Zhiyuan Huang Ziming Cheng Junting Pan Zhaohui Hou Mingjie Zhan LLMAG 101 2 0 05 Mar 2025
Vision-centric Token Compression in Large Language Model Ling Xing Alex Jinpeng Wang Rui Yan J. Tang Jinhui Tang VLM 60 0 0 02 Feb 2025
Exposing Limitations of Language Model Agents in Sequential-Task Compositions on the Web Hiroki Furuta Yutaka Matsuo Aleksandra Faust Izzeddin Gur CLL 89 14 0 03 Jan 2025
MolParser: End-to-end Visual Recognition of Molecule Structures in the Wild Xi Fang Jiankun Wang X. Cai Shangqian Chen Shuwen Yang Lin Yao Linfeng Zhang Guolin Ke Linfeng Zhang Guolin Ke 50 1 0 17 Nov 2024
M3DocRAG: Multi-modal Retrieval is What You Need for Multi-page Multi-document Understanding Jaemin Cho Debanjan Mahata Ozan Irsoy Yujie He Mohit Bansal VLM 29 9 0 07 Nov 2024
VisRAG: Vision-based Retrieval-augmented Generation on Multi-modality Documents S. Yu C. Tang Bokai Xu Junbo Cui Junhao Ran ... Zhenghao Liu Shuo Wang Xu Han Zhiyuan Liu Maosong Sun VLM 39 23 0 14 Oct 2024
Aria: An Open Multimodal Native Mixture-of-Experts Model Dongxu Li Yudong Liu Haoning Wu Yue Wang Zhiqi Shen ... Lihuan Zhang Hanshu Yan Guoyin Wang Bei Chen Junnan Li MoE 51 48 0 08 Oct 2024
HATFormer: Historic Handwritten Arabic Text Recognition with Transformers Adrian Chan Anupam Mijar Mehreen Saeed Chau-Wai Wong Akram Khater 41 0 0 03 Oct 2024
Eagle: Exploring The Design Space for Multimodal LLMs with Mixture of Encoders Min Shi Fuxiao Liu Shihao Wang Shijia Liao Subhashree Radhakrishnan ... Andrew Tao Andrew Tao Zhiding Yu Guilin Liu Guilin Liu MLLM 27 53 0 28 Aug 2024
VisFocus: Prompt-Guided Vision Encoders for OCR-Free Dense Document Understanding Ofir Abramovich Niv Nayman Sharon Fogel I. Lavi Ron Litman Shahar Tsiper Royee Tichauer Srikar Appalaraju Shai Mazor R. Manmatha VLM 33 3 0 17 Jul 2024
A Bounding Box is Worth One Token: Interleaving Layout and Text in a Large Language Model for Document Understanding Jinghui Lu Haiyang Yu Yanjie Wang Yongjie Ye Jingqun Tang ... Qi Liu Hao Feng Han Wang Hao Liu Can Huang 50 18 0 02 Jul 2024
On Efficient Language and Vision Assistants for Visually-Situated Natural Language Understanding: What Matters in Reading and Reasoning Geewook Kim Minjoon Seo VLM 36 2 0 17 Jun 2024
VideoGUI: A Benchmark for GUI Automation from Instructional Videos Kevin Qinghong Lin Linjie Li Difei Gao Qinchen Wu Mingyi Yan Zhengyuan Yang Lijuan Wang Mike Zheng Shou 46 10 0 14 Jun 2024
Wavelet-Based Image Tokenizer for Vision Transformers Zhenhai Zhu Radu Soricut ViT 47 3 0 28 May 2024
TextSquare: Scaling up Text-Centric Visual Instruction Tuning Jingqun Tang Chunhui Lin Zhen Zhao Shubo Wei Binghong Wu ... Yuliang Liu Hao Liu Yuan Xie Xiang Bai Can Huang LRM VLM MLLM 66 29 0 19 Apr 2024
MMInA: Benchmarking Multihop Multimodal Internet Agents Ziniu Zhang Shulin Tian Liangyu Chen Ziwei Liu LLMAG LM&Ro 27 13 0 15 Apr 2024
Ferret-UI: Grounded Mobile UI Understanding with Multimodal LLMs Keen You Haotian Zhang E. Schoop Floris Weers Amanda Swearngin Jeffrey Nichols Yinfei Yang Zhe Gan MLLM 47 82 0 08 Apr 2024
Tur[k]ingBench: A Challenge Benchmark for Web Agents Kevin Xu Yeganeh Kordi Kate Sanders Yizhong Wang Adam Byerly Kate Sanders Adam Byerly Jingyu Zhang Benjamin Van Durme Daniel Khashabi LLMAG 72 6 0 18 Mar 2024
Enhancing Vision-Language Pre-training with Rich Supervisions Yuan Gao Kunyu Shi Pengkai Zhu Edouard Belval Oren Nuriel Srikar Appalaraju Shabnam Ghadar Vijay Mahadevan Zhuowen Tu Stefano Soatto VLM CLIP 67 12 0 05 Mar 2024
ChartReformer: Natural Language-Driven Chart Image Editing Pengyu Yan Mahesh Bhosale Jay Lal Bikhyat Adhikari David Doermann 26 6 0 01 Mar 2024
ChartX & ChartVLM: A Versatile Benchmark and Foundation Model for Complicated Chart Reasoning Renqiu Xia Bo-Wen Zhang Hancheng Ye Xiangchao Yan Qi Liu ... Min Dou Botian Shi Junchi Yan Junchi Yan Yu Qiao LRM 57 52 0 19 Feb 2024
WebLINX: Real-World Website Navigation with Multi-Turn Dialogue Xing Han Lù Zdeněk Kasner Siva Reddy 30 59 0 08 Feb 2024
TreeForm: End-to-end Annotation and Evaluation for Form Document Parsing Ran Zmigrod Zhiqiang Ma Armineh Nourbakhsh Sameena Shah 24 4 0 07 Feb 2024
Dual-View Visual Contextualization for Web Navigation Jihyung Kil Chan Hee Song Boyuan Zheng Xiang Deng Yu-Chuan Su Wei-Lun Chao EgoV 22 12 0 06 Feb 2024
ChartAssisstant: A Universal Chart Multimodal Language Model via Chart-to-Table Pre-training and Multitask Instruction Tuning Fanqing Meng Wenqi Shao Quanfeng Lu Peng Gao Kaipeng Zhang Yu Qiao Ping Luo 29 45 0 04 Jan 2024
An Empirical Study of Scaling Law for OCR Miao Rang Zhenni Bi Chuanjian Liu Yunhe Wang Kai Han 33 6 0 29 Dec 2023
Do LLMs Work on Charts? Designing Few-Shot Prompts for Chart Question Answering and Summarization Do Xuan Long Mohammad Hassanpour Ahmed Masry P. Kavehzadeh Enamul Hoque Shafiq R. Joty LRM 30 9 0 17 Dec 2023
Text as Images: Can Multimodal Large Language Models Follow Printed Instructions in Pixels? Xiujun Li Yujie Lu Zhe Gan Jianfeng Gao William Yang Wang Yejin Choi VLM MLLM 35 1 0 29 Nov 2023
MMC: Advancing Multimodal Chart Understanding with Large-scale Instruction Tuning Fuxiao Liu Xiaoyang Wang Wenlin Yao Jianshu Chen Kaiqiang Song Sangwoo Cho Yaser Yacoob Dong Yu 24 99 0 15 Nov 2023
DCQA: Document-Level Chart Question Answering towards Complex Reasoning and Common-Sense Understanding Anran Wu Luwei Xiao Xingjiao Wu Shuwen Yang Junjie Xu Zisong Zhuang Nian Xie Cheng Jin Liang He 26 0 0 29 Oct 2023
A Multi-Modal Multilingual Benchmark for Document Image Classification Yoshinari Fujinuma Siddharth Varia Nishant Sankaran Srikar Appalaraju Bonan Min Yogarshi Vyas VLM 20 4 0 25 Oct 2023
SCOB: Universal Text Understanding via Character-wise Supervised Contrastive Learning with Online Text Rendering for Bridging Domain Gap Daehee Kim Yoon Kim Donghyun Kim Yumin Lim Geewook Kim Taeho Kil 28 3 0 21 Sep 2023
Android in the Wild: A Large-Scale Dataset for Android Device Control Christopher Rawles Alice Li Daniel Rodriguez Oriana Riva Timothy Lillicrap LM&Ro 28 137 0 19 Jul 2023
GenPlot: Increasing the Scale and Diversity of Chart Derendering Data Brendan Artley 23 1 0 20 Jun 2023
PaLI-X: On Scaling up a Multilingual Vision and Language Model Xi Chen Josip Djolonga Piotr Padlewski Basil Mustafa Soravit Changpinyo ... Mojtaba Seyedhosseini A. Angelova Xiaohua Zhai N. Houlsby Radu Soricut VLM 51 187 0 29 May 2023
Multimodal Web Navigation with Instruction-Finetuned Foundation Models Hiroki Furuta Kuang-Huei Lee Ofir Nachum Yutaka Matsuo Aleksandra Faust S. Gu Izzeddin Gur LM&Ro 36 91 0 19 May 2023
ProgSG: Cross-Modality Representation Learning for Programs in Electronic Design Automation Yunsheng Bai Atefeh Sohrabizadeh Zongyue Qin Ziniu Hu Yizhou Sun Jason Cong 18 1 0 18 May 2023
OneCAD: One Classifier for All image Datasets using multimodal learning S. Wadekar Eugenio Culurciello 37 0 0 11 May 2023
ChartReader: A Unified Framework for Chart Derendering and Comprehension without Heuristic Rules Zhi-Qi Cheng Qianwen Dai Siyao Li Jingdong Sun Teruko Mitamura Alexander G. Hauptmann 29 21 0 05 Apr 2023
Can Current Task-oriented Dialogue Models Automate Real-world Scenarios in the Wild? Sang-Woo Lee Sungdong Kim Donghyeon Ko Dong-hyun Ham Youngki Hong ... Wangkyo Jung Kyunghyun Cho Donghyun Kwak H. Noh W. Park 43 1 0 20 Dec 2022
WebFormer: The Web-page Transformer for Structure Information Extraction Qifan Wang Yi Fang Anirudh Ravula Fuli Feng Xiaojun Quan Dongfang Liu ViT 141 65 0 01 Feb 2022
Pix2seq: A Language Modeling Framework for Object Detection Ting-Li Chen Saurabh Saxena Lala Li David J. Fleet Geoffrey E. Hinton MLLM ViT VLM 241 344 0 22 Sep 2021
Screen Parsing: Towards Reverse Engineering of UI Models from Screenshots Jason Wu Xiaoyi Zhang Jeffrey Nichols Jeffrey P. Bigham 3DV 163 71 0 17 Sep 2021