v1v2v3 (latest)

Unifying Vision, Text, and Layout for Universal Document Processing

5 December 2022

Yang Liu

Papers citing "Unifying Vision, Text, and Layout for Universal Document Processing"

50 / 52 papers shown

Title
AutoGUI: Scaling GUI Grounding with Automatic Functionality Annotations from LLMs Hongxin Li Jingfan Chen Jingran Su Yuntao Chen Qing Li Zhaoxiang Zhang 451 1 0 04 Feb 2025
VisFocus: Prompt-Guided Vision Encoders for OCR-Free Dense Document Understanding Ofir Abramovich Niv Nayman Sharon Fogel I. Lavi Ron Litman Shahar Tsiper Royee Tichauer Srikar Appalaraju Shai Mazor R. Manmatha VLM 81 3 0 17 Jul 2024
A Bounding Box is Worth One Token: Interleaving Layout and Text in a Large Language Model for Document Understanding Jinghui Lu Haiyang Yu Yanjie Wang Yongjie Ye Jingqun Tang ... Qi Liu Hao Feng Han Wang Hao Liu Can Huang 138 23 0 02 Jul 2024
ColPali: Efficient Document Retrieval with Vision Language Models Manuel Faysse Hugues Sibille Tony Wu Bilel Omrani Gautier Viaud C´eline Hudelot Pierre Colombo VLM 273 29 0 27 Jun 2024
DistilDoc: Knowledge Distillation for Visually-Rich Document Applications Jordy Van Landeghem Subhajit Maity Ayan Banerjee Matthew Blaschko Marie-Francine Moens Josep Lladós Sanket Biswas 114 2 0 12 Jun 2024
Perceiver-VL: Efficient Vision-and-Language Modeling with Iterative Latent Attention Zineng Tang Jaemin Cho Jie Lei Joey Tianyi Zhou VLM 74 9 0 21 Nov 2022
Scaling Instruction-Finetuned Language Models Hyung Won Chung Le Hou Shayne Longpre Barret Zoph Yi Tay ... Jacob Devlin Adam Roberts Denny Zhou Quoc V. Le Jason W. Wei ReLM LRM 218 3,158 0 20 Oct 2022
Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Tasks Wenhui Wang Hangbo Bao Li Dong Johan Bjorck Zhiliang Peng ... Kriti Aggarwal O. Mohammed Saksham Singhal Subhojit Som Furu Wei MLLM VLM ViT 148 644 0 22 Aug 2022
Unified-IO: A Unified Model for Vision, Language, and Multi-Modal Tasks Jiasen Lu Christopher Clark Rowan Zellers Roozbeh Mottaghi Aniruddha Kembhavi ObjD VLM MLLM 141 413 0 17 Jun 2022
i-Code: An Integrative and Composable Multimodal Learning Framework Ziyi Yang Yuwei Fang Chenguang Zhu Reid Pryzant DongDong Chen ... Bin Xiao Yuanxun Lu Takuya Yoshioka Michael Zeng Xuedong Huang 76 49 0 03 May 2022
Unified Pretraining Framework for Document Understanding Jiuxiang Gu Jason Kuen Vlad I. Morariu Handong Zhao Nikolaos Barmpalios R. Jain A. Nenkova Tong Sun 78 97 0 22 Apr 2022
LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking Yupan Huang Tengchao Lv Lei Cui Yutong Lu Furu Wei 95 460 0 18 Apr 2022
FormNet: Structural Encoding beyond Sequential Modeling in Form Document Information Extraction Chen-Yu Lee Chun-Liang Li Timothy Dozat Vincent Perot Guolong Su Nan Hua Joshua Ainslie Renshen Wang Yasuhisa Fujii Tomas Pfister 73 79 0 16 Mar 2022
XYLayoutLM: Towards Layout-Aware Multimodal Networks For Visually-Rich Document Understanding Zhangxuan Gu Changhua Meng Ke Wang Jun Lan Weiqiang Wang Ming Gu Liqing Zhang 72 79 0 14 Mar 2022
LiLT: A Simple yet Effective Language-Independent Layout Transformer for Structured Document Understanding Jiapeng Wang Lianwen Jin Kai Ding VLM 69 143 0 28 Feb 2022
OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Framework Peng Wang An Yang Rui Men Junyang Lin Shuai Bai Zhikang Li Jianxin Ma Chang Zhou Jingren Zhou Hongxia Yang MLLM ObjD 157 880 0 07 Feb 2022
Masked Autoencoders Are Scalable Vision Learners Kaiming He Xinlei Chen Saining Xie Yanghao Li Piotr Dollár Ross B. Girshick ViT TPM 477 7,827 0 11 Nov 2021
Pix2seq: A Language Modeling Framework for Object Detection Ting-Li Chen Saurabh Saxena Lala Li David J. Fleet Geoffrey E. Hinton MLLM ViT VLM 271 350 0 22 Sep 2021
BROS: A Pre-trained Language Model Focusing on Text and Layout for Better Key Information Extraction from Documents Teakgyu Hong Donghyun Kim Mingi Ji Wonseok Hwang Daehyun Nam Sungrae Park VLM 76 154 0 10 Aug 2021
StrucTexT: Structured Text Understanding with Multi-Modal Transformers Yulin Li Yuxi Qian Yuchen Yu Xiameng Qin Chengquan Zhang Yan Liu Kun Yao Junyu Han Jingtuo Liu Errui Ding 79 117 0 06 Aug 2021
DocFormer: End-to-End Transformer for Document Understanding Srikar Appalaraju Bhavan A. Jasani Bhargava Urala Kota Yusheng Xie R. Manmatha ViT 91 280 0 22 Jun 2021
SelfDoc: Self-Supervised Document Representation Learning Peizhao Li Jiuxiang Gu Jason Kuen Vlad I. Morariu Handong Zhao R. Jain Varun Manjunatha Hongfu Liu ViT SSL 77 162 0 07 Jun 2021
StructuralLM: Structural Pre-training for Form Understanding Chenliang Li Bin Bi Ming Yan Wei Wang Songfang Huang Fei Huang Luo Si LMTD AI4CE 88 134 0 24 May 2021
Kleister: Key Information Extraction Datasets Involving Long Documents with Complex Layouts Tomasz Stanislawek Filip Graliñski Anna Wróblewska Dawid Lipiñski Agnieszka Kaliska Paulina Rosalska Bartosz Topolski P. Biecek 75 95 0 12 May 2021
InfographicVQA Minesh Mathew Viraj Bagal Rubèn Pérez Tito Dimosthenis Karatzas Ernest Valveny C. V. Jawahar 102 242 0 26 Apr 2021
LayoutXLM: Multimodal Pre-training for Multilingual Visually-rich Document Understanding Yiheng Xu Tengchao Lv Lei Cui Guoxin Wang Yijuan Lu D. Florêncio Cha Zhang Furu Wei MLLM VLM 78 130 0 18 Apr 2021
LAMPRET: Layout-Aware Multimodal PreTraining for Document Understanding Te-Lin Wu Cheng-rong Li Mingyang Zhang Tao Chen Spurthi Amba Hombaiah Michael Bendersky 65 14 0 16 Apr 2021
Learning Transferable Visual Models From Natural Language Supervision Alec Radford Jong Wook Kim Chris Hallacy Aditya A. Ramesh Gabriel Goh ... Amanda Askell Pamela Mishkin Jack Clark Gretchen Krueger Ilya Sutskever CLIP VLM 981 29,871 0 26 Feb 2021
Going Full-TILT Boogie on Document Understanding with Text-Image-Layout Transformer Rafal Powalski Łukasz Borchmann Dawid Jurkiewicz Tomasz Dwojak Michal Pietruszka Gabriela Pałka ViT 68 160 0 18 Feb 2021
ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision Wonjae Kim Bokyung Son Ildoo Kim VLM CLIP 134 1,761 0 05 Feb 2021
Unifying Vision-and-Language Tasks via Text Generation Jaemin Cho Jie Lei Hao Tan Joey Tianyi Zhou MLLM 339 542 0 04 Feb 2021
VisualMRC: Machine Reading Comprehension on Document Images Ryota Tanaka Kyosuke Nishida Sen Yoshida 91 145 0 27 Jan 2021
WebSRC: A Dataset for Web-Based Structural Reading Comprehension Xingyu Chen Zihan Zhao Lu Chen Danyang Zhang Jiabao Ji Ao Luo Yuxuan Xiong Kai Yu RALM 59 98 0 23 Jan 2021
LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding Yang Xu Yiheng Xu Tengchao Lv Lei Cui Furu Wei ... D. Florêncio Cha Zhang Wanxiang Che Min Zhang Lidong Zhou ViT MLLM 192 516 0 29 Dec 2020
Towards a Multi-modal, Multi-task Learning based Pre-training Framework for Document Representation Learning Subhojeet Pramanik Shashank Mujumdar Hima Patel 131 32 0 30 Sep 2020
DocVQA: A Dataset for VQA on Document Images Minesh Mathew Dimosthenis Karatzas C. V. Jawahar 146 743 0 01 Jul 2020
AxCell: Automatic Extraction of Results from Machine Learning Papers Marcin Kardas Piotr Czapla Pontus Stenetorp Sebastian Ruder Sebastian Riedel Ross Taylor Robert Stojnic 45 76 0 29 Apr 2020
Pixel-BERT: Aligning Image Pixels with Text by Deep Multi-Modal Transformers Zhicheng Huang Zhaoyang Zeng Bei Liu Dongmei Fu Jianlong Fu ViT 150 440 0 02 Apr 2020
LAMBERT: Layout-Aware (Language) Modeling for information extraction Lukasz Garncarek Rafal Powalski Tomasz Stanislawek Bartosz Topolski Piotr Halama M. Turski Filip Graliñski 55 88 0 19 Feb 2020
LayoutLM: Pre-training of Text and Layout for Document Image Understanding Yiheng Xu Minghao Li Lei Cui Shaohan Huang Furu Wei Ming Zhou 137 707 0 31 Dec 2019
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 488 20,342 0 23 Oct 2019
TabFact: A Large-scale Dataset for Table-based Fact Verification Wenhu Chen Hongmin Wang Jianshu Chen Yunkai Zhang Hong Wang Shiyang Li Xiyou Zhou William Yang Wang LMTD 109 514 0 05 Sep 2019
VL-BERT: Pre-training of Generic Visual-Linguistic Representations Weijie Su Xizhou Zhu Yue Cao Bin Li Lewei Lu Furu Wei Jifeng Dai VLM MLLM SSL 177 1,668 0 22 Aug 2019
LXMERT: Learning Cross-Modality Encoder Representations from Transformers Hao Hao Tan Joey Tianyi Zhou VLM MLLM 252 2,493 0 20 Aug 2019
PubLayNet: largest dataset ever for document layout analysis Xu Zhong Jianbin Tang Antonio Jimeno Yepes 52 461 0 16 Aug 2019
VisualBERT: A Simple and Performant Baseline for Vision and Language Liunian Harold Li Mark Yatskar Da Yin Cho-Jui Hsieh Kai-Wei Chang VLM 153 1,965 0 09 Aug 2019
FUNSD: A Dataset for Form Understanding in Noisy Scanned Documents Guillaume Jaume H. K. Ekenel Jean-Philippe Thiran 168 372 0 27 May 2019
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.8K 95,229 0 11 Oct 2018
SQuAD: 100,000+ Questions for Machine Comprehension of Text Pranav Rajpurkar Jian Zhang Konstantin Lopyrev Percy Liang RALM 316 8,174 0 16 Jun 2016
Compositional Semantic Parsing on Semi-Structured Tables Panupong Pasupat Percy Liang CoGe LMTD 125 793 0 03 Aug 2015