v1v2 (latest)

DistilDoc: Knowledge Distillation for Visually-Rich Document Applications

12 June 2024

Matthew Blaschko

Papers citing "DistilDoc: Knowledge Distillation for Visually-Rich Document Applications"

50 / 85 papers shown

Title
DocSum: Domain-Adaptive Pre-training for Document Abstractive Summarization Phan Phuong Mai Chau Souhail Bakkali Antoine Doucet 112 0 0 11 Dec 2024
PDFTriage: Question Answering over Long, Structured Documents Jon Saad-Falcon Joe Barrow Alexa F. Siu A. Nenkova David Seunghyun Yoon Ryan Rossi Franck Dernoncourt RALM 45 20 0 16 Sep 2023
Vision Grid Transformer for Document Layout Analysis Cheng Da Chuwei Luo Qi Zheng Cong Yao ViT 79 31 0 29 Aug 2023
Beyond Document Page Classification: Design, Datasets, and Challenges Jordy Van Landeghem Sanket Biswas Matthew B. Blaschko Marie-Francine Moens 81 7 0 24 Aug 2023
A Survey on Model Compression for Large Language Models Xunyu Zhu Jian Li Yong Liu Can Ma Weiping Wang 77 226 0 15 Aug 2023
Llama 2: Open Foundation and Fine-Tuned Chat Models Hugo Touvron Louis Martin Kevin R. Stone Peter Albert Amjad Almahairi ... Sharan Narang Aurelien Rodriguez Robert Stojnic Sergey Edunov Thomas Scialom AI4MH ALM 302 11,894 0 18 Jul 2023
DocTr: Document Transformer for Structured Information Extraction in Documents Haofu Liao Aruni RoyChowdhury Weijian Li Ankan Bansal Yuting Zhang Zhuowen Tu R. Satzoda R. Manmatha Vijay Mahadevan 51 11 0 16 Jul 2023
On Evaluation of Document Classification using RVL-CDIP Stefan Larson Gordon Lim Kevin Leach 76 3 0 21 Jun 2023
Do-GOOD: Towards Distribution Shift Evaluation for Pre-Trained Visual Document Understanding Models Jiabang He Yilang Hu Lei Wang Xingdong Xu Ning Liu Hui-juan Liu Hengtao Shen VLM OOD 47 2 0 05 Jun 2023
Layout and Task Aware Instruction Prompt for Zero-shot Document Image Question Answering Wenjin Wang Yunhao Li Yixin Ou Yin Zhang VLM 110 26 0 01 Jun 2023
Document Understanding Dataset and Evaluation (DUDE) Jordy Van Landeghem Rubèn Pérez Tito Łukasz Borchmann Michal Pietruszka Pawel Józiak ... Bertrand Ackaert Ernest Valveny Matthew Blaschko Sien Moens Tomasz Stanislawek VGen 56 63 0 15 May 2023
SwinDocSegmenter: An End-to-End Unified Domain Adaptive Transformer for Document Instance Segmentation Ayan Banerjee Sanket Biswas Josep Lladós Umapada Pal ViT 64 16 0 08 May 2023
Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes Lokesh Nagalapatti Chun-Liang Li Chih-Kuan Yeh Hootan Nakhost Yasuhisa Fujii Alexander Ratner Ranjay Krishna Chen-Yu Lee Tomas Pfister ALM 318 546 0 03 May 2023
SelfDocSeg: A Self-Supervised vision-based Approach towards Document Segmentation Subhajit Maity Sanket Biswas Siladittya Manna Ayan Banerjee Josep Lladós Saumik Bhattacharya Umapada Pal 64 5 0 01 May 2023
GeoLayoutLM: Geometric Pre-training for Visual Information Extraction Chuwei Luo Changxu Cheng Qi Zheng Cong Yao 74 46 0 21 Apr 2023
From Knowledge Distillation to Self-Knowledge Distillation: A Unified Approach with Normalized Loss and Customized Soft Labels Zhendong Yang Ailing Zeng Zhe Li Tianke Zhang Chun Yuan Yu Li 72 76 0 23 Mar 2023
What Can We Learn From The Selective Prediction And Uncertainty Estimation Performance Of 523 Imagenet Classifiers Ido Galil Mohammed Dabbah Ran El-Yaniv UQCV 54 30 0 23 Feb 2023
DocILE Benchmark for Document Information Localization and Extraction vStvepán vSimsa Milan vSulc Michal Uvrivcávr Yash J. Patel Ahmed Hamdi ... Matyávs Skalický Jivrí Matas Antoine Doucet Mickael Coustaty Dimosthenis Karatzas 40 36 0 11 Feb 2023
Unifying Vision, Text, and Layout for Universal Document Processing Zineng Tang Ziyi Yang Guoxin Wang Yuwei Fang Yang Liu Chenguang Zhu Michael Zeng Chao-Yue Zhang Joey Tianyi Zhou VLM 82 112 0 05 Dec 2022
A Call to Reflect on Evaluation Practices for Failure Detection in Image Classification Paul F. Jaeger Carsten T. Lüth Lukas Klein Till J. Bungert UQCV 78 37 0 28 Nov 2022
Evaluating Out-of-Distribution Performance on Document Image Classifiers Stefan Larson Gordon Lim Yutong Ai David Kuang Kevin Leach OODD OOD 78 18 0 14 Oct 2022
Efficient Knowledge Distillation from Model Checkpoints Chaofei Wang Qisen Yang Rui Huang S. Song Gao Huang FedML 46 36 0 12 Oct 2022
I-ViT: Integer-only Quantization for Efficient Vision Transformer Inference Zhikai Li Qingyi Gu MQ 103 106 0 04 Jul 2022
DocLayNet: A Large Human-Annotated Dataset for Document-Layout Analysis B. Pfitzmann Christoph Auer Michele Dolfi A. Nassar Peter W. J. Staar 64 87 0 02 Jun 2022
V-Doc : Visual questions answers with Documents Yihao Ding Zhe Huang Runlin Wang Yanhang Zhang Xianru Chen Yuzhong Ma Hyunsuk Chung S. Han 53 15 0 27 May 2022
Unified Pretraining Framework for Document Understanding Jiuxiang Gu Jason Kuen Vlad I. Morariu Handong Zhao Nikolaos Barmpalios R. Jain A. Nenkova Tong Sun 71 97 0 22 Apr 2022
LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking Yupan Huang Tengchao Lv Lei Cui Yutong Lu Furu Wei 92 454 0 18 Apr 2022
Exploring Plain Vision Transformer Backbones for Object Detection Yanghao Li Hanzi Mao Ross B. Girshick Kaiming He ViT 90 808 0 30 Mar 2022
Knowledge Distillation with the Reused Teacher Classifier Defang Chen Jianhan Mei Hailin Zhang C. Wang Yan Feng Chun-Yen Chen 71 170 0 26 Mar 2022
Decoupled Knowledge Distillation Borui Zhao Quan Cui Renjie Song Yiyu Qiu Jiajun Liang 69 544 0 16 Mar 2022
DiT: Self-supervised Pre-training for Document Image Transformer Junlong Li Yiheng Xu Tengchao Lv Lei Cui Chaoxi Zhang Furu Wei ViT VLM 97 166 0 04 Mar 2022
DocSegTr: An Instance-Level End-to-End Document Image Segmentation Transformer Sanket Biswas Ayan Banerjee Josep Lladós Umapada Pal ViT 74 23 0 27 Jan 2022
Benchmarking Detection Transfer Learning with Vision Transformers Yanghao Li Saining Xie Xinlei Chen Piotr Dollar Kaiming He Ross B. Girshick 72 168 0 22 Nov 2021
Document AI: Benchmarks, Models and Applications Lei Cui Yiheng Xu Tengchao Lv Furu Wei VLM 61 73 0 16 Nov 2021
ICDAR 2021 Competition on Document VisualQuestion Answering Rubèn Pérez Tito Minesh Mathew C. V. Jawahar Ernest Valveny Dimosthenis Karatzas 65 23 0 10 Nov 2021
DocFormer: End-to-End Transformer for Document Understanding Srikar Appalaraju Bhavan A. Jasani Bhargava Urala Kota Yusheng Xie R. Manmatha ViT 86 275 0 22 Jun 2021
LoRA: Low-Rank Adaptation of Large Language Models J. E. Hu Yelong Shen Phillip Wallis Zeyuan Allen-Zhu Yuanzhi Li Shean Wang Lu Wang Weizhu Chen OffRL AI4TS AI4CE ALM AIMat 474 10,367 0 17 Jun 2021
BEiT: BERT Pre-Training of Image Transformers Hangbo Bao Li Dong Songhao Piao Furu Wei ViT 274 2,826 0 15 Jun 2021
Does Knowledge Distillation Really Work? Samuel Stanton Pavel Izmailov Polina Kirichenko Alexander A. Alemi A. Wilson FedML 69 220 0 10 Jun 2021
SelfDoc: Self-Supervised Document Representation Learning Peizhao Li Jiuxiang Gu Jason Kuen Vlad I. Morariu Handong Zhao R. Jain Varun Manjunatha Hongfu Liu ViT SSL 70 161 0 07 Jun 2021
VILA: Improving Structured Content Extraction from Scientific PDFs Using Visual Layout Groups Zejiang Shen Kyle Lo Lucy Lu Wang Bailey Kuehl Daniel S. Weld Doug Downey VLM 79 35 0 01 Jun 2021
Comparing Kullback-Leibler Divergence and Mean Squared Error Loss in Knowledge Distillation Taehyeon Kim Jaehoon Oh Nakyil Kim Sangwook Cho Se-Young Yun 51 234 0 19 May 2021
Kleister: Key Information Extraction Datasets Involving Long Documents with Complex Layouts Tomasz Stanislawek Filip Graliñski Anna Wróblewska Dawid Lipiñski Agnieszka Kaliska Paulina Rosalska Bartosz Topolski P. Biecek 73 95 0 12 May 2021
Distilling Knowledge via Knowledge Review Pengguang Chen Shu Liu Hengshuang Zhao Jiaya Jia 189 442 0 19 Apr 2021
Distilling Virtual Examples for Long-tailed Recognition Yingying He Jianxin Wu Xiu-Shen Wei 82 102 0 28 Mar 2021
Understanding Robustness of Transformers for Image Classification Srinadh Bhojanapalli Ayan Chakrabarti Daniel Glasner Daliang Li Thomas Unterthiner Andreas Veit ViT 90 385 0 26 Mar 2021
LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding Yang Xu Yiheng Xu Tengchao Lv Lei Cui Furu Wei ... D. Florêncio Cha Zhang Wanxiang Che Min Zhang Lidong Zhou ViT MLLM 192 510 0 29 Dec 2020
Cross-Layer Distillation with Semantic Calibration Defang Chen Jian-Ping Mei Yuan Zhang Can Wang Yan Feng Chun-Yen Chen FedML 88 298 0 06 Dec 2020
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai ... Matthias Minderer G. Heigold Sylvain Gelly Jakob Uszkoreit N. Houlsby ViT 657 41,103 0 22 Oct 2020
DocVQA: A Dataset for VQA on Document Images Minesh Mathew Dimosthenis Karatzas C. V. Jawahar 142 739 0 01 Jul 2020