Title
QID: Efficient Query-Informed ViTs in Data-Scarce Regimes for OCR-free Visual Document Understanding Binh M. Le Shaoyuan Xu Jinmiao Fu Zhishen Huang Moyan Li Yanhui Guo Hongdong Li Sameera Ramasinghe Bryan Wang 35 0 0 03 Apr 2025
CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Models Qingqing Zhao Yao Lu Moo Jin Kim Zipeng Fu Zhuoyang Zhang ... Ankur Handa Xuan Li Donglai Xiang Gordon Wetzstein Nayeon Lee LM&Ro LRM 43 13 0 27 Mar 2025
MP-GUI: Modality Perception with MLLMs for GUI Understanding Ziwei Wang Weizhi Chen Leyang Yang Sheng Zhou Shengchu Zhao Hanbei Zhan Jiongchao Jin Liangcheng Li Zirui Shao Jiajun Bu 74 1 0 18 Mar 2025
Vision-Language Models for Edge Networks: A Comprehensive Survey Ahmed Sharshar Latif U. Khan Waseem Ullah Mohsen Guizani VLM 70 3 0 11 Feb 2025
Efficient Architectures for High Resolution Vision-Language Models Miguel Carvalho Bruno Martins MLLM VLM 45 0 0 05 Jan 2025
A Simple Recipe for Contrastively Pre-training Video-First Encoders Beyond 16 Frames Pinelopi Papalampidi Skanda Koppula Shreya Pathak Justin T Chiu Joseph Heyward Viorica Patraucean Jiajun Shen Antoine Miech Andrew Zisserman Aida Nematzdeh VLM 63 24 0 31 Dec 2024
A Comprehensive Survey of Large Language Models and Multimodal Large Language Models in Medicine Hanguang Xiao Feizhong Zhou X. Liu Tianqi Liu Zhipeng Li Xin Liu Xiaoxuan Huang AILaw LM&MA LRM 61 18 0 31 Dec 2024
Scaling 4D Representations João Carreira Dilara Gokay Michael King Chuhan Zhang Ignacio Rocco ... Viorica Patraucean Dima Damen Pauline Luc Mehdi S. M. Sajjadi Andrew Zisserman 85 3 0 19 Dec 2024
Neptune: The Long Orbit to Benchmarking Long Video Understanding Arsha Nagrani Mingda Zhang Ramin Mehran Rachel Hornung N. B. Gundavarapu ... Boqing Gong Cordelia Schmid Mikhail Sirotenko Yukun Zhu Tobias Weyand 103 4 0 12 Dec 2024
AnyBimanual: Transferring Unimanual Policy for General Bimanual Manipulation Guanxing Lu Tengbo Yu Haoyuan Deng Season Si Chen Yansong Tang Ziwei Wang 77 3 0 09 Dec 2024
Exploring the Adversarial Vulnerabilities of Vision-Language-Action Models in Robotics Taowen Wang Dongfang Liu James Liang Wenhao Yang Qifan Wang Cheng Han Jiebo Luo Ruixiang Tang Ruixiang Tang AAML 79 3 0 18 Nov 2024
Multimodal Instruction Tuning with Hybrid State Space Models Jianing Zhou Han Li Shuai Zhang Ning Xie Ruijie Wang Xiaohan Nie Sheng Liu Lingyun Wang 40 0 0 13 Nov 2024
TAP-VL: Text Layout-Aware Pre-training for Enriched Vision-Language Models Jonathan Fhima Elad Ben Avraham Oren Nuriel Yair Kittenplon Roy Ganz Aviad Aberdam Ron Litman VLM 34 1 0 07 Nov 2024
VL-Cache: Sparsity and Modality-Aware KV Cache Compression for Vision-Language Model Inference Acceleration Dezhan Tu Danylo Vashchilenko Yuzhe Lu Panpan Xu VLM 43 9 0 29 Oct 2024
Towards Reliable Verification of Unauthorized Data Usage in Personalized Text-to-Image Diffusion Models Boheng Li Yanhao Wei Yankai Fu Ziyi Wang Yiming Li Jie Zhang Run Wang Tianwei Zhang DiffM AAML 27 9 0 14 Oct 2024
Inferring Alt-text For UI Icons With Large Language Models During App Development Sabrina Haque Christoph Csallner VLM 36 0 0 26 Sep 2024
Playground v3: Improving Text-to-Image Alignment with Deep-Fusion Large Language Models Bingchen Liu Ehsan Akhgari Alexander Visheratin Aleks Kamko Linmiao Xu Shivam Shrirao Joao Souza Suhail Doshi Daiqing Li Daiqing Li DiffM MLLM 31 47 0 16 Sep 2024
TG-LLaVA: Text Guided LLaVA via Learnable Latent Embeddings Dawei Yan Pengcheng Li Yang Li Hao Chen Qingguo Chen Weihua Luo Wei Dong Qingsen Yan Haokui Zhang Chunhua Shen 3DV VLM 48 4 0 15 Sep 2024
Shaking Up VLMs: Comparing Transformers and Structured State Space Models for Vision & Language Modeling Georgios Pantazopoulos Malvina Nikandrou Alessandro Suglia Oliver Lemon Arash Eshghi Mamba 45 1 0 09 Sep 2024
Eagle: Exploring The Design Space for Multimodal LLMs with Mixture of Encoders Min Shi Fuxiao Liu Shihao Wang Shijia Liao Subhashree Radhakrishnan ... Andrew Tao Andrew Tao Zhiding Yu Guilin Liu Guilin Liu MLLM 30 53 0 28 Aug 2024
Building and better understanding vision-language models: insights and future directions Hugo Laurençon Andrés Marafioti Victor Sanh Léo Tronchon VLM 42 61 0 22 Aug 2024
Accelerating Pre-training of Multimodal LLMs via Chain-of-Sight Ziyuan Huang Kaixiang Ji Biao Gong Zhiwu Qing Qinglong Zhang Kecheng Zheng Jian Wang Jingdong Chen Ming Yang LRM 42 1 0 22 Jul 2024
Learning Visual Grounding from Generative Vision and Language Model Shijie Wang Dahun Kim A. Taalimi Chen Sun Weicheng Kuo ObjD 36 5 0 18 Jul 2024
Evaluating Linguistic Capabilities of Multimodal LLMs in the Lens of Few-Shot Learning Mustafa Dogan .Ilker Kesen Iacer Calixto Aykut Erdem Erkut Erdem LRM 31 1 0 17 Jul 2024
HiRes-LLaVA: Restoring Fragmentation Input in High-Resolution Large Vision-Language Models Runhui Huang Xinpeng Ding Chunwei Wang J. N. Han Yulong Liu Hengshuang Zhao Hang Xu Lu Hou Wei Zhang Xiaodan Liang VLM 28 8 0 11 Jul 2024
ChartGemma: Visual Instruction-tuning for Chart Reasoning in the Wild Ahmed Masry Megh Thakkar Aayush Bajaj Aaryaman Kartha Enamul Hoque Shafiq R. Joty VLM 41 26 0 04 Jul 2024
ColPali: Efficient Document Retrieval with Vision Language Models Manuel Faysse Hugues Sibille Tony Wu Bilel Omrani Gautier Viaud C´eline Hudelot Pierre Colombo VLM 67 21 0 27 Jun 2024
Data curation via joint example selection further accelerates multimodal learning Talfan Evans Nikhil Parthasarathy Hamza Merzic Olivier J. Hénaff 32 12 0 25 Jun 2024
OpenVLA: An Open-Source Vision-Language-Action Model Moo Jin Kim Karl Pertsch Siddharth Karamcheti Ted Xiao Ashwin Balakrishna ... Russ Tedrake Dorsa Sadigh Sergey Levine Percy Liang Chelsea Finn LM&Ro VLM 51 367 0 13 Jun 2024
mOSCAR: A Large-scale Multilingual and Multimodal Document-level Corpus Matthieu Futeral A. Zebaze Pedro Ortiz Suarez Julien Abadji Rémi Lacroix Cordelia Schmid Rachel Bawden Benoît Sagot 39 3 0 13 Jun 2024
M&M VTO: Multi-Garment Virtual Try-On and Editing Luyang Zhu Yingwei Li Nan Liu Hao Peng Dawei Yang Ira Kemelmacher-Shlizerman DiffM 52 7 0 06 Jun 2024
Selectively Answering Visual Questions Julian Martin Eisenschlos Hernán Maina Guido Ivetta Luciana Benotti 49 0 0 03 Jun 2024
StrucTexTv3: An Efficient Vision-Language Model for Text-rich Image Perception, Comprehension, and Beyond Pengyuan Lyu Yulin Li Hao Zhou Weihong Ma Xingyu Wan ... Liang Wu Chengquan Zhang Kun Yao Errui Ding Jingdong Wang 36 7 0 31 May 2024
Seeing the Image: Prioritizing Visual Correlation by Contrastive Alignment Xin Xiao Bohong Wu Jiacong Wang Chunyuan Li Xun Zhou Haoyuan Guo VLM 39 7 0 28 May 2024
No Filter: Cultural and Socioeconomic Diversity in Contrastive Vision-Language Models Angeline Pouget Lucas Beyer Emanuele Bugliarello Xiao Wang Andreas Steiner Xiao-Qi Zhai Ibrahim M. Alabdulmohsin VLM 33 7 0 22 May 2024
Adapting Multi-modal Large Language Model to Concept Drift From Pre-training Onwards Xiaoyu Yang Jie Lu Enshui Yu VLM 48 1 0 22 May 2024
Safeguarding Vision-Language Models Against Patched Visual Prompt Injectors Jiachen Sun Changsheng Wang Jiong Wang Yiwei Zhang Chaowei Xiao AAML VLM 34 3 0 17 May 2024
What matters when building vision-language models? Hugo Laurençon Léo Tronchon Matthieu Cord Victor Sanh VLM 43 156 0 03 May 2024
DOCCI: Descriptions of Connected and Contrasting Images Yasumasa Onoe Sunayana Rane Zachary Berger Yonatan Bitton Jaemin Cho ... Zarana Parekh Jordi Pont-Tuset Garrett Tanzer Su Wang Jason Baldridge 41 48 0 30 Apr 2024
HRVDA: High-Resolution Visual Document Assistant Chaohu Liu Kun Yin Haoyu Cao Xinghua Jiang Xin Li Yinsong Liu Deqiang Jiang Xing Sun Linli Xu VLM 43 24 0 10 Apr 2024
MoReVQA: Exploring Modular Reasoning Models for Video Question Answering Juhong Min Shyamal Buch Arsha Nagrani Minsu Cho Cordelia Schmid LRM 44 20 0 09 Apr 2024
Multilingual Large Language Model: A Survey of Resources, Taxonomy and Frontiers Libo Qin Qiguang Chen Yuhang Zhou Zhi Chen Hai-Tao Zheng Lizi Liao Min Li Wanxiang Che Philip S. Yu LRM 55 36 0 07 Apr 2024
ViTamin: Designing Scalable Vision Models in the Vision-Language Era Jienneg Chen Qihang Yu Xiaohui Shen Alan L. Yuille Liang-Chieh Chen 3DV VLM 36 24 0 02 Apr 2024
mChartQA: A universal benchmark for multimodal Chart Question Answer based on Vision-Language Alignment and Reasoning Jingxuan Wei Nan Xu Guiyong Chang Yin Luo Bihui Yu Ruifeng Guo 44 2 0 02 Apr 2024
LocCa: Visual Pretraining with Location-aware Captioners Bo Wan Michael Tschannen Yongqin Xian Filip Pavetić Ibrahim M. Alabdulmohsin Xiao Wang André Susano Pinto Andreas Steiner Lucas Beyer Xiao-Qi Zhai VLM 51 6 0 28 Mar 2024
Chart-based Reasoning: Transferring Capabilities from LLMs to VLMs Victor Carbune Hassan Mansoor Fangyu Liu Rahul Aralikatte Gilles Baechler Jindong Chen Abhanshu Sharma ReLM LRM 138 12 0 19 Mar 2024
Griffon v2: Advancing Multimodal Perception with High-Resolution Scaling and Visual-Language Co-Referring Yufei Zhan Yousong Zhu Hongyin Zhao Fan Yang Ming Tang Jinqiao Wang ObjD 44 12 0 14 Mar 2024
Feast Your Eyes: Mixture-of-Resolution Adaptation for Multimodal Large Language Models Gen Luo Yiyi Zhou Yuxin Zhang Xiawu Zheng Xiaoshuai Sun Rongrong Ji VLM 28 56 0 05 Mar 2024
Modeling Collaborator: Enabling Subjective Vision Classification With Minimal Human Effort via LLM Tool-Use Imad Eddine Toubal Aditya Avinash N. Alldrin Jan Dlabal Wenlei Zhou ... Chun-Ta Lu Howard Zhou Ranjay Krishna Ariel Fuxman Tom Duerig VLM 75 7 0 05 Mar 2024
Representing Online Handwriting for Recognition in Large Vision-Language Models Anastasiia Fadeeva Philippe Schlattner Andrii Maksai Mark Collier Efi Kokiopoulou Jesse Berent C. Musat 54 4 0 23 Feb 2024