Understanding Information Storage and Transfer in Multi-modal Large
Language Models

Understanding Information Storage and Transfer in Multi-modal Large Language Models

6 June 2024

Samyadeep Basu

Daniela Massiceti

Papers citing "Understanding Information Storage and Transfer in Multi-modal Large Language Models"

14 / 14 papers shown

Title
Platonic Grounding for Efficient Multimodal Language Models Moulik Choraria Xinbo Wu Akhil Bhimaraju Nitesh Sekhar Yue Wu Xu Zhang Prateek Singhal L. Varshney 59 0 0 27 Apr 2025
Skip-Vision: Efficient and Scalable Acceleration of Vision-Language Models via Adaptive Token Skipping Weili Zeng Ziyuan Huang Kaixiang Ji Yichao Yan VLM 42 1 0 26 Mar 2025
See What You Are Told: Visual Attention Sink in Large Multimodal Models Seil Kang Jinyeong Kim Junhyeok Kim Seong Jae Hwang VLM 112 6 0 05 Mar 2025
Visual Attention Never Fades: Selective Progressive Attention ReCalibration for Detailed Image Captioning in Multimodal Large Language Models Mingi Jung Saehuyng Lee Eunji Kim Sungroh Yoon 68 0 0 03 Feb 2025
Performance Gap in Entity Knowledge Extraction Across Modalities in Vision Language Models Ido Cohen Daniela Gottesman Mor Geva Raja Giryes VLM 92 0 1 18 Dec 2024
Lifelong Knowledge Editing for Vision Language Models with Low-Rank Mixture-of-Experts Qizhou Chen Chengyu Wang Dakan Wang Taolin Zhang Wangyue Li Xiaofeng He KELM 80 1 0 23 Nov 2024
Understanding Multimodal LLMs: the Mechanistic Interpretability of Llava in Visual Question Answering Zeping Yu Sophia Ananiadou 136 0 0 17 Nov 2024
From Pixels to Prose: Advancing Multi-Modal Language Models for Remote Sensing Xingchen Sun Benji Peng Charles Zhang Fei Jin Qian Niu ... Ming Li Pohsun Feng Ziqian Bi Ming Liu Yuyao Zhang 54 0 0 05 Nov 2024
Towards Interpreting Visual Information Processing in Vision-Language Models Clement Neo Luke Ong Philip H. S. Torr Mor Geva David M. Krueger Fazl Barez 86 6 0 09 Oct 2024
Locating and Editing Factual Associations in Mamba Arnab Sen Sharma David Atkinson David Bau KELM 73 28 0 04 Apr 2024
Dissecting Recall of Factual Associations in Auto-Regressive Language Models Mor Geva Jasmijn Bastings Katja Filippova Amir Globerson KELM 191 261 0 28 Apr 2023
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models Junnan Li Dongxu Li Silvio Savarese Steven C. H. Hoi VLM MLLM 270 4,244 0 30 Jan 2023
Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small Kevin Wang Alexandre Variengien Arthur Conmy Buck Shlegeris Jacob Steinhardt 212 496 0 01 Nov 2022
Reassessing Evaluation Practices in Visual Question Answering: A Case Study on Out-of-Distribution Generalization Aishwarya Agrawal Ivana Kajić Emanuele Bugliarello Elnaz Davoodi Anita Gergely Phil Blunsom Aida Nematzadeh OOD 40 17 0 24 May 2022