Title
Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attacks Erfan Shayegani Md Abdullah Al Mamun Yu Fu Pedram Zaree Yue Dong Nael B. Abu-Ghazaleh AAML 147 148 0 16 Oct 2023
Octopus: Embodied Vision-Language Programmer from Environmental Feedback Jingkang Yang Yuhao Dong Shuai Liu Bo-wen Li Ziyue Wang ... Haoran Tan Jiamu Kang Yuanhan Zhang Kaiyang Zhou Ziwei Liu LM&Ro 52 46 0 12 Oct 2023
Visual Data-Type Understanding does not emerge from Scaling Vision-Language Models Vishaal Udandarao Max F. Burg Samuel Albanie Matthias Bethge VLM 36 9 0 12 Oct 2023
MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts Pan Lu Hritik Bansal Tony Xia Jiacheng Liu Chun-yue Li Hannaneh Hajishirzi Hao Cheng Kai-Wei Chang Michel Galley Jianfeng Gao LRM MLLM 43 509 0 03 Oct 2023
HallE-Control: Controlling Object Hallucination in Large Multimodal Models Bohan Zhai Shijia Yang Chenfeng Xu Sheng Shen Kurt Keutzer Chunyuan Li Manling Li MLLM 31 12 0 03 Oct 2023
Fool Your (Vision and) Language Model With Embarrassingly Simple Permutations Yongshuo Zong Tingyang Yu Ruchika Chavhan Bingchen Zhao Timothy M. Hospedales MLLM AAML LRM 32 18 0 02 Oct 2023
Application of frozen large-scale models to multimodal task-oriented dialogue Tatsuki Kawamoto Takuma Suzuki Ko Miyama Takumi Meguro Tomohiro Takagi 27 0 0 02 Oct 2023
Beyond Task Performance: Evaluating and Reducing the Flaws of Large Multimodal Models with In-Context Learning Mustafa Shukor Alexandre Ramé Corentin Dancette Matthieu Cord LRM MLLM 46 20 0 01 Oct 2023
AnyMAL: An Efficient and Scalable Any-Modality Augmented Language Model Avamarie Brueggeman Andrea Madotto Zhaojiang Lin Tushar Nagarajan Matt Smith ... Peyman Heidari Yue Liu Kavya Srinet Babak Damavandi Anuj Kumar MLLM 34 93 0 27 Sep 2023
InternLM-XComposer: A Vision-Language Large Model for Advanced Text-image Comprehension and Composition Pan Zhang Xiaoyi Wang Bin Wang Yuhang Cao Chao Xu ... Conghui He Xingcheng Zhang Yu Qiao Da Lin Jiaqi Wang MLLM 80 225 0 26 Sep 2023
Aligning Large Multimodal Models with Factually Augmented RLHF Zhiqing Sun Sheng Shen Shengcao Cao Haotian Liu Chunyuan Li ... Liangyan Gui Yu-xiong Wang Yiming Yang Kurt Keutzer Trevor Darrell VLM 52 313 0 25 Sep 2023
DeepSpeed-VisualChat: Multi-Round Multi-Image Interleave Chat via Multi-Modal Causal Attention Z. Yao Xiaoxia Wu Conglong Li Minjia Zhang Heyang Qi Olatunji Ruwase A. A. Awan Samyam Rajbhandari Yuxiong He 39 11 0 25 Sep 2023
DreamLLM: Synergistic Multimodal Comprehension and Creation Runpei Dong Chunrui Han Yuang Peng Zekun Qi Zheng Ge ... Hao-Ran Wei Xiangwen Kong Xiangyu Zhang Kaisheng Ma Li Yi MLLM 39 173 0 20 Sep 2023
MMICL: Empowering Vision-language Model with Multi-Modal In-Context Learning Haozhe Zhao Zefan Cai Shuzheng Si Xiaojian Ma Kaikai An Liang Chen Zixuan Liu Sheng Wang Wenjuan Han Baobao Chang MLLM VLM 28 133 0 14 Sep 2023
Language Models as Black-Box Optimizers for Vision-Language Models Shihong Liu Zhiqiu Lin Samuel Yu Ryan Lee Tiffany Ling Deepak Pathak Deva Ramanan VLM 35 28 0 12 Sep 2023
Unified Language-Vision Pretraining in LLM with Dynamic Discrete Visual Tokenization Yang Jin Kun Xu Kun Xu Liwei Chen Chao Liao ... Xiaoqiang Lei Di Zhang Wenwu Ou Kun Gai Yadong Mu MLLM VLM 19 41 0 09 Sep 2023
A Joint Study of Phrase Grounding and Task Performance in Vision and Language Models Noriyuki Kojima Hadar Averbuch-Elor Yoav Artzi 28 2 0 06 Sep 2023
PointLLM: Empowering Large Language Models to Understand Point Clouds Runsen Xu Xiaolong Wang Tai Wang Yilun Chen Jiangmiao Pang Dahua Lin MLLM 61 151 0 31 Aug 2023
Sparkles: Unlocking Chats Across Multiple Images for Multimodal Instruction-Following Models Yupan Huang Zaiqiao Meng Fangyu Liu Yixuan Su Nigel Collier Yutong Lu MLLM 41 22 0 31 Aug 2023
On the Adversarial Robustness of Multi-Modal Foundation Models Christian Schlarmann Matthias Hein AAML 116 86 0 21 Aug 2023
VisIT-Bench: A Benchmark for Vision-Language Instruction Following Inspired by Real-World Use Yonatan Bitton Hritik Bansal Jack Hessel Rulin Shao Wanrong Zhu Anas Awadalla Josh Gardner Rohan Taori L. Schimdt VLM 31 77 0 12 Aug 2023
MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities Weihao Yu Zhengyuan Yang Linjie Li Jianfeng Wang Kevin Qinghong Lin Zicheng Liu Xinchao Wang Lijuan Wang MLLM 60 615 0 04 Aug 2023
Look, Remember and Reason: Grounded reasoning in videos with language models Apratim Bhattacharyya Sunny Panchal Mingu Lee Reza Pourreza Pulkit Madan Roland Memisevic LRM 35 7 0 30 Jun 2023
A Survey on Multimodal Large Language Models Shukang Yin Chaoyou Fu Sirui Zhao Ke Li Xing Sun Tong Xu Enhong Chen MLLM LRM 54 556 0 23 Jun 2023
OBELICS: An Open Web-Scale Filtered Dataset of Interleaved Image-Text Documents Hugo Laurenccon Lucile Saulnier Léo Tronchon Stas Bekman Amanpreet Singh ... Siddharth Karamcheti Alexander M. Rush Douwe Kiela Matthieu Cord Victor Sanh 25 230 0 21 Jun 2023
Investigating Prompting Techniques for Zero- and Few-Shot Visual Question Answering Rabiul Awal Le Zhang Aishwarya Agrawal LRM 46 12 0 16 Jun 2023
ArtWhisperer: A Dataset for Characterizing Human-AI Interactions in Artistic Creations Kailas Vodrahalli James Zou 27 5 0 13 Jun 2023
Exploring Diverse In-Context Configurations for Image Captioning Xu Yang Yongliang Wu Mingzhuo Yang Haokun Chen Xin Geng MLLM 27 51 0 24 May 2023
On the Hidden Mystery of OCR in Large Multimodal Models Yuliang Liu Zhang Li Mingxin Huang Chunyuan Li Dezhi Peng Mingyu Liu Lianwen Jin Xiang Bai VLM MLLM 34 54 0 13 May 2023
mPLUG-Owl: Modularization Empowers Large Language Models with Multimodality Qinghao Ye Haiyang Xu Guohai Xu Jiabo Ye Ming Yan ... Junfeng Tian Qiang Qi Ji Zhang Feiyan Huang Jingren Zhou VLM MLLM 208 905 0 27 Apr 2023
MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models Deyao Zhu Jun Chen Xiaoqian Shen Xiang Li Mohamed Elhoseiny VLM MLLM 70 1,904 0 20 Apr 2023
Multimodal Chain-of-Thought Reasoning in Language Models Zhuosheng Zhang Aston Zhang Mu Li Hai Zhao George Karypis Alexander J. Smola LRM 30 409 0 02 Feb 2023
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models Junnan Li Dongxu Li Silvio Savarese Steven C. H. Hoi VLM MLLM 308 4,261 0 30 Jan 2023
BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation Junnan Li Dongxu Li Caiming Xiong Guosheng Lin MLLM BDL VLM CLIP 392 4,154 0 28 Jan 2022
An Empirical Study of GPT-3 for Few-Shot Knowledge-Based VQA Zhengyuan Yang Zhe Gan Jianfeng Wang Xiaowei Hu Yumao Lu Zicheng Liu Lijuan Wang 180 402 0 10 Sep 2021
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision Chao Jia Yinfei Yang Ye Xia Yi-Ting Chen Zarana Parekh Hieu H. Pham Quoc V. Le Yun-hsuan Sung Zhen Li Tom Duerig VLM CLIP 337 3,708 0 11 Feb 2021
Unified Vision-Language Pre-Training for Image Captioning and VQA Luowei Zhou Hamid Palangi Lei Zhang Houdong Hu Jason J. Corso Jianfeng Gao MLLM VLM 252 927 0 24 Sep 2019