Title
Large Language Models for Education: A Survey Hanyi Xu Wensheng Gan Zhenlian Qi Jiayang Wu Philip S. Yu AI4Ed ELM 54 14 0 12 May 2024
FreeBind: Free Lunch in Unified Multimodal Space via Knowledge Fusion Zehan Wang Ziang Zhang Xize Cheng Rongjie Huang Luping Liu ... Haifeng Huang Yang Zhao Tao Jin Peng Gao Zhou Zhao 23 8 0 08 May 2024
RAG and RAU: A Survey on Retrieval-Augmented Language Model in Natural Language Processing Yucheng Hu Yuxing Lu RALM 58 17 0 30 Apr 2024
Hallucination of Multimodal Large Language Models: A Survey Zechen Bai Pichao Wang Tianjun Xiao Tong He Zongbo Han Zheng Zhang Mike Zheng Shou VLM LRM 95 139 0 29 Apr 2024
WorldGPT: Empowering LLM as Multimodal World Model Zhiqi Ge Hongzhe Huang Mingze Zhou Juncheng Li Guoming Wang Siliang Tang Yueting Zhuang 35 26 0 28 Apr 2024
BRAVE: Broadening the visual encoding of vision-language models Ouguzhan Fatih Kar A. Tonioni Petra Poklukar Achin Kulshrestha Amir Zamir Federico Tombari MLLM VLM 42 25 0 10 Apr 2024
ChatTracer: Large Language Model Powered Real-time Bluetooth Device Tracking System Qijun Wang Shichen Zhang Kunzhe Song Huacheng Zeng 22 1 0 28 Mar 2024
MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems? Renrui Zhang Dongzhi Jiang Yichi Zhang Haokun Lin Ziyu Guo ... Aojun Zhou Pan Lu Kai-Wei Chang Peng Gao Hongsheng Li 32 168 0 21 Mar 2024
Improved Baselines for Data-efficient Perceptual Augmentation of LLMs Théophane Vallaeys Mustafa Shukor Matthieu Cord Jakob Verbeek 54 12 0 20 Mar 2024
HYDRA: A Hyper Agent for Dynamic Compositional Visual Reasoning Fucai Ke Zhixi Cai Simindokht Jahangard Weiqing Wang P. D. Haghighi Hamid Rezatofighi LRM 43 10 0 19 Mar 2024
CAT: Enhancing Multimodal Large Language Model to Answer Questions in Dynamic Audio-Visual Scenarios Qilang Ye Zitong Yu Rui Shao Xinyu Xie Philip H. S. Torr Xiaochun Cao MLLM 39 24 0 07 Mar 2024
TMT: Tri-Modal Translation between Speech, Image, and Text by Processing Different Modalities as Different Languages Minsu Kim Jee-weon Jung Hyeongseop Rha Soumi Maiti Siddhant Arora Xuankai Chang Shinji Watanabe Y. Ro 28 6 0 25 Feb 2024
Where Visual Speech Meets Language: VSP-LLM Framework for Efficient and Context-Aware Visual Speech Processing Jeong Hun Yeo Seunghee Han Minsu Kim Y. Ro 48 11 0 23 Feb 2024
Model Composition for Multimodal Large Language Models Chi Chen Yiyang Du Zheng Fang Ziyue Wang Fuwen Luo ... Ming Yan Ji Zhang Fei Huang Maosong Sun Yang Janet Liu MoMe 24 3 0 20 Feb 2024
It's Never Too Late: Fusing Acoustic Information into Large Language Models for Automatic Speech Recognition Chen Chen Ruizhe Li Yuchen Hu Sabato Marco Siniscalchi Pin-Yu Chen Ensiong Chng Chao-Han Huck Yang 26 19 0 08 Feb 2024
SPHINX-X: Scaling Data and Parameters for a Family of Multi-modal Large Language Models Chris Liu Renrui Zhang Longtian Qiu Siyuan Huang Weifeng Lin ... Hao Shao Pan Lu Hongsheng Li Yu Qiao Peng Gao MLLM 128 107 0 08 Feb 2024
Audio Flamingo: A Novel Audio Language Model with Few-Shot Learning and Dialogue Abilities Zhifeng Kong Arushi Goel Rohan Badlani Wei Ping Rafael Valle Bryan Catanzaro AuLLM LM&MA MLLM 66 73 0 02 Feb 2024
A Survey on Hallucination in Large Vision-Language Models Hanchao Liu Wenyuan Xue Yifei Chen Dapeng Chen Xiutian Zhao Ke Wang Liping Hou Rong-Zhi Li Wei Peng LRM MLLM 29 112 0 01 Feb 2024
Binding Touch to Everything: Learning Unified Multimodal Tactile Representations Fengyu Yang Chao Feng Ziyang Chen Hyoungseob Park Daniel Wang ... Ziyao Zeng Xien Chen Rit Gangopadhyay Andrew Owens Alex Wong 38 53 0 31 Jan 2024
Image Anything: Towards Reasoning-coherent and Training-free Multi-modal Image Generation Yuanhuiyi Lyu Xueye Zheng Lin Wang DiffM 33 9 0 31 Jan 2024
MouSi: Poly-Visual-Expert Vision-Language Models Xiaoran Fan Tao Ji Changhao Jiang Shuo Li Senjie Jin ... Qi Zhang Xipeng Qiu Xuanjing Huang Zuxuan Wu Yunchun Jiang VLM 24 16 0 30 Jan 2024
Benchmarking Large Multimodal Models against Common Corruptions Jiawei Zhang Tianyu Pang Chao Du Yi Ren Bo-wen Li Min-Bin Lin MLLM 24 14 0 22 Jan 2024
ModaVerse: Efficiently Transforming Modalities with LLMs Xinyu Wang Bohan Zhuang Qi Wu 14 11 0 12 Jan 2024
3DMIT: 3D Multi-modal Instruction Tuning for Scene Understanding Zeju Li Chao Zhang Xiaoyan Wang Ruilong Ren Yifan Xu Ruifei Ma Xiangde Liu MLLM 13 20 0 06 Jan 2024
Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Action Jiasen Lu Christopher Clark Sangho Lee Zichen Zhang Savya Khosla Ryan Marten Derek Hoiem Aniruddha Kembhavi VLM MLLM 34 144 0 28 Dec 2023
Visual Instruction Tuning towards General-Purpose Multimodal Model: A Survey Jiaxing Huang Jingyi Zhang Kai Jiang Han Qiu Shijian Lu 35 22 0 27 Dec 2023
FM-OV3D: Foundation Model-based Cross-modal Knowledge Blending for Open-Vocabulary 3D Detection Dongmei Zhang Chang Li Ray Zhang Shenghao Xie Wei Xue Xiaodong Xie Shanghang Zhang VLM 25 14 0 22 Dec 2023
3DAxiesPrompts: Unleashing the 3D Spatial Task Capabilities of GPT-4V Dingning Liu Xiaomeng Dong Renrui Zhang Xu Luo Peng Gao Xiaoshui Huang Yongshun Gong Zhihui Wang 32 10 0 15 Dec 2023
Chat-3D v2: Bridging 3D Scene and Large Language Models with Object Identifiers Haifeng Huang Zehan Wang Rongjie Huang Luping Liu Xize Cheng Yang Zhao Tao Jin Zhou Zhao 59 42 0 13 Dec 2023
Multimodality of AI for Education: Towards Artificial General Intelligence Gyeong-Geon Lee Lehong Shi Ehsan Latif Yizhu Gao Arne Bewersdorff ... Zheng Liu Hui Wang Gengchen Mai Tiaming Liu Xiaoming Zhai 22 37 0 10 Dec 2023
LEGO: Learning EGOcentric Action Frame Generation via Visual Instruction Tuning Bolin Lai Xiaoliang Dai Lawrence Chen Guan Pang James M. Rehg Miao Liu 35 14 0 06 Dec 2023
Reason2Drive: Towards Interpretable and Chain-based Reasoning for Autonomous Driving Ming-Jun Nie Renyuan Peng Chunwei Wang Xinyue Cai Jianhua Han Hang Xu Li Zhang LRM 29 45 0 06 Dec 2023
X-InstructBLIP: A Framework for aligning X-Modal instruction-aware representations to LLMs and Emergent Cross-modal Reasoning Artemis Panagopoulou Le Xue Ning Yu Junnan Li Dongxu Li Shafiq R. Joty Ran Xu Silvio Savarese Caiming Xiong Juan Carlos Niebles VLM MLLM 38 45 0 30 Nov 2023
ViT-Lens: Towards Omni-modal Representations Weixian Lei Yixiao Ge Kun Yi Jianfeng Zhang Difei Gao Dylan Sun Yuying Ge Ying Shan Mike Zheng Shou 21 18 0 27 Nov 2023
Video-LLaVA: Learning United Visual Representation by Alignment Before Projection Bin Lin Yang Ye Bin Zhu Jiaxi Cui Munan Ning Peng Jin Li-ming Yuan VLM MLLM 194 588 0 16 Nov 2023
Vision-Language Instruction Tuning: A Review and Analysis Chen Li Yixiao Ge Dian Li Ying Shan VLM 33 12 0 14 Nov 2023
SPHINX: The Joint Mixing of Weights, Tasks, and Visual Embeddings for Multi-modal Large Language Models Ziyi Lin Chris Liu Renrui Zhang Peng Gao Longtian Qiu ... Siyuan Huang Yichi Zhang Xuming He Hongsheng Li Yu Qiao MLLM VLM 33 208 0 13 Nov 2023
Enhancing the Spatial Awareness Capability of Multi-Modal Large Language Model Yongqiang Zhao Zhenyu Li Zhi Jin Feng Zhang Haiyan Zhao Chengfeng Dou Zhengwei Tao Xinhai Xu Donghong Liu 21 4 0 31 Oct 2023
Extending Multi-modal Contrastive Representations Zehan Wang Ziang Zhang Luping Liu Yang Zhao Haifeng Huang Tao Jin Zhou Zhao 21 5 0 13 Oct 2023
LLark: A Multimodal Instruction-Following Language Model for Music Josh Gardner Simon Durand Daniel Stoller Rachel M. Bittner AuLLM 23 14 0 11 Oct 2023
Improving Compositional Text-to-image Generation with Large Vision-Language Models Song Wen Guian Fang Renrui Zhang Peng Gao Hao Dong Dimitris N. Metaxas 21 17 0 10 Oct 2023
ReForm-Eval: Evaluating Large Vision Language Models via Unified Re-Formulation of Task-Oriented Benchmarks Zejun Li Ye Wang Mengfei Du Qingwen Liu Binhao Wu ... Zhihao Fan Jie Fu Jingjing Chen Xuanjing Huang Zhongyu Wei 27 13 0 04 Oct 2023
Dynamic-SUPERB: Towards A Dynamic, Collaborative, and Comprehensive Instruction-Tuning Benchmark for Speech Chien-yu Huang Ke-Han Lu Shi Wang Chi-Yuan Hsiao Chun-Yi Kuan ... Roshan S. Sharma Shinji Watanabe Bhiksha Ramakrishnan Shady Shehata Hung-yi Lee AuLLM 32 50 0 18 Sep 2023
PointLLM: Empowering Large Language Models to Understand Point Clouds Runsen Xu Xiaolong Wang Tai Wang Yilun Chen Jiangmiao Pang Dahua Lin MLLM 56 149 0 31 Aug 2023
A Survey on Multimodal Large Language Models Shukang Yin Chaoyou Fu Sirui Zhao Ke Li Xing Sun Tong Bill Xu Enhong Chen MLLM LRM 48 553 0 23 Jun 2023
MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models Chaoyou Fu Peixian Chen Yunhang Shen Yulei Qin Mengdan Zhang ... Xiawu Zheng Ke Li Xing Sun Zhenyu Qiu Rongrong Ji ELM MLLM 42 759 0 23 Jun 2023
Referred by Multi-Modality: A Unified Temporal Transformer for Video Object Segmentation Shilin Yan Renrui Zhang Ziyu Guo Wenchao Chen Wei Zhang Hongyang Li Yu Qiao Hao Dong Zhongjiang He Peng Gao VOS 20 30 0 25 May 2023
VideoLLM: Modeling Video Sequence with Large Language Models Guo Chen Yin-Dong Zheng Jiahao Wang Jilan Xu Yifei Huang ... Yi Wang Yali Wang Yu Qiao Tong Lu Limin Wang MLLM 92 76 0 22 May 2023
Instruction Tuning with GPT-4 Baolin Peng Chunyuan Li Pengcheng He Michel Galley Jianfeng Gao SyDa ALM LM&MA 159 579 0 06 Apr 2023
LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention Renrui Zhang Jiaming Han Chris Liu Peng Gao Aojun Zhou Xiangfei Hu Shilin Yan Pan Lu Hongsheng Li Yu Qiao MLLM 35 739 0 28 Mar 2023