Title
OmniCharacter: Towards Immersive Role-Playing Agents with Seamless Speech-Language Personality Interaction Haonan Zhang Run Luo Xiong Liu Yuchuan Wu Ting-En Lin ... Min Yang Lianli Gao Jingkuan Song Fei Huang Yongbin Li AI4CE 71 0 0 26 May 2025
Prioritizing Image-Related Tokens Enhances Vision-Language Pre-Training Yiran Chen Hao Peng Tong Zhang Heng Ji VLM 67 0 0 13 May 2025
VCM: Vision Concept Modeling Based on Implicit Contrastive Learning with Vision-Language Instruction Fine-Tuning Run Luo Renke Shan Longze Chen Ziqiang Liu Lu Wang Min Yang Xiaobo Xia MLLM VLM 249 1 0 28 Apr 2025
Distilling Transitional Pattern to Large Language Models for Multimodal Session-based Recommendation Jiajie Su Qiyong Zhong Yunshan Ma Weiming Liu Chaochao Chen Xiaolin Zheng Yuxiang Cai Tat-Seng Chua 67 0 0 13 Apr 2025
Learning to Instruct for Visual Instruction Tuning Zhihan Zhou Feng Hong Jiaan Luo Jiangchao Yao Dongsheng Li Bo Han Yize Zhang Yanfeng Wang VLM 104 1 0 28 Mar 2025
Continual Multimodal Contrastive Learning Xiaohao Liu Xiaobo Xia See-Kiong Ng Tat-Seng Chua CLL 213 2 0 19 Mar 2025
Beware of Your Po! Measuring and Mitigating AI Safety Risks in Role-Play Fine-Tuning of LLMs Weixiang Zhao Yulin Hu Yang Deng Jiahe Guo Xingyu Sui ... An Zhang Yanyan Zhao Bing Qin Tat-Seng Chua Ting Liu 150 7 0 28 Feb 2025
OpenOmni: Advancing Open-Source Omnimodal Large Language Models with Progressive Multimodal Alignment and Real-Time Self-Aware Emotional Speech Synthesis Run Luo Ting-En Lin Jun Wang Yuchuan Wu Xiong Liu ... Lei Zhang Yushen Chen Xiaobo Xia Hamid Alinejad-Rokny Fei Huang VLM AuLLM 124 0 0 08 Jan 2025
Towards Modality Generalization: A Benchmark and Prospective Analysis Xiaohao Liu Xiaobo Xia Zhuo Huang See-Kiong Ng Tat-Seng Chua 54 4 0 24 Dec 2024
LaVin-DiT: Large Vision Diffusion Transformer Zhaoqing Wang Xiaobo Xia Runnan Chen Dongdong Yu Changhu Wang Mingming Gong Tongliang Liu 149 7 0 18 Nov 2024
IP-MOT: Instance Prompt Learning for Cross-Domain Multi-Object Tracking Run Luo Zikai Song Longze Chen Yunshui Li Min Yang Wei-Guo Yang 81 0 0 30 Oct 2024
Law of Vision Representation in MLLMs Shijia Yang Bohan Zhai Quanzeng You Jianbo Yuan Hongxia Yang Chenfeng Xu 96 12 0 29 Aug 2024
Autogenic Language Embedding for Coherent Point Tracking Zikai Song Ying Tang Run Luo Lintao Ma Junqing Yu Yi-Ping Phoebe Chen Wei Yang 111 4 0 30 Jul 2024
Towards Spoken Language Understanding via Multi-level Multi-grained Contrastive Learning Xuxin Cheng Wanshi Xu Zhihong Zhu Hongxiang Li Yuexian Zou 86 13 0 31 May 2024
SEED-X: Multimodal Models with Unified Multi-granularity Comprehension and Generation Yuying Ge Sijie Zhao Jinguo Zhu Yixiao Ge Kun Yi Lin Song Chen Li Xiaohan Ding Ying Shan VLM 115 142 0 22 Apr 2024
Few-Shot Adversarial Prompt Learning on Vision-Language Models Yiwei Zhou Xiaobo Xia Zhiwei Lin Bo Han Tongliang Liu VLM 87 16 0 21 Mar 2024
Open-Vocabulary Segmentation with Unpaired Mask-Text Supervision Zhaoqing Wang Xiaobo Xia Ziye Chen Xiao He Yandong Guo Biwei Huang Tongliang Liu VLM 78 13 0 14 Feb 2024
MM-Interleaved: Interleaved Image-Text Generative Modeling via Multi-modal Feature Synchronizer Changyao Tian Xizhou Zhu Yuwen Xiong Weiyun Wang Zhe Chen ... Tong Lu Jie Zhou Hongsheng Li Yu Qiao Jifeng Dai AuLLM 126 47 0 18 Jan 2024
Generative Multimodal Models are In-Context Learners Quan-Sen Sun Yufeng Cui Xiaosong Zhang Fan Zhang Qiying Yu ... Yueze Wang Yongming Rao Jingjing Liu Tiejun Huang Xinlong Wang MLLM LRM 153 288 0 20 Dec 2023
NExT-Chat: An LMM for Chat, Detection and Segmentation Ao Zhang Yuan Yao Wei Ji Zhiyuan Liu Tat-Seng Chua MLLM VLM 68 55 0 08 Nov 2023
De-Diffusion Makes Text a Strong Cross-Modal Interface Chen Wei Chenxi Liu Siyuan Qiao Zhishuai Zhang Alan Yuille Jiahui Yu VLM DiffM 76 11 0 01 Nov 2023
IDEAL: Influence-Driven Selective Annotations Empower In-Context Learners in Large Language Models Shaokun Zhang Xiaobo Xia Zhaoqing Wang Ling-Hao Chen Jiale Liu Qingyun Wu Tongliang Liu 72 21 0 16 Oct 2023
Improved Baselines with Visual Instruction Tuning Haotian Liu Chunyuan Li Yuheng Li Yong Jae Lee VLM MLLM 160 2,817 0 05 Oct 2023
DreamLLM: Synergistic Multimodal Comprehension and Creation Runpei Dong Chunrui Han Yuang Peng Zekun Qi Zheng Ge ... Hao-Ran Wei Xiangwen Kong Xiangyu Zhang Kaisheng Ma Li Yi MLLM 80 201 0 20 Sep 2023
Scaling Autoregressive Multi-Modal Models: Pretraining and Instruction Tuning L. Yu Bowen Shi Ramakanth Pasunuru Benjamin Muller O. Yu. Golovneva ... Yaniv Taigman Maryam Fazel-Zarandi Asli Celikyilmaz Luke Zettlemoyer Armen Aghajanyan MLLM 91 142 0 05 Sep 2023
DiffusionTrack: Diffusion Model For Multi-Object Tracking Run Luo Zikai Song Lintao Ma Ji Wei Wei-Guo Yang Min Yang DiffM 88 30 0 19 Aug 2023
MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities Weihao Yu Zhengyuan Yang Linjie Li Jianfeng Wang Kevin Qinghong Lin Zicheng Liu Xinchao Wang Lijuan Wang MLLM 107 716 0 04 Aug 2023
Planting a SEED of Vision in Large Language Model Yuying Ge Yixiao Ge Ziyun Zeng Xintao Wang Ying Shan VLM MLLM 33 99 0 16 Jul 2023
GPT4RoI: Instruction Tuning Large Language Model on Region-of-Interest Shilong Zhang Pei Sun Shoufa Chen Min Xiao Wenqi Shao Wenwei Zhang Yu Liu Kai-xiang Chen Ping Luo MLLM VLM 154 238 0 07 Jul 2023
mPLUG-DocOwl: Modularized Multimodal Large Language Model for Document Understanding Jiabo Ye Anwen Hu Haiyang Xu Qinghao Ye Mingshi Yan ... Chenliang Li Junfeng Tian Qiang Qi Ji Zhang Feiyan Huang VLM MLLM 78 128 0 04 Jul 2023
Shikra: Unleashing Multimodal LLM's Referential Dialogue Magic Ke Chen Zhao Zhang Weili Zeng Richong Zhang Feng Zhu Rui Zhao ObjD 105 651 0 27 Jun 2023
Kosmos-2: Grounding Multimodal Large Language Models to the World Zhiliang Peng Wenhui Wang Li Dong Y. Hao Shaohan Huang Shuming Ma Furu Wei MLLM ObjD VLM 117 763 0 26 Jun 2023
OBELICS: An Open Web-Scale Filtered Dataset of Interleaved Image-Text Documents Hugo Laurenccon Lucile Saulnier Léo Tronchon Stas Bekman Amanpreet Singh ... Siddharth Karamcheti Alexander M. Rush Douwe Kiela Matthieu Cord Victor Sanh 132 246 0 21 Jun 2023
Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena Lianmin Zheng Wei-Lin Chiang Ying Sheng Siyuan Zhuang Zhanghao Wu ... Dacheng Li Eric Xing Haotong Zhang Joseph E. Gonzalez Ion Stoica ALM OSLM ELM 426 4,422 0 09 Jun 2023
Diffusion Model for Dense Matching Jisu Nam Gyuseong Lee Sunwoo Kim Ines Hyeonsu Kim Hyoungwon Cho Seyeong Kim Seung Wook Kim DiffM 56 9 0 30 May 2023
Generating Images with Multimodal Language Models Jing Yu Koh Daniel Fried Ruslan Salakhutdinov MLLM 93 259 0 26 May 2023
Evaluating Object Hallucination in Large Vision-Language Models Yifan Li Yifan Du Kun Zhou Jinpeng Wang Wayne Xin Zhao Ji-Rong Wen MLLM LRM 303 809 0 17 May 2023
InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning Wenliang Dai Junnan Li Dongxu Li A. M. H. Tiong Junqi Zhao Weisheng Wang Boyang Albert Li Pascale Fung Steven C. H. Hoi MLLM VLM 139 2,095 0 11 May 2023
MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models Deyao Zhu Jun Chen Xiaoqian Shen Xiang Li Mohamed Elhoseiny VLM MLLM 165 2,064 0 20 Apr 2023
Visual Instruction Tuning Haotian Liu Chunyuan Li Qingyang Wu Yong Jae Lee SyDa VLM MLLM 569 4,910 0 17 Apr 2023
Multimodal C4: An Open, Billion-scale Corpus of Images Interleaved with Text Wanrong Zhu Jack Hessel Anas Awadalla S. Gadre Jesse Dodge Alex Fang Youngjae Yu Ludwig Schmidt William Yang Wang Yejin Choi VLM 96 177 0 14 Apr 2023
Denoising Diffusion Autoencoders are Unified Self-supervised Learners Weilai Xiang Hongyu Yang Di Huang Yunhong Wang DiffM 109 78 0 17 Mar 2023
Open-Vocabulary Panoptic Segmentation with Text-to-Image Diffusion Models Jiarui Xu Sifei Liu Arash Vahdat Wonmin Byeon Xiaolong Wang Shalini De Mello VLM 276 336 0 08 Mar 2023
Language Is Not All You Need: Aligning Perception with Language Models Shaohan Huang Li Dong Wenhui Wang Y. Hao Saksham Singhal ... Johan Bjorck Vishrav Chaudhary Subhojit Som Xia Song Furu Wei VLM LRM MLLM 109 565 0 27 Feb 2023
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models Junnan Li Dongxu Li Silvio Savarese Steven C. H. Hoi VLM MLLM 429 4,642 0 30 Jan 2023
GRiT: A Generative Region-to-text Transformer for Object Understanding Jialian Wu Jianfeng Wang Zhengyuan Yang Zhe Gan Zicheng Liu Junsong Yuan Lijuan Wang ObjD VLM 52 117 0 01 Dec 2022
LAION-5B: An open large-scale dataset for training next generation image-text models Christoph Schuhmann Romain Beaumont Richard Vencu Cade Gordon Ross Wightman ... Srivatsa Kundurthy Katherine Crowson Ludwig Schmidt R. Kaczmarczyk J. Jitsev VLM MLLM CLIP 200 3,500 0 16 Oct 2022
Imagen Video: High Definition Video Generation with Diffusion Models Jonathan Ho William Chan Chitwan Saharia Jay Whang Ruiqi Gao ... Diederik P. Kingma Ben Poole Mohammad Norouzi David J. Fleet Tim Salimans VGen 171 1,541 0 05 Oct 2022
Scaling Autoregressive Models for Content-Rich Text-to-Image Generation Jiahui Yu Yuanzhong Xu Jing Yu Koh Thang Luong Gunjan Baid ... Zarana Parekh Xin Li Han Zhang Jason Baldridge Yonghui Wu EGVM 197 1,129 0 22 Jun 2022
A-OKVQA: A Benchmark for Visual Question Answering using World Knowledge Dustin Schwenk Apoorv Khandelwal Christopher Clark Kenneth Marino Roozbeh Mottaghi 69 551 0 03 Jun 2022