PandaGPT: One Model To Instruction-Follow Them All

25 May 2023

Papers citing "PandaGPT: One Model To Instruction-Follow Them All"

50 / 62 papers shown

Title
GeoMM: On Geodesic Perspective for Multi-modal Learning Shibin Mei Hang Wang Bingbing Ni 27 0 0 16 May 2025
Sage Deer: A Super-Aligned Driving Generalist Is Your Copilot Hao Lu Jiaqi Tang Jiyao Wang Yaojie Lu Xu Cao ... Bin Huang Dengbo He Shuiguang Deng Hao Chen Ying-Cong Chen 43 0 0 15 May 2025
Large Language Models for Computer-Aided Design: A Survey Licheng Zhang Bach Le Naveed Akhtar Siew-Kei Lam Tuan Ngo 3DV AI4CE 48 0 0 13 May 2025
Prioritizing Image-Related Tokens Enhances Vision-Language Pre-Training Yiran Chen Hao Peng Tong Zhang Heng Ji VLM 35 0 0 13 May 2025
DEEMO: De-identity Multimodal Emotion Recognition and Reasoning Deng Li Bohao Xing Xin Liu Baiqiang Xia Bihan Wen Heikki Kälviäinen VLM 73 0 0 28 Apr 2025
FortisAVQA and MAVEN: a Benchmark Dataset and Debiasing Framework for Robust Multimodal Reasoning Jie Ma Zhitao Gao Qi Chai Jing Liu Peijie Wang Jing Tao Zhou Su 68 1 0 01 Apr 2025
Orchestrate Multimodal Data with Batch Post-Balancing to Accelerate Multimodal Large Language Model Training Yijie Zheng Bangjun Xiao Lei Shi Xiaoyang Li Faming Wu Tianyu Li Xuefeng Xiao Wenjie Qu Yali Wang Shouda Liu MLLM MoE 75 1 0 31 Mar 2025
Aurelia: Test-time Reasoning Distillation in Audio-Visual LLMs Sanjoy Chowdhury Hanan Gani Nishit Anand Sayan Nag Ruohan Gao Mohamed Elhoseiny Salman Khan Dinesh Manocha LRM 59 0 0 29 Mar 2025
Exploiting Mixture-of-Experts Redundancy Unlocks Multimodal Generative Abilities Raman Dutt Harleen Hanspal Guoxuan Xia Petru-Daniel Tudosiu Alexander Black Yongxin Yang Jingyu Sun Sarah Parisot MoE 43 0 0 28 Mar 2025
Adversarial Training for Multimodal Large Language Models against Jailbreak Attacks Liming Lu Shuchao Pang Siyuan Liang Haotian Zhu Xiyu Zeng Aishan Liu Yunhuai Liu Yongbin Zhou AAML 55 2 0 05 Mar 2025
Omni-SILA: Towards Omni-scene Driven Visual Sentiment Identifying, Locating and Attributing in Videos Jiamin Luo Jingjing Wang Junxiao Ma Yujie Jin Shoushan Li Guodong Zhou 38 0 0 26 Feb 2025
OneLLM: One Framework to Align All Modalities with Language Jiaming Han Kaixiong Gong Yiyuan Zhang Jiaqi Wang Kaipeng Zhang Dahua Lin Yu Qiao Peng Gao Xiangyu Yue MLLM 108 111 0 10 Jan 2025
Kernel-Aware Graph Prompt Learning for Few-Shot Anomaly Detection Fenfang Tao G. Xie Fang Zhao Xiangbo Shu 49 2 0 23 Dec 2024
Spider: Any-to-Many Multimodal LLM Jinxiang Lai Jie Zhang Jun Liu Jian Li Xiaocheng Lu Song Guo MLLM 72 2 0 14 Nov 2024
On Learning Multi-Modal Forgery Representation for Diffusion Generated Video Detection Xiufeng Song Xiao Guo Juntian Zhang Qirui Li Lei Bai Xiaoming Liu Guangtao Zhai Xiaohong Liu DiffM VGen 74 9 0 31 Oct 2024
GiVE: Guiding Visual Encoder to Perceive Overlooked Information Junjie Li Jianghong Ma Xiaofeng Zhang Yuhang Li Jianyang Shi 48 0 0 26 Oct 2024
AVHBench: A Cross-Modal Hallucination Benchmark for Audio-Visual Large Language Models Kim Sung-Bin Oh Hyun-Bin JungMok Lee Arda Senocak Joon Son Chung Tae-Hyun Oh MLLM VLM 55 3 0 23 Oct 2024
ForgeryGPT: Multimodal Large Language Model For Explainable Image Forgery Detection and Localization Jiawei Li Fanrui Zhang Jiaying Zhu Esther Sun Qiang Zhang Zheng-jun Zha MLLM 59 9 0 14 Oct 2024
End-to-end Semantic-centric Video-based Multimodal Affective Computing Ronghao Lin Ying Zeng Sijie Mai Haifeng Hu VGen 53 0 0 14 Aug 2024
Large Visual-Language Models Are Also Good Classifiers: A Study of In-Context Multimodal Fake News Detection Ye Jiang Yimin Wang MLLM 40 1 0 16 Jul 2024
CLIP3D-AD: Extending CLIP for 3D Few-Shot Anomaly Detection with Multi-View Images Generation Zuo Zuo Jiahao Dong Yao Wu Yanyun Qu Zongze Wu 47 3 0 27 Jun 2024
MMFakeBench: A Mixed-Source Multimodal Misinformation Detection Benchmark for LVLMs Xuannan Liu Zekun Li Peipei Li Shuhan Xia Xing Cui Linzhi Huang Huaibo Huang Weihong Deng Zhaofeng He 58 15 0 13 Jun 2024
FaceGPT: Self-supervised Learning to Chat about 3D Human Faces Haoran Wang Mohit Mendiratta Christian Theobalt Adam Kortylewski 3DH CVBM 33 3 0 11 Jun 2024
X-VILA: Cross-Modality Alignment for Large Language Model Hanrong Ye De-An Huang Yao Lu Zhiding Yu Ming-Yu Liu ... Jan Kautz Song Han Dan Xu Pavlo Molchanov Hongxu Yin MLLM VLM 49 32 0 29 May 2024
Prompt-Aware Adapter: Towards Learning Adaptive Visual Tokens for Multimodal Large Language Models Yue Zhang Hehe Fan Yi Yang 58 3 0 24 May 2024
A Survey on Vision-Language-Action Models for Embodied AI Yueen Ma Zixing Song Yuzheng Zhuang Jianye Hao Irwin King LM&Ro 82 45 0 23 May 2024
SOK-Bench: A Situated Video Reasoning Benchmark with Aligned Open-World Knowledge Andong Wang Bo Wu Sunli Chen Zhenfang Chen Haotian Guan Wei-Ning Lee Li Erran Li Chuang Gan LRM RALM 37 16 0 15 May 2024
EALD-MLLM: Emotion Analysis in Long-sequential and De-identity videos with Multi-modal Large Language Model Deng Li Xin Liu Bohao Xing Baiqiang Xia Yuan Zong Bihan Wen Heikki Kälviäinen 42 3 0 01 May 2024
Hallucination of Multimodal Large Language Models: A Survey Zechen Bai Pichao Wang Tianjun Xiao Tong He Zongbo Han Zheng Zhang Mike Zheng Shou VLM LRM 95 145 0 29 Apr 2024
Do LLMs Understand Visual Anomalies? Uncovering LLM's Capabilities in Zero-shot Anomaly Detection Jiaqi Zhu Shaofeng Cai Fang Deng Junran Wu Junran Wu 70 15 0 15 Apr 2024
Koala: Key frame-conditioned long video-LLM Reuben Tan Ximeng Sun Ping Hu Jui-hsien Wang Hanieh Deilamsalehy Bryan A. Plummer Bryan C. Russell Kate Saenko 38 36 0 05 Apr 2024
Long-Tailed Anomaly Detection with Learnable Class Names Chih-Hui Ho Kuan-Chuan Peng Nuno Vasconcelos OODD 48 6 0 29 Mar 2024
TempCompass: Do Video LLMs Really Understand Videos? Yuanxin Liu Shicheng Li Yi Liu Yuxiang Wang Shuhuai Ren Lei Li Sishuo Chen Xu Sun Lu Hou VLM 52 103 0 01 Mar 2024
SPHINX-X: Scaling Data and Parameters for a Family of Multi-modal Large Language Models Chris Liu Renrui Zhang Longtian Qiu Siyuan Huang Weifeng Lin ... Hao Shao Pan Lu Hongsheng Li Yu Qiao Peng Gao MLLM 130 110 0 08 Feb 2024
MLLM-Tool: A Multimodal Large Language Model For Tool Agent Learning Chenyu Wang Weixin Luo Qianyu Chen Haonan Mai Jindi Guo Sixun Dong Xiaohua Xuan MLLM LLMAG 52 19 0 19 Jan 2024
GroundingGPT:Language Enhanced Multi-modal Grounding Model Zhaowei Li Qi Xu Dong Zhang Hang Song Yiqing Cai ... Junting Pan Zefeng Li Van Tu Vu Zhida Huang Tao Wang 41 38 0 11 Jan 2024
Video Anomaly Detection and Explanation via Large Language Models Hui Lv Qianru Sun 33 21 0 11 Jan 2024
NLLG Quarterly arXiv Report 09/23: What are the most influential current AI Papers? Ran Zhang Aida Kostikova Christoph Leiter Jonas Belouadi Daniil Larionov Yanran Chen Vivian Fresen Steffen Eger 53 0 0 09 Dec 2023
Text as Images: Can Multimodal Large Language Models Follow Printed Instructions in Pixels? Xiujun Li Yujie Lu Zhe Gan Jianfeng Gao William Y. Wang Yejin Choi VLM MLLM 35 2 0 29 Nov 2023
Video-Bench: A Comprehensive Benchmark and Toolkit for Evaluating Video-based Large Language Models Munan Ning Bin Zhu Yujia Xie Bin Lin Jiaxi Cui Lu Yuan Dongdong Chen Li-ming Yuan ELM MLLM 27 58 0 27 Nov 2023
u-LLaVA: Unifying Multi-Modal Tasks via Large Language Model Jinjin Xu Liwu Xu Yuzhe Yang Xiang Li Fanyi Wang Yanchun Xie Yi-Jie Huang Yaqian Li MoE MLLM VLM 37 13 0 09 Nov 2023
Enhancing the Spatial Awareness Capability of Multi-Modal Large Language Model Yongqiang Zhao Zhenyu Li Zhi Jin Feng Zhang Haiyan Zhao Chengfeng Dou Zhengwei Tao Xinhai Xu Donghong Liu 27 4 0 31 Oct 2023
SALMONN: Towards Generic Hearing Abilities for Large Language Models Changli Tang Wenyi Yu Guangzhi Sun Xianzhao Chen Tian Tan Wei Li Lu Lu Zejun Ma Chao Zhang LM&MA AuLLM 44 209 0 20 Oct 2023
Extending Multi-modal Contrastive Representations Zehan Wang Ziang Zhang Luping Liu Yang Zhao Haifeng Huang Tao Jin Zhou Zhao 31 5 0 13 Oct 2023
Fine-grained Audio-Visual Joint Representations for Multimodal Large Language Models Guangzhi Sun Wenyi Yu Changli Tang Xianzhao Chen Tian Tan Wei Li Lu Lu Zejun Ma Chao Zhang 41 12 0 09 Oct 2023
AnyMAL: An Efficient and Scalable Any-Modality Augmented Language Model Avamarie Brueggeman Andrea Madotto Zhaojiang Lin Tushar Nagarajan Matt Smith ... Peyman Heidari Yue Liu Kavya Srinet Babak Damavandi Anuj Kumar MLLM 39 93 0 27 Sep 2023
Species196: A One-Million Semi-supervised Dataset for Fine-grained Species Recognition W. He Kai Han Ying Nie Chengcheng Wang Yunhe Wang VLM 48 6 0 25 Sep 2023
ImageBind-LLM: Multi-modality Instruction Tuning Jiaming Han Renrui Zhang Wenqi Shao Peng Gao Peng Xu ... Yafei Wen Xiaoxin Chen Xiangyu Yue Hongsheng Li Yu Qiao MLLM 54 117 0 07 Sep 2023
Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models Yue Zhang Yafu Li Leyang Cui Deng Cai Lemao Liu ... Longyue Wang Anh Tuan Luu Wei Bi Freda Shi Shuming Shi RALM LRM HILM 53 523 0 03 Sep 2023
Sparkles: Unlocking Chats Across Multiple Images for Multimodal Instruction-Following Models Yupan Huang Zaiqiao Meng Fangyu Liu Yixuan Su Nigel Collier Yutong Lu MLLM 41 22 0 31 Aug 2023