MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models

20 April 2023

Papers citing "MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models"

50 / 381 papers shown

Title
On the Essence and Prospect: An Investigation of Alignment Approaches for Big Models Xinpeng Wang Shitong Duan Xiaoyuan Yi Jing Yao Shanlin Zhou Zhihua Wei Peng Zhang Dongkuan Xu Maosong Sun Xing Xie OffRL 41 16 0 07 Mar 2024
CoTBal: Comprehensive Task Balancing for Multi-Task Visual Instruction Tuning Yanqi Dai Dong Jing Nanyi Fei Zhiwu Lu Nanyi Fei Guoxing Yang Zhiwu Lu 55 3 0 07 Mar 2024
Beyond Specialization: Assessing the Capabilities of MLLMs in Age and Gender Estimation Maksim Kuprashevich Grigorii Alekseenko Irina Tolstykh ELM 56 4 0 04 Mar 2024
Exploring the Potential of Large Language Models for Improving Digital Forensic Investigation Efficiency Akila Wickramasekara F. Breitinger Mark Scanlon 52 8 0 29 Feb 2024
Unveiling Typographic Deceptions: Insights of the Typographic Vulnerability in Large Vision-Language Model Hao-Ran Cheng Erjia Xiao Jindong Gu Le Yang Jinhao Duan Jize Zhang Jiahang Cao Kaidi Xu Renjing Xu 37 6 0 29 Feb 2024
From Summary to Action: Enhancing Large Language Models for Complex Tasks with Open World APIs Yulong Liu Yunlong Yuan Chunwei Wang Jianhua Han Yongqiang Ma Li Zhang Nanning Zheng Hang Xu LLMAG 39 5 0 28 Feb 2024
A Cognitive Evaluation Benchmark of Image Reasoning and Description for Large Vision-Language Models Xiujie Song Mengyue Wu Ke Zhu Chunhao Zhang Yanyi Chen LRM ELM 36 3 0 28 Feb 2024
PALO: A Polyglot Large Multimodal Model for 5B People Muhammad Maaz H. Rasheed Abdelrahman M. Shaker Salman Khan Hisham Cholakal Rao M. Anwer Timothy Baldwin M. Felsberg Fahad S. Khan VLM LRM 85 13 0 22 Feb 2024
Visual Hallucinations of Multi-modal Large Language Models Wen Huang Hongbin Liu Minxin Guo Neil Zhenqiang Gong MLLM VLM 32 24 0 22 Feb 2024
LLMBind: A Unified Modality-Task Integration Framework Bin Zhu Munan Ning Peng Jin Bin Lin Jinfa Huang ... Junwu Zhang Zhenyu Tang Mingjun Pan Xing Zhou Li-ming Yuan MLLM 40 6 0 22 Feb 2024
VL-Trojan: Multimodal Instruction Backdoor Attacks against Autoregressive Visual Language Models Jiawei Liang Siyuan Liang Man Luo Aishan Liu Dongchen Han Ee-Chien Chang Xiaochun Cao 42 38 0 21 Feb 2024
SInViG: A Self-Evolving Interactive Visual Agent for Human-Robot Interaction Jie Xu Hanbo Zhang Xinghang Li Huaping Liu Xuguang Lan Tao Kong LM&Ro 38 3 0 19 Feb 2024
RAG-Driver: Generalisable Driving Explanations with Retrieval-Augmented In-Context Learning in Multi-Modal Large Language Model Jianhao Yuan Shuyang Sun Daniel Omeiza Bo Zhao Paul Newman Lars Kunze Matthew Gadd LRM 36 49 0 16 Feb 2024
ProtChatGPT: Towards Understanding Proteins with Large Language Models Chao Wang Hehe Fan Ruijie Quan Yi Yang 26 13 0 15 Feb 2024
World Model on Million-Length Video And Language With Blockwise RingAttention Hao Liu Wilson Yan Matei A. Zaharia Pieter Abbeel VGen 31 62 0 13 Feb 2024
WebLINX: Real-World Website Navigation with Multi-Turn Dialogue Xing Han Lù Zdeněk Kasner Siva Reddy 34 60 0 08 Feb 2024
CREMA: Generalizable and Efficient Video-Language Reasoning via Multimodal Modular Fusion Shoubin Yu Jaehong Yoon Mohit Bansal 82 4 0 08 Feb 2024
SPHINX-X: Scaling Data and Parameters for a Family of Multi-modal Large Language Models Chris Liu Renrui Zhang Longtian Qiu Siyuan Huang Weifeng Lin ... Hao Shao Pan Lu Hongsheng Li Yu Qiao Peng Gao MLLM 130 109 0 08 Feb 2024
PVLR: Prompt-driven Visual-Linguistic Representation Learning for Multi-Label Image Recognition Hao Tan Zichang Tan Jun Li Jun Wan Zhen Lei VLM 32 0 0 31 Jan 2024
LLaVA-MoLE: Sparse Mixture of LoRA Experts for Mitigating Data Conflicts in Instruction Finetuning MLLMs Shaoxiang Chen Zequn Jie Lin Ma MoE 45 47 0 29 Jan 2024
Mobile-Agent: Autonomous Multi-Modal Mobile Device Agent with Visual Perception Junyang Wang Haiyang Xu Jiabo Ye Mingshi Yan Weizhou Shen Ji Zhang Fei Huang Jitao Sang 47 109 0 29 Jan 2024
MLLMReID: Multimodal Large Language Model-based Person Re-identification Shan Yang Yongfei Zhang LRM 32 2 0 24 Jan 2024
MLLM-Tool: A Multimodal Large Language Model For Tool Agent Learning Chenyu Wang Weixin Luo Qianyu Chen Haonan Mai Jindi Guo Sixun Dong Xiaohua Xuan MLLM LLMAG 52 19 0 19 Jan 2024
Temporal Insight Enhancement: Mitigating Temporal Hallucination in Multimodal Large Language Models Li Sun Liuan Wang Jun Sun Takayuki Okatani MLLM 19 0 0 18 Jan 2024
COCO is "ALL'' You Need for Visual Instruction Fine-tuning Xiaotian Han Yiqi Wang Bohan Zhai Quanzeng You Hongxia Yang VLM MLLM 33 2 0 17 Jan 2024
DoraemonGPT: Toward Understanding Dynamic Scenes with Large Language Models (Exemplified as A Video Agent) Zongxin Yang Guikun Chen Xiaodi Li Wenguan Wang Yi Yang LM&Ro LLMAG 69 35 0 16 Jan 2024
Towards A Better Metric for Text-to-Video Generation Jay Zhangjie Wu Guian Fang Haoning Wu Xintao Wang Yixiao Ge ... Rui Zhao Weisi Lin Wynne Hsu Ying Shan Mike Zheng Shou VGen 37 34 0 15 Jan 2024
Seeing the Unseen: Visual Common Sense for Semantic Placement Ram Ramrakhya Aniruddha Kembhavi Dhruv Batra Z. Kira Kuo-Hao Zeng Luca Weihs VLM 41 5 0 15 Jan 2024
AccidentGPT: Large Multi-Modal Foundation Model for Traffic Accident Analysis Kebin Wu Wenbin Li Xiaofei Xiao 21 3 0 05 Jan 2024
ChartAssisstant: A Universal Chart Multimodal Language Model via Chart-to-Table Pre-training and Multitask Instruction Tuning Fanqing Meng Wenqi Shao Quanfeng Lu Peng Gao Kaipeng Zhang Yu Qiao Ping Luo 31 46 0 04 Jan 2024
LISA++: An Improved Baseline for Reasoning Segmentation with Large Language Model Senqiao Yang Tianyuan Qu Xin Lai Zhuotao Tian Bohao Peng Shu Liu Jiaya Jia VLM 21 28 0 28 Dec 2023
LLM-SAP: Large Language Models Situational Awareness Based Planning Liman Wang Hanyang Zhong LLMAG 35 2 0 26 Dec 2023
IQAGPT: Image Quality Assessment with Vision-language and ChatGPT Models Zhihao Chen Bin Hu Chuang Niu Tao Chen Yuxin Li Hongming Shan Ge Wang LM&MA MLLM 24 4 0 25 Dec 2023
LiDAR-LLM: Exploring the Potential of Large Language Models for 3D LiDAR Understanding Senqiao Yang Jiaming Liu Ray Zhang Mingjie Pan Zoey Guo Xiaoqi Li Zehui Chen Peng Gao Yandong Guo Shanghang Zhang 3DV 26 58 0 21 Dec 2023
Jack of All Tasks, Master of Many: Designing General-purpose Coarse-to-Fine Vision-Language Model Shraman Pramanick Guangxing Han Rui Hou Sayan Nag Ser-Nam Lim Nicolas Ballas Qifan Wang Rama Chellappa Amjad Almahairi VLM MLLM 48 29 0 19 Dec 2023
One-Shot Learning as Instruction Data Prospector for Large Language Models Yunshui Li Binyuan Hui Xiaobo Xia Jiaxi Yang Min Yang ... Ling-Hao Chen Junhao Liu Tongliang Liu Fei Huang Yongbin Li 38 31 0 16 Dec 2023
CLIPSyntel: CLIP and LLM Synergy for Multimodal Question Summarization in Healthcare Akash Ghosh Arkadeep Acharya Raghav Jain Sriparna Saha Aman Chadha Setu Sinha 27 29 0 16 Dec 2023
DriveMLM: Aligning Multi-Modal Large Language Models with Behavioral Planning States for Autonomous Driving Wenhai Wang Jiangwei Xie ChuanYang Hu Haoming Zou Jianan Fan ... Lewei Lu Xizhou Zhu Xiaogang Wang Yu Qiao Jifeng Dai 36 125 0 14 Dec 2023
Depicting Beyond Scores: Advancing Image Quality Assessment through Multi-modal Language Models Zhiyuan You Zheyuan Li Jinjin Gu Zhenfei Yin Tianfan Xue Chao Dong EGVM 26 35 0 14 Dec 2023
LMDrive: Closed-Loop End-to-End Driving with Large Language Models Hao Shao Yuxuan Hu Letian Wang Steven L. Waslander Yu Liu Hongsheng Li ELM 33 113 0 12 Dec 2023
MP5: A Multi-modal Open-ended Embodied System in Minecraft via Active Perception Yiran Qin Enshen Zhou Qichang Liu Zhen-fei Yin Lu Sheng Ruimao Zhang Yu Qiao Jing Shao LM&Ro 32 39 0 12 Dec 2023
Audio-Visual LLM for Video Understanding Fangxun Shu Lei Zhang Hao Jiang Cihang Xie VLM MLLM 27 38 0 11 Dec 2023
Quilt-LLaVA: Visual Instruction Tuning by Extracting Localized Narratives from Open-Source Histopathology Videos M. S. Seyfioglu Wisdom O. Ikezogwo Fatemeh Ghezloo Ranjay Krishna Linda G. Shapiro 30 36 0 07 Dec 2023
Digital Life Project: Autonomous 3D Characters with Social Intelligence Zhongang Cai Jian-Dong Jiang Zhongfei Qing Xinying Guo Mingyuan Zhang ... Yukun Wei Xiaogang Wang Chen Change Loy Lei Yang Ziwei Liu VGen 36 24 0 07 Dec 2023
VRPTEST: Evaluating Visual Referring Prompting in Large Multimodal Models Zongjie Li Chaozheng Wang Chaowei Liu Pingchuan Ma Daoyuan Wu Shuai Wang Cuiyun Gao VLM 29 6 0 07 Dec 2023
Semantics-aware Motion Retargeting with Vision-Language Models Haodong Zhang Zhike Chen Haocheng Xu Lei Hao Xiaofei Wu Songcen Xu Zhensong Zhang Yue Wang Rong Xiong 35 4 0 04 Dec 2023
InstructTA: Instruction-Tuned Targeted Attack for Large Vision-Language Models Xunguang Wang Zhenlan Ji Pingchuan Ma Zongjie Li Shuai Wang MLLM 43 11 0 04 Dec 2023
StoryGPT-V: Large Language Models as Consistent Story Visualizers Xiaoqian Shen Mohamed Elhoseiny VLM 101 10 0 04 Dec 2023
Language-driven All-in-one Adverse Weather Removal Hao Yang Liyuan Pan Yan Yang Wei Liang VLM KELM 29 18 0 03 Dec 2023
Dolphins: Multimodal Language Model for Driving Yingzi Ma Yulong Cao Jiachen Sun Marco Pavone Chaowei Xiao MLLM 38 50 0 01 Dec 2023