MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models

20 April 2023

Papers citing "MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models"

50 / 381 papers shown

Title
GLOV: Guided Large Language Models as Implicit Optimizers for Vision Language Models Muhammad Jehanzeb Mirza Mengjie Zhao Zhuoyuan Mao Sivan Doveh Wei Lin ... Yuki Mitsufuji Horst Possegger Rogerio Feris Leonid Karlinsky James Glass VLM 84 1 0 08 Oct 2024
TRACE: Temporal Grounding Video LLM via Causal Event Modeling Yongxin Guo Jingyu Liu Mingda Li Xiaoying Tang Qingbin Liu Xiaoying Tang 42 14 0 08 Oct 2024
Geometric Analysis of Reasoning Trajectories: A Phase Space Approach to Understanding Valid and Invalid Multi-Hop Reasoning in LLMs Javier Marin LRM 85 0 0 06 Oct 2024
Structural-Entropy-Based Sample Selection for Efficient and Effective Learning Tianchi Xie Jiangning Zhu Guozu Ma Minzhi Lin Wei Chen Weikai Yang Shixia Liu 30 0 0 03 Oct 2024
Enhancing Screen Time Identification in Children with a Multi-View Vision Language Model and Screen Time Tracker Xinlong Hou Sen Shen Xueshen Li Xinran Gao Ziyi Huang Steven J. Holiday Matthew R. Cribbet Susan W. White Edward Sazonov Yu Gan 36 0 0 02 Oct 2024
Scaling Offline Model-Based RL via Jointly-Optimized World-Action Model Pretraining Jie Cheng Ruixi Qiao Gang Xiong Binhua Li Yingwei Ma Binhua Li Yongbin Li Yisheng Lv OffRL OnRL LM&Ro 50 3 0 01 Oct 2024
Image-guided topic modeling for interpretable privacy classification Alina Elena Baia Andrea Cavallaro 42 0 0 27 Sep 2024
VLMine: Long-Tail Data Mining with Vision Language Models Mao Ye Gregory P. Meyer Zaiwei Zhang Dennis Park Siva Karthik Mustikovela Yuning Chai Eric M. Wolff VLM 33 1 0 23 Sep 2024
JourneyBench: A Challenging One-Stop Vision-Language Understanding Benchmark of Generated Images Zhecan Wang Junzhang Liu Chia-Wei Tang Hani Alomari Anushka Sivakumar ... Haoxuan You A. Ishmam Kai-Wei Chang Shih-Fu Chang Chris Thomas CoGe VLM 66 2 0 19 Sep 2024
TinyVLA: Towards Fast, Data-Efficient Vision-Language-Action Models for Robotic Manipulation Junjie Wen Yichen Zhu Jinming Li Minjie Zhu Kun Wu ... Ran Cheng Chaomin Shen Yaxin Peng Feifei Feng Jian Tang LM&Ro 74 48 0 19 Sep 2024
One missing piece in Vision and Language: A Survey on Comics Understanding Emanuele Vivoli Andrey Barsky Mohamed Ali Souibgui Artemis LLabres Marco Bertini Dimosthenis Karatzas 42 3 0 14 Sep 2024
Securing Vision-Language Models with a Robust Encoder Against Jailbreak and Adversarial Attacks Md Zarif Hossain Ahmed Imteaj AAML VLM 48 3 0 11 Sep 2024
Deep Computer Vision for Solar Physics Big Data: Opportunities and Challenges Bo Shen Marco Marena Chenyang Li Qin Li Haodi Jiang Mengnan Du Jiajun Xu Haimin Wang 33 0 0 07 Sep 2024
SAM4MLLM: Enhance Multi-Modal Large Language Model for Referring Expression Segmentation Yi-Chia Chen Wei-Hua Li Cheng Sun Yu-Chiang Frank Wang Chu-Song Chen VLM 39 11 0 01 Sep 2024
StimuVAR: Spatiotemporal Stimuli-aware Video Affective Reasoning with Multimodal Large Language Models Y. Guo Faizan Siddiqui Yang Zhao Rama Chellappa Shao-Yuan Lo LRM 44 2 0 31 Aug 2024
Eagle: Exploring The Design Space for Multimodal LLMs with Mixture of Encoders Min Shi Fuxiao Liu Shihao Wang Shijia Liao Subhashree Radhakrishnan ... Andrew Tao Andrew Tao Zhiding Yu Guilin Liu Guilin Liu MLLM 33 53 0 28 Aug 2024
RSTeller: Scaling Up Visual Language Modeling in Remote Sensing with Rich Linguistic Semantics from Openly Available Data and Large Language Models Junyao Ge Yang Zheng Kaitai Guo Jimin Liang Jimin Liang 38 1 0 27 Aug 2024
K-Sort Arena: Efficient and Reliable Benchmarking for Generative Models via K-wise Human Preferences Zhikai Li Xuewen Liu Dongrong Fu Jianquan Li Qingyi Gu Kurt Keutzer Zhen Dong EGVM VGen DiffM 92 1 0 26 Aug 2024
LLM-3D Print: Large Language Models To Monitor and Control 3D Printing Yayati Jadhav P. Pak Amir Barati Farimani AI4CE 86 8 0 26 Aug 2024
EE-MLLM: A Data-Efficient and Compute-Efficient Multimodal Large Language Model Feipeng Ma Yizhou Zhou Hebei Li Zilong He Siying Wu Fengyun Rao Siying Wu Fengyun Rao Yueyi Zhang Xiaoyan Sun 39 3 0 21 Aug 2024
Visual Agents as Fast and Slow Thinkers Guangyan Sun Mingyu Jin Zhenting Wang Cheng-Long Wang Siqi Ma Qifan Wang Ying Nian Wu Ying Nian Wu Dongfang Liu Dongfang Liu LLMAG LRM 79 13 0 16 Aug 2024
Connecting Dreams with Visual Brainstorming Instruction Yasheng Sun Bohan Li Mingchen Zhuge Deng-Ping Fan Salman Khan Fahad Shahbaz Khan Hideki Koike DiffM 42 0 0 14 Aug 2024
LLMServingSim: A HW/SW Co-Simulation Infrastructure for LLM Inference Serving at Scale Jaehong Cho Minsu Kim Hyunmin Choi Guseul Heo Jongse Park 38 9 0 10 Aug 2024
AdvQDet: Detecting Query-Based Adversarial Attacks with Adversarial Contrastive Prompt Tuning Xin Wang Kai-xiang Chen Xingjun Ma Zhineng Chen Jingjing Chen Yu-Gang Jiang AAML 43 4 0 04 Aug 2024
Self-Introspective Decoding: Alleviating Hallucinations for Large Vision-Language Models Fushuo Huo Wenchao Xu Zhong Zhang Yining Qi Zhicheng Chen Peilin Zhao VLM MLLM 66 19 0 04 Aug 2024
Learning Video Context as Interleaved Multimodal Sequences S. Shao Pengchuan Zhang Y. Li Xide Xia A. Meso Ziteng Gao Jinheng Xie N. Holliman Mike Zheng Shou 46 5 0 31 Jul 2024
Prompting Medical Large Vision-Language Models to Diagnose Pathologies by Visual Question Answering Danfeng Guo Sumitaka Honji LRM 70 0 0 31 Jul 2024
HaloQuest: A Visual Hallucination Dataset for Advancing Multimodal Reasoning Zhecan Wang Garrett Bingham Adams Wei Yu Quoc V. Le Thang Luong Golnaz Ghiasi MLLM LRM 45 9 0 22 Jul 2024
ViLLa: Video Reasoning Segmentation with Large Language Model Rongkun Zheng Lu Qi Xi Chen Yi Wang Kun Wang Yu Qiao Hengshuang Zhao VOS LRM 77 2 0 18 Jul 2024
Turbo: Informativity-Driven Acceleration Plug-In for Vision-Language Large Models Chen Ju Haicheng Wang Haozhe Cheng Xu Chen Zhonghua Zhai Weilin Huang Jinsong Lan Shuai Xiao Bo Zheng VLM 49 5 0 16 Jul 2024
Controllable Contextualized Image Captioning: Directing the Visual Narrative through User-Defined Highlights Shunqi Mao Chaoyi Zhang Hang Su Hwanjun Song Igor Shalyminov Weidong Cai 39 1 0 16 Jul 2024
Reflective Instruction Tuning: Mitigating Hallucinations in Large Vision-Language Models Jinrui Zhang Teng Wang Haigang Zhang Ping Lu Feng Zheng MLLM LRM VLM 34 3 0 16 Jul 2024
Large Visual-Language Models Are Also Good Classifiers: A Study of In-Context Multimodal Fake News Detection Ye Jiang Yimin Wang MLLM 38 1 0 16 Jul 2024
FabGPT: An Efficient Large Multimodal Model for Complex Wafer Defect Knowledge Queries Yuqi Jiang Xudong Lu Qian Jin Qi Sun Hanming Wu Cheng Zhuo 39 5 0 15 Jul 2024
Bootstrapping Vision-language Models for Self-supervised Remote Physiological Measurement Zijie Yue Miaojing Shi Hanli Wang Shuai Ding Qijun Chen Shanlin Yang 42 0 0 11 Jul 2024
MIA-Bench: Towards Better Instruction Following Evaluation of Multimodal LLMs Yusu Qian Hanrong Ye J. Fauconnier Peter Grasch Yinfei Yang Zhe Gan 108 13 0 01 Jul 2024
MG-LLaVA: Towards Multi-Granularity Visual Instruction Tuning Xiangyu Zhao Xiangtai Li Haodong Duan Haian Huang Yining Li Kai Chen Hua Yang VLM MLLM 45 10 0 25 Jun 2024
DaLPSR: Leverage Degradation-Aligned Language Prompt for Real-World Image Super-Resolution Aiwen Jiang Zhi Wei Long Peng Feiqiang Liu Wenbo Li Mingwen Wang DiffM 54 2 0 24 Jun 2024
VoCo-LLaMA: Towards Vision Compression with Large Language Models Xubing Ye Yukang Gan Xiaoke Huang Yixiao Ge Yansong Tang MLLM VLM 43 23 0 18 Jun 2024
SPA-VL: A Comprehensive Safety Preference Alignment Dataset for Vision Language Model Yongting Zhang Lu Chen Guodong Zheng Yifeng Gao Rui Zheng ... Yu Qiao Xuanjing Huang Feng Zhao Tao Gui Jing Shao VLM 85 24 0 17 Jun 2024
VideoGPT+: Integrating Image and Video Encoders for Enhanced Video Understanding Muhammad Maaz H. Rasheed Salman Khan Fahad A Khan VLM MLLM 40 51 0 13 Jun 2024
MMFakeBench: A Mixed-Source Multimodal Misinformation Detection Benchmark for LVLMs Xuannan Liu Zekun Li Peipei Li Shuhan Xia Xing Cui Linzhi Huang Huaibo Huang Weihong Deng Zhaofeng He 47 14 0 13 Jun 2024
VLind-Bench: Measuring Language Priors in Large Vision-Language Models Kang-il Lee Minbeom Kim Seunghyun Yoon Minsung Kim Dongryeol Lee Hyukhun Koh Kyomin Jung CoGe VLM 92 5 0 13 Jun 2024
Grounding Multimodal Large Language Models in Actions Andrew Szot Bogdan Mazoure Harsh Agrawal Devon Hjelm Z. Kira Alexander Toshev LM&Ro 35 10 0 12 Jun 2024
MLLMGuard: A Multi-dimensional Safety Evaluation Suite for Multimodal Large Language Models Tianle Gu Zeyang Zhou Kexin Huang Dandan Liang Yixu Wang ... Keqing Wang Yujiu Yang Yan Teng Yu Qiao Yingchun Wang ELM 50 13 0 11 Jun 2024
Can I understand what I create? Self-Knowledge Evaluation of Large Language Models Zhiquan Tan Lai Wei Jindong Wang Xing Xie Weiran Huang ELM LRM 35 5 0 10 Jun 2024
M3GIA: A Cognition Inspired Multilingual and Multimodal General Intelligence Ability Benchmark Wei Song Yadong Li Jianhua Xu Guowei Wu Lingfeng Ming ... Weihua Luo Houyi Li Yi Du Fangda Guo Kaicheng Yu ELM LRM 39 7 0 08 Jun 2024
One Perturbation is Enough: On Generating Universal Adversarial Perturbations against Vision-Language Pre-training Models Hao Fang Jiawei Kong Wenbo Yu Bin Chen Jiawei Li Hao Wu Ke Xu Ke Xu AAML VLM 40 13 0 08 Jun 2024
InstructNav: Zero-shot System for Generic Instruction Navigation in Unexplored Environment Yuxing Long Wenzhe Cai Hongcheng Wang Guanqi Zhan Hao Dong 35 23 0 07 Jun 2024
3D-GRAND: A Million-Scale Dataset for 3D-LLMs with Better Grounding and Less Hallucination Jianing Yang Xuweiyi Chen Nikhil Madaan Madhavan Iyengar Shengyi Qian David Fouhey Joyce Chai 3DV 78 11 0 07 Jun 2024