ShareGPT4V: Improving Large Multi-Modal Models with Better Captions

21 November 2023

Conghui He

Dahua Lin

Papers citing "ShareGPT4V: Improving Large Multi-Modal Models with Better Captions"

50 / 471 papers shown

Title
Calibrated Self-Rewarding Vision Language Models Yiyang Zhou Zhiyuan Fan Dongjie Cheng Sihan Yang Zhaorun Chen Chenhang Cui Xiyao Wang Yun Li Linjun Zhang Huaxiu Yao VLM 79 27 0 23 May 2024
Unveiling the Tapestry of Consistency in Large Vision-Language Models Yuan Zhang Fei Xiao Tao Huang Chun-Kai Fan Hongyuan Dong Jiawen Li Jiacong Wang Kuan Cheng Shanghang Zhang Haoyuan Guo 42 7 0 23 May 2024
AlignGPT: Multi-modal Large Language Models with Adaptive Alignment Capability Fei Zhao Taotian Pang Chunhui Li Zhen Wu Junjie Guo Shangyu Xing Xinyu Dai 55 7 0 23 May 2024
Dense Connector for MLLMs Huanjin Yao Wenhao Wu Taojiannan Yang Yuxin Song Mengxi Zhang Haocheng Feng Yifan Sun Zhiheng Li Wanli Ouyang Jingdong Wang MLLM VLM 42 18 0 22 May 2024
Safety Alignment for Vision Language Models Zhendong Liu Yuanbi Nie Yingshui Tan Xiangyu Yue Qiushi Cui Chongjun Wang Xiaoyong Zhu Bo Zheng VLM MLLM 98 7 0 22 May 2024
Imp: Highly Capable Large Multimodal Models for Mobile Devices Zhenwei Shao Zhou Yu Jun Yu Xuecheng Ouyang Lihao Zheng Zhenbiao Gai Mingyang Wang Jiajun Ding 23 10 0 20 May 2024
Rethinking Overlooked Aspects in Vision-Language Models Yuan Liu Le Tian Xiao Zhou Jie Zhou VLM 34 2 0 20 May 2024
Automated Multi-level Preference for MLLMs Mengxi Zhang Wenhao Wu Yu Lu Yuxin Song Kang Rong ... Jianbo Zhao Fanglong Liu Yifan Sun Haocheng Feng Jingdong Wang MLLM 83 10 0 18 May 2024
Efficient Multimodal Large Language Models: A Survey Yizhang Jin Jian Li Yexin Liu Tianjun Gu Kai Wu ... Xin Tan Zhenye Gan Yabiao Wang Chengjie Wang Lizhuang Ma LRM 47 47 0 17 May 2024
Xmodel-VLM: A Simple Baseline for Multimodal Vision Language Model Wanting Xu Yang Liu Langping He Xucheng Huang Ling Jiang VLM MLLM 43 2 0 15 May 2024
CuMo: Scaling Multimodal LLM with Co-Upcycled Mixture-of-Experts Jiachen Li Xinyao Wang Sijie Zhu Chia-Wen Kuo Lu Xu Fan Chen Jitesh Jain Humphrey Shi Longyin Wen MLLM MoE 46 29 0 09 May 2024
What matters when building vision-language models? Hugo Laurençon Léo Tronchon Matthieu Cord Victor Sanh VLM 43 157 0 03 May 2024
MANTIS: Interleaved Multi-Image Instruction Tuning Dongfu Jiang Xuan He Huaye Zeng Cong Wei Max W.F. Ku Qian Liu Wenhu Chen VLM MLLM 33 104 0 02 May 2024
MiniGPT-3D: Efficiently Aligning 3D Point Clouds with Large Language Models using 2D Priors Yuan Tang Xu Han Xianzhi Li Qiao Yu Yixue Hao Long Hu Min Chen 37 14 0 02 May 2024
How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suites Zhe Chen Weiyun Wang Hao Tian Shenglong Ye Zhangwei Gao ... Tong Lu Dahua Lin Yu Qiao Jifeng Dai Wenhai Wang MLLM VLM 71 546 0 25 Apr 2024
List Items One by One: A New Data Source and Learning Paradigm for Multimodal LLMs An Yan Zhengyuan Yang Junda Wu Wanrong Zhu Jianwei Yang ... K. Lin Jianfeng Wang Julian McAuley Jianfeng Gao Lijuan Wang LRM 39 12 0 25 Apr 2024
ID-Animator: Zero-Shot Identity-Preserving Human Video Generation Xuanhua He Quande Liu Shengju Qian Xin Eric Wang Tao Hu Ke Cao K. Yan Jie Zhang VGen 39 40 0 23 Apr 2024
FINEMATCH: Aspect-based Fine-grained Image and Text Mismatch Detection and Correction Hang Hua Jing Shi Kushal Kafle Simon Jenni Daoan Zhang John Collomosse Scott D. Cohen Jiebo Luo CoGe VLM 50 9 0 23 Apr 2024
Graphic Design with Large Multimodal Model Yutao Cheng Zhao Zhang Maoke Yang Hui Nie Chunyuan Li Xinglong Wu Jie Shao 54 10 0 22 Apr 2024
UrbanCross: Enhancing Satellite Image-Text Retrieval with Cross-Domain Adaptation Siru Zhong Xixuan Hao Yibo Yan Ying Zhang Yangqiu Song Keli Zhang 48 8 0 22 Apr 2024
SEED-X: Multimodal Models with Unified Multi-granularity Comprehension and Generation Yuying Ge Sijie Zhao Jinguo Zhu Yixiao Ge Kun Yi Lin Song Chen Li Xiaohan Ding Ying Shan VLM 70 112 0 22 Apr 2024
MoVA: Adapting Mixture of Vision Experts to Multimodal Context Zhuofan Zong Bingqi Ma Dazhong Shen Guanglu Song Hao Shao Dongzhi Jiang Hongsheng Li Yu Liu MoE 50 42 0 19 Apr 2024
Groma: Localized Visual Tokenization for Grounding Multimodal Large Language Models Chuofan Ma Yi-Xin Jiang Jiannan Wu Zehuan Yuan Xiaojuan Qi VLM ObjD 37 53 0 19 Apr 2024
TextSquare: Scaling up Text-Centric Visual Instruction Tuning Jingqun Tang Chunhui Lin Zhen Zhao Shubo Wei Binghong Wu ... Yuliang Liu Hao Liu Yuan Xie Xiang Bai Can Huang LRM VLM MLLM 76 30 0 19 Apr 2024
BLINK: Multimodal Large Language Models Can See but Not Perceive Xingyu Fu Yushi Hu Bangzheng Li Yu Feng Haoyu Wang Xudong Lin Dan Roth Noah A. Smith Wei-Chiu Ma Ranjay Krishna VLM LRM MLLM 45 114 0 18 Apr 2024
Self-Supervised Visual Preference Alignment Ke Zhu Liang Zhao Zheng Ge Xiangyu Zhang 40 12 0 16 Apr 2024
TextCoT: Zoom In for Enhanced Multimodal Text-Rich Image Understanding Bozhi Luan Hao Feng Hong Chen Yonghui Wang Wen-gang Zhou Houqiang Li MLLM 37 11 0 15 Apr 2024
AesExpert: Towards Multi-modality Foundation Model for Image Aesthetics Perception Yipo Huang Xiangfei Sheng Zhichao Yang Quan Yuan Zhichao Duan Pengfei Chen Leida Li Weisi Lin Guangming Shi 42 23 0 15 Apr 2024
UNIAA: A Unified Multi-modal Image Aesthetic Assessment Baseline and Benchmark Zhaokun Zhou Qiulin Wang Bin Lin Yiwei Su Renqi Chen Xin Tao Amin Zheng Li-xin Yuan Pengfei Wan Di Zhang 34 8 0 15 Apr 2024
TextHawk: Exploring Efficient Fine-Grained Perception of Multimodal Large Language Models Ya-Qi Yu Minghui Liao Jihao Wu Yongxin Liao Xiaoyu Zheng Wei Zeng VLM 29 15 0 14 Apr 2024
Ferret-v2: An Improved Baseline for Referring and Grounding with Large Language Models Haotian Zhang Haoxuan You Philipp Dufter Bowen Zhang Chen Chen ... Tsu-Jui Fu William Y. Wang Shih-Fu Chang Zhe Gan Yinfei Yang ObjD MLLM 104 45 0 11 Apr 2024
InternLM-XComposer2-4KHD: A Pioneering Large Vision-Language Model Handling Resolutions from 336 Pixels to 4K HD Xiao-wen Dong Pan Zhang Yuhang Zang Yuhang Cao Bin Wang ... Xingcheng Zhang Jifeng Dai Yuxin Qiao Dahua Lin Jiaqi Wang VLM MLLM 44 114 0 09 Apr 2024
X-VARS: Introducing Explainability in Football Refereeing with Multi-Modal Large Language Model Jan Held Hani Itani A. Cioppa Silvio Giancola Guohao Li Marc Van Droogenbroeck 48 17 0 07 Apr 2024
Joint Visual and Text Prompting for Improved Object-Centric Perception with Multimodal Large Language Models Songtao Jiang Yan Zhang Chenyi Zhou Yeying Jin Yang Feng Jian Wu Zuozhu Liu LRM VLM 50 4 0 06 Apr 2024
CoMat: Aligning Text-to-Image Diffusion Model with Image-to-Text Concept Matching Dongzhi Jiang Guanglu Song Xiaoshi Wu Renrui Zhang Dazhong Shen Zhuofan Zong Yu Liu Hongsheng Li VLM 43 20 0 04 Apr 2024
No "Zero-Shot" Without Exponential Data: Pretraining Concept Frequency Determines Multimodal Model Performance Vishaal Udandarao Ameya Prabhu Adhiraj Ghosh Yash Sharma Philip Torr Adel Bibi Samuel Albanie Matthias Bethge VLM 128 45 0 04 Apr 2024
Sparse Concept Bottleneck Models: Gumbel Tricks in Contrastive Learning Andrei Semenov Vladimir Ivanov Aleksandr Beznosikov Alexander Gasnikov 42 6 0 04 Apr 2024
mChartQA: A universal benchmark for multimodal Chart Question Answer based on Vision-Language Alignment and Reasoning Jingxuan Wei Nan Xu Guiyong Chang Yin Luo Bihui Yu Ruifeng Guo 46 2 0 02 Apr 2024
Are We on the Right Way for Evaluating Large Vision-Language Models? Lin Chen Jinsong Li Xiao-wen Dong Pan Zhang Yuhang Zang ... Haodong Duan Jiaqi Wang Yu Qiao Dahua Lin Feng Zhao VLM 83 227 0 29 Mar 2024
Draw-and-Understand: Leveraging Visual Prompts to Enable MLLMs to Comprehend What You Want Weifeng Lin Xinyu Wei Ruichuan An Peng Gao Bocheng Zou Yulin Luo Siyuan Huang Shanghang Zhang Hongsheng Li VLM 71 33 0 29 Mar 2024
RH20T-P: A Primitive-Level Robotic Dataset Towards Composable Generalization Agents Zeren Chen Zhelun Shi Xiaoya Lu Lehan He Sucheng Qian ... Zhen-fei Yin Jing Shao Jing Shao Cewu Lu Cewu Lu 43 5 0 28 Mar 2024
Mini-Gemini: Mining the Potential of Multi-modality Vision Language Models Yanwei Li Yuechen Zhang Chengyao Wang Zhisheng Zhong Yixin Chen Ruihang Chu Shaoteng Liu Jiaya Jia VLM MLLM MoE 45 215 0 27 Mar 2024
Beyond Embeddings: The Promise of Visual Table in Visual Reasoning Yiwu Zhong Zi-Yuan Hu Michael R. Lyu Liwei Wang 29 1 0 27 Mar 2024
DreamLIP: Language-Image Pre-training with Long Captions Kecheng Zheng Yifei Zhang Wei Wu Fan Lu Shuailei Ma Xin Jin Wei Chen Yujun Shen VLM CLIP 47 26 0 25 Mar 2024
Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoning Hao Shao Shengju Qian Han Xiao Guanglu Song Zhuofan Zong Letian Wang Yu Liu Hongsheng Li VGen LRM MLLM 71 43 0 25 Mar 2024
UrbanVLP: Multi-Granularity Vision-Language Pretraining for Urban Socioeconomic Indicator Prediction Xixuan Hao Wei Chen Yibo Yan Siru Zhong Kun Wang Qingsong Wen Keli Zhang VLM 84 0 0 25 Mar 2024
Long-CLIP: Unlocking the Long-Text Capability of CLIP Beichen Zhang Pan Zhang Xiao-wen Dong Yuhang Zang Jiaqi Wang CLIP VLM 45 110 0 22 Mar 2024
MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems? Renrui Zhang Dongzhi Jiang Yichi Zhang Haokun Lin Ziyu Guo ... Aojun Zhou Pan Lu Kai-Wei Chang Peng Gao Hongsheng Li 34 173 0 21 Mar 2024
Cobra: Extending Mamba to Multi-Modal Large Language Model for Efficient Inference Han Zhao Min Zhang Wei Zhao Pengxiang Ding Siteng Huang Donglin Wang Mamba 54 69 0 21 Mar 2024
Pensieve: Retrospect-then-Compare Mitigates Visual Hallucination Dingchen Yang Bowen Cao Guang Chen Changjun Jiang 53 7 0 21 Mar 2024