ShareGPT4V: Improving Large Multi-Modal Models with Better Captions

21 November 2023

Conghui He

Dahua Lin

Papers citing "ShareGPT4V: Improving Large Multi-Modal Models with Better Captions"

50 / 471 papers shown

Title
VITA: Towards Open-Source Interactive Omni Multimodal LLM Chaoyou Fu Haojia Lin Zuwei Long Yunhang Shen Meng Zhao ... Ran He Rongrong Ji Yunsheng Wu Caifeng Shan Xing Sun MLLM 50 2 0 09 Aug 2024
LLaVA-OneVision: Easy Visual Task Transfer Bo Li Yuanhan Zhang Dong Guo Renrui Zhang Feng Li Hao Zhang Kaichen Zhang Yanwei Li Ziwei Liu Chunyuan Li MLLM SyDa VLM 58 599 0 06 Aug 2024
MMIU: Multimodal Multi-image Understanding for Evaluating Large Vision-Language Models Fanqing Meng Jun Wang Chuanhao Li Quanfeng Lu Hao Tian ... Jifeng Dai Ping Luo Ping Luo Kaipeng Zhang Wenqi Shao VLM 60 18 0 05 Aug 2024
Jailbreaking Text-to-Image Models with LLM-Based Agents Yingkai Dong Zheng Li Xiangtao Meng Ning Yu Shanqing Guo LLMAG 45 13 0 01 Aug 2024
GalleryGPT: Analyzing Paintings with Large Multimodal Models Yi Bin Wenhao Shi Yujuan Ding Zhiqiang Hu Zheng Wang Yang Yang See-Kiong Ng H. Shen MLLM 38 11 0 01 Aug 2024
SynthVLM: High-Efficiency and High-Quality Synthetic Data for Vision Language Models Zheng Liu Hao Liang Xijie Huang Wentao Xiong Qinhan Yu Linzhuang Sun Chong Chen Conghui He Bin Cui Wentao Zhang SyDa 59 0 0 30 Jul 2024
LLAVADI: What Matters For Multimodal Large Language Models Distillation Shilin Xu Xiangtai Li Haobo Yuan Lu Qi Yunhai Tong Ming-Hsuan Yang 36 3 0 28 Jul 2024
LLaVA-Read: Enhancing Reading Ability of Multimodal Language Models Ruiyi Zhang Yufan Zhou Jian Chen Jiuxiang Gu Changyou Chen Tongfei Sun VLM 41 6 0 27 Jul 2024
Data Processing Techniques for Modern Multimodal Models Yinheng Li Han Ding Hang Chen VLM 39 0 0 27 Jul 2024
Power-LLaVA: Large Language and Vision Assistant for Power Transmission Line Inspection Jiahao Wang Mingxuan Li Haichen Luo Jinguo Zhu A. Yang M. Rong Xiaohua Wang 27 3 0 27 Jul 2024
Dallah: A Dialect-Aware Multimodal Large Language Model for Arabic Fakhraddin Alwajih Gagan Bhatia Muhammad Abdul-Mageed 45 5 0 25 Jul 2024
$VILA^2$ : VILA Augmented VILA Yunhao Fang Ligeng Zhu Yao Lu Yan Wang Pavlo Molchanov Jang Hyun Cho Marco Pavone Song Han Hongxu Yin VLM 47 7 0 24 Jul 2024
INF-LLaVA: Dual-perspective Perception for High-Resolution Multimodal Large Language Model Yiwei Ma Zhibin Wang Xiaoshuai Sun Weihuang Lin Qiang-feng Zhou Jiayi Ji Rongrong Ji MLLM VLM 57 1 0 23 Jul 2024
MMInstruct: A High-Quality Multi-Modal Instruction Tuning Dataset with Extensive Diversity Yangzhou Liu Yue Cao Zhangwei Gao Weiyun Wang Zhe Chen ... Lewei Lu Xizhou Zhu Tong Lu Yu Qiao Jifeng Dai VLM MLLM 60 24 0 22 Jul 2024
LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding Haoning Wu Dongxu Li Bei Chen Junnan Li 40 114 0 22 Jul 2024
Evaluating Linguistic Capabilities of Multimodal LLMs in the Lens of Few-Shot Learning Mustafa Dogan .Ilker Kesen Iacer Calixto Aykut Erdem Erkut Erdem LRM 37 1 0 17 Jul 2024
F-HOI: Toward Fine-grained Semantic-Aligned 3D Human-Object Interactions Jie Yang Xuesong Niu Nan Jiang Ruimao Zhang Siyuan Huang 35 9 0 17 Jul 2024
FIRE: A Dataset for Feedback Integration and Refinement Evaluation of Multimodal Models Pengxiang Li Zhi Gao Bofei Zhang Tao Yuan Yuwei Wu Mehrtash Harandi Yunde Jia Song-Chun Zhu Qing Li VLM MLLM 48 5 0 16 Jul 2024
VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Models Haodong Duan Junming Yang Junming Yang Xinyu Fang Lin Chen ... Yuhang Zang Pan Zhang Jiaqi Wang Dahua Lin Kai Chen LM&MA VLM 41 119 0 16 Jul 2024
MAVIS: Mathematical Visual Instruction Tuning Renrui Zhang Xinyu Wei Dongzhi Jiang Yichi Zhang Ziyu Guo ... Aojun Zhou Bin Wei Shanghang Zhang Peng Gao Hongsheng Li MLLM 42 29 0 11 Jul 2024
HiRes-LLaVA: Restoring Fragmentation Input in High-Resolution Large Vision-Language Models Runhui Huang Xinpeng Ding Chunwei Wang J. N. Han Yulong Liu Hengshuang Zhao Hang Xu Lu Hou Wei Zhang Xiaodan Liang VLM 31 8 0 11 Jul 2024
The Synergy between Data and Multi-Modal Large Language Models: A Survey from Co-Development Perspective Zhen Qin Daoyuan Chen Wenhao Zhang Liuyi Yao Yilun Huang Bolin Ding Yaliang Li Shuiguang Deng 65 4 0 11 Jul 2024
DenseFusion-1M: Merging Vision Experts for Comprehensive Multimodal Perception Xiaotong Li Fan Zhang Haiwen Diao Yueze Wang Xinlong Wang Ling-yu Duan VLM 31 28 0 11 Jul 2024
IDA-VLM: Towards Movie Understanding via ID-Aware Large Vision-Language Model Yatai Ji Shilong Zhang Jie Wu Peize Sun Weifeng Chen Xuefeng Xiao Sidi Yang Yanting Yang Ping Luo VLM 53 3 0 10 Jul 2024
Multimodal Self-Instruct: Synthetic Abstract Image and Visual Reasoning Instruction Using Language Model Wenqi Zhang Zhenglin Cheng Yuanyu He Mengna Wang Yongliang Shen ... Guiyang Hou Mingqian He Yanna Ma Weiming Lu Yueting Zhuang SyDa 74 9 0 09 Jul 2024
Graph-Based Captioning: Enhancing Visual Descriptions by Interconnecting Region Captions Yu-Guan Hsieh Cheng-Yu Hsieh Shih-Ying Yeh Louis Béthune Hadi Pour Ansari Pavan Kumar Anasosalu Vasu Chun-Liang Li Ranjay Krishna Oncel Tuzel Marco Cuturi 66 4 0 09 Jul 2024
A Single Transformer for Scalable Vision-Language Modeling Yangyi Chen Xingyao Wang Hao Peng Heng Ji LRM 51 16 0 08 Jul 2024
UltraEdit: Instruction-based Fine-Grained Image Editing at Scale Haozhe Zhao Xiaojian Ma Liang Chen Shuzheng Si Rujie Wu Kaikai An Peiyu Yu Minjia Zhang Qing Li Baobao Chang 44 44 0 07 Jul 2024
OmChat: A Recipe to Train Multimodal Language Models with Strong Long Context and Video Understanding Tiancheng Zhao Qianqian Zhang Kyusong Lee Peng Liu Lu Zhang Chunxin Fang Jiajia Liao Kelei Jiang Yibo Ma Ruochen Xu MLLM VLM 54 5 0 06 Jul 2024
A Systematic Survey and Critical Review on Evaluating Large Language Models: Challenges, Limitations, and Recommendations Md Tahmid Rahman Laskar Sawsan Alqahtani M Saiful Bari Mizanur Rahman Mohammad Abdullah Matin Khan ... Chee Wei Tan Md. Rizwan Parvez Enamul Hoque Chenyu You Jimmy Huang ELM ALM 31 28 0 04 Jul 2024
Lateralization LoRA: Interleaved Instruction Tuning with Modality-Specialized Adaptations Zhiyang Xu Minqian Liu Ying Shen Joy Rimchala Jiaxin Zhang Qifan Wang Yu Cheng Lifu Huang VLM 39 2 0 04 Jul 2024
InternLM-XComposer-2.5: A Versatile Large Vision Language Model Supporting Long-Contextual Input and Output Pan Zhang Xiaoyi Dong Yuhang Zang Yuhang Cao Rui Qian ... Kai Chen Jifeng Dai Yu Qiao Dahua Lin Jiaqi Wang 47 100 0 03 Jul 2024
MIA-Bench: Towards Better Instruction Following Evaluation of Multimodal LLMs Yusu Qian Hanrong Ye J. Fauconnier Peter Grasch Yinfei Yang Zhe Gan 113 13 0 01 Jul 2024
Tarsier: Recipes for Training and Evaluating Large Video Description Models Jiawei Wang Liping Yuan Yuchen Zhang 49 52 0 30 Jun 2024
Investigating and Mitigating the Multimodal Hallucination Snowballing in Large Vision-Language Models Weihong Zhong Xiaocheng Feng Liang Zhao Qiming Li Lei Huang Yuxuan Gu Weitao Ma Yuan Xu Bing Qin MLLM 41 10 0 30 Jun 2024
Web2Code: A Large-scale Webpage-to-Code Dataset and Evaluation Framework for Multimodal LLMs Sukmin Yun Haokun Lin Rusiru Thushara Mohammad Qazim Bhat Yongxin Wang ... Timothy Baldwin Zhengzhong Liu Eric P. Xing Xiaodan Liang Zhiqiang Shen 54 10 0 28 Jun 2024
STLLaVA-Med: Self-Training Large Language and Vision Assistant for Medical Guohao Sun Can Qin Huazhu Fu Linwei Wang Zhiqiang Tao LM&MA 40 3 0 28 Jun 2024
MM-Instruct: Generated Visual Instructions for Large Multimodal Model Alignment Jihao Liu Xin Huang Jinliang Zheng Boxiao Liu Jia Wang Osamu Yoshie Yu Liu Hongsheng Li MLLM SyDa 38 4 0 28 Jun 2024
Solving Token Gradient Conflict in Mixture-of-Experts for Large Vision-Language Model Longrong Yang Dong Shen Chaoxiang Cai Fan Yang Size Li Di Zhang Xi Li MoE 62 2 0 28 Jun 2024
OMG-LLaVA: Bridging Image-level, Object-level, Pixel-level Reasoning and Understanding Tao Zhang Xiangtai Li Hao Fei Haobo Yuan Shengqiong Wu Shunping Ji Chen Change Loy Shuicheng Yan LRM MLLM VLM 54 49 0 27 Jun 2024
HuatuoGPT-Vision, Towards Injecting Medical Visual Knowledge into Multimodal LLMs at Scale Junying Chen Ruyi Ouyang Anningzhe Gao Shunian Chen Guiming Hardy Chen ... Zhenyang Cai Ke Ji Guangjun Yu Xiang Wan Benyou Wang MedIm LM&MA 51 32 0 27 Jun 2024
OmniJARVIS: Unified Vision-Language-Action Tokenization Enables Open-World Instruction Following Agents Zihao Wang Shaofei Cai Zhancun Mu Haowei Lin Ceyao Zhang Xuejie Liu Qing Li Guy Van den Broeck Xiaojian Ma Yitao Liang LM&Ro 59 12 0 27 Jun 2024
MATE: Meet At The Embedding -- Connecting Images with Long Texts Young Kyun Jang Junmo Kang Yong Jae Lee Donghyun Kim VLM 44 5 0 26 Jun 2024
Evaluating Fairness in Large Vision-Language Models Across Diverse Demographic Attributes and Prompts Xuyang Wu Yuan Wang Hsin-Tai Wu Zhiqiang Tao Yi Fang VLM 40 8 0 25 Jun 2024
MG-LLaVA: Towards Multi-Granularity Visual Instruction Tuning Xiangyu Zhao Xiangtai Li Haodong Duan Haian Huang Yining Li Kai Chen Hua Yang VLM MLLM 50 10 0 25 Jun 2024
LIPE: Learning Personalized Identity Prior for Non-rigid Image Editing Aoyang Liu Qingnan Fan Shuai Qin Hong Gu Yansong Tang DiffM 58 1 0 25 Jun 2024
Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMs Shengbang Tong Ellis L Brown Penghao Wu Sanghyun Woo Manoj Middepogu ... Xichen Pan Austin Wang Rob Fergus Yann LeCun Saining Xie 3DV MLLM 53 287 0 24 Jun 2024
ResMaster: Mastering High-Resolution Image Generation via Structural and Fine-Grained Guidance Shuwei Shi Wenbo Li Yuechen Zhang Jingwen He Biao Gong Yinqiang Zheng 57 10 0 24 Jun 2024
Prism: A Framework for Decoupling and Assessing the Capabilities of VLMs Yuxuan Qiao Haodong Duan Xinyu Fang Junming Yang Lin Chen Songyang Zhang Jiaqi Wang Dahua Lin Kai Chen LRM 50 19 0 20 Jun 2024
MMBench-Video: A Long-Form Multi-Shot Benchmark for Holistic Video Understanding Xinyu Fang Kangrui Mao Haodong Duan Xiangyu Zhao Yining Li Dahua Lin Kai Chen VLM 57 63 0 20 Jun 2024