v1v2v3 (latest)

F-LMM: Grounding Frozen Large Multimodal Models

9 June 2024

Wei Li

Papers citing "F-LMM: Grounding Frozen Large Multimodal Models"

50 / 87 papers shown

Title
MEBench: A Novel Benchmark for Understanding Mutual Exclusivity Bias in Vision-Language Models Anh Thai Stefan Stojanov Zixuan Huang Bikram Boote James M. Rehg VLM 37 0 0 26 May 2025
Pixel-SAIL: Single Transformer For Pixel-Grounded Understanding Tao Zhang Xuelong Li Zilong Huang Yuchen Li Weixian Lei XueQing Deng Shihao Chen S. Ji Jiashi Feng MLLM LRM 127 5 0 14 Apr 2025
Harmonizing Visual Representations for Unified Multimodal Understanding and Generation Size Wu Wentao Zhang Lumin Xu Sheng Jin Zhonghua Wu Qingyi Tao Wentao Liu Wei Li Chen Change Loy VGen 452 6 0 27 Mar 2025
REF-VLM: Triplet-Based Referring Paradigm for Unified Visual Decoding Yan Tai Luhao Zhu Zhiqiang Chen Ynan Ding Yiying Dong Xiaohong Liu Guodong Guo MLLM ObjD 97 0 0 10 Mar 2025
Your Large Vision-Language Model Only Needs A Few Attention Heads For Visual Grounding Seil Kang Jinyeong Kim Junhyeok Kim Seong Jae Hwang VLM 121 4 0 08 Mar 2025
Sa2VA: Marrying SAM2 with LLaVA for Dense Grounded Understanding of Images and Videos Haobo Yuan Xianrui Li Tao Zhang Zilong Huang Shilin Xu S. Ji Yunhai Tong Lu Qi Jiashi Feng Ming-Hsuan Yang VLM 174 25 0 07 Jan 2025
Towards Visual Grounding: A Survey Linhui Xiao Xiaoshan Yang X. Lan Yaowei Wang Changsheng Xu ObjD 272 5 0 31 Dec 2024
HyperSeg: Towards Universal Visual Segmentation with Large Language Model Cong Wei Yujie Zhong Haoxian Tan Yong Liu Zheng Zhao Jie Hu Yujiu Yang VOS MLLM VLM LRM 125 5 0 26 Nov 2024
Emerging Pixel Grounding in Large Multimodal Models Without Grounding Supervision Shengcao Cao Liang-Yan Gui Yu-Xiong Wang 77 3 0 10 Oct 2024
Instruction-guided Multi-Granularity Segmentation and Captioning with Large Multimodal Model Li Zhou Xu Yuan Zenghui Sun Zikun Zhou Jingsong Lan VLM MLLM 398 4 0 20 Sep 2024
Image Segmentation in Foundation Model Era: A Survey Tianfei Zhou Fei Zhang Boyu Chang Wenguan Wang Ye Yuan E. Konukoglu Daniel Cremers VLM 138 12 0 23 Aug 2024
Pseudo-RIS: Distinctive Pseudo-supervision Generation for Referring Image Segmentation Seonghoon Yu Paul Hongsuck Seo Jeany Son DiffM 124 6 0 10 Jul 2024
OMG-LLaVA: Bridging Image-level, Object-level, Pixel-level Reasoning and Understanding Tao Zhang Xiangtai Li Hao Fei Haobo Yuan Shengqiong Wu Shunping Ji Chen Change Loy Shuicheng Yan LRM MLLM VLM 126 63 0 27 Jun 2024
What matters when building vision-language models? Hugo Laurençon Léo Tronchon Matthieu Cord Victor Sanh VLM 103 177 0 03 May 2024
LaSagnA: Language-based Segmentation Assistant for Complex Queries Cong Wei Haoxian Tan Yujie Zhong Yujiu Yang Lin Ma 100 17 0 12 Apr 2024
Mini-Gemini: Mining the Potential of Multi-modality Vision Language Models Yanwei Li Yuechen Zhang Chengyao Wang Zhisheng Zhong Yixin Chen Ruihang Chu Shaoteng Liu Jiaya Jia VLM MLLM MoE 112 238 0 27 Mar 2024
PSALM: Pixelwise SegmentAtion with Large Multi-Modal Model Zheng Zhang Yeyao Ma Enming Zhang Xiang Bai VLM MLLM 100 47 0 21 Mar 2024
MM1: Methods, Analysis & Insights from Multimodal LLM Pre-training Brandon McKinzie Zhe Gan J. Fauconnier Sam Dodge Bowen Zhang ... Zirui Wang Ruoming Pang Peter Grasch Alexander Toshev Yinfei Yang MLLM 92 208 0 14 Mar 2024
Gemma: Open Models Based on Gemini Research and Technology Gemma Team Gemma Team Thomas Mesnard Cassidy Hardin Robert Dadashi Surya Bhupatiraju ... Armand Joulin Noah Fiedel Evan Senter Alek Andreev Kathleen Kenealy VLM LLMAG 229 506 0 13 Mar 2024
GROUNDHOG: Grounding Large Language Models to Holistic Segmentation Yichi Zhang Ziqiao Ma Xiaofeng Gao Suhaila Shakiah Qiaozi Gao Joyce Chai MLLM VLM 118 47 0 26 Feb 2024
GSVA: Generalized Segmentation via Multimodal Large Language Models Zhuofan Xia Dongchen Han Yizeng Han Xuran Pan Shiji Song Gao Huang VLM 138 68 0 15 Dec 2023
VILA: On Pre-training for Visual Language Models Ji Lin Hongxu Yin Ming-Yu Liu Yao Lu Pavlo Molchanov Andrew Tao Huizi Mao Jan Kautz Mohammad Shoeybi Song Han MLLM VLM 112 429 0 12 Dec 2023
LLaVA-Grounding: Grounded Visual Chat with Large Multimodal Models Hao Zhang Hongyang Li Feng Li Tianhe Ren Xueyan Zou ... Shijia Huang Jianfeng Gao Lei Zhang Chun-yue Li Jianwei Yang 179 75 0 05 Dec 2023
PixelLM: Pixel Reasoning with Large Multimodal Model Zhongwei Ren Zhicheng Huang Yunchao Wei Yao-Min Zhao Dongmei Fu Jiashi Feng Xiaojie Jin VLM MLLM LRM 99 108 0 04 Dec 2023
OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation Qidong Huang Xiao-wen Dong Pan Zhang Bin Wang Conghui He Jiaqi Wang Dahua Lin Weiming Zhang Neng H. Yu MLLM 128 206 0 29 Nov 2023
PerceptionGPT: Effectively Fusing Visual Perception into LLM Renjie Pi Lewei Yao Jiahui Gao Jipeng Zhang Tong Zhang MLLM 81 36 0 11 Nov 2023
GLaMM: Pixel Grounding Large Multimodal Model H. Rasheed Muhammad Maaz Sahal Shaji Mullappilly Abdelrahman M. Shaker Salman Khan Hisham Cholakkal Rao M. Anwer Erix Xing Ming-Hsuan Yang Fahad S. Khan MLLM VLM 138 238 0 06 Nov 2023
Ferret: Refer and Ground Anything Anywhere at Any Granularity Haoxuan You Haotian Zhang Zhe Gan Xianzhi Du Bowen Zhang Zirui Wang Liangliang Cao Shih-Fu Chang Yinfei Yang ObjD MLLM VLM 113 328 0 11 Oct 2023
Mistral 7B Albert Q. Jiang Alexandre Sablayrolles A. Mensch Chris Bamford Devendra Singh Chaplot ... Teven Le Scao Thibaut Lavril Thomas Wang Timothée Lacroix William El Sayed MoE LRM 113 2,246 0 10 Oct 2023
Improved Baselines with Visual Instruction Tuning Haotian Liu Chunyuan Li Yuheng Li Yong Jae Lee VLM MLLM 181 2,826 0 05 Oct 2023
MosaicFusion: Diffusion Models as Data Augmenters for Large Vocabulary Instance Segmentation Jiahao Xie Wei Li Xiangtai Li Ziwei Liu Yew-Soon Ong Chen Change Loy DiffM VLM 143 37 0 22 Sep 2023
MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities Weihao Yu Zhengyuan Yang Linjie Li Jianfeng Wang Kevin Qinghong Lin Zicheng Liu Xinchao Wang Lijuan Wang MLLM 117 719 0 04 Aug 2023
LISA: Reasoning Segmentation via Large Language Model Xin Lai Zhuotao Tian Yukang Chen Yanwei Li Yuhui Yuan Shu Liu Jiaya Jia LM&Ro VLM MLLM LRM 162 461 0 01 Aug 2023
Llama 2: Open Foundation and Fine-Tuned Chat Models Hugo Touvron Louis Martin Kevin R. Stone Peter Albert Amjad Almahairi ... Sharan Narang Aurelien Rodriguez Robert Stojnic Sergey Edunov Thomas Scialom AI4MH ALM 419 12,076 0 18 Jul 2023
BuboGPT: Enabling Visual Grounding in Multi-Modal LLMs Yang Zhao Zhijie Lin Daquan Zhou Zilong Huang Jiashi Feng Bingyi Kang MLLM 78 112 0 17 Jul 2023
MMBench: Is Your Multi-modal Model an All-around Player? Yuanzhan Liu Haodong Duan Yuanhan Zhang Yue Liu Songyang Zhang ... Jiaqi Wang Conghui He Ziwei Liu Kai-xiang Chen Dahua Lin 133 1,059 0 12 Jul 2023
Shikra: Unleashing Multimodal LLM's Referential Dialogue Magic Ke Chen Zhao Zhang Weili Zeng Richong Zhang Feng Zhu Rui Zhao ObjD 108 651 0 27 Jun 2023
Kosmos-2: Grounding Multimodal Large Language Models to the World Zhiliang Peng Wenhui Wang Li Dong Y. Hao Shaohan Huang Shuming Ma Furu Wei MLLM ObjD VLM 121 764 0 26 Jun 2023
MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models Chaoyou Fu Peixian Chen Yunhang Shen Yulei Qin Mengdan Zhang ... Xiawu Zheng Ke Li Xing Sun Zhenyu Qiu Rongrong Ji ELM MLLM 128 859 0 23 Jun 2023
MIMIC-IT: Multi-Modal In-Context Instruction Tuning Yue Liu Yuanhan Zhang Liangyu Chen Jinghao Wang Fanyi Pu Jingkang Yang Cuiping Li Ziwei Liu MLLM VLM 92 240 0 08 Jun 2023
GRES: Generalized Referring Expression Segmentation Chang Liu Henghui Ding Xudong Jiang 103 165 0 01 Jun 2023
Contextual Object Detection with Multimodal Large Language Models Yuhang Zang Wei Li Jun Han Kaiyang Zhou Chen Change Loy ObjD VLM MLLM 106 87 0 29 May 2023
Evaluating Object Hallucination in Large Vision-Language Models Yifan Li Yifan Du Kun Zhou Jinpeng Wang Wayne Xin Zhao Ji-Rong Wen MLLM LRM 309 814 0 17 May 2023
InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning Wenliang Dai Junnan Li Dongxu Li A. M. H. Tiong Junqi Zhao Weisheng Wang Boyang Albert Li Pascale Fung Steven C. H. Hoi MLLM VLM 150 2,098 0 11 May 2023
mPLUG-Owl: Modularization Empowers Large Language Models with Multimodality Qinghao Ye Haiyang Xu Guohai Xu Jiabo Ye Ming Yan ... Junfeng Tian Qiang Qi Ji Zhang Feiyan Huang Jingren Zhou VLM MLLM 295 956 0 27 Apr 2023
Segment Everything Everywhere All at Once Xueyan Zou Jianwei Yang Hao Zhang Feng Li Linjie Li Jianfeng Wang Lijuan Wang Jianfeng Gao Yong Jae Lee MLLM VLM 105 490 0 13 Apr 2023
Instruction Tuning with GPT-4 Baolin Peng Chunyuan Li Pengcheng He Michel Galley Jianfeng Gao SyDa ALM LM&MA 228 625 0 06 Apr 2023
Segment Anything A. Kirillov Eric Mintun Nikhila Ravi Hanzi Mao Chloe Rolland ... Spencer Whitehead Alexander C. Berg Wan-Yen Lo Piotr Dollár Ross B. Girshick MLLM VLM 395 7,421 0 05 Apr 2023
Sigmoid Loss for Language Image Pre-Training Xiaohua Zhai Basil Mustafa Alexander Kolesnikov Lucas Beyer CLIP VLM 257 1,200 0 27 Mar 2023
GPT-4 Technical Report OpenAI OpenAI OpenAI Josh Achiam Steven Adler Sandhini Agarwal Lama Ahmad ... Shengjia Zhao Tianhao Zheng Juntang Zhuang William Zhuk Barret Zoph LLMAG MLLM 1.5K 14,761 0 15 Mar 2023