v1v2v3 (latest)

VideoGLaMM: A Large Multimodal Model for Pixel-Level Visual Grounding in Videos

7 November 2024

Papers citing "VideoGLaMM: A Large Multimodal Model for Pixel-Level Visual Grounding in Videos"

50 / 52 papers shown

Title
Pixel-SAIL: Single Transformer For Pixel-Grounded Understanding Tao Zhang Xuelong Li Zilong Huang Yuchen Li Weixian Lei XueQing Deng Shihao Chen S. Ji Jiashi Feng MLLM LRM 119 5 0 14 Apr 2025
V-STaR: Benchmarking Video-LLMs on Video Spatio-Temporal Reasoning Zixu Cheng Jian Hu Ziquan Liu Chenyang Si Wei Li Shaogang Gong LRM 136 5 0 14 Mar 2025
Sa2VA: Marrying SAM2 with LLaVA for Dense Grounded Understanding of Images and Videos Haobo Yuan Xianrui Li Tao Zhang Zilong Huang Shilin Xu S. Ji Yunhai Tong Lu Qi Jiashi Feng Ming-Hsuan Yang VLM 157 25 0 07 Jan 2025
Referring Video Object Segmentation via Language-aligned Track Selection Seongchan Kim Woojeong Jin Sangbeom Lim Heeji Yoon Hyunwook Choi Seungryong Kim VOS 145 0 0 02 Dec 2024
One Token to Seg Them All: Language Instructed Reasoning Segmentation in Videos Zechen Bai Tong He Haiyang Mei Pichao Wang Ziteng Gao Joya Chen Lei Liu Zheng Zhang Mike Zheng Shou VLM VOS MLLM 85 27 0 29 Sep 2024
SAM 2: Segment Anything in Images and Videos Nikhila Ravi Valentin Gabeur Yuan-Ting Hu Ronghang Hu Chaitanya K. Ryali ... Nicolas Carion Chao-Yuan Wu Ross B. Girshick Piotr Dollár Christoph Feichtenhofer VLM MLLM 148 917 0 01 Aug 2024
VideoGPT+: Integrating Image and Video Encoders for Enhanced Video Understanding Muhammad Maaz H. Rasheed Salman Khan Fahad A Khan VLM MLLM 76 61 0 13 Jun 2024
Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone Marah Abdin Sam Ade Jacobs A. A. Awan J. Aneja Ahmed Hassan Awadallah ... Li Zhang Yi Zhang Yue Zhang Yunan Zhang Xiren Zhou LRM ALM 146 1,249 0 22 Apr 2024
InternVideo2: Scaling Video Foundation Models for Multimodal Video Understanding Yi Wang Kunchang Li Xinhao Li Jiashuo Yu Yinan He ... Hongjie Zhang Yifei Huang Yu Qiao Yali Wang Limin Wang 75 76 0 22 Mar 2024
Tracking with Human-Intent Reasoning Jiawen Zhu Zhi-Qi Cheng Jun-Yan He Chenyang Li Bin Luo Huchuan Lu Yifeng Geng Xuansong Xie LRM VOS 76 11 0 29 Dec 2023
BIVDiff: A Training-Free Framework for General-Purpose Video Synthesis via Bridging Image and Video Diffusion Models Fengyuan Shi Jiaxi Gu Hang Xu Songcen Xu Wei Zhang Limin Wang VGen DiffM 43 14 0 05 Dec 2023
PG-Video-LLaVA: Pixel Grounding Large Video-Language Models Shehan Munasinghe Rusiru Thushara Muhammad Maaz H. Rasheed Salman Khan Mubarak Shah Fahad Khan VLM MLLM 65 36 0 22 Nov 2023
Video-LLaVA: Learning United Visual Representation by Alignment Before Projection Bin Lin Yang Ye Bin Zhu Jiaxi Cui Munan Ning Peng Jin Li-ming Yuan VLM MLLM 328 706 0 16 Nov 2023
GLaMM: Pixel Grounding Large Multimodal Model H. Rasheed Muhammad Maaz Sahal Shaji Mullappilly Abdelrahman M. Shaker Salman Khan Hisham Cholakkal Rao M. Anwer Erix Xing Ming-Hsuan Yang Fahad S. Khan MLLM VLM 117 238 0 06 Nov 2023
Ferret: Refer and Ground Anything Anywhere at Any Granularity Haoxuan You Haotian Zhang Zhe Gan Xianzhi Du Bowen Zhang Zirui Wang Liangliang Cao Shih-Fu Chang Yinfei Yang ObjD MLLM VLM 109 327 0 11 Oct 2023
Improved Baselines with Visual Instruction Tuning Haotian Liu Chunyuan Li Yuheng Li Yong Jae Lee VLM MLLM 130 2,807 0 05 Oct 2023
MeViS: A Large-scale Benchmark for Video Segmentation with Motion Expressions Henghui Ding Chang Liu Shuting He Xudong Jiang Chen Change Loy VOS 110 116 0 16 Aug 2023
The All-Seeing Project: Towards Panoptic Visual Recognition and Understanding of the Open World Weiyun Wang Min Shi Qingyun Li Wen Wang Zhenhang Huang ... Zhiguo Cao Yushi Chen Tong Lu Jifeng Dai Yu Qiao LRM MLLM 86 88 0 03 Aug 2023
OpenFlamingo: An Open-Source Framework for Training Large Autoregressive Vision-Language Models Anas Awadalla Irena Gao Josh Gardner Jack Hessel Yusuf Hanafy ... Simon Kornblith Pang Wei Koh Gabriel Ilharco Mitchell Wortsman Ludwig Schmidt MLLM 118 431 0 02 Aug 2023
LISA: Reasoning Segmentation via Large Language Model Xin Lai Zhuotao Tian Yukang Chen Yanwei Li Yuhui Yuan Shu Liu Jiaya Jia LM&Ro VLM MLLM LRM 138 460 0 01 Aug 2023
BuboGPT: Enabling Visual Grounding in Multi-Modal LLMs Yang Zhao Zhijie Lin Daquan Zhou Zilong Huang Jiashi Feng Bingyi Kang MLLM 73 111 0 17 Jul 2023
Shikra: Unleashing Multimodal LLM's Referential Dialogue Magic Ke Chen Zhao Zhang Weili Zeng Richong Zhang Feng Zhu Rui Zhao ObjD 95 650 0 27 Jun 2023
Kosmos-2: Grounding Multimodal Large Language Models to the World Zhiliang Peng Wenhui Wang Li Dong Y. Hao Shaohan Huang Shuming Ma Furu Wei MLLM ObjD VLM 111 763 0 26 Jun 2023
Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Models Muhammad Maaz H. Rasheed Salman Khan Fahad Shahbaz Khan MLLM 117 658 0 08 Jun 2023
Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding Hang Zhang Xin Li Lidong Bing MLLM 157 1,059 0 05 Jun 2023
InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning Wenliang Dai Junnan Li Dongxu Li A. M. H. Tiong Junqi Zhao Weisheng Wang Boyang Albert Li Pascale Fung Steven C. H. Hoi MLLM VLM 132 2,095 0 11 May 2023
VideoChat: Chat-Centric Video Understanding Kunchang Li Yinan He Yi Wang Yizhuo Li Wen Wang Ping Luo Yali Wang Limin Wang Yu Qiao MLLM 94 585 0 10 May 2023
InternGPT: Solving Vision-Centric Tasks by Interacting with ChatGPT Beyond Language Zhaoyang Liu Yinan He Wenhai Wang Weiyun Wang Yi Wang ... Yali Wang Limin Wang Ping Luo Jifeng Dai Yu Qiao LRM MLLM 102 85 0 09 May 2023
MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models Deyao Zhu Jun Chen Xiaoqian Shen Xiang Li Mohamed Elhoseiny VLM MLLM 162 2,060 0 20 Apr 2023
Visual Instruction Tuning Haotian Liu Chunyuan Li Qingyang Wu Yong Jae Lee SyDa VLM MLLM 567 4,910 0 17 Apr 2023
Segment Anything A. Kirillov Eric Mintun Nikhila Ravi Hanzi Mao Chloe Rolland ... Spencer Whitehead Alexander C. Berg Wan-Yen Lo Piotr Dollár Ross B. Girshick MLLM VLM 362 7,365 0 05 Apr 2023
LLaMA: Open and Efficient Foundation Language Models Hugo Touvron Thibaut Lavril Gautier Izacard Xavier Martinet Marie-Anne Lachaux ... Faisal Azhar Aurelien Rodriguez Armand Joulin Edouard Grave Guillaume Lample ALM PILM 1.5K 13,420 0 27 Feb 2023
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models Junnan Li Dongxu Li Silvio Savarese Steven C. H. Hoi VLM MLLM 429 4,641 0 30 Jan 2023
PACO: Parts and Attributes of Common Objects Vignesh Ramanathan Anmol Kalia Vladan Petrovic Yiqian Wen Baixue Zheng ... Abhishek Kadian Amir Mousavi Yi-Zhe Song Abhimanyu Dubey D. Mahajan VLM 87 104 0 04 Jan 2023
Open-Vocabulary Semantic Segmentation with Mask-adapted CLIP Feng Liang Bichen Wu Xiaoliang Dai Kunpeng Li Yinan Zhao Hang Zhang Peizhao Zhang Peter Vajda Diana Marculescu CLIP VLM 102 457 0 09 Oct 2022
BURST: A Benchmark for Unifying Object Recognition, Segmentation and Tracking in Video A. Athar Jonathon Luiten P. Voigtlaender Tarasha Khurana Achal Dave Bastian Leibe Deva Ramanan VOS VLM 87 60 0 25 Sep 2022
Expanding Language-Image Pretrained Models for General Video Recognition Bolin Ni Houwen Peng Minghao Chen Songyang Zhang Gaofeng Meng Jianlong Fu Shiming Xiang Haibin Ling VLM CLIP ViT 106 326 0 04 Aug 2022
Video Mask Transfiner for High-Quality Video Instance Segmentation Lei Ke Henghui Ding Martin Danelljan Yu-Wing Tai Chi-Keung Tang Feng Yu 70 30 0 28 Jul 2022
Bridging the Gap between Object and Image-level Representations for Open-Vocabulary Detection H. Rasheed Muhammad Maaz Muhammad Uzair Khattak Salman Khan Fahad Shahbaz Khan ObjD VLM 107 154 0 07 Jul 2022
Flamingo: a Visual Language Model for Few-Shot Learning Jean-Baptiste Alayrac Jeff Donahue Pauline Luc Antoine Miech Iain Barr ... Mikolaj Binkowski Ricardo Barreira Oriol Vinyals Andrew Zisserman Karen Simonyan MLLM VLM 418 3,602 0 29 Apr 2022
Language-Grounded Indoor 3D Semantic Segmentation in the Wild Dávid Rozenberszki Or Litany Angela Dai 3DV VLM 95 189 0 16 Apr 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 883 13,176 0 04 Mar 2022
ActionCLIP: A New Paradigm for Video Action Recognition Mengmeng Wang Jiazheng Xing Yong Liu VLM 209 372 0 17 Sep 2021
LoRA: Low-Rank Adaptation of Large Language Models J. E. Hu Yelong Shen Phillip Wallis Zeyuan Allen-Zhu Yuanzhi Li Shean Wang Lu Wang Weizhu Chen OffRL AI4TS AI4CE ALM AIMat 488 10,496 0 17 Jun 2021
GLM: General Language Model Pretraining with Autoregressive Blank Infilling Zhengxiao Du Yujie Qian Xiao Liu Ming Ding J. Qiu Zhilin Yang Jie Tang BDL AI4CE 142 1,553 0 18 Mar 2021
Learning Transferable Visual Models From Natural Language Supervision Alec Radford Jong Wook Kim Chris Hallacy Aditya A. Ramesh Gabriel Goh ... Amanda Askell Pamela Mishkin Jack Clark Gretchen Krueger Ilya Sutskever CLIP VLM 967 29,810 0 26 Feb 2021
Human-centric Spatio-Temporal Video Grounding With Visual Transformers Zongheng Tang Yue Liao Si Liu Guanbin Li Xiaojie Jin Hongxu Jiang Qian Yu Dong Xu 63 99 0 10 Nov 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 853 42,332 0 28 May 2020
Where Does It Exist: Spatio-Temporal Video Grounding for Multi-Form Sentences Zhu Zhang Zhou Zhao Yang Zhao Qi. Wang Huasheng Liu Lianli Gao 73 118 0 19 Jan 2020
Grounded Video Description Luowei Zhou Yannis Kalantidis Xinlei Chen Jason J. Corso Marcus Rohrbach 83 193 0 17 Dec 2018