v1v2v3v4 (latest)

GPT4Scene: Understand 3D Scenes from Videos with Vision-Language Models

2 January 2025

Papers citing "GPT4Scene: Understand 3D Scenes from Videos with Vision-Language Models"

50 / 109 papers shown

Title
LEO-VL: Towards 3D Vision-Language Generalists via Data Scaling with Efficient Representation J. Huang Xiaojian Ma Xiongkun Linghu Yue Fan Junchao He ... Qing Li Song-Chun Zhu Yixin Chen Baoxiong Jia Siyuan Huang 88 0 0 11 Jun 2025
From Objects to Anywhere: A Holistic Benchmark for Multi-level Visual Grounding in 3D Scenes Tianxu Wang Zhuofan Zhang Ziyu Zhu Yue Fan Jing Xiong Pengxiang Li Xiaojian Ma Qing Li 111 0 0 05 Jun 2025
Struct2D: A Perception-Guided Framework for Spatial Reasoning in Large Multimodal Models Fangrui Zhu Hanhui Wang Yiming Xie Jing Gu Tianye Ding Jianwei Yang Huaizu Jiang 3DV LRM 116 0 0 04 Jun 2025
Visual Embodied Brain: Let Multimodal Large Language Models See, Think, and Control in Spaces Gen Luo Ganlin Yang Ziyang Gong Guanzhou Chen Haonan Duan ... Wenhai Wang Jifeng Dai Yu Qiao Rongrong Ji X. Zhu LM&Ro 41 1 0 30 May 2025
Learning from Videos for 3D World: Enhancing MLLMs with 3D Vision Geometry Priors Duo Zheng Shijia Huang Yanyang Li Liwei Wang 51 0 0 30 May 2025
Spatial-MLLM: Boosting MLLM Capabilities in Visual-based Spatial Intelligence Diankun Wu Fangfu Liu Yi-Hsin Hung Yueqi Duan LRM 97 1 0 29 May 2025
AdaToken-3D: Dynamic Spatial Gating for Efficient 3D Large Multimodal-Models Reasoning Kai Zhang Xingyu Chen Xiaofeng Zhang 116 0 0 19 May 2025
Ross3D: Reconstructive Visual Instruction Tuning with 3D-Awareness Haochen Wang Yucheng Zhao Tiancai Wang Haoqiang Fan Xinming Zhang Zhaoxiang Zhang 162 4 0 02 Apr 2025
MLLM-For3D: Adapting Multimodal Large Language Model for 3D Reasoning Segmentation Jiaxin Huang Runnan Chen Ziwen Li Zhengqing Gao Xiao He Yandong Guo Mingming Gong Tongliang Liu LRM 115 1 0 23 Mar 2025
Robin3D: Improving 3D Large Language Model via Robust Instruction Tuning Weitai Kang Haifeng Huang Yuzhang Shang Mubarak Shah Yan Yan 107 9 0 21 Feb 2025
Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces Jihan Yang Shusheng Yang Anjali W. Gupta Rilyn Han Li Fei-Fei Saining Xie LRM 215 107 0 18 Dec 2024
HourVideo: 1-Hour Video-Language Understanding Keshigeyan Chandrasegaran Agrim Gupta Lea M. Hadzic Taran Kota Jimming He Cristobal Eyzaguirre Zane Durante Manling Li Jiajun Wu L. Fei-Fei VLM 113 49 0 07 Nov 2024
Semantically Safe Robot Manipulation: From Semantic Scene Understanding to Motion Safeguards Lukas Brunke Yanni Zhang Ralf Romer Jack Naimer Nikola Staykov Siqi Zhou Angela P. Schoellig 122 5 0 19 Oct 2024
VLM-Grounder: A VLM Agent for Zero-Shot 3D Visual Grounding Runsen Xu Zhiwei Huang Tai Wang Yuxiao Chen Jiangmiao Pang Dahua Lin VGen 102 18 0 17 Oct 2024
LLaVA-3D: A Simple yet Effective Pathway to Empowering LMMs with 3D-awareness Chenming Zhu Tai Wang Wenwei Zhang Jiangmiao Pang Xihui Liu 255 52 0 26 Sep 2024
Lexicon3D: Probing Visual Foundation Models for Complex 3D Scene Understanding Yunze Man Shuhong Zheng Zhipeng Bao M. Hebert Liang-Yan Gui Yu-Xiong Wang 155 23 0 05 Sep 2024
LLaVA-OneVision: Easy Visual Task Transfer Bo Li Yuanhan Zhang Dong Guo Renrui Zhang Feng Li Hao Zhang Kaichen Zhang Yanwei Li Ziwei Liu Chunyuan Li MLLM SyDa VLM 174 865 0 06 Aug 2024
MiniCPM-V: A GPT-4V Level MLLM on Your Phone Yuan Yao Tianyu Yu Ao Zhang Chongyi Wang Junbo Cui ... Xu Han Guoyang Zeng Dahai Li Zhiyuan Liu Maosong Sun VLM MLLM 149 481 0 03 Aug 2024
Goldfish: Vision-Language Understanding of Arbitrarily Long Videos Kirolos Ataallah Xiaoqian Shen Eslam Abdelrahman Essam Sleiman Mingchen Zhuge Jian Ding Deyao Zhu Jürgen Schmidhuber Mohamed Elhoseiny VLM 75 20 0 17 Jul 2024
Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis Chaoyou Fu Yuhan Dai Yondong Luo Lei Li Shuhuai Ren ... Xiawu Zheng Enhong Chen Caifeng Shan Xing Sun Xing Sun VLM MLLM 187 421 0 31 May 2024
Intent3D: 3D Object Detection in RGB-D Scans Based on Human Intention Weitai Kang Mengxue Qu Jyoti Kini Yunchao Wei Mubarak Shah Yan Yan LM&Ro 3DPC 110 10 0 28 May 2024
Unifying 3D Vision-Language Understanding via Promptable Queries Ziyu Zhu Zhuofan Zhang Xiaojian Ma Xuesong Niu Yixin Chen Baoxiong Jia Zhidong Deng Siyuan Huang Qing Li 118 32 0 19 May 2024
MiniGPT4-Video: Advancing Multimodal LLMs for Video Understanding with Interleaved Visual-Textual Tokens Kirolos Ataallah Xiaoqian Shen Eslam Abdelrahman Essam Sleiman Deyao Zhu Jian Ding Mohamed Elhoseiny VLM 104 79 0 04 Apr 2024
Are We on the Right Way for Evaluating Large Vision-Language Models? Lin Chen Jinsong Li Xiao-wen Dong Pan Zhang Yuhang Zang ... Haodong Duan Jiaqi Wang Yu Qiao Dahua Lin Feng Zhao VLM 152 303 0 29 Mar 2024
LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models Yaowei Zheng Richong Zhang Junhao Zhang Yanhan Ye Zheyan Luo Zhangchi Feng Yongqiang Ma 184 559 0 20 Mar 2024
Agent3D-Zero: An Agent for Zero-shot 3D Understanding Sha Zhang Di Huang Jiajun Deng Shixiang Tang Wanli Ouyang Tong He Yanyong Zhang VGen 71 18 0 18 Mar 2024
Scene-LLM: Extending Language Model for 3D Visual Understanding and Reasoning Rao Fu Jingyu Liu Xilun Chen Yixin Nie Wenhan Xiong LM&Ro LRM 115 74 0 18 Mar 2024
ShapeLLM: Universal 3D Object Understanding for Embodied Interaction Zekun Qi Runpei Dong Shaochen Zhang Haoran Geng Chunrui Han Zheng Ge Li Yi Kaisheng Ma 206 63 0 27 Feb 2024
InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Model Xiao-wen Dong Pan Zhang Yuhang Zang Yuhang Cao Bin Wang ... Conghui He Xingcheng Zhang Yu Qiao Dahua Lin Jiaqi Wang VLM MLLM 166 268 0 29 Jan 2024
EmbodiedScan: A Holistic Multi-Modal 3D Perception Suite Towards Embodied AI Tai Wang Xiaohan Mao Chenming Zhu Runsen Xu Ruiyuan Lyu ... Tianfan Xue Xihui Liu Cewu Lu Dahua Lin Jiangmiao Pang LM&Ro 120 74 0 26 Dec 2023
InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks Zhe Chen Jiannan Wu Wenhai Wang Weijie Su Guo Chen ... Bin Li Ping Luo Tong Lu Yu Qiao Jifeng Dai VLM MLLM 318 1,217 0 21 Dec 2023
Chat-3D v2: Bridging 3D Scene and Large Language Models with Object Identifiers Haifeng Huang Zehan Wang Rongjie Huang Luping Liu Xize Cheng Yang Zhao Tao Jin Zhou Zhao 126 62 0 13 Dec 2023
GPT4Point: A Unified Framework for Point-Language Understanding and Generation Zhangyang Qi Ye Fang Zeyi Sun Xiaoyang Wu Tong Wu Jiaqi Wang Dahua Lin Hengshuang Zhao MLLM 184 36 0 05 Dec 2023
TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understanding Shuhuai Ren Linli Yao Shicheng Li Xu Sun Lu Hou VLM MLLM 112 200 0 04 Dec 2023
LL3DA: Visual Interactive Instruction Tuning for Omni-3D Understanding, Reasoning, and Planning Sijin Chen Xin Chen C. Zhang Mingsheng Li Gang Yu Hao Fei Erik Cambria Jiayuan Fan Tao Chen MLLM 103 102 0 30 Nov 2023
LLaMA-VID: An Image is Worth 2 Tokens in Large Language Models Yanwei Li Chengyao Wang Jiaya Jia VLM MLLM 101 301 0 28 Nov 2023
MVBench: A Comprehensive Multi-modal Video Understanding Benchmark Kunchang Li Yali Wang Yinan He Yizhuo Li Yi Wang ... Jilan Xu Guo Chen Ping Luo Limin Wang Yu Qiao VLM MLLM 169 507 0 28 Nov 2023
An Embodied Generalist Agent in 3D World Jiangyong Huang Silong Yong Xiaojian Ma Xiongkun Linghu Puhao Li Yan Wang Qing Li Song-Chun Zhu Baoxiong Jia Siyuan Huang LM&Ro 127 176 0 18 Nov 2023
Video-LLaVA: Learning United Visual Representation by Alignment Before Projection Bin Lin Yang Ye Bin Zhu Jiaxi Cui Munan Ning Peng Jin Li-ming Yuan VLM MLLM 393 711 0 16 Nov 2023
GLaMM: Pixel Grounding Large Multimodal Model H. Rasheed Muhammad Maaz Sahal Shaji Mullappilly Abdelrahman M. Shaker Salman Khan Hisham Cholakkal Rao M. Anwer Erix Xing Ming-Hsuan Yang Fahad S. Khan MLLM VLM 173 239 0 06 Nov 2023
CogVLM: Visual Expert for Pretrained Language Models Weihan Wang Qingsong Lv Wenmeng Yu Wenyi Hong Ji Qi ... Bin Xu Juanzi Li Yuxiao Dong Ming Ding Jie Tang VLM MLLM 194 517 0 06 Nov 2023
Set-of-Mark Prompting Unleashes Extraordinary Visual Grounding in GPT-4V Jianwei Yang Hao Zhang Feng Li Xueyan Zou Chun-yue Li Jianfeng Gao MLLM VLM 136 189 0 17 Oct 2023
MiniGPT-v2: large language model as a unified interface for vision-language multi-task learning Jun Chen Deyao Zhu Xiaoqian Shen Xiang Li Zechun Liu Pengchuan Zhang Raghuraman Krishnamoorthi Vikas Chandra Yunyang Xiong Mohamed Elhoseiny MLLM 258 475 0 14 Oct 2023
Improved Baselines with Visual Instruction Tuning Haotian Liu Chunyuan Li Yuheng Li Yong Jae Lee VLM MLLM 252 2,834 0 05 Oct 2023
Multi3DRefer: Grounding Text Description to Multiple 3D Objects Yiming Zhang ZeMing Gong Angel X. Chang 137 77 0 11 Sep 2023
Four Ways to Improve Verbo-visual Fusion for Dense 3D Visual Grounding Ozan Unal Daniel Gehrig Suman Saha Luc Van Gool 94 17 0 08 Sep 2023
Point-Bind & Point-LLM: Aligning Point Cloud with Multi-modality for 3D Understanding, Generation, and Instruction Following Ziyu Guo Renrui Zhang Xiangyang Zhu Yiwen Tang Xianzheng Ma ... Ke Chen Peng Gao Xianzhi Li Hongsheng Li Pheng-Ann Heng MLLM 110 146 0 01 Sep 2023
PointLLM: Empowering Large Language Models to Understand Point Clouds Runsen Xu Xiaolong Wang Tai Wang Yilun Chen Jiangmiao Pang Dahua Lin MLLM 135 185 0 31 Aug 2023
Chat-3D: Data-efficiently Tuning Large Language Model for Universal Dialogue of 3D Scenes Zehan Wang Haifeng Huang Yang Zhao Ziang Zhang Zhou Zhao 125 73 0 17 Aug 2023
3D-VisTA: Pre-trained Transformer for 3D Vision and Text Alignment Ziyu Zhu Xiaojian Ma Yixin Chen Zhidong Deng Siyuan Huang Qing Li LM&Ro 90 123 0 08 Aug 2023