LiDAR-LLM: Exploring the Potential of Large Language Models for 3D LiDAR Understanding

21 December 2023

Shanghang Zhang

Papers citing "LiDAR-LLM: Exploring the Potential of Large Language Models for 3D LiDAR Understanding"

47 / 47 papers shown

Title
Flex-Judge: Think Once, Judge Anywhere Jongwoo Ko S. Kim Sungwoo Cho Se-Young Yun ELM LRM 206 0 0 24 May 2025
LSceneLLM: Enhancing Large 3D Scene Understanding Using Adaptive Visual Preferences Hongyan Zhi Peihao Chen Junyan Li Shuailei Ma Xinyu Sun Tianhang Xiang Yinjie Lei Mingkui Tan Chuang Gan 164 8 0 02 Dec 2024
Guide-LLM: An Embodied LLM Agent and Text-Based Topological Map for Robotic Guidance of People with Visual Impairments Sangmim Song S. Kodagoda A. Gunatilake Marc G. Carmichael Karthick Thiyagarajan Jodi Martin LM&Ro 151 1 0 28 Oct 2024
Talk2Radar: Bridging Natural Language with 4D mmWave Radar for 3D Referring Expression Comprehension Runwei Guan Ruixiao Zhang Ningwei Ouyang Tao Huang Ka Lok Man ... Ming Xu Jeremy S. Smith Eng Gee Lim Yutao Yue Hui Xiong 190 10 0 21 May 2024
SPHINX-X: Scaling Data and Parameters for a Family of Multi-modal Large Language Models Chris Liu Renrui Zhang Longtian Qiu Siyuan Huang Weifeng Lin ... Hao Shao Pan Lu Hongsheng Li Yu Qiao Peng Gao MLLM 217 116 0 08 Feb 2024
Unlock the Power: Competitive Distillation for Multi-Modal Large Language Models Xinwei Li Li Lin Shuai Wang Chen Qian 34 4 0 14 Nov 2023
ImageManip: Image-based Robotic Manipulation with Affordance-guided Next View Selection Xiaoqi Li Yanzi Wang Yan Shen Ponomarenko Iaroslav Haoran Lu Qianxu Wang Boshi An Jiaming Liu Hao Dong 60 5 0 13 Oct 2023
Improved Baselines with Visual Instruction Tuning Haotian Liu Chunyuan Li Yuheng Li Yong Jae Lee VLM MLLM 184 2,826 0 05 Oct 2023
RenderOcc: Vision-Centric 3D Occupancy Prediction with 2D Rendering Supervision Mingjie Pan Jiaming Liu Renrui Zhang Peixiang Huang Xiaoqi Li Bing Wang Hongwei Xie Li Liu Shanghang Zhang 118 90 0 18 Sep 2023
Find What You Want: Learning Demand-conditioned Object Attribute Space for Demand-driven Navigation Hongchen Wang Andy Guan Hong Chen Xiaoqi Li Mingdong Wu Hao Dong 62 16 0 15 Sep 2023
Point-Bind & Point-LLM: Aligning Point Cloud with Multi-modality for 3D Understanding, Generation, and Instruction Following Ziyu Guo Renrui Zhang Xiangyang Zhu Yiwen Tang Xianzheng Ma ... Ke Chen Peng Gao Xianzhi Li Hongsheng Li Pheng-Ann Heng MLLM 96 144 0 01 Sep 2023
PointLLM: Empowering Large Language Models to Understand Point Clouds Runsen Xu Xiaolong Wang Tai Wang Yilun Chen Jiangmiao Pang Dahua Lin MLLM 123 185 0 31 Aug 2023
Chat-3D: Data-efficiently Tuning Large Language Model for Universal Dialogue of 3D Scenes Zehan Wang Haifeng Huang Yang Zhao Ziang Zhang Zhou Zhao 110 73 0 17 Aug 2023
ViDA: Homeostatic Visual Domain Adapter for Continual Test Time Adaptation Jiaming Liu Senqiao Yang Peidong Jia Renrui Zhang Ming Lu Yandong Guo Wei Xue Shanghang Zhang TTA OOD VLM 98 40 0 07 Jun 2023
NuScenes-QA: A Multi-modal Visual Question Answering Benchmark for Autonomous Driving Scenario Tianwen Qian Jingjing Chen Linhai Zhuo Yang Jiao Yueping Jiang 88 156 0 24 May 2023
VisionLLM: Large Language Model is also an Open-Ended Decoder for Vision-Centric Tasks Wen Wang Zhe Chen Xiaokang Chen Jiannan Wu Xizhou Zhu ... Ping Luo Tong Lu Jie Zhou Yu Qiao Jifeng Dai MLLM VLM 107 492 0 18 May 2023
MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models Deyao Zhu Jun Chen Xiaoqian Shen Xiang Li Mohamed Elhoseiny VLM MLLM 165 2,072 0 20 Apr 2023
CLIP-Guided Vision-Language Pre-training for Question Answering in 3D Scenes Maria Parelli Alexandros Delitzas Nikolas Hars G. Vlassis Sotiris Anagnostidis Gregor Bachmann Thomas Hofmann CLIP 70 51 0 12 Apr 2023
ViewRefer: Grasp the Multi-view Knowledge for 3D Visual Grounding with GPT and Prototype Guidance Zoey Guo Yiwen Tang Renrui Zhang Dong Wang Zhigang Wang Bin Zhao Xuelong Li 110 59 0 29 Mar 2023
LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention Renrui Zhang Jiaming Han Chris Liu Peng Gao Aojun Zhou Xiangfei Hu Shilin Yan Pan Lu Hongsheng Li Yu Qiao MLLM 166 787 0 28 Mar 2023
LLaMA: Open and Efficient Foundation Language Models Hugo Touvron Thibaut Lavril Gautier Izacard Xavier Martinet Marie-Anne Lachaux ... Faisal Azhar Aurelien Rodriguez Armand Joulin Edouard Grave Guillaume Lample ALM PILM 1.5K 13,490 0 27 Feb 2023
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models Junnan Li Dongxu Li Silvio Savarese Steven C. H. Hoi VLM MLLM 432 4,663 0 30 Jan 2023
BEV-SAN: Accurate BEV 3D Object Detection via Slice Attention Networks Xiaowei Chi Jiaming Liu Ming Lu Rongyu Zhang Zhaoqing Wang Yandong Guo Shanghang Zhang 3DPC 95 20 0 02 Dec 2022
Cloud-Device Collaborative Adaptation to Continual Changing Environments in the Real-world Yulu Gan Mingjie Pan Rongyu Zhang Zijian Ling Lingran Zhao Jiaming Liu Shanghang Zhang VLM 73 15 0 02 Dec 2022
UniT3D: A Unified Transformer for 3D Dense Captioning and Visual Grounding Dave Zhenyu Chen Ronghang Hu Xinlei Chen Matthias Nießner Angel X. Chang 118 54 0 01 Dec 2022
DETRDistill: A Universal Knowledge Distillation Framework for DETR-families Jiahao Chang Shuo Wang Guangkai Xu Zehui Chen Chenhongyi Yang Fengshang Zhao 79 29 0 17 Nov 2022
BEVDistill: Cross-Modal BEV Distillation for Multi-View 3D Object Detection Zehui Chen Zhenyu Li Shiquan Zhang Liangji Fang Qinhong Jiang Feng Zhao 88 61 0 17 Nov 2022
LAION-5B: An open large-scale dataset for training next generation image-text models Christoph Schuhmann Romain Beaumont Richard Vencu Cade Gordon Ross Wightman ... Srivatsa Kundurthy Katherine Crowson Ludwig Schmidt R. Kaczmarczyk J. Jitsev VLM MLLM CLIP 206 3,502 0 16 Oct 2022
SQA3D: Situated Question Answering in 3D Scenes Xiaojian Ma Silong Yong Zilong Zheng Qing Li Yitao Liang Song-Chun Zhu Siyuan Huang LM&Ro 81 159 0 14 Oct 2022
3D Concept Grounding on Neural Fields Yining Hong Yilun Du Chun-Tse Lin J. Tenenbaum Chuang Gan 94 20 0 13 Jul 2022
Flamingo: a Visual Language Model for Few-Shot Learning Jean-Baptiste Alayrac Jeff Donahue Pauline Luc Antoine Miech Iain Barr ... Mikolaj Binkowski Ricardo Barreira Oriol Vinyals Andrew Zisserman Karen Simonyan MLLM VLM 418 3,610 0 29 Apr 2022
MORE: Multi-Order RElation Mining for Dense Captioning in 3D Scenes Yang Jiao Shaoxiang Chen Zequn Jie Wenke Huang Lin Ma Yu-Gang Jiang 3DPC 110 48 0 10 Mar 2022
ScanQA: 3D Question Answering for Spatial Scene Understanding Daich Azuma Taiki Miyanishi Shuhei Kurita M. Kawanabe 90 207 0 20 Dec 2021
Graph-to-3D: End-to-End Generation and Manipulation of 3D Scenes Using Scene Graphs Helisa Dhamo Fabian Manhardt Nassir Navab F. Tombari 3DV 53 70 0 19 Aug 2021
LoRA: Low-Rank Adaptation of Large Language Models J. E. Hu Yelong Shen Phillip Wallis Zeyuan Allen-Zhu Yuanzhi Li Shean Wang Lu Wang Weizhu Chen OffRL AI4TS AI4CE ALM AIMat 511 10,563 0 17 Jun 2021
SAT: 2D Semantics Assisted Training for 3D Visual Grounding Zhengyuan Yang Songyang Zhang Liwei Wang Jiebo Luo 3DPC 91 126 0 24 May 2021
Learning Transferable Visual Models From Natural Language Supervision Alec Radford Jong Wook Kim Chris Hallacy Aditya A. Ramesh Gabriel Goh ... Amanda Askell Pamela Mishkin Jack Clark Gretchen Krueger Ilya Sutskever CLIP VLM 1.0K 29,926 0 26 Feb 2021
Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts Soravit Changpinyo P. Sharma Nan Ding Radu Soricut VLM 450 1,142 0 17 Feb 2021
Scan2Cap: Context-aware Dense Captioning in RGB-D Scans Dave Zhenyu Chen A. Gholami Matthias Nießner Angel X. Chang 3DPC 164 176 0 03 Dec 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 908 42,520 0 28 May 2020
ScanRefer: 3D Object Localization in RGB-D Scans using Natural Language Dave Zhenyu Chen Angel X. Chang Matthias Nießner 3DPC 102 379 0 18 Dec 2019
BERTScore: Evaluating Text Generation with BERT Tianyi Zhang Varsha Kishore Felix Wu Kilian Q. Weinberger Yoav Artzi 381 5,872 0 21 Apr 2019
nuScenes: A multimodal dataset for autonomous driving Holger Caesar Varun Bankiti Alex H. Lang Sourabh Vora Venice Erin Liong Qiang Xu Anush Krishnan Yuxin Pan G. Baldan Oscar Beijbom 3DPC 306 5,790 0 26 Mar 2019
3D-Aware Scene Manipulation via Inverse Graphics Shunyu Yao T. Hsu Jun-Yan Zhu Jiajun Wu Antonio Torralba William T. Freeman J. Tenenbaum 3DPC 3DV 105 86 0 28 Aug 2018
VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection Yin Zhou Oncel Tuzel 3DPC 129 3,732 0 17 Nov 2017
Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering Peter Anderson Xiaodong He Chris Buehler Damien Teney Mark Johnson Stephen Gould Lei Zhang AIMat 131 4,224 0 25 Jul 2017
VQA: Visual Question Answering Aishwarya Agrawal Jiasen Lu Stanislaw Antol Margaret Mitchell C. L. Zitnick Dhruv Batra Devi Parikh CoGe 243 5,512 0 03 May 2015