Empowering Large Language Models with 3D Situation Awareness

Empowering Large Language Models with 3D Situation Awareness

29 March 2025

Hengshuang Zhao

Papers citing "Empowering Large Language Models with 3D Situation Awareness"

17 / 17 papers shown

Title
EmbodiedScan: A Holistic Multi-Modal 3D Perception Suite Towards Embodied AI Tai Wang Xiaohan Mao Chenming Zhu Runsen Xu Ruiyuan Lyu ... Tianfan Xue Xihui Liu Cewu Lu Dahua Lin Jiangmiao Pang LM&Ro 63 73 0 26 Dec 2023
GPT4Point: A Unified Framework for Point-Language Understanding and Generation Zhangyang Qi Ye Fang Zeyi Sun Xiaoyang Wu Tong Wu Jiaqi Wang Dahua Lin Hengshuang Zhao MLLM 121 36 0 05 Dec 2023
Visual Programming for Zero-shot Open-Vocabulary 3D Visual Grounding Zhihao Yuan Jinke Ren Chun-Mei Feng Hengshuang Zhao Shuguang Cui Zhen Li 84 29 0 26 Nov 2023
Uni3D: Exploring Unified 3D Representation at Scale Junsheng Zhou Jinsheng Wang Baorui Ma Yu-Shen Liu Tiejun Huang Xinlong Wang 71 96 0 10 Oct 2023
Visual Instruction Tuning Haotian Liu Chunyuan Li Qingyang Wu Yong Jae Lee SyDa VLM MLLM 529 4,740 0 17 Apr 2023
End-to-End 3D Dense Captioning with Vote2Cap-DETR Sijin Chen Erik Cambria Xin Chen Yinjie Lei Tao Chen YU Gang ViT 54 58 0 06 Jan 2023
Language Conditioned Spatial Relation Reasoning for 3D Object Grounding Shizhe Chen Pierre-Louis Guhur Makarand Tapaswi Cordelia Schmid Ivan Laptev 87 85 0 17 Nov 2022
SQA3D: Situated Question Answering in 3D Scenes Xiaojian Ma Silong Yong Zilong Zheng Qing Li Yitao Liang Song-Chun Zhu Siyuan Huang LM&Ro 72 148 0 14 Oct 2022
Multi-View Transformer for 3D Visual Grounding Shijia Huang Yilun Chen Jiaya Jia Liwei Wang 75 124 0 05 Apr 2022
LoRA: Low-Rank Adaptation of Large Language Models J. E. Hu Yelong Shen Phillip Wallis Zeyuan Allen-Zhu Yuanzhi Li Shean Wang Lu Wang Weizhu Chen OffRL AI4TS AI4CE ALM AIMat 454 10,367 0 17 Jun 2021
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai ... Matthias Minderer G. Heigold Sylvain Gelly Jakob Uszkoreit N. Houlsby ViT 654 41,103 0 22 Oct 2020
ScanRefer: 3D Object Localization in RGB-D Scans using Natural Language Dave Zhenyu Chen Angel X. Chang Matthias Nießner 3DPC 87 370 0 18 Dec 2019
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks Nils Reimers Iryna Gurevych 1.3K 12,226 0 27 Aug 2019
RIO: 3D Object Instance Re-Localization in Changing Indoor Environments Johanna Wald A. Avetisyan Nassir Navab Federico Tombari Matthias Nießner 59 156 0 16 Aug 2019
Matterport3D: Learning from RGB-D Data in Indoor Environments Angel X. Chang Angela Dai Thomas Funkhouser Maciej Halber Matthias Nießner Manolis Savva Shuran Song Andy Zeng Yinda Zhang 3DV 3DPC 183 1,901 0 18 Sep 2017
ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes Angela Dai Angel X. Chang Manolis Savva Maciej Halber Thomas Funkhouser Matthias Nießner 3DPC 3DV 472 4,062 0 14 Feb 2017
CIDEr: Consensus-based Image Description Evaluation Ramakrishna Vedantam C. L. Zitnick Devi Parikh 289 4,488 0 20 Nov 2014