OrionNav: Online Planning for Robot Autonomy with Context-Aware LLM and Open-Vocabulary Semantic Scene Graphs

8 October 2024

Venkata Naren Devarakonda

Raktim Gautam Goswami

Ali Umut Kaypak

Naman Patel

Rooholla Khorrambakht

Prashanth Krishnamurthy

Farshad Khorrami

LM&Ro

ArXiv PDF HTML

Papers citing "OrionNav: Online Planning for Robot Autonomy with Context-Aware LLM and Open-Vocabulary Semantic Scene Graphs"

32 / 32 papers shown

Title
Multi-Object Hallucination in Vision-Language Models Xuweiyi Chen Ziqiao Ma Xuejun Zhang Sihan Xu Shengyi Qian Jianing Yang David Fouhey Joyce Chai 62 19 0 08 Jul 2024
LLM-Seg: Bridging Image Segmentation and Large Language Model Reasoning Junchi Wang Lei Ke MLLM LRM VLM 58 24 0 12 Apr 2024
ViTamin: Designing Scalable Vision Models in the Vision-Language Era Jienneg Chen Qihang Yu Xiaohui Shen Alan Yuille Liang-Chieh Chen 3DV VLM 83 26 0 02 Apr 2024
GOV-NeSF: Generalizable Open-Vocabulary Neural Semantic Fields Yunsong Wang Hanlin Chen Gim Hee Lee 88 6 0 01 Apr 2024
Hierarchical Open-Vocabulary 3D Scene Graphs for Language-Grounded Robot Navigation Abdelrhman Werby Chen Huang M. Büchner Abhinav Valada Wolfram Burgard 75 78 0 26 Mar 2024
SED: A Simple Encoder-Decoder for Open-Vocabulary Semantic Segmentation Bin Xie Jiale Cao Jin Xie Fahad Shahbaz Khan Yanwei Pang VLM 77 46 0 27 Nov 2023
Open-NeRF: Towards Open Vocabulary NeRF Decomposition Hao Zhang Fang Li Narendra Ahuja 60 12 0 25 Oct 2023
Uni3D: Exploring Unified 3D Representation at Scale Junsheng Zhou Jinsheng Wang Baorui Ma Yu-Shen Liu Tiejun Huang Xinlong Wang 62 96 0 10 Oct 2023
ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planning Yuanyi Zhong Alihusein Kuwajerwala Sacha Morin Krishna Murthy Jatavallabhula Bipasha Sen ... Celso Miguel de Melo Joshua B. Tenenbaum Antonio Torralba Florian Shkurti Liam Paull LM&Ro 96 180 0 28 Sep 2023
LLM-Grounder: Open-Vocabulary 3D Visual Grounding with Large Language Model as an Agent Jianing Yang Xuweiyi Chen Shengyi Qian Nikhil Madaan Madhavan Iyengar David Fouhey Joyce Chai LM&Ro LLMAG 115 94 0 21 Sep 2023
3D Active Metric-Semantic SLAM Yuezhan Tao Xu Liu Igor Spasojevic Saurav Agarwa Vijay Kumar 61 17 0 13 Sep 2023
Convolutions Die Hard: Open-Vocabulary Segmentation with Single Frozen Convolutional CLIP Qihang Yu Ju He XueQing Deng Xiaohui Shen Liang-Chieh Chen VLM CLIP 66 148 0 04 Aug 2023
AGILE3D: Attention Guided Interactive Multi-object 3D Segmentation Yuanwen Yue Sabarinath Mahadevan Jonas Schult Francis Engelmann Bastian Leibe Konrad Schindler Theodora Kontogianni 3DPC 75 29 0 01 Jun 2023
GPT-4 Technical Report OpenAI OpenAI OpenAI Josh Achiam Steven Adler Sandhini Agarwal Lama Ahmad ... Shengjia Zhao Tianhao Zheng Juntang Zhuang William Zhuk Barret Zoph LLMAG MLLM 1.3K 14,289 0 15 Mar 2023
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models Junnan Li Dongxu Li Silvio Savarese Steven C. H. Hoi VLM MLLM 401 4,527 0 30 Jan 2023
Visual Language Maps for Robot Navigation Chen Huang Oier Mees Andy Zeng Wolfram Burgard LM&Ro 228 362 0 11 Oct 2022
ProgPrompt: Generating Situated Robot Task Plans using Large Language Models Ishika Singh Valts Blukis Arsalan Mousavian Ankit Goyal Danfei Xu Jonathan Tremblay Dieter Fox Jesse Thomason Animesh Garg LM&Ro LLMAG 156 654 0 22 Sep 2022
Do As I Can, Not As I Say: Grounding Language in Robotic Affordances Michael Ahn Anthony Brohan Noah Brown Yevgen Chebotar Omar Cortes ... Ted Xiao Peng Xu Sichun Xu Mengyuan Yan Andy Zeng LM&Ro 165 1,946 0 04 Apr 2022
A ConvNet for the 2020s Zhuang Liu Hanzi Mao Chaozheng Wu Christoph Feichtenhofer Trevor Darrell Saining Xie ViT 148 5,158 0 10 Jan 2022
Masked-attention Mask Transformer for Universal Image Segmentation Bowen Cheng Ishan Misra Alex Schwing Alexander Kirillov Rohit Girdhar ISeg 202 2,355 0 02 Dec 2021
SceneGraphFusion: Incremental 3D Scene Graph Prediction from RGB-D Sequences Shun-cheng Wu Johanna Wald Keisuke Tateno Nassir Navab Federico Tombari 3DPC 46 160 0 27 Mar 2021
A Survey of Embodied AI: From Simulators to Research Tasks Jiafei Duan Samson Yu Tangyao Li Huaiyu Zhu Cheston Tan LM&Ro 61 286 0 08 Mar 2021
Learning Transferable Visual Models From Natural Language Supervision Alec Radford Jong Wook Kim Chris Hallacy Aditya A. Ramesh Gabriel Goh ... Amanda Askell Pamela Mishkin Jack Clark Gretchen Krueger Ilya Sutskever CLIP VLM 828 29,341 0 26 Feb 2021
Kimera: from SLAM to Spatial Perception with 3D Dynamic Scene Graphs Antoni Rosinol Andrew Violette Marcus Abate Nathan Hughes Yun Chang Jingang Shi Arjun Gupta Luca Carlone 3DV 79 235 0 18 Jan 2021
3D Dynamic Scene Graphs: Actionable Spatial Perception with Places, Objects, and Humans Antoni Rosinol Arjun Gupta Marcus Abate Jingang Shi Luca Carlone 79 194 0 15 Feb 2020
3D Scene Graph: A Structure for Unified Semantics, 3D Space, and Camera Iro Armeni Zhi-Yang He JunYoung Gwak Amir Zamir Martin Fischer Jitendra Malik Silvio Savarese 3DV 3DPC 89 345 0 06 Oct 2019
MID-Fusion: Octree-based Object-Level Multi-Instance Dynamic SLAM Binbin Xu Wenbin Li Dimos Tzoumanikas Michael Bloesch Andrew J. Davison Stefan Leutenegger 80 189 0 19 Dec 2018
Attentive Relational Networks for Mapping Images to Scene Graphs Mengshi Qi Weijian Li Zhengyuan Yang Yunhong Wang Jiebo Luo 3DPC 3DH GNN 55 169 0 26 Nov 2018
MaskFusion: Real-Time Recognition, Tracking and Reconstruction of Multiple Moving Objects Martin Rünz Maud Buffier Lourdes Agapito 96 351 0 24 Apr 2018
COCO-Stuff: Thing and Stuff Classes in Context Holger Caesar J. Uijlings V. Ferrari 116 1,385 0 12 Dec 2016
Neural Machine Translation of Rare Words with Subword Units Rico Sennrich Barry Haddow Alexandra Birch 204 7,734 0 31 Aug 2015
Microsoft COCO: Common Objects in Context Nayeon Lee Michael Maire Serge J. Belongie Lubomir Bourdev Ross B. Girshick James Hays Pietro Perona Deva Ramanan C. L. Zitnick Piotr Dollár ObjD 389 43,619 0 01 May 2014