Multimodal Grounding for Embodied AI via Augmented Reality Headsets for Natural Language Driven Task Planning

26 April 2023

Papers citing "Multimodal Grounding for Embodied AI via Augmented Reality Headsets for Natural Language Driven Task Planning"

36 / 36 papers shown

Title
ChatGPT for Robotics: Design Principles and Model Abilities Sai H. Vemprala Rogerio Bonatti A. Bucker Ashish Kapoor LM&Ro 66 472 0 20 Feb 2023
CLIP-Nav: Using CLIP for Zero-Shot Vision-and-Language Navigation Vishnu Sashank Dorbala Gunnar Sigurdsson Robinson Piramuthu Jesse Thomason Gaurav Sukhatme LM&Ro 68 56 0 30 Nov 2022
Retrospectives on the Embodied AI Workshop Matt Deitke Dhruv Batra Yonatan Bisk Tommaso Campari Angel X. Chang ... Jesse Thomason Alexander Toshev Joanne Truong Luca Weihs Jiajun Wu LM&Ro 62 51 0 13 Oct 2022
Ask Me Anything: A simple strategy for prompting language models Simran Arora A. Narayan Mayee F. Chen Laurel J. Orr Neel Guha Kush S. Bhatia Ines Chami Frederic Sala Christopher Ré ReLM LRM 252 215 0 05 Oct 2022
ProgPrompt: Generating Situated Robot Task Plans using Large Language Models Ishika Singh Valts Blukis Arsalan Mousavian Ankit Goyal Danfei Xu Jonathan Tremblay Dieter Fox Jesse Thomason Animesh Garg LM&Ro LLMAG 158 654 0 22 Sep 2022
Inner Monologue: Embodied Reasoning through Planning with Language Models Wenlong Huang F. Xia Ted Xiao Harris Chan Jacky Liang ... Tomas Jackson Linda Luu Sergey Levine Karol Hausman Brian Ichter LLMAG LM&Ro LRM 104 902 0 12 Jul 2022
The Unreliability of Explanations in Few-shot Prompting for Textual Reasoning Xi Ye Greg Durrett ReLM LRM 55 177 0 06 May 2022
PaLM: Scaling Language Modeling with Pathways Aakanksha Chowdhery Sharan Narang Jacob Devlin Maarten Bosma Gaurav Mishra ... Kathy Meier-Hellstern Douglas Eck J. Dean Slav Petrov Noah Fiedel PILM LRM 446 6,222 0 05 Apr 2022
Do As I Can, Not As I Say: Grounding Language in Robotic Affordances Michael Ahn Anthony Brohan Noah Brown Yevgen Chebotar Omar Cortes ... Ted Xiao Peng Xu Sichun Xu Mengyuan Yan Andy Zeng LM&Ro 167 1,951 0 04 Apr 2022
Socratic Models: Composing Zero-Shot Multimodal Reasoning with Language Andy Zeng Maria Attarian Brian Ichter K. Choromanski Adrian S. Wong ... Michael S. Ryoo Vikas Sindhwani Johnny Lee Vincent Vanhoucke Peter R. Florence ReLM LRM 137 582 0 01 Apr 2022
Can Prompt Probe Pretrained Language Models? Understanding the Invisible Risks from a Causal View Boxi Cao Hongyu Lin Xianpei Han Fangchao Liu Le Sun ELM AAML 37 43 0 23 Mar 2022
Visual-Language Navigation Pretraining via Prompt-based Environmental Self-exploration Xiwen Liang Fengda Zhu Lingling Li Hang Xu Xiaodan Liang LM&Ro VLM 50 30 0 08 Mar 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 811 12,893 0 04 Mar 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 763 9,330 0 28 Jan 2022
Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents Wenlong Huang Pieter Abbeel Deepak Pathak Igor Mordatch LM&Ro 81 1,102 0 18 Jan 2022
Simple but Effective: CLIP Embeddings for Embodied AI Apoorv Khandelwal Luca Weihs Roozbeh Mottaghi Aniruddha Kembhavi VLM LM&Ro 74 226 0 18 Nov 2021
On Transferability of Prompt Tuning for Natural Language Processing Yusheng Su Xiaozhi Wang Yujia Qin Chi-Min Chan Yankai Lin ... Peng Li Juanzi Li Lei Hou Maosong Sun Jie Zhou AAML VLM 52 104 0 12 Nov 2021
Text AutoAugment: Learning Compositional Augmentation Policy for Text Classification Shuhuai Ren Jinchao Zhang Lei Li Xu Sun Jie Zhou 50 34 0 01 Sep 2021
Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing Pengfei Liu Weizhe Yuan Jinlan Fu Zhengbao Jiang Hiroaki Hayashi Graham Neubig VLM SyDa 191 3,964 0 28 Jul 2021
Open-vocabulary Object Detection via Vision and Language Knowledge Distillation Xiuye Gu Nayeon Lee Weicheng Kuo Huayu Chen VLM ObjD 267 915 0 28 Apr 2021
MDETR -- Modulated Detection for End-to-End Multi-Modal Understanding Aishwarya Kamath Mannat Singh Yann LeCun Gabriel Synnaeve Ishan Misra Nicolas Carion ObjD VLM 165 881 0 26 Apr 2021
Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity Yao Lu Max Bartolo Alastair Moore Sebastian Riedel Pontus Stenetorp AILaw LRM 394 1,181 0 18 Apr 2021
Robofleet: Open Source Communication and Management for Fleets of Autonomous Robots Kavan Singh Sikand Logan Zartman Sadegh Rabiee Joydeep Biswas 27 18 0 11 Mar 2021
A Survey of Embodied AI: From Simulators to Research Tasks Jiafei Duan Samson Yu Tangyao Li Huaiyu Zhu Cheston Tan LM&Ro 61 286 0 08 Mar 2021
Learning Transferable Visual Models From Natural Language Supervision Alec Radford Jong Wook Kim Chris Hallacy Aditya A. Ramesh Gabriel Goh ... Amanda Askell Pamela Mishkin Jack Clark Gretchen Krueger Ilya Sutskever CLIP VLM 861 29,341 0 26 Feb 2021
The Pile: An 800GB Dataset of Diverse Text for Language Modeling Leo Gao Stella Biderman Sid Black Laurence Golding Travis Hoppe ... Horace He Anish Thite Noa Nabeshima Shawn Presser Connor Leahy AIMat 434 2,091 0 31 Dec 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 731 41,894 0 28 May 2020
ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Tasks Mohit Shridhar Jesse Thomason Daniel Gordon Yonatan Bisk Winson Han Roozbeh Mottaghi Luke Zettlemoyer Dieter Fox LM&Ro 104 768 0 03 Dec 2019
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks Nils Reimers Iryna Gurevych 1.2K 12,181 0 27 Aug 2019
ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks Jiasen Lu Dhruv Batra Devi Parikh Stefan Lee SSL VLM 219 3,674 0 06 Aug 2019
REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environments Yuankai Qi Qi Wu Peter Anderson Xinze Wang Wenjie Wang Chunhua Shen Anton Van Den Hengel LM&Ro 83 323 0 23 Apr 2019
EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks Jason W. Wei Kai Zou 105 1,953 0 31 Jan 2019
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.7K 94,729 0 11 Oct 2018
VirtualHome: Simulating Household Activities via Programs Xavier Puig K. Ra Marko Boben Jiaman Li Tingwu Wang Sanja Fidler Antonio Torralba LM&Ro 88 490 0 19 Jun 2018
Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environments Peter Anderson Qi Wu Damien Teney Jake Bruce Mark Johnson Niko Sünderhauf Ian Reid Stephen Gould Anton Van Den Hengel LM&Ro 95 1,306 0 20 Nov 2017
The LAMBADA dataset: Word prediction requiring a broad discourse context Denis Paperno Germán Kruszewski Angeliki Lazaridou Q. N. Pham Raffaella Bernardi Sandro Pezzelle Marco Baroni Gemma Boleda Raquel Fernández 118 715 0 20 Jun 2016