Learning Language-Conditioned Robot Behavior from Offline Data and Crowd-Sourced Annotation

2 September 2021

Silvio Savarese

Papers citing "Learning Language-Conditioned Robot Behavior from Offline Data and Crowd-Sourced Annotation"

50 / 114 papers shown

Title
Kinematic-aware Prompting for Generalizable Articulated Object Manipulation with LLMs Wenke Xia Dong Wang Xincheng Pang Zhigang Wang Bin Zhao Di Hu Xuelong Li LM&Ro 32 19 0 06 Nov 2023
RT-Trajectory: Robotic Task Generalization via Hindsight Trajectory Sketches Jiayuan Gu Sean Kirmani Paul Wohlhart Yao Lu Montse Gonzalez Arenas ... Hao Su Karol Hausman Chelsea Finn Q. Vuong Ted Xiao 28 62 0 03 Nov 2023
Open X-Embodiment: Robotic Learning Datasets and RT-X Models Open X-Embodiment Collaboration Abby OÑeill Abdul Rehman Abhinav Gupta Abhiram Maddukuri ... Zhuo Xu Zichen Jeff Cui Zichen Zhang Zipeng Fu Zipeng Lin LM&Ro 42 460 0 13 Oct 2023
GEAR: A GPU-Centric Experience Replay System for Large Reinforcement Learning Models Hanjing Wang Man-Kit Sit Cong He Ying Wen Weinan Zhang Jun Wang Yaodong Yang Luo Mai OffRL VLM 27 1 0 08 Oct 2023
LanguageMPC: Large Language Models as Decision Makers for Autonomous Driving Hao Sha Yao Mu Yuxuan Jiang Li Chen Chenfeng Xu Ping Luo Shengbo Eben Li Masayoshi Tomizuka Wei Zhan Mingyu Ding 114 159 0 04 Oct 2023
MUTEX: Learning Unified Policies from Multimodal Task Specifications Rutav Shah Roberto Martín-Martín Yuke Zhu OffRL 44 54 0 25 Sep 2023
Improve the efficiency of deep reinforcement learning through semantic exploration guided by natural language Zhourui Guo Meng Yao Yang Yu Qiyue Yin OnRL 28 1 0 21 Sep 2023
Q-Transformer: Scalable Offline Reinforcement Learning via Autoregressive Q-Functions Yevgen Chebotar Q. Vuong A. Irpan Karol Hausman F. Xia ... Brianna Zitkovich Tomas Jackson Kanishka Rao Chelsea Finn Sergey Levine OffRL 126 81 0 18 Sep 2023
LEMMA: Learning Language-Conditioned Multi-Robot Manipulation Ran Gong Xiaofeng Gao Qiaozi Gao Suhaila Shakiah Govind Thattai Gaurav Sukhatme LM&Ro 16 8 0 02 Aug 2023
Learning to Model the World with Language Jessy Lin Yuqing Du Olivia Watkins Danijar Hafner Pieter Abbeel Dan Klein Anca Dragan LM&Ro SyDa 26 50 0 31 Jul 2023
RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control Anthony Brohan Noah Brown Justice Carbajal Yevgen Chebotar Xi Chen ... Ted Xiao Peng-Tao Xu Sichun Xu Tianhe Yu Brianna Zitkovich LM&Ro LRM 30 1,099 0 28 Jul 2023
Scaling Up and Distilling Down: Language-Guided Robot Skill Acquisition Huy Ha Peter R. Florence Shuran Song LM&Ro 41 150 0 26 Jul 2023
VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models Wenlong Huang Chen Wang Ruohan Zhang Yunzhu Li Jiajun Wu Li Fei-Fei LM&Ro 33 479 0 12 Jul 2023
DoReMi: Grounding Language Model by Detecting and Recovering from Plan-Execution Misalignment Yanjiang Guo Yen-Jen Wang Lihan Zha Zheyuan Jiang Jianyu Chen LM&Ro 24 39 0 01 Jul 2023
Goal Representations for Instruction Following: A Semi-Supervised Language Interface to Control Vivek Myers Andre Wang He Kuan Fang Homer Walke Philippe Hansen-Estruch Ching-An Cheng Mihai Jalobeanu Andrey Kolobov Anca Dragan Sergey Levine LM&Ro 22 29 0 30 Jun 2023
RVT: Robotic View Transformer for 3D Object Manipulation Ankit Goyal Jie Xu Yijie Guo Valts Blukis Yu-Wei Chao D. Fox LM&Ro 42 117 0 26 Jun 2023
Towards Language-Based Modulation of Assistive Robots through Multimodal Models Philipp Wicke Lufti Kerem cSenel Shengqiang Zhang Luis F. C. Figueredo Abdeldjallil Naceri Sami Haddadin Hinrich Schütze 20 2 0 26 Jun 2023
Improving Long-Horizon Imitation Through Instruction Prediction Joey Hejna Pieter Abbeel Lerrel Pinto 19 7 0 21 Jun 2023
Language to Rewards for Robotic Skill Synthesis Wenhao Yu Nimrod Gileadi Chuyuan Fu Sean Kirmani Kuang-Huei Lee ... N. Heess Dorsa Sadigh Jie Tan Yuval Tassa F. Xia LM&Ro 37 269 0 14 Jun 2023
LIV: Language-Image Representations and Rewards for Robotic Control Yecheng Jason Ma William Liang Vaidehi Som Vikash Kumar Amy Zhang Osbert Bastani Dinesh Jayaraman LM&Ro 33 121 0 01 Jun 2023
Language-Conditioned Imitation Learning with Base Skill Priors under Unstructured Data Hongkuan Zhou Zhenshan Bing Xiangtong Yao Xiaojie Su Chenguang Yang Kai-Qi Huang Alois C. Knoll LM&Ro 41 18 0 30 May 2023
Demo2Code: From Summarizing Demonstrations to Synthesizing Code via Extended Chain-of-Thought Huaxiaoyue Wang Gonzalo Gonzalez-Pumariega Yash Sharma Sanjiban Choudhury LM&Ro 26 33 0 26 May 2023
Learning Video-Conditioned Policies for Unseen Manipulation Tasks Elliot Chane-Sane Cordelia Schmid Ivan Laptev 27 18 0 10 May 2023
Plan, Eliminate, and Track -- Language Models are Good Teachers for Embodied Agents Yu-Chih Chen So Yeon Min Chase Davis Ruslan Salakhutdinov A. Azaria Yuan-Fang Li Tom Michael Mitchell A. Bovik LM&Ro LLMAG 75 32 0 03 May 2023
Contrastive Language, Action, and State Pre-training for Robot Learning Krishan Rana Andrew Melnik Niko Sünderhauf 20 12 0 21 Apr 2023
EC^2: Emergent Communication for Embodied Control Yao Mu Shunyu Yao Mingyu Ding Ping Luo Chuang Gan LM&Ro 31 19 0 19 Apr 2023
ARNOLD: A Benchmark for Language-Grounded Task Learning With Continuous States in Realistic 3D Scenes Ran Gong Jiangyong Huang Yizhou Zhao Haoran Geng Xiaofeng Gao ... Ziheng Zhou D. Terzopoulos Song-Chun Zhu Baoxiong Jia Siyuan Huang LM&Ro 42 45 0 09 Apr 2023
ERRA: An Embodied Representation and Reasoning Architecture for Long-horizon Language-conditioned Manipulation Tasks Chao Zhao Shuai Yuan Chunli Jiang Junhao Cai Hongyu Yu M. Y. Wang Qifeng Chen LM&Ro 29 14 0 05 Apr 2023
Grounding Object Relations in Language-Conditioned Robotic Manipulation with Semantic-Spatial Reasoning Qian Luo Yunfei Li Yi Wu LM&Ro 40 5 0 31 Mar 2023
Did You Mean...? Confidence-based Trade-offs in Semantic Parsing Elias Stengel-Eskin Benjamin Van Durme 21 5 0 29 Mar 2023
Foundation Models for Decision Making: Problems, Methods, and Opportunities Sherry Yang Ofir Nachum Yilun Du Jason W. Wei Pieter Abbeel Dale Schuurmans LM&Ro OffRL LRM AI4CE 90 155 0 07 Mar 2023
PaLM-E: An Embodied Multimodal Language Model Danny Driess F. Xia Mehdi S. M. Sajjadi Corey Lynch Aakanksha Chowdhery ... Marc Toussaint Klaus Greff Andy Zeng Igor Mordatch Peter R. Florence LM&Ro 22 1,565 0 06 Mar 2023
Learning Language-Conditioned Deformable Object Manipulation with Graph Dynamics Yuhong Deng Kaichun Mo Chongkun Xia Xueqian Wang AI4CE 36 11 0 02 Mar 2023
Preference Transformer: Modeling Human Preferences using Transformers for RL Changyeon Kim Jongjin Park Jinwoo Shin Honglak Lee Pieter Abbeel Kimin Lee OffRL 30 61 0 02 Mar 2023
Open-World Object Manipulation using Pre-trained Vision-Language Models Austin Stone Ted Xiao Yao Lu K. Gopalakrishnan Kuang-Huei Lee ... Sean Kirmani Brianna Zitkovich F. Xia Chelsea Finn Karol Hausman LM&Ro 150 144 0 02 Mar 2023
Grounded Decoding: Guiding Text Generation with Grounded Models for Embodied Agents Wenlong Huang Fei Xia Dhruv Shah Danny Driess Andy Zeng ... Pete Florence Igor Mordatch Sergey Levine Karol Hausman Brian Ichter LM&Ro 21 41 0 01 Mar 2023
Semantic Mechanical Search with Large Vision and Language Models Satvik Sharma Huang Huang K. Shivakumar A. Imran Ryan Hoque Brian Ichter Ken Goldberg LM&Ro VLM 21 5 0 24 Feb 2023
Language-Driven Representation Learning for Robotics Siddharth Karamcheti Suraj Nair Annie S. Chen Thomas Kollar Chelsea Finn Dorsa Sadigh Percy Liang LM&Ro SSL 41 145 0 24 Feb 2023
Temporal Video-Language Alignment Network for Reward Shaping in Reinforcement Learning Ziyuan Cao Reshma A. Ramachandra K. Yu 20 2 0 08 Feb 2023
Understanding the Complexity Gains of Single-Task RL with a Curriculum Qiyang Li Yuexiang Zhai Yi-An Ma Sergey Levine 37 14 0 24 Dec 2022
Benchmarking Spatial Relationships in Text-to-Image Generation Tejas Gokhale Hamid Palangi Besmira Nushi Vibhav Vineet Eric Horvitz Ece Kamar Chitta Baral Yezhou Yang EGVM 40 66 0 20 Dec 2022
RT-1: Robotics Transformer for Real-World Control at Scale Anthony Brohan Noah Brown Justice Carbajal Yevgen Chebotar Joseph Dabis ... Ted Xiao Peng-Tao Xu Sichun Xu Tianhe Yu Brianna Zitkovich LM&Ro 28 1,021 0 13 Dec 2022
Robotic Skill Acquisition via Instruction Augmentation with Vision-Language Models Ted Xiao Harris Chan P. Sermanet Ayzaan Wahid Anthony Brohan Karol Hausman Sergey Levine Jonathan Tompson VLM LM&Ro 30 65 0 21 Nov 2022
Calibrated Interpretation: Confidence Estimation in Semantic Parsing Elias Stengel-Eskin Benjamin Van Durme UQLM 41 24 0 14 Nov 2022
StructDiffusion: Language-Guided Creation of Physically-Valid Structures using Unseen Objects Weiyu Liu Yilun Du Tucker Hermans Sonia Chernova Chris Paxton DiffM 23 53 0 08 Nov 2022
Instruction-Following Agents with Multimodal Transformer Hao Liu Lisa Lee Kimin Lee Pieter Abbeel LM&Ro 30 10 0 24 Oct 2022
Interactive Language: Talking to Robots in Real Time Corey Lynch Ayzaan Wahid Jonathan Tompson Tianli Ding James Betker Robert Baruch Travis Armstrong Peter R. Florence LM&Ro 35 214 0 12 Oct 2022
Generating Executable Action Plans with Environmentally-Aware Language Models Maitrey Gramopadhye D. Szafir LM&Ro LLMAG 15 22 0 10 Oct 2022
Using Both Demonstrations and Language Instructions to Efficiently Learn Robotic Tasks Albert Yu Raymond J. Mooney LM&Ro 29 19 0 10 Oct 2022
Grounding Language with Visual Affordances over Unstructured Data Oier Mees Jessica Borja-Diaz Wolfram Burgard LM&Ro 121 108 0 04 Oct 2022