Learning Language-Conditioned Robot Behavior from Offline Data and Crowd-Sourced Annotation

2 September 2021

Silvio Savarese

Papers citing "Learning Language-Conditioned Robot Behavior from Offline Data and Crowd-Sourced Annotation"

50 / 114 papers shown

Title
CrayonRobo: Object-Centric Prompt-Driven Vision-Language-Action Model for Robotic Manipulation Xiaoqi Li Lingyun Xu M. Zhang Jiaming Liu Yan Shen ... Jiahui Xu Liang Heng Siyuan Huang S. Zhang Hao Dong LM&Ro 48 0 0 04 May 2025
Joint Action Language Modelling for Transparent Policy Execution Theodor Wulff R. S. Maharjan Xinyun Chi Angelo Cangelosi 27 0 0 14 Apr 2025
Data-Agnostic Robotic Long-Horizon Manipulation with Vision-Language-Guided Closed-Loop Feedback Y. Meng Xiangtong Yao Haihui Ye Yirui Zhou Shengqiang Zhang Zhenshan Bing Alois C. Knoll LM&Ro VLM 55 0 0 27 Mar 2025
Contrastive Learning from Exploratory Actions: Leveraging Natural Interactions for Preference Elicitation N. Dennler S. Nikolaidis Maja J. Matarić 133 0 0 03 Jan 2025
Incremental Learning of Retrievable Skills For Efficient Continual Task Adaptation Daehee Lee Minjong Yoo Woo Kyung Kim Wonje Choi Honguk Woo CLL 93 3 0 30 Oct 2024
Discovering Robotic Interaction Modes with Discrete Representation Learning Liquan Wang Ankit Goyal Haoping Xu Animesh Garg 31 0 0 26 Oct 2024
GHIL-Glue: Hierarchical Control with Filtered Subgoal Images Kyle Hatch Ashwin Balakrishna Oier Mees Suraj Nair Seohong Park ... Masha Itkina Benjamin Eysenbach Sergey Levine Thomas Kollar Benjamin Burchfiel 56 2 0 26 Oct 2024
VL-GLUE: A Suite of Fundamental yet Challenging Visuo-Linguistic Reasoning Tasks Shailaja Keyur Sampat Mutsumi Nakamura Shankar Kailas Kartik Aggarwal Mandy Zhou Yezhou Yang Chitta Baral MLLM CoGe ReLM VLM LRM 32 0 0 17 Oct 2024
Steering Your Generalists: Improving Robotic Foundation Models via Value Guidance Mitsuhiko Nakamoto Oier Mees Aviral Kumar Sergey Levine OffRL 73 12 0 17 Oct 2024
CoPESD: A Multi-Level Surgical Motion Dataset for Training Large Vision-Language Models to Co-Pilot Endoscopic Submucosal Dissection Guankun Wang Han Xiao Huxin Gao Renrui Zhang Long Bai Xiaoxiao Yang Zhen Li Hongsheng Li Hongliang Ren 36 4 0 10 Oct 2024
Robo-MUTUAL: Robotic Multimodal Task Specification via Unimodal Learning Jianxiong Li Zhihao Wang Jinliang Zheng Xiaoai Zhou Guanming Wang ... Yu Liu Jingjing Liu Ya-Qin Zhang Junzhi Yu Xianyuan Zhan 38 2 0 02 Oct 2024
Robotic-CLIP: Fine-tuning CLIP on Action Data for Robotic Applications Nghia Nguyen Minh Nhat Vu Tung D. Ta Baoru Huang T. Vo Ngan Le Anh Nguyen VLM CLIP 41 3 0 26 Sep 2024
HiRT: Enhancing Robotic Control with Hierarchical Robot Transformers Jianke Zhang Yanjiang Guo Xiaoyu Chen Yen-Jen Wang Yucheng Hu Chengming Shi Jianyu Chen 29 5 0 12 Sep 2024
Robot Utility Models: General Policies for Zero-Shot Deployment in New Environments Haritheja Etukuru Norihito Naka Zijin Hu Seungjae Lee Julian Mehu Aaron Edsinger Chris Paxton Soumith Chintala Lerrel Pinto Nur Muhammad (Mahi) Shafiullah LM&Ro 31 23 0 09 Sep 2024
Contrast, Imitate, Adapt: Learning Robotic Skills From Raw Human Videos Zhifeng Qian Mingyu You Hongjun Zhou Xuanhui Xu Hao Fu Jinzhe Xue Bin He 26 0 0 10 Aug 2024
Autonomous Improvement of Instruction Following Skills via Foundation Models Zhiyuan Zhou P. Atreya Abraham Lee Homer Walke Oier Mees Sergey Levine 30 10 0 30 Jul 2024
Adapt2Reward: Adapting Video-Language Models to Generalizable Robotic Rewards via Failure Prompts Yanting Yang Minghao Chen Qibo Qiu Jiahao Wu Wenxiao Wang Binbin Lin Ziyu Guan Xiaofei He LM&Ro 39 2 0 20 Jul 2024
VLMPC: Vision-Language Model Predictive Control for Robotic Manipulation Wentao Zhao Jiaming Chen Ziyu Meng Donghui Mao Ran Song Wei Zhang 43 8 0 13 Jul 2024
RoboUniView: Visual-Language Model with Unified View Representation for Robotic Manipulaiton Fanfan Liu Feng Yan Liming Zheng Chengjian Feng Yiyang Huang Lin Ma LM&Ro 27 11 0 27 Jun 2024
Lifelong Robot Library Learning: Bootstrapping Composable and Generalizable Skills for Embodied Control with Language Models Georgios Tziafas H. Kasaei KELM LM&Ro 43 8 0 26 Jun 2024
OCALM: Object-Centric Assessment with Language Models Timo Kaufmann Jannis Blüml Antonia Wüst Quentin Delfosse Kristian Kersting Eyke Hüllermeier LM&Ro LRM 39 1 0 24 Jun 2024
LGR2: Language Guided Reward Relabeling for Accelerating Hierarchical Reinforcement Learning Utsav Singh Pramit Bhattacharyya Vinay P. Namboodiri LM&Ro 37 1 0 09 Jun 2024
A Survey of Language-Based Communication in Robotics William Hunt Sarvapali D. Ramchurn Mohammad D. Soorati LM&Ro 54 12 0 06 Jun 2024
Cross-Embodiment Robot Manipulation Skill Transfer using Latent Space Alignment Tianyu Wang Dwait Bhatt Xiaolong Wang Nikolay A. Atanasov 45 5 0 04 Jun 2024
Video-Language Critic: Transferable Reward Functions for Language-Conditioned Robotics Minttu Alakuijala Reginald McLean Isaac Woungang Nariman Farsad Samuel Kaski Pekka Marttinen Kai Yuan LM&Ro 34 0 0 30 May 2024
Interpretable Robotic Manipulation from Language Boyuan Zheng Jianlong Zhou Fang Chen LM&Ro 37 0 0 27 May 2024
VICtoR: Learning Hierarchical Vision-Instruction Correlation Rewards for Long-horizon Manipulation Kuo-Han Hung Pang-Chi Lo Jia-Fong Yeh Han-Yuan Hsu Yi-Ting Chen Winston H. Hsu 30 0 0 26 May 2024
Natural Language Can Help Bridge the Sim2Real Gap Albert Yu Adeline Foote Raymond J. Mooney Roberto Martín-Martín LM&Ro 43 11 0 16 May 2024
Learning Planning Abstractions from Language Weiyu Liu Geng Chen Joy Hsu Jiayuan Mao Jiajun Wu PINN 43 2 0 06 May 2024
Composing Pre-Trained Object-Centric Representations for Robotics From "What" and "Where" Foundation Models Junyao Shi Jianing Qian Yecheng Jason Ma Dinesh Jayaraman OCL 32 4 0 20 Apr 2024
Retrieval-Augmented Embodied Agents Yichen Zhu Zhicai Ou Xiaofeng Mou Jian Tang 41 17 0 17 Apr 2024
Language, Environment, and Robotic Navigation Johnathan E. Avery LM&Ro 32 0 0 03 Apr 2024
RH20T-P: A Primitive-Level Robotic Dataset Towards Composable Generalization Agents Zeren Chen Zhelun Shi Xiaoya Lu Lehan He Sucheng Qian ... Zhen-fei Yin Jing Shao Jing Shao Cewu Lu Cewu Lu 38 5 0 28 Mar 2024
Scaling Vision-and-Language Navigation With Offline RL Valay Bundele Mahesh Bhupati Biplab Banerjee Aditya Grover OffRL 29 1 0 27 Mar 2024
DecisionNCE: Embodied Multimodal Representations via Implicit Preference Learning Jianxiong Li Jinliang Zheng Yinan Zheng Liyuan Mao Xiaoming Hu ... Jihao Liu Yu Liu Jingjing Liu Ya-Qin Zhang Xianyuan Zhan LM&Ro OffRL 35 8 0 28 Feb 2024
Rethinking Mutual Information for Language Conditioned Skill Discovery on Imitation Learning Zhaoxun Ju Chao Yang Hongbo Wang Yu Qiao Fuchun Sun LM&Ro 28 3 0 27 Feb 2024
SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities Boyuan Chen Zhuo Xu Sean Kirmani Brian Ichter Danny Driess Pete Florence Dorsa Sadigh Leonidas J. Guibas Fei Xia LRM ReLM 44 205 0 22 Jan 2024
Visual Robotic Manipulation with Depth-Aware Pretraining Wanying Wang Jinming Li Yichen Zhu Zhiyuan Xu Zhengping Che Yaxin Peng Chaomin Shen Dong Liu Feifei Feng Jian Tang MDE 24 3 0 17 Jan 2024
Object-Centric Instruction Augmentation for Robotic Manipulation Junjie Wen Yichen Zhu Minjie Zhu Jinming Li Zhiyuan Xu ... Chaomin Shen Yaxin Peng Dong Liu Feifei Feng Jian Tang LM&Ro 62 16 0 05 Jan 2024
SkillDiffuser: Interpretable Hierarchical Planning via Skill Abstractions in Diffusion-Based Task Execution Zhixuan Liang Yao Mu Hengbo Ma Masayoshi Tomizuka Mingyu Ding Ping Luo 33 37 0 18 Dec 2023
Learning adaptive planning representations with natural language guidance L. Wong Jiayuan Mao Pratyusha Sharma Zachary S. Siegel Jiahai Feng Noa Korneev Joshua B. Tenenbaum Jacob Andreas LM&Ro 28 21 0 13 Dec 2023
Foundation Models in Robotics: Applications, Challenges, and the Future Roya Firoozi Johnathan Tucker Stephen Tian Anirudha Majumdar Jiankai Sun ... Brian Ichter Danny Driess Jiajun Wu Cewu Lu Mac Schwager LM&Ro AI4CE LRM VLM 35 139 0 13 Dec 2023
DiffVL: Scaling Up Soft Body Manipulation using Vision-Language Driven Differentiable Physics Zhiao Huang Feng Chen Yewen Pu Chun-Tse Lin Hao Su Chuang Gan 22 4 0 11 Dec 2023
PlayFusion: Skill Acquisition via Diffusion from Language-Annotated Play Lili Chen Shikhar Bahl Deepak Pathak 22 41 0 07 Dec 2023
LaMPilot: An Open Benchmark Dataset for Autonomous Driving with Language Model Programs Yunsheng Ma Can Cui Xu Cao Wenqian Ye Peiran Liu ... Rohit Gupta Kyungtae Han Aniket Bera James M. Rehg Ziran Wang 21 42 0 07 Dec 2023
FoMo Rewards: Can we cast foundation models as reward functions? Ekdeep Singh Lubana Johann Brehmer P. D. Haan Taco S. Cohen OffRL LRM 45 2 0 06 Dec 2023
RobotGPT: Robot Manipulation Learning from ChatGPT Yixiang Jin Dingzhe Li Yong A Jun Shi Peng Hao Fuchun Sun Jianwei Zhang Bin Fang 22 46 0 03 Dec 2023
Transfer Learning in Robotics: An Upcoming Breakthrough? A Review of Promises and Challenges Noémie Jaquier Michael C. Welle A. Gams Kunpeng Yao Bernardo Fichera A. Billard Aleš Ude Tamim Asfour Danica Kragic 25 14 0 29 Nov 2023
A Survey on Multimodal Large Language Models for Autonomous Driving Can Cui Yunsheng Ma Xu Cao Wenqian Ye Yang Zhou ... Xinrui Yan Shuqi Mei Jianguo Cao Ziran Wang Chao Zheng 41 249 0 21 Nov 2023
CLIP-Motion: Learning Reward Functions for Robotic Actions Using Consecutive Observations Xuzhe Dang Stefan Edelkamp 35 4 0 06 Nov 2023