Learning Generalizable Robotic Reward Functions from "In-The-Wild" Human Videos

31 March 2021

Papers citing "Learning Generalizable Robotic Reward Functions from "In-The-Wild" Human Videos"

48 / 48 papers shown

Title
Augmented Reality for RObots (ARRO): Pointing Visuomotor Policies Towards Visual Robustness Reihaneh Mirjalili Tobias Jülg Florian Walter Wolfram Burgard 32 0 0 13 May 2025
Pixel Motion as Universal Representation for Robot Control Kanchana Ranasinghe Xiang Li Cristina Mata J. Park Michael S. Ryoo VGen 32 0 0 12 May 2025
Chain-of-Modality: Learning Manipulation Programs from Multimodal Human Videos with Vision-Language-Models Chen Wang Fei Xia Wenhao Yu Tingnan Zhang Ruohan Zhang Ce Liu Li Fei-Fei Jie Tan Jacky Liang 36 0 0 17 Apr 2025
VidBot: Learning Generalizable 3D Actions from In-the-Wild 2D Human Videos for Zero-Shot Robotic Manipulation Hanzhi Chen Boyang Sun Anran Zhang Marc Pollefeys Stefan Leutenegger LM&Ro 75 0 0 10 Mar 2025
Human2Robot: Learning Robot Actions from Paired Human-Robot Videos Sicheng Xie Haidong Cao Zejia Weng Zhen Xing Shiwei Shen Jiaqi Leng Xipeng Qiu Yanwei Fu Zuxuan Wu Yu Jiang 56 0 0 23 Feb 2025
CityWalker: Learning Embodied Urban Navigation from Web-Scale Videos Xinhao Liu Jiajian Li Yichen Jiang Niranjan Sujay Zhengyuan Yang Juexiao Zhang John Abanes Jing Zhang Chen Feng 114 1 0 26 Nov 2024
Grounding Video Models to Actions through Goal Conditioned Exploration Yunhao Luo Yilun Du LM&Ro VGen 85 1 0 11 Nov 2024
VideoAgent: Self-Improving Video Generation Achint Soni Sreyas Venkataraman Abhranil Chandra Sebastian Fischmeister Percy Liang Bo Dai Sherry Yang LM&Ro VGen 58 7 0 14 Oct 2024
Learning Wheelchair Tennis Navigation from Broadcast Videos with Domain Knowledge Transfer and Diffusion Motion Planning Zixuan Wu Z. Zaidi Adithya Patil Qingyu Xiao Matthew C. Gombolay 75 1 0 29 Sep 2024
DynaMo: In-Domain Dynamics Pretraining for Visuo-Motor Control Zichen Jeff Cui Hengkai Pan Aadhithya Iyer Siddhant Haldar Lerrel Pinto VGen 36 10 0 18 Sep 2024
One-Shot Imitation under Mismatched Execution Kushal Kedia Prithwish Dan Sanjiban Choudhury Maximus Adrian Pace Sanjiban Choudhury 40 2 0 10 Sep 2024
Leveraging Unlabeled Data Sharing through Kernel Function Approximation in Offline Reinforcement Learning Yen-Ru Lai Fu-Chieh Chang Pei-Yuan Wu OffRL 79 1 0 22 Aug 2024
Adapt2Reward: Adapting Video-Language Models to Generalizable Robotic Rewards via Failure Prompts Yanting Yang Minghao Chen Qibo Qiu Jiahao Wu Wenxiao Wang Binbin Lin Ziyu Guan Xiaofei He LM&Ro 45 2 0 20 Jul 2024
R+X: Retrieval and Execution from Everyday Human Videos Georgios Papagiannis Norman Di Palo Pietro Vitiello Edward Johns 59 16 0 17 Jul 2024
Dreamitate: Real-World Visuomotor Policy Learning via Video Generation Junbang Liang Ruoshi Liu Ege Ozguroglu Sruthi Sudhakar Achal Dave P. Tokmakov Shuran Song Carl Vondrick VGen 42 23 0 24 Jun 2024
RILe: Reinforced Imitation Learning Mert Albaba Sammy Christen Christoph Gebhardt Thomas Langarek Otmar Hilliges Otmar Hilliges 50 1 0 12 Jun 2024
Video-Language Critic: Transferable Reward Functions for Language-Conditioned Robotics Minttu Alakuijala Reginald McLean Isaac Woungang Nariman Farsad Samuel Kaski Pekka Marttinen Kai Yuan LM&Ro 42 1 0 30 May 2024
VICtoR: Learning Hierarchical Vision-Instruction Correlation Rewards for Long-horizon Manipulation Kuo-Han Hung Pang-Chi Lo Jia-Fong Yeh Han-Yuan Hsu Yi-Ting Chen Winston H. Hsu 33 0 0 26 May 2024
Rank2Reward: Learning Shaped Reward Functions from Passive Video Daniel Yang Davin Tjia Jacob Berg Dima Damen Pulkit Agrawal Abhishek Gupta OffRL 40 5 0 23 Apr 2024
Human Demonstrations are Generalizable Knowledge for Robots Te Cui Guangyan Chen Tianxing Zhou Zicai Peng Mengxiao Hu Haoyang Lu Haizhou Li Meiling Wang Yi Yang Yufeng Yue LM&Ro 39 6 0 05 Dec 2023
AirExo: Low-Cost Exoskeletons for Learning Whole-Arm Manipulation in the Wild Hongjie Fang Haoshu Fang Yiming Wang Jieji Ren Jing Chen Ruo Zhang Weiming Wang Cewu Lu 39 48 0 26 Sep 2023
Language Reward Modulation for Pretraining Reinforcement Learning Ademi Adeniji Amber Xie Carmelo Sferrazza Younggyo Seo Stephen James Pieter Abbeel 39 26 0 23 Aug 2023
Structured World Models from Human Videos Russell Mendonca Shikhar Bahl Deepak Pathak LM&Ro 49 87 0 21 Aug 2023
Pre-training Contextualized World Models with In-the-wild Videos for Reinforcement Learning Jialong Wu Haoyu Ma Chao Deng Mingsheng Long OffRL 31 24 0 29 May 2023
Affordances from Human Videos as a Versatile Representation for Robotics Shikhar Bahl Russell Mendonca Lili Chen Unnat Jain Deepak Pathak 44 164 0 17 Apr 2023
Reward Design with Language Models Minae Kwon Sang Michael Xie Kalesha Bullard Dorsa Sadigh LM&Ro 41 201 0 27 Feb 2023
Language-Driven Representation Learning for Robotics Siddharth Karamcheti Suraj Nair Annie S. Chen Thomas Kollar Chelsea Finn Dorsa Sadigh Percy Liang LM&Ro SSL 47 145 0 24 Feb 2023
MimicPlay: Long-Horizon Imitation Learning by Watching Human Play Chen Wang Linxi Fan Jiankai Sun Ruohan Zhang Li Fei-Fei Danfei Xu Yuke Zhu Anima Anandkumar 44 184 0 24 Feb 2023
ALAN: Autonomously Exploring Robotic Agents in the Real World Russell Mendonca Shikhar Bahl Deepak Pathak LM&Ro 36 20 0 13 Feb 2023
SIRL: Similarity-based Implicit Representation Learning Andreea Bobu Yi Liu Rohin Shah Daniel S. Brown Anca Dragan SSL DRL 35 17 0 02 Jan 2023
Cross-Domain Transfer via Semantic Skill Imitation Karl Pertsch Ruta Desai Vikash Kumar Franziska Meier Joseph J. Lim Dhruv Batra Akshara Rai LM&Ro 16 18 0 14 Dec 2022
VideoDex: Learning Dexterity from Internet Videos Kenneth Shaw Shikhar Bahl Deepak Pathak 30 89 0 08 Dec 2022
Learning Reward Functions for Robotic Manipulation by Observing Humans Minttu Alakuijala Gabriel Dulac-Arnold Julien Mairal Jean Ponce Cordelia Schmid OffRL 37 26 0 16 Nov 2022
DALL-E-Bot: Introducing Web-Scale Diffusion Models to Robotics Ivan Kapelyukh Vitalis Vosylius Edward Johns LM&Ro DiffM 113 146 0 05 Oct 2022
Extraneousness-Aware Imitation Learning Rachel Zheng Kaizhe Hu Zhecheng Yuan Boyuan Chen Huazhe Xu SSL 33 3 0 04 Oct 2022
Learning Object Manipulation Skills from Video via Approximate Differentiable Physics Vladimir Petrik M. N. Qureshi Josef Sivic Makarand Tapaswi 41 9 0 03 Aug 2022
Graph Inverse Reinforcement Learning from Diverse Videos Sateesh Kumar Jonathan Zamora Nicklas Hansen Rishabh Jangir Xiaolong Wang 35 53 0 28 Jul 2022
Human-to-Robot Imitation in the Wild Shikhar Bahl Abhi Gupta Deepak Pathak 30 165 0 19 Jul 2022
MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge Linxi Fan Guanzhi Wang Yunfan Jiang Ajay Mandlekar Yuncong Yang Haoyi Zhu Andrew Tang De-An Huang Yuke Zhu Anima Anandkumar LM&Ro 51 352 0 17 Jun 2022
A Generalist Agent Scott E. Reed Konrad Zolna Emilio Parisotto Sergio Gomez Colmenarejo Alexander Novikov ... Yutian Chen R. Hadsell Oriol Vinyals Mahyar Bordbar Nando de Freitas LM&Ro LLMAG AI4CE 59 787 0 12 May 2022
Reinforcement Learning with Action-Free Pre-Training from Videos Younggyo Seo Kimin Lee Stephen James Pieter Abbeel SSL OnRL 18 117 0 25 Mar 2022
Reshaping Robot Trajectories Using Natural Language Commands: A Study of Multi-Modal Data Alignment Using Transformers A. Bucker Luis F. C. Figueredo Sami Haddadin Ashish Kapoor Shuang Ma Rogerio Bonatti LM&Ro 38 49 0 25 Mar 2022
Example-Driven Model-Based Reinforcement Learning for Solving Long-Horizon Visuomotor Tasks Bohan Wu Suraj Nair Li Fei-Fei Chelsea Finn OffRL LM&Ro 38 24 0 21 Sep 2021
Conservative Data Sharing for Multi-Task Offline Reinforcement Learning Tianhe Yu Aviral Kumar Yevgen Chebotar Karol Hausman Sergey Levine Chelsea Finn OffRL 35 77 0 16 Sep 2021
Learning Language-Conditioned Robot Behavior from Offline Data and Crowd-Sourced Annotation Suraj Nair E. Mitchell Kevin Chen Brian Ichter Silvio Savarese Chelsea Finn LM&Ro OffRL 37 154 0 02 Sep 2021
Learning by Watching: Physical Imitation of Manipulation Skills from Human Videos Haoyu Xiong Quanzhou Li Yun-Chun Chen Homanga Bharadhwaj Samarth Sinha Animesh Garg SSL 128 93 0 18 Jan 2021
Model-Based Inverse Reinforcement Learning from Visual Demonstrations Neha Das Sarah Bechtle Todor Davchev Dinesh Jayaraman Akshara Rai Franziska Meier 121 82 0 18 Oct 2020
Scalable Multi-Task Imitation Learning with Autonomous Improvement Avi Singh Eric Jang A. Irpan Daniel Kappler Murtaza Dalal Sergey Levine Mohi Khansari Chelsea Finn 48 35 0 25 Feb 2020