Zero-Shot Robot Manipulation from Passive Human Videos

3 February 2023

Homanga Bharadhwaj

Abhi Gupta

Shubham Tulsiani

Vikash Kumar

ArXiv PDF HTML

Papers citing "Zero-Shot Robot Manipulation from Passive Human Videos"

21 / 21 papers shown

Title
X-Sim: Cross-Embodiment Learning via Real-to-Sim-to-Real Prithwish Dan Kushal Kedia Angela Chao Edward Weiyi Duan Maximus Adrian Pace Wei-Chiu Ma Sanjiban Choudhury 44 0 0 11 May 2025
VidBot: Learning Generalizable 3D Actions from In-the-Wild 2D Human Videos for Zero-Shot Robotic Manipulation Hanzhi Chen Boyang Sun Anran Zhang Marc Pollefeys Stefan Leutenegger LM&Ro 86 0 0 10 Mar 2025
On-Robot Reinforcement Learning with Goal-Contrastive Rewards Ondrej Biza Thomas Weng Lingfeng Sun Karl Schmeckpeper Tarik Kelestemur Yecheng Jason Ma Robert Platt Jan-Willem van de Meent Lawson L. S. Wong OffRL 63 0 0 25 Oct 2024
Latent Action Pretraining from Videos Seonghyeon Ye Joel Jang Byeongguk Jeon Sejune Joo Jianwei Yang ... Kimin Lee J. Gao Luke Zettlemoyer Dieter Fox Minjoon Seo 53 34 0 15 Oct 2024
Spacewalk-18: A Benchmark for Multimodal and Long-form Procedural Video Understanding in Novel Domains Rohan Myer Krishnan Zitian Tang Zhiqiu Yu Chen Sun 78 1 0 30 Nov 2023
Human-to-Robot Imitation in the Wild Shikhar Bahl Abhi Gupta Deepak Pathak 43 167 0 19 Jul 2022
Joint Hand Motion and Interaction Hotspots Prediction from Egocentric Videos Shao-Wei Liu Subarna Tripathi Somdeb Majumdar Xiaolong Wang EgoV 60 93 0 04 Apr 2022
Human Hands as Probes for Interactive Object Understanding Mohit Goyal Sahil Modi Rishabh Goyal Saurabh Gupta 44 48 0 16 Dec 2021
Ego4D: Around the World in 3,000 Hours of Egocentric Video Kristen Grauman Andrew Westbury Eugene Byrne Zachary Chavis Antonino Furnari ... Mike Zheng Shou Antonio Torralba Lorenzo Torresani Mingfei Yan Jitendra Malik EgoV 335 1,056 0 13 Oct 2021
DexYCB: A Benchmark for Capturing Hand Grasping of Objects Yu-Wei Chao Wei Yang Yu Xiang Pavlo Molchanov Ankur Handa ... Karl Van Wyk Umar Iqbal Stan Birchfield Jan Kautz Dieter Fox 44 257 0 09 Apr 2021
Is Space-Time Attention All You Need for Video Understanding? Gedas Bertasius Heng Wang Lorenzo Torresani ViT 303 2,016 0 09 Feb 2021
Reinforcement Learning with Videos: Combining Offline Observations with Interaction Karl Schmeckpeper Oleh Rybkin Kostas Daniilidis Sergey Levine Chelsea Finn OffRL 50 105 0 12 Nov 2020
Understanding Human Hands in Contact at Internet Scale Dandan Shan Jiaqi Geng Michelle Shu David Fouhey 55 320 0 11 Jun 2020
AVID: Learning Multi-Stage Tasks via Pixel-Level Translation of Human Videos Laura M. Smith Nikita Dhawan Marvin Zhang Pieter Abbeel Sergey Levine 105 158 0 10 Dec 2019
PVN3D: A Deep Point-wise 3D Keypoints Voting Network for 6DoF Pose Estimation Yisheng He Wei Sun Haibin Huang Jianran Liu Haoqiang Fan Jian Sun 3DPC 99 464 0 11 Nov 2019
ContactGrasp: Functional Multi-finger Grasp Synthesis from Contact Samarth Brahmbhatt Ankur Handa James Hays Dieter Fox 21 134 0 07 Apr 2019
Grounded Human-Object Interaction Hotspots from Video Tushar Nagarajan Christoph Feichtenhofer Kristen Grauman 46 159 0 11 Dec 2018
Segmentation-driven 6D Object Pose Estimation Yinlin Hu Joachim Hugonot Pascal Fua Mathieu Salzmann 3DPC 46 279 0 06 Dec 2018
SFV: Reinforcement Learning of Physical Skills from Videos Xue Bin Peng Angjoo Kanazawa Jitendra Malik Pieter Abbeel Sergey Levine 33 65 0 08 Oct 2018
SSD-6D: Making RGB-based 3D detection and 6D pose estimation great again Wadim Kehl Fabian Manhardt Federico Tombari Slobodan Ilic Nassir Navab 98 922 0 27 Nov 2017
Benchmarking in Manipulation Research: The YCB Object and Model Set and Benchmarking Protocols B. Çalli Aaron Walsman Arjun Singh S. Srinivasa Pieter Abbeel A. Dollar 41 293 0 10 Feb 2015