Affordances from Human Videos as a Versatile Representation for Robotics

17 April 2023

Papers citing "Affordances from Human Videos as a Versatile Representation for Robotics"

50 / 135 papers shown

Title
$DexSim2Real$^{2}$: Building Explicit World Model for Precise Articulated Object Dexterous Manipulation$ DexSim2Real $^{2}$ : Building Explicit World Model for Precise Articulated Object Dexterous Manipulation Taoran Jiang Liqian Ma Yixuan Guan Jiaojiao Meng Weihang Chen Zecui Zeng Lusong Li Dan Wu Jing Xu Rui Chen 33 0 0 13 Sep 2024
Hand-Object Interaction Pretraining from Videos Himanshu Gaurav Singh Antonio Loquercio Carmelo Sferrazza Jane Wu Haozhi Qi Pieter Abbeel Jitendra Malik 47 15 0 12 Sep 2024
HiRT: Enhancing Robotic Control with Hierarchical Robot Transformers Jianke Zhang Yanjiang Guo Xiaoyu Chen Yen-Jen Wang Yucheng Hu Chengming Shi Jianyu Chen 31 5 0 12 Sep 2024
INTRA: Interaction Relationship-aware Weakly Supervised Affordance Grounding Ji Ha Jang H. Seo Se Young Chun 48 2 0 10 Sep 2024
One-Shot Imitation under Mismatched Execution Kushal Kedia Prithwish Dan Sanjiban Choudhury Maximus Adrian Pace Sanjiban Choudhury 47 2 0 10 Sep 2024
ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulation Wenlong Huang Chen Wang Yong Li Ruohan Zhang Li Fei-Fei 52 90 0 03 Sep 2024
Semantically Controllable Augmentations for Generalizable Robot Learning Zoey Chen Zhao Mandi Homanga Bharadhwaj Mohit Sharma Shuran Song Abhishek Gupta Vikash Kumar LM&Ro 37 5 0 02 Sep 2024
One-shot Video Imitation via Parameterized Symbolic Abstraction Graphs Jianren Wang Kangni Liu Dingkun Guo Xian Zhou Christopher G Atkeson 35 0 0 22 Aug 2024
Learning Precise Affordances from Egocentric Videos for Robotic Manipulation Gen Li Nikolaos Tsagkas Jifei Song Ruaridh Mon-Williams S. Vijayakumar Kun Shao Laura Sevilla-Lara 38 8 0 19 Aug 2024
HRP: Human Affordances for Robotic Pre-Training Mohan Kumar Srirama Sudeep Dasari Shikhar Bahl Abhinav Gupta 41 15 0 26 Jul 2024
Flow as the Cross-Domain Manipulation Interface Mengda Xu Zhenjia Xu Yinghao Xu Cheng Chi Gordon Wetzstein Manuela Veloso Shuran Song AI4CE 47 36 0 21 Jul 2024
Simultaneous Localization and Affordance Prediction for Tasks in Egocentric Video Zachary Chavis Hyun Soo Park Stephen J. Guy EgoV 44 0 0 18 Jul 2024
CookAR: Affordance Augmentations in Wearable AR to Support Kitchen Tool Interactions for People with Low Vision Jaewook Lee Andrew D. Tjahjadi Jiho Kim Junpu Yu Minji Park Jiawen Zhang Jon E. Froehlich Yapeng Tian Yuhang Zhao 33 6 0 18 Jul 2024
R+X: Retrieval and Execution from Everyday Human Videos Georgios Papagiannis Norman Di Palo Pietro Vitiello Edward Johns 59 16 0 17 Jul 2024
RAM: Retrieval-Based Affordance Transfer for Generalizable Zero-Shot Robotic Manipulation Yuxuan Kuang Junjie Ye Haoran Geng Jiageng Mao Congyue Deng Leonidas J. Guibas He Wang Yue Wang LM&Ro 51 22 0 05 Jul 2024
TieBot: Learning to Knot a Tie from Visual Demonstration through a Real-to-Sim-to-Real Approach Weikun Peng Jun Lv Yuwei Zeng Haonan Chen Siheng Zhao Jichen Sun Cewu Lu Lin Shao 31 2 0 03 Jul 2024
Mitigating the Human-Robot Domain Discrepancy in Visual Pre-training for Robotic Manipulation Jiaming Zhou Teli Ma Kun-Yu Lin Ronghe Qiu Zifan Wang Junwei Liang 52 4 0 20 Jun 2024
Learning Semantic Traversability with Egocentric Video and Automated Annotation Strategy Yunho Kim Jeong Hyun Lee Choongin Lee Juhyeok Mun D. Youm Jeongsoo Park Jemin Hwangbo 37 1 0 05 Jun 2024
Learning Manipulation by Predicting Interaction Jia Zeng Qingwen Bu Bangjun Wang Wenke Xia Li Chen ... Heming Cui Bin Zhao Xuelong Li Yu Qiao Hongyang Li 60 21 0 01 Jun 2024
SAM-E: Leveraging Visual Foundation Model with Sequence Imitation for Embodied Manipulation Junjie Zhang Chenjia Bai Haoran He Wenke Xia Zhigang Wang Bin Zhao Xiu Li Xuelong Li 46 12 0 30 May 2024
Empowering Embodied Manipulation: A Bimanual-Mobile Robot Manipulation Dataset for Household Tasks Tianle Zhang Dongjiang Li Yihang Li Zecui Zeng Lin Zhao ... Yue Chen Xuelong Wei Yibing Zhan Lusong Li Xiaodong He 40 7 0 29 May 2024
iVideoGPT: Interactive VideoGPTs are Scalable World Models Jialong Wu Shaofeng Yin Ningya Feng Xu He Dong Li Haifeng Zhang Mingsheng Long VGen 49 26 0 24 May 2024
Octo: An Open-Source Generalist Robot Policy Octo Model Team Dibya Ghosh Homer Walke Karl Pertsch Kevin Black ... Quan Vuong Ted Xiao Dorsa Sadigh Chelsea Finn Sergey Levine 69 357 0 20 May 2024
ManiFoundation Model for General-Purpose Robotic Manipulation of Contact Synthesis with Arbitrary Objects and Robots Zhixuan Xu Chongkai Gao Zixuan Liu Gang Yang Chenrui Tie ... Weikun Peng Debang Wang Tianyi Chen Zhouliang Yu Lin Shao 49 9 0 11 May 2024
Splat-MOVER: Multi-Stage, Open-Vocabulary Robotic Manipulation via Editable Gaussian Splatting O. Shorinwa Johnathan Tucker Aliyah Smith Aiden Swann Timothy Chen Roya Firoozi Monroe Kennedy Mac Schwager 37 22 0 07 May 2024
Diff-IP2D: Diffusion-Based Hand-Object Interaction Prediction on Egocentric Videos Junyi Ma Jingyi Xu Xieyuanli Chen Hesheng Wang VGen 38 7 0 07 May 2024
Plan-Seq-Learn: Language Model Guided RL for Solving Long Horizon Robotics Tasks Murtaza Dalal Tarun Chiruvolu Devendra Singh Chaplot Ruslan Salakhutdinov LM&Ro 41 39 0 02 May 2024
Track2Act: Predicting Point Tracks from Internet Videos enables Diverse Zero-shot Robot Manipulation Homanga Bharadhwaj Roozbeh Mottaghi Abhinav Gupta Shubham Tulsiani 3DPC 54 17 0 02 May 2024
What Foundation Models can Bring for Robot Learning in Manipulation : A Survey Dingzhe Li Yixiang Jin A. Yong Hongze Yu Jun Shi Xiaoshuai Hao Peng Hao Huaping Liu Gang Hua Bin Fang AI4CE LM&Ro 72 13 0 28 Apr 2024
Retrieval-Augmented Embodied Agents Yichen Zhu Zhicai Ou Xiaofeng Mou Jian Tang 51 17 0 17 Apr 2024
QuasiSim: Parameterized Quasi-Physical Simulators for Dexterous Manipulations Transfer Xueyi Liu Kangbo Lyu Jieqiong Zhang Tao Du Li Yi 43 4 0 11 Apr 2024
Self-Explainable Affordance Learning with Embodied Caption Zhipeng Zhang Zhimin Wei Guolei Sun Peng Wang Luc Van Gool 53 3 0 08 Apr 2024
Text-driven Affordance Learning from Egocentric Vision Tomoya Yoshida Shuhei Kurita Taichi Nishimura Shinsuke Mori 44 5 0 03 Apr 2024
DITTO: Demonstration Imitation by Trajectory Transformation Nick Heppert Max Argus Tim Welschehold Thomas Brox Abhinav Valada 67 16 0 22 Mar 2024
Vid2Robot: End-to-end Video-conditioned Policy Learning with Cross-Attention Transformers Vidhi Jain Maria Attarian Nikhil J. Joshi Ayzaan Wahid Danny Driess ... Stefan Welker Christine Chan Igor Gilitschenski Yonatan Bisk Debidatta Dwibedi 68 29 0 19 Mar 2024
Spatiotemporal Predictive Pre-training for Robotic Motor Control Jiange Yang Bei Liu Jianlong Fu Bocheng Pan Gangshan Wu Limin Wang 42 10 0 08 Mar 2024
Pushing the Limits of Cross-Embodiment Learning for Manipulation and Navigation Jonathan Yang Catherine Glossop Arjun Bhorkar Dhruv Shah Quan Vuong Chelsea Finn Dorsa Sadigh Sergey Levine 44 42 0 29 Feb 2024
Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robots Cheng Chi Zhenjia Xu Chuer Pan Eric A. Cousineau Benjamin Burchfiel Siyuan Feng Russ Tedrake Shuran Song 17 190 0 15 Feb 2024
Learning by Watching: A Review of Video-based Learning Approaches for Robot Manipulation Chrisantus Eze Christopher Crick SSL 82 12 0 11 Feb 2024
Real-World Robot Applications of Foundation Models: A Review Kento Kawaharazuka T. Matsushima Andrew Gambardella Jiaxian Guo Chris Paxton Andy Zeng OffRL VLM LM&Ro 51 45 0 08 Feb 2024
InCoRo: In-Context Learning for Robotics Control with Feedback Loops Jiaqiang Ye Zhu Carla Gomez Cano David Vazquez Bermudez Michal Drozdzal LRM 35 7 0 07 Feb 2024
Zero-shot Imitation Policy via Search in Demonstration Dataset Federico Malato Florian Leopold Andrew Melnik Ville Hautamaki LM&Ro OffRL 21 6 0 29 Jan 2024
Zero-Shot Learning for the Primitives of 3D Affordance in General Objects Hyeonwoo Kim Sookwan Han Patrick Kwon Hanbyul Joo DiffM 45 14 0 23 Jan 2024
General Flow as Foundation Affordance for Scalable Robot Learning Chengbo Yuan Chuan Wen Tong Zhang Yang Gao AI4CE 21 31 0 21 Jan 2024
Seeing the Unseen: Visual Common Sense for Semantic Placement Ram Ramrakhya Aniruddha Kembhavi Dhruv Batra Z. Kira Kuo-Hao Zeng Luca Weihs VLM 41 5 0 15 Jan 2024
Robo-ABC: Affordance Generalization Beyond Categories via Semantic Correspondence for Robot Manipulation Yuanchen Ju Kaizhe Hu Guowei Zhang Gu Zhang Mingrun Jiang Huazhe Xu 36 41 0 15 Jan 2024
AffordanceLLM: Grounding Affordance from Vision Language Models Shengyi Qian Weifeng Chen Min Bai Xiong Zhou Zhuowen Tu Li Erran Li 23 21 0 12 Jan 2024
Any-point Trajectory Modeling for Policy Learning Chuan Wen Xingyu Lin John So Kai-xiang Chen Qi Dou Yang Gao Pieter Abbeel PINN VGen 44 81 0 28 Dec 2023
QUAR-VLA: Vision-Language-Action Model for Quadruped Robots Pengxiang Ding Han Zhao Wenxuan Song Zhitao Wang Zhenyu Wei Shangke Lyu Ningxi Yang Donglin Wang 32 19 0 22 Dec 2023
Shared Affordance-awareness via Augmented Reality for Proactive Assistance in Human-robot Collaboration Drake Moore Mark Zolotas T. Padır 18 1 0 20 Dec 2023