Real-World Robot Learning with Masked Visual Pre-training

6 October 2022

Pieter Abbeel

Papers citing "Real-World Robot Learning with Masked Visual Pre-training"

50 / 58 papers shown

Title
DexWild: Dexterous Human Interactions for In-the-Wild Robot Policies Tony Tao M. K. Srirama Jason Jingzhou Liu Kenneth Shaw Deepak Pathak 28 0 0 12 May 2025
Learning Streaming Video Representation via Multitask Training Yibin Yan Jilan Xu Shangzhe Di Yikun Liu Yudi Shi Qirui Chen Zeqian Li Yifei Huang Weidi Xie CLL 84 0 0 28 Apr 2025
CIVIL: Causal and Intuitive Visual Imitation Learning Yinlong Dai Robert Ramirez Sanchez Ryan Jeronimus Shahabedin Sagheb Cara M. Nunez Heramb Nemlekar Dylan P. Losey 68 0 0 24 Apr 2025
ViTaMIn: Learning Contact-Rich Tasks Through Robot-Free Visuo-Tactile Manipulation Interface Fangchen Liu Chuanyu Li Yihua Qin Ankit Shaw J. Xu Pieter Abbeel Rui Chen 49 2 0 08 Apr 2025
FADConv: A Frequency-Aware Dynamic Convolution for Farmland Non-agriculturalization Identification and Segmentation Tan Shu Li Shen 44 0 0 04 Apr 2025
Machine Unlearning in Hyperbolic vs. Euclidean Multimodal Contrastive Learning: Adapting Alignment Calibration to MERU Àlex Pujol Vidal Sergio Escalera Kamal Nasrollahi T. Moeslund MU 61 0 0 19 Mar 2025
HybridVLA: Collaborative Diffusion and Autoregression in a Unified Vision-Language-Action Model Jiaming Liu Hao Chen Pengju An Zhuoyang Liu Renrui Zhang ... Chengkai Hou Mengdi Zhao KC alex Zhou Pheng-Ann Heng S. Zhang 64 7 0 13 Mar 2025
A Data-Centric Revisit of Pre-Trained Vision Models for Robot Learning Xin Wen Bingchen Zhao Yilun Chen Jiangmiao Pang Xiaojuan Qi LM&Ro 46 0 0 10 Mar 2025
VidBot: Learning Generalizable 3D Actions from In-the-Wild 2D Human Videos for Zero-Shot Robotic Manipulation Hanzhi Chen Boyang Sun Anran Zhang Marc Pollefeys Stefan Leutenegger LM&Ro 67 0 0 10 Mar 2025
VLA Model-Expert Collaboration for Bi-directional Manipulation Learning Tian-Yu Xiang Ao-Qun Jin Xiao-Hu Zhou Mei-Jiang Gui Xiao-Liang Xie ... Shuang-Yi Wang Sheng-Bin Duang Si-Cheng Wang Zheng Lei Z. Hou 58 1 0 06 Mar 2025
AirExo-2: Scaling up Generalizable Robotic Imitation Learning with Low-Cost Exoskeletons Hongjie Fang Chenxi Wang Yiming Wang J. Chen Shangning Xia ... Xinyu Zhan Lixin Yang Weiming Wang Cewu Lu Hao-Shu Fang 82 1 0 05 Mar 2025
Modeling Fine-Grained Hand-Object Dynamics for Egocentric Video Representation Learning Baoqi Pei Y. Huang Jilan Xu Guo Chen Yuping He ... Yali Wang Weidi Xie Yu Qiao Fei Wu Limin Wang 41 0 0 02 Mar 2025
Human2Robot: Learning Robot Actions from Paired Human-Robot Videos Sicheng Xie Haidong Cao Zejia Weng Zhen Xing Shiwei Shen Jiaqi Leng Xipeng Qiu Yanwei Fu Zuxuan Wu Yu Jiang 53 0 0 23 Feb 2025
ArrayBot: Reinforcement Learning for Generalizable Distributed Manipulation through Touch Zhengrong Xue H. Zhang Jin Cheng Zhengmao He Yuanchen Ju Chan-Yu Lin Gu Zhang Huazhe Xu OffRL 88 9 0 20 Feb 2025
DexVLA: Vision-Language Model with Plug-In Diffusion Expert for General Robot Control Junjie Wen Y. X. Zhu Jinming Li Zhibin Tang Chaomin Shen Feifei Feng VLM 53 12 0 09 Feb 2025
RDT-1B: a Diffusion Foundation Model for Bimanual Manipulation Songming Liu Lingxuan Wu Bangguo Li Hengkai Tan Huayu Chen Zhengyi Wang Ke Xu Hang Su Jun Zhu 29 75 0 10 Oct 2024
SPA: 3D Spatial-Awareness Enables Effective Embodied Representation Haoyi Zhu Honghui Yang Yating Wang Jiange Yang Limin Wang Tong He 3DH 51 6 0 10 Oct 2024
Towards Generalizable Vision-Language Robotic Manipulation: A Benchmark and LLM-guided 3D Policy Ricardo Garcia Shizhe Chen Cordelia Schmid LM&Ro 34 7 0 02 Oct 2024
Feature Extractor or Decision Maker: Rethinking the Role of Visual Encoders in Visuomotor Policies Ruiyu Wang Zheyu Zhuang Shutong Jin Nils Ingelhag Danica Kragic Florian T. Pokorny 26 0 0 30 Sep 2024
DynaMo: In-Domain Dynamics Pretraining for Visuo-Motor Control Zichen Jeff Cui Hengkai Pan Aadhithya Iyer Siddhant Haldar Lerrel Pinto VGen 26 10 0 18 Sep 2024
Hand-Object Interaction Pretraining from Videos Himanshu Gaurav Singh Antonio Loquercio Carmelo Sferrazza Jane Wu Haozhi Qi Pieter Abbeel Jitendra Malik 44 13 0 12 Sep 2024
Adapt2Reward: Adapting Video-Language Models to Generalizable Robotic Rewards via Failure Prompts Yanting Yang Minghao Chen Qibo Qiu Jiahao Wu Wenxiao Wang Binbin Lin Ziyu Guan Xiaofei He LM&Ro 37 2 0 20 Jul 2024
Dreamitate: Real-World Visuomotor Policy Learning via Video Generation Junbang Liang Ruoshi Liu Ege Ozguroglu Sruthi Sudhakar Achal Dave P. Tokmakov Shuran Song Carl Vondrick VGen 40 22 0 24 Jun 2024
Mitigating the Human-Robot Domain Discrepancy in Visual Pre-training for Robotic Manipulation Jiaming Zhou Teli Ma Kun-Yu Lin Ronghe Qiu Zifan Wang Junwei Liang 46 3 0 20 Jun 2024
A Survey on Vision-Language-Action Models for Embodied AI Yueen Ma Zixing Song Yuzheng Zhuang Jianye Hao Irwin King LM&Ro 71 41 0 23 May 2024
Rethinking Patch Dependence for Masked Autoencoders Letian Fu Long Lian Renhao Wang Baifeng Shi Xudong Wang Adam Yala Trevor Darrell Alexei A. Efros Ken Goldberg 26 14 0 25 Jan 2024
On Bringing Robots Home Nur Muhammad (Mahi) Shafiullah Anant Rai Haritheja Etukuru Yiqian Liu Ishan Misra Soumith Chintala Lerrel Pinto 33 76 0 27 Nov 2023
What Makes Pre-Trained Visual Representations Successful for Robust Manipulation? Kaylee Burns Zach Witzel Jubayer Ibn Hamid Tianhe Yu Chelsea Finn Karol Hausman OOD SSL 27 22 0 03 Nov 2023
H-InDex: Visual Reinforcement Learning with Hand-Informed Representations for Dexterous Manipulation Yanjie Ze Yuyao Liu Ruizhe Shi Jiaxin Qin Zhecheng Yuan Jiashun Wang Huazhe Xu 28 1 0 02 Oct 2023
AirExo: Low-Cost Exoskeletons for Learning Whole-Arm Manipulation in the Wild Hongjie Fang Haoshu Fang Yiming Wang Jieji Ren Jing Chen Ruo Zhang Weiming Wang Cewu Lu 34 46 0 26 Sep 2023
AnyOKP: One-Shot and Instance-Aware Object Keypoint Extraction with Pretrained ViT Fangbo Qin Taogang Hou Shan Lin Kaiyuan Wang Michael C. Yip Shan Yu 21 0 0 15 Sep 2023
RePo: Resilient Model-Based Reinforcement Learning by Regularizing Posterior Predictability Chuning Zhu Max Simchowitz Siri Gadipudi Abhishek Gupta 40 13 0 31 Aug 2023
Language Reward Modulation for Pretraining Reinforcement Learning Ademi Adeniji Amber Xie Carmelo Sferrazza Younggyo Seo Stephen James Pieter Abbeel 39 26 0 23 Aug 2023
Structured World Models from Human Videos Russell Mendonca Shikhar Bahl Deepak Pathak LM&Ro 32 85 0 21 Aug 2023
An Outlook into the Future of Egocentric Vision Chiara Plizzari Gabriele Goletto Antonino Furnari Siddhant Bansal Francesco Ragusa G. Farinella Dima Damen Tatiana Tommasi EgoV 32 38 0 14 Aug 2023
Exploring Visual Pre-training for Robot Manipulation: Datasets, Models and Methods Ya Jing Xuelin Zhu Xingbin Liu Qie Sima Taozheng Yang Yunhai Feng Tao Kong LM&Ro 40 16 0 07 Aug 2023
Large Language Models as General Pattern Machines Suvir Mirchandani F. Xia Peter R. Florence Brian Ichter Danny Driess Montse Gonzalez Arenas Kanishka Rao Dorsa Sadigh Andy Zeng LLMAG 48 183 0 10 Jul 2023
AnyTeleop: A General Vision-Based Dexterous Robot Arm-Hand Teleoperation System Yuzhe Qin Wei Yang Binghao Huang Karl Van Wyk Hao Su Xiaolong Wang Yu-Wei Chao D. Fox 28 91 0 10 Jul 2023
PM-DETR: Domain Adaptive Prompt Memory for Object Detection with Transformers Peidong Jia Jiaming Liu Senqiao Yang Jiarui Wu Xiaodong Xie Shanghang Zhang VLM 39 2 0 01 Jul 2023
Transferring Foundation Models for Generalizable Robotic Manipulation Jiange Yang Wenhui Tan Chuhao Jin Keling Yao Bei Liu Jianlong Fu Ruihua Song Gangshan Wu Limin Wang LM&Ro 47 6 0 09 Jun 2023
Instruct2Act: Mapping Multi-modality Instructions to Robotic Actions with Large Language Model Siyuan Huang Zhengkai Jiang Hao Dong Yu Qiao Peng Gao Hongsheng Li LM&Ro 22 92 0 18 May 2023
Latent Space Planning for Multi-Object Manipulation with Environment-Aware Relational Classifiers Yixuan Huang Nichols Crawford Taylor Adam Conkey Weiyu Liu Tucker Hermans 33 7 0 18 May 2023
Affordances from Human Videos as a Versatile Representation for Robotics Shikhar Bahl Russell Mendonca Lili Chen Unnat Jain Deepak Pathak 37 160 0 17 Apr 2023
Lossless Adaptation of Pretrained Vision Models For Robotic Manipulation Mohit Sharma Claudio Fantacci Yuxiang Zhou Skanda Koppula N. Heess Jonathan Scholz Y. Aytar VLM 42 29 0 13 Apr 2023
ENTL: Embodied Navigation Trajectory Learner Klemen Kotar Aaron Walsman Roozbeh Mottaghi 15 6 0 05 Apr 2023
UniDexGrasp++: Improving Dexterous Grasping Policy Learning via Geometry-aware Curriculum and Iterative Generalist-Specialist Learning Weikang Wan Haoran Geng Yun-Hai Liu Zikang Shan Yaodong Yang Li Yi He-Nan Wang 44 94 0 02 Apr 2023
Dexterity from Touch: Self-Supervised Pre-Training of Tactile Representations with Robotic Play Irmak Güzey Ben Evans Soumith Chintala Lerrel Pinto 61 64 0 21 Mar 2023
Open-World Object Manipulation using Pre-trained Vision-Language Models Austin Stone Ted Xiao Yao Lu K. Gopalakrishnan Kuang-Huei Lee ... Sean Kirmani Brianna Zitkovich F. Xia Chelsea Finn Karol Hausman LM&Ro 142 144 0 02 Mar 2023
Aerial Image Object Detection With Vision Transformer Detector (ViTDet) Liya Wang A. Tien 39 7 0 28 Jan 2023
Decorate the Newcomers: Visual Domain Prompt for Continual Test Time Adaptation Yulu Gan Yan Bai Yihang Lou Xianzheng Ma Renrui Zhang Nian Shi Lin Luo OOD VLM 25 91 0 08 Dec 2022