Making Sense of Vision and Touch: Self-Supervised Learning of Multimodal Representations for Contact-Rich Tasks

24 October 2018

Silvio Savarese

Li Fei-Fei

Papers citing "Making Sense of Vision and Touch: Self-Supervised Learning of Multimodal Representations for Contact-Rich Tasks"

46 / 96 papers shown

Title
Exploring Data Aggregation and Transformations to Generalize across Visual Domains Antono DÍnnocente OOD 33 0 0 20 Aug 2021
MultiBench: Multiscale Benchmarks for Multimodal Representation Learning Paul Pu Liang Yiwei Lyu Xiang Fan Zetian Wu Yun Cheng ... Peter Wu Michelle A. Lee Yuke Zhu Ruslan Salakhutdinov Louis-Philippe Morency VLM 34 159 0 15 Jul 2021
Multi-Modal Mutual Information (MuMMI) Training for Robust Self-Supervised Deep Reinforcement Learning Kaiqi Chen Yong Lee Harold Soh SSL 33 20 0 06 Jul 2021
Vision-driven Compliant Manipulation for Reliable, High-Precision Assembly Tasks A. S. Morgan Bowen Wen Junchi Liang Abdeslam Boularias A. Dollar Kostas Bekris 30 54 0 26 Jun 2021
panda-gym: Open-source goal-conditioned environments for robotic learning Quentin Gallouedec Nicolas Cazin Emmanuel Dellandrea Liming Chen OffRL 11 77 0 25 Jun 2021
What Does Rotation Prediction Tell Us about Classifier Accuracy under Varying Testing Environments? Weijian Deng Stephen Gould Liang Zheng 39 63 0 10 Jun 2021
Composable Energy Policies for Reactive Motion Generation and Reinforcement Learning Julen Urain Anqi Li Puze Liu Carlo DÉramo Jan Peters 33 26 0 11 May 2021
InsertionNet -- A Scalable Solution for Insertion Oren Spector Dotan Di Castro 32 51 0 29 Apr 2021
Tactile-RL for Insertion: Generalization to Objects of Unknown Geometry Siyuan Dong Devesh K. Jha Diego Romeres Sangwoon Kim D. Nikovski Alberto Rodriguez 40 116 0 02 Apr 2021
Sim-to-Real for Robotic Tactile Sensing via Physics-Based Simulation and Learned Latent Projections Yashraj S. Narang Balakumar Sundaralingam Miles Macklin Arsalan Mousavian Dieter Fox 30 58 0 31 Mar 2021
Learning Deep Energy Shaping Policies for Stability-Guaranteed Manipulation S. A. Khader Hang Yin Pietro Falco Danica Kragic 16 12 0 30 Mar 2021
Visionary: Vision architecture discovery for robot learning Iretiayo Akinola A. Angelova Yao Lu Yevgen Chebotar Dmitry Kalashnikov Jacob Varley Julian Ibarz Michael S. Ryoo 24 10 0 26 Mar 2021
In-air Knotting of Rope using Dual-Arm Robot based on Deep Learning Kanata Suzuki Momomi Kanamura Y. Suga Hiroki Mori T. Ogata 28 22 0 17 Mar 2021
Combining Learning from Demonstration with Learning by Exploration to Facilitate Contact-Rich Tasks Yunlei Shi Zhaopeng Chen Yansong Wu Dimitri Henkel Sebastian Riedel Hongxu Liu Qian Feng Jianwei Zhang 16 12 0 10 Mar 2021
Interpretability in Contact-Rich Manipulation via Kinodynamic Images Ioanna Mitsioni Joonatan Mänttäri Y. Karayiannidis John Folkesson Danica Kragic FAtt 11 3 0 23 Feb 2021
How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learned Julian Ibarz Jie Tan Chelsea Finn Mrinal Kalakrishnan P. Pastor Sergey Levine OffRL 16 520 0 04 Feb 2021
Detect, Reject, Correct: Crossmodal Compensation of Corrupted Sensors Michelle A. Lee Matthew Tan Yuke Zhu Jeannette Bohg 49 25 0 01 Dec 2020
COCOI: Contact-aware Online Context Inference for Generalizable Non-planar Pushing Zhuo Xu Wenhao Yu Alexander Herzog Wenlong Lu Chuyuan Fu Masayoshi Tomizuka Yunfei Bai Chenxi Liu Daniel Ho OffRL 27 17 0 23 Nov 2020
Seeing Through your Skin: Recognizing Objects with a Novel Visuotactile Sensor F. Hogan M. Jenkin S. Rezaei-Shoshtari Yogesh A. Girdhar David Meger Gregory Dudek 27 40 0 18 Nov 2020
Learning Dense Rewards for Contact-Rich Manipulation Tasks Zheng Wu Wenzhao Lian Vaibhav Unhelkar Masayoshi Tomizuka S. Schaal 8 37 0 17 Nov 2020
Joint Space Control via Deep Reinforcement Learning Visak C. V. Kumar David Hoeller Balakumar Sundaralingam Jonathan Tremblay Stan Birchfield DRL 25 15 0 12 Nov 2020
Offline Learning of Counterfactual Predictions for Real-World Robotic Reinforcement Learning Jun Jin D. Graves Cameron Haigh Jun Luo Martin Jägersand SSL OffRL 25 6 0 11 Nov 2020
Learning Bayes Filter Models for Tactile Localization Tarik Kelestemur Colin Keil J. P. Whitney Robert W. Platt T. Padır 16 5 0 11 Nov 2020
Leveraging Post Hoc Context for Faster Learning in Bandit Settings with Applications in Robot-Assisted Feeding E. Gordon Sumegh Roychowdhury T. Bhattacharjee Kevin G. Jamieson S. Srinivasa 18 18 0 05 Nov 2020
MELD: Meta-Reinforcement Learning from Images via Latent State Models Tony Zhao Anusha Nagabandi Kate Rakelly Chelsea Finn Sergey Levine OffRL 32 36 0 26 Oct 2020
S3K: Self-Supervised Semantic Keypoints for Robotic Manipulation via Multi-View Consistency Mel Vecerík Jean-Baptiste Regli Oleg O. Sushkov David Barker Rugile Pevceviciute Thomas Rothörl C. Schuster R. Hadsell Lourdes Agapito Jonathan Scholz 3DPC 23 40 0 30 Sep 2020
Learning Skills to Patch Plans Based on Inaccurate Models A. LaGrassa Steven Lee Oliver Kroemer 13 10 0 29 Sep 2020
Event-Driven Visual-Tactile Sensing and Learning for Robots Tasbolat Taunyazov Weicong Sng H. See Brian Z. H. Lim Jethro Kuan Abdul Fatir Ansari Benjamin C. K. Tee Harold Soh 28 92 0 15 Sep 2020
3D Shape Reconstruction from Vision and Touch Edward James Smith Roberto Calandra Adriana Romero Georgia Gkioxari David Meger Jitendra Malik M. Drozdzal 16 71 0 07 Jul 2020
Grasp State Assessment of Deformable Objects Using Visual-Tactile Fusion Perception Shaowei Cui Rui Wang Junhang Wei Fanrong Li Shuo Wang 30 42 0 23 Jun 2020
Sim2Real for Peg-Hole Insertion with Eye-in-Hand Camera Damian Bogunowicz A. Rybnikov Komal Vendidandi Fedor Chervinskii 17 7 0 29 May 2020
Guided Uncertainty-Aware Policy Optimization: Combining Learning and Model-Based Strategies for Sample-Efficient Policy Learning Michelle A. Lee Carlos Florensa Jonathan Tremblay Nathan D. Ratliff Animesh Garg Fabio Ramos Dieter Fox 23 60 0 21 May 2020
Never Stop Learning: The Effectiveness of Fine-Tuning in Robotic Reinforcement Learning Ryan Julian Benjamin Swanson Gaurav Sukhatme Sergey Levine Chelsea Finn Karol Hausman OnRL CLL 33 43 0 21 Apr 2020
Balance Between Efficient and Effective Learning: Dense2Sparse Reward Shaping for Robot Manipulation with Environment Uncertainty Yongle Luo Kun Dong Lili Zhao Zhiyong Sun Chao Zhou Bo Song 34 13 0 05 Mar 2020
Learning to Walk in the Real World with Minimal Human Effort Sehoon Ha P. Xu Zhenyu Tan Sergey Levine Jie Tan 29 169 0 20 Feb 2020
Self-Supervised Joint Encoding of Motion and Appearance for First Person Action Recognition M. Planamente A. Bottino Barbara Caputo EgoV 28 3 0 10 Feb 2020
Learning Selective Sensor Fusion for States Estimation Changhao Chen Stefano Rosa Chris Xiaoxuan Lu Bing Wang Niki Trigoni Andrew Markham 22 20 0 30 Dec 2019
Learning to Scaffold the Development of Robotic Manipulation Skills Lin Shao Toki Migimatsu Jeannette Bohg 21 29 0 03 Nov 2019
Tactile-Based Insertion for Dense Box-Packing Siyuan Dong Alberto Rodriguez 103 52 0 12 Sep 2019
Making Sense of Vision and Touch: Learning Multimodal Representations for Contact-Rich Tasks Michelle A. Lee Yuke Zhu Peter Zachares Matthew Tan K. Srinivasan Silvio Savarese Fei-Fei Li Animesh Garg Jeannette Bohg SSL 23 208 0 28 Jul 2019
Grasping Using Tactile Sensing and Deep Calibration Masoud Baghbahari A. Behal 16 0 0 23 Jul 2019
A Review of Robot Learning for Manipulation: Challenges, Representations, and Algorithms Oliver Kroemer S. Niekum George Konidaris 41 356 0 06 Jul 2019
Variable Impedance Control in End-Effector Space: An Action Space for Reinforcement Learning in Contact-Rich Tasks Roberto Martín-Martín Michelle A. Lee Rachel Gardner Silvio Savarese Jeannette Bohg Animesh Garg 30 194 0 20 Jun 2019
Quickly Inserting Pegs into Uncertain Holes using Multi-view Images and Deep Network Trained on Synthetic Data Joshua C. Triyonoputro Weiwei Wan Kensuke Harada 3DV 29 55 0 25 Feb 2019
Learning efficient haptic shape exploration with a rigid tactile sensor array Sascha Fleer Alexandra Moringen R. Klatzky Helge J. Ritter 35 21 0 20 Feb 2019
Revisiting Self-Supervised Visual Representation Learning Alexander Kolesnikov Xiaohua Zhai Lucas Beyer SSL 74 715 0 25 Jan 2019