CLIPort: What and Where Pathways for Robotic Manipulation

24 September 2021

Papers citing "CLIPort: What and Where Pathways for Robotic Manipulation"

50 / 477 papers shown

Title
Modularity through Attention: Efficient Training and Transfer of Language-Conditioned Policies for Robot Manipulation Yifan Zhou Shubham D. Sonawani Mariano Phielipp Simon Stepputtis H. B. Amor LM&Ro 27 27 0 08 Dec 2022
Task Bias in Vision-Language Models Sachit Menon I. Chandratreya Carl Vondrick VLM SSL 19 6 0 08 Dec 2022
Perceive, Ground, Reason, and Act: A Benchmark for General-purpose Visual Representation Jiangyong Huang William Zhu Baoxiong Jia Zan Wang Xiaojian Ma Qing Li Siyuan Huang 37 5 0 28 Nov 2022
A System for Morphology-Task Generalization via Unified Representation and Behavior Distillation Hiroki Furuta Yusuke Iwasawa Yutaka Matsuo S. Gu 18 14 0 25 Nov 2022
Robotic Skill Acquisition via Instruction Augmentation with Vision-Language Models Ted Xiao Harris Chan P. Sermanet Ayzaan Wahid Anthony Brohan Karol Hausman Sergey Levine Jonathan Tompson VLM LM&Ro 32 65 0 21 Nov 2022
Language-Conditioned Reinforcement Learning to Solve Misunderstandings with Action Corrections Frank Röder Manfred Eppe CLL LRM 22 3 0 18 Nov 2022
PARTNR: Pick and place Ambiguity Resolving by Trustworthy iNteractive leaRning Jelle Luijkx Zlatan Ajanović L. Ferranti Jens Kober 13 3 0 15 Nov 2022
Learning Neuro-symbolic Programs for Language Guided Robot Manipulation Namasivayam Kalithasan H. Singh Vishal Bindal Arnav Tuli Vishwajeet Agrawal Rahul Jain Parag Singla Rohan Paul LM&Ro 19 15 0 12 Nov 2022
Foundation Models for Semantic Novelty in Reinforcement Learning Tarun Gupta Peter Karkus Tong Che Danfei Xu Marco Pavone VLM OffRL LRM 39 7 0 09 Nov 2022
StructDiffusion: Language-Guided Creation of Physically-Valid Structures using Unseen Objects Weiyu Liu Yilun Du Tucker Hermans Sonia Chernova Chris Paxton DiffM 23 53 0 08 Nov 2022
CLIP-Sculptor: Zero-Shot Generation of High-Fidelity and Diverse Shapes from Natural Language Aditya Sanghi Rao Fu Vivian Liu Karl Willis Hooman Shayani Amir Hosein Khasahmadi Srinath Sridhar Daniel E. Ritchie 19 52 0 02 Nov 2022
Broken Neural Scaling Laws Ethan Caballero Kshitij Gupta Irina Rish David M. Krueger 30 74 0 26 Oct 2022
Instruction-Following Agents with Multimodal Transformer Hao Liu Lisa Lee Kimin Lee Pieter Abbeel LM&Ro 30 10 0 24 Oct 2022
Composing Ensembles of Pre-trained Models via Iterative Consensus Shuang Li Yilun Du J. Tenenbaum Antonio Torralba Igor Mordatch MoMe 19 23 0 20 Oct 2022
Learning and Retrieval from Prior Data for Skill-based Imitation Learning Soroush Nasiriany Tian Gao Ajay Mandlekar Yuke Zhu SSL 39 47 0 20 Oct 2022
VIOLA: Imitation Learning for Vision-Based Manipulation with Object Proposal Priors Yifeng Zhu Abhishek Joshi Peter Stone Yuke Zhu LM&Ro 25 124 0 20 Oct 2022
Robotic Table Wiping via Reinforcement Learning and Whole-body Trajectory Optimization T. Lew Sumeet Singh M. Prats Jeffrey Bingham Jonathan Weisz ... Fei Xia Peng-Tao Xu Tingnan Zhang Jie Tan Montserrat Gonzalez 32 15 0 19 Oct 2022
PI-QT-Opt: Predictive Information Improves Multi-Task Robotic Reinforcement Learning at Scale Kuang-Huei Lee Ted Xiao A. Li Paul Wohlhart Ian S. Fischer Yao Lu 48 10 0 15 Oct 2022
SQA3D: Situated Question Answering in 3D Scenes Xiaojian Ma Silong Yong Zilong Zheng Qing Li Yitao Liang Song-Chun Zhu Siyuan Huang LM&Ro 22 129 0 14 Oct 2022
Retrospectives on the Embodied AI Workshop Matt Deitke Dhruv Batra Yonatan Bisk Tommaso Campari Angel X. Chang ... Jesse Thomason Alexander Toshev Joanne Truong Luca Weihs Jiajun Wu LM&Ro 37 51 0 13 Oct 2022
Real World Offline Reinforcement Learning with Realistic Data Source G. Zhou Liyiming Ke S. Srinivasa Abhi Gupta Aravind Rajeswaran Vikash Kumar OffRL 40 21 0 12 Oct 2022
Interactive Language: Talking to Robots in Real Time Corey Lynch Ayzaan Wahid Jonathan Tompson Tianli Ding James Betker Robert Baruch Travis Armstrong Peter R. Florence LM&Ro 35 214 0 12 Oct 2022
Visual Language Maps for Robot Navigation Chen Huang Oier Mees Andy Zeng Wolfram Burgard LM&Ro 159 344 0 11 Oct 2022
CLIP-Fields: Weakly Supervised Semantic Fields for Robotic Memory Nur Muhammad (Mahi) Shafiullah Chris Paxton Lerrel Pinto Soumith Chintala Arthur Szlam VLM LM&Ro CLIP 95 156 0 11 Oct 2022
Using Both Demonstrations and Language Instructions to Efficiently Learn Robotic Tasks Albert Yu Raymond J. Mooney LM&Ro 32 19 0 10 Oct 2022
Real-World Robot Learning with Masked Visual Pre-training Ilija Radosavovic Tete Xiao Stephen James Pieter Abbeel Jitendra Malik Trevor Darrell SSL 156 239 0 06 Oct 2022
VIMA: General Robot Manipulation with Multimodal Prompts Yunfan Jiang Agrim Gupta Zichen Zhang Guanzhi Wang Yongqiang Dou Yanjun Chen Li Fei-Fei Anima Anandkumar Yuke Zhu Linxi Fan LM&Ro 28 335 0 06 Oct 2022
DALL-E-Bot: Introducing Web-Scale Diffusion Models to Robotics Ivan Kapelyukh Vitalis Vosylius Edward Johns LM&Ro DiffM 110 145 0 05 Oct 2022
Grounding Language with Visual Affordances over Unstructured Data Oier Mees Jessica Borja-Diaz Wolfram Burgard LM&Ro 121 108 0 04 Oct 2022
Enhancing Interpretability and Interactivity in Robot Manipulation: A Neurosymbolic Approach Georgios Tziafas H. Kasaei LM&Ro 20 3 0 03 Oct 2022
Differentiable Parsing and Visual Grounding of Natural Language Instructions for Object Placement Zirui Zhao W. Lee David Hsu OOD 32 9 0 01 Oct 2022
PACT: Perception-Action Causal Transformer for Autoregressive Robotics Pre-Training Rogerio Bonatti Sai H. Vemprala Shuang Ma Felipe Vieira Frujeri Shuhang Chen Ashish Kapoor 33 22 0 22 Sep 2022
Learning from Symmetry: Meta-Reinforcement Learning with Symmetrical Behaviors and Language Instructions Xiangtong Yao Zhenshan Bing Genghang Zhuang Ke Chen Hongkuan Zhou Kai Huang Alois C. Knoll 28 6 0 21 Sep 2022
Leveraging Large (Visual) Language Models for Robot 3D Scene Understanding William Chen Siyi Hu Rajat Talak Luca Carlone LM&Ro 27 0 0 12 Sep 2022
Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulation Mohit Shridhar Lucas Manuelli D. Fox LM&Ro 161 457 0 12 Sep 2022
Instruction-driven history-aware policies for robotic manipulations Pierre-Louis Guhur Shizhe Chen Ricardo Garcia Pinel Makarand Tapaswi Ivan Laptev Cordelia Schmid LM&Ro 110 102 0 11 Sep 2022
A Computational Interface to Translate Strategic Intent from Unstructured Language in a Low-Data Setting Pradyumna Tambwekar Lakshita Dodeja Nathan Vaska Wei Xu Matthew C. Gombolay 33 0 0 17 Aug 2022
Language-guided Semantic Style Transfer of 3D Indoor Scenes Bu Jin Beiwen Tian Hao Zhao Guyue Zhou 3DV 20 11 0 16 Aug 2022
Frozen CLIP Models are Efficient Video Learners Ziyi Lin Shijie Geng Renrui Zhang Peng Gao Gerard de Melo Xiaogang Wang Jifeng Dai Yu Qiao Hongsheng Li CLIP VLM 16 200 0 06 Aug 2022
LATTE: LAnguage Trajectory TransformEr A. Bucker Luis F. C. Figueredo Sami Haddadin Ashish Kapoor Shuang Ma Sai H. Vemprala Rogerio Bonatti LM&Ro 36 59 0 04 Aug 2022
Testing Relational Understanding in Text-Guided Image Generation C. Conwell T. Ullman EGVM 152 64 0 29 Jul 2022
Robots Enact Malignant Stereotypes Andrew Hundt William Agnew V. Zeng Severin Kacianka Matthew C. Gombolay LM&Ro 35 41 0 23 Jul 2022
Semantic Abstraction: Open-World 3D Scene Understanding from 2D Vision-Language Models Huy Ha Shuran Song LM&Ro VLM 37 101 0 23 Jul 2022
World Robot Challenge 2020 -- Partner Robot: A Data-Driven Approach for Room Tidying with Mobile Manipulator T. Matsushima Yukiyasu Noguchi Jumpei Arima Toshiki Aoki Yuki Okita ... Yuki Yamashita Shoichi Seto S. Gu Yusuke Iwasawa Yutaka Matsuo 32 8 0 20 Jul 2022
Inner Monologue: Embodied Reasoning through Planning with Language Models Wenlong Huang F. Xia Ted Xiao Harris Chan Jacky Liang ... Tomas Jackson Linda Luu Sergey Levine Karol Hausman Brian Ichter LLMAG LM&Ro LRM 39 856 0 12 Jul 2022
LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Action Dhruv Shah B. Osinski Brian Ichter Sergey Levine LM&Ro 158 436 0 10 Jul 2022
Recognising Affordances in Predicted Futures to Plan with Consideration of Non-canonical Affordance Effects S. Arnold Mami Kuroishi Tadashi Adachi Kimitoshi Yamazaki 16 0 0 22 Jun 2022
MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge Linxi Fan Guanzhi Wang Yunfan Jiang Ajay Mandlekar Yuncong Yang Haoyi Zhu Andrew Tang De-An Huang Yuke Zhu Anima Anandkumar LM&Ro 46 348 0 17 Jun 2022
VLMbench: A Compositional Benchmark for Vision-and-Language Manipulation Kai Zheng Xiaotong Chen Odest Chadwicke Jenkins Qing Guo LM&Ro CoGe 21 54 0 17 Jun 2022
Extracting Zero-shot Common Sense from Large Language Models for Robot 3D Scene Understanding William Chen Siyi Hu Rajat Talak Luca Carlone 3DV LM&Ro 15 2 0 09 Jun 2022