CLIPort: What and Where Pathways for Robotic Manipulation

24 September 2021

Papers citing "CLIPort: What and Where Pathways for Robotic Manipulation"

50 / 477 papers shown

Title
Affordances from Human Videos as a Versatile Representation for Robotics Shikhar Bahl Russell Mendonca Lili Chen Unnat Jain Deepak Pathak 41 164 0 17 Apr 2023
Lossless Adaptation of Pretrained Vision Models For Robotic Manipulation Mohit Sharma Claudio Fantacci Yuxiang Zhou Skanda Koppula N. Heess Jonathan Scholz Y. Aytar VLM 50 29 0 13 Apr 2023
ARNOLD: A Benchmark for Language-Grounded Task Learning With Continuous States in Realistic 3D Scenes Ran Gong Jiangyong Huang Yizhou Zhao Haoran Geng Xiaofeng Gao ... Ziheng Zhou D. Terzopoulos Song-Chun Zhu Baoxiong Jia Siyuan Huang LM&Ro 42 45 0 09 Apr 2023
Object-centric Inference for Language Conditioned Placement: A Foundation Model based Approach Zhi-Wei Xu Kechun Xu Yue Wang R. Xiong OCL 18 4 0 06 Apr 2023
Grounding Object Relations in Language-Conditioned Robotic Manipulation with Semantic-Spatial Reasoning Qian Luo Yunfei Li Yi Wu LM&Ro 40 5 0 31 Mar 2023
When Learning Is Out of Reach, Reset: Generalization in Autonomous Visuomotor Reinforcement Learning Zichen Zhang Luca Weihs OffRL 24 5 0 30 Mar 2023
Language Models can Solve Computer Tasks Geunwoo Kim Pierre Baldi Stephen Marcus McAleer LLMAG LM&Ro 43 342 0 30 Mar 2023
Seer: Language Instructed Video Prediction with Latent Diffusion Models Xianfan Gu Chuan Wen Weirui Ye Jiaming Song Yang Gao DiffM VGen 21 40 0 27 Mar 2023
SEAL: Semantic Frame Execution And Localization for Perceiving Afforded Robot Actions Cameron Kisailus Daksh Narang Matthew P Shannon Odest Chadwicke Jenkins 26 0 0 24 Mar 2023
Text2Motion: From Natural Language Instructions to Feasible Plans Kevin Qinghong Lin Christopher Agia Toki Migimatsu Marco Pavone Jeannette Bohg LM&Ro 23 266 0 21 Mar 2023
Large AI Models in Health Informatics: Applications, Challenges, and the Future Jianing Qiu Lin Li Jiankai Sun Jiachuan Peng Peilun Shi ... Bo Xiao Wu Yuan Ningli Wang Dong Xu Benny Lo AI4MH LM&MA 42 127 0 21 Mar 2023
Chat with the Environment: Interactive Multimodal Perception Using Large Language Models Xufeng Zhao Mengdi Li C. Weber Muhammad Burhan Hafez S. Wermter LLMAG LM&Ro LRM 107 47 0 14 Mar 2023
WDiscOOD: Out-of-Distribution Detection via Whitened Linear Discriminant Analysis Yiye Chen Yunzhi Lin Ruinian Xu Patricio A. Vela OODD 29 3 0 14 Mar 2023
Audio Visual Language Maps for Robot Navigation Chen Huang Oier Mees Andy Zeng Wolfram Burgard VGen 65 33 0 13 Mar 2023
Robotic Applications of Pre-Trained Vision-Language Models to Various Recognition Behaviors Kento Kawaharazuka Yoshiki Obinata Naoaki Kanazawa K. Okada Masayuki Inaba LM&Ro 30 11 0 10 Mar 2023
Foundation Models for Decision Making: Problems, Methods, and Opportunities Sherry Yang Ofir Nachum Yilun Du Jason W. Wei Pieter Abbeel Dale Schuurmans LM&Ro OffRL LRM AI4CE 90 155 0 07 Mar 2023
PaLM-E: An Embodied Multimodal Language Model Danny Driess F. Xia Mehdi S. M. Sajjadi Corey Lynch Aakanksha Chowdhery ... Marc Toussaint Klaus Greff Andy Zeng Igor Mordatch Peter R. Florence LM&Ro 22 1,565 0 06 Mar 2023
HiCLIP: Contrastive Language-Image Pretraining with Hierarchy-aware Attention Shijie Geng Jianbo Yuan Yu Tian Yuxiao Chen Yongfeng Zhang CLIP VLM 43 44 0 06 Mar 2023
Naming Objects for Vision-and-Language Manipulation Tokuhiro Nishikawa Kazumi Aoyama Shunichi Sekiguchi Takayoshi Takayanagi Jianing Wu Yu Ishihara Tamaki Kojima Jerry Jun Yokono 32 1 0 06 Mar 2023
Learning Language-Conditioned Deformable Object Manipulation with Graph Dynamics Yuhong Deng Kaichun Mo Chongkun Xia Xueqian Wang AI4CE 36 11 0 02 Mar 2023
Open-World Object Manipulation using Pre-trained Vision-Language Models Austin Stone Ted Xiao Yao Lu K. Gopalakrishnan Kuang-Huei Lee ... Sean Kirmani Brianna Zitkovich F. Xia Chelsea Finn Karol Hausman LM&Ro 156 145 0 02 Mar 2023
Grounded Decoding: Guiding Text Generation with Grounded Models for Embodied Agents Wenlong Huang Fei Xia Dhruv Shah Danny Driess Andy Zeng ... Pete Florence Igor Mordatch Sergey Levine Karol Hausman Brian Ichter LM&Ro 24 42 0 01 Mar 2023
Task-Oriented Grasp Prediction with Visual-Language Inputs Chao Tang Dehao Huang Lingxiao Meng Weiyu Liu Hong Zhang 28 33 0 28 Feb 2023
ReorientDiff: Diffusion Model based Reorientation for Object Manipulation Utkarsh Aashu Mishra Yongxin Chen 26 20 0 28 Feb 2023
Semantic Mechanical Search with Large Vision and Language Models Satvik Sharma Huang Huang K. Shivakumar A. Imran Ryan Hoque Brian Ichter Ken Goldberg LM&Ro VLM 26 5 0 24 Feb 2023
Language-Driven Representation Learning for Robotics Siddharth Karamcheti Suraj Nair Annie S. Chen Thomas Kollar Chelsea Finn Dorsa Sadigh Percy Liang LM&Ro SSL 41 145 0 24 Feb 2023
A Joint Modeling of Vision-Language-Action for Target-oriented Grasping in Clutter Kechun Xu Shuqing Zhao Zhongxiang Zhou Zizhang Li Huaijin Pi Yifeng Zhu Yue Wang R. Xiong 30 46 0 24 Feb 2023
MimicPlay: Long-Horizon Imitation Learning by Watching Human Play Chen Wang Linxi Fan Jiankai Sun Ruohan Zhang Li Fei-Fei Danfei Xu Yuke Zhu Anima Anandkumar 36 184 0 24 Feb 2023
Scaling Robot Learning with Semantically Imagined Experience Tianhe Yu Ted Xiao Austin Stone Jonathan Tompson Anthony Brohan ... M. Dee Jodilyn Peralta Brian Ichter Karol Hausman F. Xia LM&Ro DiffM 36 144 0 22 Feb 2023
Deep Reinforcement Learning Based on Local GNN for Goal-conditioned Deformable Object Rearranging Yuhong Deng Chongkun Xia Xueqian Wang Lipeng Chen 26 15 0 21 Feb 2023
Graph-Transporter: A Graph-based Learning Method for Goal-Conditioned Deformable Object Rearranging Task Yuhong Deng Chongkun Xia Xueqian Wang Lipeng Chen 21 4 0 21 Feb 2023
ChatGPT for Robotics: Design Principles and Model Abilities Sai H. Vemprala Rogerio Bonatti A. Bucker Ashish Kapoor LM&Ro 33 459 0 20 Feb 2023
Train What You Know -- Precise Pick-and-Place with Transporter Networks Gergely Sóti Xi Huang C. Wurll B. Hein 27 6 0 17 Feb 2023
GenAug: Retargeting behaviors to unseen situations via Generative Augmentation Zoey Chen Sho Kiami Abhishek Gupta Vikash Kumar LM&Ro 27 83 0 13 Feb 2023
Combining Reconstruction and Contrastive Methods for Multimodal Representations in RL P. Becker Sebastian Mossburger Fabian Otto Gerhard Neumann SSL 34 2 0 10 Feb 2023
Multi-View Masked World Models for Visual Robotic Manipulation Younggyo Seo Junsup Kim Stephen James Kimin Lee Jinwoo Shin Pieter Abbeel VGen 22 55 0 05 Feb 2023
Aligning Robot and Human Representations Andreea Bobu Andi Peng Pulkit Agrawal Julie A. Shah Anca D. Dragan 48 10 0 03 Feb 2023
Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agents Zihao Wang Shaofei Cai Guanzhou Chen Anji Liu Xiaojian Ma Yitao Liang LM&Ro LLMAG 60 315 0 03 Feb 2023
Learning Universal Policies via Text-Guided Video Generation Yilun Du Mengjiao Yang Bo Dai H. Dai Ofir Nachum J. Tenenbaum Dale Schuurmans Pieter Abbeel PINN LM&Ro 19 231 0 31 Jan 2023
Distilling Internet-Scale Vision-Language Models into Embodied Agents T. Sumers Kenneth Marino Arun Ahuja Rob Fergus Ishita Dasgupta LM&Ro 28 24 0 29 Jan 2023
LEGO-Net: Learning Regular Rearrangements of Objects in Rooms Qiuhong Anna Wei Sijie Ding Jeong Joon Park Rahul Sajnani A. Poulenard Srinath Sridhar Leonidas J. Guibas DiffM 30 61 0 23 Jan 2023
Learning Bidirectional Action-Language Translation with Limited Supervision and Incongruent Input Ozan Ozdemir Matthias Kerzel C. Weber Jae Hee Lee Muhammad Burhan Hafez P. Bruns S. Wermter 24 1 0 09 Jan 2023
"No, to the Right" -- Online Language Corrections for Robotic Manipulation via Shared Autonomy Yuchen Cui Siddharth Karamcheti Raj Palleti Nidhya Shivakumar Percy Liang Dorsa Sadigh LM&Ro 32 76 0 06 Jan 2023
On Transforming Reinforcement Learning by Transformer: The Development Trajectory Shengchao Hu Li Shen Ya-Qin Zhang Yixin Chen Dacheng Tao OffRL 27 25 0 29 Dec 2022
Pre-Trained Image Encoder for Generalizable Visual Reinforcement Learning Zhecheng Yuan Zhengrong Xue Bo Yuan Xueqian Wang Yi Wu Yang Gao Huazhe Xu SSL OffRL 38 70 0 17 Dec 2022
Policy Adaptation from Foundation Model Feedback Yuying Ge Annabella Macaluso Erran L. Li Ping Luo Xiaolong Wang LM&Ro 27 12 0 14 Dec 2022
CREPE: Can Vision-Language Foundation Models Reason Compositionally? Zixian Ma Jerry Hong Mustafa Omer Gul Mona Gandhi Irena Gao Ranjay Krishna CoGe 29 125 0 13 Dec 2022
RT-1: Robotics Transformer for Real-World Control at Scale Anthony Brohan Noah Brown Justice Carbajal Yevgen Chebotar Joseph Dabis ... Ted Xiao Peng-Tao Xu Sichun Xu Tianhe Yu Brianna Zitkovich LM&Ro 33 1,021 0 13 Dec 2022
MIRA: Mental Imagery for Robotic Affordances Lin Yen-Chen Pete Florence Andy Zeng Jonathan T. Barron Yilun Du Wei-Chiu Ma Anthony Simeonov Alberto Rodriguez Garcia Phillip Isola LM&Ro 26 33 0 12 Dec 2022
OpenD: A Benchmark for Language-Driven Door and Drawer Opening Yizhou Zhao Qiaozi Gao Liang Qiu Govind Thattai Gaurav Sukhatme 29 5 0 10 Dec 2022