Autonomous Improvement of Instruction Following Skills via Foundation Models

30 July 2024

Papers citing "Autonomous Improvement of Instruction Following Skills via Foundation Models"

43 / 43 papers shown

Title
The Latent Space Hypothesis: Toward Universal Medical Representation Learning Salil Patel 134 0 0 04 Jun 2025
Robotic Control via Embodied Chain-of-Thought Reasoning Michał Zawalski William Chen Karl Pertsch Oier Mees Chelsea Finn Sergey Levine LRM LM&Ro 123 88 0 11 Jul 2024
Long-horizon Locomotion and Manipulation on a Quadrupedal Robot with Large Language Models Yutao Ouyang Jinhan Li Yunfei Li Zhongyu Li Chao Yu Koushil Sreenath Yi Wu 108 15 0 08 Apr 2024
MLDT: Multi-Level Decomposition for Complex Long-Horizon Robotic Task Planning with Open-Source Large Language Model Yike Wu Jiatao Zhang Nan Hu LanLing Tang Guilin Qi Jun Shao Jie Ren Wei Song 87 12 0 27 Mar 2024
DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset Alexander Khazatsky Karl Pertsch Suraj Nair Ashwin Balakrishna Sudeep Dasari ... Thomas Kollar Sergey Levine Chelsea Finn Sergey Levine Chelsea Finn 223 221 0 19 Mar 2024
SERL: A Software Suite for Sample-Efficient Robotic Reinforcement Learning Jianlan Luo Zheyuan Hu Charles Xu You Liang Tan Jacob Berg Archit Sharma S. Schaal Chelsea Finn Abhishek Gupta Sergey Levine OffRL OnRL 95 48 0 29 Jan 2024
Mastering Stacking of Diverse Shapes with Large-Scale Iterative Reinforcement Learning on Real Robots Thomas Lampe A. Abdolmaleki Sarah Bechtle Sandy H. Huang Jost Tobias Springenberg ... Markus Wulfmeier Jingwei Zhang Francesco Nori N. Heess Martin Riedmiller OffRL 64 9 0 18 Dec 2023
CogVLM: Visual Expert for Pretrained Language Models Weihan Wang Qingsong Lv Wenmeng Yu Wenyi Hong Ji Qi ... Bin Xu Juanzi Li Yuxiao Dong Ming Ding Jie Tang VLM MLLM 97 502 0 06 Nov 2023
SMART-LLM: Smart Multi-Agent Robot Task Planning using Large Language Models S. S. Kannan Vishnunandan L. N. Venkatesh Byung-Cheol Min LLMAG LM&Ro 64 121 0 18 Sep 2023
VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models Wenlong Huang Chen Wang Ruohan Zhang Yunzhu Li Jiajun Wu Li Fei-Fei LM&Ro 111 510 0 12 Jul 2023
GPT-4 Technical Report OpenAI OpenAI OpenAI Josh Achiam Steven Adler Sandhini Agarwal Lama Ahmad ... Shengjia Zhao Tianhao Zheng Juntang Zhuang William Zhuk Barret Zoph LLMAG MLLM 1.5K 14,631 0 15 Mar 2023
Self-Improving Robots: End-to-End Autonomous Visuomotor Reinforcement Learning Archit Sharma Ahmed M. Ahmed Rehaan Ahmad Chelsea Finn SSL 108 17 0 02 Mar 2023
Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering Pan Lu Swaroop Mishra Tony Xia Liang Qiu Kai-Wei Chang Song-Chun Zhu Oyvind Tafjord Peter Clark Ashwin Kalyan ELM ReLM LRM 283 1,286 0 20 Sep 2022
Latent Plans for Task-Agnostic Offline Reinforcement Learning Erick Rosete-Beas Oier Mees Gabriel Kalweit Joschka Boedecker Wolfram Burgard OffRL 106 83 0 19 Sep 2022
Don't Start From Scratch: Leveraging Prior Data to Automate Robotic Reinforcement Learning Homer Walke Jonathan Yang Albert Yu Aviral Kumar Jedrzej Orbik Avi Singh Sergey Levine OffRL OnRL 58 31 0 11 Jul 2022
Contrastive Learning as Goal-Conditioned Reinforcement Learning Benjamin Eysenbach Tianjun Zhang Ruslan Salakhutdinov Sergey Levine SSL OffRL 96 162 0 15 Jun 2022
A Generalist Agent Scott E. Reed Konrad Zolna Emilio Parisotto Sergio Gomez Colmenarejo Alexander Novikov ... Yutian Chen R. Hadsell Oriol Vinyals Mahyar Bordbar Nando de Freitas LM&Ro LLMAG AI4CE 211 823 0 12 May 2022
What Matters in Language Conditioned Robotic Imitation Learning over Unstructured Data Oier Mees Lukás Hermann Wolfram Burgard LM&Ro 105 155 0 13 Apr 2022
Hierarchical Text-Conditional Image Generation with CLIP Latents Aditya A. Ramesh Prafulla Dhariwal Alex Nichol Casey Chu Mark Chen VLM DiffM 410 6,897 0 13 Apr 2022
PaLM: Scaling Language Modeling with Pathways Aakanksha Chowdhery Sharan Narang Jacob Devlin Maarten Bosma Gaurav Mishra ... Kathy Meier-Hellstern Douglas Eck J. Dean Slav Petrov Noah Fiedel PILM LRM 512 6,279 0 05 Apr 2022
Do As I Can, Not As I Say: Grounding Language in Robotic Affordances Michael Ahn Anthony Brohan Noah Brown Yevgen Chebotar Omar Cortes ... Ted Xiao Peng Xu Sichun Xu Mengyuan Yan Andy Zeng LM&Ro 189 1,984 0 04 Apr 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 880 13,148 0 04 Mar 2022
BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning Eric Jang A. Irpan Mohi Khansari Daniel Kappler F. Ebert Corey Lynch Sergey Levine Chelsea Finn LM&Ro 263 542 0 04 Feb 2022
Goal-Conditioned Reinforcement Learning: Problems and Solutions Minghuan Liu Menghui Zhu Weinan Zhang 87 142 0 20 Jan 2022
High-Resolution Image Synthesis with Latent Diffusion Models Robin Rombach A. Blattmann Dominik Lorenz Patrick Esser Bjorn Ommer 3DV 463 15,665 0 20 Dec 2021
AW-Opt: Learning Robotic Skills with Imitation and Reinforcement at Scale Yao Lu Karol Hausman Yevgen Chebotar Mengyuan Yan Eric Jang ... Ted Xiao A. Irpan Mohi Khansari Dmitry Kalashnikov Sergey Levine OffRL 179 60 0 09 Nov 2021
Learning Language-Conditioned Robot Behavior from Offline Data and Crowd-Sourced Annotation Suraj Nair E. Mitchell Kevin Chen Brian Ichter Silvio Savarese Chelsea Finn LM&Ro OffRL 105 158 0 02 Sep 2021
Goal-Conditioned Reinforcement Learning with Imagined Subgoals Elliot Chane-Sane Cordelia Schmid Ivan Laptev 77 144 0 01 Jul 2021
MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scale Dmitry Kalashnikov Jacob Varley Yevgen Chebotar Benjamin Swanson Rico Jonschkowski Chelsea Finn Sergey Levine Karol Hausman OffRL 107 280 0 16 Apr 2021
Actionable Models: Unsupervised Offline Reinforcement Learning of Robotic Skills Yevgen Chebotar Karol Hausman Yao Lu Ted Xiao Dmitry Kalashnikov ... A. Irpan Benjamin Eysenbach Ryan Julian Chelsea Finn Sergey Levine SSL OffRL 72 153 0 15 Apr 2021
Rapid Exploration for Open-World Navigation with Latent Goal Models Dhruv Shah Benjamin Eysenbach G. Kahn Nicholas Rhinehart Sergey Levine 72 75 0 12 Apr 2021
Learning Transferable Visual Models From Natural Language Supervision Alec Radford Jong Wook Kim Chris Hallacy Aditya A. Ramesh Gabriel Goh ... Amanda Askell Pamela Mishkin Jack Clark Gretchen Krueger Ilya Sutskever CLIP VLM 967 29,731 0 26 Feb 2021
The Ingredients of Real-World Robotic Reinforcement Learning Henry Zhu Justin Yu Abhishek Gupta Dhruv Shah Kristian Hartikainen Avi Singh Vikash Kumar Sergey Levine OffRL 116 179 0 27 Apr 2020
Google Landmarks Dataset v2 -- A Large-Scale Benchmark for Instance-Level Recognition and Retrieval Tobias Weyand A. Araújo Bingyi Cao Jack Sim 78 369 0 03 Apr 2020
Towards VQA Models That Can Read Amanpreet Singh Vivek Natarajan Meet Shah Yu Jiang Xinlei Chen Dhruv Batra Devi Parikh Marcus Rohrbach EgoV 100 1,244 0 18 Apr 2019
Skew-Fit: State-Covering Self-Supervised Reinforcement Learning Vitchyr H. Pong Murtaza Dalal Steven Lin Ashvin Nair Shikhar Bahl Sergey Levine OffRL SSL 91 277 0 08 Mar 2019
Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Control F. Ebert Chelsea Finn Sudeep Dasari Annie Xie Alex X. Lee Sergey Levine SSL 110 392 0 03 Dec 2018
Group Normalization Yuxin Wu Kaiming He 233 3,669 0 22 Mar 2018
GONet: A Semi-Supervised Deep Learning Approach For Traversability Estimation Noriaki Hirose Amir Sadeghian Nathan Tsoi P. Goebel Silvio Savarese 61 77 0 08 Mar 2018
Hindsight Experience Replay Marcin Andrychowicz Dwight Crow Alex Ray Jonas Schneider Rachel Fong Peter Welinder Bob McGrew Joshua Tobin Pieter Abbeel Wojciech Zaremba OffRL 268 2,337 0 05 Jul 2017
Gaussian Error Linear Units (GELUs) Dan Hendrycks Kevin Gimpel 172 5,037 0 27 Jun 2016
VQA: Visual Question Answering Aishwarya Agrawal Jiasen Lu Stanislaw Antol Margaret Mitchell C. L. Zitnick Dhruv Batra Devi Parikh CoGe 214 5,497 0 03 May 2015
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift Sergey Ioffe Christian Szegedy OOD 463 43,328 0 11 Feb 2015