What Can RL Bring to VLA Generalization? An Empirical Study

26 May 2025

Papers citing "What Can RL Bring to VLA Generalization? An Empirical Study"

50 / 58 papers shown

Title
Interleave-VLA: Enhancing Robot Manipulation with Interleaved Image-Text Instructions Cunxin Fan Xiaosong Jia Yihang Sun Yixiao Wang Jianglan Wei ... Xiangyu Zhao Masayoshi Tomizuka Xue Yang Junchi Yan Mingyu Ding LM&Ro VLM 76 4 0 04 May 2025
$$π_{0.5}$: a Vision-Language-Action Model with Open-World Generalization$ $π_{0.5}$ : a Vision-Language-Action Model with Open-World Generalization Physical Intelligence Kevin Black Noah Brown James Darpinian Karan Dhabalia ... Homer Walke Anna Walling Haohuan Wang Lili Yu Ury Zhilinsky LM&Ro VLM 75 25 0 22 Apr 2025
Rethinking RL Scaling for Vision Language Models: A Transparent, From-Scratch Framework and Comprehensive Evaluation Scheme Yan Ma Steffi Chern Xuyang Shen Yiran Zhong Pengfei Liu OffRL LRM 81 8 0 03 Apr 2025
Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models Wenxuan Huang Bohan Jia Zijie Zhai Shaosheng Cao Zheyu Ye Fei Zhao Zhe Xu Yao Hu Shaohui Lin MU OffRL LRM MLLM ReLM VLM 93 85 0 09 Mar 2025
VLAS: Vision-Language-Action Model With Speech Instructions For Customized Robot Manipulation Wei Zhao Pengxiang Ding Hao Fei Zhefei Gong Shuanghao Bai Han Zhao Donglin Wang 105 7 0 24 Feb 2025
DexVLA: Vision-Language Model with Plug-In Diffusion Expert for General Robot Control Junjie Wen Yinlin Zhu Jinming Li Zhibin Tang Yaxin Peng Feifei Feng VLM 80 18 0 09 Feb 2025
ConRFT: A Reinforced Fine-tuning Method for VLA Models via Consistency Policy Yuhui Chen Shuai Tian Shugao Liu Yingting Zhou Haoran Li Dongbin Zhao OffRL 129 6 0 08 Feb 2025
Improving Vision-Language-Action Model with Online Reinforcement Learning Yanjiang Guo Jianke Zhang Xiaoyu Chen Xiang Ji Yen-Jen Wang Yucheng Hu Jianyu Chen VLM 74 10 0 28 Jan 2025
SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training Tianzhe Chu Yuexiang Zhai Jihan Yang Shengbang Tong Saining Xie Dale Schuurmans Quoc V. Le Sergey Levine Yi-An Ma OffRL 106 85 0 28 Jan 2025
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning DeepSeek-AI Daya Guo Dejian Yang Haowei Zhang Junxiao Song ... Shiyu Wang S. Yu Shunfeng Zhou Shuting Pan S.S. Li ReLM VLM OffRL AI4TS LRM 218 1,503 0 22 Jan 2025
Kimi k1.5: Scaling Reinforcement Learning with LLMs Kimi Team Angang Du Bofei Gao Bowei Xing Changjiu Jiang ... Zihao Huang Ziyao Xu Zhiyong Yang Zonghan Yang Zongyu Lin OffRL ALM AI4TS VLM LRM 178 250 0 22 Jan 2025
Policy Agnostic RL: Offline RL and Online RL Fine-Tuning of Any Class and Backbone Max Sobol Mark Tian Gao Georgia Gabriela Sampaio Mohan Kumar Srirama Archit Sharma Chelsea Finn Aviral Kumar OffRL OnRL 138 8 0 09 Dec 2024
RDT-1B: a Diffusion Foundation Model for Bimanual Manipulation Songming Liu Lingxuan Wu Bangguo Li Hengkai Tan Huayu Chen Zhengyi Wang Ke Xu Hang Su Jun Zhu 72 102 0 10 Oct 2024
GR-2: A Generative Video-Language-Action Model with Web-Scale Knowledge for Robot Manipulation Chi-Lam Cheang Guangzeng Chen Ya Jing Tao Kong Hang Li ... Hongtao Wu Jiafeng Xu Yichu Yang Hanbo Zhang Minzhao Zhu VGen LM&Ro 85 58 0 08 Oct 2024
ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AI Stone Tao Fanbo Xiang Arth Shukla Yuzhe Qin Xander Hinrichsen ... Shan Luo Roberto Calandra Rui Chen Shan Luo Hao Su 58 33 0 01 Oct 2024
FLaRe: Achieving Masterful and Adaptive Robot Policies with Large-Scale Reinforcement Learning Fine-Tuning Jiaheng Hu Rose Hendrix Ali Farhadi Aniruddha Kembhavi Roberto Martín-Martín Peter Stone Kuo-Hao Zeng Kiana Ehsani 69 12 0 25 Sep 2024
Diffusion Policy Policy Optimization Allen Z. Ren Justin Lidard Lars L. Ankile Anthony Simeonov Pulkit Agrawal Anirudha Majumdar Benjamin Burchfiel Hongkai Dai Max Simchowitz 94 49 0 01 Sep 2024
Unpacking DPO and PPO: Disentangling Best Practices for Learning from Preference Feedback Hamish Ivison Yizhong Wang Jiacheng Liu Zeqiu Wu Valentina Pyatkin Nathan Lambert Noah A. Smith Yejin Choi Hannaneh Hajishirzi 72 50 0 13 Jun 2024
OpenVLA: An Open-Source Vision-Language-Action Model Moo Jin Kim Karl Pertsch Siddharth Karamcheti Ted Xiao Ashwin Balakrishna ... Russ Tedrake Dorsa Sadigh Sergey Levine Percy Liang Chelsea Finn LM&Ro VLM 138 425 0 13 Jun 2024
A Survey on Vision-Language-Action Models for Embodied AI Yueen Ma Zixing Song Yuzheng Zhuang Jianye Hao Irwin King LM&Ro 188 49 0 23 May 2024
Octo: An Open-Source Generalist Robot Policy Octo Model Team Dibya Ghosh Homer Walke Karl Pertsch Kevin Black ... Quan Vuong Ted Xiao Dorsa Sadigh Chelsea Finn Sergey Levine 126 397 0 20 May 2024
Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learning Yuexiang Zhai Hao Bai Zipeng Lin Jiayi Pan Shengbang Tong ... Alane Suhr Saining Xie Yann LeCun Yi-An Ma Sergey Levine LLMAG LRM 74 72 0 16 May 2024
Evaluating Real-World Robot Manipulation Policies in Simulation Xuanlin Li Kyle Hsu Jiayuan Gu Karl Pertsch Oier Mees ... Jiajun Wu Chelsea Finn Hao Su Q. Vuong Ted Xiao OffRL 70 71 0 09 May 2024
Is DPO Superior to PPO for LLM Alignment? A Comprehensive Study Shusheng Xu Wei Fu Jiaxuan Gao Wenjie Ye Weiling Liu Zhiyu Mei Guangju Wang Chao Yu Yi Wu 84 149 0 16 Apr 2024
When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Method Biao Zhang Zhongtao Liu Colin Cherry Orhan Firat LRM 86 136 0 27 Feb 2024
Prismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Models Siddharth Karamcheti Suraj Nair Ashwin Balakrishna Percy Liang Thomas Kollar Dorsa Sadigh MLLM VLM 80 108 0 12 Feb 2024
DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models Zhihong Shao Peiyi Wang Qihao Zhu Runxin Xu Jun-Mei Song ... Haowei Zhang Mingchuan Zhang Yiming Li Yu-Huan Wu Daya Guo ReLM LRM 77 953 0 05 Feb 2024
A Survey of Reinforcement Learning from Human Feedback Timo Kaufmann Paul Weng Viktor Bengs Eyke Hüllermeier OffRL 24 134 0 22 Dec 2023
Unleashing Large-Scale Video Generative Pre-training for Visual Robot Manipulation Hongtao Wu Ya Jing Chi-Hou Cheang Guangzeng Chen Jiafeng Xu Xinghang Li Minghuan Liu Hang Li Tao Kong 54 101 0 20 Dec 2023
Foundation Models in Robotics: Applications, Challenges, and the Future Roya Firoozi Johnathan Tucker Stephen Tian Anirudha Majumdar Jiankai Sun ... Brian Ichter Danny Driess Jiajun Wu Cewu Lu Mac Schwager LM&Ro AI4CE LRM VLM 67 150 0 13 Dec 2023
ReMax: A Simple, Effective, and Efficient Reinforcement Learning Method for Aligning Large Language Models Ziniu Li Tian Xu Yushun Zhang Zhihang Lin Yang Yu Ruoyu Sun Zhimin Luo 43 63 0 16 Oct 2023
BridgeData V2: A Dataset for Robot Learning at Scale Homer Walke Kevin Black Abraham Lee Moo Jin Kim Maximilian Du ... Andre Wang He Vivek Myers Kuan Fang Chelsea Finn Sergey Levine 42 219 0 24 Aug 2023
Instruction Tuning for Large Language Models: A Survey Shengyu Zhang Linfeng Dong Xiaoya Li Sen Zhang Xiaofei Sun ... Jiwei Li Runyi Hu Tianwei Zhang Leilei Gan Guoyin Wang LM&MA 71 570 0 21 Aug 2023
RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control Anthony Brohan Noah Brown Justice Carbajal Yevgen Chebotar Xi Chen ... Ted Xiao Peng Xu Sichun Xu Tianhe Yu Brianna Zitkovich LM&Ro LRM 86 1,172 0 28 Jul 2023
Llama 2: Open Foundation and Fine-Tuned Chat Models Hugo Touvron Louis Martin Kevin R. Stone Peter Albert Amjad Almahairi ... Sharan Narang Aurelien Rodriguez Robert Stojnic Sergey Edunov Thomas Scialom AI4MH ALM 206 11,636 0 18 Jul 2023
Secrets of RLHF in Large Language Models Part I: PPO Rui Zheng Shihan Dou Songyang Gao Yuan Hua Wei Shen ... Hang Yan Tao Gui Qi Zhang Xipeng Qiu Xuanjing Huang ALM OffRL 61 163 0 11 Jul 2023
Data Quality in Imitation Learning Suneel Belkhale Yuchen Cui Dorsa Sadigh 42 51 0 04 Jun 2023
Direct Preference Optimization: Your Language Model is Secretly a Reward Model Rafael Rafailov Archit Sharma E. Mitchell Stefano Ermon Christopher D. Manning Chelsea Finn ALM 276 3,712 0 29 May 2023
Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware Tony Zhao Vikash Kumar Sergey Levine Chelsea Finn 28 574 0 23 Apr 2023
DINOv2: Learning Robust Visual Features without Supervision Maxime Oquab Timothée Darcet Théo Moutakanni Huy Q. Vo Marc Szafraniec ... Hervé Jégou Julien Mairal Patrick Labatut Armand Joulin Piotr Bojanowski VLM CLIP SSL 249 3,205 0 14 Apr 2023
Sigmoid Loss for Language Image Pre-Training Xiaohua Zhai Basil Mustafa Alexander Kolesnikov Lucas Beyer CLIP VLM 83 1,076 0 27 Mar 2023
GPT-4 Technical Report OpenAI OpenAI OpenAI Josh Achiam Steven Adler Sandhini Agarwal Lama Ahmad ... Shengjia Zhao Tianhao Zheng Juntang Zhuang William Zhuk Barret Zoph LLMAG MLLM 625 13,788 0 15 Mar 2023
Diffusion Policy: Visuomotor Policy Learning via Action Diffusion Cheng Chi Zhenjia Xu S. Feng Eric A. Cousineau Yilun Du Benjamin Burchfiel Russ Tedrake Shuran Song 221 1,138 0 07 Mar 2023
Open-World Object Manipulation using Pre-trained Vision-Language Models Austin Stone Ted Xiao Yao Lu K. Gopalakrishnan Kuang-Huei Lee ... Sean Kirmani Brianna Zitkovich F. Xia Chelsea Finn Karol Hausman LM&Ro 184 148 0 02 Mar 2023
Adding Conditional Control to Text-to-Image Diffusion Models Lvmin Zhang Anyi Rao Maneesh Agrawala AI4CE 70 4,015 1 10 Feb 2023
Objaverse: A Universe of Annotated 3D Objects Matt Deitke Dustin Schwenk Jordi Salvador Luca Weihs Oscar Michel Eli VanderBilt Ludwig Schmidt Kiana Ehsani Aniruddha Kembhavi Ali Farhadi 67 913 0 15 Dec 2022
RT-1: Robotics Transformer for Real-World Control at Scale Anthony Brohan Noah Brown Justice Carbajal Yevgen Chebotar Joseph Dabis ... Ted Xiao Peng Xu Sichun Xu Tianhe Yu Brianna Zitkovich LM&Ro 60 1,068 0 13 Dec 2022
LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Action Dhruv Shah B. Osinski Brian Ichter Sergey Levine LM&Ro 201 447 0 10 Jul 2022
Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback Yuntao Bai Andy Jones Kamal Ndousse Amanda Askell Anna Chen ... Jack Clark Sam McCandlish C. Olah Benjamin Mann Jared Kaplan 212 2,457 0 12 Apr 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 694 12,525 0 04 Mar 2022