A Sober Look at Progress in Language Model Reasoning: Pitfalls and Paths to Reproducibility

9 April 2025

Papers citing "A Sober Look at Progress in Language Model Reasoning: Pitfalls and Paths to Reproducibility"

50 / 56 papers shown

Title
Walk Before You Run! Concise LLM Reasoning via Reinforcement Learning Mingyang Song Mao Zheng OffRL LRM 77 1 0 27 May 2025
SHARP: Synthesizing High-quality Aligned Reasoning Problems for Large Reasoning Models Reinforcement Learning Xiong Jun Wu Zhenduo Zhang ZuJie Wen Zhiqiang Zhang Wang Ren ... Xudong Han Chengfu Tang Dingnan Jin Qing Cui Jun Zhou LRM 190 1 0 20 May 2025
Reinforcement Learning vs. Distillation: Understanding Accuracy and Capability in LLM Reasoning Minwu Kim Anubhav Shrestha Safal Shrestha Aadim Nepal Keith Ross 58 0 0 20 May 2025
Warm Up Before You Train: Unlocking General Reasoning in Resource-Constrained Settings Safal Shrestha Minwu Kim Aadim Nepal Anubhav Shrestha Keith Ross OffRL ReLM LRM 77 0 0 19 May 2025
Tiny QA Benchmark++: Ultra-Lightweight, Synthetic Multilingual Dataset Generation & Smoke-Tests for Continuous LLM Evaluation Vincent Koc LM&MA 72 0 0 17 May 2025
X-Reasoner: Towards Generalizable Reasoning Across Modalities and Domains Qianchu Liu Sheng Zhang Guanghui Qin Timothy Ossowski Yu Gu ... Sam Preston Mu-Hsin Wei Paul Vozila Tristan Naumann Hoifung Poon OOD LRM VLM 110 8 0 06 May 2025
Phi-4-reasoning Technical Report Marah Abdin Sahaj Agarwal Ahmed Hassan Awadallah Vidhisha Balachandran Harkirat Singh Behl ... Vaishnavi Shrivastava Vibhav Vineet Yue Wu Safoora Yousefi Guoqing Zheng ReLM LRM 197 15 0 30 Apr 2025
The Hitchhiker's Guide to Program Analysis, Part II: Deep Thoughts by LLMs Haonan Li Hang Zhang Kexin Pei Zhiyun Qian 108 1 0 16 Apr 2025
VAPO: Efficient and Reliable Reinforcement Learning for Advanced Reasoning Tasks Yu Yue Yufeng Yuan Qiying Yu Xiaochen Zuo Ruofei Zhu ... Ru Zhang Xin Liu Mingxuan Wang Yonghui Wu Lin Yan OffRL LRM 117 38 0 07 Apr 2025
Rethinking RL Scaling for Vision Language Models: A Transparent, From-Scratch Framework and Comprehensive Evaluation Scheme Yan Ma Steffi Chern Xuyang Shen Yiran Zhong Pengfei Liu OffRL LRM 119 9 0 03 Apr 2025
Recitation over Reasoning: How Cutting-Edge Language Models Can Fail on Elementary School-Level Reasoning Problems? Kai Yan Yufei Xu Zhengyin Du Xuesong Yao Ziyi Wang Xiaowen Guo Jiecao Chen ReLM ELM LRM 178 5 0 01 Apr 2025
Exploring Data Scaling Trends and Effects in Reinforcement Learning from Human Feedback Wei Shen Guanlin Liu Zheng Wu Ruofei Zhu Qingping Yang Chao Xin Yu Yue Lin Yan 118 14 0 28 Mar 2025
Proof or Bluff? Evaluating LLMs on 2025 USA Math Olympiad Ivo Petrov Jasper Dekoninck Lyuben Baltadzhiev Maria Drencheva Kristian Minchev Mislav Balunović Nikola Jovanović Martin Vechev LRM ELM 115 22 0 27 Mar 2025
Video-R1: Reinforcing Video Reasoning in MLLMs Kaituo Feng Kaixiong Gong Yangqiu Song Zonghao Guo Yibing Wang Tianshuo Peng Jian Wu Xiaoying Zhang Benyou Wang Xiangyu Yue AI4TS SyDa LRM 148 62 0 27 Mar 2025
Understanding R1-Zero-Like Training: A Critical Perspective Zichen Liu Changyu Chen Wenjun Li Penghui Qi Tianyu Pang Chao Du Wee Sun Lee Min Lin OffRL LRM 194 168 0 26 Mar 2025
One Framework to Rule Them All: Unifying RL-Based and RL-Free Methods in RLHF Xin Cai 72 1 0 25 Mar 2025
SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild Weihao Zeng Yuzhen Huang Qian Liu Wei Liu Keqing He Zejun Ma Junxian He OffRL ReLM LRM 171 135 0 24 Mar 2025
Mind with Eyes: from Language Reasoning to Multimodal Reasoning Zhiyu Lin Yifei Gao Xian Zhao Yunfan Yang Jitao Sang LRM 134 5 0 23 Mar 2025
Reinforcement Learning for Reasoning in Small LLMs: What Works and What Doesn't Quy-Anh Dang Chris Ngo OffRL LRM 150 20 0 20 Mar 2025
Tapered Off-Policy REINFORCE: Stable and efficient reinforcement learning for LLMs Nicolas Le Roux Marc G. Bellemare Jonathan Lebensold Arnaud Bergeron Joshua Greaves Alex Fréchette Carolyne Pelletier Eric Thibodeau-Laufer Sándor Toth Sam Work OffRL 162 6 0 18 Mar 2025
DAPO: An Open-Source LLM Reinforcement Learning System at Scale Qiying Yu Zheng Zhang Ruofei Zhu Yufeng Yuan Xiaochen Zuo ... Ya Zhang Lin Yan Mu Qiao Yonghui Wu Mingxuan Wang OffRL LRM 202 215 0 18 Mar 2025
Enhancing LLM Reasoning with Iterative DPO: A Comprehensive Empirical Investigation Songjun Tu Jiahao Lin Xiangyu Tian Qichao Zhang Linjing Li ... Nan Xu Wei He Xiangyuan Lan D. Jiang Dongbin Zhao LRM 136 6 0 17 Mar 2025
LMM-R1: Empowering 3B LMMs with Strong Reasoning Abilities Through Two-Stage Rule-Based RL Yingzhe Peng Gongrui Zhang Miaosen Zhang Zhiyuan You Jie Liu Qipeng Zhu Kai Yang Xingzhong Xu Xin Geng Xu Yang LRM ReLM 187 87 0 10 Mar 2025
Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models Wenxuan Huang Bohan Jia Zijie Zhai Shaosheng Cao Zheyu Ye Fei Zhao Zhe Xu Yao Hu Shaohui Lin MU OffRL LRM MLLM ReLM VLM 144 130 0 09 Mar 2025
Med-RLVR: Emerging Medical Reasoning from a 3B base model via reinforcement Learning Sheng Zhang Qianchu Liu Guanghui Qin Tristan Naumann Hoifung Poon ReLM OffRL LRM 115 9 0 27 Feb 2025
Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement Learning Tian Xie Zitian Gao Qingnan Ren Haoming Luo Yuqian Hong Bryan Dai Joey Zhou Kai Qiu Zhirong Wu Chong Luo ReLM OffRL LRM 139 79 0 21 Feb 2025
NaturalReasoning: Reasoning in the Wild with 2.8M Challenging Questions Weizhe Yuan Jane Dwivedi-Yu Song Jiang Karthik Padthe Yang Li ... Ilia Kulikov Kyunghyun Cho Yuandong Tian Jason Weston Xian Li ReLM LRM 123 20 0 18 Feb 2025
VersaPRM: Multi-Domain Process Reward Model via Synthetic Reasoning Data Thomas Zeng Shuibai Zhang Shutong Wu Christian Classen Daewon Chae ... Jungtaek Kim H. Koo Kannan Ramchandran Dimitris Papailiopoulos Kangwook Lee LRM 101 4 0 10 Feb 2025
Process Reinforcement through Implicit Rewards Ganqu Cui Lifan Yuan Ziyi Wang Hanbin Wang Wendi Li ... Yu Cheng Zhiyuan Liu Maosong Sun Bowen Zhou Ning Ding OffRL LRM 153 103 0 03 Feb 2025
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning DeepSeek-AI Daya Guo Dejian Yang Haowei Zhang Junxiao Song ... Shiyu Wang S. Yu Shunfeng Zhou Shuting Pan S.S. Li ReLM VLM OffRL AI4TS LRM 380 1,970 0 22 Jan 2025
Kimi k1.5: Scaling Reinforcement Learning with LLMs Kimi Team Angang Du Bofei Gao Bowei Xing Changjiu Jiang ... Zihao Huang Ziyao Xu Zhiyong Yang Zonghan Yang Zongyu Lin OffRL ALM AI4TS VLM LRM 277 330 0 22 Jan 2025
ONEBench to Test Them All: Sample-Level Benchmarking Over Open-Ended Capabilities Adhiraj Ghosh Sebastian Dziadzio Ameya Prabhu Vishaal Udandarao Samuel Albanie Matthias Bethge MLLM ELM 103 4 0 09 Dec 2024
On Designing Effective RL Reward at Training Time for LLM Reasoning Jiaxuan Gao Shusheng Xu Wenjie Ye Weilin Liu Chuyi He Wei Fu Zhiyu Mei Guangju Wang Yi Wu OffRL LRM 106 23 0 19 Oct 2024
GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models Iman Mirzadeh Keivan Alizadeh Hooman Shahrokhi Oncel Tuzel Samy Bengio Mehrdad Farajtabar AIMat LRM 103 184 0 07 Oct 2024
CodePMP: Scalable Preference Model Pretraining for Large Language Model Reasoning Huimu Yu Xing Wu Weidong Yin Debing Zhang Songlin Hu LRM 68 5 0 03 Oct 2024
Training on the Test Task Confounds Evaluation and Emergence Ricardo Dominguez-Olmedo Florian E. Dorner Moritz Hardt ELM 133 9 1 10 Jul 2024
Position: Benchmarking is Limited in Reinforcement Learning Research Scott M. Jordan Adam White Bruno Castro da Silva Martha White Philip S. Thomas OffRL 39 8 0 23 Jun 2024
RLSF: Reinforcement Learning via Symbolic Feedback Piyush Jha Prithwish Jana Arnav Arora Vijay Ganesh LRM 60 4 0 26 May 2024
Functional Benchmarks for Robust Evaluation of Reasoning Performance, and the Reasoning Gap Saurabh Srivastava B. AnnaroseM V. AntoP Shashank Menon Ajay Sukumar T. AdwaithSamod Alan Philipose Stevin Prince Sooraj Thomas ELM ReLM LRM 63 56 0 29 Feb 2024
OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problems Chaoqun He Renjie Luo Yuzhuo Bai Shengding Hu Zhen Leng Thai ... Yuxiang Zhang Jie Liu Lei Qi Zhiyuan Liu Maosong Sun ELM AIMat 122 279 0 21 Feb 2024
DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models Zhihong Shao Peiyi Wang Qihao Zhu Runxin Xu Jun-Mei Song ... Haowei Zhang Mingchuan Zhang Yiming Li Yu-Huan Wu Daya Guo ReLM LRM 146 1,274 0 05 Feb 2024
The Neglected Tails in Vision-Language Models Shubham Parashar Zhiqiu Lin Tian Liu Xiangjue Dong Yanan Li Deva Ramanan James Caverlee Shu Kong VLM 92 38 0 23 Jan 2024
Let's Verify Step by Step Hunter Lightman V. Kosaraju Yura Burda Harrison Edwards Bowen Baker Teddy Lee Jan Leike John Schulman Ilya Sutskever K. Cobbe ALM OffRL LRM 195 1,233 0 31 May 2023
Empirical Design in Reinforcement Learning Andrew Patterson Samuel Neumann Martha White Adam White 97 28 0 03 Apr 2023
Large Language Models Struggle to Learn Long-Tail Knowledge Nikhil Kandpal H. Deng Adam Roberts Eric Wallace Colin Raffel RALM KELM 128 417 0 15 Nov 2022
Towards a Standardised Performance Evaluation Protocol for Cooperative MARL R. Gorsane Omayma Mahjoub Ruan de Kock Roland Dubb Siddarth S. Singh Arnu Pretorius OffRL 76 50 0 21 Sep 2022
Solving Quantitative Reasoning Problems with Language Models Aitor Lewkowycz Anders Andreassen David Dohan Ethan Dyer Henryk Michalewski ... Theo Gutman-Solo Yuhuai Wu Behnam Neyshabur Guy Gur-Ari Vedant Misra ReLM ELM LRM 181 857 0 29 Jun 2022
Evaluation Gaps in Machine Learning Practice Ben Hutchinson Negar Rostamzadeh Christina Greer Katherine A. Heller Vinodkumar Prabhakaran ELM 87 63 0 11 May 2022
Deep Reinforcement Learning at the Edge of the Statistical Precipice Rishabh Agarwal Max Schwarzer Pablo Samuel Castro Aaron Courville Marc G. Bellemare OffRL 123 676 0 30 Aug 2021
The Benchmark Lottery Mostafa Dehghani Yi Tay A. Gritsenko Zhe Zhao N. Houlsby Fernando Diaz Donald Metzler Oriol Vinyals 99 92 0 14 Jul 2021