Deciphering Trajectory-Aided LLM Reasoning: An Optimization Perspective

26 May 2025

Author Contacts:

zhangsongyang@pjlab.org.cn chenkai@pjlab.org.cn

LRM

ArXiv (abs)PDF HTML

Papers citing "Deciphering Trajectory-Aided LLM Reasoning: An Optimization Perspective"

50 / 58 papers shown

Title
Reasoning Models Can Be Effective Without Thinking Wenjie Ma Jingxuan He Charlie Snell Tyler Griggs Sewon Min Matei A. Zaharia ReLM LRM 107 46 1 14 Apr 2025
SFT or RL? An Early Investigation into Training R1-Like Reasoning Large Vision-Language Models Hardy Chen Haoqin Tu Fali Wang Hui Liu Xianfeng Tang Xinya Du Yuyin Zhou Cihang Xie ReLM VLM OffRL LRM 132 27 0 10 Apr 2025
VAPO: Efficient and Reliable Reinforcement Learning for Advanced Reasoning Tasks Yu Yue Yufeng Yuan Qiying Yu Xiaochen Zuo Ruofei Zhu ... Ru Zhang Xin Liu Mingxuan Wang Yonghui Wu Lin Yan OffRL LRM 112 30 0 07 Apr 2025
Reasoning Models Know When They're Right: Probing Hidden States for Self-Verification Anqi Zhang Yulin Chen Jane Pan Chen Zhao Aurojit Panda Jinyang Li He He ReLM LRM 108 15 0 07 Apr 2025
Rethinking Reflection in Pre-Training Essential AI Darsh J Shah Peter Rushton Somanshu Singla Mohit Parmar ... Philip Monk Platon Mazarakis Ritvik Kapila Saurabh Srivastava Tim Romanski ReLM LRM 130 12 0 05 Apr 2025
A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyond Xiaoye Qu Yafu Li Zhaochen Su Weigao Sun Jianhao Yan ... Chaochao Lu Yue Zhang Xian-Sheng Hua Bowen Zhou Yu Cheng ReLM OffRL LRM 165 43 0 27 Mar 2025
DAPO: An Open-Source LLM Reinforcement Learning System at Scale Qiying Yu Zheng Zhang Ruofei Zhu Yufeng Yuan Xiaochen Zuo ... Ya Zhang Lin Yan Mu Qiao Yonghui Wu Mingxuan Wang OffRL LRM 195 175 0 18 Mar 2025
All Roads Lead to Likelihood: The Value of Reinforcement Learning in Fine-Tuning Gokul Swamy Sanjiban Choudhury Wen Sun Zhiwei Steven Wu J. Andrew Bagnell OffRL 127 16 0 03 Mar 2025
Transformers Learn to Implement Multi-step Gradient Descent with Chain of Thought Jianhao Huang Zixuan Wang Jason D. Lee LRM 65 3 0 28 Feb 2025
Towards Thinking-Optimal Scaling of Test-Time Compute for LLM Reasoning Wenkai Yang Shuming Ma Yankai Lin Furu Wei LRM 94 45 0 25 Feb 2025
Atom of Thoughts for Markov LLM Test-Time Scaling Fengwei Teng Zhaoyang Yu Quan Shi Jiayi Zhang Chenglin Wu Yuyu Luo MU LRM 121 22 0 17 Feb 2025
LIMO: Less is More for Reasoning Yixin Ye Zhen Huang Yang Xiao Ethan Chern Shijie Xia Pengfei Liu LRM 164 147 0 05 Feb 2025
SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training Tianzhe Chu Yuexiang Zhai Jihan Yang Shengbang Tong Saining Xie Dale Schuurmans Quoc V. Le Sergey Levine Yi-An Ma OffRL 222 106 0 28 Jan 2025
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning DeepSeek-AI Daya Guo Dejian Yang Haowei Zhang Junxiao Song ... Shiyu Wang S. Yu Shunfeng Zhou Shuting Pan S.S. Li ReLM VLM OffRL AI4TS LRM 370 1,692 0 22 Jan 2025
Kimi k1.5: Scaling Reinforcement Learning with LLMs Kimi Team Angang Du Bofei Gao Bowei Xing Changjiu Jiang ... Zihao Huang Ziyao Xu Zhiyong Yang Zonghan Yang Zongyu Lin OffRL ALM AI4TS VLM LRM 248 274 0 22 Jan 2025
O1 Replication Journey: A Strategic Progress Report -- Part 1 Yiwei Qin Xuefeng Li Haoyang Zou Yixiu Liu Shijie Xia ... Yixin Ye Weizhe Yuan Hector Liu Yuezun Li Pengfei Liu VLM 87 88 0 08 Oct 2024
HybridFlow: A Flexible and Efficient RLHF Framework Guangming Sheng Chi Zhang Zilingfeng Ye Xibin Wu Wang Zhang Ru Zhang Size Zheng Haibin Lin Chuan Wu AI4CE 163 201 0 28 Sep 2024
Qwen2.5-Coder Technical Report Binyuan Hui Jian Yang Zeyu Cui Jiaxi Yang Dayiheng Liu ... Fei Huang Xingzhang Ren Xuancheng Ren Jingren Zhou Junyang Lin OSLM 104 306 0 18 Sep 2024
$μ$ LO: Compute-Efficient Meta-Generalization of Learned Optimizers Benjamin Thérien Charles-Étienne Joseph Boris Knyazev Edouard Oyallon Irina Rish Eugene Belilovsky AI4CE 120 3 0 31 May 2024
In-Context Learning with Long-Context Models: An In-Depth Exploration Amanda Bertsch Maor Ivgi Uri Alon Jonathan Berant Matthew R. Gormley Matthew R. Gormley Graham Neubig ReLM AIMat 170 80 0 30 Apr 2024
DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models Zhihong Shao Peiyi Wang Qihao Zhu Runxin Xu Jun-Mei Song ... Haowei Zhang Mingchuan Zhang Yiming Li Yu-Huan Wu Daya Guo ReLM LRM 138 1,119 0 05 Feb 2024
MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical Reasoning Ke Wang Houxing Ren Aojun Zhou Zimu Lu Sichun Luo Weikang Shi Renrui Zhang Linqi Song Mingjie Zhan Hongsheng Li ReLM LRM SyDa 93 103 0 05 Oct 2023
Graph of Thoughts: Solving Elaborate Problems with Large Language Models Maciej Besta Nils Blach Aleš Kubíček Robert Gerstenberger Michal Podstawski ... Joanna Gajda Tomasz Lehmann H. Niewiadomski Piotr Nyczyk Torsten Hoefler LRM AI4CE LM&Ro 136 671 0 18 Aug 2023
Transformers as Statisticians: Provable In-Context Learning with In-Context Algorithm Selection Yu Bai Fan Chen Haiquan Wang Caiming Xiong Song Mei 50 193 0 07 Jun 2023
Let's Verify Step by Step Hunter Lightman V. Kosaraju Yura Burda Harrison Edwards Bowen Baker Teddy Lee Jan Leike John Schulman Ilya Sutskever K. Cobbe ALM OffRL LRM 191 1,164 0 31 May 2023
Direct Preference Optimization: Your Language Model is Secretly a Reward Model Rafael Rafailov Archit Sharma E. Mitchell Stefano Ermon Christopher D. Manning Chelsea Finn ALM 385 3,981 0 29 May 2023
Towards Revealing the Mystery behind Chain of Thought: A Theoretical Perspective Guhao Feng Bohang Zhang Yuntian Gu Haotian Ye Di He Liwei Wang LRM 100 248 0 24 May 2023
GPT-4 Technical Report OpenAI OpenAI OpenAI Josh Achiam Steven Adler Sandhini Agarwal Lama Ahmad ... Shengjia Zhao Tianhao Zheng Juntang Zhuang William Zhuk Barret Zoph LLMAG MLLM 1.4K 14,359 0 15 Mar 2023
Tighter Bounds on the Expressivity of Transformer Encoders David Chiang Peter A. Cholak A. Pillay 79 58 0 25 Jan 2023
Transformers Learn Shortcuts to Automata Bingbin Liu Jordan T. Ash Surbhi Goel A. Krishnamurthy Cyril Zhang OffRL LRM 131 175 0 19 Oct 2022
In-context Learning and Induction Heads Catherine Olsson Nelson Elhage Neel Nanda Nicholas Joseph Nova Dassarma ... Tom B. Brown Jack Clark Jared Kaplan Sam McCandlish C. Olah 316 516 0 24 Sep 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 874 12,973 0 04 Mar 2022
A Simple Guard for Learned Optimizers Isabeau Prémont-Schwarz Jaroslav Vítkru Jan Feyereisl 99 8 0 28 Jan 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 817 9,387 0 28 Jan 2022
Learning To Retrieve Prompts for In-Context Learning Ohad Rubin Jonathan Herzig Jonathan Berant VPVLM RALM 84 702 0 16 Dec 2021
An Explanation of In-context Learning as Implicit Bayesian Inference Sang Michael Xie Aditi Raghunathan Percy Liang Tengyu Ma ReLM BDL VPVLM LRM 198 751 0 03 Nov 2021
MetaICL: Learning to Learn In Context Sewon Min M. Lewis Luke Zettlemoyer Hannaneh Hajishirzi LRM 212 489 0 29 Oct 2021
Meta-learning with an Adaptive Task Scheduler Huaxiu Yao Yu Wang Ying Wei P. Zhao M. Mahdavi Defu Lian Chelsea Finn OOD 64 48 0 26 Oct 2021
Evaluating Large Language Models Trained on Code Mark Chen Jerry Tworek Heewoo Jun Qiming Yuan Henrique Pondé ... Bob McGrew Dario Amodei Sam McCandlish Ilya Sutskever Wojciech Zaremba ELM ALM 231 5,539 0 07 Jul 2021
Saturated Transformers are Constant-Depth Threshold Circuits William Merrill Ashish Sabharwal Noah A. Smith 82 105 0 30 Jun 2021
Learning a Universal Template for Few-shot Dataset Generalization Eleni Triantafillou Hugo Larochelle R. Zemel Vincent Dumoulin 80 94 0 14 May 2021
RNNs can generate bounded hierarchical languages with optimal memory John Hewitt Michael Hahn Surya Ganguli Percy Liang Christopher D. Manning LRM 45 54 0 15 Oct 2020
Adaptive Task Sampling for Meta-Learning Chenghao Liu Zhihao Wang Doyen Sahoo Yuan Fang Kun Zhang Guosheng Lin 86 55 0 17 Jul 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 795 42,055 0 28 May 2020
Meta-Learning in Neural Networks: A Survey Timothy M. Hospedales Antreas Antoniou P. Micaelli Amos Storkey OOD 393 1,979 0 11 Apr 2020
Variational Metric Scaling for Metric-Based Meta-Learning Jiaxin Chen Li-Ming Zhan Xiao-Ming Wu K. F. Chung 52 48 0 26 Dec 2019
Are Transformers universal approximators of sequence-to-sequence functions? Chulhee Yun Srinadh Bhojanapalli A. S. Rawat Sashank J. Reddi Sanjiv Kumar 115 355 0 20 Dec 2019
PyTorch: An Imperative Style, High-Performance Deep Learning Library Adam Paszke Sam Gross Francisco Massa Adam Lerer James Bradbury ... Sasank Chilamkurthy Benoit Steiner Lu Fang Junjie Bai Soumith Chintala ODL 511 42,449 0 03 Dec 2019
How Can We Know What Language Models Know? Zhengbao Jiang Frank F. Xu Jun Araki Graham Neubig KELM 132 1,405 0 28 Nov 2019
Meta-Learning with Implicit Gradients Aravind Rajeswaran Chelsea Finn Sham Kakade Sergey Levine 110 855 0 10 Sep 2019