Step-level Value Preference Optimization for Mathematical Reasoning

16 June 2024

Papers citing "Step-level Value Preference Optimization for Mathematical Reasoning"

32 / 32 papers shown

Title
Iterative Tool Usage Exploration for Multimodal Agents via Step-wise Preference Tuning Pengxiang Li Zhi Gao Bofei Zhang Yapeng Mi Xiaojian Ma ... Tao Yuan Yuwei Wu Yunde Jia Song-Chun Zhu Qing Li LLMAG 70 0 0 30 Apr 2025
Efficient Pretraining Length Scaling Bohong Wu Shen Yan Sijun Zhang Jianqiao Lu Yutao Zeng Ya Wang Xun Zhou 133 0 0 21 Apr 2025
RealSafe-R1: Safety-Aligned DeepSeek-R1 without Compromising Reasoning Capability Y. Zhang Zihao Zeng Dongbai Li Yao Huang Zhijie Deng Yinpeng Dong LRM 32 4 0 14 Apr 2025
R1-VL: Learning to Reason with Multimodal Large Language Models via Step-wise Group Relative Policy Optimization Jingyi Zhang Jiaxing Huang H. Yao Shunyu Liu Xikun Zhang Shijian Lu Dacheng Tao LRM 57 18 0 17 Mar 2025
Thinking Machines: A Survey of LLM based Reasoning Strategies Dibyanayan Bandyopadhyay Soham Bhattacharjee Asif Ekbal LRM ELM 48 4 0 13 Mar 2025
Local Look-Ahead Guidance via Verifier-in-the-Loop for Automated Theorem Proving Sara Rajaee Kumar Pratik Gabriele Cesa Arash Behboodi OffRL LRM 61 0 0 12 Mar 2025
Process-based Self-Rewarding Language Models Shimao Zhang Xiao Liu Xin Zhang Junxiao Liu Zheheng Luo Shujian Huang Yeyun Gong ReLM SyDa LRM 95 2 0 05 Mar 2025
An Efficient and Precise Training Data Construction Framework for Process-supervised Reward Model in Mathematical Reasoning Wei Sun Qianlong Du Fuwei Cui Jiajun Zhang OffRL LRM 40 0 0 04 Mar 2025
What's Behind PPO's Collapse in Long-CoT? Value Optimization Holds the Secret Yufeng Yuan Yu Yue Ruofei Zhu Tiantian Fan Lin Yan OffRL 62 10 0 03 Mar 2025
Two Heads Are Better Than One: Dual-Model Verbal Reflection at Inference-Time Jiazheng Li Yuxiang Zhou Junru Lu Gladys Tyen Lin Gui Cesare Aloisi Yulan He LRM 39 2 0 26 Feb 2025
Self-rewarding correction for mathematical reasoning Wei Xiong Hanning Zhang Chenlu Ye Lichang Chen Nan Jiang Tong Zhang ReLM KELM LRM 75 9 0 26 Feb 2025
Table-Critic: A Multi-Agent Framework for Collaborative Criticism and Refinement in Table Reasoning Peiying Yu Guoxin Chen Jingjing Wang LLMAG LMTD LRM 70 2 0 17 Feb 2025
Leveraging Constrained Monte Carlo Tree Search to Generate Reliable Long Chain-of-Thought for Mathematical Reasoning Qingwen Lin Boyan Xu Zijian Li Zhifeng Hao Keli Zhang Ruichu Cai LRM 52 2 0 16 Feb 2025
PIPA: Preference Alignment as Prior-Informed Statistical Estimation Junbo Li Zhangyang Wang Qiang Liu OffRL 102 0 0 09 Feb 2025
Iterative Deepening Sampling for Large Language Models Weizhe Chen Sven Koenig B. Dilkina LRM ReLM 88 1 0 08 Feb 2025
STAIR: Improving Safety Alignment with Introspective Reasoning Y. Zhang Siyuan Zhang Yao Huang Zeyu Xia Zhengwei Fang Xiao Yang Ranjie Duan Dong Yan Yinpeng Dong Jun Zhu LRM LLMSV 56 3 0 04 Feb 2025
Domaino1s: Guiding LLM Reasoning for Explainable Answers in High-Stakes Domains Xu Chu Zhijie Tan Hanlin Xue Guanyu Wang Tong Mo Weiping Li ELM LRM 55 1 0 24 Jan 2025
BoostStep: Boosting mathematical capability of Large Language Models via improved single-step reasoning Beichen Zhang Yuhong Liu Xiaoyi Dong Yuhang Zang Pan Zhang Haodong Duan Yuhang Cao D. Lin J. T. Wang LRM ReLM 61 2 0 06 Jan 2025
Markov Chain of Thought for Efficient Mathematical Reasoning Wen Yang Kai Fan Minpeng Liao LRM 39 4 0 23 Oct 2024
Process Reward Model with Q-Value Rankings W. Li Yixuan Li LRM 59 15 0 15 Oct 2024
Enhancing Multi-Step Reasoning Abilities of Language Models through Direct Q-Function Optimization Guanlin Liu Kaixuan Ji Ning Dai Zheng Wu Chen Dun Q. Gu Lin Yan Quanquan Gu Lin Yan OffRL LRM 48 9 0 11 Oct 2024
Towards Self-Improvement of LLMs via MCTS: Leveraging Stepwise Knowledge with Curriculum Preference Learning Xiyao Wang Linfeng Song Ye Tian Dian Yu Baolin Peng Haitao Mi Furong Huang Dong Yu LRM 52 9 0 09 Oct 2024
Learning Evolving Tools for Large Language Models Guoxin Chen Zhong Zhang Xin Cong Fangda Guo Yesai Wu Yankai Lin Wenzheng Feng Yasheng Wang KELM 52 1 0 09 Oct 2024
Subtle Errors Matter: Preference Learning via Error-injected Self-editing Kaishuai Xu Tiezheng YU Wenjun Hou Yi Cheng Chak Tou Leong Liangyou Li Xin Jiang Lifeng Shang Qun Liu Wenjie Li LRM 151 0 0 09 Oct 2024
LLaMA-Berry: Pairwise Optimization for O1-like Olympiad-Level Mathematical Reasoning Di Zhang Jianbo Wu Jingdi Lei Tong Che Jiatong Li ... Shufei Zhang Marco Pavone Yuqiang Li Wanli Ouyang Dongzhan Zhou LRM 33 43 0 03 Oct 2024
BEATS: Optimizing LLM Mathematical Capabilities with BackVerify and Adaptive Disambiguate based Efficient Tree Search Linzhuang Sun Hao Liang Jingxuan Wei Bihui Yu Conghui He Zenan Zhou Wentao Zhang 21 4 0 26 Sep 2024
Selective Preference Optimization via Token-Level Reward Function Estimation Kailai Yang Zhiwei Liu Qianqian Xie Jimin Huang Erxue Min Sophia Ananiadou 33 10 0 24 Aug 2024
MARIO Eval: Evaluate Your Math LLM with your Math LLM--A mathematical dataset evaluation toolkit Boning Zhang Chengxi Li Kai Fan ELM 43 10 0 22 Apr 2024
MathGenie: Generating Synthetic Data with Question Back-translation for Enhancing Mathematical Reasoning of LLMs Zimu Lu Aojun Zhou Houxing Ren Ke Wang Weikang Shi Junting Pan Mingjie Zhan Hongsheng Li SyDa LRM 50 43 0 26 Feb 2024
Noise Contrastive Alignment of Language Models with Explicit Rewards Huayu Chen Guande He Lifan Yuan Ganqu Cui Hang Su Jun Zhu 57 38 0 08 Feb 2024
KTO: Model Alignment as Prospect Theoretic Optimization Kawin Ethayarajh Winnie Xu Niklas Muennighoff Dan Jurafsky Douwe Kiela 173 449 0 02 Feb 2024
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 313 11,953 0 04 Mar 2022