AutoPRM: Automating Procedural Supervision for Multi-Step Reasoning via Controllable Question Decomposition

18 February 2024

Zhaorun Chen

Zhuokai Zhao

Zhihong Zhu

Ruiqi Zhang

Xiang Li

Bhiksha Raj

Huaxiu Yao

LRM

ArXiv (abs)PDF HTML

Papers citing "AutoPRM: Automating Procedural Supervision for Multi-Step Reasoning via Controllable Question Decomposition"

23 / 23 papers shown

Title
QLASS: Boosting Language Agent Inference via Q-Guided Stepwise Search Zongyu Lin Yao Tang Xingcheng Yao Da Yin Ziniu Hu Ningyu Zhang Kai-Wei Chang LRM 127 6 0 04 Feb 2025
MJ-VIDEO: Fine-Grained Benchmarking and Rewarding Video Preferences in Video Generation Haibo Tong Zhaoyang Wang Zhe Chen Haonian Ji Shi Qiu ... Peng Xia Mingyu Ding Rafael Rafailov Chelsea Finn Huaxiu Yao EGVM VGen 178 4 0 03 Feb 2025
Beyond Reward Hacking: Causal Rewards for Large Language Model Alignment Chaoqi Wang Zhuokai Zhao Yibo Jiang Zhaorun Chen Chen Zhu ... Jiayi Liu Lizhu Zhang Xiangjun Fan Hao Ma Sinong Wang 132 5 0 16 Jan 2025
Taming Overconfidence in LLMs: Reward Calibration in RLHF Jixuan Leng Chengsong Huang Banghua Zhu Jiaxin Huang 98 16 0 13 Oct 2024
RankCLIP: Ranking-Consistent Language-Image Pretraining Yiming Zhang Zhuokai Zhao Zhaorun Chen Zhili Feng Zenghui Ding Yining Sun SSL VLM 101 7 0 15 Apr 2024
Fine-tuning Language Models for Factuality Katherine Tian Eric Mitchell Huaxiu Yao Christopher D. Manning Chelsea Finn KELM HILM SyDa 75 184 0 14 Nov 2023
MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models L. Yu Weisen Jiang Han Shi Jincheng Yu Zhengying Liu Yu Zhang James T. Kwok Zheng Li Adrian Weller Weiyang Liu OSLM LRM 100 394 0 21 Sep 2023
Graph of Thoughts: Solving Elaborate Problems with Large Language Models Maciej Besta Nils Blach Aleš Kubíček Robert Gerstenberger Michal Podstawski ... Joanna Gajda Tomasz Lehmann H. Niewiadomski Piotr Nyczyk Torsten Hoefler LRM AI4CE LM&Ro 150 701 0 18 Aug 2023
WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instruct Haipeng Luo Qingfeng Sun Can Xu Pu Zhao Jian-Guang Lou ... Xiubo Geng Qingwei Lin Shifeng Chen Yansong Tang Dongmei Zhang LRM OSLM 233 466 0 18 Aug 2023
SelfCheck: Using LLMs to Zero-Shot Check Their Own Step-by-Step Reasoning Ning Miao Yee Whye Teh Tom Rainforth ReLM LRM 55 135 0 01 Aug 2023
Fine-Grained Human Feedback Gives Better Rewards for Language Model Training Zeqiu Wu Yushi Hu Weijia Shi Nouha Dziri Alane Suhr Prithviraj Ammanabrolu Noah A. Smith Mari Ostendorf Hannaneh Hajishirzi ALM 147 334 0 02 Jun 2023
A Causal Framework to Quantify the Robustness of Mathematical Reasoning with Language Models Alessandro Stolfo Zhijing Jin Kumar Shridhar Bernhard Schölkopf Mrinmaya Sachan ELM OOD LRM 84 66 0 21 Oct 2022
Automatic Chain of Thought Prompting in Large Language Models Zhuosheng Zhang Aston Zhang Mu Li Alexander J. Smola ReLM LRM 150 632 0 07 Oct 2022
Is Reinforcement Learning (Not) for Natural Language Processing: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimization Rajkumar Ramamurthy Prithviraj Ammanabrolu Kianté Brantley Jack Hessel R. Sifa Christian Bauckhage Hannaneh Hajishirzi Yejin Choi OffRL 101 248 0 03 Oct 2022
Complexity-Based Prompting for Multi-Step Reasoning Yao Fu Hao-Chun Peng Ashish Sabharwal Peter Clark Tushar Khot ReLM LRM 232 444 0 03 Oct 2022
Solving Quantitative Reasoning Problems with Language Models Aitor Lewkowycz Anders Andreassen David Dohan Ethan Dyer Henryk Michalewski ... Theo Gutman-Solo Yuhuai Wu Behnam Neyshabur Guy Gur-Ari Vedant Misra ReLM ELM LRM 181 857 0 29 Jun 2022
Training Compute-Optimal Large Language Models Jordan Hoffmann Sebastian Borgeaud A. Mensch Elena Buchatskaya Trevor Cai ... Karen Simonyan Erich Elsen Jack W. Rae Oriol Vinyals Laurent Sifre AI4TS 208 1,976 0 29 Mar 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 823 9,644 0 28 Jan 2022
Training Verifiers to Solve Math Word Problems K. Cobbe V. Kosaraju Mohammad Bavarian Mark Chen Heewoo Jun ... Jerry Tworek Jacob Hilton Reiichiro Nakano Christopher Hesse John Schulman ReLM OffRL LRM 326 4,569 0 27 Oct 2021
Evaluating Large Language Models Trained on Code Mark Chen Jerry Tworek Heewoo Jun Qiming Yuan Henrique Pondé ... Bob McGrew Dario Amodei Sam McCandlish Ilya Sutskever Wojciech Zaremba ELM ALM 236 5,647 0 07 Jul 2021
Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies Mor Geva Daniel Khashabi Elad Segal Tushar Khot Dan Roth Jonathan Berant RALM 345 735 0 06 Jan 2021
Language Models as Knowledge Bases? Fabio Petroni Tim Rocktaschel Patrick Lewis A. Bakhtin Yuxiang Wu Alexander H. Miller Sebastian Riedel KELM AI4MH 576 2,674 0 03 Sep 2019
Selective Classification for Deep Neural Networks Yonatan Geifman Ran El-Yaniv CVBM 95 529 0 23 May 2017