Teaching Language Models to Self-Improve by Learning from Language Feedback

11 June 2024

Jingbo Zhu

Papers citing "Teaching Language Models to Self-Improve by Learning from Language Feedback"

32 / 32 papers shown

Title
Self-Improving Robust Preference Optimization Eugene Choi Arash Ahmadian Matthieu Geist Oilvier Pietquin M. G. Azar 52 9 0 03 Jun 2024
RankPrompt: Step-by-Step Comparisons Make Language Models Better Reasoners Chi Hu Yuan Ge Xiangnan Ma Hang Cao Qiang Li Yonghua Yang Tong Xiao Jingbo Zhu ReLM ELM LRM ALM 54 9 0 19 Mar 2024
Self-Rewarding Language Models Weizhe Yuan Richard Yuanzhe Pang Kyunghyun Cho Xian Li Sainbayar Sukhbaatar Jing Xu Jason Weston ReLM SyDa ALM LRM 287 312 0 18 Jan 2024
Zephyr: Direct Distillation of LM Alignment Lewis Tunstall E. Beeching Nathan Lambert Nazneen Rajani Kashif Rasul ... Nathan Habib Nathan Sarrazin Omar Sanseviero Alexander M. Rush Thomas Wolf ALM 78 382 0 25 Oct 2023
Teaching Language Models to Self-Improve through Interactive Demonstrations Xiao Yu Baolin Peng Michel Galley Jianfeng Gao Zhou Yu LRM ReLM 62 20 0 20 Oct 2023
Can Large Language Models Really Improve by Self-critiquing Their Own Plans? Karthik Valmeekam Matthew Marquez Subbarao Kambhampati LRM 50 84 0 12 Oct 2023
Large Language Models Cannot Self-Correct Reasoning Yet Jie Huang Xinyun Chen Swaroop Mishra Huaixiu Steven Zheng Adams Wei Yu Xinying Song Denny Zhou ReLM LRM 63 445 0 03 Oct 2023
SELF: Self-Evolution with Language Feedback Jianqiao Lu Wanjun Zhong Wenyong Huang Yufei Wang Qi Zhu ... Weichao Wang Xingshan Zeng Lifeng Shang Xin Jiang Qun Liu LRM SyDa 52 7 0 01 Oct 2023
Large Language Model Alignment: A Survey Tianhao Shen Renren Jin Yufei Huang Chuang Liu Weilong Dong Zishan Guo Xinwei Wu Yan Liu Deyi Xiong LM&MA 54 191 0 26 Sep 2023
Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategies Liangming Pan Michael Stephen Saxon Wenda Xu Deepak Nathani Xinyi Wang William Yang Wang KELM LRM 88 206 0 06 Aug 2023
Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback Stephen Casper Xander Davies Claudia Shi T. Gilbert Jérémy Scheurer ... Erdem Biyik Anca Dragan David M. Krueger Dorsa Sadigh Dylan Hadfield-Menell ALM OffRL 98 497 0 27 Jul 2023
FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning Tri Dao LRM 97 1,221 0 17 Jul 2023
Factually Consistent Summarization via Reinforcement Learning with Textual Entailment Feedback Paul Roit Johan Ferret Lior Shani Roee Aharoni Geoffrey Cideron ... Olivier Bachem G. Elidan Avinatan Hassidim Olivier Pietquin Idan Szpektor HILM 54 81 0 31 May 2023
Direct Preference Optimization: Your Language Model is Secretly a Reward Model Rafael Rafailov Archit Sharma E. Mitchell Stefano Ermon Christopher D. Manning Chelsea Finn ALM 300 3,712 0 29 May 2023
Teaching Large Language Models to Self-Debug Xinyun Chen Maxwell Lin Nathanael Scharli Denny Zhou LRM 74 677 0 11 Apr 2023
Self-Refine: Iterative Refinement with Self-Feedback Aman Madaan Niket Tandon Prakhar Gupta Skyler Hallinan Luyu Gao ... Bodhisattwa Prasad Majumder Katherine Hermann Sean Welleck Amir Yazdanbakhsh Peter Clark ReLM LRM DiffM 112 1,577 0 30 Mar 2023
Training Language Models with Language Feedback at Scale Jérémy Scheurer Jon Ander Campos Tomasz Korbak Jun Shern Chan Angelica Chen Kyunghyun Cho Ethan Perez ALM 71 104 0 28 Mar 2023
Improving Code Generation by Training with Natural Language Feedback Angelica Chen Jérémy Scheurer Tomasz Korbak Jon Ander Campos Jun Shern Chan Samuel R. Bowman Kyunghyun Cho Ethan Perez SyDa ALM AI4CE 51 77 0 28 Mar 2023
Constitutional AI: Harmlessness from AI Feedback Yuntao Bai Saurav Kadavath Sandipan Kundu Amanda Askell John Kernion ... Dario Amodei Nicholas Joseph Sam McCandlish Tom B. Brown Jared Kaplan SyDa MoMe 156 1,583 0 15 Dec 2022
Generating Sequences by Learning to Self-Correct Sean Welleck Ximing Lu Peter West Faeze Brahman T. Shen Daniel Khashabi Yejin Choi LRM 64 225 0 31 Oct 2022
Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them Mirac Suzgun Nathan Scales Nathanael Scharli Sebastian Gehrmann Yi Tay ... Aakanksha Chowdhery Quoc V. Le Ed H. Chi Denny Zhou Jason W. Wei ALM ELM LRM ReLM 167 1,068 0 17 Oct 2022
Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback Yuntao Bai Andy Jones Kamal Ndousse Amanda Askell Anna Chen ... Jack Clark Sam McCandlish C. Olah Benjamin Mann Jared Kaplan 226 2,457 0 12 Apr 2022
PaLM: Scaling Language Modeling with Pathways Aakanksha Chowdhery Sharan Narang Jacob Devlin Maarten Bosma Gaurav Mishra ... Kathy Meier-Hellstern Douglas Eck J. Dean Slav Petrov Noah Fiedel PILM LRM 355 6,132 0 05 Apr 2022
Red Teaming Language Models with Language Models Ethan Perez Saffron Huang Francis Song Trevor Cai Roman Ring John Aslanides Amelia Glaese Nat McAleese G. Irving AAML 73 627 0 07 Feb 2022
Ethical and social risks of harm from Language Models Laura Weidinger John F. J. Mellor Maribeth Rauh Conor Griffin J. Uesato ... Lisa Anne Hendricks William S. Isaac Sean Legassick G. Irving Iason Gabriel PILM 73 1,009 0 08 Dec 2021
Training Verifiers to Solve Math Word Problems K. Cobbe V. Kosaraju Mohammad Bavarian Mark Chen Heewoo Jun ... Jerry Tworek Jacob Hilton Reiichiro Nakano Christopher Hesse John Schulman ReLM OffRL LRM 214 4,175 0 27 Oct 2021
Learning Multimodal Rewards from Rankings Vivek Myers Erdem Biyik Nima Anari Dorsa Sadigh OffRL 49 50 0 27 Sep 2021
Learning to summarize from human feedback Nisan Stiennon Long Ouyang Jeff Wu Daniel M. Ziegler Ryan J. Lowe Chelsea Voss Alec Radford Dario Amodei Paul Christiano ALM 191 2,071 0 02 Sep 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 546 41,106 0 28 May 2020
Graph-based, Self-Supervised Program Repair from Diagnostic Feedback Michihiro Yasunaga Percy Liang LRM 102 173 0 20 May 2020
Way Off-Policy Batch Deep Reinforcement Learning of Implicit Human Preferences in Dialog Natasha Jaques Asma Ghandeharioun J. Shen Craig Ferguson Àgata Lapedriza Noah J. Jones S. Gu Rosalind W. Picard OffRL 88 338 0 30 Jun 2019
Deep reinforcement learning from human preferences Paul Christiano Jan Leike Tom B. Brown Miljan Martic Shane Legg Dario Amodei 105 3,243 0 12 Jun 2017