Title
Fine-Grained Human Feedback Gives Better Rewards for Language Model Training Zeqiu Wu Yushi Hu Weijia Shi Nouha Dziri Alane Suhr Prithviraj Ammanabrolu Noah A. Smith Mari Ostendorf Hannaneh Hajishirzi ALM 136 329 0 02 Jun 2023
Instructions as Backdoors: Backdoor Vulnerabilities of Instruction Tuning for Large Language Models Lyne Tchapmi Mingyu Derek Ma Fei Wang Chaowei Xiao Muhao Chen SILM 108 82 0 24 May 2023
Shattering the Agent-Environment Interface for Fine-Tuning Inclusive Language Models Wanqiao Xu Shi Dong Dilip Arumugam Benjamin Van Roy 60 8 0 19 May 2023
RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignment Hanze Dong Wei Xiong Deepanshu Goyal Yihan Zhang Winnie Chow Boyao Wang Shizhe Diao Jipeng Zhang Kashun Shum Tong Zhang ALM 71 455 0 13 Apr 2023
RRHF: Rank Responses to Align Language Models with Human Feedback without tears Zheng Yuan Hongyi Yuan Chuanqi Tan Wei Wang Songfang Huang Feiran Huang ALM 159 374 0 11 Apr 2023
Whose Opinions Do Language Models Reflect? Shibani Santurkar Esin Durmus Faisal Ladhak Cinoo Lee Percy Liang Tatsunori Hashimoto 76 432 0 30 Mar 2023
Training Language Models with Language Feedback at Scale Jérémy Scheurer Jon Ander Campos Tomasz Korbak Jun Shern Chan Angelica Chen Kyunghyun Cho Ethan Perez ALM 84 106 0 28 Mar 2023
What does it take to catch a Chinchilla? Verifying Rules on Large-Scale Neural Network Training via Compute Monitoring Yonadav Shavit 49 23 0 20 Mar 2023
Susceptibility to Influence of Large Language Models Lewis D. Griffin Bennett Kleinberg Maximilian Mozes Kimberly T. Mai Maria Vau M. Caldwell Augustine N. Mavor-Parker 71 15 0 10 Mar 2023
Personalisation within bounds: A risk taxonomy and policy framework for the alignment of large language models with personalised feedback Hannah Rose Kirk Bertie Vidgen Paul Röttger Scott A. Hale 72 106 0 09 Mar 2023
Ground(less) Truth: A Causal Framework for Proxy Labels in Human-Algorithm Decision-Making Luke M. Guerdan Amanda Coston Zhiwei Steven Wu Kenneth Holstein CML 51 29 0 13 Feb 2023
Settling the Reward Hypothesis Michael Bowling John D. Martin David Abel Will Dabney LRM 71 33 0 20 Dec 2022
Constitutional AI: Harmlessness from AI Feedback Yuntao Bai Saurav Kadavath Sandipan Kundu Amanda Askell John Kernion ... Dario Amodei Nicholas Joseph Sam McCandlish Tom B. Brown Jared Kaplan SyDa MoMe 184 1,618 0 15 Dec 2022
On the Sensitivity of Reward Inference to Misspecified Human Models Joey Hong Kush S. Bhatia Anca Dragan 50 26 0 09 Dec 2022
Few-Shot Preference Learning for Human-in-the-Loop RL Joey Hejna Dorsa Sadigh OffRL 95 99 0 06 Dec 2022
Misspecification in Inverse Reinforcement Learning Joar Skalse Alessandro Abate 53 23 0 06 Dec 2022
Fine-tuning language models to find agreement among humans with diverse preferences Michiel A. Bakker Martin Chadwick Hannah R. Sheahan Michael Henry Tessler Lucy Campbell-Gillingham ... Nat McAleese Amelia Glaese John Aslanides M. Botvinick Christopher Summerfield ALM 100 231 0 28 Nov 2022
Measuring Progress on Scalable Oversight for Large Language Models Sam Bowman Jeeyoon Hyun Ethan Perez Edwin Chen Craig Pettit ... Tristan Hume Yuntao Bai Zac Hatfield-Dodds Benjamin Mann Jared Kaplan ALM ELM 69 129 0 04 Nov 2022
Scaling Instruction-Finetuned Language Models Hyung Won Chung Le Hou Shayne Longpre Barret Zoph Yi Tay ... Jacob Devlin Adam Roberts Denny Zhou Quoc V. Le Jason W. Wei ReLM LRM 183 3,117 0 20 Oct 2022
Mass-Editing Memory in a Transformer Kevin Meng Arnab Sen Sharma A. Andonian Yonatan Belinkov David Bau KELM VLM 128 585 0 13 Oct 2022
Goal Misgeneralization: Why Correct Specifications Aren't Enough For Correct Goals Rohin Shah Vikrant Varma Ramana Kumar Mary Phuong Victoria Krakovna J. Uesato Zachary Kenton 73 72 0 04 Oct 2022
Is Reinforcement Learning (Not) for Natural Language Processing: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimization Rajkumar Ramamurthy Prithviraj Ammanabrolu Kianté Brantley Jack Hessel R. Sifa Christian Bauckhage Hannaneh Hajishirzi Yejin Choi OffRL 91 247 0 03 Oct 2022
Defining and Characterizing Reward Hacking Joar Skalse Nikolaus H. R. Howe Dmitrii Krasheninnikov David M. Krueger 86 61 0 27 Sep 2022
The Alignment Problem from a Deep Learning Perspective Richard Ngo Lawrence Chan Sören Mindermann 100 192 0 30 Aug 2022
Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networks Tilman Raukur A. Ho Stephen Casper Dylan Hadfield-Menell AAML AI4CE 93 132 0 27 Jul 2022
Self-critiquing models for assisting human evaluators William Saunders Catherine Yeh Jeff Wu Steven Bills Ouyang Long Jonathan Ward Jan Leike ALM ELM 92 302 0 12 Jun 2022
Adversarial Training for High-Stakes Reliability Daniel M. Ziegler Seraphina Nix Lawrence Chan Tim Bauman Peter Schmidt-Nielsen ... Noa Nabeshima Benjamin Weinstein-Raun D. Haas Buck Shlegeris Nate Thomas AAML 97 61 0 03 May 2022
Training Language Models with Language Feedback Jérémy Scheurer Jon Ander Campos Jun Shern Chan Angelica Chen Kyunghyun Cho Ethan Perez ALM 102 51 0 29 Apr 2022
Estimating and Penalizing Induced Preference Shifts in Recommender Systems Micah Carroll Anca Dragan Stuart J. Russell Dylan Hadfield-Menell OffRL 88 44 0 25 Apr 2022
Correcting Robot Plans with Natural Language Feedback Pratyusha Sharma Balakumar Sundaralingam Valts Blukis Chris Paxton Tucker Hermans Antonio Torralba Jacob Andreas Dieter Fox 3DV LM&Ro 59 93 0 11 Apr 2022
Inferring Rewards from Language in Context Jessy Lin Daniel Fried Dan Klein Anca Dragan LM&Ro 76 55 0 05 Apr 2022
Locating and Editing Factual Associations in GPT Kevin Meng David Bau A. Andonian Yonatan Belinkov KELM 248 1,350 0 10 Feb 2022
Red Teaming Language Models with Language Models Ethan Perez Saffron Huang Francis Song Trevor Cai Roman Ring John Aslanides Amelia Glaese Nat McAleese G. Irving AAML 155 655 0 07 Feb 2022
The Effects of Reward Misspecification: Mapping and Mitigating Misaligned Models Alexander Pan Kush S. Bhatia Jacob Steinhardt 86 179 0 10 Jan 2022
Ethical and social risks of harm from Language Models Laura Weidinger John F. J. Mellor Maribeth Rauh Conor Griffin J. Uesato ... Lisa Anne Hendricks William S. Isaac Sean Legassick G. Irving Iason Gabriel PILM 108 1,036 0 08 Dec 2021
Scalar reward is not enough: A response to Silver, Singh, Precup and Sutton (2021) Peter Vamplew Benjamin J. Smith Johan Källström G. Ramos Roxana Rădulescu ... Fredrik Heintz Patrick Mannion Pieter J. K. Libin Richard Dazeley Cameron Foale LRM 49 67 0 25 Nov 2021
Confidence-Aware Imitation Learning from Demonstrations with Varying Optimality Songyuan Zhang Zhangjie Cao Dorsa Sadigh Yanan Sui 44 54 0 27 Oct 2021
Multitask Prompted Training Enables Zero-Shot Task Generalization Victor Sanh Albert Webson Colin Raffel Stephen H. Bach Lintang Sutawika ... T. Bers Stella Biderman Leo Gao Thomas Wolf Alexander M. Rush LRM 342 1,702 0 15 Oct 2021
On Releasing Annotator-Level Labels and Information in Datasets Vinodkumar Prabhakaran Aida Mostafazadeh Davani Mark Díaz 87 149 0 12 Oct 2021
Learning Reward Functions from Scale Feedback Nils Wilde Erdem Biyik Dorsa Sadigh Stephen L. Smith 81 33 0 01 Oct 2021
Unsolved Problems in ML Safety Dan Hendrycks Nicholas Carlini John Schulman Jacob Steinhardt 242 289 0 28 Sep 2021
Learning Multimodal Rewards from Rankings Vivek Myers Erdem Biyik Nima Anari Dorsa Sadigh OffRL 58 50 0 27 Sep 2021
Recursively Summarizing Books with Human Feedback Jeff Wu Long Ouyang Daniel M. Ziegler Nissan Stiennon Ryan J. Lowe Jan Leike Paul Christiano ALM 154 303 0 22 Sep 2021
A Survey of Exploration Methods in Reinforcement Learning Susan Amin Maziar Gomrokchi Harsh Satija H. V. Hoof Doina Precup OffRL 68 84 0 01 Sep 2021
Process for Adapting Language Models to Society (PALMS) with Values-Targeted Datasets Irene Solaiman Christy Dennison 90 225 0 18 Jun 2021
Hard Choices in Artificial Intelligence Roel Dobbe T. Gilbert Yonatan Dov Mintz 50 56 0 10 Jun 2021
PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-training Kimin Lee Laura M. Smith Pieter Abbeel OffRL 63 284 0 09 Jun 2021
Survey on reinforcement learning for language processing Víctor Uc Cetina Nicolás Navarro-Guerrero A. Martín-González C. Weber S. Wermter OffRL 62 106 0 12 Apr 2021
Planning for Safe Abortable Overtaking Maneuvers in Autonomous Driving Jiyo Palatti Andrei Aksjonov G. Alcan Ville Kyrki 53 22 0 31 Mar 2021
Consequences of Misaligned AI Simon Zhuang Dylan Hadfield-Menell 62 75 0 07 Feb 2021