Distributional Preference Learning: Understanding and Accounting for Hidden Context in RLHF

13 December 2023

Papers citing "Distributional Preference Learning: Understanding and Accounting for Hidden Context in RLHF"

43 / 43 papers shown

Title
Learning Guarantee of Reward Modeling Using Deep Neural Networks Yuanhang Luo Yeheng Ge Ruijian Han Guohao Shen 34 0 0 10 May 2025
The Mind in the Machine: A Survey of Incorporating Psychological Theories in LLMs Zizhou Liu Ziwei Gong Lin Ai Zheng Hui Run Chen Colin Wayne Leach Michelle R. Greene Julia Hirschberg LLMAG 165 0 0 28 Mar 2025
Capturing Individual Human Preferences with Reward Features André Barreto Vincent Dumoulin Yiran Mao Nicolas Perez-Nieves Bobak Shahriari Yann Dauphin Doina Precup Hugo Larochelle ALM 67 1 0 21 Mar 2025
A Survey on Personalized Alignment -- The Missing Piece for Large Language Models in Real-World Applications Jian Guan Jian Wu J. Li Chuanqi Cheng Wei Wu LM&MA 74 0 0 21 Mar 2025
From 1,000,000 Users to Every User: Scaling Up Personalized Preference for User-level Alignment J. Li Jian Guan Songhao Wu Wei Wu Rui Yan 67 1 0 19 Mar 2025
Strategyproof Reinforcement Learning from Human Feedback Thomas Kleine Buening Jiarui Gan Debmalya Mandal Marta Z. Kwiatkowska 52 0 0 13 Mar 2025
Improving LLM-as-a-Judge Inference with the Judgment Distribution Victor Wang Michael J.Q. Zhang Eunsol Choi 58 1 0 04 Mar 2025
Building Safe GenAI Applications: An End-to-End Overview of Red Teaming for Large Language Models Alberto Purpura Sahil Wadhwa Jesse Zymet Akshay Gupta Andy Luo Melissa Kazemi Rad Swapnil Shinde Mohammad Sorower AAML 203 0 0 03 Mar 2025
CoPL: Collaborative Preference Learning for Personalizing LLMs Youngbin Choi Seunghyuk Cho M. Lee Moonjeong Park Yesong Ko Jungseul Ok Dongwoo Kim 68 0 0 03 Mar 2025
FSPO: Few-Shot Preference Optimization of Synthetic Preference Data in LLMs Elicits Effective Personalization to Real Users Anikait Singh Sheryl Hsu Kyle Hsu E. Mitchell Stefano Ermon Tatsunori Hashimoto Archit Sharma Chelsea Finn SyDa OffRL 59 1 0 26 Feb 2025
Language Model Fine-Tuning on Scaled Survey Data for Predicting Distributions of Public Opinions Joseph Suh Erfan Jahanparast Suhong Moon Minwoo Kang Serina Chang ALM LM&MA 57 1 0 24 Feb 2025
Bone Soups: A Seek-and-Soup Model Merging Approach for Controllable Multi-Objective Generation Guofu Xie Xiao Zhang Ting Yao Yunsheng Shi MoMe 63 1 0 15 Feb 2025
CTR-Driven Advertising Image Generation with Multimodal Large Language Models Xingye Chen Wei Feng Zhenbang Du Weizhen Wang Yuxiao Chen ... Jingping Shao Yuanjie Shao Xinge You Changxin Gao Nong Sang OffRL 47 2 0 05 Feb 2025
Clone-Robust AI Alignment Ariel D. Procaccia Benjamin G. Schiffer Shirley Zhang 35 1 0 17 Jan 2025
Geometric-Averaged Preference Optimization for Soft Preference Labels Hiroki Furuta Kuang-Huei Lee Shixiang Shane Gu Y. Matsuo Aleksandra Faust Heiga Zen Izzeddin Gur 58 7 0 31 Dec 2024
Test-Time Alignment via Hypothesis Reweighting Yoonho Lee Jonathan Williams Henrik Marklund Archit Sharma E. Mitchell Anikait Singh Chelsea Finn 91 3 0 11 Dec 2024
Contextualized Evaluations: Taking the Guesswork Out of Language Model Evaluations Chaitanya Malaviya Joseph Chee Chang Dan Roth Mohit Iyyer Mark Yatskar Kyle Lo ELM 45 4 0 11 Nov 2024
BlueSuffix: Reinforced Blue Teaming for Vision-Language Models Against Jailbreak Attacks Yunhan Zhao Xiang Zheng Lin Luo Yige Li Xingjun Ma Yu-Gang Jiang VLM AAML 62 3 0 28 Oct 2024
Rethinking the Uncertainty: A Critical Review and Analysis in the Era of Large Language Models Mohammad Beigi Sijia Wang Ying Shen Zihao Lin Adithya Kulkarni ... Ming Jin Jin-Hee Cho Dawei Zhou Chang-Tien Lu Lifu Huang 29 1 0 26 Oct 2024
SafetyAnalyst: Interpretable, transparent, and steerable safety moderation for AI behavior Jing-Jing Li Valentina Pyatkin Max Kleiman-Weiner Liwei Jiang Nouha Dziri Anne Collins Jana Schaich Borg Maarten Sap Yejin Choi Sydney Levine 29 1 0 22 Oct 2024
Diverging Preferences: When do Annotators Disagree and do Models Know? Michael J.Q. Zhang Zhilin Wang Jena D. Hwang Yi Dong Olivier Delalleau Yejin Choi Eunsol Choi Xiang Ren Valentina Pyatkin 32 7 0 18 Oct 2024
Quantile Regression for Distributional Reward Models in RLHF Nicolai Dorka 37 17 0 16 Sep 2024
Beyond Preferences in AI Alignment Tan Zhi-Xuan Micah Carroll Matija Franklin Hal Ashton 41 16 0 30 Aug 2024
Personalizing Reinforcement Learning from Human Feedback with Variational Preference Learning S. Poddar Yanming Wan Hamish Ivison Abhishek Gupta Natasha Jaques 37 35 0 19 Aug 2024
Improving Context-Aware Preference Modeling for Language Models Silviu Pitis Ziang Xiao Nicolas Le Roux Alessandro Sordoni 38 8 0 20 Jul 2024
Jailbreak Attacks and Defenses Against Large Language Models: A Survey Sibo Yi Yule Liu Zhen Sun Tianshuo Cong Xinlei He Jiaxing Song Ke Xu Qi Li AAML 39 82 0 05 Jul 2024
GIEBench: Towards Holistic Evaluation of Group Identity-based Empathy for Large Language Models Leyan Wang Yonggang Jin Tianhao Shen Tianyu Zheng Xinrun Du ... Jiaheng Liu Shi Wang Ge Zhang Liuyu Xiang Zhaofeng He VLM AI4MH 40 0 0 21 Jun 2024
Pareto-Optimal Learning from Preferences with Hidden Context Ryan Boldi Li Ding Lee Spector S. Niekum 70 6 0 21 Jun 2024
Humor in AI: Massive Scale Crowd-Sourced Preferences and Benchmarks for Cartoon Captioning Jifan Zhang Lalit P. Jain Yang Guo Jiayi Chen Kuan Lok Zhou ... Scott Sievert Timothy T. Rogers Kevin Jamieson Robert Mankoff Robert Nowak 39 5 0 15 Jun 2024
Multi-objective Reinforcement learning from AI Feedback Marcus Williams 46 1 0 11 Jun 2024
Aligning to Thousands of Preferences via System Message Generalization Seongyun Lee Sue Hyun Park Seungone Kim Minjoon Seo ALM 44 38 0 28 May 2024
Direct Preference Optimization With Unobserved Preference Heterogeneity Keertana Chidambaram Karthik Vinay Seetharaman Vasilis Syrgkanis 39 7 0 23 May 2024
Axioms for AI Alignment from Human Feedback Luise Ge Daniel Halpern Evi Micha Ariel D. Procaccia Itai Shapira Yevgeniy Vorobeychik Junlin Wu 49 15 0 23 May 2024
Hummer: Towards Limited Competitive Preference Dataset Li Jiang Yusen Wu Junwu Xiong Jingqing Ruan Yichuan Ding Qingpei Guo Zujie Wen Jun Zhou Xiaotie Deng 34 6 0 19 May 2024
Mapping Social Choice Theory to RLHF Jessica Dai Eve Fleisig 35 11 0 19 Apr 2024
Social Choice Should Guide AI Alignment in Dealing with Diverse Human Feedback Vincent Conitzer Rachel Freedman J. Heitzig Wesley H. Holliday Bob M. Jacobs ... Eric Pacuit Stuart Russell Hailey Schoelkopf Emanuel Tewolde W. Zwicker 43 28 0 16 Apr 2024
Post-Hoc Reversal: Are We Selecting Models Prematurely? Rishabh Ranjan Saurabh Garg Mrigank Raman Carlos Guestrin Zachary Chase Lipton 42 0 0 11 Apr 2024
Scalable Interactive Machine Learning for Future Command and Control Anna Madison Ellen R. Novoseller Vinicius G. Goecks Benjamin T. Files Nicholas R. Waytowich Alfred Yu Vernon J. Lawhern Steven Thurman Christopher Kelshaw Kaleb McDowell 35 4 0 09 Feb 2024
A Roadmap to Pluralistic Alignment Taylor Sorensen Jared Moore Jillian R. Fisher Mitchell L. Gordon Niloofar Mireshghallah ... Liwei Jiang Ximing Lu Nouha Dziri Tim Althoff Yejin Choi 65 80 0 07 Feb 2024
Crowd-PrefRL: Preference-Based Reward Learning from Crowds David Chhan Ellen R. Novoseller Vernon J. Lawhern 29 5 0 17 Jan 2024
A Minimaximalist Approach to Reinforcement Learning from Human Feedback Gokul Swamy Christoph Dann Rahul Kidambi Zhiwei Steven Wu Alekh Agarwal OffRL 41 96 0 08 Jan 2024
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 339 12,003 0 04 Mar 2022
Fine-Tuning Language Models from Human Preferences Daniel M. Ziegler Nisan Stiennon Jeff Wu Tom B. Brown Alec Radford Dario Amodei Paul Christiano G. Irving ALM 298 1,610 0 18 Sep 2019