Fine-Tuning Language Models from Human Preferences

18 September 2019

Papers citing "Fine-Tuning Language Models from Human Preferences"

50 / 351 papers shown

Title
AI Alignment at Your Discretion Maarten Buyl Hadi Khalaf C. M. Verdun Lucas Monteiro Paes Caio Vieira Machado Flavio du Pin Calmon 45 0 0 10 Feb 2025
Large Language Models for Multi-Robot Systems: A Survey Peihan Li Zijian An Shams Abrar Lifeng Zhou LM&Ro LRM 62 6 0 06 Feb 2025
$deCIFer: Crystal Structure Prediction from Powder Diffraction Data using Autoregressive Language Models$ deCIFer: Crystal Structure Prediction from Powder Diffraction Data using Autoregressive Language Models Frederik L. Johansen Ulrik Friis-Jensen Erik B. Dam Kirsten M. Ø. Jensen Rocío Mercado Raghavendra Selvan 88 0 0 04 Feb 2025
MJ-VIDEO: Fine-Grained Benchmarking and Rewarding Video Preferences in Video Generation Haibo Tong Zhaoyang Wang Zhengzhang Chen Haonian Ji Shi Qiu ... Peng Xia Mingyu Ding Rafael Rafailov Chelsea Finn Huaxiu Yao EGVM VGen 104 2 0 03 Feb 2025
The Energy Loss Phenomenon in RLHF: A New Perspective on Mitigating Reward Hacking Yuchun Miao Sen Zhang Liang Ding Yuqi Zhang L. Zhang Dacheng Tao 81 3 0 31 Jan 2025
GuardReasoner: Towards Reasoning-based LLM Safeguards Yue Liu Hongcheng Gao Shengfang Zhai Jun-Xiong Xia Tianyi Wu Zhiwei Xue Y. Chen Kenji Kawaguchi Jiaheng Zhang Bryan Hooi AI4TS LRM 131 14 0 30 Jan 2025
Diverse Preference Optimization Jack Lanchantin Angelica Chen S. Dhuliawala Ping Yu Jason Weston Sainbayar Sukhbaatar Ilia Kulikov 93 4 0 30 Jan 2025
Think Smarter not Harder: Adaptive Reasoning with Inference Aware Optimization Zishun Yu Tengyu Xu Di Jin Karthik Abinav Sankararaman Yun He ... Eryk Helenowski Chen Zhu Sinong Wang Hao Ma Han Fang LRM 54 4 0 29 Jan 2025
SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training Tianzhe Chu Yuexiang Zhai Jihan Yang Shengbang Tong Saining Xie Dale Schuurmans Quoc V. Le Sergey Levine Yi Ma OffRL 70 58 0 28 Jan 2025
Training Dialogue Systems by AI Feedback for Improving Overall Dialogue Impression Kai Yoshida M. Mizukami Seiya Kawano Canasai Kruengkrai Hiroaki Sugiyama Koichiro Yoshino ALM OffRL 84 1 0 28 Jan 2025
BoKDiff: Best-of-K Diffusion Alignment for Target-Specific 3D Molecule Generation Ali Khodabandeh Yalabadi Mehdi Yazdani-Jahromi O. Garibay 44 0 0 28 Jan 2025
Reinforcement Learning from Human Feedback: Whose Culture, Whose Values, Whose Perspectives? Kristian González Barman Simon Lohse H. Regt OffRL 143 0 0 20 Jan 2025
Can Safety Fine-Tuning Be More Principled? Lessons Learned from Cybersecurity David Williams-King Linh Le Adam Oberman Yoshua Bengio AAML 56 0 0 19 Jan 2025
Beyond Reward Hacking: Causal Rewards for Large Language Model Alignment Chaoqi Wang Zhuokai Zhao Yibo Jiang Zhaorun Chen Chen Zhu ... Jiayi Liu Lizhu Zhang Xiangjun Fan Hao Ma Sinong Wang 77 3 0 17 Jan 2025
LLMs as Workers in Human-Computational Algorithms? Replicating Crowdsourcing Pipelines with LLMs Tongshuang Wu Haiyi Zhu Maya Albayrak Alexis Axon Amanda Bertsch ... Ying-Jui Tseng Patricia Vaidos Zhijin Wu Wei Yu Wu Chenyang Yang 86 30 0 10 Jan 2025
CREW: Facilitating Human-AI Teaming Research Lingyu Zhang Zhengran Ji Boyuan Chen 44 3 0 03 Jan 2025
WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instruct Haipeng Luo Qingfeng Sun Can Xu Pu Zhao Jian-Guang Lou ... Xiubo Geng Qingwei Lin Shifeng Chen Yansong Tang Dongmei Zhang OSLM LRM 110 408 0 03 Jan 2025
An Overview and Discussion on Using Large Language Models for Implementation Generation of Solutions to Open-Ended Problems Hashmath Shaik Alex Doboli OffRL ELM 149 0 0 31 Dec 2024
Disentangling Preference Representation and Text Generation for Efficient Individual Preference Alignment Jianfei Zhang Jun Bai Yangqiu Song Yanmeng Wang Rumei Li Chenghua Lin Wenge Rong 44 0 0 31 Dec 2024
Understanding the Logic of Direct Preference Alignment through Logic Kyle Richardson Vivek Srikumar Ashish Sabharwal 85 2 0 23 Dec 2024
FedRLHF: A Convergence-Guaranteed Federated Framework for Privacy-Preserving and Personalized RLHF Flint Xiaofeng Fan Cheston Tan Yew-Soon Ong Roger Wattenhofer Wei Tsang Ooi 85 1 0 20 Dec 2024
REFA: Reference Free Alignment for multi-preference optimization Taneesh Gupta Rahul Madhavan Xuchao Zhang Chetan Bansal Saravan Rajmohan 91 1 0 20 Dec 2024
Efficient Diversity-Preserving Diffusion Alignment via Gradient-Informed GFlowNets Zhen Liu Tim Z. Xiao Weiyang Liu Yoshua Bengio Dinghuai Zhang 123 2 0 10 Dec 2024
All Languages Matter: Evaluating LMMs on Culturally Diverse 100 Languages Ashmal Vayani Dinura Dissanayake Hasindri Watawana Noor Ahsan Nevasini Sasikumar ... Monojit Choudhury Ivan Laptev Mubarak Shah Salman Khan Fahad A Khan 124 8 0 25 Nov 2024
Sharp Analysis for KL-Regularized Contextual Bandits and RLHF Heyang Zhao Chenlu Ye Quanquan Gu Tong Zhang OffRL 57 3 0 07 Nov 2024
TODO: Enhancing LLM Alignment with Ternary Preferences Yuxiang Guo Lu Yin Bo Jiang Jiaqi Zhang 38 1 0 02 Nov 2024
Enhancing Safety in Reinforcement Learning with Human Feedback via Rectified Policy Optimization Xiyue Peng Hengquan Guo Jiawei Zhang Dongqing Zou Ziyu Shao Honghao Wei Xin Liu 44 0 0 25 Oct 2024
End-to-end Training for Recommendation with Language-based User Profiles Zhaolin Gao Joyce Zhou Yijia Dai Thorsten Joachims AI4Ed 59 3 0 24 Oct 2024
Improving Model Factuality with Fine-grained Critique-based Evaluator Yiqing Xie Wenxuan Zhou Pradyot Prakash Di Jin Yuning Mao ... Sinong Wang Han Fang Carolyn Rose Daniel Fried Hejia Zhang HILM 33 6 0 24 Oct 2024
Cross-lingual Transfer of Reward Models in Multilingual Alignment Jiwoo Hong Noah Lee Rodrigo Martínez-Castaño César Rodríguez James Thorne 48 4 0 23 Oct 2024
Asynchronous RLHF: Faster and More Efficient Off-Policy RL for Language Models Michael Noukhovitch Shengyi Huang Sophie Xhonneux Arian Hosseini Rishabh Agarwal Aaron C. Courville OffRL 79 5 0 23 Oct 2024
Science Out of Its Ivory Tower: Improving Accessibility with Reinforcement Learning Haining Wang Jason Clark Hannah McKelvey Leila Sterman Zheng Gao Zuoyu Tian Sandra Kübler Xiaozhong Liu 33 1 0 22 Oct 2024
Magnetic Preference Optimization: Achieving Last-iterate Convergence for Language Model Alignment Mingzhi Wang Chengdong Ma Qizhi Chen Linjian Meng Yang Han Jiancong Xiao Zhaowei Zhang Jing Huo Weijie Su Yaodong Yang 32 4 0 22 Oct 2024
Bridging the Training-Inference Gap in LLMs by Leveraging Self-Generated Tokens Zhepeng Cen Yao Liu Siliang Zeng Pratik Chaudhar Huzefa Rangwala George Karypis Rasool Fakoor SyDa AIFin 34 3 0 18 Oct 2024
Fine-Grained Verifiers: Preference Modeling as Next-token Prediction in Vision-Language Alignment Chenhang Cui An Zhang Yiyang Zhou Zhaorun Chen Gelei Deng Huaxiu Yao Tat-Seng Chua 73 4 0 18 Oct 2024
SPIN: Self-Supervised Prompt INjection Leon Zhou Junfeng Yang Chengzhi Mao AAML SILM 30 0 0 17 Oct 2024
Fine-Tuning Discrete Diffusion Models via Reward Optimization with Applications to DNA and Protein Design Chenyu Wang Masatoshi Uehara Yichun He Amy Wang Tommaso Biancalani Avantika Lal Tommi Jaakkola Sergey Levine Hanchen Wang Aviv Regev 53 8 0 17 Oct 2024
Negative-Prompt-driven Alignment for Generative Language Model Shiqi Qiao Ning Xv Biao Liu Xin Geng ALM SyDa 33 0 0 16 Oct 2024
JudgeBench: A Benchmark for Evaluating LLM-based Judges Sijun Tan Siyuan Zhuang Kyle Montgomery William Y. Tang Alejandro Cuadron Chenguang Wang Raluca A. Popa Ion Stoica ELM ALM 51 38 0 16 Oct 2024
CREAM: Consistency Regularized Self-Rewarding Language Models Zekun Wang Weilei He Zhiyuan Liang Xuchao Zhang Chetan Bansal Ying Wei Weitong Zhang Huaxiu Yao ALM 101 7 0 16 Oct 2024
Multi-round jailbreak attack on large language models Yihua Zhou Xiaochuan Shi AAML 21 0 0 15 Oct 2024
AdvBDGen: Adversarially Fortified Prompt-Specific Fuzzy Backdoor Generator Against LLM Alignment Pankayaraj Pathmanathan Udari Madhushani Sehwag Michael-Andrei Panaitescu-Liess Furong Huang SILM AAML 43 0 0 15 Oct 2024
FB-Bench: A Fine-Grained Multi-Task Benchmark for Evaluating LLMs' Responsiveness to Human Feedback Heng Chang Miao Zheng Fan Yang Guosheng Dong Bin Cui Xin Wu Zenan Zhou Wentao Zhang ALM 46 6 0 12 Oct 2024
Simultaneous Reward Distillation and Preference Learning: Get You a Language Model Who Can Do Both Abhijnan Nath Changsoo Jung Ethan Seefried Nikhil Krishnaswamy 134 1 0 11 Oct 2024
Unintentional Unalignment: Likelihood Displacement in Direct Preference Optimization Noam Razin Sadhika Malladi Adithya Bhaskar Danqi Chen Sanjeev Arora Boris Hanin 93 16 0 11 Oct 2024
RDT-1B: a Diffusion Foundation Model for Bimanual Manipulation Songming Liu Lingxuan Wu Bangguo Li Hengkai Tan Huayu Chen Zhengyi Wang Ke Xu Hang Su Jun Zhu 34 77 0 10 Oct 2024
GenARM: Reward Guided Generation with Autoregressive Reward Model for Test-time Alignment Yuancheng Xu Udari Madhushani Sehwag Alec Koppel Sicheng Zhu Bang An Furong Huang Sumitra Ganesh 55 6 0 10 Oct 2024
From Exploration to Mastery: Enabling LLMs to Master Tools via Self-Driven Interactions Changle Qu Sunhao Dai Xiaochi Wei Hengyi Cai Shuaiqiang Wang Dawei Yin Jun Xu Ji-Rong Wen 60 9 0 10 Oct 2024
Simplicity Prevails: Rethinking Negative Preference Optimization for LLM Unlearning Chongyu Fan Jiancheng Liu Licong Lin Jinghan Jia Ruiqi Zhang Song Mei Sijia Liu MU 43 16 0 09 Oct 2024
RL, but don't do anything I wouldn't do Michael K. Cohen Marcus Hutter Yoshua Bengio Stuart J. Russell OffRL 35 2 0 08 Oct 2024