Noise Contrastive Alignment of Language Models with Explicit Rewards

8 February 2024

Jun Zhu

Papers citing "Noise Contrastive Alignment of Language Models with Explicit Rewards"

34 / 34 papers shown

Title
InfoPO: On Mutual Information Maximization for Large Language Model Alignment Teng Xiao Zhen Ge Sujay Sanghavi Tian Wang Julian Katz-Samuels Marc Versage Qingjun Cui Trishul Chilimbi 31 0 0 13 May 2025
Training Small Reasoning LLMs with Cognitive Preference Alignment Wenrui Cai Chengyu Wang Junbing Yan Jun Huang Xiangzhong Fang LRM 26 1 0 14 Apr 2025
A Comprehensive Survey of Reward Models: Taxonomy, Applications, Challenges, and Future Jialun Zhong Wei Shen Yanzeng Li Songyang Gao Hua Lu Yicheng Chen Yang Zhang Wei Zhou Jinjie Gu Lei Zou LRM 45 2 0 12 Apr 2025
Debiasing Multimodal Large Language Models via Noise-Aware Preference Optimization Zefeng Zhang Hengzhu Tang Jiawei Sheng Zhenyu Zhang Yiming Ren Zhenyang Li Dawei Yin Duohe Ma Tingwen Liu 47 0 0 23 Mar 2025
A Survey on Mathematical Reasoning and Optimization with Large Language Models Ali Forootani OffRL LRM AI4CE 45 0 0 22 Mar 2025
IterPref: Focal Preference Learning for Code Generation via Iterative Debugging Jie Wu Haoling Li Xin Zhang Jianwen Luo Yangyu Huang Ruihang Chu Yuqing Yang Scarlett Li 75 0 0 04 Mar 2025
Discriminative Finetuning of Generative Large Language Models without Reward Models and Human Preference Data Siqi Guo Ilgee Hong Vicente Balmaseda Changlong Yu Liang Qiu Xin Liu Haoming Jiang Tuo Zhao Tianbao Yang 43 0 0 25 Feb 2025
AMPO: Active Multi-Preference Optimization Taneesh Gupta Rahul Madhavan Xuchao Zhang Chetan Bansal Saravan Rajmohan 55 0 0 25 Feb 2025
SimPER: A Minimalist Approach to Preference Alignment without Hyperparameters Teng Xiao Yige Yuan Z. Chen Mingxiao Li Shangsong Liang Z. Ren V. Honavar 95 5 0 21 Feb 2025
Process Reinforcement through Implicit Rewards Ganqu Cui Lifan Yuan Zhilin Wang Hanbin Wang Wendi Li ... Yu Cheng Zhiyuan Liu Maosong Sun Bowen Zhou Ning Ding OffRL LRM 73 52 0 03 Feb 2025
Visual Generation Without Guidance Huayu Chen Kai Jiang Kaiwen Zheng Jianfei Chen Hang Su Jun Zhu 57 0 0 28 Jan 2025
REFA: Reference Free Alignment for multi-preference optimization Taneesh Gupta Rahul Madhavan Xuchao Zhang Chetan Bansal Saravan Rajmohan 91 1 0 20 Dec 2024
Energy-Based Preference Model Offers Better Offline Alignment than the Bradley-Terry Preference Model Yuzhong Hong Hanshan Zhang Junwei Bao Hongfei Jiang Yang Song OffRL 77 1 0 18 Dec 2024
Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimization Weiyun Wang Zhe Chen Wenhai Wang Yue Cao Yangzhou Liu ... Jinguo Zhu X. Zhu Lewei Lu Yu Qiao Jifeng Dai LRM 62 47 1 15 Nov 2024
Modality-Fair Preference Optimization for Trustworthy MLLM Alignment Songtao Jiang Yan Zhang Ruizhe Chen Yeying Jin Zuozhu Liu MLLM MoE 28 6 0 20 Oct 2024
Toward Guidance-Free AR Visual Generation via Condition Contrastive Alignment Huayu Chen Hang Su Peize Sun Jun Zhu VLM 48 3 0 12 Oct 2024
TPO: Aligning Large Language Models with Multi-branch & Multi-step Preference Trees Weibin Liao Xu Chu Yasha Wang LRM 48 6 0 10 Oct 2024
Reward-Augmented Data Enhances Direct Preference Alignment of LLMs Shenao Zhang Zhihan Liu Boyi Liu Yuhang Zhang Yingxiang Yang Y. Liu Liyu Chen Tao Sun Ziyi Wang 98 3 0 10 Oct 2024
Evaluation of Large Language Models for Summarization Tasks in the Medical Domain: A Narrative Review Emma Croxford Yanjun Gao Nicholas Pellegrino Karen K. Wong Graham Wills Elliot First Frank J. Liao Cherodeep Goswami Brian Patterson Majid Afshar HILM ELM LM&MA 37 1 0 26 Sep 2024
Alignment with Preference Optimization Is All You Need for LLM Safety Réda Alami Ali Khalifa Almansoori Ahmed Alzubaidi M. Seddik Mugariya Farooq Hakim Hacid 34 1 0 12 Sep 2024
Aligning Diffusion Behaviors with Q-functions for Efficient Continuous Control Huayu Chen Kaiwen Zheng Hang Su Jun Zhu 51 1 0 12 Jul 2024
Aligning Target-Aware Molecule Diffusion Models with Exact Energy Optimization Siyi Gu Minkai Xu Alexander Powers Weili Nie Tomas Geffner Karsten Kreis J. Leskovec Arash Vahdat Stefano Ermon 48 7 0 01 Jul 2024
On the Transformations across Reward Model, Parameter Update, and In-Context Prompt Deng Cai Huayang Li Tingchen Fu Siheng Li Weiwen Xu ... Leyang Cui Yan Wang Lemao Liu Taro Watanabe Shuming Shi KELM 30 2 0 24 Jun 2024
mDPO: Conditional Preference Optimization for Multimodal Large Language Models Fei Wang Wenxuan Zhou James Y. Huang Nan Xu Sheng Zhang Hoifung Poon Muhao Chen 66 15 0 17 Jun 2024
Eliminating Biased Length Reliance of Direct Preference Optimization via Down-Sampled KL Divergence Junru Lu Jiazheng Li Siyu An Meng Zhao Yulan He Di Yin Xing Sun 41 14 0 16 Jun 2024
Step-level Value Preference Optimization for Mathematical Reasoning Guoxin Chen Minpeng Liao Chengxi Li Kai Fan LRM 37 34 0 16 Jun 2024
On Softmax Direct Preference Optimization for Recommendation Yuxin Chen Junfei Tan An Zhang Zhengyi Yang Leheng Sheng Enzhi Zhang Xiang Wang Tat-Seng Chua 34 24 0 13 Jun 2024
3D-Properties: Identifying Challenges in DPO and Charting a Path Forward Yuzi Yan Yibo Miao J. Li Yipin Zhang Jian Xie Zhijie Deng Dong Yan 54 11 0 11 Jun 2024
360Zhinao Technical Report 360Zhinao Team 40 0 0 22 May 2024
RLHF Workflow: From Reward Modeling to Online RLHF Hanze Dong Wei Xiong Bo Pang Haoxiang Wang Han Zhao Yingbo Zhou Nan Jiang Doyen Sahoo Caiming Xiong Tong Zhang OffRL 29 95 0 13 May 2024
Advancing LLM Reasoning Generalists with Preference Trees Lifan Yuan Ganqu Cui Hanbin Wang Ning Ding Xingyao Wang ... Zhenghao Liu Bowen Zhou Hao Peng Zhiyuan Liu Maosong Sun LRM 39 98 0 02 Apr 2024
KTO: Model Alignment as Prospect Theoretic Optimization Kawin Ethayarajh Winnie Xu Niklas Muennighoff Dan Jurafsky Douwe Kiela 173 449 0 02 Feb 2024
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 313 11,953 0 04 Mar 2022
Fine-Tuning Language Models from Human Preferences Daniel M. Ziegler Nisan Stiennon Jeff Wu Tom B. Brown Alec Radford Dario Amodei Paul Christiano G. Irving ALM 280 1,595 0 18 Sep 2019