Online Merging Optimizers for Boosting Rewards and Mitigating Tax in
Alignment

Online Merging Optimizers for Boosting Rewards and Mitigating Tax in Alignment

28 May 2024

Keming Lu

Bowen Yu

Fei Huang

Yang Fan

Runji Lin

Chang Zhou

Papers citing "Online Merging Optimizers for Boosting Rewards and Mitigating Tax in Alignment"

12 / 12 papers shown

Title
Reinforcement Learning Enhanced LLMs: A Survey Shuhe Wang Shengyu Zhang Jingyang Zhang Runyi Hu Xiaoya Li Tianwei Zhang Jiwei Li Fei Wu G. Wang Eduard H. Hovy OffRL 134 7 0 05 Dec 2024
From Lists to Emojis: How Format Bias Affects Model Alignment Xuanchang Zhang Wei Xiong Lichang Chen Dinesh Manocha Heng Huang Tong Zhang ALM 35 11 0 18 Sep 2024
Unlocking Decoding-time Controllability: Gradient-Free Multi-Objective Alignment with Contrastive Prompts Tingchen Fu Yupeng Hou Julian McAuley Rui Yan 38 3 0 09 Aug 2024
Model Surgery: Modulating LLM's Behavior Via Simple Parameter Editing Huanqian Wang Yang Yue Rui Lu Jingxin Shi Andrew Zhao Shenzhi Wang Shiji Song Gao Huang LM&Ro KELM 51 6 0 11 Jul 2024
Merge, Ensemble, and Cooperate! A Survey on Collaborative Strategies in the Era of Large Language Models Jinliang Lu Ziliang Pang Min Xiao Yaochen Zhu Rui Xia Jiajun Zhang MoMe 49 18 0 08 Jul 2024
Merging Improves Self-Critique Against Jailbreak Attacks Victor Gallego AAML MoMe 44 3 0 11 Jun 2024
Direct Preference Optimization with an Offset Afra Amini Tim Vieira Ryan Cotterell 73 55 0 16 Feb 2024
KTO: Model Alignment as Prospect Theoretic Optimization Kawin Ethayarajh Winnie Xu Niklas Muennighoff Dan Jurafsky Douwe Kiela 176 449 0 02 Feb 2024
Language Model Alignment with Elastic Reset Michael Noukhovitch Samuel Lavoie Florian Strub Aaron Courville KELM 91 25 0 06 Dec 2023
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 313 11,953 0 04 Mar 2022
Raise a Child in Large Language Model: Towards Effective and Generalizable Fine-tuning Runxin Xu Fuli Luo Zhiyuan Zhang Chuanqi Tan Baobao Chang Songfang Huang Fei Huang LRM 145 178 0 13 Sep 2021
Mixout: Effective Regularization to Finetune Large-scale Pretrained Language Models Cheolhyoung Lee Kyunghyun Cho Wanmo Kang MoE 249 205 0 25 Sep 2019