WARP: On the Benefits of Weight Averaged Rewarded Policies

24 June 2024

Papers citing "WARP: On the Benefits of Weight Averaged Rewarded Policies"

33 / 33 papers shown

Title
A Modular Approach for Clinical SLMs Driven by Synthetic Data with Pre-Instruction Tuning, Model Merging, and Clinical-Tasks Alignment Jean-Philippe Corbeil Amin Dada Jean-Michel Attendu Asma Ben Abacha Alessandro Sordoni Lucas Caccia François Beaulieu Thomas Lin Jens Kleesiek Paul Vozila LM&MA 12 0 0 15 May 2025
Gemma 3 Technical Report Gemma Team Aishwarya B Kamath Johan Ferret Shreya Pathak Nino Vieillard ... Harshal Tushar Lehri Hussein Hazimeh Ian Ballantyne Idan Szpektor Ivan Nardini VLM 90 32 0 25 Mar 2025
RL-finetuning LLMs from on- and off-policy data with a single algorithm Yunhao Tang Taco Cohen David W. Zhang Michal Valko Rémi Munos OffRL 44 2 0 25 Mar 2025
Low-rank bias, weight decay, and model merging in neural networks Ilja Kuzborskij Yasin Abbasi-Yadkori 47 0 0 24 Feb 2025
Rethinking Mixture-of-Agents: Is Mixing Different Large Language Models Beneficial? Wenzhe Li Yong Lin Mengzhou Xia Chi Jin MoE 91 2 0 02 Feb 2025
Parameter-Efficient Interventions for Enhanced Model Merging Marcin Osial Daniel Marczak Bartosz Zieliñski MoMe 84 1 0 22 Dec 2024
Exponential Moving Average of Weights in Deep Learning: Dynamics and Benefits Daniel Morales-Brotons Thijs Vogels Hadrien Hendrikx 123 17 0 27 Nov 2024
Model Swarms: Collaborative Search to Adapt LLM Experts via Swarm Intelligence Shangbin Feng Zifeng Wang Yike Wang Sayna Ebrahimi Hamid Palangi ... Nathalie Rauschmayr Yejin Choi Yulia Tsvetkov Chen-Yu Lee Tomas Pfister MoMe 35 3 0 15 Oct 2024
What Matters for Model Merging at Scale? Prateek Yadav Tu Vu Jonathan Lai Alexandra Chronopoulou Manaal Faruqui Joey Tianyi Zhou Tsendsuren Munkhdalai MoMe 46 15 0 04 Oct 2024
RRM: Robust Reward Model Training Mitigates Reward Hacking Tianqi Liu Wei Xiong Jie Jessie Ren Lichang Chen Junru Wu ... Yuan Liu Bilal Piot Abe Ittycheriah Aviral Kumar Mohammad Saleh AAML 56 13 0 20 Sep 2024
From Lists to Emojis: How Format Bias Affects Model Alignment Xuanchang Zhang Wei Xiong Lichang Chen Dinesh Manocha Heng Huang Tong Zhang ALM 35 11 0 18 Sep 2024
Gemma 2: Improving Open Language Models at a Practical Size Gemma Team Gemma Team Morgane Riviere Shreya Pathak Pier Giuseppe Sessa Cassidy Hardin ... Noah Fiedel Armand Joulin Kathleen Kenealy Robert Dadashi Alek Andreev VLM MoE OSLM 37 663 0 31 Jul 2024
MAP: Low-compute Model Merging with Amortized Pareto Fronts via Quadratic Approximation Lu Li T. Zhang Zhiqi Bu Suyuchen Wang Huan He Jie Fu Yonghui Wu Jiang Bian Yong Chen Yoshua Bengio FedML MoMe 100 3 0 11 Jun 2024
Gemma: Open Models Based on Gemini Research and Technology Gemma Team Gemma Team Thomas Mesnard Cassidy Hardin Robert Dadashi Surya Bhupatiraju ... Armand Joulin Noah Fiedel Evan Senter Alek Andreev Kathleen Kenealy VLM LLMAG 131 431 0 13 Mar 2024
BioMistral: A Collection of Open-Source Pretrained Large Language Models for Medical Domains Yanis Labrak Adrien Bazoge Emmanuel Morin P. Gourraud Mickael Rouvier Richard Dufour 105 191 0 15 Feb 2024
Decoding-time Realignment of Language Models Tianlin Liu Shangmin Guo Leonardo Bianco Daniele Calandriello Quentin Berthet Felipe Llinares-López Jessica Hoffmann Lucas Dixon Michal Valko Mathieu Blondel AI4CE 54 35 0 05 Feb 2024
More Agents Is All You Need Junyou Li Qin Zhang Yangbin Yu Qiang Fu Deheng Ye LLMAG 144 63 0 03 Feb 2024
WARM: On the Benefits of Weight Averaged Reward Models Alexandre Ramé Nino Vieillard Léonard Hussenot Robert Dadashi Geoffrey Cideron Olivier Bachem Johan Ferret 114 93 0 22 Jan 2024
Language Model Alignment with Elastic Reset Michael Noukhovitch Samuel Lavoie Florian Strub Aaron Courville KELM 91 25 0 06 Dec 2023
LM-Cocktail: Resilient Tuning of Language Models via Model Merging Shitao Xiao Zheng Liu Peitian Zhang Xingrun Xing MoMe KELM 91 24 0 22 Nov 2023
DiLoCo: Distributed Low-Communication Training of Language Models Arthur Douillard Qixuang Feng Andrei A. Rusu Rachita Chhaparia Yani Donchev A. Kuncoro MarcÁurelio Ranzato Arthur Szlam Jiajun Shen 58 31 0 14 Nov 2023
Towards Understanding Sycophancy in Language Models Mrinank Sharma Meg Tong Tomasz Korbak David Duvenaud Amanda Askell ... Oliver Rausch Nicholas Schiefer Da Yan Miranda Zhang Ethan Perez 213 192 0 20 Oct 2023
Understanding the Effects of RLHF on LLM Generalisation and Diversity Robert Kirk Ishita Mediratta Christoforos Nalmpantis Jelena Luketina Eric Hambro Edward Grefenstette Roberta Raileanu AI4CE ALM 106 122 0 10 Oct 2023
Sparks of Artificial General Intelligence: Early experiments with GPT-4 Sébastien Bubeck Varun Chandrasekaran Ronen Eldan J. Gehrke Eric Horvitz ... Scott M. Lundberg Harsha Nori Hamid Palangi Marco Tulio Ribeiro Yi Zhang ELM AI4MH AI4CE ALM 298 3,007 0 22 Mar 2023
Weight Averaging: A Simple Yet Effective Method to Overcome Catastrophic Forgetting in Automatic Speech Recognition Steven Vander Eeckt Hugo Van hamme CLL MoMe 58 14 0 27 Oct 2022
Git Re-Basin: Merging Models modulo Permutation Symmetries Samuel K. Ainsworth J. Hayase S. Srinivasa MoMe 255 314 0 11 Sep 2022
Diverse Weight Averaging for Out-of-Distribution Generalization Alexandre Ramé Matthieu Kirchmeyer Thibaud Rahier A. Rakotomamonjy Patrick Gallinari Matthieu Cord OOD 199 128 0 19 May 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 313 11,953 0 04 Mar 2022
Emerging Properties in Self-Supervised Vision Transformers Mathilde Caron Hugo Touvron Ishan Misra Hervé Jégou Julien Mairal Piotr Bojanowski Armand Joulin 317 5,785 0 29 Apr 2021
SWAD: Domain Generalization by Seeking Flat Minima Junbum Cha Sanghyuk Chun Kyungjae Lee Han-Cheol Cho Seunghyun Park Yunsung Lee Sungrae Park MoMe 216 423 0 17 Feb 2021
Fine-Tuning Language Models from Human Preferences Daniel M. Ziegler Nisan Stiennon Jeff Wu Tom B. Brown Alec Radford Dario Amodei Paul Christiano G. Irving ALM 280 1,595 0 18 Sep 2019
AI safety via debate G. Irving Paul Christiano Dario Amodei 204 200 0 02 May 2018
Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles Balaji Lakshminarayanan Alexander Pritzel Charles Blundell UQCV BDL 276 5,661 0 05 Dec 2016