Diverse Weight Averaging for Out-of-Distribution Generalization

19 May 2022

Papers citing "Diverse Weight Averaging for Out-of-Distribution Generalization"

25 / 25 papers shown

Title
Param $Δ$ for Direct Weight Mixing: Post-Train Large Language Model at Zero Cost Sheng Cao Mingrui Wu Karthik Prasad Yuandong Tian Zechun Liu MoMe 80 0 0 23 Apr 2025
When is Task Vector Provably Effective for Model Editing? A Generalization Analysis of Nonlinear Transformers Hongkang Li Yihua Zhang Shuai Zhang M. Wang Sijia Liu Pin-Yu Chen MoMe 69 2 0 15 Apr 2025
Generalizable Prompt Learning of CLIP: A Brief Overview Fangming Cui Yonggang Zhang Xuan Wang Xule Wang Liang Xiao VPVLM VLM 153 0 0 03 Mar 2025
LED-Merging: Mitigating Safety-Utility Conflicts in Model Merging with Location-Election-Disjoint Qianli Ma Dongrui Liu Qian Chen Linfeng Zhang Jing Shao MoMe 136 0 0 24 Feb 2025
Task Singular Vectors: Reducing Task Interference in Model Merging Antonio Andrea Gargiulo Donato Crisostomi Maria Sofia Bucarelli Simone Scardapane Fabrizio Silvestri Emanuele Rodolà MoMe 87 8 0 26 Nov 2024
QT-DoG: Quantization-aware Training for Domain Generalization Saqib Javed Hieu Le Mathieu Salzmann OOD MQ 28 1 0 08 Oct 2024
Merge, Ensemble, and Cooperate! A Survey on Collaborative Strategies in the Era of Large Language Models Jinliang Lu Ziliang Pang Min Xiao Yaochen Zhu Rui Xia Jiajun Zhang MoMe 38 18 0 08 Jul 2024
WARP: On the Benefits of Weight Averaged Rewarded Policies Alexandre Ramé Johan Ferret Nino Vieillard Robert Dadashi Léonard Hussenot Pierre-Louis Cedoz Pier Giuseppe Sessa Sertan Girgin Arthur Douillard Olivier Bachem 54 13 0 24 Jun 2024
TS-Align: A Teacher-Student Collaborative Framework for Scalable Iterative Finetuning of Large Language Models Chen Zhang Chengguang Tang Dading Chong Ke Shi Guohua Tang Feng Jiang Haizhou Li 31 4 0 30 May 2024
Merging Text Transformer Models from Different Initializations Neha Verma Maha Elbayad MoMe 56 7 0 01 Mar 2024
Fine-tuning with Very Large Dropout Jianyu Zhang Léon Bottou 37 1 0 01 Mar 2024
Knowledge Fusion of Large Language Models Fanqi Wan Xinting Huang Deng Cai Xiaojun Quan Wei Bi Shuming Shi MoMe 29 61 0 19 Jan 2024
Domain Aligned Prefix Averaging for Domain Generalization in Abstractive Summarization Pranav Ajit Nair Sukomal Pal Pradeepika Verm MoMe 32 2 0 26 May 2023
HGFormer: Hierarchical Grouping Transformer for Domain Generalized Semantic Segmentation Jian Ding Nan Xue Guisong Xia Bernt Schiele Dengxin Dai ViT 17 30 0 22 May 2023
Task Arithmetic in the Tangent Space: Improved Editing of Pre-Trained Models Guillermo Ortiz-Jiménez Alessandro Favero P. Frossard MoMe 42 103 0 22 May 2023
Personalised Federated Learning On Heterogeneous Feature Spaces A. Rakotomamonjy Maxime Vono H. M. Ruiz L. Ralaivola FedML 18 8 0 26 Jan 2023
Dataless Knowledge Fusion by Merging Weights of Language Models Xisen Jin Xiang Ren Daniel Preotiuc-Pietro Pengxiang Cheng FedML MoMe 18 211 0 19 Dec 2022
Learning useful representations for shifting tasks and distributions Jianyu Zhang Léon Bottou OOD 34 13 0 14 Dec 2022
Towards Generalized Few-Shot Open-Set Object Detection Binyi Su Hua Zhang Jingzhi Li Zhongjun Zhou 43 9 0 28 Oct 2022
Diversify and Disambiguate: Learning From Underspecified Data Yoonho Lee Huaxiu Yao Chelsea Finn 205 64 0 07 Feb 2022
Neural Tangent Kernel Beyond the Infinite-Width Limit: Effects of Depth and Initialization Mariia Seleznova Gitta Kutyniok 179 16 0 01 Feb 2022
Fishr: Invariant Gradient Variances for Out-of-Distribution Generalization Alexandre Ramé Corentin Dancette Matthieu Cord OOD 38 204 0 07 Sep 2021
SWAD: Domain Generalization by Seeking Flat Minima Junbum Cha Sanghyuk Chun Kyungjae Lee Han-Cheol Cho Seunghyun Park Yunsung Lee Sungrae Park MoMe 216 423 0 17 Feb 2021
Out-of-Distribution Generalization via Risk Extrapolation (REx) David M. Krueger Ethan Caballero J. Jacobsen Amy Zhang Jonathan Binas Dinghuai Zhang Rémi Le Priol Aaron Courville OOD 215 901 0 02 Mar 2020
Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles Balaji Lakshminarayanan Alexander Pritzel Charles Blundell UQCV BDL 270 5,660 0 05 Dec 2016