TIES-Merging: Resolving Interference When Merging Models

2 June 2023

Papers citing "TIES-Merging: Resolving Interference When Merging Models"

50 / 221 papers shown

Title
Knowledge Mechanisms in Large Language Models: A Survey and Perspective Meng Wang Yunzhi Yao Ziwen Xu Shuofei Qiao Shumin Deng ... Yong-jia Jiang Pengjun Xie Fei Huang Huajun Chen Ningyu Zhang 55 28 0 22 Jul 2024
Mitigating Catastrophic Forgetting in Language Transfer via Model Merging Anton Alexandrov Veselin Raychev Mark Niklas Muller Ce Zhang Martin Vechev Kristina Toutanova MoMe CLL KELM 42 14 0 11 Jul 2024
Foundation Model Engineering: Engineering Foundation Models Just as Engineering Software Dezhi Ran Mengzhou Wu Wei Yang Tao Xie AI4CE 39 1 0 11 Jul 2024
Pareto Low-Rank Adapters: Efficient Multi-Task Learning with Preferences Nikolaos Dimitriadis Pascal Frossard F. Fleuret MoE 67 6 0 10 Jul 2024
MagMax: Leveraging Model Merging for Seamless Continual Learning Daniel Marczak Bartłomiej Twardowski Tomasz Trzciñski Sebastian Cygert MoMe CLL 53 18 0 08 Jul 2024
Merge, Ensemble, and Cooperate! A Survey on Collaborative Strategies in the Era of Large Language Models Jinliang Lu Ziliang Pang Min Xiao Yaochen Zhu Rui Xia Jiajun Zhang MoMe 52 18 0 08 Jul 2024
Harmony in Diversity: Merging Neural Networks with Canonical Correlation Analysis Stefan Horoi Albert Manuel Orozco Camacho Eugene Belilovsky Guy Wolf FedML MoMe 35 9 0 07 Jul 2024
Unlocking the Potential of Model Merging for Low-Resource Languages Mingxu Tao Chen Zhang Quzhe Huang Tianyao Ma Songfang Huang Dongyan Zhao Yansong Feng CLL MoMe 32 3 0 04 Jul 2024
Knowledge Composition using Task Vectors with Learned Anisotropic Scaling Frederic Z. Zhang Paul Albert Cristian Rodriguez-Opazo Anton van den Hengel Ehsan Abbasnejad MoMe 61 8 0 03 Jul 2024
PLeaS -- Merging Models with Permutations and Least Squares Anshul Nasery J. Hayase Pang Wei Koh Sewoong Oh MoMe 51 3 0 02 Jul 2024
DogeRM: Equipping Reward Models with Domain Knowledge through Model Merging Tzu-Han Lin Chen An Li Hung-yi Lee Yun-Nung Chen VLM ALM 26 4 0 01 Jul 2024
It's Morphing Time: Unleashing the Potential of Multiple LLMs via Multi-objective Optimization Bingdong Li Zixiang Di Yanting Yang Hong Qian Peng Yang Hao Hao Ke Tang Aimin Zhou MoMe 21 5 0 29 Jun 2024
Sequential Editing for Lifelong Training of Speech Recognition Models Devang Kulshreshtha Saket Dingliwal Brady C. Houston Nikolaos Pappas S. Ronanki KELM CLL 34 1 0 25 Jun 2024
Lottery Ticket Adaptation: Mitigating Destructive Interference in LLMs Ashwinee Panda Berivan Isik Xiangyu Qi Sanmi Koyejo Tsachy Weissman Prateek Mittal MoMe 45 14 0 24 Jun 2024
WARP: On the Benefits of Weight Averaged Rewarded Policies Alexandre Ramé Johan Ferret Nino Vieillard Robert Dadashi Léonard Hussenot Pierre-Louis Cedoz Pier Giuseppe Sessa Sertan Girgin Arthur Douillard Olivier Bachem 62 14 0 24 Jun 2024
DEM: Distribution Edited Model for Training with Mixed Data Distributions Dhananjay Ram Aditya Rawal Momchil Hardalov Nikolaos Pappas Sheng Zha MoMe 59 1 0 21 Jun 2024
Model Merging and Safety Alignment: One Bad Model Spoils the Bunch Hasan Hammoud Umberto Michieli Fabio Pizzati Philip Torr Adel Bibi Guohao Li Mete Ozay MoMe 31 15 0 20 Jun 2024
Self-MoE: Towards Compositional Large Language Models with Self-Specialized Experts Junmo Kang Leonid Karlinsky Hongyin Luo Zhen Wang Jacob A. Hansen James Glass David D. Cox Yikang Shen Rogerio Feris Alan Ritter MoMe MoE 42 8 0 17 Jun 2024
Safety Arithmetic: A Framework for Test-time Safety Alignment of Language Models by Steering Parameters and Activations Rima Hazra Sayan Layek Somnath Banerjee Soujanya Poria KELM LLMSV 37 7 0 17 Jun 2024
Split, Unlearn, Merge: Leveraging Data Attributes for More Effective Unlearning in LLMs S. Kadhe Farhan Ahmed Dennis Wei Nathalie Baracaldo Inkit Padhi MoMe MU 28 7 0 17 Jun 2024
MetaGPT: Merging Large Language Models Using Model Exclusive Task Arithmetic Yuyan Zhou Liang Song Bingning Wang Weipeng Chen MoMe 30 17 0 17 Jun 2024
On Giant's Shoulders: Effortless Weak to Strong by Dynamic Logits Fusion Chenghao Fan Zhenyi Lu Wei Wei Jie Tian Xiaoye Qu Dangyang Chen Yu Cheng MoMe 48 5 0 17 Jun 2024
Twin-Merging: Dynamic Integration of Modular Expertise in Model Merging Zhenyi Lu Chenghao Fan Wei Wei Xiaoye Qu Dangyang Chen Yu Cheng MoMe 50 48 0 17 Jun 2024
Compress then Serve: Serving Thousands of LoRA Adapters with Little Overhead Rickard Brüel-Gabrielsson Jiacheng Zhu Onkar Bhardwaj Leshem Choshen Kristjan Greenewald Mikhail Yurochkin Justin Solomon 51 5 0 17 Jun 2024
A Survey on Large Language Models from General Purpose to Medical Applications: Datasets, Methodologies, and Evaluations Jinqiang Wang Huansheng Ning Yi Peng Qikai Wei Daniel Tesfai Wenwei Mao Tao Zhu Runhe Huang LM&MA AI4MH ELM 49 5 0 14 Jun 2024
ME-Switch: A Memory-Efficient Expert Switching Framework for Large Language Models Jing Liu Ruihao Gong Mingyang Zhang Yefei He Jianfei Cai Bohan Zhuang MoE 45 0 0 13 Jun 2024
Merging Improves Self-Critique Against Jailbreak Attacks Victor Gallego AAML MoMe 44 3 0 11 Jun 2024
MAP: Low-compute Model Merging with Amortized Pareto Fronts via Quadratic Approximation Lu Li Tianze Zhang Zhiqi Bu Suyuchen Wang Huan He Jie Fu Yonghui Wu Jiang Bian Yong Chen Yoshua Bengio FedML MoMe 100 3 0 11 Jun 2024
LLM-based speaker diarization correction: A generalizable approach Georgios Efstathiadis Vijay Yadav Anzar Abbas 45 3 0 07 Jun 2024
HPE-CogVLM: New Head Pose Grounding Task Exploration on Vision Language Model Yu Tian Tianqi Shao Tsukasa Demizu Xuyang Wu Hsin-Tai Wu 29 0 0 04 Jun 2024
Pretrained Hybrids with MAD Skills Nicholas Roberts Samuel Guo Zhiqi Gao Satya Sai Srinath Namburi Sonia Cromp Chengjun Wu Chengyu Duan Frederic Sala Mamba 42 0 0 02 Jun 2024
Online Merging Optimizers for Boosting Rewards and Mitigating Tax in Alignment Keming Lu Bowen Yu Fei Huang Yang Fan Runji Lin Chang Zhou MoMe 32 18 0 28 May 2024
Exploring the LLM Journey from Cognition to Expression with Linear Representations Yuzi Yan J. Li Yipin Zhang Dong Yan 49 1 0 27 May 2024
ConStat: Performance-Based Contamination Detection in Large Language Models Jasper Dekoninck Mark Niklas Muller Martin Vechev 37 7 0 25 May 2024
WISE: Rethinking the Knowledge Memory for Lifelong Model Editing of Large Language Models Peng Wang Zexi Li Ningyu Zhang Ziwen Xu Yunzhi Yao Yong-jia Jiang Pengjun Xie Fei Huang Huajun Chen KELM CLL 53 20 0 23 May 2024
MiniCache: KV Cache Compression in Depth Dimension for Large Language Models Akide Liu Jing Liu Zizheng Pan Yefei He Gholamreza Haffari Bohan Zhuang MQ 35 30 0 23 May 2024
EMR-Merging: Tuning-Free High-Performance Model Merging Chenyu Huang Peng Ye Tao Chen Tong He Xiangyu Yue Wanli Ouyang MoMe 46 29 0 23 May 2024
Evaluating Large Language Models with Human Feedback: Establishing a Swedish Benchmark Birger Moëll ALM ELM 27 1 0 22 May 2024
Exploring and Exploiting the Asymmetric Valley of Deep Neural Networks Xin-Chun Li Jinli Tang Bo Zhang Lan Li De-Chuan Zhan 49 2 0 21 May 2024
MeteoRA: Multiple-tasks Embedded LoRA for Large Language Models Jingwei Xu Junyu Lai Yunpeng Huang MoE MoMe 38 9 0 19 May 2024
Towards Modular LLMs by Building and Reusing a Library of LoRAs O. Ostapenko Zhan Su Edoardo Ponti Laurent Charlin Nicolas Le Roux Matheus Pereira Lucas Caccia Alessandro Sordoni MoMe 44 31 0 18 May 2024
A safety realignment framework via subspace-oriented model fusion for large language models Xin Yi Shunfan Zheng Linlin Wang Xiaoling Wang Liang He 60 21 0 15 May 2024
Zero-Shot Tokenizer Transfer Benjamin Minixhofer Edoardo Ponti Ivan Vulić VLM 44 9 0 13 May 2024
Localizing Task Information for Improved Model Merging and Compression Ke Wang Nikolaos Dimitriadis Guillermo Ortiz-Jimenez Franccois Fleuret Pascal Frossard MoMe 33 46 0 13 May 2024
HFT: Half Fine-Tuning for Large Language Models Tingfeng Hui Zhenyu Zhang Shuohuan Wang Weiran Xu Yu Sun Hua Wu CLL 45 4 0 29 Apr 2024
No Train but Gain: Language Arithmetic for training-free Language Adapters enhancement Mateusz Klimaszewski Piotr Andruszkiewicz Alexandra Birch MoMe 47 4 0 24 Apr 2024
A Survey on Self-Evolution of Large Language Models Zhengwei Tao Ting-En Lin Xiancai Chen Hangyu Li Yuchuan Wu Yongbin Li Zhi Jin Fei Huang Dacheng Tao Jingren Zhou LRM LM&Ro 57 22 0 22 Apr 2024
MaxFusion: Plug&Play Multi-Modal Generation in Text-to-Image Diffusion Models Nithin Gopalakrishnan Nair Jeya Maria Jose Valanarasu Vishal M. Patel MoMe 33 7 0 15 Apr 2024
MedExpQA: Multilingual Benchmarking of Large Language Models for Medical Question Answering Inigo Alonso Maite Oronoz Rodrigo Agerri AI4MH LM&MA ELM 52 16 1 08 Apr 2024
Have You Merged My Model? On The Robustness of Large Language Model IP Protection Methods Against Model Merging Tianshuo Cong Delong Ran Zesen Liu Xinlei He Jinyuan Liu Yichen Gong Qi Li Anyu Wang Xiaoyun Wang MoMe 46 7 0 08 Apr 2024