Extend Model Merging from Fine-Tuned to Pre-Trained Large Language
Models via Weight Disentanglement

Extend Model Merging from Fine-Tuned to Pre-Trained Large Language Models via Weight Disentanglement

6 August 2024

Yongbin Li

Papers citing "Extend Model Merging from Fine-Tuned to Pre-Trained Large Language Models via Weight Disentanglement"

6 / 6 papers shown

Title
From Task-Specific Models to Unified Systems: A Review of Model Merging Approaches Wei Ruan Tianze Yang Yue Zhou Tianming Liu Jin Lu MoMe 90 0 0 13 Mar 2025
Scalable Model Merging with Progressive Layer-wise Distillation Jing Xu Jiazheng Li Junzhe Zhang MoMe FedML 90 0 0 18 Feb 2025
On the Role of Attention Heads in Large Language Model Safety Zhenhong Zhou Haiyang Yu Xinghua Zhang Rongwu Xu Fei Huang Kun Wang Yang Liu Fan Zhang Yongbin Li 59 5 0 17 Oct 2024
What Matters for Model Merging at Scale? Prateek Yadav Tu Vu Jonathan Lai Alexandra Chronopoulou Manaal Faruqui Joey Tianyi Zhou Tsendsuren Munkhdalai MoMe 46 15 0 04 Oct 2024
Continual Training of Language Models for Few-Shot Learning Zixuan Ke Haowei Lin Yijia Shao Hu Xu Lei Shu Bin Liu KELM BDL CLL 87 34 0 11 Oct 2022
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 280 3,848 0 18 Apr 2021