Knowledge Fusion of Large Language Models

19 January 2024

Wei Bi

Papers citing "Knowledge Fusion of Large Language Models"

50 / 57 papers shown

Title
A Survey on Collaborative Mechanisms Between Large and Small Language Models Yi Chen JiaHao Zhao HaoHao Han 38 0 0 12 May 2025
A Weighted Byzantine Fault Tolerance Consensus Driven Trusted Multiple Large Language Models Network Haoxiang Luo Gang Sun Yinqiu Liu Dongcheng Zhao Dusit Niyato Hongfang Yu Schahram Dustdar 48 0 0 08 May 2025
Towards Harnessing the Collaborative Power of Large and Small Models for Domain Tasks Yang Janet Liu Bingjie Yan Tianyuan Zou Jianqing Zhang Zixuan Gu ... J. Li Xiaozhou Ye Ye Ouyang Qiang Yang Y. Zhang ALM 149 1 0 24 Apr 2025
DMM: Building a Versatile Image Generation Model via Distillation-Based Model Merging Tianhui Song Weixin Feng Shuai Wang X. Li Tiezheng Ge Bo Zheng Limin Wang MoMe 62 0 0 16 Apr 2025
A Dual-Space Framework for General Knowledge Distillation of Large Language Models X. Zhang Songming Zhang Yunlong Liang Fandong Meng Yufeng Chen Jinan Xu Jie Zhou 19 0 0 15 Apr 2025
Leveraging Submodule Linearity Enhances Task Arithmetic Performance in LLMs Rui Dai Sile Hu Xu Shen Yonggang Zhang Xinmei Tian Jieping Ye MoMe 44 2 0 15 Apr 2025
FuseRL: Dense Preference Optimization for Heterogeneous Model Fusion Longguang Zhong Fanqi Wan Ziyi Yang Guosheng Liang Tianyuan Shi Xiaojun Quan MoMe 57 0 0 09 Apr 2025
LeForecast: Enterprise Hybrid Forecast by Time Series Intelligence Zheng Tan Yiwen Nie Wenfa Wu Guanyu Zhang Yanze Liu ... Chao Yang Jiaxuan Fan Yuan He Hongsheng Qi Yangzhou Du AI4TS 42 0 0 27 Mar 2025
Cross-Tokenizer Distillation via Approximate Likelihood Matching Benjamin Minixhofer Ivan Vulić E. Ponti 148 0 0 25 Mar 2025
Overcoming Vocabulary Mismatch: Vocabulary-agnostic Teacher Guided Language Modeling Haebin Shin Lei Ji Xiao Liu Yeyun Gong 52 0 0 24 Mar 2025
Collaborative Speculative Inference for Efficient LLM Inference Serving Luyao Gao Jianchun Liu Hongli Xu Xichong Zhang Yunming Liao Liusheng Huang 46 0 0 13 Mar 2025
Ensemble Learning for Large Language Models in Text and Code Generation: A Survey Mari Ashiga Wei Jie Fan Wu Vardan K. Voskanyan Fateme Dinmohammadi P. Brookes Jingzhi Gong Zheng Wang 44 0 0 13 Mar 2025
System 0/1/2/3: Quad-process theory for multi-timescale embodied collective cognitive systems Tadahiro Taniguchi Yasushi Hirai Masahiro Suzuki Shingo Murata Takato Horii Kazutoshi Tanaka AI4CE 61 0 0 08 Mar 2025
Rethinking Data: Towards Better Performing Domain-Specific Small Language Models Boris Nazarov Darya Frolova Yackov Lubarsky Alexei Gaissinski Pavel Kisilev ALM 65 1 0 03 Mar 2025
Scalable Model Merging with Progressive Layer-wise Distillation Jing Xu Jiazheng Li J. Zhang MoMe FedML 90 0 0 18 Feb 2025
Speculate, then Collaborate: Fusing Knowledge of Language Models during Decoding Z. Wang Muneeza Azmart Ang Li R. Horesh Mikhail Yurochkin 118 1 0 11 Feb 2025
TAID: Temporally Adaptive Interpolated Distillation for Efficient Knowledge Transfer in Language Models Makoto Shing Kou Misaki Han Bao Sho Yokoi Takuya Akiba VLM 57 1 0 28 Jan 2025
Multi-Task Model Merging via Adaptive Weight Disentanglement Feng Xiong Runxi Cheng Wang Chen Zhanqiu Zhang Yiwen Guo Chun Yuan Ruifeng Xu MoMe 99 4 0 10 Jan 2025
Large Language Monkeys: Scaling Inference Compute with Repeated Sampling Bradley Brown Jordan Juravsky Ryan Ehrlich Ronald Clark Quoc V. Le Christopher Ré Azalia Mirhoseini ALM LRM 81 220 0 03 Jan 2025
Copyright-Protected Language Generation via Adaptive Model Fusion Javier Abad Konstantin Donhauser Francesco Pinto Fanny Yang 74 1 0 09 Dec 2024
Enhancing Perception Capabilities of Multimodal LLMs with Training-Free Fusion Zhuokun Chen Jinwu Hu Zeshuai Deng Yufeng Wang Bohan Zhuang Mingkui Tan 71 0 0 02 Dec 2024
$H^3$ Fusion: Helpful, Harmless, Honest Fusion of Aligned LLMs Selim Furkan Tekin Fatih Ilhan Tiansheng Huang Sihao Hu Zachary Yahn Ling Liu MoMe 86 3 0 26 Nov 2024
Exploring Model Kinship for Merging Large Language Models Yedi Hu Yunzhi Yao N. Zhang Shumin Deng H. Chen MoMe 36 1 0 16 Oct 2024
Model Swarms: Collaborative Search to Adapt LLM Experts via Swarm Intelligence Shangbin Feng Zifeng Wang Yike Wang Sayna Ebrahimi Hamid Palangi ... Nathalie Rauschmayr Yejin Choi Yulia Tsvetkov Chen-Yu Lee Tomas Pfister MoMe 35 3 0 15 Oct 2024
Exact Byte-Level Probabilities from Tokenized Language Models for FIM-Tasks and Model Ensembles Buu Phan Brandon Amos Itai Gat Marton Havasi Matthew Muckley Karen Ullrich 47 1 0 11 Oct 2024
Model-GLUE: Democratized LLM Scaling for A Large Model Zoo in the Wild Xinyu Zhao Guoheng Sun Ruisi Cai Yukun Zhou Pingzhi Li ... Binhang Yuan Hongyi Wang Ang Li Zhangyang Wang Tianlong Chen MoMe ALM 28 3 0 07 Oct 2024
What Matters for Model Merging at Scale? Prateek Yadav Tu Vu Jonathan Lai Alexandra Chronopoulou Manaal Faruqui Mohit Bansal Tsendsuren Munkhdalai MoMe 46 13 0 04 Oct 2024
Parameter Competition Balancing for Model Merging Guodong Du Junlin Lee Jing Li Runhua Jiang Yifei Guo ... Hanting Liu S. Goh Ho-Kin Tang Daojing He Min Zhang MoMe 35 12 0 03 Oct 2024
Determine-Then-Ensemble: Necessity of Top-k Union for Large Language Model Ensembling Yuxuan Yao Han Wu Mingyang Liu Sichun Luo Xiongwei Han Jie Liu Zhijiang Guo Linqi Song 58 4 0 03 Oct 2024
Disentangling Latent Shifts of In-Context Learning Through Self-Training Josip Jukić Jan Snajder 21 0 0 02 Oct 2024
SMILE: Zero-Shot Sparse Mixture of Low-Rank Experts Construction From Pre-Trained Foundation Models A. Tang Li Shen Yong Luo Shuai Xie Han Hu Lefei Zhang Bo Du Dacheng Tao MoMe 36 4 0 19 Aug 2024
FuseChat: Knowledge Fusion of Chat Models Fanqi Wan Longguang Zhong Ziyi Yang Ruijun Chen Xiaojun Quan ALM KELM MoMe 32 23 0 15 Aug 2024
Computer Audition: From Task-Specific Machine Learning to Foundation Models Andreas Triantafyllopoulos Iosif Tsangko Alexander Gebhard A. Mesaros Tuomas Virtanen Björn Schuller 45 4 0 22 Jul 2024
Recent Advances in Generative AI and Large Language Models: Current Status, Challenges, and Perspectives D. Hagos Rick Battle Danda B. Rawat LM&MA OffRL 28 22 0 20 Jul 2024
OmniBind: Large-scale Omni Multimodal Representation via Binding Spaces Zehan Wang Ziang Zhang Hang Zhang Luping Liu Rongjie Huang Xize Cheng Hengshuang Zhao Zhou Zhao 43 9 0 16 Jul 2024
Aligning Teacher with Student Preferences for Tailored Training Data Generation Yantao Liu Zhao Zhang Zijun Yao S. Cao Lei Hou Juanzi Li 49 1 0 27 Jun 2024
WARP: On the Benefits of Weight Averaged Rewarded Policies Alexandre Ramé Johan Ferret Nino Vieillard Robert Dadashi Léonard Hussenot Pierre-Louis Cedoz Pier Giuseppe Sessa Sertan Girgin Arthur Douillard Olivier Bachem 56 14 0 24 Jun 2024
FuseGen: PLM Fusion for Data-generation based Zero-shot Learning Tianyuan Zou Yang Janet Liu Peng Li Jianqing Zhang Jingjing Liu Ya-Qin Zhang 31 3 0 18 Jun 2024
Reversing the Forget-Retain Objectives: An Efficient LLM Unlearning Framework from Logit Difference Jiabao Ji Yujian Liu Yang Zhang Gaowen Liu Ramana Rao Kompella Sijia Liu Shiyu Chang KELM MU 29 22 0 12 Jun 2024
FusionBench: A Comprehensive Benchmark of Deep Model Fusion A. Tang Li Shen Yong Luo Han Hu Bo Du Dacheng Tao ELM MoMe VLM 41 21 0 05 Jun 2024
FedMKT: Federated Mutual Knowledge Transfer for Large and Small Language Models Tao Fan Guoqiang Ma Yan Kang Hanlin Gu Yuanfeng Song Lixin Fan Kai Chen Qiang Yang 26 9 0 04 Jun 2024
A Survey on Self-Evolution of Large Language Models Zhengwei Tao Ting-En Lin Xiancai Chen Hangyu Li Yuchuan Wu Yongbin Li Zhi Jin Fei Huang Dacheng Tao Jingren Zhou LRM LM&Ro 54 22 0 22 Apr 2024
Ensemble Learning for Heterogeneous Large Language Models with Deep Parallel Collaboration Yi-Chong Huang Xiaocheng Feng Baohang Li Yang Xiang Hui Wang Bing Qin Ting Liu FedML 41 22 0 19 Apr 2024
Pack of LLMs: Model Fusion at Test-Time via Perplexity Optimization Costas Mavromatis Petros Karypis George Karypis MoMe 29 24 0 17 Apr 2024
Distilling Reasoning Ability from Large Language Models with Adaptive Thinking Xiao Chen Sihang Zhou K. Liang Xinwang Liu ReLM LRM 37 2 0 14 Apr 2024
Rho-1: Not All Tokens Are What You Need Zheng-Wen Lin Zhibin Gou Yeyun Gong Xiao Liu Yelong Shen ... Chen Lin Yujiu Yang Jian Jiao Nan Duan Weizhu Chen CLL 50 55 0 11 Apr 2024
Mastering Text, Code and Math Simultaneously via Fusing Highly Specialized Language Models Ning Ding Yulin Chen Ganqu Cui Xingtai Lv Weilin Zhao Ruobing Xie Bowen Zhou Zhiyuan Liu Maosong Sun ALM MoMe AI4CE 38 7 0 13 Mar 2024
Knowledge Fusion of Chat LLMs: A Preliminary Technical Report Fanqi Wan Ziyi Yang Longguang Zhong Xiaojun Quan Xinting Huang Wei Bi MoMe 27 1 0 25 Feb 2024
A Survey on Knowledge Distillation of Large Language Models Xiaohan Xu Ming Li Chongyang Tao Tao Shen Reynold Cheng Jinyang Li Can Xu Dacheng Tao Tianyi Zhou KELM VLM 44 101 0 20 Feb 2024
Large Language Models: A Survey Shervin Minaee Tomáš Mikolov Narjes Nikzad M. Asgari-Chenaghlu R. Socher Xavier Amatriain Jianfeng Gao ALM LM&MA ELM 125 369 0 09 Feb 2024