Seeking Neural Nuggets: Knowledge Transfer in Large Language Models from
a Parametric Perspective

Seeking Neural Nuggets: Knowledge Transfer in Large Language Models from a Parametric Perspective

17 October 2023

Papers citing "Seeking Neural Nuggets: Knowledge Transfer in Large Language Models from a Parametric Perspective"

16 / 16 papers shown

Title
Towards Harnessing the Collaborative Power of Large and Small Models for Domain Tasks Yang Liu Bingjie Yan Tianyuan Zou Jianqing Zhang Zixuan Gu ... J. Li Xiaozhou Ye Ye Ouyang Qiang Yang Yuhang Zhang ALM 155 1 0 24 Apr 2025
APEX $^2$ : Adaptive and Extreme Summarization for Personalized Knowledge Graphs Zihao Li Dongqi Fu Mengting Ai Jingrui He 47 4 0 23 Dec 2024
Beyond Task Vectors: Selective Task Arithmetic Based on Importance Metrics Tian Bowen Lai Songning Wu Jiemin Shuai Zhihao Ge Shiming Yue Yutao MoMe 70 4 0 25 Nov 2024
Scaling Smart: Accelerating Large Language Model Pre-training with Small Model Initialization Mohammad Samragh Iman Mirzadeh Keivan Alizadeh Vahid Fartash Faghri Minsik Cho Moin Nabi Devang Naik Mehrdad Farajtabar LRM AI4CE 27 6 0 19 Sep 2024
Unveiling Factual Recall Behaviors of Large Language Models through Knowledge Neurons Yifei Wang Yuheng Chen Wanting Wen Yu Sheng Linjing Li D. Zeng KELM 39 5 0 06 Aug 2024
Multi-LoRA Composition for Image Generation Ming Zhong Yelong Shen Shuohang Wang Yadong Lu Yizhu Jiao Siru Ouyang Donghan Yu Jiawei Han Weizhu Chen MoMe 43 37 0 26 Feb 2024
A Comprehensive Study of Knowledge Editing for Large Language Models Ningyu Zhang Yunzhi Yao Bo Tian Peng Wang Shumin Deng ... Lei Liang Qing Cui Xiao-Jun Zhu Jun Zhou Huajun Chen KELM 47 76 0 02 Jan 2024
LaMini-LM: A Diverse Herd of Distilled Models from Large-Scale Instructions Minghao Wu Abdul Waheed Chiyu Zhang Muhammad Abdul-Mageed Alham Fikri Aji ALM 132 119 0 27 Apr 2023
Instruction Tuning with GPT-4 Baolin Peng Chunyuan Li Pengcheng He Michel Galley Jianfeng Gao SyDa ALM LM&MA 159 579 0 06 Apr 2023
Git Re-Basin: Merging Models modulo Permutation Symmetries Samuel K. Ainsworth J. Hayase S. Srinivasa MoMe 255 314 0 11 Sep 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 370 8,495 0 28 Jan 2022
Learning Student-Friendly Teacher Networks for Knowledge Distillation D. Park Moonsu Cha C. Jeong Daesin Kim Bohyung Han 121 100 0 12 Feb 2021
On the Transformer Growth for Progressive BERT Training Xiaotao Gu Liyuan Liu Hongkun Yu Jing Li Cheng Chen Jiawei Han VLM 69 51 0 23 Oct 2020
Meta Pseudo Labels Hieu H. Pham Zihang Dai Qizhe Xie Minh-Thang Luong Quoc V. Le VLM 256 656 0 23 Mar 2020
Language Models as Knowledge Bases? Fabio Petroni Tim Rocktaschel Patrick Lewis A. Bakhtin Yuxiang Wu Alexander H. Miller Sebastian Riedel KELM AI4MH 415 2,588 0 03 Sep 2019
Knowledge Distillation by On-the-Fly Native Ensemble Xu Lan Xiatian Zhu S. Gong 195 473 0 12 Jun 2018