Efficient Ensemble for Fine-tuning Language Models on Multiple Datasets

28 May 2025

Papers citing "Efficient Ensemble for Fine-tuning Language Models on Multiple Datasets"

29 / 29 papers shown

Title
Scalable Fine-tuning from Multiple Data Sources: A First-Order Approximation Approach Dongyue Li Ziniu Zhang Lu Wang Hongyang R. Zhang 79 1 0 28 Sep 2024
Scalable Multitask Learning Using Gradient-based Estimation of Task Affinity Dongyue Li Aneesh Sharma Hongyang R. Zhang 113 3 0 09 Sep 2024
Interactive Machine Teaching by Labeling Rules and Instances Giannis Karamanolakis Daniel J. Hsu Luis Gravano 60 1 0 08 Sep 2024
CogBench: a large language model walks into a psychology lab Julian Coda-Forno Marcel Binz Jane X. Wang Eric Schulz ELM ALM LLMAG LM&MA 113 38 0 28 Feb 2024
LESS: Selecting Influential Data for Targeted Instruction Tuning Mengzhou Xia Sadhika Malladi Suchin Gururangan Sanjeev Arora Danqi Chen 148 242 0 06 Feb 2024
LQ-LoRA: Low-rank Plus Quantized Matrix Decomposition for Efficient Language Model Finetuning Han Guo P. Greengard Eric P. Xing Yoon Kim MQ 96 57 0 20 Nov 2023
QLoRA: Efficient Finetuning of Quantized LLMs Tim Dettmers Artidoro Pagnoni Ari Holtzman Luke Zettlemoyer ALM 154 2,602 0 23 May 2023
TRAK: Attributing Model Behavior at Scale Sung Min Park Kristian Georgiev Andrew Ilyas Guillaume Leclerc Aleksander Madry TDI 102 156 0 24 Mar 2023
Parameter-Efficient Fine-Tuning Design Spaces Jiaao Chen Aston Zhang Xingjian Shi Mu Li Alexander J. Smola Diyi Yang 110 67 0 04 Jan 2023
Robust Fine-Tuning of Deep Neural Networks with Hessian-based Generalization Guarantees Haotian Ju Dongyue Li Hongyang R. Zhang 88 30 0 06 Jun 2022
Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time Mitchell Wortsman Gabriel Ilharco S. Gadre Rebecca Roelofs Raphael Gontijo-Lopes ... Hongseok Namkoong Ali Farhadi Y. Carmon Simon Kornblith Ludwig Schmidt MoMe 155 1,009 1 10 Mar 2022
Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey Bonan Min Hayley L Ross Elior Sulem Amir Pouran Ben Veyseh Thien Huu Nguyen Oscar Sainz Eneko Agirre Ilana Heinz Dan Roth LM&MA VLM AI4CE 162 1,081 0 01 Nov 2021
Efficiently Identifying Task Groupings for Multi-Task Learning Christopher Fifty Ehsan Amid Zhe Zhao Tianhe Yu Rohan Anil Chelsea Finn 290 255 1 10 Sep 2021
LoRA: Low-Rank Adaptation of Large Language Models J. E. Hu Yelong Shen Phillip Wallis Zeyuan Allen-Zhu Yuanzhi Li Shean Wang Lu Wang Weizhu Chen OffRL AI4TS AI4CE ALM AIMat 490 10,496 0 17 Jun 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 584 4,084 0 18 Apr 2021
Multi-Task Reinforcement Learning with Context-based Representations Shagun Sodhani Amy Zhang Joelle Pineau 84 191 0 11 Feb 2021
Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity W. Fedus Barret Zoph Noam M. Shazeer MoE 88 2,220 0 11 Jan 2021
Prefix-Tuning: Optimizing Continuous Prompts for Generation Xiang Lisa Li Percy Liang 250 4,299 0 01 Jan 2021
WILDS: A Benchmark of in-the-Wild Distribution Shifts Pang Wei Koh Shiori Sagawa Henrik Marklund Sang Michael Xie Marvin Zhang ... A. Kundaje Emma Pierson Sergey Levine Chelsea Finn Percy Liang OOD 225 1,445 0 14 Dec 2020
Precise High-Dimensional Asymptotics for Quantifying Heterogeneous Transfers Fan Yang Hongyang R. Zhang Sen Wu Christopher Ré Weijie J. Su 102 11 0 22 Oct 2020
Measuring Massive Multitask Language Understanding Dan Hendrycks Collin Burns Steven Basart Andy Zou Mantas Mazeika Basel Alomair Jacob Steinhardt ELM RALM 184 4,553 0 07 Sep 2020
Understanding and Improving Information Transfer in Multi-Task Learning Sen Wu Hongyang R. Zhang Christopher Ré 42 158 0 02 May 2020
AdapterFusion: Non-Destructive Task Composition for Transfer Learning Jonas Pfeiffer Aishwarya Kamath Andreas Rucklé Kyunghyun Cho Iryna Gurevych CLL MoMe 145 858 0 01 May 2020
SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems Alex Jinpeng Wang Yada Pruksachatkun Nikita Nangia Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 274 2,323 0 02 May 2019
Parameter-Efficient Transfer Learning for NLP N. Houlsby A. Giurgiu Stanislaw Jastrzebski Bruna Morrone Quentin de Laroussilhe Andrea Gesmundo Mona Attariyan Sylvain Gelly 221 4,514 0 02 Feb 2019
Multi-Task Deep Neural Networks for Natural Language Understanding Xiaodong Liu Pengcheng He Weizhu Chen Jianfeng Gao AI4CE 141 1,273 0 31 Jan 2019
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 1.1K 7,196 0 20 Apr 2018
A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networks Behnam Neyshabur Srinadh Bhojanapalli Nathan Srebro 83 610 0 29 Jul 2017
Understanding Black-box Predictions via Influence Functions Pang Wei Koh Percy Liang TDI 216 2,905 0 14 Mar 2017