Preserving Pre-trained Features Helps Calibrate Fine-tuned Language Models

30 May 2023

Jianfei Chen

Jun Zhu

Papers citing "Preserving Pre-trained Features Helps Calibrate Fine-tuned Language Models"

22 / 22 papers shown

Title
Comparing Uncertainty Measurement and Mitigation Methods for Large Language Models: A Systematic Review Toghrul Abbasli Kentaroh Toyoda Yuan Wang Leon Witt Muhammad Asif Ali Yukai Miao Dan Li Qingsong Wei UQCV 92 0 0 25 Apr 2025
ResMoE: Space-efficient Compression of Mixture of Experts LLMs via Residual Restoration Mengting Ai Tianxin Wei Yifan Chen Zhichen Zeng Ritchie Zhao G. Varatkar B. Rouhani Xianfeng Tang Hanghang Tong Jingrui He MoE 51 1 0 10 Mar 2025
Uncertainty-Aware Adaptation of Large Language Models for Protein-Protein Interaction Analysis Sanket R. Jantre Tianle Wang Gilchan Park Kriti Chopra Nicholas Jeon Xiaoning Qian Nathan M. Urban Byung-Jun Yoon 62 0 0 10 Feb 2025
DiTASK: Multi-Task Fine-Tuning with Diffeomorphic Transformations Krishna Sri Ipsit Mantri Carola-Bibiane Schönlieb Bruno Ribeiro Chaim Baskin Moshe Eliasof 43 0 0 09 Feb 2025
The Best Instruction-Tuning Data are Those That Fit Dylan Zhang Qirun Dai Hao Peng ALM 117 3 0 06 Feb 2025
CleaR: Towards Robust and Generalized Parameter-Efficient Fine-Tuning for Noisy Label Learning Yeachan Kim Junho Kim SangKeun Lee NoLa AAML 35 2 0 31 Oct 2024
Functional-level Uncertainty Quantification for Calibrated Fine-tuning on LLMs Ruijia Niu D. Wu Rose Yu Yi Ma 33 1 0 09 Oct 2024
CONTESTS: a Framework for Consistency Testing of Span Probabilities in Language Models Eitan Wagner Yuli Slavutsky Omri Abend 18 1 0 30 Sep 2024
Correcting Negative Bias in Large Language Models through Negative Attention Score Alignment Sangwon Yu Jongyoon Song Bongkyu Hwang Hoyoung Kang Sooah Cho Junhwa Choi Seongho Joe Taehee Lee Youngjune Gwon Sungroh Yoon 117 4 0 31 Jul 2024
Instruction Tuning With Loss Over Instructions Zhengyan Shi Adam X. Yang Bin Wu Laurence Aitchison Emine Yilmaz Aldo Lipani ALM 24 20 0 23 May 2024
Gaussian Stochastic Weight Averaging for Bayesian Low-Rank Adaptation of Large Language Models Emre Onal Klemens Flöge Emma Caldwell A. Sheverdin Vincent Fortuin UQCV BDL 45 9 0 06 May 2024
LoRA Dropout as a Sparsity Regularizer for Overfitting Control Yang Lin Xinyu Ma Xu Chu Yujie Jin Zhibang Yang Yasha Wang Hong-yan Mei 52 19 0 15 Apr 2024
Predict the Next Word: Humans exhibit uncertainty in this task and language models _____ Evgenia Ilia Wilker Aziz 29 2 0 27 Feb 2024
Uncertainty quantification in fine-tuned LLMs using LoRA ensembles Oleksandr Balabanov H. Linander UQCV 36 13 0 19 Feb 2024
On the Calibration of Large Language Models and Alignment Chiwei Zhu Benfeng Xu Quan Wang Yongdong Zhang Zhendong Mao 74 32 0 22 Nov 2023
LoRA ensembles for large language model fine-tuning Xi Wang Laurence Aitchison Maja Rudolph UQCV 34 34 0 29 Sep 2023
CATfOOD: Counterfactual Augmented Training for Improving Out-of-Domain Performance and Calibration Rachneet Sachdeva Martin Tutek Iryna Gurevych OODD 32 10 0 14 Sep 2023
Fine-tuning can cripple your foundation model; preserving features may be the solution Jishnu Mukhoti Y. Gal Philip H. S. Torr P. Dokania CLL 37 31 0 25 Aug 2023
Bayesian Low-rank Adaptation for Large Language Models Adam X. Yang Maxime Robeyns Xi Wang Laurence Aitchison AI4CE BDL 18 45 0 24 Aug 2023
Bayesian Attention Modules Xinjie Fan Shujian Zhang Bo Chen Mingyuan Zhou 117 59 0 20 Oct 2020
Calibration of Pre-trained Transformers Shrey Desai Greg Durrett UQLM 243 289 0 17 Mar 2020
Mixout: Effective Regularization to Finetune Large-scale Pretrained Language Models Cheolhyoung Lee Kyunghyun Cho Wanmo Kang MoE 249 205 0 25 Sep 2019