LoRA ensembles for large language model fine-tuning

29 September 2023

Papers citing "LoRA ensembles for large language model fine-tuning"

31 / 31 papers shown

Title
Parallel Scaling Law for Language Models Mouxiang Chen Binyuan Hui Zeyu Cui Jiaxi Yang Dayiheng Liu Jianling Sun Junyang Lin Zhongxin Liu MoE LRM 37 0 0 15 May 2025
Beyond Negation Detection: Comprehensive Assertion Detection Models for Clinical NLP Veysel Kocaman Yigit Gul M. Aytug Kaya H. Haq Mehmet Butgul Cabir Celik D. Talby MedIm 39 0 0 21 Mar 2025
Investigating Human-Aligned Large Language Model Uncertainty Kyle Moore Jesse Roberts Daryl Watson Pamela Wisniewski 58 0 0 16 Mar 2025
Uncertainty-Aware Adaptation of Large Language Models for Protein-Protein Interaction Analysis Sanket R. Jantre Tianle Wang Gilchan Park Kriti Chopra Nicholas Jeon Xiaoning Qian Nathan M. Urban Byung-Jun Yoon 62 0 0 10 Feb 2025
Ensembles of Low-Rank Expert Adapters Yinghao Li Vianne Gao Chao Zhang MohamadAli Torkamani 67 0 0 31 Jan 2025
BLoB: Bayesian Low-Rank Adaptation by Backpropagation for Large Language Models Yibin Wang Haizhou Shi Ligong Han Dimitris N. Metaxas Hao Wang BDL UQLM 116 6 0 28 Jan 2025
Optimizing Large Language Models with an Enhanced LoRA Fine-Tuning Algorithm for Efficiency and Robustness in NLP Tasks Jiacheng Hu Xiaoxuan Liao Jia Gao Zhen Qi Hongye Zheng Chihang Wang 33 1 0 25 Dec 2024
Training-Free Bayesianization for Low-Rank Adapters of Large Language Models Haizhou Shi Yibin Wang Ligong Han H. M. Zhang Hao Wang UQCV 83 0 0 07 Dec 2024
Variational Low-Rank Adaptation Using IVON Bai Cong Nico Daheim Yuesong Shen Daniel Cremers Rio Yokota Mohammad Emtiyaz Khan Thomas Möllenhoff 41 3 0 07 Nov 2024
Functional-level Uncertainty Quantification for Calibrated Fine-tuning on LLMs Ruijia Niu D. Wu Rose Yu Yi Ma 33 1 0 09 Oct 2024
Hyperbolic Fine-tuning for Large Language Models Menglin Yang Aosong Feng Bo Xiong Jihong Liu Irwin King Rex Ying 23 1 0 05 Oct 2024
Hallucination Detection in LLMs: Fast and Memory-Efficient Finetuned Models Gabriel Y. Arteaga Thomas B. Schon Nicolas Pielawski 35 7 0 04 Sep 2024
CoRA: Optimizing Low-Rank Adaptation with Common Subspace of Large Language Models Xiaojun Xiao Sen Shen Qiming Bao Hongfei Rong Kairui Liu Zhongsheng Wang J. Liu 36 2 0 31 Aug 2024
A Survey on LoRA of Large Language Models Yuren Mao Yuhang Ge Yijiang Fan Wenyi Xu Yu Mi Zhonghao Hu Yunjun Gao ALM 54 24 0 08 Jul 2024
Expressive and Generalizable Low-rank Adaptation for Large Models via Slow Cascaded Learning Siwei Li Yifan Yang Yifei Shen Fangyun Wei Zongqing Lu L. Qiu Yuqing Yang AI4CE 43 1 0 01 Jul 2024
LoRA-Ensemble: Efficient Uncertainty Modelling for Self-attention Networks Michelle Halbheer Dominik J. Mühlematter Alexander Becker Dominik Narnhofer Helge Aasen Konrad Schindler Mehmet Özgür Türkoglu UQCV 35 1 0 23 May 2024
Just rephrase it! Uncertainty estimation in closed-source language models via multiple rephrased queries Adam Yang Chen Chen Konstantinos Pitas 32 7 0 22 May 2024
Gaussian Stochastic Weight Averaging for Bayesian Low-Rank Adaptation of Large Language Models Emre Onal Klemens Flöge Emma Caldwell A. Sheverdin Vincent Fortuin UQCV BDL 45 9 0 06 May 2024
STAR: Constraint LoRA with Dynamic Active Learning for Data-Efficient Fine-Tuning of Large Language Models Linhai Zhang Jialong Wu Deyu Zhou Guoqiang Xu 30 4 0 02 Mar 2024
LoRA-SP: Streamlined Partial Parameter Adaptation for Resource-Efficient Fine-Tuning of Large Language Models Yichao Wu Yafei Xiang Shuning Huo Yulu Gong Penghao Liang 25 7 0 28 Feb 2024
On the Challenges and Opportunities in Generative AI Laura Manduchi Kushagra Pandey Robert Bamler Ryan Cotterell Sina Daubener ... F. Wenzel Frank Wood Stephan Mandt Vincent Fortuin Vincent Fortuin 56 17 0 28 Feb 2024
Bayesian Reward Models for LLM Alignment Adam X. Yang Maxime Robeyns Thomas Coste Zhengyan Shi Jun Wang Haitham Bou-Ammar Laurence Aitchison 34 17 0 20 Feb 2024
Uncertainty quantification in fine-tuned LLMs using LoRA ensembles Oleksandr Balabanov H. Linander UQCV 33 13 0 19 Feb 2024
Improving Reinforcement Learning from Human Feedback with Efficient Reward Model Ensemble Shun Zhang Zhenfang Chen Sunli Chen Yikang Shen Zhiqing Sun Chuang Gan 23 26 0 30 Jan 2024
Uncertainty-Penalized Reinforcement Learning from Human Feedback with Diverse Reward LoRA Ensembles Yuanzhao Zhai Han Zhang Yu Lei Yue Yu Kele Xu Dawei Feng Bo Ding Huaimin Wang AI4CE 72 32 0 30 Dec 2023
On Task Performance and Model Calibration with Supervised and Self-Ensembled In-Context Learning Chengzu Li Han Zhou Goran Glavavs Anna Korhonen Ivan Vulić 21 11 0 21 Dec 2023
Bayesian Low-rank Adaptation for Large Language Models Adam X. Yang Maxime Robeyns Xi Wang Laurence Aitchison AI4CE BDL 18 44 0 24 Aug 2023
Large Language Models are Zero-Shot Reasoners Takeshi Kojima S. Gu Machel Reid Yutaka Matsuo Yusuke Iwasawa ReLM LRM 325 4,077 0 24 May 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 313 11,953 0 04 Mar 2022
Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles Balaji Lakshminarayanan Alexander Pritzel Charles Blundell UQCV BDL 276 5,661 0 05 Dec 2016
Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning Y. Gal Zoubin Ghahramani UQCV BDL 285 9,138 0 06 Jun 2015