BiLoRA: A Bi-level Optimization Framework for Overfitting-Resilient Low-Rank Adaptation of Large Pre-trained Models

19 March 2024

Papers citing "BiLoRA: A Bi-level Optimization Framework for Overfitting-Resilient Low-Rank Adaptation of Large Pre-trained Models"

23 / 23 papers shown

Title
LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Composition Chengsong Huang Qian Liu Bill Yuchen Lin Tianyu Pang Chao Du Min Lin MoMe 94 210 0 25 Jul 2023
DyLoRA: Parameter Efficient Tuning of Pre-trained Models using Dynamic Search-Free Low-Rank Adaptation Mojtaba Valipour Mehdi Rezagholizadeh I. Kobyzev A. Ghodsi 71 176 0 14 Oct 2022
UniPELT: A Unified Framework for Parameter-Efficient Language Model Tuning Yuning Mao Lambert Mathias Rui Hou Amjad Almahairi Hao Ma Jiawei Han Wen-tau Yih Madian Khabsa 58 190 0 14 Oct 2021
BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models Elad Ben-Zaken Shauli Ravfogel Yoav Goldberg 162 1,218 0 18 Jun 2021
Compacter: Efficient Low-Rank Hypercomplex Adapter Layers Rabeeh Karimi Mahabadi James Henderson Sebastian Ruder MoE 100 485 0 08 Jun 2021
Prefix-Tuning: Optimizing Continuous Prompts for Generation Xiang Lisa Li Percy Liang 223 4,254 0 01 Jan 2021
Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning Armen Aghajanyan Luke Zettlemoyer Sonal Gupta 90 562 1 22 Dec 2020
DeBERTa: Decoding-enhanced BERT with Disentangled Attention Pengcheng He Xiaodong Liu Jianfeng Gao Weizhu Chen AAML 138 2,731 0 05 Jun 2020
AdapterFusion: Non-Destructive Task Composition for Transfer Learning Jonas Pfeiffer Aishwarya Kamath Andreas Rucklé Kyunghyun Cho Iryna Gurevych CLL MoMe 129 849 0 01 May 2020
A Game Theoretic Framework for Model Based Reinforcement Learning Aravind Rajeswaran Igor Mordatch Vikash Kumar OffRL 49 127 0 16 Apr 2020
Exploring Versatile Generative Language Model Via Parameter-Efficient Transfer Learning Zhaojiang Lin Andrea Madotto Pascale Fung 94 160 0 08 Apr 2020
Pre-trained Models for Natural Language Processing: A Survey Xipeng Qiu Tianxiang Sun Yige Xu Yunfan Shao Ning Dai Xuanjing Huang LM&MA VLM 356 1,484 0 18 Mar 2020
Optimizing Millions of Hyperparameters by Implicit Differentiation Jonathan Lorraine Paul Vicol David Duvenaud DD 106 414 0 06 Nov 2019
Meta-Learning with Implicit Gradients Aravind Rajeswaran Chelsea Finn Sham Kakade Sergey Levine 102 855 0 10 Sep 2019
RoBERTa: A Robustly Optimized BERT Pretraining Approach Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy M. Lewis Luke Zettlemoyer Veselin Stoyanov AIMat 615 24,431 0 26 Jul 2019
DARTS: Differentiable Architecture Search Hanxiao Liu Karen Simonyan Yiming Yang 199 4,354 0 24 Jun 2018
Know What You Don't Know: Unanswerable Questions for SQuAD Pranav Rajpurkar Robin Jia Percy Liang RALM ELM 268 2,840 0 11 Jun 2018
Neural Network Acceptability Judgments Alex Warstadt Amanpreet Singh Samuel R. Bowman 230 1,407 0 31 May 2018
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 1.1K 7,154 0 20 Apr 2018
The E2E Dataset: New Challenges For End-to-End Generation Jekaterina Novikova Ondrej Dusek Verena Rieser 87 459 0 28 Jun 2017
A Review on Bilevel Optimization: From Classical to Evolutionary Approaches and Applications Ankur Sinha P. Malo Kalyanmoy Deb 46 755 0 17 May 2017
A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference Adina Williams Nikita Nangia Samuel R. Bowman 520 4,476 0 18 Apr 2017
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks Chelsea Finn Pieter Abbeel Sergey Levine OOD 823 11,899 0 09 Mar 2017