Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning

11 May 2022

Papers citing "Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning"

50 / 162 papers shown

Title
The Impact of Initialization on LoRA Finetuning Dynamics Soufiane Hayou Nikhil Ghosh Bin Yu AI4CE 36 13 0 12 Jun 2024
More Than Catastrophic Forgetting: Integrating General Capabilities For Domain-Specific LLMs Chengyuan Liu Shihang Wang Yangyang Kang Lizhi Qing Fubang Zhao Changlong Sun Kun Kuang Fei Wu ELM AILaw CLL 27 6 0 28 May 2024
$$\textit{Trans-LoRA}$: towards data-free Transferable Parameter Efficient Finetuning$ $\textit{Trans-LoRA}$ : towards data-free Transferable Parameter Efficient Finetuning Runqian Wang Soumya Ghosh David D. Cox Diego Antognini Aude Oliva Rogerio Feris Leonid Karlinsky 40 1 0 27 May 2024
On Fairness of Low-Rank Adaptation of Large Models Zhoujie Ding Ken Ziyu Liu Pura Peetathawatchai Berivan Isik Sanmi Koyejo 48 4 0 27 May 2024
Rényi Neural Processes Xuesong Wang He Zhao Edwin V. Bonilla UQCV BDL 32 0 0 25 May 2024
Implicit In-context Learning Zhuowei Li Zihao Xu Ligong Han Yunhe Gao Song Wen Di Liu Hao Wang Dimitris N. Metaxas 38 1 0 23 May 2024
Towards Modular LLMs by Building and Reusing a Library of LoRAs O. Ostapenko Zhan Su E. Ponti Laurent Charlin Nicolas Le Roux Matheus Pereira Lucas Caccia Alessandro Sordoni MoMe 44 31 0 18 May 2024
Contextual Emotion Recognition using Large Vision Language Models Yasaman Etesam Özge Nilay Yalçin Chuxuan Zhang Angelica Lim VLM 73 3 0 14 May 2024
CANAL -- Cyber Activity News Alerting Language Model: Empirical Approach vs. Expensive LLM Urjitkumar Patel Fang-Chun Yeh Chinmay Gondhalekar 29 3 0 10 May 2024
In-Context Learning with Long-Context Models: An In-Depth Exploration Amanda Bertsch Maor Ivgi Uri Alon Jonathan Berant Matthew R. Gormley Matthew R. Gormley Graham Neubig ReLM AIMat 93 64 0 30 Apr 2024
LLMParser: An Exploratory Study on Using Large Language Models for Log Parsing Zeyang Ma A. Chen Dong Jae Kim Tse-Husn Chen Shaowei Wang 35 45 0 27 Apr 2024
Evaluation of Few-Shot Learning for Classification Tasks in the Polish Language Tsimur Hadeliya D. Kajtoch 46 0 0 27 Apr 2024
Boosting Architectural Generation via Prompts: Report Xin Zhang Wenwen Liu AI4CE 35 1 0 24 Apr 2024
Leveraging Large Language Models for Multimodal Search Oriol Barbany Michael Huang Xinliang Zhu Arnab Dhua 31 9 0 24 Apr 2024
ColA: Collaborative Adaptation with Gradient Learning Enmao Diao Qi Le Suya Wu Xinran Wang Ali Anwar Jie Ding Vahid Tarokh 35 1 0 22 Apr 2024
MM-PhyRLHF: Reinforcement Learning Framework for Multimodal Physics Question-Answering Avinash Anand Janak Kapuriya Chhavi Kirtani Apoorv Singh Jay Saraf Naman Lal Jatin Kumar A. Shivam Astha Verma R. Shah OffRL 51 9 0 19 Apr 2024
Navigating the Landscape of Large Language Models: A Comprehensive Review and Analysis of Paradigms and Fine-Tuning Strategies Benjue Weng LM&MA 46 8 0 13 Apr 2024
AnchorAL: Computationally Efficient Active Learning for Large and Imbalanced Datasets Pietro Lesci Andreas Vlachos 35 2 0 08 Apr 2024
Unlocking Parameter-Efficient Fine-Tuning for Low-Resource Language Translation Tong Su Xin Peng Sarubi Thillainathan David Guzmán Surangika Ranathunga En-Shiun Annie Lee 43 2 0 05 Apr 2024
Parameter Efficient Quasi-Orthogonal Fine-Tuning via Givens Rotation Xinyu Ma Xu Chu Zhibang Yang Yang Lin Xin Gao Junfeng Zhao 46 7 0 05 Apr 2024
ILLUMINER: Instruction-tuned Large Language Models as Few-shot Intent Classifier and Slot Filler Paramita Mirza Viju Sudhi S. Sahoo Sinchana Ramakanth Bhat 25 4 0 26 Mar 2024
AutoLoRA: Automatically Tuning Matrix Ranks in Low-Rank Adaptation Based on Meta Learning Ruiyi Zhang Rushi Qiang Sai Ashish Somayajula Pengtao Xie 42 13 0 14 Mar 2024
Block-wise LoRA: Revisiting Fine-grained LoRA for Effective Personalization and Stylization in Text-to-Image Generation Likun Li Haoqi Zeng Changpeng Yang Haozhe Jia Di Xu DiffM 34 4 0 12 Mar 2024
Can LLMs' Tuning Methods Work in Medical Multimodal Domain? Jiawei Chen Yue Jiang Dingkang Yang Mingcheng Li Jinjie Wei Ziyun Qian Lihua Zhang LM&MA 27 9 0 11 Mar 2024
Tracking Meets LoRA: Faster Training, Larger Model, Stronger Performance Liting Lin Heng Fan Zhipeng Zhang Yaowei Wang Yong-mei Xu Haibin Ling 52 26 0 08 Mar 2024
Dual-path Frequency Discriminators for Few-shot Anomaly Detection Yuhu Bai Jiangning Zhang Yuhang Dong Guanzhong Tian Liang Liu Yunkang Cao Yabiao Wang Chengjie Wang 44 2 0 07 Mar 2024
FlexLLM: A System for Co-Serving Large Language Model Inference and Parameter-Efficient Finetuning Xupeng Miao Gabriele Oliaro Xinhao Cheng Vineeth Kada Ruohan Gao ... April Yang Yingcheng Wang Mengdi Wu Colin Unger Zhihao Jia MoE 94 9 0 29 Feb 2024
Large Language Models(LLMs) on Tabular Data: Prediction, Generation, and Understanding -- A Survey Xi Fang Weijie Xu Fiona Anting Tan Jiani Zhang Ziqing Hu Yanjun Qi Scott Nickleach Diego Socolinsky Srinivasan H. Sengamedu Christos Faloutsos LMTD ALM 50 66 0 27 Feb 2024
LoRA+: Efficient Low Rank Adaptation of Large Models Soufiane Hayou Nikhil Ghosh Bin Yu AI4CE 46 148 0 19 Feb 2024
How Useful is Continued Pre-Training for Generative Unsupervised Domain Adaptation? Rheeya Uppaal Yixuan Li Junjie Hu 37 4 0 31 Jan 2024
Low-Resource Vision Challenges for Foundation Models Yunhua Zhang Hazel Doughty Cees G. M. Snoek VLM 30 5 0 09 Jan 2024
A Comprehensive Evaluation of Parameter-Efficient Fine-Tuning on Software Engineering Tasks Wentao Zou Qi Li Jidong Ge Chuanyi Li Xiaoyu Shen LiGuo Huang Bin Luo 32 5 0 25 Dec 2023
Towards Better Serialization of Tabular Data for Few-shot Classification with Large Language Models Sukriti Jaitly Tanay Shah Ashish Shugani Razik Singh Grewal LMTD 18 5 0 18 Dec 2023
Traffic Signal Control Using Lightweight Transformers: An Offline-to-Online RL Approach Xingshuai Huang Di Wu Benoit Boulet OffRL 27 2 0 12 Dec 2023
Diversified in-domain synthesis with efficient fine-tuning for few-shot classification Victor G. Turrisi da Costa Nicola Dall’Asen Yiming Wang N. Sebe Elisa Ricci 46 3 0 05 Dec 2023
A Rank Stabilization Scaling Factor for Fine-Tuning with LoRA Damjan Kalajdzievski ALM 33 80 0 28 Nov 2023
PrivateLoRA For Efficient Privacy Preserving LLM Yiming Wang Yu Lin Xiaodong Zeng Guannan Zhang 62 11 0 23 Nov 2023
PEFT-MedAware: Large Language Model for Medical Awareness Keivalya Pandya MedIm AI4MH LM&MA 16 0 0 17 Nov 2023
Modality-invariant and Specific Prompting for Multimodal Human Perception Understanding Hao Sun Ziwei Niu Xinyao Yu Jiaqing Liu Yen-Wei Chen Lanfen Lin 29 0 0 17 Nov 2023
Tied-Lora: Enhancing parameter efficiency of LoRA with weight tying Adithya Renduchintala Tugrul Konuk Oleksii Kuchaiev MoMe 38 42 0 16 Nov 2023
SiRA: Sparse Mixture of Low Rank Adaptation Yun Zhu Nevan Wichers Chu-Cheng Lin Xinyi Wang Tianlong Chen ... Han Lu Canoee Liu Liangchen Luo Jindong Chen Lei Meng MoE 30 27 0 15 Nov 2023
When does In-context Learning Fall Short and Why? A Study on Specification-Heavy Tasks Hao Peng Xiaozhi Wang Jianhui Chen Weikai Li Y. Qi ... Zhili Wu Kaisheng Zeng Bin Xu Lei Hou Juanzi Li 34 28 0 15 Nov 2023
AdaSent: Efficient Domain-Adapted Sentence Embeddings for Few-Shot Classification Yongxin Huang Kexin Wang Sourav Dutta Raj Nath Patel Goran Glavas Iryna Gurevych VLM 20 4 0 01 Nov 2023
FedPEAT: Convergence of Federated Learning, Parameter-Efficient Fine Tuning, and Emulator Assisted Tuning for Artificial Intelligence Foundation Models with Mobile Edge Computing Terence Jie Chua Wen-li Yu Junfeng Zhao Kwok-Yan Lam FedML 32 5 0 26 Oct 2023
Improving Few-shot Generalization of Safety Classifiers via Data Augmented Parameter-Efficient Fine-Tuning Ananth Balashankar Xiao Ma Aradhana Sinha Ahmad Beirami Yao Qin Jilin Chen Alex Beutel 24 2 0 25 Oct 2023
Decomposed Prompt Tuning via Low-Rank Reparameterization Yao Xiao Lu Xu Jiaxi Li Wei Lu Xiaoli Li VLM 25 6 0 16 Oct 2023
In-Context Learning for Few-Shot Molecular Property Prediction Christopher Fifty J. Leskovec Sebastian Thrun 36 5 0 13 Oct 2023
Comparing Styles across Languages: A Cross-Cultural Exploration of Politeness Shreya Havaldar Matthew Pressimone Eric Wong Lyle Ungar 58 2 0 11 Oct 2023
Fine-tune Language Models to Approximate Unbiased In-context Learning Timothy Chu Zhao Song Chiwun Yang 27 15 0 05 Oct 2023
NOLA: Compressing LoRA using Linear Combination of Random Basis Soroush Abbasi Koohpayegani K. Navaneet Parsa Nooralinejad Soheil Kolouri Hamed Pirsiavash 40 12 0 04 Oct 2023