What Would Elsa Do? Freezing Layers During Transformer Fine-Tuning

8 November 2019

Papers citing "What Would Elsa Do? Freezing Layers During Transformer Fine-Tuning"

28 / 28 papers shown

Title
ICE-Pruning: An Iterative Cost-Efficient Pruning Pipeline for Deep Neural Networks Wenhao Hu Paul Henderson José Cano 32 0 0 12 May 2025
Memory-Efficient Fine-Tuning of Transformers via Token Selection Antoine Simoulin Namyong Park Xiaoyi Liu Grey Yang 115 0 0 31 Jan 2025
Budgeted Online Continual Learning by Adaptive Layer Freezing and Frequency-based Sampling Minhyuk Seo Hyunseo Koh Jonghyun Choi 39 1 0 19 Oct 2024
AdaShadow: Responsive Test-time Model Adaptation in Non-stationary Mobile Environments Cheng Fang Sicong Liu Zimu Zhou Bin Guo Jiaqi Tang Ke Ma Zhiwen Yu TTA 39 1 0 10 Oct 2024
Fusion Transformer with Object Mask Guidance for Image Forgery Analysis Dimitrios Karageorgiou Giorgos Kordopatis-Zilos Symeon Papadopoulos ViT 28 5 0 18 Mar 2024
InfFeed: Influence Functions as a Feedback to Improve the Performance of Subjective Tasks Somnath Banerjee Maulindu Sarkar Punyajoy Saha Binny Mathew Animesh Mukherjee TDI 34 0 0 22 Feb 2024
Why Lift so Heavy? Slimming Large Language Models by Cutting Off the Layers Shuzhou Yuan Ercong Nie Bolei Ma Michael Farber 42 3 0 18 Feb 2024
Heterogeneous LoRA for Federated Fine-tuning of On-Device Foundation Models Yae Jee Cho Luyang Liu Zheng Xu Aldi Fahrezi Gauri Joshi 38 47 0 12 Jan 2024
Scalable Neural Network Kernels Arijit Sehanobish Krzysztof Choromanski Yunfan Zhao Kumar Avinava Dubey Valerii Likhosherstov 41 5 0 20 Oct 2023
PEFT-Ref: A Modular Reference Architecture and Typology for Parameter-Efficient Finetuning Techniques Mohammed Sabry Anya Belz 38 8 0 24 Apr 2023
Adopting the Multi-answer Questioning Task with an Auxiliary Metric for Extreme Multi-label Text Classification Utilizing the Label Hierarchy Li Wang Y. Teh M. Al-garadi 20 2 0 02 Mar 2023
SPARTAN: Sparse Hierarchical Memory for Parameter-Efficient Transformers Ameet Deshpande Md Arafat Sultan Anthony Ferritto Ashwin Kalyan Karthik R. Narasimhan Avirup Sil MoE 43 1 0 29 Nov 2022
AdaMix: Mixture-of-Adaptations for Parameter-efficient Model Tuning Yaqing Wang Sahaj Agarwal Subhabrata Mukherjee Xiaodong Liu Jing Gao Ahmed Hassan Awadallah Jianfeng Gao MoE 22 118 0 31 Oct 2022
Parameter-Efficient Tuning Makes a Good Classification Head Zhuoyi Yang Ming Ding Yanhui Guo Qingsong Lv Jie Tang VLM 58 14 0 30 Oct 2022
Fully-attentive and interpretable: vision and video vision transformers for pain detection Giacomo Fiorentini Itir Onal Ertugrul A. A. Salah MedIm ViT 21 2 0 27 Oct 2022
Surgical Fine-Tuning Improves Adaptation to Distribution Shifts Yoonho Lee Annie S. Chen Fahim Tajwar Ananya Kumar Huaxiu Yao Percy Liang Chelsea Finn OOD 61 198 0 20 Oct 2022
Tiny-Attention Adapter: Contexts Are More Important Than the Number of Parameters Hongyu Zhao Hao Tan Hongyuan Mei MoE 39 16 0 18 Oct 2022
Efficient NLP Model Finetuning via Multistage Data Filtering Ouyang Xu S. Ansari F. Lin Yangfeng Ji 35 2 0 28 Jul 2022
Delta Tuning: A Comprehensive Study of Parameter Efficient Methods for Pre-trained Language Models Ning Ding Yujia Qin Guang Yang Fu Wei Zonghan Yang ... Jianfei Chen Yang Liu Jie Tang Juan Li Maosong Sun 32 196 0 14 Mar 2022
HyperPELT: Unified Parameter-Efficient Language Model Tuning for Both Language and Vision-and-Language Tasks Zhengkun Zhang Wenya Guo Xiaojun Meng Yasheng Wang Yadao Wang Xin Jiang Qun Liu Zhenglu Yang 34 15 0 08 Mar 2022
Revisiting Parameter-Efficient Tuning: Are We Really There Yet? Guanzheng Chen Fangyu Liu Zaiqiao Meng Shangsong Liang 26 88 0 16 Feb 2022
Transformer-based Models of Text Normalization for Speech Applications Jae Hun Ro Felix Stahlberg Ke Wu Shankar Kumar 14 7 0 01 Feb 2022
Semi-Siamese Bi-encoder Neural Ranking Model Using Lightweight Fine-Tuning Euna Jung Jaekeol Choi Wonjong Rhee 22 13 0 28 Oct 2021
A Conditional Generative Matching Model for Multi-lingual Reply Suggestion Budhaditya Deb Guoqing Zheng Milad Shokouhi Ahmed Hassan Awadallah 36 1 0 15 Sep 2021
AutoFreeze: Automatically Freezing Model Blocks to Accelerate Fine-tuning Yuhan Liu Saurabh Agarwal Shivaram Venkataraman OffRL 19 53 0 02 Feb 2021
FastIF: Scalable Influence Functions for Efficient Model Interpretation and Debugging Han Guo Nazneen Rajani Peter Hase Joey Tianyi Zhou Caiming Xiong TDI 41 102 0 31 Dec 2020
Pretrained Transformers for Text Ranking: BERT and Beyond Jimmy J. Lin Rodrigo Nogueira Andrew Yates VLM 244 612 0 13 Oct 2020
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 299 6,984 0 20 Apr 2018