Mixout: Effective Regularization to Finetune Large-scale Pretrained Language Models

25 September 2019

Papers citing "Mixout: Effective Regularization to Finetune Large-scale Pretrained Language Models"

50 / 134 papers shown

Title
LLM+P: Empowering Large Language Models with Optimal Planning Proficiency B. Liu Yuqian Jiang Xiaohan Zhang Qian Liu Shiqi Zhang Joydeep Biswas Peter Stone LM&Ro LLMAG 23 380 0 22 Apr 2023
Measuring the Instability of Fine-Tuning Yupei Du D. Nguyen 18 4 0 15 Feb 2023
Backward Compatibility During Data Updates by Weight Interpolation Raphael Schumann Elman Mansimov Yi-An Lai Nikolaos Pappas Xibin Gao Yi Zhang 11 4 0 25 Jan 2023
A Stability Analysis of Fine-Tuning a Pre-Trained Model Z. Fu Anthony Man-Cho So Nigel Collier 23 3 0 24 Jan 2023
Continuously Reliable Detection of New-Normal Misinformation: Semantic Masking and Contrastive Smoothing in High-Density Latent Regions Abhijit Suprem J. Ferreira C. Pu AAML 29 1 0 19 Jan 2023
NarrowBERT: Accelerating Masked Language Model Pretraining and Inference Haoxin Li Phillip Keung Daniel Cheng Jungo Kasai Noah A. Smith 12 3 0 11 Jan 2023
KL Regularized Normalization Framework for Low Resource Tasks Neeraj Kumar Ankur Narang Brejesh Lall 23 1 0 21 Dec 2022
Localising In-Domain Adaptation of Transformer-Based Biomedical Language Models T. M. Buonocore Claudio Crema A. Redolfi Riccardo Bellazzi Enea Parimbelli LM&MA 15 20 0 20 Dec 2022
HyPe: Better Pre-trained Language Model Fine-tuning with Hidden Representation Perturbation Hongyi Yuan Zheng Yuan Chuanqi Tan Fei Huang Songfang Huang 27 15 0 17 Dec 2022
G-MAP: General Memory-Augmented Pre-trained Language Model for Domain Tasks Zhongwei Wan Yichun Yin Wei Zhang Jiaxin Shi Lifeng Shang Guangyong Chen Xin Jiang Qun Liu VLM CLL 28 16 0 07 Dec 2022
On the Effectiveness of Parameter-Efficient Fine-Tuning Z. Fu Haoran Yang Anthony Man-Cho So Wai Lam Lidong Bing Nigel Collier 19 156 0 28 Nov 2022
Alignment-Enriched Tuning for Patch-Level Pre-trained Document Image Models Lei Wang Jian He Xingdong Xu Ning Liu Hui-juan Liu 33 2 0 27 Nov 2022
AF Adapter: Continual Pretraining for Building Chinese Biomedical Language Model Yongyu Yan Kui Xue Xiaoming Shi Qi Ye Jingping Liu Tong Ruan CLL 42 1 0 21 Nov 2022
Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors Thomas Hartvigsen S. Sankaranarayanan Hamid Palangi Yoon Kim Marzyeh Ghassemi KELM 14 143 0 20 Nov 2022
Towards Robust Low-Resource Fine-Tuning with Multi-View Compressed Representations Linlin Liu Xingxuan Li Megh Thakkar Xin Li Shafiq R. Joty Luo Si Lidong Bing 27 2 0 16 Nov 2022
Fine-Tuning Pre-Trained Language Models Effectively by Optimizing Subnetworks Adaptively Haojie Zhang Ge Li Jia Li Zhongjin Zhang Yuqi Zhu Zhi Jin AI4CE 8 26 0 03 Nov 2022
Parameter-Efficient Tuning Makes a Good Classification Head Zhuoyi Yang Ming Ding Yanhui Guo Qingsong Lv Jie Tang VLM 35 14 0 30 Oct 2022
PATS: Sensitivity-aware Noisy Learning for Pretrained Language Models Yupeng Zhang Hongzhi Zhang Sirui Wang Wei Yu Wu Zhoujun Li AAML 25 1 0 22 Oct 2022
Surgical Fine-Tuning Improves Adaptation to Distribution Shifts Yoonho Lee Annie S. Chen Fahim Tajwar Ananya Kumar Huaxiu Yao Percy Liang Chelsea Finn OOD 51 197 0 20 Oct 2022
Improving Stability of Fine-Tuning Pretrained Language Models via Component-Wise Gradient Norm Clipping Chenghao Yang Xuezhe Ma 32 6 0 19 Oct 2022
ROSE: Robust Selective Fine-tuning for Pre-trained Language Models Lan Jiang Hao Zhou Yankai Lin Peng Li Jie Zhou R. Jiang AAML 29 8 0 18 Oct 2022
Fine-mixing: Mitigating Backdoors in Fine-tuned Language Models Zhiyuan Zhang Lingjuan Lyu Xingjun Ma Chenguang Wang Xu Sun AAML 23 41 0 18 Oct 2022
AD-DROP: Attribution-Driven Dropout for Robust Language Model Fine-Tuning Tao Yang Jinghao Deng Xiaojun Quan Qifan Wang Shaoliang Nie 28 3 0 12 Oct 2022
SparseAdapter: An Easy Approach for Improving the Parameter-Efficiency of Adapters Shwai He Liang Ding Daize Dong Miao Zhang Dacheng Tao MoE 24 87 0 09 Oct 2022
Exploring Effective Knowledge Transfer for Few-shot Object Detection Zhiyuan Zhao Qingjie Liu Yunhong Wang 35 9 0 05 Oct 2022
Combating high variance in Data-Scarce Implicit Hate Speech Classification Debaditya Pal Kaustubh Chaudhari Harsh Sharma 21 1 0 29 Aug 2022
u-HuBERT: Unified Mixed-Modal Speech Pretraining And Zero-Shot Transfer to Unlabeled Modality Wei-Ning Hsu Bowen Shi SSL VLM 19 41 0 14 Jul 2022
Zero-shot Cross-lingual Transfer is Under-specified Optimization Shijie Wu Benjamin Van Durme Mark Dredze 25 6 0 12 Jul 2022
Improving Pre-trained Language Model Fine-tuning with Noise Stability Regularization Hang Hua Xingjian Li Dejing Dou Chengzhong Xu Jiebo Luo 31 15 0 12 Jun 2022
DynaMaR: Dynamic Prompt with Mask Token Representation Xiaodi Sun Sunny Rajagopalan Priyank Nigam Weiyi Lu Yi Xu Belinda Zeng Trishul M. Chilimbi 17 1 0 07 Jun 2022
Representation Projection Invariance Mitigates Representation Collapse Anastasia Razdaibiedina A. Khetan Zohar S. Karnin Daniel Khashabi Vishaal Kapoor V. Madan 32 5 0 23 May 2022
Embedding Hallucination for Few-Shot Language Fine-tuning Yiren Jian Chongyang Gao Soroush Vosoughi 23 4 0 03 May 2022
Robust Fine-tuning via Perturbation and Interpolation from In-batch Instances Shoujie Tong Qingxiu Dong Damai Dai Yifan Song Tianyu Liu Baobao Chang Zhifang Sui AAML 11 6 0 02 May 2022
Super-Prompting: Utilizing Model-Independent Contextual Data to Reduce Data Annotation Required in Visual Commonsense Tasks Navid Rezaei Marek Reformat VLM 17 2 0 25 Apr 2022
Event Transition Planning for Open-ended Text Generation Qintong Li Pijian Li Wei Bi Z. Ren Yuxuan Lai Lingpeng Kong 23 12 0 20 Apr 2022
Bridging Cross-Lingual Gaps During Leveraging the Multilingual Sequence-to-Sequence Pretraining for Text Generation and Understanding Changtong Zan Liang Ding Li Shen Yu Cao Weifeng Liu Dacheng Tao LRM 34 8 0 16 Apr 2022
Adaptive Transformers for Robust Few-shot Cross-domain Face Anti-spoofing Hsin-Ping Huang Deqing Sun Yaojie Liu Wen-Sheng Chu Taihong Xiao Jinwei Yuan Hartwig Adam Ming-Hsuan Yang CVBM 36 56 0 23 Mar 2022
Task-guided Disentangled Tuning for Pretrained Language Models Jiali Zeng Yu Jiang Shuangzhi Wu Yongjing Yin Mu Li DRL 17 3 0 22 Mar 2022
NoisyTune: A Little Noise Can Help You Finetune Pretrained Language Models Better Chuhan Wu Fangzhao Wu Tao Qi Yongfeng Huang Xing Xie 25 58 0 24 Feb 2022
Revisiting Parameter-Efficient Tuning: Are We Really There Yet? Guanzheng Chen Fangyu Liu Zaiqiao Meng Shangsong Liang 26 88 0 16 Feb 2022
A Differential Entropy Estimator for Training Neural Networks Georg Pichler Pierre Colombo Malik Boudiaf Günther Koliander Pablo Piantanida 20 21 0 14 Feb 2022
Adaptive Fine-Tuning of Transformer-Based Language Models for Named Entity Recognition Felix Stollenwerk 12 3 0 05 Feb 2022
Transferability in Deep Learning: A Survey Junguang Jiang Yang Shu Jianmin Wang Mingsheng Long OOD 34 101 0 15 Jan 2022
Pretrained Language Models for Text Generation: A Survey Junyi Li Tianyi Tang Wayne Xin Zhao J. Nie Ji-Rong Wen AI4CE 36 125 0 14 Jan 2022
MERLOT Reserve: Neural Script Knowledge through Vision and Language and Sound Rowan Zellers Jiasen Lu Ximing Lu Youngjae Yu Yanpeng Zhao Mohammadreza Salehi Aditya Kusupati Jack Hessel Ali Farhadi Yejin Choi 26 207 0 07 Jan 2022
How Should Pre-Trained Language Models Be Fine-Tuned Towards Adversarial Robustness? Xinhsuai Dong Anh Tuan Luu Min-Bin Lin Shuicheng Yan Hanwang Zhang SILM AAML 20 55 0 22 Dec 2021
Fine-Tuning Large Neural Language Models for Biomedical Natural Language Processing Robert Tinn Hao Cheng Yu Gu Naoto Usuyama Xiaodong Liu Tristan Naumann Jianfeng Gao Hoifung Poon LM&MA 17 111 0 15 Dec 2021
From Dense to Sparse: Contrastive Pruning for Better Pre-trained Language Model Compression Runxin Xu Fuli Luo Chengyu Wang Baobao Chang Jun Huang Songfang Huang Fei Huang VLM 27 25 0 14 Dec 2021
Discriminative and Generative Transformer-based Models For Situation Entity Classification Mehdi Rezaee Kasra Darvish Gaoussou Youssouf Kebe Francis Ferraro ViT 26 2 0 15 Sep 2021
Raise a Child in Large Language Model: Towards Effective and Generalizable Fine-tuning Runxin Xu Fuli Luo Zhiyuan Zhang Chuanqi Tan Baobao Chang Songfang Huang Fei Huang LRM 145 178 0 13 Sep 2021