SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization

8 November 2019

Xiaodong Liu

Papers citing "SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization"

50 / 103 papers shown

Title
Amortized Proximal Optimization Juhan Bae Paul Vicol Jeff Z. HaoChen Roger C. Grosse ODL 25 14 0 28 Feb 2022
Is Neuro-Symbolic AI Meeting its Promise in Natural Language Processing? A Structured Review Kyle Hamilton Aparna Nayak Bojan Bozic Luca Longo NAI 29 57 0 24 Feb 2022
NoisyTune: A Little Noise Can Help You Finetune Pretrained Language Models Better Chuhan Wu Fangzhao Wu Tao Qi Yongfeng Huang Xing Xie 25 58 0 24 Feb 2022
GatorTron: A Large Clinical Language Model to Unlock Patient Information from Unstructured Electronic Health Records Xi Yang Aokun Chen Nima M. Pournejatian Hoo-Chang Shin Kaleb E. Smith ... Duane A. Mitchell W. Hogan E. Shenkman Jiang Bian Yonghui Wu AI4MH LM&MA 37 507 0 02 Feb 2022
Identifying Adversarial Attacks on Text Classifiers Zhouhang Xie Jonathan Brophy Adam Noack Wencong You Kalyani Asthana Carter Perkins Sabrina Reis Sameer Singh Daniel Lowd AAML 27 9 0 21 Jan 2022
Transferability in Deep Learning: A Survey Junguang Jiang Yang Shu Jianmin Wang Mingsheng Long OOD 34 101 0 15 Jan 2022
Sharpness-Aware Minimization with Dynamic Reweighting Wenxuan Zhou Fangyu Liu Huan Zhang Muhao Chen AAML 19 8 0 16 Dec 2021
Measure and Improve Robustness in NLP Models: A Survey Xuezhi Wang Haohan Wang Diyi Yang 139 130 0 15 Dec 2021
Survey of Generative Methods for Social Media Analysis Stan Matwin Aristides Milios P. Prałat Amílcar Soares Franccois Théberge 27 3 0 13 Dec 2021
Human Parity on CommonsenseQA: Augmenting Self-Attention with External Attention Yichong Xu Chenguang Zhu Shuohang Wang Siqi Sun Hao Cheng Xiaodong Liu Jianfeng Gao Pengcheng He Michael Zeng Xuedong Huang LRM 254 55 0 06 Dec 2021
MNet-Sim: A Multi-layered Semantic Similarity Network to Evaluate Sentence Similarity Manuela Nayantara Jeyaraj D. Kasthurirathna 11 3 0 09 Nov 2021
Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Models Wei Ping Chejian Xu Shuohang Wang Zhe Gan Yu Cheng Jianfeng Gao Ahmed Hassan Awadallah Yangqiu Song VLM ELM AAML 27 214 0 04 Nov 2021
Smooth Imitation Learning via Smooth Costs and Smooth Policies Sapana Chaudhary Balaraman Ravindran 21 1 0 03 Nov 2021
Generating artificial texts as substitution or complement of training data Vincent Claveau Antoine Chaffin Ewa Kijak 28 9 0 25 Oct 2021
CLLD: Contrastive Learning with Label Distance for Text Classification Jinhe Lan Qingyuan Zhan Chenhao Jiang Kunping Yuan Desheng Wang VLM 34 2 0 25 Oct 2021
Sharpness-Aware Minimization Improves Language Model Generalization Dara Bahri H. Mobahi Yi Tay 133 98 0 16 Oct 2021
SGD-X: A Benchmark for Robust Generalization in Schema-Guided Dialogue Systems Harrison Lee Raghav Gupta Abhinav Rastogi Yuan Cao Bin Zhang Yonghui Wu 74 33 0 13 Oct 2021
Taming Sparsely Activated Transformer with Stochastic Experts Simiao Zuo Xiaodong Liu Jian Jiao Young Jin Kim Hany Hassan Ruofei Zhang T. Zhao Jianfeng Gao MoE 39 108 0 08 Oct 2021
Using Psuedolabels for training Sentiment Classifiers makes the model generalize better across datasets N. Reddy Muktabh Mayank Srivastava 24 0 0 05 Oct 2021
Virtual Data Augmentation: A Robust and General Framework for Fine-tuning Pre-trained Models Kun Zhou Wayne Xin Zhao Sirui Wang Fuzheng Zhang Wei Yu Wu Ji-Rong Wen AAML 21 7 0 13 Sep 2021
TREATED:Towards Universal Defense against Textual Adversarial Attacks Bin Zhu Zhaoquan Gu Le Wang Zhihong Tian AAML 36 8 0 13 Sep 2021
Robust fine-tuning of zero-shot models Mitchell Wortsman Gabriel Ilharco Jong Wook Kim Mike Li Simon Kornblith ... Raphael Gontijo-Lopes Hannaneh Hajishirzi Ali Farhadi Hongseok Namkoong Ludwig Schmidt VLM 64 689 0 04 Sep 2021
How Does Adversarial Fine-Tuning Benefit BERT? J. Ebrahimi Hao Yang Wei Zhang AAML 26 4 0 31 Aug 2021
Noise Stability Regularization for Improving BERT Fine-tuning Hang Hua Xingjian Li Dejing Dou Chengzhong Xu Jiebo Luo 19 43 0 10 Jul 2021
A Survey on Data Augmentation for Text Classification Markus Bayer M. Kaufhold Christian A. Reuter 36 334 0 07 Jul 2021
CLINE: Contrastive Learning with Semantic Negative Examples for Natural Language Understanding Dong Wang Ning Ding Pijian Li Haitao Zheng AAML 37 115 0 01 Jul 2021
Learn to Resolve Conversational Dependency: A Consistency Training Framework for Conversational Question Answering Gangwoo Kim Hyunjae Kim Jungsoo Park Jaewoo Kang 30 38 0 22 Jun 2021
Consistency Regularization for Cross-Lingual Fine-Tuning Bo Zheng Li Dong Shaohan Huang Wenhui Wang Zewen Chi Saksham Singhal Wanxiang Che Ting Liu Xia Song Furu Wei 19 58 0 15 Jun 2021
An Empirical Survey of Data Augmentation for Limited Data Learning in NLP Jiaao Chen Derek Tam Colin Raffel Joey Tianyi Zhou Diyi Yang 28 172 0 14 Jun 2021
Defending Against Backdoor Attacks in Natural Language Generation Xiaofei Sun Xiaoya Li Yuxian Meng Xiang Ao Fei Wu Jiwei Li Tianwei Zhang AAML SILM 31 47 0 03 Jun 2021
Self-Guided Contrastive Learning for BERT Sentence Representations Taeuk Kim Kang Min Yoo Sang-goo Lee SSL 39 202 0 03 Jun 2021
Empirical Evaluation of Pre-trained Transformers for Human-Level NLP: The Role of Sample Size and Dimensionality Adithya V Ganesan Matthew Matero Aravind Reddy Ravula Huy-Hien Vu H. A. Schwartz 22 35 0 07 May 2021
Playing Lottery Tickets with Vision and Language Zhe Gan Yen-Chun Chen Linjie Li Tianlong Chen Yu Cheng Shuohang Wang Jingjing Liu Lijuan Wang Zicheng Liu VLM 106 54 0 23 Apr 2021
Reweighting Augmented Samples by Minimizing the Maximal Expected Loss Mingyang Yi Lu Hou Lifeng Shang Xin Jiang Qun Liu Zhi-Ming Ma 12 19 0 16 Mar 2021
A Closer Look at the Robustness of Vision-and-Language Pre-trained Models Linjie Li Zhe Gan Jingjing Liu VLM 33 42 0 15 Dec 2020
Contrastive Learning with Adversarial Perturbations for Conditional Text Generation Seanie Lee Dong Bok Lee Sung Ju Hwang 24 106 0 14 Dec 2020
Infusing Finetuning with Semantic Dependencies Zhaofeng Wu Hao Peng Noah A. Smith 22 36 0 10 Dec 2020
Supervised Contrastive Learning for Pre-trained Language Model Fine-tuning Beliz Gunel Jingfei Du Alexis Conneau Ves Stoyanov 18 497 0 03 Nov 2020
Posterior Differential Regularization with f-divergence for Improving Model Robustness Hao Cheng Xiaodong Liu L. Pereira Yaoliang Yu Jianfeng Gao 248 31 0 23 Oct 2020
Calibrated Language Model Fine-Tuning for In- and Out-of-Distribution Data Lingkai Kong Haoming Jiang Yuchen Zhuang Jie Lyu T. Zhao Chao Zhang OODD 24 26 0 22 Oct 2020
Robust Optimization as Data Augmentation for Large-scale Graphs Kezhi Kong Bernard Ghanem Mucong Ding Zuxuan Wu Chen Zhu Guohao Li Gavin Taylor Tom Goldstein 106 74 0 19 Oct 2020
InfoBERT: Improving Robustness of Language Models from An Information Theoretic Perspective Wei Ping Shuohang Wang Yu Cheng Zhe Gan R. Jia Bo-wen Li Jingjing Liu AAML 46 113 0 05 Oct 2020
A Simple but Tough-to-Beat Data Augmentation Approach for Natural Language Understanding and Generation Dinghan Shen Ming Zheng Yelong Shen Yanru Qu Weizhu Chen AAML 29 130 0 29 Sep 2020
Conditionally Adaptive Multi-Task Learning: Improving Transfer Learning in NLP Using Fewer Parameters & Less Data Jonathan Pilault Amine Elhattami C. Pal CLL MoE 24 89 0 19 Sep 2020
FarsTail: A Persian Natural Language Inference Dataset Hossein Amirkhani Mohammad AzariJafari Zohreh Pourjafari Soroush Faridan-Jahromi Zeinab Kouhkan Azadeh Amirak 19 30 0 18 Sep 2020
STEAM: Self-Supervised Taxonomy Expansion with Mini-Paths Yue Yu Yinghao Li Jiaming Shen Haoyang Feng Jimeng Sun Chao Zhang 26 58 0 18 Jun 2020
Large-Scale Adversarial Training for Vision-and-Language Representation Learning Zhe Gan Yen-Chun Chen Linjie Li Chen Zhu Yu Cheng Jingjing Liu ObjD VLM 35 488 0 11 Jun 2020
DeBERTa: Decoding-enhanced BERT with Disentangled Attention Pengcheng He Xiaodong Liu Jianfeng Gao Weizhu Chen AAML 62 2,622 0 05 Jun 2020
Adversarial Training for Large Neural Language Models Xiaodong Liu Hao Cheng Pengcheng He Weizhu Chen Yu-Chiang Frank Wang Hoifung Poon Jianfeng Gao AAML 26 183 0 20 Apr 2020
XGPT: Cross-modal Generative Pre-Training for Image Captioning Qiaolin Xia Haoyang Huang Nan Duan Dongdong Zhang Lei Ji Zhifang Sui Edward Cui Taroon Bharti Xin Liu Ming Zhou MLLM VLM 25 74 0 03 Mar 2020