SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization

8 November 2019

Xiaodong Liu

Papers citing "SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization"

50 / 103 papers shown

Title
Adaptive Latent-Space Constraints in Personalized FL Sana Ayromlou D. B. Emerson FedML 49 0 0 12 May 2025
IM-BERT: Enhancing Robustness of BERT through the Implicit Euler Method Mihyeon Kim Juhyoung Park Youngbin Kim 34 0 0 11 May 2025
Do we really have to filter out random noise in pre-training data for language models? Jinghan Ru Yuxin Xie Xianwei Zhuang Yuguo Yin Zhihui Guo Zhiming Liu Qianli Ren Yuexian Zou 83 2 0 10 Feb 2025
Impeding LLM-assisted Cheating in Introductory Programming Assignments via Adversarial Perturbation Saiful Islam Salim Rubin Yuchan Yang Alexander Cooper Suryashree Ray Saumya Debray Sazzadur Rahaman AAML 47 0 0 12 Oct 2024
Robust LLM safeguarding via refusal feature adversarial training L. Yu Virginie Do Karen Hambardzumyan Nicola Cancedda AAML 62 10 0 30 Sep 2024
GAMedX: Generative AI-based Medical Entity Data Extractor Using Large Language Models Mohammed-Khalil Ghali Abdelrahman Farrag Hajar Sakai Hicham El Baz Yu Jin Sarah Lam LM&MA MedIm 40 8 0 31 May 2024
Generalization Measures for Zero-Shot Cross-Lingual Transfer Saksham Bassi Duygu Ataman Kyunghyun Cho 29 0 0 24 Apr 2024
From Robustness to Improved Generalization and Calibration in Pre-trained Language Models Josip Jukić Jan Snajder 37 0 0 31 Mar 2024
Soft Prompt Threats: Attacking Safety Alignment and Unlearning in Open-Source LLMs through the Embedding Space Leo Schwinn David Dobre Sophie Xhonneux Gauthier Gidel Stephan Gunnemann AAML 51 38 0 14 Feb 2024
Black-Box Access is Insufficient for Rigorous AI Audits Stephen Casper Carson Ezell Charlotte Siegmann Noam Kolt Taylor Lynn Curtis ... Michael Gerovitch David Bau Max Tegmark David M. Krueger Dylan Hadfield-Menell AAML 34 78 0 25 Jan 2024
Dynamic Corrective Self-Distillation for Better Fine-Tuning of Pretrained Models Ibtihel Amara Vinija Jain Aman Chadha 32 0 0 12 Dec 2023
Weigh Your Own Words: Improving Hate Speech Counter Narrative Generation via Attention Regularization Helena Bonaldi Giuseppe Attanasio Debora Nozza Marco Guerini 20 6 0 05 Sep 2023
Efficient Discovery and Effective Evaluation of Visual Perceptual Similarity: A Benchmark and Beyond Oren Barkan Tal Reiss Jonathan Weill Ori Katz Roy Hirsch Itzik Malkiel Noam Koenigstein 42 6 0 28 Aug 2023
LimeAttack: Local Explainable Method for Textual Hard-Label Adversarial Attack HaiXiang Zhu Zhaoqing Yang Weiwei Shang Yuren Wu AAML FAtt 10 3 0 01 Aug 2023
Bi-Drop: Enhancing Fine-tuning Generalization via Synchronous sub-net Estimation and Optimization Shoujie Tong Heming Xia Damai Dai Runxin Xu Tianyu Liu Binghuai Lin Yunbo Cao Zhifang Sui 20 0 0 24 May 2023
Out-of-Distribution Generalization in Text Classification: Past, Present, and Future Linyi Yang Yangqiu Song Xuan Ren Chenyang Lyu Yidong Wang Lingqiao Liu Jindong Wang Jennifer Foster Yue Zhang OOD 37 2 0 23 May 2023
SHINE: Syntax-augmented Hierarchical Interactive Encoder for Zero-shot Cross-lingual Information Extraction Jun-Yu Ma Jia-Chen Gu Zhen-Hua Ling Quan Liu Cong Liu Guoping Hu 57 1 0 21 May 2023
USTC-NELSLIP at SemEval-2023 Task 2: Statistical Construction and Dual Adaptation of Gazetteer for Multilingual Complex NER Jun-Yu Ma Jia-Chen Gu Jiajun Qi Zhen-Hua Ling Quan Liu Xiaoyi Zhao 29 2 0 04 May 2023
Tailoring Language Generation Models under Total Variation Distance Haozhe Ji Pei Ke Zhipeng Hu Rongsheng Zhang Minlie Huang 28 18 0 26 Feb 2023
LabelPrompt: Effective Prompt-based Learning for Relation Classification Wenbo Zhang Xiaoning Song Zhenhua Feng Tianyang Xu Xiaojun Wu VLM 35 4 0 16 Feb 2023
HateProof: Are Hateful Meme Detection Systems really Robust? Piush Aggarwal Pranit Chawla Mithun Das Punyajoy Saha Binny Mathew Torsten Zesch Animesh Mukherjee AAML 31 8 0 11 Feb 2023
ZhichunRoad at Amazon KDD Cup 2022: MultiTask Pre-Training for E-Commerce Product Search Xuange Cui Wei Xiong Songlin Wang 35 1 0 31 Jan 2023
WIDER & CLOSER: Mixture of Short-channel Distillers for Zero-shot Cross-lingual Named Entity Recognition Jun-Yu Ma Beiduo Chen Jia-Chen Gu Zhen-Hua Ling Wu Guo Quan Liu Zhigang Chen Cong Liu 31 10 0 07 Dec 2022
Finetune like you pretrain: Improved finetuning of zero-shot vision models Sachin Goyal Ananya Kumar Sankalp Garg Zico Kolter Aditi Raghunathan CLIP VLM 41 136 0 01 Dec 2022
Language Model Pre-training on True Negatives Zhuosheng Zhang Hai Zhao Masao Utiyama Eiichiro Sumita 34 2 0 01 Dec 2022
Alignment-Enriched Tuning for Patch-Level Pre-trained Document Image Models Lei Wang Jian He Xingdong Xu Ning Liu Hui-juan Liu 36 2 0 27 Nov 2022
Robust Lottery Tickets for Pre-trained Language Models Rui Zheng Rong Bao Yuhao Zhou Di Liang Sirui Wang Wei Yu Wu Tao Gui Qi Zhang Xuanjing Huang AAML 14 13 0 06 Nov 2022
BERT-Deep CNN: State-of-the-Art for Sentiment Analysis of COVID-19 Tweets Javad Hassannataj Joloudari Sadiq Hussain M. Nematollahi Rouhollah Bagheri Fatemeh Fazl R. Alizadehsani Reza Lashgari Ashis Talukder 18 38 0 04 Nov 2022
Parameter-Efficient Tuning Makes a Good Classification Head Zhuoyi Yang Ming Ding Yanhui Guo Qingsong Lv Jie Tang VLM 40 14 0 30 Oct 2022
Precisely the Point: Adversarial Augmentations for Faithful and Informative Text Generation Wenhao Wu Wei Li Jiachen Liu Xinyan Xiao Sujian Li Yajuan Lyu 33 3 0 22 Oct 2022
TCAB: A Large-Scale Text Classification Attack Benchmark Kalyani Asthana Zhouhang Xie Wencong You Adam Noack Jonathan Brophy Sameer Singh Daniel Lowd 39 3 0 21 Oct 2022
BioLORD: Learning Ontological Representations from Definitions (for Biomedical Concepts and their Textual Descriptions) François Remy Kris Demuynck Thomas Demeester 16 13 0 21 Oct 2022
Surgical Fine-Tuning Improves Adaptation to Distribution Shifts Yoonho Lee Annie S. Chen Fahim Tajwar Ananya Kumar Huaxiu Yao Percy Liang Chelsea Finn OOD 58 197 0 20 Oct 2022
ROSE: Robust Selective Fine-tuning for Pre-trained Language Models Lan Jiang Hao Zhou Yankai Lin Peng Li Jie Zhou R. Jiang AAML 37 8 0 18 Oct 2022
Short Text Pre-training with Extended Token Classification for E-commerce Query Understanding Haoming Jiang Tianyu Cao Zheng Li Cheng-hsin Luo Xianfeng Tang Qingyu Yin Danqing Zhang R. Goutam Bing Yin RALM 27 11 0 08 Oct 2022
State-of-the-art generalisation research in NLP: A taxonomy and review Dieuwke Hupkes Mario Giulianelli Verna Dankers Mikel Artetxe Yanai Elazar ... Leila Khalatbari Maria Ryskina Rita Frieske Ryan Cotterell Zhijing Jin 114 93 0 06 Oct 2022
InFi: End-to-End Learning to Filter Input for Resource-Efficiency in Mobile-Centric Inference Mu Yuan Lan Zhang Fengxiang He Xueting Tong Miao-Hui Song Zhengyuan Xu Xiang-Yang Li 26 2 0 28 Sep 2022
Linear Transformations for Cross-lingual Sentiment Analysis Pavel Přibáň Jakub Šmíd Adam Mištera Pavel Král 26 3 0 15 Sep 2022
Socially Enhanced Situation Awareness from Microblogs using Artificial Intelligence: A Survey Rabindra Lamsal Aaron Harwood M. Read 34 20 0 13 Sep 2022
Multi-Level Fine-Tuning, Data Augmentation, and Few-Shot Learning for Specialized Cyber Threat Intelligence Markus Bayer Tobias Frey Christian A. Reuter AAML 21 15 0 22 Jul 2022
Domain Confused Contrastive Learning for Unsupervised Domain Adaptation Quanyu Long Tianze Luo Wenya Wang Sinno Jialin Pan 57 8 0 10 Jul 2022
Dual Decomposition of Convex Optimization Layers for Consistent Attention in Medical Images Tom Ron M. Weiler-Sagie Tamir Hazan FAtt MedIm 21 6 0 06 Jun 2022
TreeMix: Compositional Constituency-based Data Augmentation for Natural Language Understanding Le Zhang Zichao Yang Diyi Yang 36 24 0 12 May 2022
Few-shot Mining of Naturally Occurring Inputs and Outputs Mandar Joshi Terra Blevins M. Lewis Daniel S. Weld Luke Zettlemoyer 30 1 0 09 May 2022
Embedding Hallucination for Few-Shot Language Fine-tuning Yiren Jian Chongyang Gao Soroush Vosoughi 23 4 0 03 May 2022
METRO: Efficient Denoising Pretraining of Large Scale Autoencoding Language Models with Model Generated Signals Payal Bajaj Chenyan Xiong Guolin Ke Xiaodong Liu Di He Saurabh Tiwary Tie-Yan Liu Paul N. Bennett Xia Song Jianfeng Gao 47 32 0 13 Apr 2022
Impossible Triangle: What's Next for Pre-trained Language Models? Chenguang Zhu Michael Zeng 21 1 0 13 Apr 2022
Incremental Few-Shot Learning via Implanting and Compressing Yiting Li H. Zhu Xijia Feng Zilong Cheng Jun Ma Cheng Xiang P. Vadakkepat T. Lee CLL VLM 21 2 0 19 Mar 2022
elBERto: Self-supervised Commonsense Learning for Question Answering Xunlin Zhan Yuan Li Xiao Dong Xiaodan Liang Zhiting Hu Lawrence Carin SSL RALM LRM 24 7 0 17 Mar 2022
USTC-NELSLIP at SemEval-2022 Task 11: Gazetteer-Adapted Integration Network for Multilingual Complex Named Entity Recognition Beiduo Chen Jun-Yu Ma Jiajun Qi Wu Guo Zhen-Hua Ling Quan Liu 18 16 0 07 Mar 2022