Pretrained Transformers Improve Out-of-Distribution Robustness

13 April 2020

Papers citing "Pretrained Transformers Improve Out-of-Distribution Robustness"

50 / 109 papers shown

Title
Out-of-Distribution Detection and Selective Generation for Conditional Language Models Jie Jessie Ren Jiaming Luo Yao-Min Zhao Kundan Krishna Mohammad Saleh Balaji Lakshminarayanan Peter J. Liu OODD 75 96 0 30 Sep 2022
T-NER: An All-Round Python Library for Transformer-based Named Entity Recognition Asahi Ushio Jose Camacho-Collados 20 72 0 09 Sep 2022
Towards explainable evaluation of language models on the semantic similarity of visual concepts Maria Lymperaiou George Manoliadis Orfeas Menis Mastromichalakis Edmund Dervakos Giorgos Stamou AAML 24 5 0 08 Sep 2022
Selective Text Augmentation with Word Roles for Low-Resource Text Classification Biyang Guo Songqiao Han Hailiang Huang 19 9 0 04 Sep 2022
Doge Tickets: Uncovering Domain-general Language Models by Playing Lottery Tickets Yi Yang Chen Zhang Benyou Wang Dawei Song LRM 24 6 0 20 Jul 2022
POEM: Out-of-Distribution Detection with Posterior Sampling Yifei Ming Ying Fan Yixuan Li OODD 29 114 0 28 Jun 2022
Post-hoc Concept Bottleneck Models Mert Yuksekgonul Maggie Wang James Zou 145 185 0 31 May 2022
Detecting Label Errors by using Pre-Trained Language Models Derek Chong Jenny Hong Christopher D. Manning NoLa 55 21 0 25 May 2022
Self-Supervised Losses for One-Class Textual Anomaly Detection Kimberly T. Mai Toby O. Davies Lewis D. Griffin 17 7 0 12 Apr 2022
Metaphors in Pre-Trained Language Models: Probing and Generalization Across Datasets and Languages Ehsan Aghazadeh Mohsen Fayyaz Yadollah Yaghoobzadeh 33 51 0 26 Mar 2022
WOODS: Benchmarks for Out-of-Distribution Generalization in Time Series Jean-Christophe Gagnon-Audet Kartik Ahuja Mohammad Javad Darvishi Bayazi Pooneh Mousavi G. Dumas Irina Rish OOD CML AI4TS 32 29 0 18 Mar 2022
Generalized but not Robust? Comparing the Effects of Data Modification Methods on Out-of-Domain Generalization and Adversarial Robustness Tejas Gokhale Swaroop Mishra Man Luo Bhavdeep Singh Sachdeva Chitta Baral 52 29 0 15 Mar 2022
Investigating Selective Prediction Approaches Across Several Tasks in IID, OOD, and Adversarial Settings Neeraj Varshney Swaroop Mishra Chitta Baral 27 55 0 01 Mar 2022
Deconstructing Distributions: A Pointwise Framework of Learning Gal Kaplun Nikhil Ghosh Saurabh Garg Boaz Barak Preetum Nakkiran OOD 33 21 0 20 Feb 2022
Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model Shaden Smith M. Patwary Brandon Norick P. LeGresley Samyam Rajbhandari ... M. Shoeybi Yuxiong He Michael Houston Saurabh Tiwary Bryan Catanzaro MoE 90 732 0 28 Jan 2022
Video Transformers: A Survey Javier Selva A. S. Johansen Sergio Escalera Kamal Nasrollahi T. Moeslund Albert Clapés ViT 22 103 0 16 Jan 2022
Robust Natural Language Processing: Recent Advances, Challenges, and Future Directions Marwan Omar Soohyeon Choi Daehun Nyang David A. Mohaisen 32 57 0 03 Jan 2022
Measure and Improve Robustness in NLP Models: A Survey Xuezhi Wang Haohan Wang Diyi Yang 139 130 0 15 Dec 2021
The King is Naked: on the Notion of Robustness for Natural Language Processing Emanuele La Malfa Marta Z. Kwiatkowska 20 28 0 13 Dec 2021
Transformaly -- Two (Feature Spaces) Are Better Than One M. Cohen S. Avidan ViT 13 27 0 08 Dec 2021
Dyna-bAbI: unlocking bAbI's potential with dynamic synthetic benchmarking Ronen Tamari Kyle Richardson Aviad Sar-Shalom Noam Kahlon Nelson F. Liu Reut Tsarfaty Dafna Shahaf 43 5 0 30 Nov 2021
Understanding Out-of-distribution: A Perspective of Data Dynamics Dyah Adila Dongyeop Kang 38 12 0 29 Nov 2021
Towards Principled Disentanglement for Domain Generalization Hanlin Zhang Yi-Fan Zhang Weiyang Liu Adrian Weller Bernhard Schölkopf Eric Xing OOD 39 112 0 27 Nov 2021
Out-of-Category Document Identification Using Target-Category Names as Weak Supervision Dongha Lee Dongmin Hyun Jiawei Han Hwanjo Yu OOD 37 1 0 24 Nov 2021
Building Legal Datasets Jerrold Soh ELM AILaw 42 3 0 03 Nov 2021
Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey Bonan Min Hayley L Ross Elior Sulem Amir Pouran Ben Veyseh Thien Huu Nguyen Oscar Sainz Eneko Agirre Ilana Heinz Dan Roth LM&MA VLM AI4CE 83 1,035 0 01 Nov 2021
Generalized Out-of-Distribution Detection: A Survey Jingkang Yang Kaiyang Zhou Yixuan Li Ziwei Liu 188 879 0 21 Oct 2021
Identifying and Mitigating Spurious Correlations for Improving Robustness in NLP Models Tianlu Wang Rohit Sridhar Diyi Yang Xuezhi Wang AAML 120 72 0 14 Oct 2021
Adversarial Token Attacks on Vision Transformers Ameya Joshi Gauri Jagatap C. Hegde ViT 30 19 0 08 Oct 2021
Adversarial Robustness Comparison of Vision Transformer and MLP-Mixer to CNNs Philipp Benz Soomin Ham Chaoning Zhang Adil Karjauv In So Kweon AAML ViT 47 79 0 06 Oct 2021
Pretraining & Reinforcement Learning: Sharpening the Axe Before Cutting the Tree Saurav Kadavath Samuel Paradis Brian Yao VLM CLIP OffRL OnRL 22 1 0 06 Oct 2021
KroneckerBERT: Learning Kronecker Decomposition for Pre-trained Language Models via Knowledge Distillation Marzieh S. Tahaei Ella Charlaix V. Nia A. Ghodsi Mehdi Rezagholizadeh 46 22 0 13 Sep 2021
Efficient Test Time Adapter Ensembling for Low-resource Language Varieties Xinyi Wang Yulia Tsvetkov Sebastian Ruder Graham Neubig 35 34 0 10 Sep 2021
Context-aware Adversarial Training for Name Regularity Bias in Named Entity Recognition Abbas Ghaddar Philippe Langlais Ahmad Rashid Mehdi Rezagholizadeh 39 42 0 24 Jul 2021
Uncertainty-Aware Reliable Text Classification Yibo Hu Latifur Khan EDL UQCV 38 33 0 15 Jul 2021
On the Robustness of Pretraining and Self-Supervision for a Deep Learning-based Analysis of Diabetic Retinopathy Vignesh Srinivasan Nils Strodthoff Jackie Ma Alexander Binder Klaus-Robert Muller Wojciech Samek OOD 20 6 0 25 Jun 2021
Out-of-Scope Intent Detection with Self-Supervision and Discriminative Training Li-Ming Zhan Haowen Liang Bo Liu Lu Fan Xiao-Ming Wu Albert Y. S. Lam OODD 21 75 0 16 Jun 2021
How Robust are Model Rankings: A Leaderboard Customization Approach for Equitable Evaluation Swaroop Mishra Anjana Arunkumar 34 24 0 10 Jun 2021
RobustNav: Towards Benchmarking Robustness in Embodied Navigation Prithvijit Chattopadhyay Judy Hoffman Roozbeh Mottaghi Aniruddha Kembhavi 25 55 0 08 Jun 2021
On the Efficacy of Adversarial Data Collection for Question Answering: Results from a Large-Scale Randomized Study Divyansh Kaushik Douwe Kiela Zachary Chase Lipton Wen-tau Yih AAML 11 36 0 02 Jun 2021
Adversarially Adaptive Normalization for Single Domain Generalization Xinjie Fan Qifei Wang Junjie Ke Feng Yang Boqing Gong Mingyuan Zhou 27 129 0 01 Jun 2021
Improved OOD Generalization via Adversarial Training and Pre-training Mingyang Yi Lu Hou Jiacheng Sun Lifeng Shang Xin Jiang Qun Liu Zhi-Ming Ma VLM 31 83 0 24 May 2021
Reliability Testing for Natural Language Processing Systems Samson Tan Chenyu You K. Baxter Araz Taeihagh G. Bennett Min-Yen Kan 15 38 0 06 May 2021
Does Your Dermatology Classifier Know What It Doesn't Know? Detecting the Long-Tail of Unseen Conditions Abhijit Guha Roy Jie Jessie Ren Shekoofeh Azizi Aaron Loh Vivek Natarajan ... Yun-Hui Liu taylan. cemgil Alan Karthikesalingam Balaji Lakshminarayanan Jim Winkens 18 104 0 08 Apr 2021
On the Robustness of Vision Transformers to Adversarial Examples Kaleel Mahmood Rigel Mahmood Marten van Dijk ViT 25 217 0 31 Mar 2021
Learning Transferable Visual Models From Natural Language Supervision Alec Radford Jong Wook Kim Chris Hallacy Aditya A. Ramesh Gabriel Goh ... Amanda Askell Pamela Mishkin Jack Clark Gretchen Krueger Ilya Sutskever CLIP VLM 183 27,846 0 26 Feb 2021
Mind the Gap: Assessing Temporal Generalization in Neural Language Models Angeliki Lazaridou A. Kuncoro E. Gribovskaya Devang Agrawal Adam Liska ... Sebastian Ruder Dani Yogatama Kris Cao Susannah Young Phil Blunsom VLM 41 207 0 03 Feb 2021
Automated Lay Language Summarization of Biomedical Scientific Reviews Yue Guo Weijian Qiu Yizhong Wang T. Cohen 38 77 0 23 Dec 2020
Exploring Fluent Query Reformulations with Text-to-Text Transformers and Reinforcement Learning Jerry Zikun Chen S. Yu Haoran Wang 178 5 0 18 Dec 2020
WILDS: A Benchmark of in-the-Wild Distribution Shifts Pang Wei Koh Shiori Sagawa Henrik Marklund Sang Michael Xie Marvin Zhang ... A. Kundaje Emma Pierson Sergey Levine Chelsea Finn Percy Liang OOD 92 1,377 0 14 Dec 2020