Distributionally Robust Language Modeling

4 September 2019

Yonatan Oren

Shiori Sagawa

Tatsunori B. Hashimoto

Percy Liang

OOD

ArXiv PDF HTML

Papers citing "Distributionally Robust Language Modeling"

50 / 53 papers shown

Title
Class-Conditional Distribution Balancing for Group Robust Classification Miaoyun Zhao Qiang Zhang C. Li 70 1 0 24 Apr 2025
Upsample or Upweight? Balanced Training on Heavily Imbalanced Datasets Tianjian Li Haoran Xu Weiting Tan Kenton Murray Daniel Khashabi 35 1 0 06 Oct 2024
ExPLoRA: Parameter-Efficient Extended Pre-Training to Adapt Vision Transformers under Domain Shifts Samar Khanna Medhanie Irgau David B. Lobell Stefano Ermon VLM 37 4 0 16 Jun 2024
Rho-1: Not All Tokens Are What You Need Zheng-Wen Lin Zhibin Gou Yeyun Gong Xiao Liu Yelong Shen ... Chen Lin Yujiu Yang Jian Jiao Nan Duan Weizhu Chen CLL 50 58 0 11 Apr 2024
Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models Samuel Marks Can Rager Eric J. Michaud Yonatan Belinkov David Bau Aaron Mueller 53 122 0 28 Mar 2024
To Pool or Not To Pool: Analyzing the Regularizing Effects of Group-Fair Training on Shared Models Cyrus Cousins I. E. Kumar Suresh Venkatasubramanian FedML 43 1 0 29 Feb 2024
Group Distributionally Robust Dataset Distillation with Risk Minimization Saeed Vahidian Mingyu Wang Jianyang Gu Vyacheslav Kungurtsev Wei Jiang Yiran Chen OOD DD 43 6 0 07 Feb 2024
REDUCR: Robust Data Downsampling Using Class Priority Reweighting William Bankes George Hughes Ilija Bogunovic Zi Wang 38 3 0 01 Dec 2023
Bias Amplification Enhances Minority Group Performance Gaotang Li Jiarui Liu Wei Hu 30 5 0 13 Sep 2023
Efficient Stochastic Approximation of Minimax Excess Risk Optimization Lijun Zhang Haomin Bai W. Tu Ping Yang Yao Hu 19 4 0 31 May 2023
From Adversarial Arms Race to Model-centric Evaluation: Motivating a Unified Automatic Robustness Evaluation Framework Yangyi Chen Hongcheng Gao Ganqu Cui Lifan Yuan Dehan Kong ... Longtao Huang H. Xue Zhiyuan Liu Maosong Sun Heng Ji AAML ELM 38 6 0 29 May 2023
Out-of-Distribution Generalization in Text Classification: Past, Present, and Future Linyi Yang Yangqiu Song Xuan Ren Chenyang Lyu Yidong Wang Lingqiao Liu Jindong Wang Jennifer Foster Yue Zhang OOD 44 2 0 23 May 2023
DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretraining Sang Michael Xie Hieu H. Pham Xuanyi Dong Nan Du Hanxiao Liu Yifeng Lu Percy Liang Quoc V. Le Tengyu Ma Adams Wei Yu MoMe MoE 61 180 0 17 May 2023
Diffusion Theory as a Scalpel: Detecting and Purifying Poisonous Dimensions in Pre-trained Language Models Caused by Backdoor or Bias Zhiyuan Zhang Deli Chen Hao Zhou Fandong Meng Jie Zhou Xu Sun 36 5 0 08 May 2023
Re-weighting Based Group Fairness Regularization via Classwise Robust Optimization Sangwon Jung Taeeon Park Sanghyuk Chun Taesup Moon 13 19 0 01 Mar 2023
Hyper-parameter Tuning for Fair Classification without Sensitive Attribute Access A. Veldanda Ivan Brugere Sanghamitra Dutta Alan Mishler S. Garg 47 5 0 02 Feb 2023
Model Agnostic Sample Reweighting for Out-of-Distribution Learning Xiao Zhou Yong Lin Renjie Pi Weizhong Zhang Renzhe Xu Peng Cui Tong Zhang OODD 44 61 0 24 Jan 2023
Foundation models in brief: A historical, socio-technical focus Johannes Schneider VLM 34 9 0 17 Dec 2022
Improving group robustness under noisy labels using predictive uncertainty Dongpin Oh Dae Lee Jeunghyun Byun Bonggun Shin UQCV 25 3 0 14 Dec 2022
The future is different: Large pre-trained language models fail in prediction tasks K. Cvejoski Ramses J. Sanchez C. Ojeda 35 3 0 01 Nov 2022
COCO-DR: Combating Distribution Shifts in Zero-Shot Dense Retrieval with Contrastive and Distributionally Robust Learning Yue Yu Chenyan Xiong Si Sun Chao Zhang Arnold Overwijk VLM OOD 57 22 0 27 Oct 2022
Tencent's Multilingual Machine Translation System for WMT22 Large-Scale African Languages Wenxiang Jiao Zhaopeng Tu Jiarui Li Wenxuan Wang Jen-tse Huang Shuming Shi 54 15 0 18 Oct 2022
Learning from a Biased Sample Roshni Sahoo Lihua Lei Stefan Wager 27 17 0 05 Sep 2022
Learning to Split for Automatic Bias Detection Yujia Bao Regina Barzilay 26 20 0 28 Apr 2022
Distributionally Robust Models with Parametric Likelihood Ratios Paul Michel Tatsunori Hashimoto Graham Neubig OOD 35 15 0 13 Apr 2022
Last Layer Re-Training is Sufficient for Robustness to Spurious Correlations Polina Kirichenko Pavel Izmailov A. Wilson OOD 56 320 0 06 Apr 2022
Distributionally Robust Optimization via Ball Oracle Acceleration Y. Carmon Danielle Hausler 23 12 0 24 Mar 2022
Evaluating Distributional Distortion in Neural Language Modeling Benjamin LeBrun Alessandro Sordoni Timothy J. O'Donnell 24 22 0 24 Mar 2022
Challenges and Strategies in Cross-Cultural NLP Daniel Hershcovich Stella Frank Heather Lent Miryam de Lhoneux Mostafa Abdou ... Ruixiang Cui Constanza Fierro Katerina Margatina Phillip Rust Anders Søgaard 48 163 0 18 Mar 2022
WOODS: Benchmarks for Out-of-Distribution Generalization in Time Series Jean-Christophe Gagnon-Audet Kartik Ahuja Mohammad Javad Darvishi Bayazi Pooneh Mousavi G. Dumas Irina Rish OOD CML AI4TS 37 29 0 18 Mar 2022
Towards Group Robustness in the presence of Partial Group Labels Vishnu Suresh Lokhande Kihyuk Sohn Jinsung Yoon Madeleine Udell Chen-Yu Lee Tomas Pfister OOD 45 11 0 10 Jan 2022
Measure and Improve Robustness in NLP Models: A Survey Xuezhi Wang Haohan Wang Diyi Yang 139 131 0 15 Dec 2021
Editing a classifier by rewriting its prediction rules Shibani Santurkar Dimitris Tsipras Mahalaxmi Elango David Bau Antonio Torralba Aleksander Madry KELM 188 89 0 02 Dec 2021
Distributionally Robust Recurrent Decoders with Random Network Distillation Antonio Valerio Miceli Barone Alexandra Birch Rico Sennrich 39 1 0 25 Oct 2021
Symbolic Brittleness in Sequence Models: on Systematic Generalization in Symbolic Mathematics Sean Welleck Peter West Jize Cao Yejin Choi 32 28 0 28 Sep 2021
On Language Models for Creoles Heather Lent Emanuele Bugliarello Miryam de Lhoneux Chen Qiu Anders Søgaard 44 20 0 13 Sep 2021
A comparison of approaches to improve worst-case predictive model performance over patient subpopulations Stephen R. Pfohl Haoran Zhang Yizhe Xu Agata Foryciarz Marzyeh Ghassemi N. Shah OOD 34 22 0 27 Aug 2021
Just Train Twice: Improving Group Robustness without Training Group Information E. Liu Behzad Haghgoo Annie S. Chen Aditi Raghunathan Pang Wei Koh Shiori Sagawa Percy Liang Chelsea Finn OOD 37 540 0 19 Jul 2021
The Spotlight: A General Method for Discovering Systematic Errors in Deep Learning Models G. dÉon Jason dÉon J. R. Wright Kevin Leyton-Brown 33 74 0 01 Jul 2021
Learning Stable Classifiers by Transferring Unstable Features Yujia Bao Shiyu Chang Regina Barzilay OOD 32 8 0 15 Jun 2021
Examining and Combating Spurious Features under Distribution Shift Chunting Zhou Xuezhe Ma Paul Michel Graham Neubig OOD 37 67 0 14 Jun 2021
DORO: Distributional and Outlier Robust Optimization Runtian Zhai Chen Dan J. Zico Kolter Pradeep Ravikumar 33 60 0 11 Jun 2021
Predict then Interpolate: A Simple Algorithm to Learn Stable Classifiers Yujia Bao Shiyu Chang Regina Barzilay 27 20 0 26 May 2021
Detoxifying Language Models Risks Marginalizing Minority Voices Albert Xu Eshaan Pathak Eric Wallace Suchin Gururangan Maarten Sap Dan Klein 24 123 0 13 Apr 2021
Label-Imbalanced and Group-Sensitive Classification under Overparameterization Ganesh Ramachandra Kini Orestis Paraskevas Samet Oymak Christos Thrampoulidis 36 94 0 02 Mar 2021
PADA: Example-based Prompt Learning for on-the-fly Adaptation to Unseen Domains Eyal Ben-David Nadav Oved Roi Reichart VLM OOD 22 88 0 24 Feb 2021
Mind the Gap: Assessing Temporal Generalization in Neural Language Models Angeliki Lazaridou A. Kuncoro E. Gribovskaya Devang Agrawal Adam Liska ... Sebastian Ruder Dani Yogatama Kris Cao Susannah Young Phil Blunsom VLM 46 207 0 03 Feb 2021
WILDS: A Benchmark of in-the-Wild Distribution Shifts Pang Wei Koh Shiori Sagawa Henrik Marklund Sang Michael Xie Marvin Zhang ... A. Kundaje Emma Pierson Sergey Levine Chelsea Finn Percy Liang OOD 106 1,386 0 14 Dec 2020
Utility is in the Eye of the User: A Critique of NLP Leaderboards Kawin Ethayarajh Dan Jurafsky ELM 24 51 0 29 Sep 2020
An Empirical Study on Robustness to Spurious Correlations using Pre-trained Language Models Lifu Tu Garima Lalwani Spandana Gella He He LRM 33 184 0 14 Jul 2020