Title
FLUKE: A Linguistically-Driven and Task-Agnostic Framework for Robustness Evaluation Yulia Otmakhova Hung Thinh Truong Rahmad Mahendra Zenan Zhai Rongxin Zhu Daniel Beck Jey Han Lau ELM 70 0 0 24 Apr 2025
MultiLoKo: a multilingual local knowledge benchmark for LLMs spanning 31 languages Dieuwke Hupkes Nikolay Bogoychev 124 0 0 14 Apr 2025
TRA: Better Length Generalisation with Threshold Relative Attention Mattia Opper Roland Fernandez P. Smolensky Jianfeng Gao 46 0 0 29 Mar 2025
Gradient-Guided Annealing for Domain Generalization Aristotelis Ballas Christos Diou OOD 174 0 0 27 Feb 2025
Frequency matters: Modeling irregular morphological patterns in Spanish with Transformers Akhilesh Kakolu Ramarao Kevin Tang Dinah Baer-Henney 33 0 0 28 Oct 2024
Filtered Corpus Training (FiCT) Shows that Language Models can Generalize from Indirect Evidence Abhinav Patil Jaap Jumelet Yu Ying Chiu Andy Lapastora Peter Shen Lexie Wang Clevis Willrich Shane Steinert-Threlkeld 35 13 0 24 May 2024
From Frege to chatGPT: Compositionality in language, cognition, and deep neural networks Jacob Russin Sam Whitman McGrath Danielle J. Williams Lotem Elber-Dorozko AI4CE 73 3 0 24 May 2024
From Robustness to Improved Generalization and Calibration in Pre-trained Language Models Josip Jukić Jan Snajder 37 0 0 31 Mar 2024
Walking a Tightrope -- Evaluating Large Language Models in High-Risk Domains Chia-Chien Hung Wiem Ben-Rim Lindsay Frost Lars Bruckner Carolin (Haas) Lawrence AILaw ALM ELM 25 9 0 25 Nov 2023
On Using Distribution-Based Compositionality Assessment to Evaluate Compositional Generalisation in Machine Translation Anssi Moisio Mathias Creutz M. Kurimo CoGe 19 1 0 14 Nov 2023
Mind the instructions: a holistic evaluation of consistency and interactions in prompt-based learning Lucas Weber Elia Bruni Dieuwke Hupkes 32 24 0 20 Oct 2023
Cross-Lingual Consistency of Factual Knowledge in Multilingual Language Models Jirui Qi Raquel Fernández Arianna Bisazza KELM HILM 24 60 0 16 Oct 2023
Joint Dropout: Improving Generalizability in Low-Resource Neural Machine Translation through Phrase Pair Variables Ali Araabi Vlad Niculae Christof Monz 52 1 0 24 Jul 2023
Classical Out-of-Distribution Detection Methods Benchmark in Text Classification Tasks M. Baran Joanna Baran Mateusz Wójcik Maciej Ziȩba Adam Gonczarek OODD 44 4 0 13 Jul 2023
Measuring the Robustness of NLP Models to Domain Shifts Nitay Calderon Naveh Porat Eyal Ben-David Alexander Chapanin Zorik Gekhman Nadav Oved Vitaly Shalumov Roi Reichart 21 7 0 31 May 2023
Towards More Robust NLP System Evaluation: Handling Missing Scores in Benchmarks Anas Himmi Ekhine Irurozki Nathan Noiry Stéphan Clémençon Pierre Colombo 31 5 0 17 May 2023
Investigating Multi-source Active Learning for Natural Language Inference Ard Snijders Douwe Kiela Katerina Margatina 24 7 0 14 Feb 2023
Language model acceptability judgements are not always robust to context Koustuv Sinha Jon Gauthier Aaron Mueller Kanishka Misra Keren Fuentes R. Levy Adina Williams 21 17 0 18 Dec 2022
Are Prompt-based Models Clueless? Pride Kavumba Ryo Takahashi Yusuke Oda VLM 139 13 0 19 May 2022
Naturalistic Causal Probing for Morpho-Syntax Afra Amini Tiago Pimentel Clara Meister Ryan Cotterell MILM 103 18 0 14 May 2022
Better Language Model with Hypernym Class Prediction Richard He Bai Tong Wang Alessandro Sordoni Peng Shi 81 15 0 21 Mar 2022
PromptSource: An Integrated Development Environment and Repository for Natural Language Prompts Stephen H. Bach Victor Sanh Zheng-Xin Yong Albert Webson Colin Raffel ... Khalid Almubarak Xiangru Tang Dragomir R. Radev Mike Tian-Jian Jiang Alexander M. Rush VLM 225 338 0 02 Feb 2022
A Unified Speaker Adaptation Approach for ASR Yingzhu Zhao Chongjia Ni C. Leung Shafiq R. Joty Chng Eng Siong B. Ma CLL 92 9 0 16 Oct 2021
Sharpness-Aware Minimization Improves Language Model Generalization Dara Bahri H. Mobahi Yi Tay 127 98 0 16 Oct 2021
ASPECTNEWS: Aspect-Oriented Summarization of News Documents Ojas Ahuja Jiacheng Xu A. Gupta Kevin Horecka Greg Durrett 84 46 0 15 Oct 2021
Multitask Prompted Training Enables Zero-Shot Task Generalization Victor Sanh Albert Webson Colin Raffel Stephen H. Bach Lintang Sutawika ... T. Bers Stella Biderman Leo Gao Thomas Wolf Alexander M. Rush LRM 213 1,657 0 15 Oct 2021
BBQ: A Hand-Built Bias Benchmark for Question Answering Alicia Parrish Angelica Chen Nikita Nangia Vishakh Padmakumar Jason Phang Jana Thompson Phu Mon Htut Sam Bowman 223 367 0 15 Oct 2021
Why don't people use character-level machine translation? Jindrich Libovický Helmut Schmid Alexander Fraser 65 28 0 15 Oct 2021
Zero-Shot Dense Retrieval with Momentum Adversarial Domain Invariant Representations Ji Xin Chenyan Xiong A. Srinivasan Ankita Sharma Damien Jose Paul N. Bennett VLM 83 41 0 14 Oct 2021
LAGr: Labeling Aligned Graphs for Improving Systematic Generalization in Semantic Parsing Dora Jambor Dzmitry Bahdanau 105 9 0 14 Oct 2021
Causal Transformers Perform Below Chance on Recursive Nested Constructions, Unlike Humans Yair Lakretz T. Desbordes Dieuwke Hupkes S. Dehaene 233 11 0 14 Oct 2021
Situated Dialogue Learning through Procedural Environment Generation Prithviraj Ammanabrolu Renee Jia Mark O. Riedl 109 14 0 07 Oct 2021
Generalization in NLI: Ways (Not) To Go Beyond Simple Heuristics Prajjwal Bhargava Aleksandr Drozd Anna Rogers 98 101 0 04 Oct 2021
JuriBERT: A Masked-Language Model Adaptation for French Legal Text S. Douka Hadi Abdine Michalis Vazirgiannis Rajaa El Hamdani David Restrepo Amariles AILaw 87 31 0 04 Oct 2021
LexGLUE: A Benchmark Dataset for Legal Language Understanding in English Ilias Chalkidis Abhik Jana D. Hartung M. Bommarito Ion Androutsopoulos Daniel Martin Katz Nikolaos Aletras AILaw ELM 130 248 0 03 Oct 2021
Inducing Transformer's Compositional Generalization Ability via Auxiliary Sequence Prediction Tasks Yichen Jiang Joey Tianyi Zhou 98 25 0 30 Sep 2021
Single-dataset Experts for Multi-dataset Question Answering Dan Friedman Ben Dodge Danqi Chen MoMe 132 26 0 28 Sep 2021
On Generalization in Coreference Resolution Shubham Toshniwal Patrick Xia Sam Wiseman Karen Livescu Kevin Gimpel 118 36 0 20 Sep 2021
RnG-KBQA: Generation Augmented Iterative Ranking for Knowledge Base Question Answering Xi Ye Semih Yavuz Kazuma Hashimoto Yingbo Zhou Caiming Xiong 146 138 0 17 Sep 2021
Frequency Effects on Syntactic Rule Learning in Transformers Jason W. Wei Dan Garrette Tal Linzen Ellie Pavlick 88 62 0 14 Sep 2021
Effectiveness of Pre-training for Few-shot Intent Classification Haode Zhang Yuwei Zhang Li-Ming Zhan Jiaxin Chen Guangyuan Shi Xiao-Ming Wu Albert Y. S. Lam VLM 77 44 0 13 Sep 2021
Honey or Poison? Solving the Trigger Curse in Few-shot Event Detection via Causal Intervention Jiawei Chen Hongyu Lin Xianpei Han Le Sun 95 27 0 13 Sep 2021
Raise a Child in Large Language Model: Towards Effective and Generalizable Fine-tuning Runxin Xu Fuli Luo Zhiyuan Zhang Chuanqi Tan Baobao Chang Songfang Huang Fei Huang LRM 148 178 0 13 Sep 2021
Exploring Underexplored Limitations of Cross-Domain Text-to-SQL Generalization Yujian Gan Xinyun Chen Matthew Purver 77 78 0 11 Sep 2021
Entity-Based Knowledge Conflicts in Question Answering Shayne Longpre Kartik Perisetla Anthony Chen Nikhil Ramesh Chris DuBois Sameer Singh HILM 245 237 0 10 Sep 2021
Low-Resource Dialogue Summarization with Domain-Agnostic Multi-Source Pretraining Yicheng Zou Bolin Zhu Xingwu Hu Tao Gui Qi Zhang 86 31 0 09 Sep 2021
The paradox of the compositionality of natural language: a neural machine translation case study Verna Dankers Elia Bruni Dieuwke Hupkes CoGe 162 75 0 12 Aug 2021
(Un)solving Morphological Inflection: Lemma Overlap Artificially Inflates Models' Performance Omer Goldman David Guriel Reut Tsarfaty 92 28 0 12 Aug 2021
FLEX: Unifying Evaluation for Few-Shot NLP Jonathan Bragg Arman Cohan Kyle Lo Iz Beltagy 205 104 0 15 Jul 2021
On Compositional Generalization of Neural Machine Translation Yafu Li Yongjing Yin Yulong Chen Yue Zhang 156 44 0 31 May 2021