We Need to Talk About Random Splits

1 May 2020

Papers citing "We Need to Talk About Random Splits"

32 / 32 papers shown

Title
Exploring Diachronic and Diatopic Changes in Dialect Continua: Tasks, Datasets and Challenges Melis Çelikkol Lydia Körber Wei Zhao 19 0 0 04 Jul 2024
Gaining More Insight into Neural Semantic Parsing with Challenging Benchmarks Xiao Zhang Chunliu Wang Rik van Noord Johan Bos 28 3 0 12 Apr 2024
A Comprehensive Review of Machine Learning Advances on Data Change: A Cross-Field Perspective Jeng-Lin Li Chih-Fan Hsu Ming-Ching Chang Wei-Chao Chen OOD 44 2 0 20 Feb 2024
Latent Feature-based Data Splits to Improve Generalisation Evaluation: A Hate Speech Detection Case Study Maike Zufle Verna Dankers Ivan Titov 42 0 0 16 Nov 2023
On Using Distribution-Based Compositionality Assessment to Evaluate Compositional Generalisation in Machine Translation Anssi Moisio Mathias Creutz M. Kurimo CoGe 19 1 0 14 Nov 2023
Examining the Limitations of Computational Rumor Detection Models Trained on Static Datasets Yida Mu Xingyi Song Kalina Bontcheva Nikolaos Aletras 22 3 0 20 Sep 2023
On Evaluation of Document Classification using RVL-CDIP Stefan Larson Gordon Lim Kevin Leach 26 3 0 21 Jun 2023
On the Limitations of Simulating Active Learning Katerina Margatina Nikolaos Aletras 31 11 0 21 May 2023
What's the Meaning of Superhuman Performance in Today's NLU? Simone Tedeschi Johan Bos T. Declerck Jan Hajic Daniel Hershcovich ... Simon Krek Steven Schockaert Rico Sennrich Ekaterina Shutova Roberto Navigli ELM LM&MA VLM ReLM LRM 34 26 0 15 May 2023
Dynamic Benchmarking of Masked Language Models on Temporal Concept Drift with Multiple Views Katerina Margatina Shuai Wang Yogarshi Vyas Neha Ann John Yassine Benajiba Miguel Ballesteros 17 15 0 23 Feb 2023
Predicting Long-Term Citations from Short-Term Linguistic Influence Sandeep Soni David Bamman Jacob Eisenstein 17 2 0 24 Oct 2022
Testing Independence of Exchangeable Random Variables Marcus Hutter 21 2 0 22 Oct 2022
Sequential Learning Of Neural Networks for Prequential MDL J. Bornschein Yazhe Li Marcus Hutter AI4TS 27 6 0 14 Oct 2022
State-of-the-art generalisation research in NLP: A taxonomy and review Dieuwke Hupkes Mario Giulianelli Verna Dankers Mikel Artetxe Yanai Elazar ... Leila Khalatbari Maria Ryskina Rita Frieske Ryan Cotterell Zhijing Jin 114 93 0 06 Oct 2022
The Fragility of Multi-Treebank Parsing Evaluation I. Alonso-Alonso David Vilares Carlos Gómez-Rodríguez 17 1 0 14 Sep 2022
Investigating data partitioning strategies for crosslinguistic low-resource ASR evaluation Zoey Liu J. Spence Emily Tucker Prudhommeaux 24 8 0 26 Aug 2022
What do we Really Know about State of the Art NER? Sowmya Vajjala Ramya Balasubramaniam 19 15 0 29 Apr 2022
What do You Mean by Relation Extraction? A Survey on Datasets and Study on Scientific Relation Classification Elisa Bassignana Barbara Plank 21 20 0 28 Apr 2022
You Are What You Write: Preserving Privacy in the Era of Large Language Models Richard Plant V. Giuffrida Dimitra Gkatzia PILM 20 19 0 20 Apr 2022
FairLex: A Multilingual Benchmark for Evaluating Fairness in Legal Text Processing Ilias Chalkidis Tommaso Pasini Shenmin Zhang Letizia Tomada Sebastian Felix Schwemer Anders Søgaard AILaw 37 54 0 14 Mar 2022
Data-driven Model Generalizability in Crosslinguistic Low-resource Morphological Segmentation Zoey Liu Emily Tucker Prudhommeaux 43 4 0 05 Jan 2022
Temporal Effects on Pre-trained Models for Language Processing Tasks Oshin Agarwal A. Nenkova VLM 22 53 0 24 Nov 2021
SynthBio: A Case Study in Human-AI Collaborative Curation of Text Datasets Ann Yuan Daphne Ippolito Vitaly Nikolaev Chris Callison-Burch Andy Coenen Sebastian Gehrmann SyDa 112 20 0 11 Nov 2021
MultiEURLEX -- A multi-lingual and multi-label legal document classification dataset for zero-shot cross-lingual transfer Ilias Chalkidis Manos Fergadiotis Ion Androutsopoulos AILaw 16 106 0 02 Sep 2021
Randomness In Neural Network Training: Characterizing The Impact of Tooling Donglin Zhuang Xingyao Zhang Shuaiwen Leon Song Sara Hooker 25 75 0 22 Jun 2021
Automatic Construction of Evaluation Suites for Natural Language Generation Datasets Simon Mille Kaustubh D. Dhole Saad Mahamood Laura Perez-Beltrachini Varun Gangal Mihir Kale Emiel van Miltenburg Sebastian Gehrmann ELM 39 22 0 16 Jun 2021
SemEval-2021 Task 1: Lexical Complexity Prediction Matthew Shardlow R. Evans Gustavo Henrique Paetzold Marcos Zampieri 19 97 0 01 Jun 2021
Reliability Testing for Natural Language Processing Systems Samson Tan Shafiq R. Joty K. Baxter Araz Taeihagh G. Bennett Min-Yen Kan 15 38 0 06 May 2021
Mind the Gap: Assessing Temporal Generalization in Neural Language Models Angeliki Lazaridou A. Kuncoro E. Gribovskaya Devang Agrawal Adam Liska ... Sebastian Ruder Dani Yogatama Kris Cao Susannah Young Phil Blunsom VLM 30 207 0 03 Feb 2021
What you can cram into a single vector: Probing sentence embeddings for linguistic properties Alexis Conneau Germán Kruszewski Guillaume Lample Loïc Barrault Marco Baroni 201 882 0 03 May 2018
Six Challenges for Neural Machine Translation Philipp Koehn Rebecca Knowles AAML AIMat 221 1,208 0 12 Jun 2017
Effective Approaches to Attention-based Neural Machine Translation Thang Luong Hieu H. Pham Christopher D. Manning 218 7,926 0 17 Aug 2015