Anything Goes? A Crosslinguistic Study of (Im)possible Language Learning in LMs

26 February 2025

Papers citing "Anything Goes? A Crosslinguistic Study of (Im)possible Language Learning in LMs"

33 / 33 papers shown

Title
Can Language Models Learn Typologically Implausible Languages? Tianyang Xu Tatsuki Kuribayashi Yohei Oseki Ryan Cotterell Alex Warstadt 93 1 0 17 Feb 2025
Large Language Models Are Human-Like Internally Tatsuki Kuribayashi Yohei Oseki Souhaib Ben Taieb Kentaro Inui Timothy Baldwin 116 5 0 03 Feb 2025
Why do language models perform worse for morphologically complex languages? Catherine Arnett Benjamin Bergen 77 10 0 21 Nov 2024
Surprise! Uniform Information Density Isn't the Whole Story: Predicting Surprisal Contours in Long-form Discourse Eleftheria Tsipidi Franz Nowak Ryan Cotterell Ethan Gotlieb Wilcox Mario Giulianelli Alex Warstadt 55 4 0 21 Oct 2024
Language Models as Models of Language Raphaël Millière 21 19 0 13 Aug 2024
What Languages are Easy to Language-Model? A Perspective from Learning Probabilistic Regular Languages Nadav Borenstein Anej Svete R. Chan Josef Valvoda Franz Nowak Isabelle Augenstein Eleanor Chodroff Ryan Cotterell 56 13 0 06 Jun 2024
Introducing cosmosGPT: Monolingual Training for Turkish Language Models Himmet Toprak Kesgin M. K. Yuce Eren Dogan M. E. Uzun Atahan Uz H. E. Seyrek Ahmed Zeer M. Amasyalı 67 9 0 26 Apr 2024
Language in Vivo vs. in Silico: Size Matters but Larger Language Models Still Do Not Comprehend Language on a Par with Humans Vittoria Dentella Fritz Guenther Evelina Leivada ELM 69 1 0 23 Apr 2024
Unpacking Tokenization: Evaluating Text Compression and its Correlation with Model Performance Omer Goldman Avi Caciularu Matan Eyal Kris Cao Idan Szpektor Reut Tsarfaty 53 24 0 10 Mar 2024
Tokenization Is More Than Compression Craig W. Schmidt Varshini Reddy Haoran Zhang Alec Alameddine Omri Uzan Yuval Pinter Chris Tanner 72 34 0 28 Feb 2024
GlórIA -- A Generative and Open Large Language Model for Portuguese Ricardo Lopes João Magalhães David Semedo 49 8 0 20 Feb 2024
A Family of Pretrained Transformer Language Models for Russian Dmitry Zmitrovich Alexander Abramov Andrey Kalmykov Maria Tikhonova Ekaterina Taktasheva ... Vitalii Kadulin Sergey Markov Tatiana Shavrina Vladislav Mikhailov Alena Fenogenova 48 26 0 19 Sep 2023
Testing the Predictions of Surprisal Theory in 11 Languages Ethan Gotlieb Wilcox Tiago Pimentel Clara Meister Ryan Cotterell R. Levy LRM 78 66 0 07 Jul 2023
Tokenization and the Noiseless Channel Vilém Zouhar Clara Meister Juan Luis Gastaldi Li Du Mrinmaya Sachan Ryan Cotterell 48 32 0 29 Jun 2023
A Cross-Linguistic Pressure for Uniform Information Density in Word Order T. H. Clark Clara Meister Tiago Pimentel Michael Hahn Ryan Cotterell Richard Futrell Saarland University 55 13 0 06 Jun 2023
Transformer-Based Language Model Surprisal Predicts Human Reading Times Best with About Two Billion Training Tokens Byung-Doh Oh William Schuler 56 27 0 22 Apr 2023
XLM-V: Overcoming the Vocabulary Bottleneck in Multilingual Masked Language Models Davis Liang Hila Gonen Yuning Mao Rui Hou Naman Goyal Marjan Ghazvininejad Luke Zettlemoyer Madian Khabsa 30 73 0 25 Jan 2023
Why Does Surprisal From Larger Transformer-Based Language Models Provide a Poorer Fit to Human Reading Times? Byung-Doh Oh William Schuler 38 108 0 23 Dec 2022
Structural generalization is hard for sequence-to-sequence models Yuekun Yao Alexander Koller 59 22 0 24 Oct 2022
PAGnol: An Extra-Large French Generative Model Julien Launay E. L. Tommasone B. Pannier Franccois Boniface A. Chatelain Alessandro Cappelli Iacopo Poli Djamé Seddah AILaw MoE AI4CE 63 8 0 16 Oct 2021
Revisiting the Uniform Information Density Hypothesis Clara Meister Tiago Pimentel Patrick Haller Lena Jäger Ryan Cotterell R. Levy 69 73 0 23 Sep 2021
AraGPT2: Pre-Trained Transformer for Arabic Language Generation Wissam Antoun Fady Baly Hazem M. Hajj VLM 39 104 0 31 Dec 2020
COGS: A Compositional Generalization Challenge Based on Semantic Interpretation Najoung Kim Tal Linzen CoGe 41 276 0 12 Oct 2020
Recurrent Neural Network Language Models Always Learn English-Like Relative Clause Attachment Forrest Davis Marten van Schijndel 31 23 0 01 May 2020
Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation Nils Reimers Iryna Gurevych 69 1,014 0 21 Apr 2020
Stanza: A Python Natural Language Processing Toolkit for Many Human Languages Peng Qi Yuhao Zhang Yuhui Zhang Jason Bolton Christopher D. Manning AI4TS 232 1,681 0 16 Mar 2020
CCMatrix: Mining Billions of High-Quality Parallel Sentences on the WEB Holger Schwenk Guillaume Wenzek Sergey Edunov Edouard Grave Armand Joulin 59 259 0 10 Nov 2019
CCAligned: A Massive Collection of Cross-Lingual Web-Document Pairs Ahmed El-Kishky Vishrav Chaudhary Francisco Guzman Philipp Koehn 72 199 0 10 Nov 2019
What Kind of Language Is Hard to Language-Model? Sabrina J. Mielke Ryan Cotterell Kyle Gorman Brian Roark Jason Eisner 59 78 0 11 Jun 2019
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.1K 93,936 0 11 Oct 2018
Recurrent Neural Networks in Linguistic Theory: Revisiting Pinker and Prince (1988) and the Past Tense Debate Christo Kirov Ryan Cotterell 26 79 0 12 Jul 2018
Are All Languages Equally Hard to Language-Model? Ryan Cotterell Sabrina J. Mielke Jason Eisner Brian Roark 40 95 0 10 Jun 2018
Bag of Tricks for Efficient Text Classification Armand Joulin Edouard Grave Piotr Bojanowski Tomas Mikolov VLM 99 4,596 0 06 Jul 2016