Systematic Inequalities in Language Technology Performance across the World's Languages

13 October 2021

Graham Neubig

Papers citing "Systematic Inequalities in Language Technology Performance across the World's Languages"

29 / 29 papers shown

Title
mHumanEval -- A Multilingual Benchmark to Evaluate Large Language Models for Code Generation Nishat Raihan Antonios Anastasopoulos Marcos Zampieri ELM 43 6 0 28 Jan 2025
Building Better: Avoiding Pitfalls in Developing Language Resources when Data is Scarce N. Ousidhoum Meriem Beloucif Saif M. Mohammad 25 1 0 16 Oct 2024
Bilingual Evaluation of Language Models on General Knowledge in University Entrance Exams with Minimal Contamination Eva Sánchez Salido Roser Morante Julio Gonzalo Guillermo Marco Jorge Carrillo-de-Albornoz ... Enrique Amigó Andrés Fernández Alejandro Benito-Santos Adrián Ghajari Espinosa Victor Fresno ELM 39 0 0 19 Sep 2024
Native Design Bias: Studying the Impact of English Nativeness on Language Model Performance Manon Reusens Philipp Borchert Jochen De Weerdt Bart Baesens 36 0 0 25 Jun 2024
SEACrowd: A Multilingual Multimodal Data Hub and Benchmark Suite for Southeast Asian Languages Holy Lovenia Rahmad Mahendra Salsabil Maulana Akbar Lester James Validad Miranda Jennifer Santoso ... Genta Indra Winata Ruochen Zhang Fajri Koto Zheng-Xin Yong Samuel Cahyawijaya 84 9 0 14 Jun 2024
What Drives Performance in Multilingual Language Models? Sina Bagheri Nezhad Ameeta Agrawal LRM 37 9 0 29 Apr 2024
What Is Missing in Multilingual Visual Reasoning and How to Fix It Yueqi Song Simran Khanuja Graham Neubig VLM LRM 87 6 0 03 Mar 2024
When Is Multilinguality a Curse? Language Modeling for 250 High- and Low-Resource Languages Tyler A. Chang Catherine Arnett Zhuowen Tu Benjamin Bergen LRM 30 7 0 15 Nov 2023
A Systematic Study of Performance Disparities in Multilingual Task-Oriented Dialogue Systems Songbo Hu Han Zhou Moy Yuan Milan Gritta Guchun Zhang Ignacio Iacobacci Anna Korhonen Ivan Vulić 28 3 0 19 Oct 2023
BUFFET: Benchmarking Large Language Models for Few-shot Cross-lingual Transfer Akari Asai Sneha Kudugunta Xinyan Velocity Yu Terra Blevins Hila Gonen Machel Reid Yulia Tsvetkov Sebastian Ruder Hannaneh Hajishirzi 31 54 0 24 May 2023
How Good are Commercial Large Language Models on African Languages? Jessica Ojo Kelechi Ogueji 21 5 0 11 May 2023
Train Global, Tailor Local: Minimalist Multilingual Translation into Endangered Languages Zhong Zhou J. Niehues Alexander Waibel 24 0 0 05 May 2023
GMNLP at SemEval-2023 Task 12: Sentiment Analysis with Phylogeny-Based Adapters Md Mahfuz Ibn Alam Ruoyu Xie Fahim Faisal Antonios Anastasopoulos 30 3 0 25 Apr 2023
Language Embeddings Sometimes Contain Typological Generalizations Robert Östling Murathan Kurfali NAI 24 9 0 19 Jan 2023
Dissociating language and thought in large language models Kyle Mahowald Anna A. Ivanova I. Blank Nancy Kanwisher J. Tenenbaum Evelina Fedorenko ELM ReLM 25 209 0 16 Jan 2023
Multi-VALUE: A Framework for Cross-Dialectal English NLP Caleb Ziems William B. Held Jingfeng Yang Jwala Dhamala Rahul Gupta Diyi Yang 31 40 0 15 Dec 2022
Beyond Counting Datasets: A Survey of Multilingual Dataset Construction and Necessary Resources Xinyan Velocity Yu Akari Asai Trina Chatterjee Junjie Hu Eunsol Choi 16 21 0 28 Nov 2022
GLUE-X: Evaluating Natural Language Understanding Models from an Out-of-distribution Generalization Perspective Linyi Yang Shuibai Zhang Libo Qin Yafu Li Yidong Wang Hanmeng Liu Jindong Wang Xingxu Xie Yue Zhang ELM 39 79 0 15 Nov 2022
Bloom Library: Multimodal Datasets in 300+ Languages for a Variety of Downstream Tasks Colin Leong Joshua Nemecek Jacob Mansdorfer Anna Filighera A. Owodunni Daniel Whitenack VLM AI4CE 39 24 0 26 Oct 2022
Some Languages are More Equal than Others: Probing Deeper into the Linguistic Disparity in the NLP World Surangika Ranathunga Nisansa de Silva 31 34 0 16 Oct 2022
Assessing Digital Language Support on a Global Scale Gary F. Simons Abbey L Thomas Chad White ELM 33 14 0 27 Sep 2022
$Meta-X$_{NLG}$: A Meta-Learning Approach Based on Language Clustering for Zero-Shot Cross-Lingual Transfer and Generation$ Meta-X $_{NLG}$ : A Meta-Learning Approach Based on Language Clustering for Zero-Shot Cross-Lingual Transfer and Generation Kaushal Kumar Maurya M. Desarkar 14 8 0 19 Mar 2022
Expanding Pretrained Models to Thousands More Languages via Lexicon-based Adaptation Xinyi Wang Sebastian Ruder Graham Neubig 23 60 0 17 Mar 2022
IGLUE: A Benchmark for Transfer Learning across Modalities, Tasks, and Languages Emanuele Bugliarello Fangyu Liu Jonas Pfeiffer Siva Reddy Desmond Elliott E. Ponti Ivan Vulić MLLM VLM ELM 42 62 0 27 Jan 2022
Dataset Geography: Mapping Language Data to Language Users Fahim Faisal Yinkai Wang Antonios Anastasopoulos 54 23 0 07 Dec 2021
SD-QA: Spoken Dialectal Question Answering for the Real World Fahim Faisal Sharlina Keshava ibn Alam Antonios Anastasopoulos 100 29 0 24 Sep 2021
When Being Unseen from mBERT is just the Beginning: Handling New Languages With Multilingual Language Models Benjamin Muller Antonis Anastasopoulos Benoît Sagot Djamé Seddah LRM 126 165 0 24 Oct 2020
MLQA: Evaluating Cross-lingual Extractive Question Answering Patrick Lewis Barlas Oğuz Ruty Rinott Sebastian Riedel Holger Schwenk ELM 246 491 0 16 Oct 2019
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 297 6,956 0 20 Apr 2018