v1v2v3v4v5 (latest)

XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization

24 March 2020

Graham Neubig

Papers citing "XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization"

50 / 666 papers shown

Title
Mitigating Negative Interference in Multilingual Sequential Knowledge Editing through Null-Space Constraints Wei Sun Tingyu Qu Mingxiao Li Jesse Davis Marie-Francine Moens KELM 127 0 0 12 Jun 2025
A Culturally-Rich Romanian NLP Dataset from "Who Wants to Be a Millionaire?" Videos Alexandru-Gabriel Ganea Antonia-Adelina Popovici Adrian-Marius Dumitran 51 0 0 06 Jun 2025
Charting the Landscape of African NLP: Mapping Progress and Shaping the Road Ahead Jesujoba Oluwadara Alabi Michael A. Hedderich David Ifeoluwa Adelani Dietrich Klakow 131 0 0 27 May 2025
Fluent but Culturally Distant: Can Regional Training Teach Cultural Understanding? Dhruv Agarwal Anya Shukla Sunayana Sitaram Aditya Vashistha 58 0 0 25 May 2025
The Unreasonable Effectiveness of Model Merging for Cross-Lingual Transfer in LLMs Lucas Bandarkar Nanyun Peng MoMe LRM 51 0 0 23 May 2025
Evaluating Large Language Model with Knowledge Oriented Language Specific Simple Question Answering Bowen Jiang Runchuan Zhu Jiang Wu Zinco Jiang Yifan He ... Haote Yang Songyang Zhang Dahua Lin Lijun Wu Conghui He ELM 56 0 0 22 May 2025
On Multilingual Encoder Language Model Compression for Low-Resource Languages Daniil Gurgurov Michal Gregor Josef van Genabith Simon Ostermann 190 0 0 22 May 2025
MAPS: A Multilingual Benchmark for Global Agent Performance and Security Omer Hofman Oren Rachmil Shamik Bose Vikas Pahuja Jonathan Brokman Toshiya Shimizu Trisha Starostina Kelly Marchisio Seraphina Goldfarb-Tarrant Roman Vainshtein 63 0 0 21 May 2025
Cross-Linguistic Transfer in Multilingual NLP: The Role of Language Families and Morphology Ajitesh Bankula Praney Bankula 13 0 0 20 May 2025
JNLP at SemEval-2025 Task 11: Cross-Lingual Multi-Label Emotion Detection Using Generative Models Jieying Xue Phuong Minh Nguyen Minh Le Nguyen Xin Liu 48 0 0 19 May 2025
The Devil Is in the Word Alignment Details: On Translation-Based Cross-Lingual Transfer for Token Classification Tasks Benedikt Ebing Goran Glavaš 93 0 0 15 May 2025
What Causes Knowledge Loss in Multilingual Language Models? Maria Khelli Samuel Cahyawijaya Ayu Purwarianti Genta Indra Winata CLL 121 0 0 29 Apr 2025
Long-context Non-factoid Question Answering in Indic Languages Ritwik Mishra R. Shah Ponnurangam Kumaraguru 76 0 0 18 Apr 2025
Bias Beyond English: Evaluating Social Bias and Debiasing Methods in a Low-Resource Setting Ej Zhou Weiming Lu 66 0 0 15 Apr 2025
Cross-Document Cross-Lingual NLI via RST-Enhanced Graph Fusion and Interpretability Prediction Mengying Yuan Wenhao Wang Zixuan Wang Yujie Huang Kangli Wei Fei Li Chong Teng Donghong Ji 121 0 0 11 Apr 2025
Enhancing NER Performance in Low-Resource Pakistani Languages using Cross-Lingual Data Augmentation Toqeer Ehsan Thamar Solorio 482 0 0 07 Apr 2025
Language-specific Neurons Do Not Facilitate Cross-Lingual Transfer Soumen Kumar Mondal Sayambhu Sen Abhishek Singhania Preethi Jyothi 94 0 0 21 Mar 2025
Fragile Mastery: Are Domain-Specific Trade-Offs Undermining On-Device Language Models? Basab Jha Firoj Paudel 89 0 0 16 Mar 2025
TLUE: A Tibetan Language Understanding Evaluation Benchmark Fan Gao Cheng Huang Nyima Tashi Xiangxiang Wang Thupten Tsering ... Gadeng Luosang Rinchen Dongrub Dorje Tashi Xiao Feng Yongbin Yu ELM 257 2 0 15 Mar 2025
LAG-MMLU: Benchmarking Frontier LLM Understanding in Latvian and Giriama Naome A. Etori Kevin Lu Randu Karisa Arturs Kanepajs LRM ELM 495 0 0 14 Mar 2025
On the Acquisition of Shared Grammatical Representations in Bilingual Language Models Catherine Arnett Tyler A. Chang J. Michaelov Benjamin Bergen 92 0 0 05 Mar 2025
MiLiC-Eval: Benchmarking Multilingual LLMs for China's Minority Languages Chen Zhang Mingxu Tao Zhiyuan Liao Yansong Feng 80 0 0 03 Mar 2025
ECLeKTic: a Novel Challenge Set for Evaluation of Cross-Lingual Knowledge Transfer Omer Goldman Uri Shaham Dan Malkin Sivan Eiger Avinatan Hassidim ... Shruti Rijhwani Laura Rimell Idan Szpektor Reut Tsarfaty Matan Eyal 125 7 0 28 Feb 2025
XCOMPS: A Multilingual Benchmark of Conceptual Minimal Pairs Linyang He Ercong Nie Sukru Samet Dindar Arsalan Firoozi Adrian Nicolas Florea ... Haotian Ye Jonathan R. Brennan Helmut Schmid Hinrich Schütze Nima Mesgarani 107 1 0 27 Feb 2025
Where Are We? Evaluating LLM Performance on African Languages Ife Adebara Hawau Olamide Toyin Nahom Tesfu Ghebremichael AbdelRahim Elmadany Muhammad Abdul-Mageed 136 2 0 26 Feb 2025
NusaAksara: A Multimodal and Multilingual Benchmark for Preserving Indonesian Indigenous Scripts Muhammad Farid Adilazuarda M. Wijanarko Lucky Susanto Khumaisa Nuráini Derry Wijaya Alham Fikri Aji 119 0 0 25 Feb 2025
Language Models' Factuality Depends on the Language of Inquiry Tushar Aggarwal Kumar Tanmay Ayush Agrawal Kumar Ayush Hamid Palangi Paul Pu Liang HILM KELM 121 2 0 25 Feb 2025
Steering into New Embedding Spaces: Analyzing Cross-Lingual Alignment Induced by Model Interventions in Multilingual Language Models Anirudh Sundar Sinead Williamson Katherine Metcalf B. Theobald Skyler Seto Masha Fedzechkina LLMSV 139 1 0 24 Feb 2025
Batayan: A Filipino NLP benchmark for evaluating Large Language Models Jann Railey Montalan Jimson Paulo Layacan David Demitri Africa Richell Isaiah Flores Michael T. Lopez II Theresa Denise Magsajo Anjanette Cayabyab William-Chandra Tjhi 71 0 0 19 Feb 2025
Blessing of Multilinguality: A Systematic Analysis of Multilingual In-Context Learning Yilei Tu Andrew Xue Freda Shi 92 0 0 17 Feb 2025
A Large-Scale Benchmark for Vietnamese Sentence Paraphrases Sang Quang Nguyen Kiet Van Nguyen 149 0 0 11 Feb 2025
Beyond Literal Token Overlap: Token Alignability for Multilinguality Katharina Hämmerl Tomasz Limisiewicz Jindrich Libovický Alexander Fraser 75 0 0 10 Feb 2025
Revisiting Projection-based Data Transfer for Cross-Lingual Named Entity Recognition in Low-Resource Languages Andrei Politov Oleh Shkalikov René Jäkel Michael Färber 136 0 0 30 Jan 2025
IndicMMLU-Pro: Benchmarking Indic Large Language Models on Multi-Task Language Understanding Sankalp KJ Ashutosh Kumar Laxmaan Balaji Nikunj Kotecha Vinija Jain Aman Chadha S. Bhaduri ELM 469 1 0 27 Jan 2025
Can linguists better understand DNA? Wang Liang 148 1 0 20 Jan 2025
Gradient-Based Multi-Objective Deep Learning: Algorithms, Theories, Applications, and Beyond Weiyu Chen Xiaoyuan Zhang Baijiong Lin Xi Lin Han Zhao Qingfu Zhang James T. Kwok 176 5 0 19 Jan 2025
LinguaLIFT: An Effective Two-stage Instruction Tuning Framework for Low-Resource Language Reasoning Hongbin Zhang Kai Chen Xuefeng Bai Yang Xiang Min Zhang 130 0 0 17 Dec 2024
PolyIPA -- Multilingual Phoneme-to-Grapheme Conversion Model Davor Lauc 109 0 0 12 Dec 2024
SailCompass: Towards Reproducible and Robust Evaluation for Southeast Asian Languages Jia Guo Longxu Dou Guangtao Zeng Stanley Kok Wei Lu Qian Liu ELM LRM 129 2 0 02 Dec 2024
ChemTEB: Chemical Text Embedding Benchmark, an Overview of Embedding Models Performance & Efficiency on a Specific Domain Ali Shiraee Kasmaee Mohammad Khodadad Mohammad Arshi Saloot Nick Sherck Stephen Dokas H. Mahyar Soheila Samiee ELM 642 2 0 30 Nov 2024
INCLUDE: Evaluating Multilingual Language Understanding with Regional Knowledge Angelika Romanou Negar Foroutan Anna Sotnikova Zeming Chen Sree Harsha Nelaturu ... Mike Zhang Imanol Schlag Marzieh Fadaee Sara Hooker Antoine Bosselut ELM 181 8 0 29 Nov 2024
Dynamic Strategy Planning for Efficient Question Answering with Large Language Models Tanmay Parekh Pradyot Prakash Alexander Radovic Akshay Shekher Denis Savenkov LRM 382 2 0 30 Oct 2024
Thank You, Stingray: Multilingual Large Language Models Can Not (Yet) Disambiguate Cross-Lingual Word Sense Samuel Cahyawijaya Ruochen Zhang Holy Lovenia Jan Christian Blaise Cruz Elisa Gilbert Hiroki Nomoto Alham Fikri Aji LRM 127 0 0 28 Oct 2024
SandboxAQ's submission to MRL 2024 Shared Task on Multi-lingual Multi-task Information Retrieval Isidora Chara Tourni Sayontan Ghosh Brenda Miao Constantijn van der Poel LRM 80 0 0 28 Oct 2024
Bridge-Coder: Unlocking LLMs' Potential to Overcome Language Gaps in Low-Resource Code Jipeng Zhang Jianshu Zhang Yuanzhe Li Renjie Pi Boyao Wang Runtao Liu Ziqiang Zheng Tong Zhang 62 0 0 24 Oct 2024
VL-GLUE: A Suite of Fundamental yet Challenging Visuo-Linguistic Reasoning Tasks Shailaja Keyur Sampat Mutsumi Nakamura Shankar Kailas Kartik Aggarwal Mandy Zhou Yezhou Yang Chitta Baral MLLM CoGe ReLM VLM LRM 83 0 0 17 Oct 2024
Cross-Lingual Auto Evaluation for Assessing Multilingual LLMs Sumanth Doddapaneni Mohammed Safi Ur Rahman Khan Dilip Venkatesh Raj Dabre Anoop Kunchukuttan Mitesh M. Khapra ELM 72 1 0 17 Oct 2024
MEXA: Multilingual Evaluation of English-Centric LLMs via Cross-Lingual Alignment Amir Hossein Kargaran Ali Modarressi Nafiseh Nikeghbal Jana Diesner François Yvon Hinrich Schütze ELM 106 7 0 08 Oct 2024
Task Diversity Shortens the ICL Plateau Jaeyeon Kim Sehyun Kwon Joo Young Choi Jongho Park Jaewoong Cho Jason D. Lee Ernest K. Ryu MoMe 101 3 0 07 Oct 2024
How Transliterations Improve Crosslingual Alignment Yihong Liu Mingyang Wang Amir Hossein Kargaran Ayyoob Imani Orgest Xhelili Haotian Ye Chunlan Ma François Yvon Hinrich Schütze 96 4 0 25 Sep 2024