v1v2v3v4v5 (latest)

XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization

24 March 2020

Graham Neubig

Papers citing "XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization"

50 / 666 papers shown

Title
Can current NLI systems handle German word order? Investigating language model performance on a new German challenge set of minimal pairs Ines Reinig K. Markert 53 0 0 07 Jun 2023
Multilingual Clinical NER: Translation or Cross-lingual Transfer? X. Fontaine Félix Gaschi Parisa Rastin Y. Toussaint 99 10 0 07 Jun 2023
XSemPLR: Cross-Lingual Semantic Parsing in Multiple Natural Languages and Meaning Representations Yusen Zhang Jun Wang Zhiguo Wang Rui Zhang VLM 125 9 0 07 Jun 2023
Cross-Lingual Transfer with Target Language-Ready Task Adapters Marinela Parović Alan Ansell Ivan Vulić Anna Korhonen 97 11 0 05 Jun 2023
bgGLUE: A Bulgarian General Language Understanding Evaluation Benchmark Momchil Hardalov Pepa Atanasova Todor Mihaylov G. Angelova K. Simov P. Osenova Ves Stoyanov Ivan Koychev Preslav Nakov Dragomir R. Radev ELM FedML 81 4 0 04 Jun 2023
Exploring Anisotropy and Outliers in Multilingual Language Models for Cross-Lingual Semantic Sentence Similarity Katharina Hämmerl Alina Fastowski Jindrich Libovický Alexander Fraser 121 7 0 01 Jun 2023
What does the Failure to Reason with "Respectively" in Zero/Few-Shot Settings Tell Us about Language Models? Ruixiang Cui Seolhwa Lee Daniel Hershcovich Anders Søgaard 52 2 0 31 May 2023
Translation-Enhanced Multilingual Text-to-Image Generation Yaoyiran Li Ching-Yun Chang Stephen Rawls Ivan Vulić Anna Korhonen 57 8 0 30 May 2023
A Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark Datasets Md Tahmid Rahman Laskar M Saiful Bari Mizanur Rahman Md Amran Hossen Bhuiyan Shafiq Joty J. Huang LM&MA ELM ALM 127 193 0 29 May 2023
Why Does Zero-Shot Cross-Lingual Generation Fail? An Explanation and a Solution Tianjian Li Kenton W. Murray 104 26 0 27 May 2023
Free Lunch: Robust Cross-Lingual Transfer via Model Checkpoint Averaging Fabian David Schmidt Ivan Vulić Goran Glavaš 75 9 0 26 May 2023
Towards a Common Understanding of Contributing Factors for Cross-Lingual Transfer in Multilingual Language Models: A Review Fred Philippy Siwen Guo Shohreh Haddadan LRM 72 37 0 26 May 2023
Revisiting non-English Text Simplification: A Unified Multilingual Benchmark Michael Joseph Ryan Tarek Naous Wei Xu 87 26 0 25 May 2023
Dolphin: A Challenging and Diverse Benchmark for Arabic NLG El Moatez Billah Nagoudi AbdelRahim Elmadany Ahmed Oumar El-Shangiti Muhammad Abdul-Mageed LM&MA 83 20 0 24 May 2023
BUFFET: Benchmarking Large Language Models for Few-shot Cross-lingual Transfer Akari Asai Sneha Kudugunta Xinyan Velocity Yu Terra Blevins Hila Gonen Machel Reid Yulia Tsvetkov Sebastian Ruder Hannaneh Hajishirzi 122 63 0 24 May 2023
GlobalBench: A Benchmark for Global Progress in Natural Language Processing Yueqi Song Catherine Cui Simran Khanuja Pengfei Liu Fahim Faisal ... Alham Fikri Aji Samuel Cahyawijaya Yulia Tsvetkov Antonios Anastasopoulos Graham Neubig 101 9 0 24 May 2023
Having Beer after Prayer? Measuring Cultural Bias in Large Language Models Tarek Naous Michael Joseph Ryan Alan Ritter Wei Xu 117 96 0 23 May 2023
Revisiting Machine Translation for Cross-lingual Classification Mikel Artetxe Vedanuj Goswami Shruti Bhosale Angela Fan Luke Zettlemoyer LRM 102 39 0 23 May 2023
mmT5: Modular Multilingual Pre-Training Solves Source Language Hallucinations Jonas Pfeiffer Francesco Piccinno Massimo Nicosia Xinyi Wang Machel Reid Sebastian Ruder VLM LRM 98 31 0 23 May 2023
$μ$ PLAN: Summarizing using a Content Plan as Cross-Lingual Bridge Fantine Huot Joshua Maynez Chris Alberti Reinald Kim Amplayo Priyanka Agrawal Constanza Fierro Shashi Narayan Mirella Lapata 105 7 0 23 May 2023
Out-of-Distribution Generalization in Text Classification: Past, Present, and Future Linyi Yang Yangqiu Song Xuan Ren Chenyang Lyu Yidong Wang Lingqiao Liu Jindong Wang Jennifer Foster Yue Zhang OOD 131 3 0 23 May 2023
mPLM-Sim: Better Cross-Lingual Similarity and Transfer in Multilingual Pretrained Language Models Peiqin Lin Chengzhi Hu Zheyu Zhang André F. T. Martins Hinrich Schütze 77 1 0 23 May 2023
Understanding and Mitigating Spurious Correlations in Text Classification with Neighborhood Analysis Oscar Chew Hsuan-Tien Lin Kai-Wei Chang Kuan-Hao Huang 90 6 0 23 May 2023
mPMR: A Multilingual Pre-trained Machine Reader at Scale Weiwen Xu Xin Li Wai Lam Lidong Bing LRM 66 5 0 23 May 2023
Detecting and Mitigating Hallucinations in Multilingual Summarisation Yifu Qiu Yftah Ziser Anna Korhonen Edoardo Ponti Shay B. Cohen HILM 134 46 0 23 May 2023
Improving Self-training for Cross-lingual Named Entity Recognition with Contrastive and Prototype Learning Ran Zhou Xin Li Lidong Bing Min Zhang Steven C. H. Hoi 97 19 0 23 May 2023
Translation and Fusion Improves Zero-shot Cross-lingual Information Extraction Yang Chen Vedaant Shah Alan Ritter 108 4 0 23 May 2023
Extrapolating Multilingual Understanding Models as Multilingual Generators Bohong Wu Fei Yuan Hai Zhao Lei Li Jingjing Xu AI4CE 82 2 0 22 May 2023
DUMB: A Benchmark for Smart Evaluation of Dutch Models Wietse de Vries Martijn B. Wieling Malvina Nissim ELM ALM MoE 62 6 0 22 May 2023
Is Translation Helpful? An Empirical Analysis of Cross-Lingual Transfer in Low-Resource Dialog Generation Lei Shen Shuai Yu Xiaoyu Shen 44 0 0 21 May 2023
Glot500: Scaling Multilingual Corpora and Language Models to 500 Languages Ayyoob Imani Peiqin Lin Amir Hossein Kargaran Silvia Severini Masoud Jalili Sabet ... Chunlan Ma Helmut Schmid André F. T. Martins François Yvon Hinrich Schütze ALM LRM 139 107 0 20 May 2023
ESCOXLM-R: Multilingual Taxonomy-driven Pre-training for the Job Market Domain Mike Zhang Rob van der Goot Barbara Plank 63 16 0 20 May 2023
XTREME-UP: A User-Centric Scarce-Data Benchmark for Under-Represented Languages Sebastian Ruder J. Clark Alexander Gutkin Mihir Kale Min Ma ... Dan Garrette R. Ingle Melvin Johnson Dmitry Panteleev Partha P. Talukdar ELM 87 40 0 19 May 2023
Separating form and meaning: Using self-consistency to quantify task understanding across multiple senses Xenia Ohmer Elia Bruni Dieuwke Hupkes LRM 113 16 0 19 May 2023
CCT-Code: Cross-Consistency Training for Multilingual Clone Detection and Code Search Nikita Sorokin Dmitry Abulkhanov Sergey I. Nikolenko Valentin Malykh 66 3 0 19 May 2023
Analyzing and Reducing the Performance Gap in Cross-Lingual Transfer with Fine-tuning Slow and Fast Yiduo Guo Yaobo Liang Dongyan Zhao Bin Liu Du Nan CLL 81 1 0 19 May 2023
Massively Multi-Lingual Event Understanding: Extraction, Visualization, and Search Chris Jenkins Shantanu Agarwal Joel Barry Steven Fincke Elizabeth Boschee 52 6 0 17 May 2023
Towards More Robust NLP System Evaluation: Handling Missing Scores in Benchmarks Anas Himmi Ekhine Irurozki Nathan Noiry Stephan Clémençon Pierre Colombo 193 9 0 17 May 2023
Variable-length Neural Interlingua Representations for Zero-shot Neural Machine Translation Zhuoyuan Mao Haiyue Song Raj Dabre Chenhui Chu Sadao Kurohashi 77 0 0 17 May 2023
xPQA: Cross-Lingual Product Question Answering across 12 Languages Xiaoyu Shen Akari Asai Bill Byrne Adria de Gispert 98 8 0 16 May 2023
Dual-Alignment Pre-training for Cross-lingual Sentence Embedding Ziheng Li Shaohan Huang Zi-qiang Zhang Zhi-Hong Deng Qiang Lou Haizhen Huang Jian Jiao Furu Wei Weiwei Deng Qi Zhang 64 11 0 16 May 2023
It Takes Two to Tango: Navigating Conceptualizations of NLP Tasks and Measurements of Performance Arjun Subramonian Xingdi Yuan Hal Daumé Su Lin Blodgett 98 18 0 15 May 2023
Measuring Cross-Lingual Transferability of Multilingual Transformers on Sentence Classification Zewen Chi Heyan Huang Xian-Ling Mao 129 0 0 15 May 2023
Taxi1500: A Multilingual Dataset for Text Classification in 1500 Languages Chunlan Ma Ayyoob Imani Haotian Ye Renhao Pei Ehsaneddin Asgari Hinrich Schütze 82 25 0 15 May 2023
What's the Meaning of Superhuman Performance in Today's NLU? Simone Tedeschi Johan Bos T. Declerck Jan Hajic Daniel Hershcovich ... Simon Krek Steven Schockaert Rico Sennrich Ekaterina Shutova Roberto Navigli ELM LM&MA VLM ReLM LRM 98 27 0 15 May 2023
A Comprehensive Analysis of Adapter Efficiency Nandini Mundra Sumanth Doddapaneni Raj Dabre Anoop Kunchukuttan Ratish Puduppully Mitesh M. Khapra 60 11 0 12 May 2023
A General-Purpose Multilingual Document Encoder Onur Galoglu Robert Litschko Goran Glavaš 87 2 0 11 May 2023
GeoGLUE: A GeoGraphic Language Understanding Evaluation Benchmark Dongyang Li Ruixue Ding Qiang-Wei Zhang Zheng Li Boli Chen ... Yao Xu Xin Li Ning Guo Fei Huang Xiaofeng He ELM VLM 65 6 0 11 May 2023
MultiTACRED: A Multilingual Version of the TAC Relation Extraction Dataset Leonhard Hennig Philippe E. Thomas Sebastian Möller 103 10 0 08 May 2023
Test-Time Adaptation with Perturbation Consistency Learning Yi Su Yixin Ji Juntao Li Hai Ye Hao Fei VLM 68 2 0 25 Apr 2023