v1v2v3v4v5 (latest)

XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization

24 March 2020

Graham Neubig

Papers citing "XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization"

50 / 666 papers shown

Title
Exposing Assumptions in AI Benchmarks through Cognitive Modelling Jonathan H. Rystrøm Kenneth C. Enevoldsen 67 0 0 25 Sep 2024
IRSC: A Zero-shot Evaluation Benchmark for Information Retrieval through Semantic Comprehension in Retrieval-Augmented Generation Scenarios Hai Lin Shaoxiong Zhan Junyou Su Haitao Zheng Hui Wang RALM 60 1 0 24 Sep 2024
Mitigating Semantic Leakage in Cross-lingual Embeddings via Orthogonality Constraint Dayeon Ki Cheonbok Park H. Kim FedML 62 0 0 24 Sep 2024
XTRUST: On the Multilingual Trustworthiness of Large Language Models Yahan Li Yi Wang Yi-Ju Chang Yuan Wu LRM HILM 43 0 0 24 Sep 2024
Bilingual Evaluation of Language Models on General Knowledge in University Entrance Exams with Minimal Contamination Eva Sánchez Salido Roser Morante Julio Gonzalo Guillermo Marco Jorge Carrillo-de-Albornoz ... Enrique Amigó Andrés Fernández Alejandro Benito-Santos Adrián Ghajari Espinosa Victor Fresno ELM 142 0 0 19 Sep 2024
MEXMA: Token-level objectives improve sentence representations Joao Maria Janeiro Benjamin Piwowarski Patrick Gallinari Loïc Barrault 41 2 0 19 Sep 2024
AraDiCE: Benchmarks for Dialectal and Cultural Capabilities in LLMs Basel Mousi Nadir Durrani Fatema Ahmad Md. Arid Hasan Maram Hasanain Tameem Kabbani Fahim Dalvi Shammur A. Chowdhury Firoj Alam 102 9 0 17 Sep 2024
Exploring syntactic information in sentence embeddings through multilingual subject-verb agreement Vivi Nastase Chunyang Jiang Giuseppe Samo Paola Merlo 60 1 0 10 Sep 2024
CLEANANERCorp: Identifying and Correcting Incorrect Labels in the ANERcorp Dataset Mashael Al-Duwais H. Al-Khalifa Abdulmalik Al-Salman 128 0 0 22 Aug 2024
Against All Odds: Overcoming Typology, Script, and Language Confusion in Multilingual Embedding Inversion Attacks Yiyi Chen Russa Biswas Heather Lent Johannes Bjerva AAML 92 5 0 21 Aug 2024
Assessing the Role of Lexical Semantics in Cross-lingual Transfer through Controlled Manipulations Roy Ilani Taelin Karidi Omri Abend 54 0 0 14 Aug 2024
Do Large Language Models Speak All Languages Equally? A Comparative Study in Low-Resource Settings Md. Arid Hasan Prerona Tarannum Krishno Dey Imran Razzak Usman Naseem 75 4 0 05 Aug 2024
mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrieval Xin Zhang Yanzhao Zhang Dingkun Long Wen Xie Ziqi Dai ... Pengjun Xie Fei Huang Meishan Zhang Wenjie Li Min Zhang 141 108 0 29 Jul 2024
Multilingual Fine-Grained News Headline Hallucination Detection Jiaming Shen Tianqi Liu Jialu Liu Zhen Qin Jay Pavagadhi Simon Baumgartner Michael Bendersky 87 0 0 22 Jul 2024
MASIVE: Open-Ended Affective State Identification in English and Spanish Nicholas Deas Elsbeth Turcan Iván Pérez Mejía Kathleen McKeown CVBM 65 1 0 16 Jul 2024
NativQA: Multilingual Culturally-Aligned Natural Query for LLMs Md. Arid Hasan Maram Hasanain Fatema Ahmad Sahinur Rahman Laskar Sunaya Upadhyay Vrunda N. Sukhadia Mucahid Kutlu Shammur A. Chowdhury Firoj Alam 171 7 0 13 Jul 2024
sPhinX: Sample Efficient Multilingual Instruction Fine-Tuning Through N-shot Guided Prompting Sanchit Ahuja Kumar Tanmay Hardik Hansrajbhai Chauhan Barun Patra Kriti Aggarwal ... Tejas I. Dhamecha Ahmed Awadallah Monojit Choudhary Vishrav Chaudhary Sunayana Sitaram 87 4 0 13 Jul 2024
MAGNET: Improving the Multilingual Fairness of Language Models with Adaptive Gradient-Based Tokenization Orevaoghene Ahia Sachin Kumar Hila Gonen Valentin Hoffman Tomasz Limisiewicz Yulia Tsvetkov Noah A. Smith 103 5 0 11 Jul 2024
Data, Data Everywhere: A Guide for Pretraining Dataset Construction Jupinder Parmar Shrimai Prabhumoye Pritam Gundecha Bo Liu Aastha Jhunjhunwala Zhilin Wang M. Patwary Mohammad Shoeybi Bryan Catanzaro 126 10 0 08 Jul 2024
IL-TUR: Benchmark for Indian Legal Text Understanding and Reasoning Abhinav Joshi Shounak Paul Akshat Sharma Pawan Goyal Saptarshi Ghosh Ashutosh Modi AILaw ELM 71 12 0 07 Jul 2024
Faux Polyglot: A Study on Information Disparity in Multilingual Large Language Models Nikhil Sharma Kenton Murray Ziang Xiao 153 1 0 07 Jul 2024
Cross-Lingual Word Alignment for ASEAN Languages with Contrastive Learning Jingshen Zhang Xinying Qiu Teng Shen Wenyu Wang Kailin Zhang Wenhe Feng 89 0 0 06 Jul 2024
Soft Language Prompts for Language Transfer Ivan Vykopal Simon Ostermann Marian Simko AAML 81 2 0 02 Jul 2024
GemmAr: Enhancing LLMs Through Arabic Instruction-Tuning Hasna Chouikhi Manel Aloui Cyrine Ben Hammou Ghaith Chaabane Haithem Kchaou Chehir Dhaouadi 76 0 0 02 Jul 2024
M2QA: Multi-domain Multilingual Question Answering Leon Arne Engländer Hannah Sterz Clifton A. Poth Jonas Pfeiffer Ilia Kuznetsov Iryna Gurevych VLM 78 2 0 01 Jul 2024
Self-Translate-Train: A Simple but Strong Baseline for Cross-lingual Transfer of Large Language Models Ryokan Ri Shun Kiyono Sho Takase SyDa 49 0 0 29 Jun 2024
Understanding and Mitigating Language Confusion in LLMs Kelly Marchisio Wei-Yin Ko Alexandre Berard Théo Dehaze Sebastian Ruder 162 32 0 28 Jun 2024
Teaching LLMs to Abstain across Languages via Multilingual Feedback Shangbin Feng Weijia Shi Yike Wang Wenxuan Ding Orevaoghene Ahia Shuyue Stella Li Vidhisha Balachandran Sunayana Sitaram Yulia Tsvetkov 161 8 0 22 Jun 2024
PARIKSHA : A Large-Scale Investigation of Human-LLM Evaluator Agreement on Multilingual and Multi-Cultural Data Ishaan Watts Varun Gumma Aditya Yadavalli Vivek Seshadri Manohar Swaminathan Sunayana Sitaram ELM 97 9 0 21 Jun 2024
Data-Centric AI in the Age of Large Language Models Xinyi Xu Zhaoxuan Wu Rui Qiao Arun Verma Yao Shu ... Xiaoqiang Lin Wenyang Hu Zhongxiang Dai Pang Wei Koh Bryan Kian Hsiang Low ALM 126 3 0 20 Jun 2024
On the Evaluation Practices in Multilingual NLP: Can Machine Translation Offer an Alternative to Human Translations? Rochelle Choenni Sara Rajaee Christof Monz Ekaterina Shutova 84 2 0 20 Jun 2024
Towards Robust Evaluation: A Comprehensive Taxonomy of Datasets and Metrics for Open Domain Question Answering in the Era of Large Language Models Akchay Srivastava Atif Memon ELM 85 1 0 19 Jun 2024
Probing the Emergence of Cross-lingual Alignment during LLM Training Hetong Wang Pasquale Minervini Edoardo Ponti 139 15 0 19 Jun 2024
Self-Distillation for Model Stacking Unlocks Cross-Lingual NLU in 200+ Languages Fabian David Schmidt Philipp Borchert Ivan Vulić Goran Glavaš 81 6 0 18 Jun 2024
OlympicArena: Benchmarking Multi-discipline Cognitive Reasoning for Superintelligent AI Zhen Huang Zengzhi Wang Shijie Xia Xuefeng Li Haoyang Zou ... Yuxiang Zheng Shaoting Zhang Dahua Lin Yu Qiao Pengfei Liu ELM LRM 140 43 0 18 Jun 2024
UniBridge: A Unified Approach to Cross-Lingual Transfer Learning for Low-Resource Languages Trinh Pham Khoi M. Le Luu Anh Tuan 125 1 0 14 Jun 2024
Decipherment-Aware Multilingual Learning in Jointly Trained Language Models Grandee Lee 76 0 0 11 Jun 2024
Thai Winograd Schemas: A Benchmark for Thai Commonsense Reasoning Phakphum Artkaew LRM 58 0 0 28 May 2024
Exploring Alignment in Shared Cross-lingual Spaces Basel Mousi Nadir Durrani Fahim Dalvi Majd Hawasly Ahmed Abdelali 94 2 0 23 May 2024
XAMPLER: Learning to Retrieve Cross-Lingual In-Context Examples Peiqin Lin André F. T. Martins Hinrich Schütze RALM 144 4 0 08 May 2024
Continual Pre-Training for Cross-Lingual LLM Adaptation: Enhancing Japanese Language Capabilities Kazuki Fujii Taishi Nakamura Mengsay Loem Hiroki Iida Masanari Ohi Kakeru Hattori Hirai Shota Sakae Mizuki Rio Yokota Naoaki Okazaki CLL 131 73 0 27 Apr 2024
IndicGenBench: A Multilingual Benchmark to Evaluate Generation Capabilities of LLMs on Indic Languages Harman Singh Nitish Gupta Shikhar Bharadwaj Dinesh Tewari Partha P. Talukdar ELM 84 28 0 25 Apr 2024
Incorporating Lexical and Syntactic Knowledge for Unsupervised Cross-Lingual Transfer Jianyu Zheng Fengfei Fan Jianquan Li 85 2 0 25 Apr 2024
Holistic Safety and Responsibility Evaluations of Advanced AI Models Laura Weidinger Joslyn Barnhart Jenny Brennan Christina Butterfield Susie Young ... Sebastian Farquhar Lewis Ho Iason Gabriel Allan Dafoe William S. Isaac ELM 90 9 0 22 Apr 2024
CORI: CJKV Benchmark with Romanization Integration -- A step towards Cross-lingual Transfer Beyond Textual Scripts Hoang Nguyen Chenwei Zhang Ye Liu Natalie Parde Eugene Rohrbaugh Philip S. Yu 133 1 0 19 Apr 2024
Reuse Your Rewards: Reward Model Transfer for Zero-Shot Cross-Lingual Alignment Zhaofeng Wu Ananth Balashankar Yoon Kim Jacob Eisenstein Ahmad Beirami 119 15 0 18 Apr 2024
From Form(s) to Meaning: Probing the Semantic Depths of Language Models Using Multisense Consistency Xenia Ohmer Elia Bruni Dieuwke Hupkes AI4CE 113 7 0 18 Apr 2024
GeMQuAD : Generating Multilingual Question Answering Datasets from Large Language Models using Few Shot Learning Amani Namboori Shivam Mangale Andrew Rosenbaum Saleh Soltan 85 0 0 14 Apr 2024
Understanding Cross-Lingual Alignment -- A Survey Katharina Hämmerl Jindvrich Libovický Alexander Fraser 85 14 0 09 Apr 2024
PORTULAN ExtraGLUE Datasets and Models: Kick-starting a Benchmark for the Neural Processing of Portuguese T. Osório Bernardo Leite Henrique Lopes Cardoso Luís Gomes João Rodrigues Rodrigo Santos António Branco 96 3 0 08 Apr 2024