A Primer on Pretrained Multilingual Language Models

1 July 2021

Mitesh M. Khapra

Papers citing "A Primer on Pretrained Multilingual Language Models"

50 / 55 papers shown

Title
Duluth at SemEval-2025 Task 7: TF-IDF with Optimized Vector Dimensions for Multilingual Fact-Checked Claim Retrieval Shujauddin Syed Ted Pedersen 27 0 0 19 May 2025
Better To Ask in English? Evaluating Factual Accuracy of Multilingual LLMs in English and Low-Resource Languages Pritika Rohera Chaitrali Ginimav Gayatri Sawant Raviraj Joshi 34 0 0 28 Apr 2025
LangTime: A Language-Guided Unified Model for Time Series Forecasting with Proximal Policy Optimization Wenzhe Niu Zongxia Xie Yanru Sun Wei He Man Xu Chao Hao AI4TS 55 1 0 11 Mar 2025
Can MLLMs Generalize to Multi-Party dialog? Exploring Multilingual Response Generation in Complex Scenarios Zhongtian Hu Yiwen Cui Ronghan Li Meng Zhao Lifang Wang 46 0 0 20 Jan 2025
The Model Arena for Cross-lingual Sentiment Analysis: A Comparative Study in the Era of Large Language Models Xiliang Zhu Shayna Gardiner T. Roldán David Rossouw ELM 22 2 0 27 Jun 2024
The Ghanaian NLP Landscape: A First Look Sheriff Issaka Zhaoyi Zhang Mihir Heda Keyi Wang Yinka Ajibola Ryan DeMar Xuefeng Du 46 1 0 10 May 2024
Understanding the role of FFNs in driving multilingual behaviour in LLMs Sunit Bhattacharya Ondrej Bojar 26 2 0 22 Apr 2024
Mitigating Language-Level Performance Disparity in mPLMs via Teacher Language Selection and Cross-lingual Self-Distillation Haozhe Zhao Zefan Cai Shuzheng Si Liang Chen Yufeng He Kaikai An Baobao Chang 38 0 0 12 Apr 2024
Privacy Preserving Prompt Engineering: A Survey Kennedy Edemacu Xintao Wu 63 18 0 09 Apr 2024
Multilingual Brain Surgeon: Large Language Models Can be Compressed Leaving No Language Behind Hongchuan Zeng Hongshen Xu Lu Chen Kai Yu 59 5 0 06 Apr 2024
A Survey on Multilingual Large Language Models: Corpora, Alignment, and Bias Yuemei Xu Ling Hu Jiayi Zhao Zihan Qiu Yuqi Ye Hanwen Gu LRM 32 37 0 01 Apr 2024
Towards a World-English Language Model for On-Device Virtual Assistants Rricha Jalota Lyan Verwimp Markus Nussbaum-Thom Amr Mousa Arturo Argueta Youssef Oualil 34 0 0 27 Mar 2024
Data Mixing Laws: Optimizing Data Mixtures by Predicting Language Modeling Performance Jiasheng Ye Peiju Liu Tianxiang Sun Yunhua Zhou Jun Zhan Xipeng Qiu 59 64 0 25 Mar 2024
Analyzing and Adapting Large Language Models for Few-Shot Multilingual NLU: Are We There Yet? E. Razumovskaia Ivan Vulić Anna Korhonen 51 6 0 04 Mar 2024
How do Large Language Models Handle Multilingualism? Yiran Zhao Wenxuan Zhang Guizhen Chen Kenji Kawaguchi Lidong Bing LRM 46 57 0 29 Feb 2024
The Less the Merrier? Investigating Language Representation in Multilingual Models H. Nigatu A. Tonja Jugal Kalita 28 0 0 20 Oct 2023
How Different Is Stereotypical Bias Across Languages? Ibrahim Tolga Ozturk R. Nedelchev C. Heumann Esteban Garces Arias Marius Roger Bernd Bischl Matthias Aßenmacher 51 2 0 14 Jul 2023
Testing the Predictions of Surprisal Theory in 11 Languages Ethan Gotlieb Wilcox Tiago Pimentel Clara Meister Ryan Cotterell R. Levy LRM 54 63 0 07 Jul 2023
Multilingual Multiword Expression Identification Using Lateral Inhibition and Domain Adaptation Andrei-Marius Avram V. Mititelu V. Pais Dumitru-Clementin Cercel Stefan Trausan-Matu 43 3 0 17 Jun 2023
Towards a Common Understanding of Contributing Factors for Cross-Lingual Transfer in Multilingual Language Models: A Review Fred Philippy Siwen Guo Shohreh Haddadan LRM 30 30 0 26 May 2023
Revisiting Machine Translation for Cross-lingual Classification Mikel Artetxe Vedanuj Goswami Shruti Bhosale Angela Fan Luke Zettlemoyer LRM 54 34 0 23 May 2023
Exploring Large Language Models for Classical Philology Frederick Riemenschneider Anette Frank 27 15 0 23 May 2023
How do languages influence each other? Studying cross-lingual data sharing during LM fine-tuning Rochelle Choenni Dan Garrette Ekaterina Shutova 42 16 0 22 May 2023
Investigating Lexical Sharing in Multilingual Machine Translation for Indian Languages Sonal Sannigrahi Rachel Bawden 37 0 0 04 May 2023
L3Cube-IndicSBERT: A simple approach for learning cross-lingual sentence representations using multilingual BERT Samruddhi Deode Janhavi Gadre Aditi Kajale Ananya Joshi Raviraj Joshi 25 20 0 22 Apr 2023
Evaluating the Effectiveness of Pre-trained Language Models in Predicting the Helpfulness of Online Product Reviews Ali Boluki Javad Pourmostafa Roshan Sharami D. Shterionov 27 1 0 19 Feb 2023
SERENGETI: Massively Multilingual Language Models for Africa Ife Adebara AbdelRahim Elmadany Muhammad Abdul-Mageed Alcides Alcoba Inciarte 27 29 0 21 Dec 2022
Mini-Model Adaptation: Efficiently Extending Pretrained Models to New Languages via Aligned Shallow Training Kelly Marchisio Patrick Lewis Yihong Chen Mikel Artetxe 35 16 0 20 Dec 2022
Towards Leaving No Indic Language Behind: Building Monolingual Corpora, Benchmark and Models for Indic Languages Sumanth Doddapaneni Rahul Aralikatte Gowtham Ramesh Shreyansh Goyal Mitesh M. Khapra Anoop Kunchukuttan Pratyush Kumar ELM 50 81 0 11 Dec 2022
Adapters for Enhanced Modeling of Multilingual Knowledge and Text Yifan Hou Wenxiang Jiao Mei-Jun Liu Carl Allen Zhaopeng Tu Mrinmaya Sachan 41 11 0 24 Oct 2022
BERTifying Sinhala -- A Comprehensive Analysis of Pre-trained Language Models for Sinhala Text Classification Vinura Dhananjaya Piyumal Demotte Surangika Ranathunga Sanath Jayasena 29 14 0 16 Aug 2022
Multilingual Transformer Encoders: a Word-Level Task-Agnostic Evaluation Félix Gaschi François Plesse Parisa Rastin Y. Toussaint 22 8 0 19 Jul 2022
Linguistically inspired roadmap for building biologically reliable protein language models Mai Ha Vu Rahmad Akbar Philippe A. Robert B. Swiatczak Victor Greiff G. K. Sandve Dag Trygve Tryslew Haug 52 35 0 03 Jul 2022
mGPT: Few-Shot Learners Go Multilingual Oleh Shliazhko Alena Fenogenova Maria Tikhonova Vladislav Mikhailov Anastasia Kozlova Tatiana Shavrina 51 149 0 15 Apr 2022
The Impact of Cross-Lingual Adjustment of Contextual Word Representations on Zero-Shot Transfer Pavel Efimov Leonid Boytsov E. Arslanova Pavel Braslavski 30 7 0 13 Apr 2022
Pretraining with Artificial Language: Studying Transferable Knowledge in Language Models Ryokan Ri Yoshimasa Tsuruoka 32 26 0 19 Mar 2022
Improving Word Translation via Two-Stage Contrastive Learning Yaoyiran Li Fangyu Liu Nigel Collier Anna Korhonen Ivan Vulić 47 29 0 15 Mar 2022
CALCS 2021 Shared Task: Machine Translation for Code-Switched Data Shuguang Chen Gustavo Aguilar A. Srinivasan Mona T. Diab Thamar Solorio 55 15 0 19 Feb 2022
When is BERT Multilingual? Isolating Crucial Ingredients for Cross-lingual Transfer Ameet Deshpande Partha P. Talukdar Karthik Narasimhan 39 38 0 27 Oct 2021
mLUKE: The Power of Entity Representations in Multilingual Pretrained Language Models Ryokan Ri Ikuya Yamada Yoshimasa Tsuruoka 31 30 0 15 Oct 2021
EENLP: Cross-lingual Eastern European NLP Index Alexey Tikhonov Alex Malkhasov A. Manoshin George-Andrei Dima Réka Cserháti Md. Sadek Hossain Asif Matt Sárdi 34 2 0 05 Aug 2021
Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing Pengfei Liu Weizhe Yuan Jinlan Fu Zhengbao Jiang Hiroaki Hayashi Graham Neubig VLM SyDa 85 3,858 0 28 Jul 2021
Word Alignment by Fine-tuning Embeddings on Parallel Corpora Zi-Yi Dou Graham Neubig 100 258 0 20 Jan 2021
Trankit: A Light-Weight Transformer-based Toolkit for Multilingual Natural Language Processing Minh Nguyen Viet Dac Lai Amir Pouran Ben Veyseh Thien Huu Nguyen 52 132 0 09 Jan 2021
ERNIE-M: Enhanced Multilingual Representation by Aligning Cross-lingual Semantics with Monolingual Corpora Ouyang Xuan Shuohuan Wang Chao Pang Yu Sun Hao Tian Hua Wu Haifeng Wang 62 100 0 31 Dec 2020
How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Models Phillip Rust Jonas Pfeiffer Ivan Vulić Sebastian Ruder Iryna Gurevych 80 235 0 31 Dec 2020
Orthogonal Language and Task Adapters in Zero-Shot Cross-Lingual Transfer M. Vidoni Ivan Vulić Goran Glavaš 33 27 0 11 Dec 2020
When Being Unseen from mBERT is just the Beginning: Handling New Languages With Multilingual Language Models Benjamin Muller Antonis Anastasopoulos Benoît Sagot Djamé Seddah LRM 136 165 0 24 Oct 2020
Rethinking embedding coupling in pre-trained language models Hyung Won Chung Thibault Févry Henry Tsai Melvin Johnson Sebastian Ruder 95 142 0 24 Oct 2020
Improving Multilingual Models with Language-Clustered Vocabularies Hyung Won Chung Dan Garrette Kiat Chuan Tan Jason Riesa VLM 77 65 0 24 Oct 2020