Are All Languages Created Equal in Multilingual BERT?

18 May 2020

Papers citing "Are All Languages Created Equal in Multilingual BERT?"

50 / 174 papers shown

Title
The Geometry of Multilingual Language Model Representations Tyler A. Chang Z. Tu Benjamin Bergen 21 56 0 22 May 2022
Pre-training Data Quality and Quantity for a Low-Resource Language: New Corpus and BERT Models for Maltese Kurt Micallef Albert Gatt Marc Tanti Lonneke van der Plas Claudia Borg 28 28 0 21 May 2022
Overcoming Language Disparity in Online Content Classification with Multimodal Learning Gaurav Verma Rohit Mujumdar Zijie J. Wang M. D. Choudhury Srijan Kumar 23 14 0 19 May 2022
OneAligner: Zero-shot Cross-lingual Transfer with One Rich-Resource Language Pair for Low-Resource Sentence Retrieval Tong Niu Kazuma Hashimoto Yingbo Zhou Caiming Xiong VLM 29 5 0 17 May 2022
Beyond Static Models and Test Sets: Benchmarking the Potential of Pre-trained Models Across Tasks and Languages Kabir Ahuja Sandipan Dandapat Sunayana Sitaram Monojit Choudhury LRM 39 16 0 12 May 2022
On the Economics of Multilingual Few-shot Learning: Modeling the Cost-Performance Trade-offs of Machine Translated and Manual Data Kabir Ahuja Monojit Choudhury Sandipan Dandapat 21 3 0 12 May 2022
Lifting the Curse of Multilinguality by Pre-training Modular Transformers Jonas Pfeiffer Naman Goyal Xi Lin Xian Li James Cross Sebastian Riedel Mikel Artetxe LRM 40 139 0 12 May 2022
Multi Task Learning For Zero Shot Performance Prediction of Multilingual Models Kabir Ahuja Shanu Kumar Sandipan Dandapat Monojit Choudhury 11 25 0 12 May 2022
Enhancing Cross-lingual Transfer by Manifold Mixup Huiyun Yang Huadong Chen Hao Zhou Lei Li AAML 20 44 0 09 May 2022
A Balanced Data Approach for Evaluating Cross-Lingual Transfer: Mapping the Linguistic Blood Bank Daniel Malkin Tomasz Limisiewicz Gabriel Stanovsky 22 24 0 09 May 2022
mGPT: Few-Shot Learners Go Multilingual Oleh Shliazhko Alena Fenogenova Maria Tikhonova Vladislav Mikhailov Anastasia Kozlova Tatiana Shavrina 49 149 0 15 Apr 2022
Label Semantic Aware Pre-training for Few-shot Text Classification Aaron Mueller Jason Krone Salvatore Romeo Saab Mansour Elman Mansimov Yi Zhang Dan Roth VLM 17 38 0 14 Apr 2022
A Comparative Study of Pre-trained Encoders for Low-Resource Named Entity Recognition Yuxuan Chen Jonas Mikkelsen Arne Binder Christoph Alt Leonhard Hennig 32 2 0 11 Apr 2022
Assessment of Massively Multilingual Sentiment Classifiers Krzysztof Rajda Lukasz Augustyniak Piotr Gramacki Marcin Gruza Szymon Wo'zniak Tomasz Kajdanowicz 28 5 0 11 Apr 2022
Global Readiness of Language Technology for Healthcare: What would it Take to Combat the Next Pandemic? Ishani Mondal Kabir Ahuja Mohit Jain Jacki O Neil Kalika Bali Monojit Choudhury ELM LM&MA 26 4 0 06 Apr 2022
Considerations for Multilingual Wikipedia Research Isaac Johnson Emily A. Lescak 24 3 0 05 Apr 2022
Match the Script, Adapt if Multilingual: Analyzing the Effect of Multilingual Pretraining on Cross-lingual Transferability Yoshinari Fujinuma Jordan L. Boyd-Graber Katharina Kann AAML 62 23 0 21 Mar 2022
Combining Static and Contextualised Multilingual Embeddings Katharina Hämmerl Jindrich Libovický Alexander Fraser 25 10 0 17 Mar 2022
Overlap-based Vocabulary Generation Improves Cross-lingual Transfer Among Related Languages Vaidehi Patil Partha P. Talukdar Sunita Sarawagi 24 21 0 03 Mar 2022
CINO: A Chinese Minority Pre-trained Language Model Ziqing Yang Zihang Xu Yiming Cui Baoxin Wang Min-Bin Lin Dayong Wu Zhigang Chen 21 25 0 28 Feb 2022
Punctuation restoration in Swedish through fine-tuned KB-BERT J. Nilsson 13 0 0 14 Feb 2022
Does Transliteration Help Multilingual Language Modeling? Ibraheem Muhammad Moosa Mahmud Elahi Akhter Ashfia Binte Habib 45 11 0 29 Jan 2022
A Warm Start and a Clean Crawled Corpus -- A Recipe for Good Language Models Vésteinn Snæbjarnarson Haukur Barri Símonarson Pétur Orri Ragnarsson Svanhvít Lilja Ingólfsdóttir H. Jónsson Vilhjálmur Þorsteinsson H. Einarsson 18 26 0 14 Jan 2022
WECHSEL: Effective initialization of subword embeddings for cross-lingual transfer of monolingual language models Benjamin Minixhofer Fabian Paischer Navid Rekabsaz 24 73 0 13 Dec 2021
Can Character-based Language Models Improve Downstream Task Performance in Low-Resource and Noisy Language Scenarios? Arij Riabi Benoît Sagot Djamé Seddah 31 15 0 26 Oct 2021
Predicting the Performance of Multilingual NLP Models A. Srinivasan Sunayana Sitaram T. Ganu Sandipan Dandapat Kalika Bali Monojit Choudhury LRM 30 27 0 17 Oct 2021
Multilingual unsupervised sequence segmentation transfers to extremely low-resource languages C.M. Downey Shannon Drizin Levon Haroutunian Shivin Thukral 28 2 0 16 Oct 2021
Cross-Lingual Fine-Grained Entity Typing N. Selvaraj Yasumasa Onoe Greg Durrett 14 2 0 15 Oct 2021
On the Prunability of Attention Heads in Multilingual BERT Aakriti Budhraja Madhura Pande Pratyush Kumar Mitesh M. Khapra 50 4 0 26 Sep 2021
Unsupervised Translation of German--Lower Sorbian: Exploring Training and Novel Transfer Methods on a Low-Resource Language Lukas Edman Ahmet Üstün Antonio Toral Gertjan van Noord 15 6 0 24 Sep 2021
On the Universality of Deep Contextual Language Models Shaily Bhatt Poonam Goyal Sandipan Dandapat Monojit Choudhury Sunayana Sitaram ELM 25 5 0 15 Sep 2021
Everything Is All It Takes: A Multipronged Strategy for Zero-Shot Cross-Lingual Information Extraction M. Yarmohammadi Shijie Wu Marc Marone Haoran Xu Seth Ebner ... Craig Harman Kenton W. Murray Aaron Steven White Mark Dredze Benjamin Van Durme 31 28 0 14 Sep 2021
Evaluating Transferability of BERT Models on Uralic Languages Judit Ács Dániel Lévai András Kornai 27 6 0 13 Sep 2021
A Massively Multilingual Analysis of Cross-linguality in Shared Embedding Space Alex Jones Luu Anh Tuan Kyle Mahowald 23 8 0 13 Sep 2021
Mitigating Language-Dependent Ethnic Bias in BERT Jaimeen Ahn Alice H. Oh 142 92 0 13 Sep 2021
Compositional Generalization in Multilingual Semantic Parsing over Wikidata Ruixiang Cui Rahul Aralikatte Heather Lent Daniel Hershcovich 39 11 0 07 Aug 2021
Deriving Disinformation Insights from Geolocalized Twitter Callouts David Tuxworth Dimosthenis Antypas Luis Espinosa-Anke Jose Camacho-Collados Alun D. Preece David Rogers 23 0 0 06 Aug 2021
EENLP: Cross-lingual Eastern European NLP Index Alexey Tikhonov Alex Malkhasov A. Manoshin George-Andrei Dima Réka Cserháti Md. Sadek Hossain Asif Matt Sárdi 28 2 0 05 Aug 2021
gaBERT -- an Irish Language Model James Barry Joachim Wagner Lauren Cassidy Alan Cowap Teresa Lynn Abigail Walsh Mícheál J. Ó Meachair Jennifer Foster 13 18 0 27 Jul 2021
A Primer on Pretrained Multilingual Language Models Sumanth Doddapaneni Gowtham Ramesh Mitesh M. Khapra Anoop Kunchukuttan Pratyush Kumar LRM 43 74 0 01 Jul 2021
Specializing Multilingual Language Models: An Empirical Study Ethan C. Chau Noah A. Smith 27 27 0 16 Jun 2021
Can BERT Dig It? -- Named Entity Recognition for Information Retrieval in the Archaeology Domain Alex Brandsen Suzan Verberne K. Lambers M. Wansleeben 27 37 0 14 Jun 2021
Assessing Multilingual Fairness in Pre-trained Multimodal Representations Jialu Wang Yang Liu Qing Guo EGVM 26 35 0 12 Jun 2021
Investigating Transfer Learning in Multilingual Pre-trained Language Models through Chinese Natural Language Inference Hai Hu He Zhou Zuoyu Tian Yiwen Zhang Yina Ma Yanting Li Yixin Nie Kyle Richardson 27 11 0 07 Jun 2021
MergeDistill: Merging Pre-trained Language Models using Distillation Simran Khanuja Melvin Johnson Partha P. Talukdar 27 16 0 05 Jun 2021
How to Adapt Your Pretrained Multilingual Model to 1600 Languages Abteen Ebrahimi Katharina Kann LRM VLM 27 67 0 03 Jun 2021
A Multilingual Entity Linking System for Wikipedia with a Machine-in-the-Loop Approach Martin Gerlach M. Miller Rita Ho Kosta Harlan D. Difallah KELM 11 10 0 31 May 2021
Adapting Monolingual Models: Data can be Scarce when Language Similarity is High Wietse de Vries Martijn Bartelds Malvina Nissim Martijn B. Wieling 26 22 0 06 May 2021
Operationalizing a National Digital Library: The Case for a Norwegian Transformer Model P. Kummervold Javier de la Rosa Freddy Wetjen Svein Arne Brygfjeld 14 55 0 19 Apr 2021
AmericasNLI: Evaluating Zero-shot Natural Language Understanding of Pretrained Multilingual Models in Truly Low-resource Languages Abteen Ebrahimi Manuel Mager Arturo Oncevay Vishrav Chaudhary Luis Chiruzzo ... Graham Neubig Alexis Palmer Rolando A. Coto Solano Ngoc Thang Vu Katharina Kann 109 72 0 18 Apr 2021