v1v2 (latest)

Discovering Language-neutral Sub-networks in Multilingual Language Models

25 May 2022

Papers citing "Discovering Language-neutral Sub-networks in Multilingual Language Models"

39 / 39 papers shown

Title
Linguistically Grounded Analysis of Language Models using Shapley Head Values Marcell Richard Fekete Johannes Bjerva 169 0 0 17 Oct 2024
Training Neural Networks with Fixed Sparse Masks Yi-Lin Sung Varun Nair Colin Raffel FedML 99 209 0 18 Nov 2021
Composable Sparse Fine-Tuning for Cross-Lingual Transfer Alan Ansell Edoardo Ponti Anna Korhonen Ivan Vulić CLL MoE 143 143 0 14 Oct 2021
Learning Language Specific Sub-network for Multilingual Machine Translation Zehui Lin Liwei Wu Mingxuan Wang Lei Li 71 82 0 19 May 2021
First Align, then Predict: Understanding the Cross-Lingual Ability of Multilingual BERT Benjamin Muller Yanai Elazar Benoît Sagot Djamé Seddah LRM 62 77 0 26 Jan 2021
mT5: A massively multilingual pre-trained text-to-text transformer Linting Xue Noah Constant Adam Roberts Mihir Kale Rami Al-Rfou Aditya Siddhant Aditya Barua Colin Raffel 151 2,561 0 22 Oct 2020
It's not Greek to mBERT: Inducing Word-Level Translations from Multilingual BERT Hila Gonen Shauli Ravfogel Yanai Elazar Yoav Goldberg 58 51 0 16 Oct 2020
On Negative Interference in Multilingual Models: Findings and A Meta-Learning Treatment Zirui Wang Zachary Chase Lipton Yulia Tsvetkov 77 32 0 06 Oct 2020
What does it mean to be language-agnostic? Probing multilingual sentence encoders for typological properties Rochelle Choenni Ekaterina Shutova 91 37 0 27 Sep 2020
Inducing Language-Agnostic Multilingual Representations Wei Zhao Steffen Eger Johannes Bjerva Isabelle Augenstein 72 65 0 20 Aug 2020
Finding Universal Grammatical Relations in Multilingual BERT Ethan A. Chi John Hewitt Christopher D. Manning 64 151 0 09 May 2020
When BERT Plays the Lottery, All Tickets Are Winning Sai Prasanna Anna Rogers Anna Rumshisky MILM 77 187 0 01 May 2020
What Happens To BERT Embeddings During Fine-tuning? Amil Merchant Elahe Rahimtoroghi Ellie Pavlick Ian Tenney 95 188 0 29 Apr 2020
What's so special about BERT's layers? A closer look at the NLP pipeline in monolingual and multilingual models Wietse de Vries Andreas van Cranenburgh Malvina Nissim MILM SSeg MoE 142 66 0 14 Apr 2020
On the Language Neutrality of Pre-trained Multilingual Representations Jindrich Libovický Rudolf Rosa Alexander Fraser 69 106 0 09 Apr 2020
XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization Junjie Hu Sebastian Ruder Aditya Siddhant Graham Neubig Orhan Firat Melvin Johnson ELM 211 977 0 24 Mar 2020
Cross-Lingual Ability of Multilingual BERT: An Empirical Study Karthikeyan K Zihan Wang Stephen D. Mayhew Dan Roth LRM 79 339 0 17 Dec 2019
Linear Mode Connectivity and the Lottery Ticket Hypothesis Jonathan Frankle Gintare Karolina Dziugaite Daniel M. Roy Michael Carbin MoMe 163 630 0 11 Dec 2019
How Language-Neutral is Multilingual BERT? Jindrich Libovický Rudolf Rosa Alexander Fraser 85 116 0 08 Nov 2019
Unsupervised Cross-lingual Representation Learning at Scale Alexis Conneau Kartikay Khandelwal Naman Goyal Vishrav Chaudhary Guillaume Wenzek Francisco Guzmán Edouard Grave Myle Ott Luke Zettlemoyer Veselin Stoyanov 228 6,593 0 05 Nov 2019
On the Cross-lingual Transferability of Monolingual Representations Mikel Artetxe Sebastian Ruder Dani Yogatama 226 800 0 25 Oct 2019
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 506 20,376 0 23 Oct 2019
One ticket to win them all: generalizing lottery ticket initializations across datasets and optimizers Ari S. Morcos Haonan Yu Michela Paganini Yuandong Tian 79 229 0 06 Jun 2019
Playing the lottery with rewards and multiple languages: lottery tickets in RL and NLP Haonan Yu Sergey Edunov Yuandong Tian Ari S. Morcos 58 150 0 06 Jun 2019
How multilingual is Multilingual BERT? Telmo Pires Eva Schlinger Dan Garrette LRM VLM 164 1,415 0 04 Jun 2019
Similarity of Neural Network Representations Revisited Simon Kornblith Mohammad Norouzi Honglak Lee Geoffrey E. Hinton 145 1,436 0 01 May 2019
Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERT Shijie Wu Mark Dredze VLM SSeg 114 681 0 19 Apr 2019
The State of Sparsity in Deep Neural Networks Trevor Gale Erich Elsen Sara Hooker 167 763 0 25 Feb 2019
Cross-lingual Language Model Pretraining Guillaume Lample Alexis Conneau 116 2,751 0 22 Jan 2019
Margin-based Parallel Corpus Mining with Multilingual Sentence Embeddings Mikel Artetxe Holger Schwenk 67 202 0 03 Nov 2018
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.8K 95,324 0 11 Oct 2018
XNLI: Evaluating Cross-lingual Sentence Representations Alexis Conneau Guillaume Lample Ruty Rinott Adina Williams Samuel R. Bowman Holger Schwenk Veselin Stoyanov ELM 93 1,388 0 13 Sep 2018
Insights on representational similarity in neural networks with canonical correlation Ari S. Morcos M. Raghu Samy Bengio DRL 96 447 0 14 Jun 2018
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 1.1K 7,201 0 20 Apr 2018
The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks Jonathan Frankle Michael Carbin 288 3,489 0 09 Mar 2018
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 811 132,725 0 12 Jun 2017
A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference Adina Williams Nikita Nangia Samuel R. Bowman 526 4,497 0 18 Apr 2017
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation Yonghui Wu M. Schuster Zhiwen Chen Quoc V. Le Mohammad Norouzi ... Alex Rudnick Oriol Vinyals G. Corrado Macduff Hughes J. Dean AIMat 918 6,799 0 26 Sep 2016
Learning both Weights and Connections for Efficient Neural Networks Song Han Jeff Pool J. Tran W. Dally CVBM 320 6,715 0 08 Jun 2015