Unveiling Multilinguality in Transformer Models: Exploring Language Specificity in Feed-Forward Networks

24 October 2023

Papers citing "Unveiling Multilinguality in Transformer Models: Exploring Language Specificity in Feed-Forward Networks"

11 / 11 papers shown

Title
Analyzing the Mono- and Cross-Lingual Pretraining Dynamics of Multilingual Language Models Terra Blevins Hila Gonen Luke Zettlemoyer LRM 101 31 0 24 May 2022
Lifting the Curse of Multilinguality by Pre-training Modular Transformers Jonas Pfeiffer Naman Goyal Xi Lin Xian Li James Cross Sebastian Riedel Mikel Artetxe LRM 93 146 0 12 May 2022
Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Space Mor Geva Avi Caciularu Ke Wang Yoav Goldberg KELM 127 387 0 28 Mar 2022
Locating and Editing Factual Associations in GPT Kevin Meng David Bau A. Andonian Yonatan Belinkov KELM 251 1,381 0 10 Feb 2022
Kformer: Knowledge Injection in Transformer Feed-Forward Layers Yunzhi Yao Shaohan Huang Li Dong Furu Wei Huajun Chen Ningyu Zhang KELM MedIm 78 42 0 15 Jan 2022
MoEfication: Transformer Feed-forward Layers are Mixtures of Experts Zhengyan Zhang Yankai Lin Zhiyuan Liu Peng Li Maosong Sun Jie Zhou MoE 91 128 0 05 Oct 2021
Transformer Feed-Forward Layers Are Key-Value Memories Mor Geva R. Schuster Jonathan Berant Omer Levy KELM 174 847 0 29 Dec 2020
Announcing CzEng 2.0 Parallel Corpus with over 2 Gigawords Tom Kocmi Martin Popel Ondrej Bojar 35 38 0 06 Jul 2020
How Language-Neutral is Multilingual BERT? Jindrich Libovický Rudolf Rosa Alexander Fraser 75 116 0 08 Nov 2019
Investigating Multilingual NMT Representations at Scale Sneha Kudugunta Ankur Bapna Isaac Caswell N. Arivazhagan Orhan Firat LRM 184 125 0 05 Sep 2019
BERT Rediscovers the Classical NLP Pipeline Ian Tenney Dipanjan Das Ellie Pavlick MILM SSeg 143 1,482 0 15 May 2019