Oolong: Investigating What Makes Transfer Learning Hard with Controlled Studies

24 February 2022

Papers citing "Oolong: Investigating What Makes Transfer Learning Hard with Controlled Studies"

34 / 34 papers shown

Title
We're Calling an Intervention: Exploring Fundamental Hurdles in Adapting Language Models to Nonstandard Text Aarohi Srivastava David Chiang 75 0 0 10 Apr 2024
XLM-V: Overcoming the Vocabulary Bottleneck in Multilingual Masked Language Models Davis Liang Hila Gonen Yuning Mao Rui Hou Naman Goyal Marjan Ghazvininejad Luke Zettlemoyer Madian Khabsa 32 79 0 25 Jan 2023
Word Order Does Matter (And Shuffled Language Models Know It) Vinit Ravishankar Mostafa Abdou Artur Kulmizev Anders Søgaard 55 45 0 21 Mar 2022
Overlap-based Vocabulary Generation Improves Cross-lingual Transfer Among Related Languages Vaidehi Patil Partha P. Talukdar Sunita Sarawagi 56 22 0 03 Mar 2022
When is BERT Multilingual? Isolating Crucial Ingredients for Cross-lingual Transfer Ameet Deshpande Partha P. Talukdar Karthik Narasimhan 60 39 0 27 Oct 2021
Cross-lingual Transfer of Monolingual Models Evangelia Gogoulou Ariel Ekgren T. Isbister Magnus Sahlgren 44 18 0 15 Sep 2021
Subword Mapping and Anchoring across Languages Giorgos Vernikos Andrei Popescu-Belis 86 12 0 09 Sep 2021
Masked Language Modeling and the Distributional Hypothesis: Order Word Matters Pre-training for Little Koustuv Sinha Robin Jia Dieuwke Hupkes J. Pineau Adina Williams Douwe Kiela 73 246 0 14 Apr 2021
How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Models Phillip Rust Jonas Pfeiffer Ivan Vulić Sebastian Ruder Iryna Gurevych 117 250 0 31 Dec 2020
UNKs Everywhere: Adapting Multilingual Language Models to New Scripts Jonas Pfeiffer Ivan Vulić Iryna Gurevych Sebastian Ruder 63 131 0 31 Dec 2020
Out of Order: How Important Is The Sequential Order of Words in a Sentence in Natural Language Understanding Tasks? Thang M. Pham Trung Bui Long Mai Anh Totti Nguyen 239 123 0 30 Dec 2020
When Being Unseen from mBERT is just the Beginning: Handling New Languages With Multilingual Language Models Benjamin Muller Antonis Anastasopoulos Benoît Sagot Djamé Seddah LRM 162 168 0 24 Oct 2020
Reusing a Pretrained Language Model on Languages with Limited Corpora for Unsupervised NMT Alexandra Chronopoulou Dario Stojanovski Alexander Fraser 48 33 0 16 Sep 2020
Identifying Necessary Elements for BERT's Multilinguality Philipp Dufter Hinrich Schütze 44 17 0 01 May 2020
MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfer Jonas Pfeiffer Ivan Vulić Iryna Gurevych Sebastian Ruder 96 625 0 30 Apr 2020
Investigating Transferability in Pretrained Language Models Alex Tamkin Trisha Singh D. Giovanardi Noah D. Goodman MILM 58 48 0 30 Apr 2020
Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation Nils Reimers Iryna Gurevych 94 1,022 0 21 Apr 2020
Stanza: A Python Natural Language Processing Toolkit for Many Human Languages Peng Qi Yuhao Zhang Yuhui Zhang Jason Bolton Christopher D. Manning AI4TS 236 1,690 0 16 Mar 2020
From English To Foreign Languages: Transferring Pre-trained Language Models Ke M. Tran 42 51 0 18 Feb 2020
BERTje: A Dutch BERT Model Wietse de Vries Andreas van Cranenburgh Arianna Bisazza Tommaso Caselli Gertjan van Noord Malvina Nissim VLM SSeg 63 293 0 19 Dec 2019
Cross-Lingual Ability of Multilingual BERT: An Empirical Study Karthikeyan K Zihan Wang Stephen D. Mayhew Dan Roth LRM 62 338 0 17 Dec 2019
FlauBERT: Unsupervised Language Model Pre-training for French Hang Le Loïc Vial Jibril Frej Vincent Segonne Maximin Coavoux Benjamin Lecouteux A. Allauzen Benoît Crabbé Laurent Besacier D. Schwab AI4CE 83 398 0 11 Dec 2019
On the Cross-lingual Transferability of Monolingual Representations Mikel Artetxe Sebastian Ruder Dani Yogatama 179 793 0 25 Oct 2019
Quantifying the Carbon Emissions of Machine Learning Alexandre Lacoste A. Luccioni Victor Schmidt Thomas Dandres 86 695 0 21 Oct 2019
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations Zhenzhong Lan Mingda Chen Sebastian Goodman Kevin Gimpel Piyush Sharma Radu Soricut SSL AIMat 338 6,441 0 26 Sep 2019
RoBERTa: A Robustly Optimized BERT Pretraining Approach Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy M. Lewis Luke Zettlemoyer Veselin Stoyanov AIMat 538 24,422 0 26 Jul 2019
Transfusion: Understanding Transfer Learning for Medical Imaging M. Raghu Chiyuan Zhang Jon M. Kleinberg Samy Bengio MedIm 77 983 0 14 Feb 2019
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.6K 94,511 0 11 Oct 2018
SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing Taku Kudo John Richardson 180 3,514 0 19 Aug 2018
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 1.0K 7,152 0 20 Apr 2018
The Galactic Dependencies Treebanks: Getting More Data by Synthesizing New Languages D. Wang Jason Eisner 46 58 0 10 Oct 2017
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation Yonghui Wu M. Schuster Zhiwen Chen Quoc V. Le Mohammad Norouzi ... Alex Rudnick Oriol Vinyals G. Corrado Macduff Hughes J. Dean AIMat 887 6,787 0 26 Sep 2016
Pointer Sentinel Mixture Models Stephen Merity Caiming Xiong James Bradbury R. Socher RALM 282 2,844 0 26 Sep 2016
Neural Machine Translation of Rare Words with Subword Units Rico Sennrich Barry Haddow Alexandra Birch 195 7,729 0 31 Aug 2015