On the importance of pre-training data volume for compact language models

8 October 2020

Papers citing "On the importance of pre-training data volume for compact language models"

10 / 10 papers shown

Title
Feature-Level Debiased Natural Language Understanding Yougang Lyu Piji Li Yechang Yang Maarten de Rijke Pengjie Ren Yukun Zhao Dawei Yin Z. Ren 32 10 0 11 Dec 2022
Benchmarking Transformers-based models on French Spoken Language Understanding tasks Oralie Cattan Sahar Ghannay Christophe Servan Sophie Rosset 36 4 0 19 Jul 2022
Match the Script, Adapt if Multilingual: Analyzing the Effect of Multilingual Pretraining on Cross-lingual Transferability Yoshinari Fujinuma Jordan L. Boyd-Graber Katharina Kann AAML 62 23 0 21 Mar 2022
Can Character-based Language Models Improve Downstream Task Performance in Low-Resource and Noisy Language Scenarios? Arij Riabi Benoît Sagot Djamé Seddah 31 15 0 26 Oct 2021
FQuAD2.0: French Question Answering and knowing that you know nothing Quentin Heinrich Gautier Viaud Wacim Belblidia 11 8 0 27 Sep 2021
On the Transferability of Pre-trained Language Models: A Study from Artificial Datasets Cheng-Han Chiang Hung-yi Lee SyDa 32 24 0 08 Sep 2021
The Interplay of Variant, Size, and Task Type in Arabic Pre-trained Language Models Go Inoue Bashar Alhafni Nurpeiis Baimukan Houda Bouamor Nizar Habash 35 224 0 11 Mar 2021
When Do You Need Billions of Words of Pretraining Data? Yian Zhang Alex Warstadt Haau-Sing Li Samuel R. Bowman 29 136 0 10 Nov 2020
Q-BERT: Hessian Based Ultra Low Precision Quantization of BERT Sheng Shen Zhen Dong Jiayu Ye Linjian Ma Z. Yao A. Gholami Michael W. Mahoney Kurt Keutzer MQ 233 576 0 12 Sep 2019
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 299 6,984 0 20 Apr 2018