v1v2 (latest)

ALBETO and DistilBETO: Lightweight Spanish Language Models

19 April 2022

Papers citing "ALBETO and DistilBETO: Lightweight Spanish Language Models"

24 / 24 papers shown

Title
BERTIN: Efficient Pre-Training of a Spanish Language Model using Perplexity Sampling Javier de la Rosa E. G. Ponferrada Paulo Villegas Pablo González de Prado Salas Manu Romero María Grandury 74 95 0 14 Jul 2022
RoBERTuito: a pre-trained language model for social media text in Spanish Juan Manuel Pérez D. Furman Laura Alonso Alemany Franco Luque 69 100 0 18 Nov 2021
MarIA: Spanish Language Models Asier Gutiérrez-Fandiño Jordi Armengol-Estapé Marc Pàmies Joan Llop-Palao Joaquín Silveira-Ocampo C. Carrino Aitor Gonzalez-Agirre Carme Armentano-Oller Carlos Rodríguez-Penagos Marta Villegas VLM 79 119 0 15 Jul 2021
MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices Zhiqing Sun Hongkun Yu Xiaodan Song Renjie Liu Yiming Yang Denny Zhou MQ 124 819 0 06 Apr 2020
RobBERT: a Dutch RoBERTa-based Language Model Pieter Delobelle Thomas Winters Bettina Berendt 79 239 0 17 Jan 2020
BERTje: A Dutch BERT Model Wietse de Vries Andreas van Cranenburgh Arianna Bisazza Tommaso Caselli Gertjan van Noord Malvina Nissim VLM SSeg 86 295 0 19 Dec 2019
Multilingual is not enough: BERT for Finnish Antti Virtanen Jenna Kanerva Rami Ilo Jouni Luoma Juhani Luotolahti T. Salakoski Filip Ginter S. Pyysalo 88 281 0 15 Dec 2019
FlauBERT: Unsupervised Language Model Pre-training for French Hang Le Loïc Vial Jibril Frej Vincent Segonne Maximin Coavoux Benjamin Lecouteux A. Allauzen Benoît Crabbé Laurent Besacier D. Schwab AI4CE 96 400 0 11 Dec 2019
Automatic Spanish Translation of the SQuAD Dataset for Multilingual Question Answering C. Carrino Marta R. Costa-jussá José A. R. Fonollosa 58 89 0 11 Dec 2019
SemEval-2017 Task 4: Sentiment Analysis in Twitter Sara Rosenthal N. Farra Preslav Nakov VLM 102 802 0 02 Dec 2019
CamemBERT: a Tasty French Language Model Louis Martin Benjamin Muller Pedro Ortiz Suarez Yoann Dupont Laurent Romary Eric Villemonte de la Clergerie Djamé Seddah Benoît Sagot 130 976 0 10 Nov 2019
On the Cross-lingual Transferability of Monolingual Representations Mikel Artetxe Sebastian Ruder Dani Yogatama 226 800 0 25 Oct 2019
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter Victor Sanh Lysandre Debut Julien Chaumond Thomas Wolf 276 7,564 0 02 Oct 2019
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations Zhenzhong Lan Mingda Chen Sebastian Goodman Kevin Gimpel Piyush Sharma Radu Soricut SSL AIMat 377 6,472 0 26 Sep 2019
TinyBERT: Distilling BERT for Natural Language Understanding Xiaoqi Jiao Yichun Yin Lifeng Shang Xin Jiang Xiao Chen Linlin Li F. Wang Qun Liu VLM 113 1,875 0 23 Sep 2019
How multilingual is Multilingual BERT? Telmo Pires Eva Schlinger Dan Garrette LRM VLM 167 1,415 0 04 Jun 2019
Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERT Shijie Wu Mark Dredze VLM SSeg 116 681 0 19 Apr 2019
PAWS: Paraphrase Adversaries from Word Scrambling Yuan Zhang Jason Baldridge Luheng He 83 544 0 01 Apr 2019
Large Batch Optimization for Deep Learning: Training BERT in 76 minutes Yang You Jing Li Sashank J. Reddi Jonathan Hseu Sanjiv Kumar Srinadh Bhojanapalli Xiaodan Song J. Demmel Kurt Keutzer Cho-Jui Hsieh ODL 274 998 0 01 Apr 2019
SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing Taku Kudo John Richardson 214 3,534 0 19 Aug 2018
Neural Network Acceptability Judgments Alex Warstadt Amanpreet Singh Samuel R. Bowman 255 1,413 0 31 May 2018
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 1.1K 7,201 0 20 Apr 2018
A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference Adina Williams Nikita Nangia Samuel R. Bowman 528 4,497 0 18 Apr 2017
SQuAD: 100,000+ Questions for Machine Comprehension of Text Pranav Rajpurkar Jian Zhang Konstantin Lopyrev Percy Liang RALM 318 8,177 0 16 Jun 2016