PeLLE: Encoder-based language models for Brazilian Portuguese based on
open data

PeLLE: Encoder-based language models for Brazilian Portuguese based on open data

29 February 2024

Guilherme Lamartine de Mello

Marcos Menon Jose

Pedro Henrique Domingues

ArXiv (abs)PDF HTML

Papers citing "PeLLE: Encoder-based language models for Brazilian Portuguese based on open data"

17 / 17 papers shown

Title
Advancing Neural Encoding of Portuguese with Transformer Albertina PT-* João Rodrigues Luís Gomes Joao Silva António Branco Rodrigo Santos Henrique Lopes Cardoso T. Osório 32 44 0 11 May 2023
Carolina: a General Corpus of Contemporary Brazilian Portuguese with Provenance, Typology and Versioning Information MariaJose Crespo Maria Lina Rocha Mariana Sturzeneker F. Serras Guilherme Lamartine de Mello ... Mariana M. Silva Marcelo Finger Maria Clara Paixão de Sousa Cristiane Namiuti Vanessa Martins do Monte 16 3 0 28 Mar 2023
Towards a Cleaner Document-Oriented Multilingual Crawled Corpus Julien Abadji Pedro Ortiz Suarez Laurent Romary Benoît Sagot CLL 89 159 0 17 Jan 2022
How to Train BERT with an Academic Budget Peter Izsak Moshe Berchansky Omer Levy 98 118 0 15 Apr 2021
HateBR: A Large Expert Annotated Corpus of Brazilian Instagram Comments for Offensive Language and Hate Speech Detection F. Vargas Isabelle Carvalho F. Góes Fabrício Benevenuto T. Pardo AILaw 27 17 0 27 Mar 2021
mT5: A massively multilingual pre-trained text-to-text transformer Linting Xue Noah Constant Adam Roberts Mihir Kale Rami Al-Rfou Aditya Siddhant Aditya Barua Colin Raffel 142 2,559 0 22 Oct 2020
DeBERTa: Decoding-enhanced BERT with Disentangled Attention Pengcheng He Xiaodong Liu Jianfeng Gao Weizhu Chen AAML 163 2,750 0 05 Jun 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 865 42,379 0 28 May 2020
The State and Fate of Linguistic Diversity and Inclusion in the NLP World Pratik M. Joshi Sebastin Santy A. Budhiraja Kalika Bali Monojit Choudhury LMTD 122 856 0 20 Apr 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 611 4,905 0 23 Jan 2020
Unsupervised Cross-lingual Representation Learning at Scale Alexis Conneau Kartikay Khandelwal Naman Goyal Vishrav Chaudhary Guillaume Wenzek Francisco Guzmán Edouard Grave Myle Ott Luke Zettlemoyer Veselin Stoyanov 228 6,587 0 05 Nov 2019
On the Cross-lingual Transferability of Monolingual Representations Mikel Artetxe Sebastian Ruder Dani Yogatama 202 798 0 25 Oct 2019
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 470 20,317 0 23 Oct 2019
RoBERTa: A Robustly Optimized BERT Pretraining Approach Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy M. Lewis Luke Zettlemoyer Veselin Stoyanov AIMat 677 24,541 0 26 Jul 2019
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.8K 95,175 0 11 Oct 2018
Population Based Training of Neural Networks Max Jaderberg Valentin Dalibard Simon Osindero Wojciech M. Czarnecki Jeff Donahue ... Tim Green Iain Dunning Karen Simonyan Chrisantha Fernando Koray Kavukcuoglu 93 744 0 27 Nov 2017
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 732 132,363 0 12 Jun 2017

We use cookies and other tracking technologies to improve your browsing experience on our website, to show you personalized content and targeted ads, to analyze our website traffic, and to understand where our visitors are coming from. See our policy.