mT5: A massively multilingual pre-trained text-to-text transformer

22 October 2020

Papers citing "mT5: A massively multilingual pre-trained text-to-text transformer"

50 / 469 papers shown

Title
LiT: Zero-Shot Transfer with Locked-image text Tuning Xiaohua Zhai Tianlin Li Basil Mustafa Andreas Steiner Daniel Keysers Alexander Kolesnikov Lucas Beyer VLM 48 543 0 15 Nov 2021
ÚFAL at MultiLexNorm 2021: Improving Multilingual Lexical Normalization by Fine-tuning ByT5 David Samuel Milan Straka 12 15 0 28 Oct 2021
SLAM: A Unified Encoder for Speech and Language Modeling via Speech-Text Joint Pre-Training Ankur Bapna Yu-An Chung Na Wu Anmol Gulati Ye Jia J. Clark Melvin Johnson Jason Riesa Alexis Conneau Yu Zhang VLM 61 94 0 20 Oct 2021
Sharpness-Aware Minimization Improves Language Model Generalization Dara Bahri H. Mobahi Yi Tay 133 98 0 16 Oct 2021
Prix-LM: Pretraining for Multilingual Knowledge Base Construction Wenxuan Zhou Fangyu Liu Ivan Vulić Nigel Collier Muhao Chen KELM 72 18 0 16 Oct 2021
Why don't people use character-level machine translation? Jindrich Libovický Helmut Schmid Alexander Fraser 65 28 0 15 Oct 2021
GlobalWoZ: Globalizing MultiWoZ to Develop Multilingual Task-Oriented Dialogue Systems Bosheng Ding Junjie Hu Lidong Bing Sharifah Aljunied Mahani Chenyu You Luo Si Chunyan Miao 45 41 0 14 Oct 2021
Few-shot Controllable Style Transfer for Low-Resource Multilingual Settings Kalpesh Krishna Deepak Nathani Xavier Garcia Bidisha Samanta Partha P. Talukdar 40 24 0 14 Oct 2021
Cross-Lingual Open-Domain Question Answering with Answer Sentence Generation Benjamin Muller Luca Soldaini Rik Koncel-Kedziorski Eric Lind Alessandro Moschitti LRM 38 7 0 14 Oct 2021
Learning Compact Metrics for MT Amy Pu Hyung Won Chung Ankur P. Parikh Sebastian Gehrmann Thibault Sellam 33 98 0 12 Oct 2021
Unsupervised Neural Machine Translation with Generative Language Models Only Jesse Michael Han Igor Babuschkin Harrison Edwards Arvind Neelakantan Tao Xu ... Alex Ray Pranav Shyam Aditya A. Ramesh Alec Radford Ilya Sutskever 52 36 0 11 Oct 2021
Sequential Reptile: Inter-Task Gradient Alignment for Multilingual Learning Seanie Lee Haebeom Lee Juho Lee Sung Ju Hwang MoMe CLL 48 16 0 06 Oct 2021
Multilingual Document-Level Translation Enables Zero-Shot Transfer From Sentences to Documents Biao Zhang Ankur Bapna Melvin Johnson A. Dabirmoghaddam N. Arivazhagan Orhan Firat 34 12 0 21 Sep 2021
BARTpho: Pre-trained Sequence-to-Sequence Models for Vietnamese Nguyen Luong Tran Duong Minh Le Dat Quoc Nguyen 19 52 0 20 Sep 2021
Enforcing fairness in private federated learning via the modified method of differential multipliers Borja Rodríguez Gálvez Filip Granqvist Rogier van Dalen M. Seigel FedML 48 52 0 17 Sep 2021
Language Models are Few-shot Multilingual Learners Genta Indra Winata Andrea Madotto Zhaojiang Lin Rosanne Liu J. Yosinski Pascale Fung ELM LRM 36 132 0 16 Sep 2021
Cross-lingual Transfer of Monolingual Models Evangelia Gogoulou Ariel Ekgren T. Isbister Magnus Sahlgren 29 17 0 15 Sep 2021
Allocating Large Vocabulary Capacity for Cross-lingual Language Model Pre-training Bo Zheng Li Dong Shaohan Huang Saksham Singhal Wanxiang Che Ting Liu Xia Song Furu Wei VLM 21 22 0 15 Sep 2021
CPT: A Pre-Trained Unbalanced Transformer for Both Chinese Language Understanding and Generation Yunfan Shao Zhichao Geng Yitao Liu Junqi Dai Hang Yan Fei Yang Li Zhe Hujun Bao Xipeng Qiu MedIm 70 148 0 13 Sep 2021
Multilingual Translation via Grafting Pre-trained Language Models Zewei Sun Mingxuan Wang Lei Li AI4CE 191 22 0 11 Sep 2021
AfroMT: Pretraining Strategies and Reproducible Benchmarks for Translation of 8 African Languages Machel Reid Junjie Hu Graham Neubig Y. Matsuo 77 31 0 10 Sep 2021
What Changes Can Large-scale Language Models Bring? Intensive Study on HyperCLOVA: Billions-scale Korean Generative Pretrained Transformers Boseop Kim Hyoungseok Kim Sang-Woo Lee Gichang Lee Donghyun Kwak ... Jaewook Kang Inho Kang Jung-Woo Ha W. Park Nako Sung VLM 249 121 0 10 Sep 2021
PPT: Pre-trained Prompt Tuning for Few-shot Learning Yuxian Gu Xu Han Zhiyuan Liu Minlie Huang VLM 54 403 0 09 Sep 2021
IndicBART: A Pre-trained Model for Indic Natural Language Generation Raj Dabre Himani Shrotriya Anoop Kunchukuttan Ratish Puduppully Mitesh M. Khapra Pratyush Kumar 39 70 0 07 Sep 2021
Nearest Neighbour Few-Shot Learning for Cross-lingual Classification M Saiful Bari Batool Haider Saab Mansour VLM 19 13 0 06 Sep 2021
MultiEURLEX -- A multi-lingual and multi-label legal document classification dataset for zero-shot cross-lingual transfer Ilias Chalkidis Manos Fergadiotis Ion Androutsopoulos AILaw 27 107 0 02 Sep 2021
Boosting Cross-Lingual Transfer via Self-Learning with Uncertainty Estimation Liyan Xu Xuchao Zhang Xujiang Zhao Haifeng Chen F. Chen Jinho Choi 25 14 0 01 Sep 2021
mMARCO: A Multilingual Version of the MS MARCO Passage Ranking Dataset L. Bonifacio Vitor Jeronymo Hugo Queiroz Abonizio Israel Campiotti Marzieh Fadaee R. Lotufo Rodrigo Nogueira 40 108 0 31 Aug 2021
AraT5: Text-to-Text Transformers for Arabic Language Generation El Moatez Billah Nagoudi AbdelRahim Elmadany Muhammad Abdul-Mageed 92 118 0 31 Aug 2021
LOT: A Story-Centric Benchmark for Evaluating Chinese Long Text Understanding and Generation Jian Guan Zhuoer Feng Yamei Chen Ru He Xiaoxi Mao Changjie Fan Minlie Huang 39 32 0 30 Aug 2021
Multi-Task Self-Training for Learning General Representations Golnaz Ghiasi Barret Zoph E. D. Cubuk Quoc V. Le Nayeon Lee SSL 24 100 0 25 Aug 2021
Are the Multilingual Models Better? Improving Czech Sentiment with Transformers Pavel Přibáň J. Steinberger 36 11 0 24 Aug 2021
MTG: A Benchmark Suite for Multilingual Text Generation Yiran Chen Zhenqiao Song Xianze Wu Danqing Wang Jingjing Xu Jiaze Chen Hao Zhou Lei Li LRM VLM 37 22 0 13 Aug 2021
PARADISE: Exploiting Parallel Data for Multilingual Sequence-to-Sequence Pretraining Machel Reid Mikel Artetxe VLM 50 26 0 04 Aug 2021
Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing Pengfei Liu Weizhe Yuan Jinlan Fu Zhengbao Jiang Hiroaki Hayashi Graham Neubig VLM SyDa 55 3,838 0 28 Jul 2021
One Question Answering Model for Many Languages with Cross-lingual Dense Passage Retrieval Akari Asai Xinyan Velocity Yu Jungo Kasai Hannaneh Hajishirzi RALM LRM 27 70 0 26 Jul 2021
Modelling Latent Translations for Cross-Lingual Transfer Edoardo Ponti Julia Kreutzer Ivan Vulić Siva Reddy 32 18 0 23 Jul 2021
Deduplicating Training Data Makes Language Models Better Katherine Lee Daphne Ippolito A. Nystrom Chiyuan Zhang Douglas Eck Chris Callison-Burch Nicholas Carlini SyDa 242 593 0 14 Jul 2021
A Primer on Pretrained Multilingual Language Models Sumanth Doddapaneni Gowtham Ramesh Mitesh M. Khapra Anoop Kunchukuttan Pratyush Kumar LRM 43 74 0 01 Jul 2021
Revisiting the Primacy of English in Zero-shot Cross-lingual Transfer Iulia Turc Kenton Lee Jacob Eisenstein Ming-Wei Chang Kristina Toutanova 26 58 0 30 Jun 2021
Overview of BioASQ 2021: The ninth BioASQ challenge on Large-Scale Biomedical Semantic Indexing and Question Answering A. Nentidis K. Bougiatiotis Carlos Rodríguez-Penagos Anastasia Krithara Marta Villegas Martin Krallinger George Giannakopoulos 29 45 0 28 Jun 2021
XL-Sum: Large-Scale Multilingual Abstractive Summarization for 44 Languages Tahmid Hasan Abhik Bhattacharjee Md. Saiful Islam Kazi Samin Mubasshir Yuan-Fang Li Yong-Bin Kang M. Rahman Rifat Shahriyar 37 344 0 25 Jun 2021
Charformer: Fast Character Transformers via Gradient-based Subword Tokenization Yi Tay Vinh Q. Tran Sebastian Ruder Jai Gupta Hyung Won Chung Dara Bahri Zhen Qin Simon Baumgartner Cong Yu Donald Metzler 51 152 0 23 Jun 2021
CPM-2: Large-scale Cost-effective Pre-trained Language Models Zhengyan Zhang Yuxian Gu Xu Han Shengqi Chen Chaojun Xiao ... Minlie Huang Wentao Han Yang Liu Xiaoyan Zhu Maosong Sun MoE 37 86 0 20 Jun 2021
Automatic Construction of Evaluation Suites for Natural Language Generation Datasets Simon Mille Kaustubh D. Dhole Saad Mahamood Laura Perez-Beltrachini Varun Gangal Mihir Kale Emiel van Miltenburg Sebastian Gehrmann ELM 42 22 0 16 Jun 2021
Specializing Multilingual Language Models: An Empirical Study Ethan C. Chau Noah A. Smith 27 27 0 16 Jun 2021
BEiT: BERT Pre-Training of Image Transformers Hangbo Bao Li Dong Songhao Piao Furu Wei ViT 68 2,749 0 15 Jun 2021
Common Sense Beyond English: Evaluating and Improving Multilingual Language Models for Commonsense Reasoning Bill Yuchen Lin Seyeon Lee Xiaoyang Qiao Xiang Ren ReLM LRM 27 61 0 13 Jun 2021
Improving Pretrained Cross-Lingual Language Models via Self-Labeled Word Alignment Zewen Chi Li Dong Bo Zheng Shaohan Huang Xian-Ling Mao Heyan Huang Furu Wei 45 67 0 11 Jun 2021
Towards More Equitable Question Answering Systems: How Much More Data Do You Need? Arnab Debnath Navid Rajabi F. Alam Antonios Anastasopoulos 22 11 0 28 May 2021