Data Scaling Laws in NMT: The Effect of Noise and Architecture

4 February 2022

Colin Cherry

Papers citing "Data Scaling Laws in NMT: The Effect of Noise and Architecture"

35 / 35 papers shown

Title
RouterEval: A Comprehensive Benchmark for Routing LLMs to Explore Model-level Scaling Up in LLMs Zhongzhan Huang Guoming Ling Vincent S. Liang Yupei Lin Yandong Chen Shanshan Zhong Hefeng Wu LRM 200 7 0 08 Mar 2025
LaMDA: Language Models for Dialog Applications R. Thoppilan Daniel De Freitas Jamie Hall Noam M. Shazeer Apoorv Kulshreshtha ... Blaise Aguera-Arcas Claire Cui M. Croak Ed H. Chi Quoc Le ALM 146 1,602 0 20 Jan 2022
Scaling Laws for Neural Machine Translation Behrooz Ghorbani Orhan Firat Markus Freitag Ankur Bapna M. Krikun Xavier Garcia Ciprian Chelba Colin Cherry 85 102 0 16 Sep 2021
An Empirical Exploration in Quality Filtering of Text Data Leo Gao 58 11 0 02 Sep 2021
To Ship or Not to Ship: An Extensive Evaluation of Automatic Metrics for Machine Translation Tom Kocmi C. Federmann Roman Grundkiewicz Marcin Junczys-Dowmunt Hitokazu Matsushita Arul Menezes 93 210 0 22 Jul 2021
Accuracy on the Line: On the Strong Correlation Between Out-of-Distribution and In-Distribution Generalization John Miller Rohan Taori Aditi Raghunathan Shiori Sagawa Pang Wei Koh Vaishaal Shankar Percy Liang Y. Carmon Ludwig Schmidt OODD OOD 112 278 0 09 Jul 2021
Language Models are Good Translators Shuo Wang Zhaopeng Tu Zhixing Tan Wenxuan Wang Maosong Sun Yang Liu 67 21 0 25 Jun 2021
Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translation Markus Freitag George F. Foster David Grangier Viresh Ratnakar Qijun Tan Wolfgang Macherey 160 392 0 29 Apr 2021
Explaining Neural Scaling Laws Yasaman Bahri Ethan Dyer Jared Kaplan Jaehoon Lee Utkarsh Sharma 83 269 0 12 Feb 2021
Beyond English-Centric Multilingual Machine Translation Angela Fan Shruti Bhosale Holger Schwenk Zhiyi Ma Ahmed El-Kishky ... Vitaliy Liptchinsky Sergey Edunov Edouard Grave Michael Auli Armand Joulin LRM 96 859 0 21 Oct 2020
Learning Curves for Analysis of Deep Networks Derek Hoiem Tanmay Gupta Zhizhong Li Michal Shlapentokh-Rothman 77 26 0 21 Oct 2020
Recipes for Safety in Open-domain Chatbots Jing Xu Da Ju Margaret Li Y-Lan Boureau Jason Weston Emily Dinan 85 234 0 14 Oct 2020
COMET: A Neural Framework for MT Evaluation Ricardo Rei Craig Alan Stewart Ana C. Farinha A. Lavie 154 1,098 0 18 Sep 2020
GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding Dmitry Lepikhin HyoukJoong Lee Yuanzhong Xu Dehao Chen Orhan Firat Yanping Huang M. Krikun Noam M. Shazeer Zhiwen Chen MoE 138 1,194 0 30 Jun 2020
Deep Encoder, Shallow Decoder: Reevaluating Non-autoregressive Machine Translation Jungo Kasai Nikolaos Pappas Hao Peng James Cross Noah A. Smith 113 140 0 18 Jun 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 908 42,520 0 28 May 2020
Leveraging Monolingual Data with Self-Supervision for Multilingual Neural Machine Translation Aditya Siddhant Ankur Bapna Yuan Cao Orhan Firat Mengzhao Chen Sneha Kudugunta N. Arivazhagan Yonghui Wu 75 86 0 11 May 2020
A Neural Scaling Law from the Dimension of the Data Manifold Utkarsh Sharma Jared Kaplan 79 53 0 22 Apr 2020
BLEU might be Guilty but References are not Innocent Markus Freitag David Grangier Isaac Caswell 74 149 0 13 Apr 2020
BLEURT: Learning Robust Metrics for Text Generation Thibault Sellam Dipanjan Das Ankur P. Parikh 113 1,507 0 09 Apr 2020
On Layer Normalization in the Transformer Architecture Ruibin Xiong Yunchang Yang Di He Kai Zheng Shuxin Zheng Chen Xing Huishuai Zhang Yanyan Lan Liwei Wang Tie-Yan Liu AI4CE 153 998 0 12 Feb 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 651 4,925 0 23 Jan 2020
Multilingual Denoising Pre-training for Neural Machine Translation Yinhan Liu Jiatao Gu Naman Goyal Xian Li Sergey Edunov Marjan Ghazvininejad M. Lewis Luke Zettlemoyer AI4CE AIMat 128 1,816 0 22 Jan 2020
Big Transfer (BiT): General Visual Representation Learning Alexander Kolesnikov Lucas Beyer Xiaohua Zhai J. Puigcerver Jessica Yung Sylvain Gelly N. Houlsby MQ 295 1,211 0 24 Dec 2019
CCAligned: A Massive Collection of Cross-Lingual Web-Document Pairs Ahmed El-Kishky Vishrav Chaudhary Francisco Guzman Philipp Koehn 103 199 0 10 Nov 2019
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 515 20,376 0 23 Oct 2019
A Constructive Prediction of the Generalization Error Across Scales Jonathan S. Rosenfeld Amir Rosenfeld Yonatan Belinkov Nir Shavit 105 215 0 27 Sep 2019
Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges N. Arivazhagan Ankur Bapna Orhan Firat Dmitry Lepikhin Melvin Johnson ... George F. Foster Colin Cherry Wolfgang Macherey Zhiwen Chen Yonghui Wu 96 428 0 11 Jul 2019
The Effect of Translationese in Machine Translation Test Sets Mike Zhang Antonio Toral 71 88 0 19 Jun 2019
Dual Conditional Cross-Entropy Filtering of Noisy Parallel Corpora Marcin Junczys-Dowmunt 63 135 0 01 Sep 2018
Denoising Neural Machine Translation Training with Trusted Data and Online Data Selection Wei Wang Taro Watanabe Macduff Hughes Tetsuji Nakagawa Ciprian Chelba 79 91 0 31 Aug 2018
On the Impact of Various Types of Noise on Neural Machine Translation Huda Khayrallah Philipp Koehn AAML 70 220 0 31 May 2018
The Best of Both Worlds: Combining Recent Advances in Neural Machine Translation Mengzhao Chen Orhan Firat Ankur Bapna Melvin Johnson Wolfgang Macherey ... Niki Parmar M. Schuster Zhifeng Chen Yonghui Wu Macduff Hughes AIMat 80 457 0 26 Apr 2018
Adafactor: Adaptive Learning Rates with Sublinear Memory Cost Noam M. Shazeer Mitchell Stern ODL 90 1,053 0 11 Apr 2018
Improving Neural Machine Translation Models with Monolingual Data Rico Sennrich Barry Haddow Alexandra Birch 266 2,724 0 20 Nov 2015