Title
Examining Scaling and Transfer of Language Model Architectures for Machine Translation Biao Zhang Behrooz Ghorbani Ankur Bapna Yong Cheng Xavier Garcia Jonathan Shen Orhan Firat 25 21 0 01 Feb 2022
Is the Performance of My Deep Network Too Good to Be True? A Direct Approach to Estimating the Bayes Error in Binary Classification Takashi Ishida Ikko Yamane Nontawat Charoenphakdee Gang Niu Masashi Sugiyama BDL UQCV 48 14 0 01 Feb 2022
Error Scaling Laws for Kernel Classification under Source and Capacity Conditions Hugo Cui Bruno Loureiro Florent Krzakala Lenka Zdeborová 46 10 0 29 Jan 2022
Safe Deep RL in 3D Environments using Human Feedback Matthew Rahtz Vikrant Varma Ramana Kumar Zachary Kenton Shane Legg Jan Leike 32 4 0 20 Jan 2022
Bringing Atomistic Deep Learning to Prime Time Nathan C. Frey S. Samsi Bharath Ramsundar Connor W. Coley V. Gadepally AI4CE 31 0 0 09 Dec 2021
Scalable Geometric Deep Learning on Molecular Graphs Nathan C. Frey S. Samsi Joseph McDonald Lin Li Connor W. Coley V. Gadepally GNN AI4CE 32 4 0 06 Dec 2021
Investigation of Training Label Error Impact on RNN-T I-Fan Chen Brian King J. Droppo 16 5 0 01 Dec 2021
Scaling Up Vision-Language Pre-training for Image Captioning Xiaowei Hu Zhe Gan Jianfeng Wang Zhengyuan Yang Zicheng Liu Yumao Lu Lijuan Wang MLLM VLM 34 246 0 24 Nov 2021
DP-REC: Private & Communication-Efficient Federated Learning Aleksei Triastcyn M. Reisser Christos Louizos FedML 18 16 0 09 Nov 2021
Turing-Universal Learners with Optimal Scaling Laws Preetum Nakkiran 21 2 0 09 Nov 2021
LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs Christoph Schuhmann Richard Vencu Romain Beaumont R. Kaczmarczyk Clayton Mullis Aarush Katta Theo Coombes J. Jitsev Aran Komatsuzaki VLM MLLM CLIP 36 1,377 0 03 Nov 2021
PAGnol: An Extra-Large French Generative Model Julien Launay E. L. Tommasone B. Pannier Franccois Boniface A. Chatelain Alessandro Cappelli Iacopo Poli Djamé Seddah AILaw MoE AI4CE 38 8 0 16 Oct 2021
Scaling Laws for the Few-Shot Adaptation of Pre-trained Image Classifiers Gabriele Prato Simon Guiroy Ethan Caballero Irina Rish Sarath Chandar VLM 34 11 0 13 Oct 2021
Unsupervised Neural Machine Translation with Generative Language Models Only Jesse Michael Han Igor Babuschkin Harrison Edwards Arvind Neelakantan Tao Xu ... Alex Ray Pranav Shyam Aditya A. Ramesh Alec Radford Ilya Sutskever 52 36 0 11 Oct 2021
8-bit Optimizers via Block-wise Quantization Tim Dettmers M. Lewis Sam Shleifer Luke Zettlemoyer MQ 34 273 0 06 Oct 2021
Symbolic Brittleness in Sequence Models: on Systematic Generalization in Symbolic Mathematics Sean Welleck Peter West Jize Cao Yejin Choi 21 28 0 28 Sep 2021
Unsolved Problems in ML Safety Dan Hendrycks Nicholas Carlini John Schulman Jacob Steinhardt 186 275 0 28 Sep 2021
Is the Number of Trainable Parameters All That Actually Matters? A. Chatelain Amine Djeghri Daniel Hesslow Julien Launay Iacopo Poli 51 7 0 24 Sep 2021
Scale Efficiently: Insights from Pre-training and Fine-tuning Transformers Yi Tay Mostafa Dehghani J. Rao W. Fedus Samira Abnar Hyung Won Chung Sharan Narang Dani Yogatama Ashish Vaswani Donald Metzler 206 110 0 22 Sep 2021
Scaling Laws for Neural Machine Translation Behrooz Ghorbani Orhan Firat Markus Freitag Ankur Bapna M. Krikun Xavier Garcia Ciprian Chelba Colin Cherry 40 99 0 16 Sep 2021
Compute and Energy Consumption Trends in Deep Learning Inference Radosvet Desislavov Fernando Martínez-Plumed José Hernández-Orallo 35 113 0 12 Sep 2021
Teaching Autoregressive Language Models Complex Tasks By Demonstration Gabriel Recchia 26 22 0 05 Sep 2021
Robust fine-tuning of zero-shot models Mitchell Wortsman Gabriel Ilharco Jong Wook Kim Mike Li Simon Kornblith ... Raphael Gontijo-Lopes Hannaneh Hajishirzi Ali Farhadi Hongseok Namkoong Ludwig Schmidt VLM 64 691 0 04 Sep 2021
On the Multilingual Capabilities of Very Large-Scale English Language Models Jordi Armengol-Estapé Ona de Gibert Bonet Maite Melero ELM LRM 11 34 0 30 Aug 2021
Why and How Governments Should Monitor AI Development Jess Whittlestone Jack Clark 29 30 0 28 Aug 2021
A Scaling Law for Synthetic-to-Real Transfer: How Much Is Your Pre-training Effective? Hiroaki Mikami Kenji Fukumizu Shogo Murai Shuji Suzuki Yuta Kikuchi Taiji Suzuki S. Maeda Kohei Hayashi 40 12 0 25 Aug 2021
Declarative Machine Learning Systems Piero Molino Christopher Ré AI4TS 21 19 0 16 Jul 2021
A Reinforcement Learning Environment for Mathematical Reasoning via Program Synthesis Joseph Palermo Johnny Ye Alok Singh AIMat 20 2 0 15 Jul 2021
MarIA: Spanish Language Models Asier Gutiérrez-Fandiño Jordi Armengol-Estapé Marc Pàmies Joan Llop-Palao Joaquín Silveira-Ocampo C. Carrino Aitor Gonzalez-Agirre Carme Armentano-Oller Carlos Rodríguez-Penagos Marta Villegas VLM 24 119 0 15 Jul 2021
Compact and Optimal Deep Learning with Recurrent Parameter Generators Jiayun Wang Yubei Chen Stella X. Yu Brian Cheung Yann LeCun BDL 40 4 0 15 Jul 2021
Transflower: probabilistic autoregressive dance generation with multimodal attention Guillermo Valle Pérez G. Henter Jonas Beskow A. Holzapfel Pierre-Yves Oudeyer Simon Alexanderson 30 42 0 25 Jun 2021
Scaling Laws for Acoustic Models J. Droppo Oguz H. Elibol 15 22 0 11 Jun 2021
Scaling Vision Transformers Xiaohua Zhai Alexander Kolesnikov N. Houlsby Lucas Beyer ViT 73 1,060 0 08 Jun 2021
Redundant representations help generalization in wide neural networks Diego Doimo Aldo Glielmo Sebastian Goldt A. Laio AI4CE 33 9 0 07 Jun 2021
Effect of Pre-Training Scale on Intra- and Inter-Domain Full and Few-Shot Transfer Learning for Natural and Medical X-Ray Chest Images Mehdi Cherti J. Jitsev LM&MA 22 23 0 31 May 2021
Search Spaces for Neural Model Training Darko Stosic Dusan Stosic 25 4 0 27 May 2021
High-Resolution Complex Scene Synthesis with Transformers Manuel Jahn Robin Rombach Bjorn Ommer ViT 15 37 0 13 May 2021
Are Larger Pretrained Language Models Uniformly Better? Comparing Performance at the Instance Level Ruiqi Zhong Dhruba Ghosh Dan Klein Jacob Steinhardt 33 35 0 13 May 2021
Which transformer architecture fits my data? A vocabulary bottleneck in self-attention Noam Wies Yoav Levine Daniel Jannai Amnon Shashua 40 20 0 09 May 2021
AI Risk Skepticism Roman V. Yampolskiy 29 4 0 02 May 2021
Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus Jesse Dodge Maarten Sap Ana Marasović William Agnew Gabriel Ilharco Dirk Groeneveld Margaret Mitchell Matt Gardner AILaw 32 425 0 18 Apr 2021
Scaling Scaling Laws with Board Games Andrew Jones 8 39 0 07 Apr 2021
TENT: Efficient Quantization of Neural Networks on the tiny Edge with Tapered FixEd PoiNT H. F. Langroudi Vedant Karia Tej Pandit Dhireesha Kudithipudi MQ 21 10 0 06 Apr 2021
Revisiting ResNets: Improved Training and Scaling Strategies Irwan Bello W. Fedus Xianzhi Du E. D. Cubuk A. Srinivas Nayeon Lee Jonathon Shlens Barret Zoph 29 298 0 13 Mar 2021
Measuring Mathematical Problem Solving With the MATH Dataset Dan Hendrycks Collin Burns Saurav Kadavath Akul Arora Steven Basart Eric Tang D. Song Jacob Steinhardt ReLM FaML 84 1,840 0 05 Mar 2021
Generating Images with Sparse Representations C. Nash Jacob Menick Sander Dieleman Peter W. Battaglia 33 199 0 05 Mar 2021
A Minimalist Dataset for Systematic Generalization of Perception, Syntax, and Semantics Qing Li Siyuan Huang Yining Hong Yixin Zhu Ying Nian Wu Song-Chun Zhu AIMat 19 6 0 02 Mar 2021
Investigating the Limitations of Transformers with Simple Arithmetic Tasks Rodrigo Nogueira Zhiying Jiang Jimmy J. Li LRM 24 122 0 25 Feb 2021
Improved Denoising Diffusion Probabilistic Models Alex Nichol Prafulla Dhariwal DiffM 60 3,526 0 18 Feb 2021
Explaining Neural Scaling Laws Yasaman Bahri Ethan Dyer Jared Kaplan Jaehoon Lee Utkarsh Sharma 27 250 0 12 Feb 2021