Randomness In Neural Network Training: Characterizing The Impact of Tooling

22 June 2021

ArXiv (abs)PDF HTML Github (14★)

Papers citing "Randomness In Neural Network Training: Characterizing The Impact of Tooling"

50 / 51 papers shown

Title
Mixtera: A Data Plane for Foundation Model Training Maximilian Böther Xiaozhe Yao Tolga Kerimoglu Ana Klimovic Viktor Gsteiger Ana Klimovic MoE 192 0 0 27 Feb 2025
Deep Learning-Based Identification of Inconsistent Method Names: How Far Are We? Taiming Wang Yuxia Zhang Lin Jiang Yi Tang Guangjie Li Hui Liu 177 2 0 22 Jan 2025
Generalizability of experimental studies Federico Matteucci Vadim Arzamasov Jose Cribeiro-Ramallo Marco Heyden Konstantin Ntounas Klemens Bohm 102 0 0 25 Jun 2024
MLPerf Tiny Benchmark Colby R. Banbury Vijay Janapa Reddi P. Torelli J. Holleman Nat Jeffries ... Videet Parekh Honson Tran Nhan Tran Niu Wenxu Xu Xuesong VLM 125 190 0 14 Jun 2021
Manipulating SGD with Data Ordering Attacks Ilia Shumailov Zakhar Shumaylov Dmitry Kazhdan Yiren Zhao Nicolas Papernot Murat A. Erdogdu Ross J. Anderson AAML 152 97 0 19 Apr 2021
Nondeterminism and Instability in Neural Network Optimization Cecilia Summers M. Dinneen 70 41 0 08 Mar 2021
Accounting for Variance in Machine Learning Benchmarks Xavier Bouthillier Pierre Delaunay Mirko Bronzi Assya Trofimov Brennan Nichyporuk ... Dmitriy Serdyuk Tal Arbel C. Pal Gaël Varoquaux Pascal Vincent 109 151 0 01 Mar 2021
Synthesizing Irreproducibility in Deep Networks R. Snapp G. Shamir OOD 62 10 0 21 Feb 2021
Underspecification Presents Challenges for Credibility in Modern Machine Learning Alexander DÁmour Katherine A. Heller D. Moldovan Ben Adlam B. Alipanahi ... Kellie Webster Steve Yadlowsky T. Yun Xiaohua Zhai D. Sculley OffRL 143 688 0 06 Nov 2020
Smooth activations and reproducibility in deep networks G. Shamir Dong Lin Lorenzo Coviello 66 23 0 20 Oct 2020
Anti-Distillation: Improving reproducibility of deep networks G. Shamir Lorenzo Coviello 101 19 0 19 Oct 2020
Characterising Bias in Compressed Models Sara Hooker Nyalleng Moorosi Gregory Clark Samy Bengio Emily L. Denton 79 185 0 06 Oct 2020
What is being transferred in transfer learning? Behnam Neyshabur Hanie Sedghi Chiyuan Zhang 122 528 0 26 Aug 2020
Beyond Point Estimate: Inferring Ensemble Prediction Variation from Neuron Activation Strength in Recommender Systems Zhe Chen Yuyan Wang Dong Lin D. Cheng Lichan Hong Ed H. Chi Claire Cui 100 17 0 17 Aug 2020
Distributed Training of Deep Learning Models: A Taxonomic Perspective M. Langer Zhen He W. Rahayu Yanbo Xue 70 78 0 08 Jul 2020
We Need to Talk About Random Splits Anders Søgaard Sebastian Ebert Jasmijn Bastings Katja Filippova 89 100 0 01 May 2020
PyTorch: An Imperative Style, High-Performance Deep Learning Library Adam Paszke Sam Gross Francisco Massa Adam Lerer James Bradbury ... Sasank Chilamkurthy Benoit Steiner Lu Fang Junjie Bai Soumith Chintala ODL 580 42,677 0 03 Dec 2019
What Do Compressed Deep Neural Networks Forget? Sara Hooker Aaron Courville Gregory Clark Yann N. Dauphin Andrea Frome 107 185 0 13 Nov 2019
MLPerf Training Benchmark Arya D. McCarthy Christine Cheng Cody Coleman Greg Diamos Paulius Micikevicius ... Carole-Jean Wu Lingjie Xu Masafumi Yamazaki C. Young Matei A. Zaharia 103 316 0 02 Oct 2019
Hidden Stratification Causes Clinically Meaningful Failures in Machine Learning for Medical Imaging Luke Oakden-Rayner Jared A. Dunnmon G. Carneiro Christopher Ré OOD 83 383 0 27 Sep 2019
On Model Stability as a Function of Random Seed Pranava Madhyastha Dhruv Batra 97 63 0 23 Sep 2019
A Step Toward Quantifying Independently Reproducible Machine Learning Research Edward Raff 67 131 0 14 Sep 2019
Reproducibility in Machine Learning for Health Matthew B. A. McDermott Shirly Wang N. Marinsek Rajesh Ranganath Marzyeh Ghassemi L. Foschini AI4TS 46 52 0 02 Jul 2019
Further advantages of data augmentation on convolutional neural networks Alex Hernández-García Peter König 70 108 0 26 Jun 2019
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks Mingxing Tan Quoc V. Le 3DV MedIm 197 18,224 0 28 May 2019
Are All Layers Created Equal? Chiyuan Zhang Samy Bengio Y. Singer 107 140 0 06 Feb 2019
Deep Learning Predicts Hip Fracture using Confounding Patient and Healthcare Variables Giovanni Sutanto J. Zech Luke Oakden-Rayner Yevgen Chebotar Manway Liu William Gale M. McConnell Ankur Handa Thomas M. Snyder Dieter Fox AI4CE OOD 89 245 0 08 Nov 2018
Measuring the Effects of Data Parallelism on Neural Network Training Christopher J. Shallue Jaehoon Lee J. Antognini J. Mamou J. Ketterling Yao Wang 100 409 0 08 Nov 2018
Mesh-TensorFlow: Deep Learning for Supercomputers Noam M. Shazeer Youlong Cheng Niki Parmar Dustin Tran Ashish Vaswani ... HyoukJoong Lee O. Milenkovic C. Young Ryan Sepassi Blake Hechtman GNN MoE AI4CE 92 393 0 05 Nov 2018
Are GANs Created Equal? A Large-Scale Study Mario Lucic Karol Kurach Marcin Michalski Sylvain Gelly Olivier Bousquet EGVM 122 1,013 0 28 Nov 2017
DLPaper2Code: Auto-generation of Code from Deep Learning Research Papers Akshay Sethi A. Sankaran Naveen Panwar Shreya Khare Senthil Mani 3DV 73 34 0 09 Nov 2017
Don't Decay the Learning Rate, Increase the Batch Size Samuel L. Smith Pieter-Jan Kindermans Chris Ying Quoc V. Le ODL 124 996 0 01 Nov 2017
Regularization for Deep Learning: A Taxonomy J. Kukačka Vladimir Golkov Daniel Cremers 88 336 0 29 Oct 2017
Deep Reinforcement Learning that Matters Peter Henderson Riashat Islam Philip Bachman Joelle Pineau Doina Precup David Meger OffRL 147 1,963 0 19 Sep 2017
Random Erasing Data Augmentation Zhun Zhong Liang Zheng Guoliang Kang Shaozi Li Yi Yang 116 3,652 0 16 Aug 2017
Regularizing and Optimizing LSTM Language Models Stephen Merity N. Keskar R. Socher 178 1,096 0 07 Aug 2017
Cut, Paste and Learn: Surprisingly Easy Synthesis for Instance Detection Debidatta Dwibedi Ishan Misra M. Hebert 108 624 0 04 Aug 2017
On the State of the Art of Evaluation in Neural Language Models Gábor Melis Chris Dyer Phil Blunsom 75 536 0 18 Jul 2017
Knowledge Base Completion: Baselines Strike Back Rudolf Kadlec Ondrej Bajgar Jan Kleindienst 76 190 0 30 May 2017
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications Andrew G. Howard Menglong Zhu Bo Chen Dmitry Kalenichenko Weijun Wang Tobias Weyand M. Andreetto Hartwig Adam 3DH 1.2K 20,918 0 17 Apr 2017
In-Datacenter Performance Analysis of a Tensor Processing Unit N. Jouppi C. Young Nishant Patil David Patterson Gaurav Agrawal ... Vijay Vasudevan Richard Walter Walter Wang Eric Wilcox Doe Hyun Yoon 239 4,648 0 16 Apr 2017
Densely Connected Convolutional Networks Gao Huang Zhuang Liu Laurens van der Maaten Kilian Q. Weinberger PINN 3DV 881 36,910 0 25 Aug 2016
TensorFlow: A system for large-scale machine learning Martín Abadi P. Barham Jianmin Chen Zhiwen Chen Andy Davis ... Vijay Vasudevan Pete Warden Martin Wicke Yuan Yu Xiaoqiang Zhang GNN AI4CE 435 18,361 0 27 May 2016
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 2.3K 194,641 0 10 Dec 2015
Rethinking the Inception Architecture for Computer Vision Christian Szegedy Vincent Vanhoucke Sergey Ioffe Jonathon Shlens Z. Wojna 3DV BDL 886 27,444 0 02 Dec 2015
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift Sergey Ioffe Christian Szegedy OOD 471 43,357 0 11 Feb 2015
Deep Learning Face Attributes in the Wild Ziwei Liu Ping Luo Xiaogang Wang Xiaoou Tang CVBM 268 8,433 0 28 Nov 2014
cuDNN: Efficient Primitives for Deep Learning Sharan Chetlur Cliff Woolley Philippe Vandermersch Jonathan M. Cohen J. Tran Bryan Catanzaro Evan Shelhamer 144 1,849 0 03 Oct 2014
Very Deep Convolutional Networks for Large-Scale Image Recognition Karen Simonyan Andrew Zisserman FAtt MDE 1.7K 100,575 0 04 Sep 2014
ImageNet Large Scale Visual Recognition Challenge Olga Russakovsky Jia Deng Hao Su J. Krause S. Satheesh ... A. Karpathy A. Khosla Michael S. Bernstein Alexander C. Berg Li Fei-Fei VLM ObjD 1.7K 39,637 0 01 Sep 2014