Qualitatively characterizing neural network optimization problems

19 December 2014

Papers citing "Qualitatively characterizing neural network optimization problems"

50 / 125 papers shown

Title
Extracting Global Dynamics of Loss Landscape in Deep Learning Models Mohammed Eslami Hamed Eramian Marcio Gameiro W. Kalies Konstantin Mischaikow 23 1 0 14 Jun 2021
Analyzing Monotonic Linear Interpolation in Neural Network Loss Landscapes James Lucas Juhan Bae Michael Ruogu Zhang Stanislav Fort R. Zemel Roger C. Grosse MoMe 172 28 0 22 Apr 2021
The Low-Rank Simplicity Bias in Deep Networks Minyoung Huh H. Mobahi Richard Y. Zhang Brian Cheung Pulkit Agrawal Phillip Isola 30 110 0 18 Mar 2021
Loss Surface Simplexes for Mode Connecting Volumes and Fast Ensembling Gregory W. Benton Wesley J. Maddox Sanae Lotfi A. Wilson UQCV 33 67 0 25 Feb 2021
Visualization of Nonlinear Programming for Robot Motion Planning David Hägele Moataz Abdelaal Ozgur S. Oguz Marc Toussaint Daniel Weiskopf 20 3 0 28 Jan 2021
Combating Mode Collapse in GAN training: An Empirical Analysis using Hessian Eigenvalues Ricard Durall Avraam Chatzimichailidis P. Labus J. Keuper GAN 30 58 0 17 Dec 2020
PEP: Parameter Ensembling by Perturbation Alireza Mehrtash Purang Abolmaesumi Polina Golland Tina Kapur Demian Wassermann W. Wells 25 10 0 24 Oct 2020
Softmax Deep Double Deterministic Policy Gradients Ling Pan Qingpeng Cai Longbo Huang 72 86 0 19 Oct 2020
Gradient Flow in Sparse Neural Networks and How Lottery Tickets Win Utku Evci Yani Andrew Ioannou Cem Keskin Yann N. Dauphin 40 87 0 07 Oct 2020
A Comparative Study of Deep Learning Loss Functions for Multi-Label Remote Sensing Image Classification Hichame Yessou Gencer Sumbul Begüm Demir 19 31 0 29 Sep 2020
Distributed Training of Deep Learning Models: A Taxonomic Perspective M. Langer Zhen He W. Rahayu Yanbo Xue 30 76 0 08 Jul 2020
Understanding the Role of Training Regimes in Continual Learning Seyed Iman Mirzadeh Mehrdad Farajtabar Razvan Pascanu H. Ghasemzadeh CLL 21 219 0 12 Jun 2020
Pruning artificial neural networks: a way to find well-generalizing, high-entropy sharp minima Enzo Tartaglione Andrea Bragagnolo Marco Grangetto 31 11 0 30 Apr 2020
Symmetry & critical points for a model shallow neural network Yossi Arjevani M. Field 36 13 0 23 Mar 2020
Critical Point-Finding Methods Reveal Gradient-Flat Regions of Deep Network Losses Charles G. Frye James B. Simon Neha S. Wadia A. Ligeralde M. DeWeese K. Bouchard ODL 16 2 0 23 Mar 2020
On the Effectiveness of Mitigating Data Poisoning Attacks with Gradient Shaping Sanghyun Hong Varun Chandrasekaran Yigitcan Kaya Tudor Dumitras Nicolas Papernot AAML 28 136 0 26 Feb 2020
SNIFF: Reverse Engineering of Neural Networks with Fault Attacks J. Breier Dirmanto Jap Xiaolu Hou S. Bhasin Yang Liu 17 53 0 23 Feb 2020
The Break-Even Point on Optimization Trajectories of Deep Neural Networks Stanislaw Jastrzebski Maciej Szymczak Stanislav Fort Devansh Arpit Jacek Tabor Kyunghyun Cho Krzysztof J. Geras 50 155 0 21 Feb 2020
Gradient Surgery for Multi-Task Learning Tianhe Yu Saurabh Kumar Abhishek Gupta Sergey Levine Karol Hausman Chelsea Finn 41 1,174 0 19 Jan 2020
Optimization for deep learning: theory and algorithms Ruoyu Sun ODL 27 168 0 19 Dec 2019
Deep Ensembles: A Loss Landscape Perspective Stanislav Fort Huiyi Hu Balaji Lakshminarayanan OOD UQCV 29 619 0 05 Dec 2019
GradVis: Visualization and Second Order Analysis of Optimization Surfaces during the Training of Deep Neural Networks Avraam Chatzimichailidis Franz-Josef Pfreundt N. Gauger J. Keuper 21 10 0 26 Sep 2019
Visualizing Movement Control Optimization Landscapes Perttu Hämäläinen Juuso Toikka Amin Babadi Karen Liu 24 7 0 17 Sep 2019
Visualizing and Understanding the Effectiveness of BERT Y. Hao Li Dong Furu Wei Ke Xu 27 181 0 15 Aug 2019
Visualizing the PHATE of Neural Networks Scott A. Gigante Adam S. Charles Smita Krishnaswamy Gal Mishne 36 37 0 07 Aug 2019
Weight-space symmetry in deep networks gives rise to permutation saddles, connected by equal-loss valleys across the loss landscape Johanni Brea Berfin Simsek Bernd Illing W. Gerstner 23 55 0 05 Jul 2019
Decentralized Bayesian Learning over Graphs Anusha Lalitha Xinghan Wang O. Kilinc Y. Lu T. Javidi F. Koushanfar FedML 28 25 0 24 May 2019
How degenerate is the parametrization of neural networks with the ReLU activation function? Julius Berner Dennis Elbrächter Philipp Grohs ODL 33 28 0 23 May 2019
Interpreting and Evaluating Neural Network Robustness Fuxun Yu Zhuwei Qin Chenchen Liu Liang Zhao Yanzhi Wang Xiang Chen AAML 15 55 0 10 May 2019
Parameter Efficient Training of Deep Convolutional Neural Networks by Dynamic Sparse Reparameterization Hesham Mostafa Xin Wang 37 307 0 15 Feb 2019
Asymmetric Valleys: Beyond Sharp and Flat Local Minima Haowei He Gao Huang Yang Yuan ODL MLT 28 148 0 02 Feb 2019
An Investigation into Neural Net Optimization via Hessian Eigenvalue Density Behrooz Ghorbani Shankar Krishnan Ying Xiao ODL 18 317 0 29 Jan 2019
Overfitting Mechanism and Avoidance in Deep Neural Networks Shaeke Salman Xiuwen Liu 17 139 0 19 Jan 2019
On the Convergence Rate of Training Recurrent Neural Networks Zeyuan Allen-Zhu Yuanzhi Li Zhao Song 32 191 0 29 Oct 2018
Collaborative Deep Learning Across Multiple Data Centers Kele Xu Haibo Mi Dawei Feng Huaimin Wang Chuan Chen Zibin Zheng Xu Lan FedML 134 18 0 16 Oct 2018
Distributed learning of deep neural network over multiple agents O. Gupta Ramesh Raskar FedML OOD 19 598 0 14 Oct 2018
Bayesian Deep Convolutional Networks with Many Channels are Gaussian Processes Roman Novak Lechao Xiao Jaehoon Lee Yasaman Bahri Greg Yang Jiri Hron Daniel A. Abolafia Jeffrey Pennington Jascha Narain Sohl-Dickstein UQCV BDL 25 307 0 11 Oct 2018
Implicit Self-Regularization in Deep Neural Networks: Evidence from Random Matrix Theory and Implications for Learning Charles H. Martin Michael W. Mahoney AI4CE 47 192 0 02 Oct 2018
Interpreting Adversarial Robustness: A View from Decision Surface in Input Space Fuxun Yu Chenchen Liu Yanzhi Wang Liang Zhao Xiang Chen AAML OOD 36 27 0 29 Sep 2018
A theoretical framework for deep locally connected ReLU network Yuandong Tian PINN 25 10 0 28 Sep 2018
On the loss landscape of a class of deep neural networks with no bad local valleys Quynh N. Nguyen Mahesh Chandra Mukkamala Matthias Hein 16 87 0 27 Sep 2018
Benchmarking five global optimization approaches for nano-optical shape optimization and parameter reconstruction Philipp‐Immanuel Schneider Xavier Garcia Santiago V. Soltwisch M. Hammerschmidt Sven Burger C. Rockstuhl 19 88 0 18 Sep 2018
Don't Use Large Mini-Batches, Use Local SGD Tao R. Lin Sebastian U. Stich Kumar Kshitij Patel Martin Jaggi 57 429 0 22 Aug 2018
Troubling Trends in Machine Learning Scholarship Zachary Chase Lipton Jacob Steinhardt 29 288 0 09 Jul 2018
PCA of high dimensional random walks with comparison to neural network training J. Antognini Jascha Narain Sohl-Dickstein OOD 27 27 0 22 Jun 2018
Using transfer learning to detect galaxy mergers Sandro Ackermann Kevin Schawinski Ce Zhang Anna K. Weigel M. D. Turp 3DPC 14 110 0 25 May 2018
SmoothOut: Smoothing Out Sharp Minima to Improve Generalization in Deep Learning W. Wen Yandan Wang Feng Yan Cong Xu Chunpeng Wu Yiran Chen H. Li 24 50 0 21 May 2018
Measuring the Intrinsic Dimension of Objective Landscapes Chunyuan Li Heerad Farkhoor Rosanne Liu J. Yosinski 29 401 0 24 Apr 2018
The Loss Surface of XOR Artificial Neural Networks D. Mehta Xiaojun Zhao Edgar A. Bernal D. Wales 34 19 0 06 Apr 2018
Averaging Weights Leads to Wider Optima and Better Generalization Pavel Izmailov Dmitrii Podoprikhin T. Garipov Dmitry Vetrov A. Wilson FedML MoMe 60 1,621 0 14 Mar 2018