Topology and Geometry of Half-Rectified Network Optimization

v1v2v3v4 (latest)

Topology and Geometry of Half-Rectified Network Optimization

4 November 2016

Joan Bruna

ArXiv (abs)PDF HTML

Papers citing "Topology and Geometry of Half-Rectified Network Optimization"

19 / 19 papers shown

Title
Understanding Mode Connectivity via Parameter Space Symmetry B. Zhao Nima Dehmamy Robin Walters Rose Yu 211 8 0 29 May 2025
Sens-Merging: Sensitivity-Guided Parameter Balancing for Merging Large Language Models Shuqi Liu Han Wu Bowei He Xiongwei Han Mingxuan Yuan Linqi Song MoMe 109 3 0 20 Feb 2025
Federated Learning over Connected Modes Dennis Grinwald Philipp Wiesner Shinichi Nakajima FedML 141 0 0 05 Mar 2024
Analysis of Linear Mode Connectivity via Permutation-Based Weight Matching: With Insights into Other Permutation Search Methods Akira Ito Masanori Yamada Atsutoshi Kumagai MoMe 99 6 0 06 Feb 2024
Deep Semi-Random Features for Nonlinear Function Approximation Kenji Kawaguchi Bo Xie Vikas Verma Le Song 118 15 0 28 Feb 2017
Local minima in training of neural networks G. Swirszcz Wojciech M. Czarnecki Razvan Pascanu ODL 61 73 0 19 Nov 2016
Distribution-Specific Hardness of Learning Neural Networks Ohad Shamir 77 117 0 05 Sep 2016
No bad local minima: Data independent training error guarantees for multilayer neural networks Daniel Soudry Y. Carmon 185 235 0 26 May 2016
Deep Learning without Poor Local Minima Kenji Kawaguchi ODL 219 923 0 23 May 2016
Gradient Descent Converges to Minimizers Jason D. Lee Max Simchowitz Michael I. Jordan Benjamin Recht 71 211 0 16 Feb 2016
On the Quality of the Initial Basin in Overspecified Neural Networks Itay Safran Ohad Shamir 72 127 0 13 Nov 2015
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift Sergey Ioffe Christian Szegedy OOD 463 43,305 0 11 Feb 2015
Adam: A Method for Stochastic Optimization Diederik P. Kingma Jimmy Ba ODL 1.9K 150,115 0 22 Dec 2014
Explorations on high dimensional landscapes Levent Sagun V. U. Güney Gerard Ben Arous Yann LeCun 52 65 0 20 Dec 2014
Qualitatively characterizing neural network optimization problems Ian Goodfellow Oriol Vinyals Andrew M. Saxe ODL 110 522 0 19 Dec 2014
The Loss Surfaces of Multilayer Networks A. Choromańska Mikael Henaff Michaël Mathieu Gerard Ben Arous Yann LeCun ODL 261 1,198 0 30 Nov 2014
Identifying and attacking the saddle point problem in high-dimensional non-convex optimization Yann N. Dauphin Razvan Pascanu Çağlar Gülçehre Kyunghyun Cho Surya Ganguli Yoshua Bengio ODL 129 1,385 0 10 Jun 2014
Exact solutions to the nonlinear dynamics of learning in deep linear neural networks Andrew M. Saxe James L. McClelland Surya Ganguli ODL 173 1,849 0 20 Dec 2013
Convex relaxations of structured matrix factorizations Francis R. Bach 87 38 0 12 Sep 2013