Global Optimality Beyond Two Layers: Training Deep ReLU Networks via Convex Programs

11 October 2021

Papers citing "Global Optimality Beyond Two Layers: Training Deep ReLU Networks via Convex Programs"

24 / 24 papers shown

Title
The Real Tropical Geometry of Neural Networks Marie-Charlotte Brandenburg Georg Loho Guido Montúfar 59 7 0 18 Mar 2024
Convex Relaxations of ReLU Neural Networks Approximate Global Optima in Polynomial Time Sungyoon Kim Mert Pilanci 48 4 0 06 Feb 2024
The Convex Landscape of Neural Networks: Characterizing Global Optima and Stationary Points via Lasso Models Tolga Ergen Mert Pilanci 16 2 0 19 Dec 2023
From Complexity to Clarity: Analytical Expressions of Deep Neural Network Weights via Clifford's Geometric Algebra and Convexity Mert Pilanci 42 2 0 28 Sep 2023
Fixing the NTK: From Neural Network Linearizations to Exact Convex Programs Rajat Vadiraj Dwaraknath Tolga Ergen Mert Pilanci 35 0 0 26 Sep 2023
Optimal Sets and Solution Paths of ReLU Networks Aaron Mishkin Mert Pilanci 42 3 0 31 May 2023
When Deep Learning Meets Polyhedral Theory: A Survey Joey Huchette Gonzalo Muñoz Thiago Serra Calvin Tsay AI4CE 94 32 0 29 Apr 2023
Convex Dual Theory Analysis of Two-Layer Convolutional Neural Networks with Soft-Thresholding Chunyan Xiong Meng Lu Xiaotong Yu JIAN-PENG Cao Zhong Chen D. Guo X. Qu MLT 40 0 0 14 Apr 2023
Globally Optimal Training of Neural Networks with Threshold Activation Functions Tolga Ergen Halil Ibrahim Gulluk Jonathan Lacotte Mert Pilanci 65 8 0 06 Mar 2023
Piecewise Linear Neural Networks and Deep Learning Qinghua Tao Li Li Xiaolin Huang Xiangming Xi Shuning Wang Johan A. K. Suykens 17 28 0 18 Jun 2022
Unraveling Attention via Convex Duality: Analysis and Interpretations of Vision Transformers Arda Sahiner Tolga Ergen Batu Mehmet Ozturkler John M. Pauly Morteza Mardani Mert Pilanci 37 33 0 17 May 2022
Deep Learning meets Nonparametric Regression: Are Weight-Decayed DNNs Locally Adaptive? Kaiqi Zhang Yu-Xiang Wang 17 12 0 20 Apr 2022
Fast Convex Optimization for Two-Layer ReLU Networks: Equivalent Model Classes and Cone Decompositions Aaron Mishkin Arda Sahiner Mert Pilanci OffRL 77 30 0 02 Feb 2022
Efficient Global Optimization of Two-layer ReLU Networks: Quadratic-time Algorithms and Adversarial Training Yatong Bai Tanmay Gautam Somayeh Sojoudi AAML 22 17 0 06 Jan 2022
Neural networks with linear threshold activations: structure and algorithms Sammy Khalife Hongyu Cheng A. Basu 42 14 0 15 Nov 2021
Path Regularization: A Convexity and Sparsity Inducing Regularization for Parallel ReLU Networks Tolga Ergen Mert Pilanci 32 16 0 18 Oct 2021
The Convex Geometry of Backpropagation: Neural Network Gradient Flows Converge to Extreme Points of the Dual Convex Program Yifei Wang Mert Pilanci MLT MDE 55 11 0 13 Oct 2021
Parallel Deep Neural Networks Have Zero Duality Gap Yifei Wang Tolga Ergen Mert Pilanci 79 10 0 13 Oct 2021
Hidden Convexity of Wasserstein GANs: Interpretable Generative Models with Closed-Form Solutions Arda Sahiner Tolga Ergen Batu Mehmet Ozturkler Burak Bartan John M. Pauly Morteza Mardani Mert Pilanci GAN 20 20 0 12 Jul 2021
Practical Convex Formulation of Robust One-hidden-layer Neural Network Training Yatong Bai Tanmay Gautam Yujie Gai Somayeh Sojoudi AAML 19 3 0 25 May 2021
Demystifying Batch Normalization in ReLU Networks: Equivalent Convex Optimization Models and Implicit Regularization Tolga Ergen Arda Sahiner Batu Mehmet Ozturkler John M. Pauly Morteza Mardani Mert Pilanci 19 31 0 02 Mar 2021
Vector-output ReLU Neural Network Problems are Copositive Programs: Convex Analysis of Two Layer Networks and Polynomial-time Algorithms Arda Sahiner Tolga Ergen John M. Pauly Mert Pilanci MLT 14 43 0 24 Dec 2020
Principled Deep Neural Network Training through Linear Programming D. Bienstock Gonzalo Muñoz Sebastian Pokutta 29 24 0 07 Oct 2018
Aggregated Residual Transformations for Deep Neural Networks Saining Xie Ross B. Girshick Piotr Dollár Z. Tu Kaiming He 297 10,220 0 16 Nov 2016