A global convergence theory for deep ReLU implicit networks via
over-parameterization

A global convergence theory for deep ReLU implicit networks via over-parameterization

11 October 2021

Papers citing "A global convergence theory for deep ReLU implicit networks via over-parameterization"

14 / 14 papers shown

Title
KIWI: A Dataset of Knowledge-Intensive Writing Instructions for Answering Research Questions Fangyuan Xu Kyle Lo Luca Soldaini Bailey Kuehl Eunsol Choi David Wadden 37 6 0 06 Mar 2024
Deep Equilibrium Models are Almost Equivalent to Not-so-deep Explicit Models for High-dimensional Gaussian Mixtures Zenan Ling Longbo Li Zhanbo Feng Yixuan Zhang Feng Zhou Robert C. Qiu Zhenyu Liao 32 4 0 05 Feb 2024
Wide Neural Networks as Gaussian Processes: Lessons from Deep Equilibrium Models Tianxiang Gao Xiaokai Huo Hailiang Liu Hongyang Gao BDL 17 8 0 16 Oct 2023
Homotopy Relaxation Training Algorithms for Infinite-Width Two-Layer ReLU Neural Networks Yahong Yang Qipin Chen Wenrui Hao 29 4 0 26 Sep 2023
Global Convergence Rate of Deep Equilibrium Models with General Activations Lan V. Truong 39 2 0 11 Feb 2023
On the optimization and generalization of overparameterized implicit neural networks Tianxiang Gao Hongyang Gao MLT AI4CE 19 3 0 30 Sep 2022
Magnitude and Angle Dynamics in Training Single ReLU Neurons Sangmin Lee Byeongsu Sim Jong Chul Ye MLT 96 6 0 27 Sep 2022
Deep equilibrium networks are sensitive to initialization statistics Atish Agarwala S. Schoenholz 16 7 0 19 Jul 2022
Informed Learning by Wide Neural Networks: Convergence, Generalization and Sampling Complexity Jianyi Yang Shaolei Ren 29 3 0 02 Jul 2022
Global Convergence of Over-parameterized Deep Equilibrium Models Zenan Ling Xingyu Xie Qiuhao Wang Zongpeng Zhang Zhouchen Lin 32 12 0 27 May 2022
Gradient Descent Optimizes Infinite-Depth ReLU Implicit Networks with Linear Widths Tianxiang Gao Hongyang Gao MLT 35 5 0 16 May 2022
On the Proof of Global Convergence of Gradient Descent for Deep ReLU Networks with Linear Widths Quynh N. Nguyen 38 49 0 24 Jan 2021
Almost Sure Convergence of Dropout Algorithms for Neural Networks Albert Senen-Cerda J. Sanders 16 8 0 06 Feb 2020
Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Łojasiewicz Condition Hamed Karimi J. Nutini Mark W. Schmidt 139 1,199 0 16 Aug 2016