The Shattered Gradients Problem: If resnets are the answer, then what is the question?

28 February 2017

Papers citing "The Shattered Gradients Problem: If resnets are the answer, then what is the question?"

50 / 65 papers shown

Title
SpINR: Neural Volumetric Reconstruction for FMCW Radars Harshvardhan Takawale Nirupam Roy 30 0 0 30 Mar 2025
Lambda-Skip Connections: the architectural component that prevents Rank Collapse Federico Arangath Joseph Jerome Sieber M. Zeilinger Carmen Amo Alonso 33 0 0 14 Oct 2024
Normalization and effective learning rates in reinforcement learning Clare Lyle Zeyu Zheng Khimya Khetarpal James Martens H. V. Hasselt Razvan Pascanu Will Dabney 19 7 0 01 Jul 2024
Explaining Text Similarity in Transformer Models Alexandros Vasileiou Oliver Eberle 43 7 0 10 May 2024
Development of Skip Connection in Deep Neural Networks for Computer Vision and Medical Image Analysis: A Survey Guoping Xu Xiaxia Wang Xinglong Wu Xuesong Leng Yongchao Xu 3DPC 39 8 0 02 May 2024
Activating Wider Areas in Image Super-Resolution Cheng Cheng Hang Wang Hongbin Sun 37 10 0 13 Mar 2024
Neural Redshift: Random Networks are not Random Functions Damien Teney A. Nicolicioiu Valentin Hartmann Ehsan Abbasnejad 103 18 0 04 Mar 2024
Explaining Deep Face Algorithms through Visualization: A Survey Thrupthi Ann S. M. I. C. V. Balasubramanian M. Jawahar CVBM 32 1 0 26 Sep 2023
Densely Connected $G$ -invariant Deep Neural Networks with Signed Permutation Representations Devanshu Agrawal James Ostrowski AI4CE 36 0 0 08 Mar 2023
Understanding plasticity in neural networks Clare Lyle Zeyu Zheng Evgenii Nikishin Bernardo Avila-Pires Razvan Pascanu Will Dabney AI4CE 35 97 0 02 Mar 2023
Investigating Pulse-Echo Sound Speed Estimation in Breast Ultrasound with Deep Learning Walter Simson Magdalini Paschali Vasiliki Sideri-Lampretsa Nassir Navab J. Dahl OOD 13 15 0 06 Feb 2023
Understanding the Spectral Bias of Coordinate Based MLPs Via Training Dynamics J. Lazzari Xiuwen Liu 24 3 0 14 Jan 2023
Shared Coupling-bridge for Weakly Supervised Local Feature Learning Jiayu Sun Jie Zhu Luping Ji 27 6 0 14 Dec 2022
Data-driven Science and Machine Learning Methods in Laser-Plasma Physics Andreas Döpp C. Eberle S. Howard F. Irshad Jinpu Lin M. Streeter AI4CE 32 63 0 30 Nov 2022
SML:Enhance the Network Smoothness with Skip Meta Logit for CTR Prediction Wenlong Deng Lang Lang Ziqiang Liu B. Liu 26 0 0 09 Oct 2022
Stimulative Training of Residual Networks: A Social Psychology Perspective of Loafing Peng Ye Shengji Tang Baopu Li Tao Chen Wanli Ouyang 31 13 0 09 Oct 2022
Dynamical Isometry for Residual Networks Advait Gadhikar R. Burkholz ODL AI4CE 40 2 0 05 Oct 2022
Understanding the Generalization Benefit of Normalization Layers: Sharpness Reduction Kaifeng Lyu Zhiyuan Li Sanjeev Arora FAtt 40 69 0 14 Jun 2022
Model Degradation Hinders Deep Graph Neural Networks Wentao Zhang Zeang Sheng Ziqi Yin Yuezihan Jiang Yikuan Xia Jun Gao Zhi-Xin Yang Bin Cui GNN AI4CE 26 39 0 09 Jun 2022
Dual Decomposition of Convex Optimization Layers for Consistent Attention in Medical Images Tom Ron M. Weiler-Sagie Tamir Hazan FAtt MedIm 24 6 0 06 Jun 2022
Entangled Residual Mappings Mathias Lechner Ramin Hasani Z. Babaiee Radu Grosu Daniela Rus T. Henzinger Sepp Hochreiter 11 4 0 02 Jun 2022
Backdooring Explainable Machine Learning Maximilian Noppel Lukas Peter Christian Wressnegger AAML 16 5 0 20 Apr 2022
Online Convolutional Re-parameterization Mu Hu Junyi Feng Jiashen Hua Baisheng Lai Jianqiang Huang Xiaojin Gong Xiansheng Hua 21 26 0 02 Apr 2022
Deep Learning without Shortcuts: Shaping the Kernel with Tailored Rectifiers Guodong Zhang Aleksandar Botev James Martens OffRL 21 26 0 15 Mar 2022
Investigating the fidelity of explainable artificial intelligence methods for applications of convolutional neural networks in geoscience Antonios Mamalakis E. Barnes I. Ebert‐Uphoff 23 73 0 07 Feb 2022
A Structured Dictionary Perspective on Implicit Neural Representations Gizem Yüce Guillermo Ortiz-Jiménez Beril Besbinar P. Frossard 31 86 0 03 Dec 2021
RMNet: Equivalently Removing Residual Connection from Networks Fanxu Meng Hao Cheng Jia-Xin Zhuang Ke Li Xing Sun 23 11 0 01 Nov 2021
Lottery Tickets with Nonzero Biases Jonas Fischer Advait Gadhikar R. Burkholz 19 6 0 21 Oct 2021
NormFormer: Improved Transformer Pretraining with Extra Normalization Sam Shleifer Jason Weston Myle Ott AI4CE 33 74 0 18 Oct 2021
Reconstructing Cosmic Polarization Rotation with ResUNet-CMB E. Guzman Joel Meyers 30 9 0 20 Sep 2021
An Embedding of ReLU Networks and an Analysis of their Identifiability Pierre Stock Rémi Gribonval 31 17 0 20 Jul 2021
Augmented Shortcuts for Vision Transformers Yehui Tang Kai Han Chang Xu An Xiao Yiping Deng Chao Xu Yunhe Wang ViT 14 39 0 30 Jun 2021
The Future is Log-Gaussian: ResNets and Their Infinite-Depth-and-Width Limit at Initialization Mufan Bill Li Mihai Nica Daniel M. Roy 30 33 0 07 Jun 2021
Understanding Neural Code Intelligence Through Program Simplification Md Rafiqul Islam Rabin Vincent J. Hellendoorn Mohammad Amin Alipour AAML 49 58 0 07 Jun 2021
"BNN - BN = ?": Training Binary Neural Networks without Batch Normalization Tianlong Chen Zhenyu (Allen) Zhang Xu Ouyang Zechun Liu Zhiqiang Shen Zhangyang Wang MQ 40 36 0 16 Apr 2021
Augmenting Deep Classifiers with Polynomial Neural Networks Grigorios G. Chrysos Markos Georgopoulos Jiankang Deng Jean Kossaifi Yannis Panagakis Anima Anandkumar 19 18 0 16 Apr 2021
White Box Methods for Explanations of Convolutional Neural Networks in Image Classification Tasks Meghna P. Ayyar J. Benois-Pineau A. Zemmari FAtt 9 17 0 06 Apr 2021
Attention is Not All You Need: Pure Attention Loses Rank Doubly Exponentially with Depth Yihe Dong Jean-Baptiste Cordonnier Andreas Loukas 46 373 0 05 Mar 2021
High-Performance Large-Scale Image Recognition Without Normalization Andrew Brock Soham De Samuel L. Smith Karen Simonyan VLM 223 512 0 11 Feb 2021
Deep Isometric Learning for Visual Recognition Haozhi Qi Chong You Xinyu Wang Yi Ma Jitendra Malik VLM 30 53 0 30 Jun 2020
Higher-Order Explanations of Graph Neural Networks via Relevant Walks Thomas Schnake Oliver Eberle Jonas Lederer Shinichi Nakajima Kristof T. Schütt Klaus-Robert Muller G. Montavon 32 215 0 05 Jun 2020
Explaining Deep Neural Networks and Beyond: A Review of Methods and Applications Wojciech Samek G. Montavon Sebastian Lapuschkin Christopher J. Anders K. Müller XAI 44 82 0 17 Mar 2020
Batch Normalization Biases Residual Blocks Towards the Identity Function in Deep Networks Soham De Samuel L. Smith ODL 14 20 0 24 Feb 2020
Empirical Studies on the Properties of Linear Regions in Deep Neural Networks Xiao Zhang Dongrui Wu 16 38 0 04 Jan 2020
When Explanations Lie: Why Many Modified BP Attributions Fail Leon Sixt Maximilian Granz Tim Landgraf BDL FAtt XAI 13 132 0 20 Dec 2019
Optimization for deep learning: theory and algorithms Ruoyu Sun ODL 16 168 0 19 Dec 2019
On the Explanation of Machine Learning Predictions in Clinical Gait Analysis D. Slijepcevic Fabian Horst Sebastian Lapuschkin Anna-Maria Raberger Matthias Zeppelzauer Wojciech Samek C. Breiteneder W. Schöllhorn B. Horsak 36 50 0 16 Dec 2019
Towards Best Practice in Explaining Neural Network Decisions with LRP M. Kohlbrenner Alexander Bauer Shinichi Nakajima Alexander Binder Wojciech Samek Sebastian Lapuschkin 16 148 0 22 Oct 2019
Switchable Normalization for Learning-to-Normalize Deep Representation Ping Luo Ruimao Zhang Jiamin Ren Zhanglin Peng Jingyu Li 30 73 0 22 Jul 2019
Enhancing the Robustness of Deep Neural Networks by Boundary Conditional GAN Ke Sun Zhanxing Zhu Zhouchen Lin AAML 19 20 0 28 Feb 2019