Most Activation Functions Can Win the Lottery Without Excessive Depth

4 May 2022

Papers citing "Most Activation Functions Can Win the Lottery Without Excessive Depth"

15 / 15 papers shown

Title
Sign-In to the Lottery: Reparameterizing Sparse Training From Scratch Advait Gadhikar Tom Jacobs Chao Zhou R. Burkholz 32 0 0 17 Apr 2025
On the Surprising Effectiveness of Attention Transfer for Vision Transformers Alexander C. Li Yuandong Tian Bin Chen Deepak Pathak Xinlei Chen 43 0 0 14 Nov 2024
Mask in the Mirror: Implicit Sparsification Tom Jacobs R. Burkholz 47 3 0 19 Aug 2024
Cyclic Sparse Training: Is it Enough? Advait Gadhikar Sree Harsha Nelaturu R. Burkholz CLL 47 0 0 04 Jun 2024
Spectral Graph Pruning Against Over-Squashing and Over-Smoothing Adarsh Jamadandi Celia Rubio-Madrigal R. Burkholz 45 1 0 06 Apr 2024
A Survey of Lottery Ticket Hypothesis Bohan Liu Zijie Zhang Peixiong He Zhensen Wang Yang Xiao Ruimeng Ye Yang Zhou Wei-Shinn Ku Bo Hui UQCV 39 12 0 07 Mar 2024
Masks, Signs, And Learning Rate Rewinding Advait Gadhikar R. Burkholz 57 8 0 29 Feb 2024
Polynomially Over-Parameterized Convolutional Neural Networks Contain Structured Strong Winning Lottery Tickets A. D. Cunha Francesco d’Amore Emanuele Natale MLT 27 1 0 16 Nov 2023
Quantifying lottery tickets under label noise: accuracy, calibration, and complexity V. Arora Daniele Irto Sebastian Goldt G. Sanguinetti 36 2 0 21 Jun 2023
Workload-Balanced Pruning for Sparse Spiking Neural Networks Ruokai Yin Youngeun Kim Yuhang Li Abhishek Moitra Nitin Satpute Anna Hambitzer Priyadarshini Panda 37 19 0 13 Feb 2023
Why Random Pruning Is All We Need to Start Sparse Advait Gadhikar Sohom Mukherjee R. Burkholz 43 19 0 05 Oct 2022
Dynamical Isometry for Residual Networks Advait Gadhikar R. Burkholz ODL AI4CE 40 2 0 05 Oct 2022
A General Framework For Proving The Equivariant Strong Lottery Ticket Hypothesis Damien Ferbach Christos Tsirigotis Gauthier Gidel Avishek A. Bose 34 16 0 09 Jun 2022
Convolutional and Residual Networks Provably Contain Lottery Tickets R. Burkholz UQCV MLT 37 13 0 04 May 2022
Comparing Rewinding and Fine-tuning in Neural Network Pruning Alex Renda Jonathan Frankle Michael Carbin 224 383 0 05 Mar 2020