Gradient Flow in Sparse Neural Networks and How Lottery Tickets Win

7 October 2020

Papers citing "Gradient Flow in Sparse Neural Networks and How Lottery Tickets Win"

21 / 21 papers shown

Title
Sparse Training from Random Initialization: Aligning Lottery Ticket Masks using Weight Symmetry Mohammed Adnan Rohan Jain Ekansh Sharma Rahul Krishnan Yani Andrew Ioannou 56 0 0 08 May 2025
Onboard Optimization and Learning: A Survey Monirul Islam Pavel Siyi Hu Mahardhika Pratama Ryszard Kowalczyk 26 0 0 07 May 2025
Information Consistent Pruning: How to Efficiently Search for Sparse Networks? Soheil Gharatappeh Salimeh Yasaei Sekeh 59 0 0 28 Jan 2025
Pushing the Limits of Sparsity: A Bag of Tricks for Extreme Pruning Andy Li A. Durrant Milan Markovic Lu Yin Georgios Leontidis Tianlong Chen Lu Yin Georgios Leontidis 75 0 0 20 Nov 2024
Navigating Extremes: Dynamic Sparsity in Large Output Spaces Nasib Ullah Erik Schultheis Mike Lasby Yani Andrew Ioannou Rohit Babbar 35 0 0 05 Nov 2024
Dynamic Sparse Training versus Dense Training: The Unexpected Winner in Image Corruption Robustness Boqian Wu Q. Xiao Shunxin Wang N. Strisciuglio Mykola Pechenizkiy M. V. Keulen Decebal Constantin Mocanu Elena Mocanu OOD 3DH 57 0 0 03 Oct 2024
Sparse Weight Averaging with Multiple Particles for Iterative Magnitude Pruning Moonseok Choi Hyungi Lee G. Nam Juho Lee 40 2 0 24 May 2023
NTK-SAP: Improving neural network pruning by aligning training dynamics Yite Wang Dawei Li Ruoyu Sun 42 19 0 06 Apr 2023
Considering Layerwise Importance in the Lottery Ticket Hypothesis Benjamin Vandersmissen José Oramas 37 1 0 22 Feb 2023
Balance is Essence: Accelerating Sparse Training via Adaptive Gradient Correction Bowen Lei Dongkuan Xu Ruqi Zhang Shuren He Bani Mallick 37 6 0 09 Jan 2023
Unmasking the Lottery Ticket Hypothesis: What's Encoded in a Winning Ticket's Mask? Mansheej Paul F. Chen Brett W. Larsen Jonathan Frankle Surya Ganguli Gintare Karolina Dziugaite UQCV 35 38 0 06 Oct 2022
Comprehensive Graph Gradual Pruning for Sparse Training in Graph Neural Networks Chuang Liu Xueqi Ma Yinbing Zhan Liang Ding Dapeng Tao Bo Du Wenbin Hu Danilo Mandic 42 28 0 18 Jul 2022
Deep Ensembling with No Overhead for either Training or Testing: The All-Round Blessings of Dynamic Sparsity Shiwei Liu Tianlong Chen Zahra Atashgahi Xiaohan Chen Ghada Sokar Elena Mocanu Mykola Pechenizkiy Zhangyang Wang Decebal Constantin Mocanu OOD 31 49 0 28 Jun 2021
The Elastic Lottery Ticket Hypothesis Xiaohan Chen Yu Cheng Shuohang Wang Zhe Gan Jingjing Liu Zhangyang Wang OOD 23 34 0 30 Mar 2021
Recent Advances on Neural Network Pruning at Initialization Huan Wang Can Qin Yue Bai Yulun Zhang Yun Fu CVBM 33 64 0 11 Mar 2021
Sparse Training Theory for Scalable and Efficient Agents Decebal Constantin Mocanu Elena Mocanu T. Pinto Selima Curci Phuong H. Nguyen M. Gibescu D. Ernst Z. Vale 45 17 0 02 Mar 2021
Truly Sparse Neural Networks at Scale Selima Curci Decebal Constantin Mocanu Mykola Pechenizkiy 35 19 0 02 Feb 2021
Efficient Estimation of Influence of a Training Instance Sosuke Kobayashi Sho Yokoi Jun Suzuki Kentaro Inui TDI 32 15 0 08 Dec 2020
Comparing Rewinding and Fine-tuning in Neural Network Pruning Alex Renda Jonathan Frankle Michael Carbin 229 383 0 05 Mar 2020
The large learning rate phase of deep learning: the catapult mechanism Aitor Lewkowycz Yasaman Bahri Ethan Dyer Jascha Narain Sohl-Dickstein Guy Gur-Ari ODL 159 235 0 04 Mar 2020
Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000-Layer Vanilla Convolutional Neural Networks Lechao Xiao Yasaman Bahri Jascha Narain Sohl-Dickstein S. Schoenholz Jeffrey Pennington 244 349 0 14 Jun 2018