Optimal Lottery Tickets via SubsetSum: Logarithmic Over-Parameterization is Sufficient

14 June 2020

Dimitris Papailiopoulos

ArXiv PDF HTML

Papers citing "Optimal Lottery Tickets via SubsetSum: Logarithmic Over-Parameterization is Sufficient"

29 / 29 papers shown

Title
Mask in the Mirror: Implicit Sparsification Tom Jacobs R. Burkholz 47 3 0 19 Aug 2024
Survival of the Fittest Representation: A Case Study with Modular Addition Xiaoman Delores Ding Zifan Carl Guo Eric J. Michaud Ziming Liu Max Tegmark 48 3 0 27 May 2024
Polynomially Over-Parameterized Convolutional Neural Networks Contain Structured Strong Winning Lottery Tickets A. D. Cunha Francesco d’Amore Emanuele Natale MLT 27 1 0 16 Nov 2023
Communication-Efficient Federated Learning via Regularized Sparse Random Networks Mohamad Mestoukirdi Omid Esrafilian David Gesbert Qianrui Li N. Gresset FedML 32 0 0 19 Sep 2023
Quantifying lottery tickets under label noise: accuracy, calibration, and complexity V. Arora Daniele Irto Sebastian Goldt G. Sanguinetti 36 2 0 21 Jun 2023
Considering Layerwise Importance in the Lottery Ticket Hypothesis Benjamin Vandersmissen José Oramas 23 1 0 22 Feb 2023
Workload-Balanced Pruning for Sparse Spiking Neural Networks Ruokai Yin Youngeun Kim Yuhang Li Abhishek Moitra Nitin Satpute Anna Hambitzer Priyadarshini Panda 37 19 0 13 Feb 2023
Quantum Neuron Selection: Finding High Performing Subnetworks With Quantum Algorithms Tim Whitaker 30 1 0 12 Feb 2023
Quantum Ridgelet Transform: Winning Lottery Ticket of Neural Networks with Quantum Computation H. Yamasaki Sathyawageeswar Subramanian Satoshi Hayakawa Sho Sonoda MLT 30 4 0 27 Jan 2023
Exploiting the Partly Scratch-off Lottery Ticket for Quantization-Aware Training Yunshan Zhong Gongrui Nan Yu-xin Zhang Rongrong Ji Rongrong Ji MQ 18 3 0 12 Nov 2022
LOFT: Finding Lottery Tickets through Filter-wise Training Qihan Wang Chen Dun Fangshuo Liao C. Jermaine Anastasios Kyrillidis 23 3 0 28 Oct 2022
A General Framework For Proving The Equivariant Strong Lottery Ticket Hypothesis Damien Ferbach Christos Tsirigotis Gauthier Gidel Avishek A. Bose 34 16 0 09 Jun 2022
Convolutional and Residual Networks Provably Contain Lottery Tickets R. Burkholz UQCV MLT 37 13 0 04 May 2022
Most Activation Functions Can Win the Lottery Without Excessive Depth R. Burkholz MLT 71 18 0 04 May 2022
Monarch: Expressive Structured Matrices for Efficient and Accurate Training Tri Dao Beidi Chen N. Sohoni Arjun D Desai Michael Poli Jessica Grogan Alexander Liu Aniruddh Rao Atri Rudra Christopher Ré 22 87 0 01 Apr 2022
The Combinatorial Brain Surgeon: Pruning Weights That Cancel One Another in Neural Networks Xin Yu Thiago Serra Srikumar Ramalingam Shandian Zhe 42 48 0 09 Mar 2022
Extracting Effective Subnetworks with Gumbel-Softmax Robin Dupont M. Alaoui H. Sahbi A. Lebois 14 6 0 25 Feb 2022
Rare Gems: Finding Lottery Tickets at Initialization Kartik K. Sreenivasan Jy-yong Sohn Liu Yang Matthew Grinde Alliot Nagle Hongyi Wang Eric P. Xing Kangwook Lee Dimitris Papailiopoulos 30 42 0 24 Feb 2022
SHRIMP: Sparser Random Feature Models via Iterative Magnitude Pruning Yuege Xie Bobby Shi Hayden Schaeffer Rachel A. Ward 81 9 0 07 Dec 2021
Pixelated Butterfly: Simple and Efficient Sparse training for Neural Network Models Tri Dao Beidi Chen Kaizhao Liang Jiaming Yang Zhao-quan Song Atri Rudra Christopher Ré 33 75 0 30 Nov 2021
Lottery Tickets with Nonzero Biases Jonas Fischer Advait Gadhikar R. Burkholz 19 6 0 21 Oct 2021
A Winning Hand: Compressing Deep Networks Can Improve Out-Of-Distribution Robustness James Diffenderfer Brian Bartoldson Shreya Chaganti Jize Zhang B. Kailkhura OOD 31 69 0 16 Jun 2021
GANs Can Play Lottery Tickets Too Xuxi Chen Zhenyu (Allen) Zhang Yongduo Sui Tianlong Chen GAN 19 58 0 31 May 2021
Playing Lottery Tickets with Vision and Language Zhe Gan Yen-Chun Chen Linjie Li Tianlong Chen Yu Cheng Shuohang Wang Jingjing Liu Lijuan Wang Zicheng Liu VLM 106 54 0 23 Apr 2021
Multi-Prize Lottery Ticket Hypothesis: Finding Accurate Binary Neural Networks by Pruning A Randomly Weighted Network James Diffenderfer B. Kailkhura MQ 35 75 0 17 Mar 2021
Recent Advances on Neural Network Pruning at Initialization Huan Wang Can Qin Yue Bai Yulun Zhang Yun Fu CVBM 33 64 0 11 Mar 2021
Provable Benefits of Overparameterization in Model Compression: From Double Descent to Pruning Neural Networks Xiangyu Chang Yingcong Li Samet Oymak Christos Thrampoulidis 35 50 0 16 Dec 2020
Logarithmic Pruning is All You Need Laurent Orseau Marcus Hutter Omar Rivasplata 23 88 0 22 Jun 2020
What is the State of Neural Network Pruning? Davis W. Blalock Jose Javier Gonzalez Ortiz Jonathan Frankle John Guttag 191 1,027 0 06 Mar 2020