Optimal Lottery Tickets via SubsetSum: Logarithmic Over-Parameterization is Sufficient

14 June 2020

Dimitris Papailiopoulos

ArXiv PDF HTML

Papers citing "Optimal Lottery Tickets via SubsetSum: Logarithmic Over-Parameterization is Sufficient"

50 / 95 papers shown

Title
Communication-Efficient Personalized Distributed Learning with Data and Node Heterogeneity Zhuojun Tian Zhaoyang Zhang Yiwei Li Mehdi Bennis 172 0 0 24 Apr 2025
Trading-off Accuracy and Communication Cost in Federated Learning Mattia Jacopo Villani Emanuele Natale Frederik Mallmann-Trenn FedML 115 0 0 18 Mar 2025
On the Surprising Effectiveness of Attention Transfer for Vision Transformers Alexander C. Li Yuandong Tian Bin Chen Deepak Pathak Xinlei Chen 61 3 0 14 Nov 2024
Finding Strong Lottery Ticket Networks with Genetic Algorithms Philipp Altmann Julian Schonberger Maximilian Zorn Thomas Gabor 63 1 0 07 Nov 2024
Mask in the Mirror: Implicit Sparsification Tom Jacobs R. Burkholz 170 4 0 19 Aug 2024
Masked Random Noise for Communication Efficient Federaetd Learning Shiwei Li Yingyi Cheng Haozhao Wang Xing Tang Shijie Xu Weihong Luo Yuhua Li Dugang Liu Xiuqiang He and Ruixuan Li FedML 68 1 0 06 Aug 2024
Cyclic Sparse Training: Is it Enough? Advait Gadhikar Sree Harsha Nelaturu R. Burkholz CLL 82 0 0 04 Jun 2024
Sparser, Better, Deeper, Stronger: Improving Sparse Training with Exact Orthogonal Initialization A. Nowak Lukasz Gniecki Filip Szatkowski Jacek Tabor 75 1 0 03 Jun 2024
Survival of the Fittest Representation: A Case Study with Modular Addition Xiaoman Delores Ding Zifan Carl Guo Eric J. Michaud Ziming Liu Max Tegmark 103 4 0 27 May 2024
Online Algorithms with Limited Data Retention Nicole Immorlica Brendan Lucier Markus Mobius James Siderius 65 1 0 17 Apr 2024
A Survey of Lottery Ticket Hypothesis Bohan Liu Zijie Zhang Peixiong He Zhensen Wang Yang Xiao Ruimeng Ye Yang Zhou Wei-Shinn Ku Bo Hui UQCV 79 15 0 07 Mar 2024
Masks, Signs, And Learning Rate Rewinding Advait Gadhikar R. Burkholz 82 9 0 29 Feb 2024
Multicoated and Folded Graph Neural Networks with Strong Lottery Tickets Jiale Yan Hiroaki Ito Ángel López García-Arias Yasuyuki Okoshi Hikari Otsuka Kazushi Kawamura Thiem Van Chu Masato Motomura 67 3 0 06 Dec 2023
Transformers are uninterpretable with myopic methods: a case study with bounded Dyck grammars Kaiyue Wen Yuchen Li Bing Liu Andrej Risteski 56 24 0 03 Dec 2023
Polynomially Over-Parameterized Convolutional Neural Networks Contain Structured Strong Winning Lottery Tickets A. D. Cunha Francesco d’Amore Emanuele Natale MLT 59 1 0 16 Nov 2023
Bridging Lottery Ticket and Grokking: Understanding Grokking from Inner Structure of Networks Gouki Minegishi Yusuke Iwasawa Yutaka Matsuo 46 3 0 30 Oct 2023
Proving Linear Mode Connectivity of Neural Networks via Optimal Transport Damien Ferbach Baptiste Goujaud Gauthier Gidel Aymeric Dieuleveut MoMe 103 16 0 29 Oct 2023
Communication-Efficient Federated Learning via Regularized Sparse Random Networks Mohamad Mestoukirdi Omid Esrafilian David Gesbert Qianrui Li N. Gresset FedML 64 0 0 19 Sep 2023
Sparse Binary Transformers for Multivariate Time Series Modeling Matt Gorbett Hossein Shirazi I. Ray AI4TS 67 14 0 09 Aug 2023
Quantifying lottery tickets under label noise: accuracy, calibration, and complexity V. Arora Daniele Irto Sebastian Goldt G. Sanguinetti 73 2 0 21 Jun 2023
Pruning at Initialization -- A Sketching Perspective Noga Bar Raja Giryes 100 1 0 27 May 2023
Probabilistic Modeling: Proving the Lottery Ticket Hypothesis in Spiking Neural Network Man Yao Yu-Liang Chou Guangshe Zhao Xiawu Zheng Yonghong Tian Boxing Xu Guoqi Li 66 4 0 20 May 2023
Sparsifying Bayesian neural networks with latent binary variables and normalizing flows Lars Skaaret-Lund G. Storvik A. Hubin BDL UQCV 62 3 0 05 May 2023
Randomly Initialized Subnetworks with Iterative Weight Recycling Matt Gorbett L. D. Whitley 51 4 0 28 Mar 2023
ExplainFix: Explainable Spatially Fixed Deep Networks Alex Gaudio Christos Faloutsos A. Smailagic P. Costa A. Campilho FAtt 52 3 0 18 Mar 2023
Considering Layerwise Importance in the Lottery Ticket Hypothesis Benjamin Vandersmissen José Oramas 52 1 0 22 Feb 2023
Workload-Balanced Pruning for Sparse Spiking Neural Networks Ruokai Yin Youngeun Kim Yuhang Li Abhishek Moitra Nitin Satpute Anna Hambitzer Priyadarshini Panda 81 21 0 13 Feb 2023
Quantum Neuron Selection: Finding High Performing Subnetworks With Quantum Algorithms Tim Whitaker 58 1 0 12 Feb 2023
Quantum Ridgelet Transform: Winning Lottery Ticket of Neural Networks with Quantum Computation H. Yamasaki Sathyawageeswar Subramanian Satoshi Hayakawa Sho Sonoda MLT 61 4 0 27 Jan 2023
Pruning Before Training May Improve Generalization, Provably Hongru Yang Yingbin Liang Xiaojie Guo Lingfei Wu Zhangyang Wang MLT 46 2 0 01 Jan 2023
Exploiting the Partly Scratch-off Lottery Ticket for Quantization-Aware Training Mingliang Xu Gongrui Nan Yuxin Zhang Chia-Wen Lin Rongrong Ji MQ 48 3 0 12 Nov 2022
$Strong Lottery Ticket Hypothesis with $\varepsilon$--perturbation$ Strong Lottery Ticket Hypothesis with $\varepsilon$ --perturbation Zheyang Xiong Fangshuo Liao Anastasios Kyrillidis 51 2 0 29 Oct 2022
LOFT: Finding Lottery Tickets through Filter-wise Training Qihan Wang Chen Dun Fangshuo Liao C. Jermaine Anastasios Kyrillidis 57 3 0 28 Oct 2022
Why Random Pruning Is All We Need to Start Sparse Advait Gadhikar Sohom Mukherjee R. Burkholz 77 20 0 05 Oct 2022
Sparse Random Networks for Communication-Efficient Federated Learning Berivan Isik Francesco Pase Deniz Gunduz Tsachy Weissman M. Zorzi FedML 100 53 0 30 Sep 2022
On the Multidimensional Random Subset Sum Problem L. Becchetti Arthur Carvalho Walraven da Cuhna A. Clementi Francesco d’Amore Hicham Lesfari Emanuele Natale Luca Trevisan 60 3 0 28 Jul 2022
The Lottery Ticket Hypothesis for Self-attention in Convolutional Neural Network Zhongzhan Huang Senwei Liang Mingfu Liang Wei He Haizhao Yang Liang Lin 65 9 0 16 Jul 2022
A General Framework For Proving The Equivariant Strong Lottery Ticket Hypothesis Damien Ferbach Christos Tsirigotis Gauthier Gidel Avishek A. Bose 65 16 0 09 Jun 2022
Meta-ticket: Finding optimal subnetworks for few-shot learning within randomly initialized neural networks Daiki Chijiwa Shin'ya Yamaguchi Atsutoshi Kumagai Yasutoshi Ida 77 9 0 31 May 2022
Dimensionality Reduced Training by Pruning and Freezing Parts of a Deep Neural Network, a Survey Paul Wimmer Jens Mehnert Alexandru Paul Condurache DD 64 21 0 17 May 2022
Convolutional and Residual Networks Provably Contain Lottery Tickets R. Burkholz UQCV MLT 70 13 0 04 May 2022
Most Activation Functions Can Win the Lottery Without Excessive Depth R. Burkholz MLT 104 18 0 04 May 2022
Monarch: Expressive Structured Matrices for Efficient and Accurate Training Tri Dao Beidi Chen N. Sohoni Arjun D Desai Michael Poli Jessica Grogan Alexander Liu Aniruddh Rao Atri Rudra Christopher Ré 95 93 0 01 Apr 2022
On the Neural Tangent Kernel Analysis of Randomly Pruned Neural Networks Hongru Yang Zhangyang Wang MLT 86 8 0 27 Mar 2022
The Combinatorial Brain Surgeon: Pruning Weights That Cancel One Another in Neural Networks Xin Yu Thiago Serra Srikumar Ramalingam Shandian Zhe 67 48 0 09 Mar 2022
Provable and Efficient Continual Representation Learning Yingcong Li Mingchen Li M. Salman Asif Samet Oymak CLL 43 12 0 03 Mar 2022
Extracting Effective Subnetworks with Gumbel-Softmax Robin Dupont M. Alaoui H. Sahbi A. Lebois 39 6 0 25 Feb 2022
Rare Gems: Finding Lottery Tickets at Initialization Kartik K. Sreenivasan Jy-yong Sohn Liu Yang Matthew Grinde Alliot Nagle Hongyi Wang Eric P. Xing Kangwook Lee Dimitris Papailiopoulos 48 42 0 24 Feb 2022
OptG: Optimizing Gradient-driven Criteria in Network Sparsity Yuxin Zhang Mingbao Lin Mengzhao Chen Chia-Wen Lin Rongrong Ji 66 5 0 30 Jan 2022
SHRIMP: Sparser Random Feature Models via Iterative Magnitude Pruning Yuege Xie Bobby Shi Hayden Schaeffer Rachel A. Ward 114 9 0 07 Dec 2021