Pruning's Effect on Generalization Through the Lens of Training and Regularization

25 October 2022

Gintare Karolina Dziugaite

ArXiv (abs)PDF HTML

Papers citing "Pruning's Effect on Generalization Through the Lens of Training and Regularization"

30 / 30 papers shown

Title
CRoP: Context-wise Robust Static Human-Sensing Personalization Sawinder Kaur Avery Gump Yi Xiao Jingyu Xin Harshit Sharma Nina R Benway Jonathan L Preston Asif Salekin 88 0 0 26 Sep 2024
Mask in the Mirror: Implicit Sparsification Tom Jacobs R. Burkholz 173 4 0 19 Aug 2024
Only Train Once: A One-Shot Neural Network Training And Pruning Framework Tianyi Chen Bo Ji Tianyu Ding Biyi Fang Guanyi Wang Zhihui Zhu Luming Liang Yixin Shi Sheng Yi Xiao Tu 95 105 0 15 Jul 2021
Deep Learning on a Data Diet: Finding Important Examples Early in Training Mansheej Paul Surya Ganguli Gintare Karolina Dziugaite 117 457 0 15 Jul 2021
Lost in Pruning: The Effects of Pruning Neural Networks beyond Test Accuracy Lucas Liebenwein Cenk Baykal Brandon Carter David K Gifford Daniela Rus AAML 55 73 0 04 Mar 2021
Characterising Bias in Compressed Models Sara Hooker Nyalleng Moorosi Gregory Clark Samy Bengio Emily L. Denton 67 185 0 06 Oct 2020
What is the State of Neural Network Pruning? Davis W. Blalock Jose Javier Gonzalez Ortiz Jonathan Frankle John Guttag 267 1,052 0 06 Mar 2020
Comparing Rewinding and Fine-tuning in Neural Network Pruning Alex Renda Jonathan Frankle Michael Carbin 272 387 0 05 Mar 2020
Linear Mode Connectivity and the Lottery Ticket Hypothesis Jonathan Frankle Gintare Karolina Dziugaite Daniel M. Roy Michael Carbin MoMe 156 619 0 11 Dec 2019
Provable Filter Pruning for Efficient Neural Networks Lucas Liebenwein Cenk Baykal Harry Lang Dan Feldman Daniela Rus VLM 3DPC 74 141 0 18 Nov 2019
The Generalization-Stability Tradeoff In Neural Network Pruning Brian Bartoldson Ari S. Morcos Adrian Barbu G. Erlebacher 77 76 0 09 Jun 2019
EigenDamage: Structured Pruning in the Kronecker-Factored Eigenbasis Chaoqi Wang Roger C. Grosse Sanja Fidler Guodong Zhang 65 124 0 15 May 2019
Reconciling modern machine learning practice and the bias-variance trade-off M. Belkin Daniel J. Hsu Siyuan Ma Soumik Mandal 240 1,650 0 28 Dec 2018
An Empirical Study of Example Forgetting during Deep Neural Network Learning Mariya Toneva Alessandro Sordoni Rémi Tachet des Combes Adam Trischler Yoshua Bengio Geoffrey J. Gordon 118 734 0 12 Dec 2018
Rethinking the Value of Network Pruning Zhuang Liu Mingjie Sun Tinghui Zhou Gao Huang Trevor Darrell 36 1,470 0 11 Oct 2018
SNIP: Single-shot Network Pruning based on Connection Sensitivity Namhoon Lee Thalaiyasingam Ajanthan Philip Torr VLM 263 1,205 0 04 Oct 2018
Data-Dependent Coresets for Compressing Neural Networks with Applications to Generalization Bounds Cenk Baykal Lucas Liebenwein Igor Gilitschenski Dan Feldman Daniela Rus 74 79 0 15 Apr 2018
The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks Jonathan Frankle Michael Carbin 240 3,473 0 09 Mar 2018
NISP: Pruning Networks using Neuron Importance Score Propagation Ruichi Yu Ang Li Chun-Fu Chen Jui-Hsin Lai Vlad I. Morariu Xintong Han M. Gao Ching-Yung Lin L. Davis 74 800 0 16 Nov 2017
Learning Efficient Convolutional Networks through Network Slimming Zhuang Liu Jianguo Li Zhiqiang Shen Gao Huang Shoumeng Yan Changshui Zhang 125 2,421 0 22 Aug 2017
Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Data Gintare Karolina Dziugaite Daniel M. Roy 106 815 0 31 Mar 2017
Equality of Opportunity in Supervised Learning Moritz Hardt Eric Price Nathan Srebro FaML 230 4,312 0 07 Oct 2016
Pruning Filters for Efficient ConvNets Hao Li Asim Kadav Igor Durdanovic H. Samet H. Graf 3DPC 193 3,697 0 31 Aug 2016
Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding Song Han Huizi Mao W. Dally 3DGS 261 8,842 0 01 Oct 2015
Learning both Weights and Connections for Efficient Neural Networks Song Han Jeff Pool J. Tran W. Dally CVBM 313 6,681 0 08 Jun 2015
Fast ConvNets Using Group-wise Brain Damage V. Lebedev Victor Lempitsky AAML 178 449 0 08 Jun 2015
Cyclical Learning Rates for Training Neural Networks L. Smith ODL 210 2,529 0 03 Jun 2015
Norm-Based Capacity Control in Neural Networks Behnam Neyshabur Ryota Tomioka Nathan Srebro 292 588 0 27 Feb 2015
In Search of the Real Inductive Bias: On the Role of Implicit Regularization in Deep Learning Behnam Neyshabur Ryota Tomioka Nathan Srebro AI4CE 94 658 0 20 Dec 2014
Going Deeper with Convolutions Christian Szegedy Wei Liu Yangqing Jia P. Sermanet Scott E. Reed Dragomir Anguelov D. Erhan Vincent Vanhoucke Andrew Rabinovich 477 43,658 0 17 Sep 2014