The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks

9 March 2018

Papers citing "The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks"

50 / 745 papers shown

Title
Performance-aware Approximation of Global Channel Pruning for Multitask CNNs Hancheng Ye Bo Zhang Tao Chen Jiayuan Fan Bin Wang 37 18 0 21 Mar 2023
Sparse-IFT: Sparse Iso-FLOP Transformations for Maximizing Training Efficiency Vithursan Thangarasa Shreyas Saxena Abhay Gupta Sean Lie 41 3 0 21 Mar 2023
Model Stitching: Looking For Functional Similarity Between Representations Adriano Hernandez Rumen Dangovski Peter Y. Lu Marin Soljacic 29 3 0 20 Mar 2023
Induced Feature Selection by Structured Pruning Nathan Hubens V. Delvigne M. Mancas B. Gosselin Marius Preda T. Zaharia 22 0 0 20 Mar 2023
Automatic Attention Pruning: Improving and Automating Model Pruning using Attentions Kaiqi Zhao Animesh Jain Ming Zhao 26 10 0 14 Mar 2023
Can Adversarial Examples Be Parsed to Reveal Victim Model Information? Yuguang Yao Jiancheng Liu Yifan Gong Xiaoming Liu Yanzhi Wang X. Lin Sijia Liu AAML MLAU 39 1 0 13 Mar 2023
Efficient Transformer-based 3D Object Detection with Dynamic Token Halting Mao Ye Gregory P. Meyer Yuning Chai Qiang Liu 34 9 0 09 Mar 2023
Sparsity May Cry: Let Us Fail (Current) Sparse Neural Networks Together! Shiwei Liu Tianlong Chen Zhenyu Zhang Xuxi Chen Tianjin Huang Ajay Jaiswal Zhangyang Wang 37 29 0 03 Mar 2023
Rotation Invariant Quantization for Model Compression Dor-Joseph Kampeas Yury Nahshan Hanoch Kremer Gil Lederman Shira Zaloshinski Zheng Li E. Haleva MQ 23 1 0 03 Mar 2023
DSD $^2$ : Can We Dodge Sparse Double Descent and Compress the Neural Network Worry-Free? Victor Quétu Enzo Tartaglione 37 7 0 02 Mar 2023
Average of Pruning: Improving Performance and Stability of Out-of-Distribution Detection Zhen Cheng Fei Zhu Xu-Yao Zhang Cheng-Lin Liu MoMe OODD 45 11 0 02 Mar 2023
Parameter Sharing with Network Pruning for Scalable Multi-Agent Deep Reinforcement Learning Woojun Kim Young-Jin Sung 32 7 0 02 Mar 2023
Balanced Training for Sparse GANs Yite Wang Jing Wu N. Hovakimyan Ruoyu Sun 48 9 0 28 Feb 2023
Permutation Equivariant Neural Functionals Allan Zhou Kaien Yang Kaylee Burns Adriano Cardace Yiding Jiang Samuel Sokota J. Zico Kolter Chelsea Finn 35 47 0 27 Feb 2023
Full Stack Optimization of Transformer Inference: a Survey Sehoon Kim Coleman Hooper Thanakul Wattanawong Minwoo Kang Ruohan Yan ... Qijing Huang Kurt Keutzer Michael W. Mahoney Y. Shao A. Gholami MQ 38 102 0 27 Feb 2023
Neural Graph Revealers H. Shrivastava Urszula Chajewska BDL 36 6 0 27 Feb 2023
Can we avoid Double Descent in Deep Neural Networks? Victor Quétu Enzo Tartaglione AI4CE 20 3 0 26 Feb 2023
A Unified Framework for Soft Threshold Pruning Yanqing Chen Zhengyu Ma Wei Fang Xiawu Zheng Zhaofei Yu Yonghong Tian 88 19 0 25 Feb 2023
MUX-PLMs: Data Multiplexing for High-throughput Language Models Vishvak Murahari Ameet Deshpande Carlos E. Jimenez Izhak Shafran Mingqiu Wang Yuan Cao Karthik Narasimhan MoE 31 5 0 24 Feb 2023
Modular Deep Learning Jonas Pfeiffer Sebastian Ruder Ivan Vulić Edoardo Ponti MoMe OOD 34 73 0 22 Feb 2023
Considering Layerwise Importance in the Lottery Ticket Hypothesis Benjamin Vandersmissen José Oramas 37 1 0 22 Feb 2023
Less is More: The Influence of Pruning on the Explainability of CNNs David Weber F. Merkle Pascal Schöttle Stephan Schlögl Martin Nocker FAtt 34 1 0 17 Feb 2023
Learning a Consensus Sub-Network with Polarization Regularization and One Pass Training Xiaoying Zhi Varun Babbar P. Sun Fran Silavong Ruibo Shi Sean J. Moran Sean Moran 60 1 0 17 Feb 2023
DP-BART for Privatized Text Rewriting under Local Differential Privacy Timour Igamberdiev Ivan Habernal 23 17 0 15 Feb 2023
Workload-Balanced Pruning for Sparse Spiking Neural Networks Ruokai Yin Youngeun Kim Yuhang Li Abhishek Moitra Nitin Satpute Anna Hambitzer Priyadarshini Panda 37 19 0 13 Feb 2023
Simple Hardware-Efficient Long Convolutions for Sequence Modeling Daniel Y. Fu Elliot L. Epstein Eric N. D. Nguyen A. Thomas Michael Zhang Tri Dao Atri Rudra Christopher Ré 25 52 0 13 Feb 2023
Bi-directional Masks for Efficient N:M Sparse Training Yuxin Zhang Yiting Luo Mingbao Lin Mingliang Xu Jingjing Xie Rongrong Ji Rongrong Ji 52 15 0 13 Feb 2023
Quantum Neuron Selection: Finding High Performing Subnetworks With Quantum Algorithms Tim Whitaker 33 1 0 12 Feb 2023
Pruning Deep Neural Networks from a Sparsity Perspective Enmao Diao G. Wang Jiawei Zhan Yuhong Yang Jie Ding Vahid Tarokh 27 30 0 11 Feb 2023
Deep Learning on Implicit Neural Representations of Shapes Luca de Luigi Adriano Cardace Riccardo Spezialetti Pierluigi Zama Ramirez Samuele Salti Luigi Di Stefano 31 46 0 10 Feb 2023
What Matters In The Structured Pruning of Generative Language Models? Michael Santacroce Zixin Wen Yelong Shen Yuan-Fang Li 28 33 0 07 Feb 2023
A Toy Model of Universality: Reverse Engineering How Networks Learn Group Operations Bilal Chughtai Lawrence Chan Neel Nanda 21 96 0 06 Feb 2023
Fast, Differentiable and Sparse Top-k: a Convex Analysis Perspective Michael E. Sander J. Puigcerver Josip Djolonga Gabriel Peyré Mathieu Blondel 21 19 0 02 Feb 2023
A Survey on Efficient Training of Transformers Bohan Zhuang Jing Liu Zizheng Pan Haoyu He Yuetian Weng Chunhua Shen 33 47 0 02 Feb 2023
An Empirical Study on the Transferability of Transformer Modules in Parameter-Efficient Fine-Tuning Mohammad AkbarTajari S. Rajaee Mohammad Taher Pilehvar 19 2 0 01 Feb 2023
Towards Inference Efficient Deep Ensemble Learning Ziyue Li Kan Ren Yifan Yang Xinyang Jiang Yuqing Yang Dongsheng Li BDL 31 12 0 29 Jan 2023
Quantum Ridgelet Transform: Winning Lottery Ticket of Neural Networks with Quantum Computation H. Yamasaki Sathyawageeswar Subramanian Satoshi Hayakawa Sho Sonoda MLT 30 4 0 27 Jan 2023
Rewarded meta-pruning: Meta Learning with Rewards for Channel Pruning Athul Shibu Abhishek Kumar Heechul Jung Dong-Gyu Lee 19 1 0 26 Jan 2023
Modality-Agnostic Variational Compression of Implicit Neural Representations Jonathan Richard Schwarz Jihoon Tack Yee Whye Teh Jaeho Lee Jinwoo Shin 34 25 0 23 Jan 2023
Getting Away with More Network Pruning: From Sparsity to Geometry and Linear Regions Junyang Cai Khai-Nguyen Nguyen Nishant Shrestha Aidan Good Ruisen Tu Xin Yu Shandian Zhe Thiago Serra MLT 40 7 0 19 Jan 2023
Adaptive Deep Neural Network Inference Optimization with EENet Fatih Ilhan Ka-Ho Chow Sihao Hu Tiansheng Huang Selim Tekin ... Myungjin Lee Ramana Rao Kompella Hugo Latapie Gan Liu Ling Liu 41 11 0 15 Jan 2023
Empirical study of the modulus as activation function in computer vision applications Iván Vallés-Pérez E. Soria-Olivas M. Martínez-Sober Antonio J. Serrano Joan Vila-Francés J. Gómez-Sanchís 33 15 0 15 Jan 2023
Adaptive Neural Networks Using Residual Fitting N. Ford J. Winder Josh Mcclellan 27 0 0 13 Jan 2023
Why is the State of Neural Network Pruning so Confusing? On the Fairness, Comparison Setup, and Trainability in Network Pruning Huan Wang Can Qin Yue Bai Yun Fu 37 20 0 12 Jan 2023
Progress measures for grokking via mechanistic interpretability Neel Nanda Lawrence Chan Tom Lieberum Jess Smith Jacob Steinhardt 49 386 0 12 Jan 2023
Toward Building General Foundation Models for Language, Vision, and Vision-Language Understanding Tasks Xinsong Zhang Yan Zeng Jipeng Zhang Hang Li VLM AI4CE LRM 32 17 0 12 Jan 2023
Pruning Compact ConvNets for Efficient Inference Sayan Ghosh Karthik Prasad Xiaoliang Dai Peizhao Zhang Bichen Wu Graham Cormode Peter Vajda VLM 34 4 0 11 Jan 2023
Balance is Essence: Accelerating Sparse Training via Adaptive Gradient Correction Bowen Lei Dongkuan Xu Ruqi Zhang Shuren He Bani Mallick 42 6 0 09 Jan 2023
A Theory of I/O-Efficient Sparse Neural Network Inference Niels Gleinig Tal Ben-Nun Torsten Hoefler 33 0 0 03 Jan 2023
SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot Elias Frantar Dan Alistarh VLM 35 643 0 02 Jan 2023