The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks

9 March 2018

Papers citing "The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks"

50 / 745 papers shown

Title
BERT Busters: Outlier Dimensions that Disrupt Transformers Olga Kovaleva Saurabh Kulshreshtha Anna Rogers Anna Rumshisky 27 85 0 14 May 2021
Troubleshooting Blind Image Quality Models in the Wild Zhihua Wang Haotao Wang Tianlong Chen Zhangyang Wang Kede Ma 15 19 0 14 May 2021
Dynamic Multi-Branch Layers for On-Device Neural Machine Translation Zhixing Tan Zeyuan Yang Meng Zhang Qun Liu Maosong Sun Yang Liu AI4CE 24 4 0 14 May 2021
What Kinds of Functions do Deep Neural Networks Learn? Insights from Variational Spline Theory Rahul Parhi Robert D. Nowak MLT 38 70 0 07 May 2021
Structured Ensembles: an Approach to Reduce the Memory Footprint of Ensemble Methods Jary Pomponi Simone Scardapane A. Uncini UQCV 49 7 0 06 May 2021
Initialization and Regularization of Factorized Neural Layers M. Khodak Neil A. Tenenholtz Lester W. Mackey Nicolò Fusi 65 56 0 03 May 2021
Effective Sparsification of Neural Networks with Global Sparsity Constraint Xiao Zhou Weizhong Zhang Hang Xu Tong Zhang 21 61 0 03 May 2021
MineGAN++: Mining Generative Models for Efficient Knowledge Transfer to Limited Data Domains Yaxing Wang Abel Gonzalez-Garcia Chenshen Wu Luis Herranz Fahad Shahbaz Khan Shangling Jui Joost van de Weijer 32 6 0 28 Apr 2021
Sifting out the features by pruning: Are convolutional networks the winning lottery ticket of fully connected ones? Franco Pellegrini Giulio Biroli 54 6 0 27 Apr 2021
Playing Lottery Tickets with Vision and Language Zhe Gan Yen-Chun Chen Linjie Li Tianlong Chen Yu Cheng Shuohang Wang Jingjing Liu Lijuan Wang Zicheng Liu VLM 109 54 0 23 Apr 2021
Analyzing Monotonic Linear Interpolation in Neural Network Loss Landscapes James Lucas Juhan Bae Michael Ruogu Zhang Stanislav Fort R. Zemel Roger C. Grosse MoMe 172 28 0 22 Apr 2021
Distilling Knowledge via Knowledge Review Pengguang Chen Shu Liu Hengshuang Zhao Jiaya Jia 155 424 0 19 Apr 2021
Case-based Reasoning for Natural Language Queries over Knowledge Bases Rajarshi Das Manzil Zaheer Dung Ngoc Thai Ameya Godbole Ethan Perez Jay Yoon Lee Lizhen Tan L. Polymenakos Andrew McCallum 36 163 0 18 Apr 2021
Lottery Jackpots Exist in Pre-trained Models Yuxin Zhang Mingbao Lin Yan Wang Rongrong Ji Rongrong Ji 35 15 0 18 Apr 2021
Rethinking Network Pruning -- under the Pre-train and Fine-tune Paradigm Dongkuan Xu Ian En-Hsu Yen Jinxi Zhao Zhibin Xiao VLM AAML 31 56 0 18 Apr 2021
Generalization bounds via distillation Daniel J. Hsu Ziwei Ji Matus Telgarsky Lan Wang FedML 35 32 0 12 Apr 2021
A hybrid inference system for improved curvature estimation in the level-set method using machine learning Luis Ángel Larios-Cárdenas Frédéric Gibou 21 6 0 07 Apr 2021
Going deeper with Image Transformers Hugo Touvron Matthieu Cord Alexandre Sablayrolles Gabriel Synnaeve Hervé Jégou ViT 27 990 0 31 Mar 2021
The Elastic Lottery Ticket Hypothesis Xiaohan Chen Yu Cheng Shuohang Wang Zhe Gan Jingjing Liu Zhangyang Wang OOD 28 34 0 30 Mar 2021
Active multi-fidelity Bayesian online changepoint detection Gregory W. Gundersen Diana Cai Chuteng Zhou Barbara E. Engelhardt Ryan P. Adams 22 10 0 26 Mar 2021
Multi-Prize Lottery Ticket Hypothesis: Finding Accurate Binary Neural Networks by Pruning A Randomly Weighted Network James Diffenderfer B. Kailkhura MQ 35 75 0 17 Mar 2021
Recent Advances on Neural Network Pruning at Initialization Huan Wang Can Qin Yue Bai Yulun Zhang Yun Fu CVBM 38 64 0 11 Mar 2021
Quantization-Guided Training for Compact TinyML Models Sedigh Ghamari Koray Ozcan Thu Dinh A. Melnikov Juan Carvajal Jan Ernst S. Chai MQ 21 16 0 10 Mar 2021
Proof-of-Learning: Definitions and Practice Hengrui Jia Mohammad Yaghini Christopher A. Choquette-Choo Natalie Dullerud Anvith Thudi Varun Chandrasekaran Nicolas Papernot AAML 25 99 0 09 Mar 2021
Knowledge Evolution in Neural Networks Ahmed Taha Abhinav Shrivastava L. Davis 49 21 0 09 Mar 2021
Teachers Do More Than Teach: Compressing Image-to-Image Models Qing Jin Jian Ren Oliver J. Woodford Jiazhuo Wang Geng Yuan Yanzhi Wang Sergey Tulyakov 39 54 0 05 Mar 2021
Clusterability in Neural Networks Daniel Filan Stephen Casper Shlomi Hod Cody Wild Andrew Critch Stuart J. Russell GNN 32 30 0 04 Mar 2021
Lost in Pruning: The Effects of Pruning Neural Networks beyond Test Accuracy Lucas Liebenwein Cenk Baykal Brandon Carter David K Gifford Daniela Rus AAML 40 71 0 04 Mar 2021
Improving Computational Efficiency in Visual Reinforcement Learning via Stored Embeddings Lili Chen Kimin Lee A. Srinivas Pieter Abbeel OffRL 24 11 0 04 Mar 2021
The Rediscovery Hypothesis: Language Models Need to Meet Linguistics Vassilina Nikoulina Maxat Tezekbayev Nuradil Kozhakhmet Madina Babazhanova Matthias Gallé Z. Assylbekov 34 8 0 02 Mar 2021
Sparse Training Theory for Scalable and Efficient Agents Decebal Constantin Mocanu Elena Mocanu T. Pinto Selima Curci Phuong H. Nguyen M. Gibescu D. Ernst Z. Vale 45 17 0 02 Mar 2021
Early-Bird GCNs: Graph-Network Co-Optimization Towards More Efficient GCN Training and Inference via Drawing Early-Bird Lottery Tickets Haoran You Zhihan Lu Zijian Zhou Y. Fu Yingyan Lin GNN 41 30 0 01 Mar 2021
Consistent Sparse Deep Learning: Theory and Computation Y. Sun Qifan Song F. Liang BDL 48 27 0 25 Feb 2021
Understanding Catastrophic Forgetting and Remembering in Continual Learning with Optimal Relevance Mapping Prakhar Kaushik Alex Gain Adam Kortylewski Alan Yuille CLL 11 69 0 22 Feb 2021
Lottery Ticket Preserves Weight Correlation: Is It Desirable or Not? Ning Liu Geng Yuan Zhengping Che Xuan Shen Xiaolong Ma Qing Jin Jian Ren Jian Tang Sijia Liu Yanzhi Wang 41 31 0 19 Feb 2021
An Information-Theoretic Justification for Model Pruning Berivan Isik Tsachy Weissman Albert No 95 35 0 16 Feb 2021
Accelerated Sparse Neural Training: A Provable and Efficient Method to Find N:M Transposable Masks Itay Hubara Brian Chmiel Moshe Island Ron Banner S. Naor Daniel Soudry 59 111 0 16 Feb 2021
Learning N:M Fine-grained Structured Sparse Neural Networks From Scratch Aojun Zhou Yukun Ma Junnan Zhu Jianbo Liu Zhijie Zhang Kun Yuan Wenxiu Sun Hongsheng Li 69 241 0 08 Feb 2021
SeReNe: Sensitivity based Regularization of Neurons for Structured Sparsity in Neural Networks Enzo Tartaglione Andrea Bragagnolo Francesco Odierna Attilio Fiandrotti Marco Grangetto 43 18 0 07 Feb 2021
A Local Convergence Theory for Mildly Over-Parameterized Two-Layer Neural Network Mo Zhou Rong Ge Chi Jin 79 45 0 04 Feb 2021
Pruning and Quantization for Deep Neural Network Acceleration: A Survey Tailin Liang C. Glossner Lei Wang Shaobo Shi Xiaotong Zhang MQ 150 678 0 24 Jan 2021
Time-Correlated Sparsification for Communication-Efficient Federated Learning Emre Ozfatura Kerem Ozfatura Deniz Gunduz FedML 43 47 0 21 Jan 2021
Zero-Cost Proxies for Lightweight NAS Mohamed S. Abdelfattah Abhinav Mehrotra L. Dudziak Nicholas D. Lane 30 253 0 20 Jan 2021
Learnable Embedding Sizes for Recommender Systems Siyi Liu Chen Gao Yihong Chen Depeng Jin Yong Li 61 83 0 19 Jan 2021
Activation Density based Mixed-Precision Quantization for Energy Efficient Neural Networks Karina Vasquez Yeshwanth Venkatesha Abhiroop Bhattacharjee Abhishek Moitra Priyadarshini Panda MQ 43 15 0 12 Jan 2021
Reservoir Transformers Sheng Shen Alexei Baevski Ari S. Morcos Kurt Keutzer Michael Auli Douwe Kiela 35 17 0 30 Dec 2020
Enabling Retrain-free Deep Neural Network Pruning using Surrogate Lagrangian Relaxation Deniz Gurevin Shangli Zhou Lynn Pepin Bingbing Li Mikhail A. Bragin Caiwen Ding Fei Miao 26 3 0 18 Dec 2020
Provable Benefits of Overparameterization in Model Compression: From Double Descent to Pruning Neural Networks Xiangyu Chang Yingcong Li Samet Oymak Christos Thrampoulidis 35 50 0 16 Dec 2020
NeurIPS 2020 Competition: Predicting Generalization in Deep Learning Yiding Jiang Pierre Foret Scott Yak Daniel M. Roy H. Mobahi Gintare Karolina Dziugaite Samy Bengio Suriya Gunasekar Isabelle M Guyon Behnam Neyshabur Google Research OOD 24 55 0 14 Dec 2020
The Lottery Tickets Hypothesis for Supervised and Self-supervised Pre-training in Computer Vision Models Tianlong Chen Jonathan Frankle Shiyu Chang Sijia Liu Yang Zhang Michael Carbin Zhangyang Wang 27 123 0 12 Dec 2020