The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks

9 March 2018

Papers citing "The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks"

50 / 729 papers shown

Title
Leveraging Active Subspaces to Capture Epistemic Model Uncertainty in Deep Generative Models for Molecular Design A. N. M. N. Abeer Sanket R. Jantre Nathan M. Urban Byung-Jun Yoon 55 1 0 30 Apr 2024
Hidden Synergy: $L_1$ Weight Normalization and 1-Path-Norm Regularization Aditya Biswas 41 0 0 29 Apr 2024
Rapid Deployment of DNNs for Edge Computing via Structured Pruning at Initialization Bailey J. Eccles Leon Wong Blesson Varghese 41 2 0 22 Apr 2024
FedSelect: Personalized Federated Learning with Customized Selection of Parameters for Fine-Tuning Rishub Tamirisa Chulin Xie Wenxuan Bao Andy Zhou Ron Arel Aviv Shamsian 33 6 0 03 Apr 2024
The Unreasonable Ineffectiveness of the Deeper Layers Andrey Gromov Kushal Tirumala Hassan Shapourian Paolo Glorioso Daniel A. Roberts 54 83 0 26 Mar 2024
Searching Search Spaces: Meta-evolving a Geometric Encoding for Neural Networks Tarek Kunze Paul Templier Dennis G. Wilson 35 0 0 20 Mar 2024
Graph Expansion in Pruned Recurrent Neural Network Layers Preserve Performance Suryam Arnav Kalra Arindam Biswas Pabitra Mitra Biswajit Basu GNN 46 0 0 17 Mar 2024
Merging Text Transformer Models from Different Initializations Neha Verma Maha Elbayad MoMe 67 7 0 01 Mar 2024
NeuroPrune: A Neuro-inspired Topological Sparse Training Algorithm for Large Language Models Amit Dhurandhar Tejaswini Pedapati Ronny Luss Soham Dan Aurélie C. Lozano Payel Das Georgios Kollias 22 3 0 28 Feb 2024
SequentialAttention++ for Block Sparsification: Differentiable Pruning Meets Combinatorial Optimization T. Yasuda Kyriakos Axiotis Gang Fu M. Bateni Vahab Mirrokni 47 0 0 27 Feb 2024
Towards Meta-Pruning via Optimal Transport Alexander Theus Olin Geimer Friedrich Wicke Thomas Hofmann Sotiris Anagnostidis Sidak Pal Singh MoMe 24 3 0 12 Feb 2024
Continual Learning on Graphs: A Survey Zonggui Tian Duanhao Zhang Hong-Ning Dai 57 5 0 09 Feb 2024
Everybody Prune Now: Structured Pruning of LLMs with only Forward Passes Lucio Dery Steven Kolawole Jean-Francois Kagey Virginia Smith Graham Neubig Ameet Talwalkar 47 28 0 08 Feb 2024
Analysis of Linear Mode Connectivity via Permutation-Based Weight Matching: With Insights into Other Permutation Search Methods Akira Ito Masanori Yamada Atsutoshi Kumagai MoMe 67 5 0 06 Feb 2024
Faster and Lighter LLMs: A Survey on Current Challenges and Way Forward Arnav Chavan Raghav Magazine Shubham Kushwaha M. Debbah Deepak Gupta 23 18 0 02 Feb 2024
Manipulating Sparse Double Descent Ya Shi Zhang 24 0 0 19 Jan 2024
Stochastic Subnetwork Annealing: A Regularization Technique for Fine Tuning Pruned Subnetworks Tim Whitaker Darrell Whitley 35 0 0 16 Jan 2024
Convolutional Neural Network Compression via Dynamic Parameter Rank Pruning Manish Sharma Jamison Heard Eli Saber Panos P. Markopoulos 31 1 0 15 Jan 2024
Always-Sparse Training by Growing Connections with Guided Stochastic Exploration Mike Heddes Narayan Srinivasa T. Givargis Alexandru Nicolau 91 0 0 12 Jan 2024
PERP: Rethinking the Prune-Retrain Paradigm in the Era of LLMs Max Zimmer Megi Andoni Christoph Spiegel Sebastian Pokutta VLM 55 10 0 23 Dec 2023
The Truth is in There: Improving Reasoning in Language Models with Layer-Selective Rank Reduction Pratyusha Sharma Jordan T. Ash Dipendra Kumar Misra LRM 19 79 0 21 Dec 2023
DSFormer: Effective Compression of Text-Transformers by Dense-Sparse Weight Factorization Rahul Chand Yashoteja Prabhu Pratyush Kumar 20 3 0 20 Dec 2023
SCoTTi: Save Computation at Training Time with an adaptive framework Ziyu Li Enzo Tartaglione Van-Tam Nguyen 42 0 0 19 Dec 2023
Gradient-based Parameter Selection for Efficient Fine-Tuning Zhi Zhang Qizhe Zhang Zijun Gao Renrui Zhang Ekaterina Shutova Shiji Zhou Shanghang Zhang 33 15 0 15 Dec 2023
Cooperative Learning for Cost-Adaptive Inference Xingli Fang Richard M. Bradford Jung-Eun Kim 45 1 0 13 Dec 2023
MaxQ: Multi-Axis Query for N:M Sparsity Network Jingyang Xiang Siqi Li Junhao Chen Zhuangzhi Chen Tianxin Huang Linpeng Peng Yong-Jin Liu 18 0 0 12 Dec 2023
Deeper Understanding of Black-box Predictions via Generalized Influence Functions Hyeonsu Lyu Jonggyu Jang Sehyun Ryu H. Yang TDI AI4CE 27 5 0 09 Dec 2023
Towards On-device Learning on the Edge: Ways to Select Neurons to Update under a Budget Constraint Ael Quélennec Enzo Tartaglione Pavlo Mozharovskyi Van-Tam Nguyen 36 2 0 08 Dec 2023
Accelerating Convolutional Neural Network Pruning via Spatial Aura Entropy Bogdan Musat Razvan Andonie 26 0 0 08 Dec 2023
Pursing the Sparse Limitation of Spiking Deep Learning Structures Hao-Ran Cheng Jiahang Cao Erjia Xiao Mengshu Sun Le Yang Jize Zhang Xue Lin B. Kailkhura Kaidi Xu Renjing Xu 18 1 0 18 Nov 2023
Polynomially Over-Parameterized Convolutional Neural Networks Contain Structured Strong Winning Lottery Tickets A. D. Cunha Francesco d’Amore Emanuele Natale MLT 27 1 0 16 Nov 2023
Language and Task Arithmetic with Parameter-Efficient Layers for Zero-Shot Summarization Alexandra Chronopoulou Jonas Pfeiffer Joshua Maynez Xinyi Wang Sebastian Ruder Priyanka Agrawal MoMe 26 16 0 15 Nov 2023
One is More: Diverse Perspectives within a Single Network for Efficient DRL Yiqin Tan Ling Pan Longbo Huang OffRL 40 0 0 21 Oct 2023
How a student becomes a teacher: learning and forgetting through Spectral methods Lorenzo Giambagli L. Buffoni Lorenzo Chicchi Duccio Fanelli 19 7 0 19 Oct 2023
Dynamic Sparse No Training: Training-Free Fine-tuning for Sparse LLMs Yuxin Zhang Lirui Zhao Mingbao Lin Yunyun Sun Yiwu Yao Xingjia Han Jared Tanner Shiwei Liu Rongrong Ji SyDa 45 40 0 13 Oct 2023
A Mass-Conserving-Perceptron for Machine Learning-Based Modeling of Geoscientific Systems Yuan-Heng Wang Hoshin V. Gupta AI4CE 40 6 0 12 Oct 2023
Outlier Weighed Layerwise Sparsity (OWL): A Missing Secret Sauce for Pruning LLMs to High Sparsity Lu Yin You Wu Zhenyu Zhang Cheng-Yu Hsieh Yaqing Wang ... Mykola Pechenizkiy Yi Liang Michael Bendersky Zhangyang Wang Shiwei Liu 36 79 0 08 Oct 2023
Spectral alignment of stochastic gradient descent for high-dimensional classification tasks Gerard Ben Arous Reza Gheissari Jiaoyang Huang Aukosh Jagannath 35 13 0 04 Oct 2023
Elephant Neural Networks: Born to Be a Continual Learner Qingfeng Lan A. Rupam Mahmood CLL 56 9 0 02 Oct 2023
Detach-ROCKET: Sequential feature selection for time series classification with random convolutional kernels Gonzalo Uribarri Federico Barone A. Ansuini Erik Fransén AI4TS 50 6 0 25 Sep 2023
Sparse Autoencoders Find Highly Interpretable Features in Language Models Hoagy Cunningham Aidan Ewart Logan Riggs R. Huben Lee Sharkey MILM 33 347 0 15 Sep 2023
Pareto Frontiers in Neural Feature Learning: Data, Compute, Width, and Luck Benjamin L. Edelman Surbhi Goel Sham Kakade Eran Malach Cyril Zhang 50 8 0 07 Sep 2023
Adaptive Consensus: A network pruning approach for decentralized optimization S. Shah A. Berahas Raghu Bollapragada 27 2 0 06 Sep 2023
T-SaS: Toward Shift-aware Dynamic Adaptation for Streaming Data Weijieying Ren Tianxiang Zhao Wei Qin Kunpeng Liu TTA AI4TS 29 5 0 05 Sep 2023
Uncovering the Hidden Cost of Model Compression Diganta Misra Muawiz Chaudhary Agam Goyal Bharat Runwal Pin-Yu Chen VLM 38 0 0 29 Aug 2023
Maestro: Uncovering Low-Rank Structures via Trainable Decomposition Samuel Horváth Stefanos Laskaridis Shashank Rajput Hongyi Wang BDL 37 4 0 28 Aug 2023
Differentiable Weight Masks for Domain Transfer Samarth Khanna Skanda Vaidyanath Akash Velu 34 0 0 26 Aug 2023
Tryage: Real-time, intelligent Routing of User Prompts to Large Language Models S. N. Hari Matt Thomson 32 11 0 22 Aug 2023
The Snowflake Hypothesis: Training Deep GNN with One Node One Receptive field Kun Wang Guohao Li Shilong Wang Guibin Zhang Kaidi Wang Yang You Xiaojiang Peng Keli Zhang Yang Wang 42 8 0 19 Aug 2023
Neural Networks at a Fraction with Pruned Quaternions Sahel Mohammad Iqbal Subhankar Mishra 28 4 0 13 Aug 2023