Sparse Training via Boosting Pruning Plasticity with Neuroregeneration

19 June 2021

Lu Yin

Papers citing "Sparse Training via Boosting Pruning Plasticity with Neuroregeneration"

50 / 78 papers shown

Title
Efficient Shapley Value-based Non-Uniform Pruning of Large Language Models Chuan Sun Han Yu Lizhen Cui Xiaoxiao Li 96 0 0 03 May 2025
Adapting In-Domain Few-Shot Segmentation to New Domains without Retraining Qi Fan Kaiqi Liu Nian Liu Hisham Cholakkal Rao Muhammad Anwer Wenbin Li Yang Gao 77 0 0 30 Apr 2025
Sparse-to-Sparse Training of Diffusion Models Inês Cardoso Oliveira Decebal Constantin Mocanu Luis A. Leiva DiffM 86 0 0 30 Apr 2025
Hyperflows: Pruning Reveals the Importance of Weights Eugen Barbulescu Antonio Alexoaie 31 0 0 06 Apr 2025
E2ENet: Dynamic Sparse Feature Fusion for Accurate and Efficient 3D Medical Image Segmentation Boqian Wu Q. Xiao Shiwei Liu Lu Yin Mykola Pechenizkiy D. Mocanu M. V. Keulen Elena Mocanu MedIm 53 4 0 20 Feb 2025
Signal Collapse in One-Shot Pruning: When Sparse Models Fail to Distinguish Neural Representations Dhananjay Saikumar Blesson Varghese 41 0 0 18 Feb 2025
Advancing Weight and Channel Sparsification with Enhanced Saliency Xinglong Sun Maying Shen Hongxu Yin Lei Mao Pavlo Molchanov Jose M. Alvarez 54 1 0 05 Feb 2025
Brain-inspired sparse training enables Transformers and LLMs to perform as fully connected Yingtao Zhang Jialin Zhao Wenjing Wu Ziheng Liao Umberto Michieli C. Cannistraci 51 0 0 31 Jan 2025
Pushing the Limits of Sparsity: A Bag of Tricks for Extreme Pruning Andy Li A. Durrant Milan Markovic Lu Yin Georgios Leontidis Tianlong Chen Lu Yin Georgios Leontidis 75 0 0 20 Nov 2024
Dynamic Sparse Training versus Dense Training: The Unexpected Winner in Image Corruption Robustness Boqian Wu Q. Xiao Shunxin Wang N. Strisciuglio Mykola Pechenizkiy M. V. Keulen D. Mocanu Elena Mocanu OOD 3DH 54 0 0 03 Oct 2024
$Mixed Sparsity Training: Achieving 4$\times$ FLOP Reduction for Transformer Pretraining$ Mixed Sparsity Training: Achieving 4 $\times$ FLOP Reduction for Transformer Pretraining Pihe Hu Shaolong Li Longbo Huang 33 0 0 21 Aug 2024
Torch2Chip: An End-to-end Customizable Deep Neural Network Compression and Deployment Toolkit for Prototype Hardware Accelerator Design Jian Meng Yuan Liao Anupreetham Anupreetham Ahmed Hassan Shixing Yu Han-Sok Suh Xiaofeng Hu Jae-sun Seo MQ 49 1 0 02 May 2024
Separate, Dynamic and Differentiable (SMART) Pruner for Block/Output Channel Pruning on Computer Vision Tasks Guanhua Ding Zexi Ye Zhen Zhong Gang Li David Shao 44 0 0 29 Mar 2024
MediSwift: Efficient Sparse Pre-trained Biomedical Language Models Vithursan Thangarasa Mahmoud Salem Shreyas Saxena Kevin Leong Joel Hestness Sean Lie MedIm 32 1 0 01 Mar 2024
SequentialAttention++ for Block Sparsification: Differentiable Pruning Meets Combinatorial Optimization T. Yasuda Kyriakos Axiotis Gang Fu M. Bateni Vahab Mirrokni 44 0 0 27 Feb 2024
Always-Sparse Training by Growing Connections with Guided Stochastic Exploration Mike Heddes Narayan Srinivasa T. Givargis Alexandru Nicolau 91 0 0 12 Jan 2024
Exploring Sparsity in Graph Transformers Chuang Liu Yibing Zhan Xueqi Ma Liang Ding Dapeng Tao Jia Wu Wenbin Hu Bo Du 34 6 0 09 Dec 2023
REST: Enhancing Group Robustness in DNNs through Reweighted Sparse Training Jiaxu Zhao Lu Yin Shiwei Liu Meng Fang Mykola Pechenizkiy 28 2 0 05 Dec 2023
Towards Higher Ranks via Adversarial Weight Pruning Yuchuan Tian Hanting Chen Tianyu Guo Chao Xu Yunhe Wang 29 2 0 29 Nov 2023
Outlier Weighed Layerwise Sparsity (OWL): A Missing Secret Sauce for Pruning LLMs to High Sparsity Lu Yin You Wu Zhenyu (Allen) Zhang Cheng-Yu Hsieh Yaqing Wang ... Mykola Pechenizkiy Yi Liang Michael Bendersky Zhangyang Wang Shiwei Liu 28 78 0 08 Oct 2023
Compressing LLMs: The Truth is Rarely Pure and Never Simple Ajay Jaiswal Zhe Gan Xianzhi Du Bowen Zhang Zhangyang Wang Yinfei Yang MQ 41 45 0 02 Oct 2023
Deep Model Fusion: A Survey Weishi Li Yong Peng Miao Zhang Liang Ding Han Hu Li Shen FedML MoMe 33 52 0 27 Sep 2023
Cluster-based pruning techniques for audio data Boris Bergsma Marta Brzezinska O. Yazyev Milos Cernak 19 2 0 21 Sep 2023
Uncovering the Hidden Cost of Model Compression Diganta Misra Muawiz Chaudhary Agam Goyal Bharat Runwal Pin-Yu Chen VLM 36 0 0 29 Aug 2023
Accurate Neural Network Pruning Requires Rethinking Sparse Optimization Denis Kuznedelev Eldar Kurtic Eugenia Iofinova Elias Frantar Alexandra Peste Dan Alistarh VLM 35 11 0 03 Aug 2023
Systematic Investigation of Sparse Perturbed Sharpness-Aware Minimization Optimizer Peng Mi Li Shen Tianhe Ren Yiyi Zhou Tianshuo Xu Xiaoshuai Sun Tongliang Liu Rongrong Ji Dacheng Tao AAML 35 2 0 30 Jun 2023
Spatial Re-parameterization for N:M Sparsity Yuxin Zhang Mingbao Lin Mingliang Xu Yonghong Tian Rongrong Ji 44 2 0 09 Jun 2023
Magnitude Attention-based Dynamic Pruning Jihye Back Namhyuk Ahn Jang-Hyun Kim 28 2 0 08 Jun 2023
The Emergence of Essential Sparsity in Large Pre-trained Models: The Weights that Matter Ajay Jaiswal Shiwei Liu Tianlong Chen Zhangyang Wang VLM 21 33 0 06 Jun 2023
Dynamic Sparsity Is Channel-Level Sparsity Learner Lu Yin Gen Li Meng Fang Lijuan Shen Tianjin Huang Zhangyang Wang Vlado Menkovski Xiaolong Ma Mykola Pechenizkiy Shiwei Liu 30 20 0 30 May 2023
A Three-regime Model of Network Pruning Yefan Zhou Yaoqing Yang Arin Chang Michael W. Mahoney 31 10 0 28 May 2023
Adaptive Sparsity Level during Training for Efficient Time Series Forecasting with Transformers Zahra Atashgahi Mykola Pechenizkiy Raymond N. J. Veldhuis D. Mocanu AI4TS AI4CE 29 1 0 28 May 2023
PDP: Parameter-free Differentiable Pruning is All You Need Minsik Cho Saurabh N. Adya Devang Naik VLM 17 10 0 18 May 2023
Dynamic Sparse Training with Structured Sparsity Mike Lasby A. Golubeva Utku Evci Mihai Nica Yani Andrew Ioannou 29 19 0 03 May 2023
Neurogenesis Dynamics-inspired Spiking Neural Network Training Acceleration Shaoyi Huang Haowen Fang Kaleel Mahmood Bowen Lei Nuo Xu Bin Lei Yue Sun Dongkuan Xu Wujie Wen Caiwen Ding 22 10 0 24 Apr 2023
NTK-SAP: Improving neural network pruning by aligning training dynamics Yite Wang Dawei Li Ruoyu Sun 39 19 0 06 Apr 2023
Self-building Neural Networks Andrea Ferigo Giovanni Iacca 10 2 0 03 Apr 2023
Sparse-IFT: Sparse Iso-FLOP Transformations for Maximizing Training Efficiency Vithursan Thangarasa Shreyas Saxena Abhay Gupta Sean Lie 31 3 0 21 Mar 2023
SPDF: Sparse Pre-training and Dense Fine-tuning for Large Language Models Vithursan Thangarasa Abhay Gupta William Marshall Tianda Li Kevin Leong D. DeCoste Sean Lie Shreyas Saxena MoE AI4CE 18 18 0 18 Mar 2023
Supervised Feature Selection with Neuron Evolution in Sparse Neural Networks Zahra Atashgahi Xuhao Zhang Neil Kichler Shiwei Liu Lu Yin Mykola Pechenizkiy Raymond N. J. Veldhuis D. Mocanu 18 10 0 10 Mar 2023
Sparsity May Cry: Let Us Fail (Current) Sparse Neural Networks Together! Shiwei Liu Tianlong Chen Zhenyu (Allen) Zhang Xuxi Chen Tianjin Huang Ajay Jaiswal Zhangyang Wang 32 29 0 03 Mar 2023
Average of Pruning: Improving Performance and Stability of Out-of-Distribution Detection Zhen Cheng Fei Zhu Xu-Yao Zhang Cheng-Lin Liu MoMe OODD 40 11 0 02 Mar 2023
Balanced Training for Sparse GANs Yite Wang Jing Wu N. Hovakimyan Ruoyu Sun 46 9 0 28 Feb 2023
A Unified Framework for Soft Threshold Pruning Yanqing Chen Zhengyu Ma Wei Fang Xiawu Zheng Zhaofei Yu Yonghong Tian 80 19 0 25 Feb 2023
Ten Lessons We Have Learned in the New "Sparseland": A Short Handbook for Sparse Neural Network Researchers Shiwei Liu Zhangyang Wang 32 30 0 06 Feb 2023
Getting Away with More Network Pruning: From Sparsity to Geometry and Linear Regions Junyang Cai Khai-Nguyen Nguyen Nishant Shrestha Aidan Good Ruisen Tu Xin Yu Shandian Zhe Thiago Serra MLT 40 7 0 19 Jan 2023
Pruning Before Training May Improve Generalization, Provably Hongru Yang Yingbin Liang Xiaojie Guo Lingfei Wu Zhangyang Wang MLT 21 1 0 01 Jan 2023
Dynamic Sparse Network for Time Series Classification: Learning What to "see'' Qiao Xiao Boqian Wu Yu Zhang Shiwei Liu Mykola Pechenizkiy Elena Mocanu D. Mocanu AI4TS 38 28 0 19 Dec 2022
Optimizing Learning Rate Schedules for Iterative Pruning of Deep Neural Networks Shiyu Liu Rohan Ghosh John Tan Chong Min Mehul Motani 37 0 0 09 Dec 2022
Are Straight-Through gradients and Soft-Thresholding all you need for Sparse Training? A. Vanderschueren Christophe De Vleeschouwer MQ 25 9 0 02 Dec 2022