Parameter Efficient Training of Deep Convolutional Neural Networks by Dynamic Sparse Reparameterization

15 February 2019

Papers citing "Parameter Efficient Training of Deep Convolutional Neural Networks by Dynamic Sparse Reparameterization"

50 / 75 papers shown

Title
Sparse-to-Sparse Training of Diffusion Models Inês Cardoso Oliveira Decebal Constantin Mocanu Luis A. Leiva DiffM 86 0 0 30 Apr 2025
GFT: Gradient Focal Transformer Boris Kriuk Simranjit Kaur Gill Shoaib Aslam Amir Fakhrutdinov 31 0 0 14 Apr 2025
Sculpting Memory: Multi-Concept Forgetting in Diffusion Models via Dynamic Mask and Concept-Aware Optimization Gen Li Yang Xiao Jie Ji Kaiyuan Deng Bo Hui Linke Guo Xiaolong Ma 24 0 0 12 Apr 2025
E2ENet: Dynamic Sparse Feature Fusion for Accurate and Efficient 3D Medical Image Segmentation Boqian Wu Q. Xiao Shiwei Liu Lu Yin Mykola Pechenizkiy D. Mocanu M. V. Keulen Elena Mocanu MedIm 53 4 0 20 Feb 2025
Advancing Weight and Channel Sparsification with Enhanced Saliency Xinglong Sun Maying Shen Hongxu Yin Lei Mao Pavlo Molchanov Jose M. Alvarez 54 1 0 05 Feb 2025
Symmetric Pruning of Large Language Models Kai Yi Peter Richtárik AAML VLM 62 0 0 31 Jan 2025
Pushing the Limits of Sparsity: A Bag of Tricks for Extreme Pruning Andy Li A. Durrant Milan Markovic Lu Yin Georgios Leontidis Tianlong Chen Lu Yin Georgios Leontidis 75 0 0 20 Nov 2024
More Experts Than Galaxies: Conditionally-overlapping Experts With Biologically-Inspired Fixed Routing Sagi Shaier Francisco Pereira K. Wense Lawrence E Hunter Matt Jones MoE 46 0 0 10 Oct 2024
Dynamic Sparse Training versus Dense Training: The Unexpected Winner in Image Corruption Robustness Boqian Wu Q. Xiao Shunxin Wang N. Strisciuglio Mykola Pechenizkiy M. V. Keulen D. Mocanu Elena Mocanu OOD 3DH 54 0 0 03 Oct 2024
AdapMTL: Adaptive Pruning Framework for Multitask Learning Model Mingcan Xiang Steven Jiaxun Tang Qizheng Yang Hui Guan Tongping Liu VLM 39 0 0 07 Aug 2024
Scorch: A Library for Sparse Deep Learning Bobby Yan Alexander J. Root Trevor Gale David Broman Fredrik Kjolstad 33 0 0 27 May 2024
Fast and Controllable Post-training Sparsity: Learning Optimal Sparsity Allocation with Global Constraint in Minutes Ruihao Gong Yang Yong Zining Wang Jinyang Guo Xiuying Wei Yuqing Ma Xianglong Liu 38 5 0 09 May 2024
Embracing Unknown Step by Step: Towards Reliable Sparse Training in Real World Bowen Lei Dongkuan Xu Ruqi Zhang Bani Mallick UQCV 39 0 0 29 Mar 2024
Always-Sparse Training by Growing Connections with Guided Stochastic Exploration Mike Heddes Narayan Srinivasa T. Givargis Alexandru Nicolau 91 0 0 12 Jan 2024
Dynamic Sparse No Training: Training-Free Fine-tuning for Sparse LLMs Yuxin Zhang Lirui Zhao Mingbao Lin Yunyun Sun Yiwu Yao Xingjia Han Jared Tanner Shiwei Liu Rongrong Ji SyDa 42 40 0 13 Oct 2023
Magnitude Attention-based Dynamic Pruning Jihye Back Namhyuk Ahn Jang-Hyun Kim 28 2 0 08 Jun 2023
NTK-SAP: Improving neural network pruning by aligning training dynamics Yite Wang Dawei Li Ruoyu Sun 39 19 0 06 Apr 2023
Scaling Expert Language Models with Unsupervised Domain Discovery Suchin Gururangan Margaret Li M. Lewis Weijia Shi Tim Althoff Noah A. Smith Luke Zettlemoyer MoE 25 46 0 24 Mar 2023
Sparsity May Cry: Let Us Fail (Current) Sparse Neural Networks Together! Shiwei Liu Tianlong Chen Zhenyu (Allen) Zhang Xuxi Chen Tianjin Huang Ajay Jaiswal Zhangyang Wang 32 29 0 03 Mar 2023
Balanced Training for Sparse GANs Yite Wang Jing Wu N. Hovakimyan Ruoyu Sun 46 9 0 28 Feb 2023
Bi-directional Masks for Efficient N:M Sparse Training Yuxin Zhang Yiting Luo Mingbao Lin Mingliang Xu Jingjing Xie Rongrong Ji Rongrong Ji 49 15 0 13 Feb 2023
Balance is Essence: Accelerating Sparse Training via Adaptive Gradient Correction Bowen Lei Dongkuan Xu Ruqi Zhang Shuren He Bani Mallick 35 6 0 09 Jan 2023
Pruning On-the-Fly: A Recoverable Pruning Method without Fine-tuning Danyang Liu Xue Liu 25 0 0 24 Dec 2022
Navigating causal deep learning Jeroen Berrevoets Krzysztof Kacprzyk Zhaozhi Qian M. Schaar CML 41 2 0 01 Dec 2022
Dynamic Sparse Training via Balancing the Exploration-Exploitation Trade-off Shaoyi Huang Bowen Lei Dongkuan Xu Hongwu Peng Yue Sun Mimi Xie Caiwen Ding 26 19 0 30 Nov 2022
Exploiting the Partly Scratch-off Lottery Ticket for Quantization-Aware Training Mingliang Xu Gongrui Nan Yuxin Zhang Rongrong Ji Rongrong Ji MQ 18 3 0 12 Nov 2022
LOFT: Finding Lottery Tickets through Filter-wise Training Qihan Wang Chen Dun Fangshuo Liao C. Jermaine Anastasios Kyrillidis 23 3 0 28 Oct 2022
Gradient-based Weight Density Balancing for Robust Dynamic Sparse Training Mathias Parger Alexander Ertl Paul Eibensteiner J. H. Mueller Martin Winter M. Steinberger 34 0 0 25 Oct 2022
On the optimization and pruning for Bayesian deep learning X. Ke Yanan Fan BDL UQCV 35 1 0 24 Oct 2022
Parameter-Efficient Masking Networks Yue Bai Huan Wang Xu Ma Yitian Zhang Zhiqiang Tao Yun Fu 23 10 0 13 Oct 2022
Make Sharpness-Aware Minimization Stronger: A Sparsified Perturbation Approach Peng Mi Li Shen Tianhe Ren Yiyi Zhou Xiaoshuai Sun Rongrong Ji Dacheng Tao AAML 27 69 0 11 Oct 2022
Efficient Methods for Natural Language Processing: A Survey Marcos Vinícius Treviso Ji-Ung Lee Tianchu Ji Betty van Aken Qingqing Cao ... Emma Strubell Niranjan Balasubramanian Leon Derczynski Iryna Gurevych Roy Schwartz 30 109 0 31 Aug 2022
Comprehensive Graph Gradual Pruning for Sparse Training in Graph Neural Networks Chuang Liu Xueqi Ma Yinbing Zhan Liang Ding Dapeng Tao Bo Du Wenbin Hu Danilo Mandic 39 28 0 18 Jul 2022
Fine-tuning Language Models over Slow Networks using Activation Compression with Guarantees Jue Wang Binhang Yuan Luka Rimanic Yongjun He Tri Dao Beidi Chen Christopher Ré Ce Zhang AI4CE 24 11 0 02 Jun 2022
Spartan: Differentiable Sparsity via Regularized Transportation Kai Sheng Tai Taipeng Tian Ser-Nam Lim 31 11 0 27 May 2022
Don't Be So Dense: Sparse-to-Sparse GAN Training Without Sacrificing Performance Shiwei Liu Yuesong Tian Tianlong Chen Li Shen 36 8 0 05 Mar 2022
Sparsity Winning Twice: Better Robust Generalization from More Efficient Training Tianlong Chen Zhenyu (Allen) Zhang Pengju Wang Santosh Balachandra Haoyu Ma Zehao Wang Zhangyang Wang OOD AAML 85 46 0 20 Feb 2022
Bit-wise Training of Neural Network Weights Cristian Ivan MQ 18 0 0 19 Feb 2022
On the Convergence of Heterogeneous Federated Learning with Arbitrary Adaptive Online Model Pruning Hanhan Zhou Tian-Shing Lan Guru Venkataramani Wenbo Ding FedML 32 6 0 27 Jan 2022
Achieving Personalized Federated Learning with Sparse Local Models Tiansheng Huang Shiwei Liu Li Shen Fengxiang He Weiwei Lin Dacheng Tao FedML 30 43 0 27 Jan 2022
Mesa: A Memory-saving Training Framework for Transformers Zizheng Pan Peng Chen Haoyu He Jing Liu Jianfei Cai Bohan Zhuang 31 20 0 22 Nov 2021
Efficient Neural Network Training via Forward and Backward Propagation Sparsification Xiao Zhou Weizhong Zhang Zonghao Chen Shizhe Diao Tong Zhang 34 46 0 10 Nov 2021
BitTrain: Sparse Bitmap Compression for Memory-Efficient Training on the Edge Abdelrahman I. Hosny Marina Neseem Sherief Reda MQ 35 4 0 29 Oct 2021
MEST: Accurate and Fast Memory-Economic Sparse Training Framework on the Edge Geng Yuan Xiaolong Ma Wei Niu Zhengang Li Zhenglun Kong ... Minghai Qin Bin Ren Yanzhi Wang Sijia Liu Xue Lin 20 89 0 26 Oct 2021
CHIP: CHannel Independence-based Pruning for Compact Neural Networks Yang Sui Miao Yin Yi Xie Huy Phan S. Zonouz Bo Yuan VLM 33 128 0 26 Oct 2021
Powerpropagation: A sparsity inducing weight reparameterisation Jonathan Richard Schwarz Siddhant M. Jayakumar Razvan Pascanu P. Latham Yee Whye Teh 90 54 0 01 Oct 2021
Architecture Aware Latency Constrained Sparse Neural Networks Tianli Zhao Qinghao Hu Xiangyu He Weixiang Xu Jiaxing Wang Cong Leng Jian Cheng 33 0 0 01 Sep 2021
Towards Low-Latency Energy-Efficient Deep SNNs via Attention-Guided Compression Souvik Kundu Gourav Datta Massoud Pedram P. Beerel 15 14 0 16 Jul 2021
Deep Ensembling with No Overhead for either Training or Testing: The All-Round Blessings of Dynamic Sparsity Shiwei Liu Tianlong Chen Zahra Atashgahi Xiaohan Chen Ghada Sokar Elena Mocanu Mykola Pechenizkiy Zhangyang Wang D. Mocanu OOD 31 49 0 28 Jun 2021
Sparse Training via Boosting Pruning Plasticity with Neuroregeneration Shiwei Liu Tianlong Chen Xiaohan Chen Zahra Atashgahi Lu Yin Huanyu Kou Li Shen Mykola Pechenizkiy Zhangyang Wang D. Mocanu 37 111 0 19 Jun 2021