Dynamic Sparsity Is Channel-Level Sparsity Learner

Dynamic Sparsity Is Channel-Level Sparsity Learner

30 May 2023

Lu Yin

Vlado Menkovski

Mykola Pechenizkiy

Papers citing "Dynamic Sparsity Is Channel-Level Sparsity Learner"

18 / 18 papers shown

Title
Sculpting Memory: Multi-Concept Forgetting in Diffusion Models via Dynamic Mask and Concept-Aware Optimization Gen Li Yang Xiao Jie Ji Kaiyuan Deng Bo Hui Linke Guo Xiaolong Ma 24 0 0 12 Apr 2025
Keeping Yourself is Important in Downstream Tuning Multimodal Large Language Model Wenke Huang Jian Liang Xianda Guo Yiyang Fang Guancheng Wan ... Bin Yang He Li Jiawei Shao Mang Ye Bo Du OffRL LRM MLLM KELM VLM 65 1 0 06 Mar 2025
Pushing the Limits of Sparsity: A Bag of Tricks for Extreme Pruning Andy Li A. Durrant Milan Markovic Lu Yin Georgios Leontidis Tianlong Chen Lu Yin Georgios Leontidis 75 0 0 20 Nov 2024
Full-Rank No More: Low-Rank Weight Training for Modern Speech Recognition Models Adriana Fernandez-Lopez Shiwei Liu L. Yin Stavros Petridis Maja Pantic 29 0 0 10 Oct 2024
Is C4 Dataset Optimal for Pruning? An Investigation of Calibration Data for LLM Pruning Abhinav Bandari L. Yin Cheng-Yu Hsieh Ajay Kumar Jaiswal Tianlong Chen Li Shen Ranjay Krishna Shiwei Liu 31 6 0 09 Oct 2024
S-STE: Continuous Pruning Function for Efficient 2:4 Sparse Pre-training Yuezhou Hu Jun-Jie Zhu Jianfei Chen 38 0 0 13 Sep 2024
Data Overfitting for On-Device Super-Resolution with Dynamic Algorithm and Compiler Co-Design Gen Li Zhihao Shu Jie Ji Minghai Qin Fatemeh Afghah Wei Niu Xiaolong Ma SupR 38 0 0 03 Jul 2024
MSRS: Training Multimodal Speech Recognition Models from Scratch with Sparse Mask Optimization Adriana Fernandez-Lopez Honglie Chen Pingchuan Ma Lu Yin Q. Xiao Stavros Petridis Shiwei Liu Maja Pantic 46 2 0 25 Jun 2024
Maxwell's Demon at Work: Efficient Pruning by Leveraging Saturation of Neurons Simon Dufort-Labbé P. DÓro Evgenii Nikishin Razvan Pascanu Pierre-Luc Bacon A. Baratin 42 1 0 12 Mar 2024
Visual Prompting Upgrades Neural Network Sparsification: A Data-Model Perspective Can Jin Tianjin Huang Yihua Zhang Mykola Pechenizkiy Sijia Liu Shiwei Liu Tianlong Chen VLM 30 26 0 03 Dec 2023
Pruning Small Pre-Trained Weights Irreversibly and Monotonically Impairs "Difficult" Downstream Tasks in LLMs Lu Yin Ajay Jaiswal Shiwei Liu Souvik Kundu Zhangyang Wang 24 7 0 29 Sep 2023
Accurate Neural Network Pruning Requires Rethinking Sparse Optimization Denis Kuznedelev Eldar Kurtic Eugenia Iofinova Elias Frantar Alexandra Peste Dan Alistarh VLM 35 11 0 03 Aug 2023
Dynamic Sparse Training with Structured Sparsity Mike Lasby A. Golubeva Utku Evci Mihai Nica Yani Andrew Ioannou 29 19 0 03 May 2023
Powerpropagation: A sparsity inducing weight reparameterisation Jonathan Richard Schwarz Siddhant M. Jayakumar Razvan Pascanu P. Latham Yee Whye Teh 90 54 0 01 Oct 2021
Carbon Emissions and Large Neural Network Training David A. Patterson Joseph E. Gonzalez Quoc V. Le Chen Liang Lluís-Miquel Munguía D. Rothchild David R. So Maud Texier J. Dean AI4CE 244 644 0 21 Apr 2021
Accelerated Sparse Neural Training: A Provable and Efficient Method to Find N:M Transposable Masks Itay Hubara Brian Chmiel Moshe Island Ron Banner S. Naor Daniel Soudry 50 110 0 16 Feb 2021
SCOP: Scientific Control for Reliable Neural Network Pruning Yehui Tang Yunhe Wang Yixing Xu Dacheng Tao Chunjing Xu Chao Xu Chang Xu AAML 50 166 0 21 Oct 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 243 4,469 0 23 Jan 2020