v1v2v3v4 (latest)

Advancing Model Pruning via Bi-level Optimization

8 October 2022

Papers citing "Advancing Model Pruning via Bi-level Optimization"

50 / 71 papers shown

Title
RoRA: Efficient Fine-Tuning of LLM with Reliability Optimization for Rank Adaptation Jun Liu Zhenglun Kong Peiyan Dong Changdi Yang Xuan Shen ... Wei Niu Wenbin Zhang Xue Lin Dong Huang Yanzhi Wang ALM 90 5 0 08 Jan 2025
WAGLE: Strategic Weight Attribution for Effective and Modular Unlearning in Large Language Models Jinghan Jia Jiancheng Liu Yihua Zhang Parikshit Ram Nathalie Baracaldo Sijia Liu MU 148 8 0 23 Oct 2024
Relax and penalize: a new bilevel approach to mixed-binary hyperparameter optimization M. D. Santis Jordan Frécon Francesco Rinaldi Saverio Salzo Martin Schmidt Martin Schmidt 100 0 0 21 Aug 2023
Gradient-based Bi-level Optimization for Deep Learning: A Survey Can Chen Xiangshan Chen Chen Ma Zixuan Liu Xue Liu 176 39 0 24 Jul 2022
Prospect Pruning: Finding Trainable Weights at Initialization using Meta-Gradients Milad Alizadeh Shyam A. Tailor L. Zintgraf Joost R. van Amersfoort Sebastian Farquhar Nicholas D. Lane Y. Gal 149 41 0 16 Feb 2022
Coarsening the Granularity: Towards Structurally Sparse Lottery Tickets Tianlong Chen Xuxi Chen Xiaolong Ma Yanzhi Wang Zhangyang Wang 64 34 0 09 Feb 2022
Revisiting and Advancing Fast Adversarial Training Through The Lens of Bi-Level Optimization Yihua Zhang Guanhua Zhang Prashant Khanduri Min-Fong Hong Shiyu Chang Sijia Liu AAML 83 89 0 23 Dec 2021
GDP: Stabilized Neural Network Pruning via Gates with Differentiable Polarization Yi Guo Huan Yuan Jianchao Tan Zhangyang Wang Sen Yang Ji Liu 72 46 0 06 Sep 2021
Sanity Checks for Lottery Tickets: Does Your Winning Ticket Really Win the Jackpot? Xiaolong Ma Geng Yuan Xuan Shen Tianlong Chen Xuxi Chen ... Ning Liu Minghai Qin Sijia Liu Zhangyang Wang Yanzhi Wang 159 64 0 01 Jul 2021
AC/DC: Alternating Compressed/DeCompressed Training of Deep Neural Networks Alexandra Peste Eugenia Iofinova Adrian Vladu Dan Alistarh AI4CE 402 72 0 23 Jun 2021
Efficient Lottery Ticket Finding: Less Data is More Zhenyu Zhang Xuxi Chen Tianlong Chen Zhangyang Wang 94 54 0 06 Jun 2021
GANs Can Play Lottery Tickets Too Xuxi Chen Zhenyu Zhang Yongduo Sui Tianlong Chen GAN 71 58 0 31 May 2021
Effective Sparsification of Neural Networks with Global Sparsity Constraint Xiao Zhou Weizhong Zhang Hang Xu Tong Zhang 150 63 0 03 May 2021
Playing Lottery Tickets with Vision and Language Zhe Gan Yen-Chun Chen Linjie Li Tianlong Chen Yu Cheng Shuohang Wang Jingjing Liu Lijuan Wang Zicheng Liu VLM 138 56 0 23 Apr 2021
Deep learning: a statistical viewpoint Peter L. Bartlett Andrea Montanari Alexander Rakhlin 70 279 0 16 Mar 2021
Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks Torsten Hoefler Dan Alistarh Tal Ben-Nun Nikoli Dryden Alexandra Peste MQ 316 725 0 31 Jan 2021
Neural Pruning via Growing Regularization Huan Wang Can Qin Yulun Zhang Y. Fu 94 146 0 16 Dec 2020
The Lottery Tickets Hypothesis for Supervised and Self-supervised Pre-training in Computer Vision Models Tianlong Chen Jonathan Frankle Shiyu Chang Sijia Liu Yang Zhang Michael Carbin Zhangyang Wang 68 123 0 12 Dec 2020
Layer-adaptive sparsity for the Magnitude-based Pruning Jaeho Lee Sejun Park Sangwoo Mo SungSoo Ahn Jinwoo Shin 93 203 0 15 Oct 2020
Winning Lottery Tickets in Deep Generative Models Neha Kalibhat Yogesh Balaji Soheil Feizi WIGM 80 42 0 05 Oct 2020
Are Neural Nets Modular? Inspecting Functional Modularity Through Differentiable Weight Masks Róbert Csordás Sjoerd van Steenkiste Jürgen Schmidhuber 98 97 0 05 Oct 2020
Sanity-Checking Pruning Methods: Random Tickets can Win the Jackpot Jingtong Su Yihang Chen Tianle Cai Tianhao Wu Ruiqi Gao Liwei Wang Jason D. Lee 64 86 0 22 Sep 2020
Pruning Neural Networks at Initialization: Why are We Missing the Mark? Jonathan Frankle Gintare Karolina Dziugaite Daniel M. Roy Michael Carbin 67 240 0 18 Sep 2020
Improved Bilevel Model: Fast and Optimal Algorithm with Theoretical Guarantee Junyi Li Bin Gu Heng-Chiao Huang 66 41 0 01 Sep 2020
Single Shot Structured Pruning Before Training Joost R. van Amersfoort Milad Alizadeh Sebastian Farquhar Nicholas D. Lane Y. Gal 64 22 0 01 Jul 2020
On the Iteration Complexity of Hypergradient Computation Riccardo Grazzi Luca Franceschi Massimiliano Pontil Saverio Salzo 117 197 0 29 Jun 2020
A Primer on Zeroth-Order Optimization in Signal Processing and Machine Learning Sijia Liu Pin-Yu Chen B. Kailkhura Gaoyuan Zhang A. Hero III P. Varshney 76 235 0 11 Jun 2020
Pruning neural networks without any data by iteratively conserving synaptic flow Hidenori Tanaka D. Kunin Daniel L. K. Yamins Surya Ganguli 188 650 0 09 Jun 2020
A Generic First-Order Algorithmic Framework for Bi-Level Programming Beyond Lower-Level Singleton Risheng Liu Pan Mu Xiaoming Yuan Shangzhi Zeng Jin Zhang 63 131 0 07 Jun 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 898 42,463 0 28 May 2020
DSA: More Efficient Budgeted Pruning via Differentiable Sparsity Allocation Xuefei Ning Tianchen Zhao Wenshuo Li Peng Lei Yu Wang Huazhong Yang 90 104 0 05 Apr 2020
MetaPoison: Practical General-purpose Clean-label Data Poisoning Wenjie Huang Jonas Geiping Liam H. Fowl Gavin Taylor Tom Goldstein 115 190 0 01 Apr 2020
What is the State of Neural Network Pruning? Davis W. Blalock Jose Javier Gonzalez Ortiz Jonathan Frankle John Guttag 280 1,054 0 06 Mar 2020
Comparing Rewinding and Fine-tuning in Neural Network Pruning Alex Renda Jonathan Frankle Michael Carbin 301 388 0 05 Mar 2020
Is the Meta-Learning Idea Able to Improve the Generalization of Deep Neural Networks on the Standard Supervised Learning? Xiang Deng Zhongfei Zhang AI4CE 49 5 0 27 Feb 2020
On the Decision Boundaries of Neural Networks: A Tropical Geometry Perspective Motasem Alfarra Adel Bibi Hasan Hammoud M. Gaafar Guohao Li 62 26 0 20 Feb 2020
Soft Threshold Weight Reparameterization for Learnable Sparsity Aditya Kusupati Vivek Ramanujan Raghav Somani Mitchell Wortsman Prateek Jain Sham Kakade Ali Farhadi 155 247 0 08 Feb 2020
An Image Enhancing Pattern-based Sparsity for Real-time Inference on Mobile Devices Xiaolong Ma Wei Niu Tianyun Zhang Sijia Liu Sheng Lin ... Xiang Chen Jian Tang Kaisheng Ma Bin Ren Yanzhi Wang 91 27 0 20 Jan 2020
PatDNN: Achieving Real-Time DNN Execution on Mobile Devices with Pattern-based Weight Pruning Wei Niu Xiaolong Ma Sheng Lin Shihao Wang Xuehai Qian Xinyu Lin Yanzhi Wang Bin Ren MQ 100 228 0 01 Jan 2020
PyHessian: Neural Networks Through the Lens of the Hessian Z. Yao A. Gholami Kurt Keutzer Michael W. Mahoney ODL 81 304 0 16 Dec 2019
Linear Mode Connectivity and the Lottery Ticket Hypothesis Jonathan Frankle Gintare Karolina Dziugaite Daniel M. Roy Michael Carbin MoMe 163 630 0 11 Dec 2019
What's Hidden in a Randomly Weighted Neural Network? Vivek Ramanujan Mitchell Wortsman Aniruddha Kembhavi Ali Farhadi Mohammad Rastegari 66 361 0 29 Nov 2019
Rigging the Lottery: Making All Tickets Winners Utku Evci Trevor Gale Jacob Menick Pablo Samuel Castro Erich Elsen 199 607 0 25 Nov 2019
Meta-Learning with Implicit Gradients Aravind Rajeswaran Chelsea Finn Sham Kakade Sergey Levine 116 858 0 10 Sep 2019
PCONV: The Missing but Desirable Sparsity in DNN Weight Pruning for Real-time Execution on Mobile Devices Xiaolong Ma Fu-Ming Guo Wei Niu Xue Lin Jian Tang Kaisheng Ma Bin Ren Yanzhi Wang CVBM 78 176 0 06 Sep 2019
Importance Estimation for Neural Network Pruning Pavlo Molchanov Arun Mallya Stephen Tyree I. Frosio Jan Kautz 3DPC 92 885 0 25 Jun 2019
Playing the lottery with rewards and multiple languages: lottery tickets in RL and NLP Haonan Yu Sergey Edunov Yuandong Tian Ari S. Morcos 56 150 0 06 Jun 2019
An ADMM Based Framework for AutoML Pipeline Configuration Sijia Liu Parikshit Ram Deepak Vijaykeerthy Djallel Bouneffouf Gregory Bramble Horst Samulowitz Dakuo Wang A. Conn Alexander G. Gray 87 76 0 01 May 2019
The State of Sparsity in Deep Neural Networks Trevor Gale Erich Elsen Sara Hooker 165 763 0 25 Feb 2019
ADMM-NN: An Algorithm-Hardware Co-Design Framework of DNNs Using Alternating Direction Method of Multipliers Ao Ren Tianyun Zhang Shaokai Ye Jiayu Li Wenyao Xu Xuehai Qian Xinyu Lin Yanzhi Wang MQ 103 161 0 31 Dec 2018