Pruning Small Pre-Trained Weights Irreversibly and Monotonically Impairs
"Difficult" Downstream Tasks in LLMs

Pruning Small Pre-Trained Weights Irreversibly and Monotonically Impairs "Difficult" Downstream Tasks in LLMs

29 September 2023

Lu Yin

Papers citing "Pruning Small Pre-Trained Weights Irreversibly and Monotonically Impairs "Difficult" Downstream Tasks in LLMs"

5 / 5 papers shown

Title
Instant Soup: Cheap Pruning Ensembles in A Single Pass Can Draw Lottery Tickets from Large Models A. Jaiswal Shiwei Liu Tianlong Chen Ying Ding Zhangyang Wang VLM 32 22 0 18 Jun 2023
Towards More Effective and Economic Sparsely-Activated Model Hao Jiang Ke Zhan Jianwei Qu Yongkang Wu Zhaoye Fei ... Enrui Hu Yinxia Zhang Yantao Jia Fan Yu Zhao Cao MoE 152 12 0 14 Oct 2021
I-BERT: Integer-only BERT Quantization Sehoon Kim A. Gholami Z. Yao Michael W. Mahoney Kurt Keutzer MQ 102 341 0 05 Jan 2021
The Lottery Ticket Hypothesis for Pre-trained BERT Networks Tianlong Chen Jonathan Frankle Shiyu Chang Sijia Liu Yang Zhang Zhangyang Wang Michael Carbin 156 345 0 23 Jul 2020
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 297 6,959 0 20 Apr 2018