NASH: A Simple Unified Framework of Structured Pruning for Accelerating Encoder-Decoder Language Models

16 October 2023

Papers citing "NASH: A Simple Unified Framework of Structured Pruning for Accelerating Encoder-Decoder Language Models"

24 / 24 papers shown

Title
PASER: Post-Training Data Selection for Efficient Pruned Large Language Model Recovery Bowei He Lihao Yin Hui-Ling Zhen Xiaokun Zhang Mingxuan Yuan Chen Ma 153 0 0 18 Feb 2025
HESSO: Towards Automatic Efficient and User Friendly Any Neural Network Training and Pruning Tianyi Chen Xiaoyi Qu David Aponte Colby R. Banbury Jongwoo Ko Tianyu Ding Yong Ma Vladimir Lyapunov Ilya Zharkov Luming Liang 144 2 0 11 Sep 2024
ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers Z. Yao Reza Yazdani Aminabadi Minjia Zhang Xiaoxia Wu Conglong Li Yuxiong He VLM MQ 125 479 0 04 Jun 2022
Structured Pruning Learns Compact and Accurate Models Mengzhou Xia Zexuan Zhong Danqi Chen VLM 66 187 0 01 Apr 2022
TextPruner: A Model Pruning Toolkit for Pre-Trained Language Models Ziqing Yang Yiming Cui Zhigang Chen SyDa VLM 50 12 0 30 Mar 2022
Scale Efficiently: Insights from Pre-training and Fine-tuning Transformers Yi Tay Mostafa Dehghani J. Rao W. Fedus Samira Abnar Hyung Won Chung Sharan Narang Dani Yogatama Ashish Vaswani Donald Metzler 258 115 0 22 Sep 2021
EfficientBERT: Progressively Searching Multilayer Perceptron via Warm-up Knowledge Distillation Chenhe Dong Guangrun Wang Hang Xu Jiefeng Peng Xiaozhe Ren Xiaodan Liang 75 28 0 15 Sep 2021
MiniLMv2: Multi-Head Self-Attention Relation Distillation for Compressing Pretrained Transformers Wenhui Wang Hangbo Bao Shaohan Huang Li Dong Furu Wei MQ 89 269 0 31 Dec 2020
Deep Encoder, Shallow Decoder: Reevaluating Non-autoregressive Machine Translation Jungo Kasai Nikolaos Pappas Hao Peng James Cross Noah A. Smith 87 140 0 18 Jun 2020
DynaBERT: Dynamic BERT with Adaptive Width and Depth Lu Hou Zhiqi Huang Lifeng Shang Xin Jiang Xiao Chen Qun Liu MQ 79 322 0 08 Apr 2020
MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices Zhiqing Sun Hongkun Yu Xiaodan Song Renjie Liu Yiming Yang Denny Zhou MQ 109 817 0 06 Apr 2020
MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers Wenhui Wang Furu Wei Li Dong Hangbo Bao Nan Yang Ming Zhou VLM 156 1,278 0 25 Feb 2020
SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarization Bogdan Gliwa Iwona Mochol M. Biesek A. Wawer 124 638 0 27 Nov 2019
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 456 20,298 0 23 Oct 2019
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter Victor Sanh Lysandre Debut Julien Chaumond Thomas Wolf 237 7,547 0 02 Oct 2019
TinyBERT: Distilling BERT for Natural Language Understanding Xiaoqi Jiao Yichun Yin Lifeng Shang Xin Jiang Xiao Chen Linlin Li F. Wang Qun Liu VLM 109 1,869 0 23 Sep 2019
SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems Alex Jinpeng Wang Yada Pruksachatkun Nikita Nangia Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 274 2,323 0 02 May 2019
Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization Shashi Narayan Shay B. Cohen Mirella Lapata AILaw 146 1,682 0 27 Aug 2018
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 1.1K 7,196 0 20 Apr 2018
The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks Jonathan Frankle Michael Carbin 245 3,484 0 09 Mar 2018
Effective Quantization Approaches for Recurrent Neural Networks Md. Zahangir Alom A. Moody N. Maruyama B. Van Essen T. Taha MQ 45 35 0 07 Feb 2018
Get To The Point: Summarization with Pointer-Generator Networks A. See Peter J. Liu Christopher D. Manning 3DPC 306 4,025 0 14 Apr 2017
Effective Quantization Methods for Recurrent Neural Networks Qinyao He He Wen Shuchang Zhou Yuxin Wu Cong Yao Xinyu Zhou Yuheng Zou MQ 71 75 0 30 Nov 2016
EIE: Efficient Inference Engine on Compressed Deep Neural Network Song Han Xingyu Liu Huizi Mao Jing Pu A. Pedram M. Horowitz W. Dally 127 2,459 0 04 Feb 2016