Efficient Transformer-based Large Scale Language Representations using
Hardware-friendly Block Structured Pruning

Efficient Transformer-based Large Scale Language Representations using Hardware-friendly Block Structured Pruning

17 September 2020

Ji Li

Caiwen Ding

Papers citing "Efficient Transformer-based Large Scale Language Representations using Hardware-friendly Block Structured Pruning"

13 / 13 papers shown

Title
RoRA: Efficient Fine-Tuning of LLM with Reliability Optimization for Rank Adaptation Jun Liu Zhenglun Kong Peiyan Dong Changdi Yang Xuan Shen ... Wei Niu Wenbin Zhang Xue Lin Dong Huang Yanzhi Wang ALM 46 2 0 08 Jan 2025
Only Send What You Need: Learning to Communicate Efficiently in Federated Multilingual Machine Translation Yun-Wei Chu Dong-Jun Han Christopher G. Brinton 28 4 0 15 Jan 2024
RecycleGPT: An Autoregressive Language Model with Recyclable Module Yu Jiang Qiaozhi He Xiaomin Zhuang Zhihua Wu Kunpeng Wang Wenlai Zhao Guangwen Yang KELM 28 3 0 07 Aug 2023
What Matters In The Structured Pruning of Generative Language Models? Michael Santacroce Zixin Wen Yelong Shen Yuan-Fang Li 21 33 0 07 Feb 2023
Exploring Attention Map Reuse for Efficient Transformer Neural Networks Kyuhong Shim Jungwook Choi Wonyong Sung ViT 24 3 0 29 Jan 2023
SpeechNet: Weakly Supervised, End-to-End Speech Recognition at Industrial Scale Raphael Tang K. Kumar Gefei Yang Akshat Pandey Yajie Mao Vladislav Belyaev Madhuri Emmadi Craig Murray Ferhan Ture Jimmy J. Lin 24 4 0 21 Nov 2022
Efficiently Scaling Transformer Inference Reiner Pope Sholto Douglas Aakanksha Chowdhery Jacob Devlin James Bradbury Anselm Levskaya Jonathan Heek Kefan Xiao Shivani Agrawal J. Dean 34 295 0 09 Nov 2022
From Dense to Sparse: Contrastive Pruning for Better Pre-trained Language Model Compression Runxin Xu Fuli Luo Chengyu Wang Baobao Chang Jun Huang Songfang Huang Fei Huang VLM 27 25 0 14 Dec 2021
Pruning Self-attentions into Convolutional Layers in Single Path Haoyu He Jianfei Cai Jing Liu Zizheng Pan Jing Zhang Dacheng Tao Bohan Zhuang ViT 34 40 0 23 Nov 2021
Learned Token Pruning for Transformers Sehoon Kim Sheng Shen D. Thorsley A. Gholami Woosuk Kwon Joseph Hassoun Kurt Keutzer 14 145 0 02 Jul 2021
Dancing along Battery: Enabling Transformer with Run-time Reconfigurability on Mobile Devices Yuhong Song Weiwen Jiang Bingbing Li Panjie Qi Qingfeng Zhuge E. Sha Sakyasingha Dasgupta Yiyu Shi Caiwen Ding 18 18 0 12 Feb 2021
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism M. Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 245 1,821 0 17 Sep 2019
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 297 6,959 0 20 Apr 2018