Rethinking the Value of Transformer Components

7 November 2020

Papers citing "Rethinking the Value of Transformer Components"

13 / 13 papers shown

Title
Outlier Weighed Layerwise Sparsity (OWL): A Missing Secret Sauce for Pruning LLMs to High Sparsity Lu Yin You Wu Zhenyu Zhang Cheng-Yu Hsieh Yaqing Wang ... Mykola Pechenizkiy Yi Liang Michael Bendersky Zhangyang Wang Shiwei Liu 36 79 0 08 Oct 2023
Transformer-based models and hardware acceleration analysis in autonomous driving: A survey J. Zhong Zheng Liu Xiangshan Chen ViT 48 17 0 21 Apr 2023
Transferable Adversarial Attacks on Vision Transformers with Token Gradient Regularization Jianping Zhang Yizhan Huang Weibin Wu Michael R. Lyu AAML ViT 20 50 0 28 Mar 2023
Multimodal Learning with Transformers: A Survey Peng Xu Xiatian Zhu David Clifton ViT 79 530 0 13 Jun 2022
Understanding and Mitigating the Uncertainty in Zero-Shot Translation Wenxuan Wang Wenxiang Jiao Shuo Wang Zhaopeng Tu Michael R. Lyu UQLM 42 9 0 20 May 2022
Training-free Transformer Architecture Search Qinqin Zhou Kekai Sheng Xiawu Zheng Ke Li Xing Sun Yonghong Tian Jie Chen Rongrong Ji ViT 45 46 0 23 Mar 2022
Kformer: Knowledge Injection in Transformer Feed-Forward Layers Yunzhi Yao Shaohan Huang Li Dong Furu Wei Huajun Chen Ningyu Zhang KELM MedIm 31 42 0 15 Jan 2022
MoEfication: Transformer Feed-forward Layers are Mixtures of Experts Zhengyan Zhang Yankai Lin Zhiyuan Liu Peng Li Maosong Sun Jie Zhou MoE 29 118 0 05 Oct 2021
Towards Efficient Post-training Quantization of Pre-trained Language Models Haoli Bai Lu Hou Lifeng Shang Xin Jiang Irwin King M. Lyu MQ 82 47 0 30 Sep 2021
Bag of Tricks for Optimizing Transformer Efficiency Ye Lin Yanyang Li Tong Xiao Jingbo Zhu 34 6 0 09 Sep 2021
How Does Selective Mechanism Improve Self-Attention Networks? Xinwei Geng Longyue Wang Xing Wang Bing Qin Ting Liu Zhaopeng Tu AAML 39 35 0 03 May 2020
Comparing Rewinding and Fine-tuning in Neural Network Pruning Alex Renda Jonathan Frankle Michael Carbin 235 383 0 05 Mar 2020
Improving neural networks by preventing co-adaptation of feature detectors Geoffrey E. Hinton Nitish Srivastava A. Krizhevsky Ilya Sutskever Ruslan Salakhutdinov VLM 266 7,639 0 03 Jul 2012