SmartFRZ: An Efficient Training Framework using Attention-Based Layer
Freezing

SmartFRZ: An Efficient Training Framework using Attention-Based Layer Freezing

30 January 2024

Papers citing "SmartFRZ: An Efficient Training Framework using Attention-Based Layer Freezing"

16 / 16 papers shown

Title
Budgeted Online Continual Learning by Adaptive Layer Freezing and Frequency-based Sampling Minhyuk Seo Hyunseo Koh Jonghyun Choi 79 2 0 19 Oct 2024
MEST: Accurate and Fast Memory-Economic Sparse Training Framework on the Edge Geng Yuan Xiaolong Ma Wei Niu Zhengang Li Zhenglun Kong ... Minghai Qin Bin Ren Yanzhi Wang Sijia Liu Xue Lin 50 93 0 26 Oct 2021
Contextual Transformer Networks for Visual Recognition Yehao Li Ting Yao Yingwei Pan Tao Mei ViT 68 482 0 26 Jul 2021
Enabling On-Device Self-Supervised Contrastive Learning With Selective Data Contrast Yawen Wu Zhepeng Wang Dewen Zeng Yiyu Shi Jingtong Hu SSL 61 28 0 07 Jun 2021
PipeTransformer: Automated Elastic Pipelining for Distributed Training of Transformers Chaoyang He Shen Li Mahdi Soltanolkotabi Salman Avestimehr 39 29 0 05 Feb 2021
Training data-efficient image transformers & distillation through attention Hugo Touvron Matthieu Cord Matthijs Douze Francisco Massa Alexandre Sablayrolles Hervé Jégou ViT 345 6,731 0 23 Dec 2020
Accelerating Training of Transformer-Based Language Models with Progressive Layer Dropping Minjia Zhang Yuxiong He AI4CE 39 102 0 26 Oct 2020
PatDNN: Achieving Real-Time DNN Execution on Mobile Devices with Pattern-based Weight Pruning Wei Niu Xiaolong Ma Sheng Lin Shihao Wang Xuehai Qian Xinyu Lin Yanzhi Wang Bin Ren MQ 59 228 0 01 Jan 2020
Rigging the Lottery: Making All Tickets Winners Utku Evci Trevor Gale Jacob Menick Pablo Samuel Castro Erich Elsen 166 600 0 25 Nov 2019
Training High-Performance and Large-Scale Deep Neural Networks with Full 8-bit Integers Yukuan Yang Shuang Wu Lei Deng Tianyi Yan Yuan Xie Guoqi Li MQ 129 112 0 05 Sep 2019
Neural Network Acceptability Judgments Alex Warstadt Amanpreet Singh Samuel R. Bowman 209 1,406 0 31 May 2018
MobileNetV2: Inverted Residuals and Linear Bottlenecks Mark Sandler Andrew G. Howard Menglong Zhu A. Zhmoginov Liang-Chieh Chen 169 19,204 0 13 Jan 2018
FreezeOut: Accelerate Training by Progressively Freezing Layers Andrew Brock Theodore Lim J. Ritchie Nick Weston 47 123 0 15 Jun 2017
Neural Attention Models for Sequence Classification: Analysis and Application to Key Term Extraction and Dialogue Act Detection Sheng-syun Shen Hung-yi Lee 58 66 0 31 Mar 2016
How transferable are features in deep neural networks? J. Yosinski Jeff Clune Yoshua Bengio Hod Lipson OOD 196 8,321 0 06 Nov 2014
Visualizing and Understanding Convolutional Networks Matthew D. Zeiler Rob Fergus FAtt SSL 486 15,861 0 12 Nov 2013