Title
An Automatic and Efficient BERT Pruning for Edge AI Systems Shaoyi Huang Ning Liu Yueying Liang Hongwu Peng Hongjia Li Dongkuan Xu Mimi Xie Caiwen Ding 107 22 0 21 Jun 2022
Accelerating Framework of Transformer by Hardware Design and Model Compression Co-Optimization Panjie Qi E. Sha Qingfeng Zhuge Hongwu Peng Shaoyi Huang Zhenglun Kong Yuhong Song Bingbing Li 67 51 0 19 Oct 2021
Binary Complex Neural Network Acceleration on FPGA Hongwu Peng Shangli Zhou Scott Weitze Jiaxin Li Sahidul Islam ... Wei Zhang M. Song Mimi Xie Hang Liu Caiwen Ding MQ 53 20 0 10 Aug 2021
Improving DNN Fault Tolerance using Weight Pruning and Differential Crossbar Mapping for ReRAM-based Edge AI Geng Yuan Zhiheng Liao Xiaolong Ma Yuxuan Cai Zhenglun Kong ... Hongwu Peng Ning Liu Ao Ren Jinhui Wang Yanzhi Wang AAML 78 33 0 16 Jun 2021
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows Ze Liu Yutong Lin Yue Cao Han Hu Yixuan Wei Zheng Zhang Stephen Lin B. Guo ViT 463 21,566 0 25 Mar 2021
SpAtten: Efficient Sparse Attention Architecture with Cascade Token and Head Pruning Hanrui Wang Zhekai Zhang Song Han 130 393 0 17 Dec 2020
End-to-End Human Pose and Mesh Reconstruction with Transformers Kevin Qinghong Lin Lijuan Wang Zicheng Liu ViT 74 626 0 17 Dec 2020
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai ... Matthias Minderer G. Heigold Sylvain Gelly Jakob Uszkoreit N. Houlsby ViT 670 41,430 0 22 Oct 2020
TurboTransformers: An Efficient GPU Serving System For Transformer Models Jiarui Fang Yang Yu Chen-liang Zhao Jie Zhou 56 139 0 09 Oct 2020
TernaryBERT: Distillation-aware Ultra-low Bit BERT Wei Zhang Lu Hou Yichun Yin Lifeng Shang Xiao Chen Xin Jiang Qun Liu MQ 93 211 0 27 Sep 2020
Big Bird: Transformers for Longer Sequences Manzil Zaheer Guru Guruganesh Kumar Avinava Dubey Joshua Ainslie Chris Alberti ... Philip Pham Anirudh Ravula Qifan Wang Li Yang Amr Ahmed VLM 556 2,099 0 28 Jul 2020
A $^3$ : Accelerating Attention Mechanisms in Neural Networks with Approximation Tae Jun Ham Sungjun Jung Seonghak Kim Young H. Oh Yeonhong Park ... Jung-Hun Park Sanghee Lee Kyoung Park Jae W. Lee D. Jeong 68 219 0 22 Feb 2020
Reformer: The Efficient Transformer Nikita Kitaev Lukasz Kaiser Anselm Levskaya VLM 199 2,332 0 13 Jan 2020
BP-Transformer: Modelling Long-Range Context via Binary Partitioning Zihao Ye Qipeng Guo Quan Gan Xipeng Qiu Zheng Zhang 78 77 0 11 Nov 2019
On the Relationship between Self-Attention and Convolutional Layers Jean-Baptiste Cordonnier Andreas Loukas Martin Jaggi 116 535 0 08 Nov 2019
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter Victor Sanh Lysandre Debut Julien Chaumond Thomas Wolf 255 7,547 0 02 Oct 2019
RoBERTa: A Robustly Optimized BERT Pretraining Approach Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy M. Lewis Luke Zettlemoyer Veselin Stoyanov AIMat 677 24,541 0 26 Jul 2019
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.8K 95,175 0 11 Oct 2018
Know What You Don't Know: Unanswerable Questions for SQuAD Pranav Rajpurkar Robin Jia Percy Liang RALM ELM 292 2,853 0 11 Jun 2018
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 786 132,363 0 12 Jun 2017
Pointer Sentinel Mixture Models Stephen Merity Caiming Xiong James Bradbury R. Socher RALM 338 2,898 0 26 Sep 2016
SQuAD: 100,000+ Questions for Machine Comprehension of Text Pranav Rajpurkar Jian Zhang Konstantin Lopyrev Percy Liang RALM 316 8,169 0 16 Jun 2016
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 2.2K 194,426 0 10 Dec 2015