NN-LUT: Neural Approximation of Non-Linear Operations for Efficient
Transformer Inference

NN-LUT: Neural Approximation of Non-Linear Operations for Efficient Transformer Inference

3 December 2021

Papers citing "NN-LUT: Neural Approximation of Non-Linear Operations for Efficient Transformer Inference"

14 / 14 papers shown

Title
LUT-DLA: Lookup Table as Efficient Extreme Low-Bit Deep Learning Accelerator Guoyu Li Shengyu Ye Chong Chen Yang Wang Fan Yang Ting Cao Cheng Liu Mohamed M. Sabry Mao Yang MQ 275 0 0 18 Jan 2025
I-BERT: Integer-only BERT Quantization Sehoon Kim A. Gholami Z. Yao Michael W. Mahoney Kurt Keutzer MQ 130 348 0 05 Jan 2021
SpAtten: Efficient Sparse Attention Architecture with Cascade Token and Head Pruning Hanrui Wang Zhekai Zhang Song Han 87 384 0 17 Dec 2020
TernaryBERT: Distillation-aware Ultra-low Bit BERT Wei Zhang Lu Hou Yichun Yin Lifeng Shang Xiao Chen Xin Jiang Qun Liu MQ 63 209 0 27 Sep 2020
MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices Zhiqing Sun Hongkun Yu Xiaodan Song Renjie Liu Yiming Yang Denny Zhou MQ 77 807 0 06 Apr 2020
Q8BERT: Quantized 8Bit BERT Ofir Zafrir Guy Boudoukh Peter Izsak Moshe Wasserblat MQ 49 502 0 14 Oct 2019
RoBERTa: A Robustly Optimized BERT Pretraining Approach Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy M. Lewis Luke Zettlemoyer Veselin Stoyanov AIMat 368 24,160 0 26 Jul 2019
Efficient 8-Bit Quantization of Transformer Neural Machine Language Translation Model Aishwarya Bhandare Vamsi Sripathi Deepthi Karkada Vivek V. Menon Sun Choi Kushal Datta V. Saletore MQ 45 132 0 03 Jun 2019
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 853 93,936 0 11 Oct 2018
AXNet: ApproXimate computing using an end-to-end trainable neural network Zhenghao Peng Xuyang Chen Chengwen Xu Naifeng Jing Xiaoyao Liang Cewu Lu Li Jiang 18 19 0 27 Jul 2018
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 572 7,080 0 20 Apr 2018
The Expressive Power of Neural Networks: A View from the Width Zhou Lu Hongming Pu Feicheng Wang Zhiqiang Hu Liwei Wang 62 886 0 08 Sep 2017
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 338 129,831 0 12 Jun 2017
SQuAD: 100,000+ Questions for Machine Comprehension of Text Pranav Rajpurkar Jian Zhang Konstantin Lopyrev Percy Liang RALM 123 8,067 0 16 Jun 2016