Transkimmer: Transformer Learns to Layer-wise Skim

Transkimmer: Transformer Learns to Layer-wise Skim

15 May 2022

Jingwen Leng

Papers citing "Transkimmer: Transformer Learns to Layer-wise Skim"

7 / 7 papers shown

Title
Fovea Transformer: Efficient Long-Context Modeling with Structured Fine-to-Coarse Attention Ziwei He Jian Yuan Le Zhou Jingwen Leng Bo Jiang 29 1 0 13 Nov 2023
Efficiently Controlling Multiple Risks with Pareto Testing Bracha Laufer-Goldshtein Adam Fisch Regina Barzilay Tommi Jaakkola 36 16 0 14 Oct 2022
ANT: Exploiting Adaptive Numerical Data Type for Low-bit Deep Neural Network Quantization Cong Guo Chen Zhang Jingwen Leng Zihan Liu Fan Yang Yun-Bo Liu Minyi Guo Yuhao Zhu MQ 16 55 0 30 Aug 2022
SQuant: On-the-Fly Data-Free Quantization via Diagonal Hessian Approximation Cong Guo Yuxian Qiu Jingwen Leng Xiaotian Gao Chen Zhang Yunxin Liu Fan Yang Yuhao Zhu Minyi Guo MQ 70 70 0 14 Feb 2022
On the Transformer Growth for Progressive BERT Training Xiaotao Gu Liyuan Liu Hongkun Yu Jing Li C. L. P. Chen Jiawei Han VLM 66 51 0 23 Oct 2020
Big Bird: Transformers for Longer Sequences Manzil Zaheer Guru Guruganesh Kumar Avinava Dubey Joshua Ainslie Chris Alberti ... Philip Pham Anirudh Ravula Qifan Wang Li Yang Amr Ahmed VLM 274 2,013 0 28 Jul 2020
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 297 6,956 0 20 Apr 2018