Castling-ViT: Compressing Self-Attention via Switching Towards
Linear-Angular Attention at Vision Transformer Inference

v1v2v3v4 (latest)

Castling-ViT: Compressing Self-Attention via Switching Towards Linear-Angular Attention at Vision Transformer Inference

18 November 2022

ArXiv (abs)PDF HTML

Papers citing "Castling-ViT: Compressing Self-Attention via Switching Towards Linear-Angular Attention at Vision Transformer Inference"

16 / 66 papers shown

Title
FBNetV3: Joint Architecture-Recipe Search using Predictor Pretraining Xiaoliang Dai Alvin Wan Peizhao Zhang Bichen Wu Zijian He ... Kan Chen Yuandong Tian Matthew Yu Peter Vajda Joseph E. Gonzalez 73 73 0 03 Jun 2020
End-to-End Object Detection with Transformers Nicolas Carion Francisco Massa Gabriel Synnaeve Nicolas Usunier Alexander Kirillov Sergey Zagoruyko ViT 3DV PINN 454 13,130 0 26 May 2020
MobileDets: Searching for Object Detection Architectures for Mobile Accelerators Yunyang Xiong Hanxiao Liu Suyog Gupta Berkin Akin Gabriel Bender Yongzhe Wang Pieter-Jan Kindermans Mingxing Tan Vikas Singh Bo Chen ObjD 45 133 0 30 Apr 2020
Lite Transformer with Long-Short Range Attention Zhanghao Wu Zhijian Liu Ji Lin Chengyue Wu Song Han 60 322 0 24 Apr 2020
Longformer: The Long-Document Transformer Iz Beltagy Matthew E. Peters Arman Cohan RALM VLM 187 4,100 0 10 Apr 2020
Designing Network Design Spaces Ilija Radosavovic Raj Prateek Kosaraju Ross B. Girshick Kaiming He Piotr Dollár GNN 107 1,697 0 30 Mar 2020
Reformer: The Efficient Transformer Nikita Kitaev Lukasz Kaiser Anselm Levskaya VLM 207 2,333 0 13 Jan 2020
EfficientDet: Scalable and Efficient Object Detection Mingxing Tan Ruoming Pang Quoc V. Le 120 5,076 0 20 Nov 2019
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks Mingxing Tan Quoc V. Le 3DV MedIm 172 18,193 0 28 May 2019
Searching for MobileNetV3 Andrew G. Howard Mark Sandler Grace Chu Liang-Chieh Chen Bo Chen ... Yukun Zhu Ruoming Pang Vijay Vasudevan Quoc V. Le Hartwig Adam 394 6,811 0 06 May 2019
Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networks Zhi-Qin John Xu Yaoyu Zhang Yaoyu Zhang Yan Xiao Zheng Ma 129 520 0 19 Jan 2019
Efficient Attention: Attention with Linear Complexities Zhuoran Shen Mingyuan Zhang Haiyu Zhao Shuai Yi Hongsheng Li 101 534 0 04 Dec 2018
MobileNetV2: Inverted Residuals and Linear Bottlenecks Mark Sandler Andrew G. Howard Menglong Zhu A. Zhmoginov Liang-Chieh Chen 218 19,353 0 13 Jan 2018
Revisiting Unreasonable Effectiveness of Data in Deep Learning Era Chen Sun Abhinav Shrivastava Saurabh Singh Abhinav Gupta VLM 207 2,407 0 10 Jul 2017
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 805 132,725 0 12 Jun 2017
Microsoft COCO: Common Objects in Context Nayeon Lee Michael Maire Serge J. Belongie Lubomir Bourdev Ross B. Girshick James Hays Pietro Perona Deva Ramanan C. L. Zitnick Piotr Dollár ObjD 437 43,875 0 01 May 2014