FlashAttention: Fast and Memory-Efficient Exact Attention with
IO-Awareness

v1v2 (latest)

FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness

27 May 2022

Christopher Ré

ArXiv (abs)PDF HTML

Papers citing "FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness"

8 / 1,508 papers shown

Title
Group DETR: Fast DETR Training with Group-Wise One-to-Many Assignment Qiang Chen Xiaokang Chen Jian Wang Shan Zhang Kun Yao Haocheng Feng Junyu Han Errui Ding Gang Zeng Jingdong Wang ViT 143 135 0 26 Jul 2022
DETRs with Hybrid Matching Ding Jia Yuhui Yuan Hao He Xiao-pei Wu Haojun Yu Weihong Lin Lei-huan Sun Chao Zhang Hanhua Hu 69 198 0 26 Jul 2022
Efficient High-Resolution Deep Learning: A Survey Arian Bakhtiarnia Qi Zhang Alexandros Iosifidis MedIm 158 21 0 26 Jul 2022
Vision Transformers: From Semantic Segmentation to Dense Prediction Li Zhang Jiachen Lu Sixiao Zheng Xinxuan Zhao Xiatian Zhu Yanwei Fu Tao Xiang Jianfeng Feng Philip H. S. Torr ViT 99 8 0 19 Jul 2022
Understanding Performance of Long-Document Ranking Models through Comprehensive Evaluation and Leaderboarding Leonid Boytsov David Akinpelu Tianyi Lin Fangwei Gao Yutian Zhao Jeffrey Huang Nipun Katyal Eric Nyberg 174 9 0 04 Jul 2022
LIFT: Language-Interfaced Fine-Tuning for Non-Language Machine Learning Tasks Tuan Dinh Yuchen Zeng Ruisu Zhang Ziqian Lin Michael Gira Shashank Rajput Jy-yong Sohn Dimitris Papailiopoulos Kangwook Lee LMTD 171 140 0 14 Jun 2022
Multimodal Learning with Transformers: A Survey Peng Xu Xiatian Zhu David Clifton ViT 233 574 0 13 Jun 2022
Self-attention Does Not Need $O(n^2)$ Memory M. Rabe Charles Staats LRM 125 150 0 10 Dec 2021