FLAT: An Optimized Dataflow for Mitigating Attention Bottlenecks

FLAT: An Optimized Dataflow for Mitigating Attention Bottlenecks

13 July 2021

Suvinay Subramanian

Amir Yazdanbakhsh

Papers citing "FLAT: An Optimized Dataflow for Mitigating Attention Bottlenecks"

9 / 9 papers shown

Title
PAPI: Exploiting Dynamic Parallelism in Large Language Model Decoding with a Processing-In-Memory-Enabled Computing System Yintao He Haiyu Mao Christina Giannoula Mohammad Sadrosadati Juan Gómez Luna Huawei Li Xiaowei Li Ying Wang O. Mutlu 41 5 0 21 Feb 2025
Unifying KV Cache Compression for Large Language Models with LeanKV Yanqi Zhang Yuwei Hu Runyuan Zhao John C. S. Lui Haibo Chen MQ 130 5 0 04 Dec 2024
MAS-Attention: Memory-Aware Stream Processing for Attention Acceleration on Resource-Constrained Edge Devices Mohammadali Shakerdargah Shan Lu Chao Gao Di Niu 70 0 0 20 Nov 2024
Characterizing the Accuracy - Efficiency Trade-off of Low-rank Decomposition in Language Models Chakshu Moar Michael Pellauer Hyoukjun Kwon 35 1 0 10 May 2024
Compound Word Transformer: Learning to Compose Full-Song Music over Dynamic Directed Hypergraphs Wen-Yi Hsiao Jen-Yu Liu Yin-Cheng Yeh Yi-Hsuan Yang 107 180 0 07 Jan 2021
I-BERT: Integer-only BERT Quantization Sehoon Kim A. Gholami Z. Yao Michael W. Mahoney Kurt Keutzer MQ 96 341 0 05 Jan 2021
TransTrack: Multiple Object Tracking with Transformer Pei Sun Jinkun Cao Yi-Xin Jiang Rufeng Zhang Enze Xie Zehuan Yuan Changhu Wang Ping Luo ViT VOT 255 565 0 31 Dec 2020
Efficient Content-Based Sparse Attention with Routing Transformers Aurko Roy M. Saffar Ashish Vaswani David Grangier MoE 243 580 0 12 Mar 2020
Q-BERT: Hessian Based Ultra Low Precision Quantization of BERT Sheng Shen Zhen Dong Jiayu Ye Linjian Ma Z. Yao A. Gholami Michael W. Mahoney Kurt Keutzer MQ 227 575 0 12 Sep 2019