FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness

27 May 2022

Papers citing "FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness"

50 / 1,427 papers shown

Title
Antidistillation Sampling Yash Savani Asher Trockman Zhili Feng Avi Schwarzschild Alexander Robey Marc Finzi J. Zico Kolter 46 0 0 17 Apr 2025
A Review of YOLOv12: Attention-Based Enhancements vs. Previous Versions Rahima Khanam Muhammad Hussain 36 0 0 16 Apr 2025
Shared Disk KV Cache Management for Efficient Multi-Instance Inference in RAG-Powered LLMs Hyungwoo Lee Kihyun Kim Jinwoo Kim Jungmin So Myung-Hoon Cha H. Kim James J. Kim Youngjae Kim 32 0 0 16 Apr 2025
MOM: Memory-Efficient Offloaded Mini-Sequence Inference for Long Context Language Models Junyang Zhang Tianyi Zhu Cheng Luo A. Anandkumar RALM 42 0 0 16 Apr 2025
TacoDepth: Towards Efficient Radar-Camera Depth Estimation with One-stage Fusion Yunhong Wang J. Li Chaoyi Hong Ruibo Li Liusheng Sun Xiao-yang Song Zhe Wang Zhiguo Cao Guosheng Lin MDE 29 0 0 16 Apr 2025
VEXP: A Low-Cost RISC-V ISA Extension for Accelerated Softmax Computation in Transformers Run Wang Gamze Islamoglu Andrea Belano Viviane Potocnik Francesco Conti Angelo Garofalo Luca Benini 26 0 0 15 Apr 2025
Understanding and Optimizing Multi-Stage AI Inference Pipelines Abhimanyu Bambhaniya Hanjiang Wu Suvinay Subramanian Sudarshan Srinivasan Souvik Kundu Amir Yazdanbakhsh Suvinay Subramanian Madhu Kumar Tushar Krishna 135 0 0 14 Apr 2025
VDocRAG: Retrieval-Augmented Generation over Visually-Rich Documents Ryota Tanaka Taichi Iki Taku Hasegawa Kyosuke Nishida Kuniko Saito Jun Suzuki VLM 52 0 0 14 Apr 2025
AlayaDB: The Data Foundation for Efficient and Effective Long-context LLM Inference Yangshen Deng Zhengxin You Long Xiang Qilong Li Peiqi Yuan ... Man Lung Yiu Huan Li Qiaomu Shen Rui Mao Bo Tang 42 0 0 14 Apr 2025
Towards Quantifying Commonsense Reasoning with Mechanistic Insights Abhinav Joshi A. Ahmad Divyaksh Shukla Ashutosh Modi ReLM LRM 36 0 0 14 Apr 2025
OVERLORD: Ultimate Scaling of DataLoader for Multi-Source Large Foundation Model Training Juntao Zhao Qi Lu Wei Jia Borui Wan Lei Zuo ... Size Zheng H. Lin Xin Liu Xin Liu Chuan Wu AI4CE 34 0 0 14 Apr 2025
Summarization of Multimodal Presentations with Vision-Language Models: Study of the Effect of Modalities and Structure Théo Gigant Camille Guinaudeau Frédéric Dufaux 26 0 0 14 Apr 2025
Efficient LLM Serving on Hybrid Real-time and Best-effort Requests Wan Borui Zhao Juntao Jiang Chenyu Guo Chuanxiong Wu Chuan VLM 79 1 0 13 Apr 2025
MoE-Lens: Towards the Hardware Limit of High-Throughput MoE LLM Serving Under Resource Constraints Yichao Yuan Lin Ma Nishil Talati MoE 64 0 0 12 Apr 2025
FVQ: A Large-Scale Dataset and A LMM-based Method for Face Video Quality Assessment Sijing Wu Yunhao Li Ziwen Xu Yixuan Gao Huiyu Duan Wei Sun Guangtao Zhai 82 1 0 12 Apr 2025
PACT: Pruning and Clustering-Based Token Reduction for Faster Visual Language Models M. Dhouib Davide Buscaldi Sonia Vanier A. Shabou VLM 36 0 0 11 Apr 2025
Particle Hit Clustering and Identification Using Point Set Transformers in Liquid Argon Time Projection Chambers Edgar E. Robles A. Yankelevich Wenjie Wu J. Bian Pierre Baldi 33 0 0 11 Apr 2025
ModernBERT or DeBERTaV3? Examining Architecture and Data Influence on Transformer Encoder Models Performance Wissam Antoun B. Sagot Djamé Seddah MQ 37 0 0 11 Apr 2025
Apt-Serve: Adaptive Request Scheduling on Hybrid Cache for Scalable LLM Inference Serving Shihong Gao Xuzhi Zhang Yanyan Shen Lei Chen 22 1 0 10 Apr 2025
Token Level Routing Inference System for Edge Devices Jianshu She Wenhao Zheng Zhengzhong Liu Hongyi Wang Eric P. Xing Huaxiu Yao Qirong Ho 36 0 0 10 Apr 2025
Kimi-VL Technical Report Kimi Team Angang Du B. Yin Bowei Xing Bowen Qu ... Zhiqi Huang Zihao Huang Zijia Zhao Zhengzhang Chen Zongyu Lin MLLM VLM MoE 204 2 0 10 Apr 2025
Distilling Textual Priors from LLM to Efficient Image Fusion Ran Zhang Xuanhua He Ke Cao L. Liu Li Zhang Man Zhou Jie Zhang 24 0 0 09 Apr 2025
CAT: Circular-Convolutional Attention for Sub-Quadratic Transformers Yoshihiro Yamada ViT 30 0 0 09 Apr 2025
CHIME: A Compressive Framework for Holistic Interest Modeling Yong Bai Rui Xiang Kaiyuan Li Yongxiang Tang Yanhua Cheng Xialong Liu Peng Jiang Kun Gai 29 0 0 09 Apr 2025
Crafting Query-Aware Selective Attention for Single Image Super-Resolution Junyoung Kim Youngrok Kim Siyeol Jung Donghyun Min 37 0 0 09 Apr 2025
Accelerating LLM Inference Throughput via Asynchronous KV Cache Prefetching Yanhao Dong Yubo Miao Weinan Li Xiao Zheng Chao Wang Feng Lyu 24 0 0 08 Apr 2025
TAGC: Optimizing Gradient Communication in Distributed Transformer Training Igor Polyakov Alexey Dukhanov Egor Spirin 41 0 0 08 Apr 2025
PaMi-VDPO: Mitigating Video Hallucinations by Prompt-Aware Multi-Instance Video Preference Learning Xinpeng Ding Kaipeng Zhang Jinahua Han Lanqing Hong Hang Xu Xiaomeng Li MLLM VLM 178 0 0 08 Apr 2025
High-Resource Translation:Turning Abundance into Accessibility Abhiram Reddy Yanampally 19 0 0 08 Apr 2025
SPIRe: Boosting LLM Inference Throughput with Speculative Decoding Sanjit Neelam Daniel Heinlein Vaclav Cvicek Akshay Mishra Reiner Pope LRM 40 0 0 08 Apr 2025
AccLLM: Accelerating Long-Context LLM Inference Via Algorithm-Hardware Co-Design Yanbiao Liang Huihong Shi Haikuo Shao Zhongfeng Wang 33 0 0 07 Apr 2025
One-Minute Video Generation with Test-Time Training Karan Dalal Daniel Koceja Gashon Hussein Jiarui Xu Yue Zhao ... Tatsunori Hashimoto Sanmi Koyejo Yejin Choi Yu Sun Xiaolong Wang ViT 91 3 0 07 Apr 2025
Reasoning on Multiple Needles In A Haystack Yidong Wang LRM 31 0 0 05 Apr 2025
PipeDec: Low-Latency Pipeline-based Inference with Dynamic Speculative Decoding towards Large-scale Models Haofei Yin Mengbai Xiao Rouzhou Lu Xiao Zhang Dongxiao Yu Guanghui Zhang AI4CE 24 0 0 05 Apr 2025
Window Token Concatenation for Efficient Visual Large Language Models Yifan Li Wentao Bao Botao Ye Zhen Tan Tianlong Chen Huan Liu Yu Kong VLM 44 0 0 05 Apr 2025
Clinical ModernBERT: An efficient and long context encoder for biomedical text Simon A. Lee Anthony Wu Jeffrey N. Chiang MedIm 46 3 0 04 Apr 2025
Classic Video Denoising in a Machine Learning World: Robust, Fast, and Controllable Xin Jin Simon Niklaus Zhoutong Zhang Zhihao Xia Chunle Guo Yuting Yang J. Chen Chongyi Li VGen 44 0 0 04 Apr 2025
HeterMoE: Efficient Training of Mixture-of-Experts Models on Heterogeneous GPUs Yongji Wu Xueshen Liu Shuowei Jin Ceyu Xu Feng Qian Ziming Mao Matthew Lentz Danyang Zhuo Ion Stoica MoMe MoE 61 0 0 04 Apr 2025
A Framework for Robust Cognitive Evaluation of LLMs Karin de Langis J. Park Bin Hu Khanh Chi Le Andreas Schramm Michael C. Mensink Andrew Elfenbein Dongyeop Kang 32 0 0 03 Apr 2025
Large (Vision) Language Models are Unsupervised In-Context Learners Artyom Gadetsky Andrei Atanov Yulun Jiang Zhitong Gao Ghazal Hosseini Mighan Amir Zamir Maria Brbić VLM MLLM LRM 69 0 0 03 Apr 2025
FT-Transformer: Resilient and Reliable Transformer with End-to-End Fault Tolerant Attention Huangliang Dai Shixun Wu Hairui Zhao Jiajun Huang Zizhe Jian Yue Zhu Haiyang Hu Zizhong Chen 49 0 0 03 Apr 2025
Urban Computing in the Era of Large Language Models Zhonghang Li Lianghao Xia Xubin Ren J. Tang Tianyi Chen Yong-mei Xu Chenyu Huang 83 0 0 02 Apr 2025
UniViTAR: Unified Vision Transformer with Native Resolution Limeng Qiao Yiyang Gan Bairui Wang Jie Qin Shuang Xu Siqi Yang Lin Ma 57 0 0 02 Apr 2025
Scaling Test-Time Inference with Policy-Optimized, Dynamic Retrieval-Augmented Generation via KV Caching and Decoding Sakhinana Sagar Srinivas Venkataramana Runkana OffRL 47 1 0 02 Apr 2025
CASCADE Your Datasets for Cross-Mode Knowledge Retrieval of Language Models Runlong Zhou Yi Zhang RALM 56 0 0 02 Apr 2025
ParallelFlow: Parallelizing Linear Transformers via Flow Discretization Nicola Muca Cirone C. Salvi 52 1 0 01 Apr 2025
TransMamba: Flexibly Switching between Transformer and Mamba Yixing Li Ruobing Xie Zhen Yang Xingchen Sun Shuaipeng Li ... Zhanhui Kang Yu Cheng C. Xu Di Wang Jie Jiang Mamba 65 1 0 31 Mar 2025
Benchmarking Systematic Relational Reasoning with Large Language and Reasoning Models Irtaza Khalid Amir Masoud Nourollah Steven Schockaert LRM 40 0 0 30 Mar 2025
FastVAR: Linear Visual Autoregressive Modeling via Cached Token Pruning Hang Guo Yawei Li Taolin Zhang J. Wang Tao Dai Shu-Tao Xia Luca Benini 72 1 0 30 Mar 2025
Cocktail: Chunk-Adaptive Mixed-Precision Quantization for Long-Context LLM Inference Wei Tao Bin Zhang Xiaoyang Qu Jiguang Wan Jianzong Wang 39 1 0 30 Mar 2025