v1v2 (latest)

FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness

27 May 2022

Papers citing "FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness"

50 / 1,508 papers shown

Title
TopV: Compatible Token Pruning with Inference Time Optimization for Fast and Low-Memory Multimodal Vision Language Model Cheng Yang Yang Sui Jinqi Xiao Lingyi Huang Yu Gong ... Jinghua Yan Y. Bai P. Sadayappan Helen Zhou Bo Yuan VLM 151 2 0 24 Mar 2025
Mist: Efficient Distributed Training of Large Language Models via Memory-Parallelism Co-Optimization Zhanda Zhu Christina Giannoula Muralidhar Andoorveedu Qidong Su Karttikeya Mangalam Bojian Zheng Gennady Pekhimenko VLM MoE 94 0 0 24 Mar 2025
Oaken: Fast and Efficient LLM Serving with Online-Offline Hybrid KV Cache Quantization Minsu Kim Seongmin Hong RyeoWook Ko S. Choi Hunjong Lee Junsoo Kim Joo-Young Kim Jongse Park 122 0 0 24 Mar 2025
Your ViT is Secretly an Image Segmentation Model Tommie Kerssies Niccolò Cavagnero Alexander Hermans Narges Norouzi Giuseppe Averta Bastian Leibe Gijs Dubbelman Daan de Geus ViT VLM 123 5 0 24 Mar 2025
WLB-LLM: Workload-Balanced 4D Parallelism for Large Language Model Training Ziyi Wang Anna Cai Xinfeng Xie Zaifeng Pan Yue Guan ... Shikai Li Jianyu Huang Chris Cai Yuchen Hao Yufei Ding 99 3 0 23 Mar 2025
OmniScience: A Domain-Specialized LLM for Scientific Reasoning and Discovery Vignesh Prabhakar Md Amirul Islam Adam Atanas Yansen Wang J. N. Han ... Rucha Apte Robert Clark Kang Xu Zihan Wang Kai Liu LRM 226 5 0 22 Mar 2025
Strong Baseline: Multi-UAV Tracking via YOLOv12 with BoT-SORT-ReID Yu-Hsi Chen 118 0 0 21 Mar 2025
iFlame: Interleaving Full and Linear Attention for Efficient Mesh Generation Hanxiao Wang Biao Zhang Weize Quan Dong-ming Yan Peter Wonka 110 2 0 20 Mar 2025
FutureGen: LLM-RAG Approach to Generate the Future Work of Scientific Article Ibrahim Al Azher Miftahul Jannat Mokarrama Zhishuai Guo Sagnik Ray Choudhury Hamed Alhoori LLMAG 104 2 0 20 Mar 2025
ATTENTION2D: Communication Efficient Distributed Self-Attention Mechanism Venmugil Elango 106 0 0 20 Mar 2025
PSA-MIL: A Probabilistic Spatial Attention-Based Multiple Instance Learning for Whole Slide Image Classification Sharon Peled Y. Maruvka Moti Freiman 126 0 0 20 Mar 2025
Uni-3DAR: Unified 3D Generation and Understanding via Autoregression on Compressed Spatial Tokens Shuqi Lu Haowei Lin Lin Yao Zhifeng Gao Xiaohong Ji Weinan E Linfeng Zhang Guolin Ke 103 0 0 20 Mar 2025
Prada: Black-Box LLM Adaptation with Private Data on Resource-Constrained Devices Ziyi Wang Yexiao He Zheyu Shen Yu Li Guoheng Sun Myungjin Lee Ang Li 89 0 0 19 Mar 2025
EgoDTM: Towards 3D-Aware Egocentric Video-Language Pretraining Boshen Xu Yuting Mei Xinbi Liu Sipeng Zheng Qin Jin VLM MDE 105 0 0 19 Mar 2025
EmpathyAgent: Can Embodied Agents Conduct Empathetic Actions? Xinyan Chen Jiaxin Ge Hongming Dai Qiang Zhou Qiuxuan Feng Jingtong Hu Yun Wang Jiaming Liu Shanghang Zhang LM&Ro 97 0 0 19 Mar 2025
Benchmarking Large Language Models for Handwritten Text Recognition Giorgia Crosilla Lukas Klic Giovanni Colavizza 104 0 0 19 Mar 2025
Bolt3D: Generating 3D Scenes in Seconds Stanislaw Szymanowicz Jason Y. Zhang P. Srinivasan Ruiqi Gao Arthur Brussee Aleksander Holynski Ricardo Martín Brualla Jonathan T. Barron Philipp Henzler 151 7 0 18 Mar 2025
Fake Runs, Real Fixes -- Analyzing xPU Performance Through Simulation Ioannis Zarkadas Amanda Tomlinson Asaf Cidon Baris Kasikci Ofir Weisse 70 0 0 18 Mar 2025
Growing a Twig to Accelerate Large Vision-Language Models Zhenwei Shao Mingyang Wang Zhou Yu Wenwen Pan Yan Yang Tao Wei Hao Zhang Ning Mao Wei Chen Jun Yu VLM 90 2 0 18 Mar 2025
Tiled Flash Linear Attention: More Efficient Linear RNN and xLSTM Kernels M. Beck Korbinian Poppel Phillip Lippe Sepp Hochreiter 159 4 0 18 Mar 2025
State Space Model Meets Transformer: A New Paradigm for 3D Object Detection Chuxin Wang Wenfei Yang Xiang Liu Tianzhu Zhang 137 1 0 18 Mar 2025
SplatVoxel: History-Aware Novel View Streaming without Temporal Training Yiming Wang Lucy Chai Xuan Luo Michael Niemeyer Manuel Lagunas Stephen Lombardi Siyu Tang Tiancheng Sun 3DGS 144 0 0 18 Mar 2025
Theoretical Foundation of Flow-Based Time Series Generation: Provable Approximation, Generalization, and Efficiency Jiangxuan Long Zhao Song Chiwun Yang AI4TS 480 0 0 18 Mar 2025
Identifying and Mitigating Position Bias of Multi-image Vision-Language Models Xinyu Tian Shu Zou Zhaoyuan Yang Jing Zhang 108 3 0 18 Mar 2025
Lifting the Veil on Visual Information Flow in MLLMs: Unlocking Pathways to Faster Inference Hao Yin Guangzong Si Zilei Wang 132 1 0 17 Mar 2025
MaTVLM: Hybrid Mamba-Transformer for Efficient Vision-Language Modeling Yingyue Li Bencheng Liao Wenyu Liu Xinggang Wang Mamba 122 0 0 17 Mar 2025
Towards Hierarchical Multi-Step Reward Models for Enhanced Reasoning in Large Language Models Teng Wang Zhangyi Jiang Zhenqi He Wenhan Yang Yanan Zheng Zeyu Li Zifan He Shenyang Tong Hailei Gong LRM 167 2 0 16 Mar 2025
Hydra-NeXt: Robust Closed-Loop Driving with Open-Loop Training Zhenxin Li Shihao Wang Shiyi Lan Zhiding Yu Zuxuan Wu Jose M. Alvarez 107 4 0 15 Mar 2025
Changing Base Without Losing Pace: A GPU-Efficient Alternative to MatMul in DNNs Nir Ailon Akhiad Bercovich Omri Weinstein 133 0 0 15 Mar 2025
Similarity-Aware Token Pruning: Your VLM but Faster Ahmadreza Jeddi Negin Baghbanzadeh Elham Dolatabadi Babak Taati 3DV VLM 114 2 0 14 Mar 2025
TigerLLM -- A Family of Bangla Large Language Models Nishat Raihan Marcos Zampieri 102 0 0 14 Mar 2025
FastVID: Dynamic Density Pruning for Fast Video Large Language Models Leqi Shen Guoqiang Gong Tao He Yifeng Zhang Pengzhang Liu Sicheng Zhao Guiguang Ding VLM 109 2 0 14 Mar 2025
TransiT: Transient Transformer for Non-line-of-sight Videography Ruiqian Li Siyuan Shen Suan Xia Zehao Wang Xingyue Peng Chengxuan Song Yingsheng Zhu Tao Wu Shiying Li Jingyi Yu 92 0 0 14 Mar 2025
FasterCache: Training-Free Video Diffusion Model Acceleration with High Quality Zhengyao Lv Chenyang Si Junhao Song Zhenyu Yang Ping Luo Ziwei Liu Kwan-Yee K. Wong VGen DiffM 166 18 0 13 Mar 2025
Speedy MASt3R Jingxing Li Yongjae Lee Abhay Kumar Yadav Cheng-Fang Peng Rama Chellappa Deliang Fan 3DGS 123 0 0 13 Mar 2025
EEdit: Rethinking the Spatial and Temporal Redundancy for Efficient Image Editing Zexuan Yan Yue Ma Chang Zou Wenteng Chen Qifeng Chen Linfeng Zhang 175 4 0 13 Mar 2025
Take Off the Training Wheels Progressive In-Context Learning for Effective Alignment Zhenyu Liu Dongfang Li Xinshuo Hu X. Zhao Yibin Chen Baotian Hu Min Zhang 113 1 0 13 Mar 2025
Beyond Atoms: Enhancing Molecular Pretrained Representations with 3D Space Modeling Shuqi Lu Xiaohong Ji Bohang Zhang Lin Yao Siyuan Liu Zhifeng Gao Linfeng Zhang Guolin Ke AI4CE 128 1 0 13 Mar 2025
Samoyeds: Accelerating MoE Models with Structured Sparsity Leveraging Sparse Tensor Cores Chenpeng Wu Qiqi Gu Heng Shi Jianguo Yao Haibing Guan MoE 78 0 0 13 Mar 2025
V2Edit: Versatile Video Diffusion Editor for Videos and 3D Scenes Yanming Zhang Jun-Kun Chen Jipeng Lyu Yu-Xiong Wang DiffM VGen 114 0 0 13 Mar 2025
Autoregressive Image Generation with Randomized Parallel Decoding Haopeng Li Jinyue Yang Guoqi Li Huan Wang 100 1 0 13 Mar 2025
Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models Marianne Arriola Aaron Gokaslan Justin T Chiu Zhihan Yang Zhixuan Qi Jiaqi Han Subham Sekhar Sahoo Volodymyr Kuleshov DiffM 269 25 0 12 Mar 2025
Exo2Ego: Exocentric Knowledge Guided MLLM for Egocentric Video Understanding Haoyu Zhang Qiaohui Chu Meng Liu Yunxiao Wang Bin Wen Fan Yang Yan Li Di Zhang Yaowei Wang Liqiang Nie EgoV 112 5 0 12 Mar 2025
AI-native Memory 2.0: Second Me Jiale Wei Xiang Ying Tao Gao Fangyi Bao Felix Tao Jingbo Shang 132 1 0 11 Mar 2025
Representing 3D Shapes With 64 Latent Vectors for 3D Diffusion Models I. Cho Youngbeom Yoo Subin Jeon Seon Joo Kim DiffM 127 0 0 11 Mar 2025
RigoChat 2: an adapted language model to Spanish using a bounded dataset and reduced hardware Gonzalo Santamaría Gómez Guillem García Subies Pablo Gutiérrez Ruiz Mario González Valero Natàlia Fuertes ... Nuria Aldama García David Betancur Sánchez Kateryna Sushkova Marta Guerrero Nieto Á. Jiménez 122 0 0 11 Mar 2025
Mind the Memory Gap: Unveiling GPU Bottlenecks in Large-Batch LLM Inference Pol G. Recasens Ferran Agullo Yue Zhu Chen Wang Eun Kyung Lee Olivier Tardieu Jordi Torres Josep Ll. Berral 92 1 0 11 Mar 2025
LiSu: A Dataset and Method for LiDAR Surface Normal Estimation Dušan Malić Christian Fruhwirth-Reisinger Samuel Schulter Horst Possegger 3DV 111 0 0 11 Mar 2025
Queueing, Predictions, and LLMs: Challenges and Open Problems Michael Mitzenmacher Rana Shahout AI4TS LRM 103 1 0 10 Mar 2025
When Large Vision-Language Model Meets Large Remote Sensing Imagery: Coarse-to-Fine Text-Guided Token Pruning Junwei Luo Yingying Zhang Xiaoyu Yang Kang Wu Qi Zhu Lei Liang Jingdong Chen Yansheng Li 148 2 0 10 Mar 2025