FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning

17 July 2023

Papers citing "FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning"

50 / 329 papers shown

Title
Task-level Distributionally Robust Optimization for Large Language Model-based Dense Retrieval Guangyuan Ma Yongliang Ma Xing Wu Zhenpeng Su Ming Zhou Songlin Hu OOD 190 3 0 20 Aug 2024
Turning Trash into Treasure: Accelerating Inference of Large Language Models with Token Recycling Xianzhen Luo Yixuan Wang Qingfu Zhu Zhiming Zhang Xuanyu Zhang Qing Yang Dongliang Xu 97 9 0 16 Aug 2024
Lumina-mGPT: Illuminate Flexible Photorealistic Text-to-Image Generation with Multimodal Generative Pretraining Dongyang Liu Shitian Zhao Le Zhuo Weifeng Lin Ping Luo Xinyue Li Qi Qin Yu Qiao Hongsheng Li Peng Gao MLLM 168 59 0 05 Aug 2024
Bailing-TTS: Chinese Dialectal Speech Synthesis Towards Human-like Spontaneous Representation Xinhan Di Jiahao Lu Yunming Liang Junjie Zheng Yihua Wang Chaofan Ding ALM 91 1 0 01 Aug 2024
ThinK: Thinner Key Cache by Query-Driven Pruning Yuhui Xu Zhanming Jie Hanze Dong Lei Wang Xudong Lu Aojun Zhou Amrita Saha Caiming Xiong Doyen Sahoo 178 23 0 30 Jul 2024
Small Molecule Optimization with Large Language Models Philipp Guevorguian Menua Bedrosian Tigran Fahradyan Gayane Chilingaryan Hrant Khachatrian Armen Aghajanyan 76 1 0 26 Jul 2024
Inverted Activations Georgii Sergeevich Novikov Ivan Oseledets 33 0 0 22 Jul 2024
$$\infty$-Brush: Controllable Large Image Synthesis with Diffusion Models in Infinite Dimensions$ $\infty$ -Brush: Controllable Large Image Synthesis with Diffusion Models in Infinite Dimensions Minh-Quan Le Alexandros Graikos Srikar Yellapragada Rajarsi R. Gupta Joel H. Saltz Dimitris Samaras 96 10 0 20 Jul 2024
Human-like Episodic Memory for Infinite Context LLMs Zafeirios Fountas Martin A Benfeghoul Adnan Oomerjee Fenia Christopoulou Gerasimos Lampouras Haitham Bou-Ammar Jun Wang 88 21 0 12 Jul 2024
Inference Optimization of Foundation Models on AI Accelerators Youngsuk Park Kailash Budhathoki Liangfu Chen Jonas M. Kübler Jiaji Huang Matthäus Kleindessner Jun Huan Volkan Cevher Yida Wang George Karypis 117 5 0 12 Jul 2024
Mitigating Catastrophic Forgetting in Language Transfer via Model Merging Anton Alexandrov Veselin Raychev Mark Niklas Muller Ce Zhang Martin Vechev Kristina Toutanova MoMe CLL KELM 117 20 0 11 Jul 2024
How Well Can a Long Sequence Model Model Long Sequences? Comparing Architechtural Inductive Biases on Long-Context Abilities Jerry Huang 137 7 0 11 Jul 2024
Convolutional vs Large Language Models for Software Log Classification in Edge-Deployable Cellular Network Testing A. Ihalage Sayed M. Taheri Faris Muhammad Hamed Al-Raweshidy 78 2 0 04 Jul 2024
Let the Code LLM Edit Itself When You Edit the Code Zhenyu He Jun Zhang Shengjie Luo Jingjing Xu Zongzhang Zhang Di He KELM 105 1 0 03 Jul 2024
Efficient Sparse Attention needs Adaptive Token Release Chaoran Zhang Lixin Zou Dan Luo Min Tang Xiangyang Luo Zihao Li Chenliang Li 108 5 0 02 Jul 2024
Badllama 3: removing safety finetuning from Llama 3 in minutes Dmitrii Volkov 49 5 0 01 Jul 2024
ColPali: Efficient Document Retrieval with Vision Language Models Manuel Faysse Hugues Sibille Tony Wu Bilel Omrani Gautier Viaud C´eline Hudelot Pierre Colombo VLM 377 29 0 27 Jun 2024
Scaling Laws for Linear Complexity Language Models Xuyang Shen Dong Li Ruitao Leng Zhen Qin Weigao Sun Yiran Zhong LRM 83 8 0 24 Jun 2024
RouteFinder: Towards Foundation Models for Vehicle Routing Problems Federico Berto Chuanbo Hua Nayeli Gast Zepeda André Hottung N. Wouda Leon Lan Kevin Tierney J. Park Jinkyoo Park 166 16 0 21 Jun 2024
DeciMamba: Exploring the Length Extrapolation Potential of Mamba Assaf Ben-Kish Itamar Zimerman Shady Abu Hussein Nadav Cohen Amir Globerson Lior Wolf Raja Giryes Mamba 206 20 0 20 Jun 2024
XLand-100B: A Large-Scale Multi-Task Dataset for In-Context Reinforcement Learning Alexander Nikulin Ilya Zisman Alexey Zemtsov Viacheslav Sinii 203 7 0 13 Jun 2024
Needle In A Multimodal Haystack Weiyun Wang Shuibo Zhang Yiming Ren Yuchen Duan Tiantong Li ... Ping Luo Yu Qiao Jifeng Dai Wenqi Shao Wenhai Wang VLM 114 24 0 11 Jun 2024
Teaching Language Models to Self-Improve by Learning from Language Feedback Chi Hu Yimin Hu Hang Cao Tong Xiao Jingbo Zhu LRM VLM 79 5 0 11 Jun 2024
Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling Liliang Ren Yang Liu Yadong Lu Yelong Shen Chen Liang Weizhu Chen Mamba 182 69 0 11 Jun 2024
3D-GRAND: A Million-Scale Dataset for 3D-LLMs with Better Grounding and Less Hallucination Jianing Yang Xuweiyi Chen Nikhil Madaan Madhavan Iyengar Shengyi Qian David Fouhey Joyce Chai 3DV 160 16 0 07 Jun 2024
S3D: A Simple and Cost-Effective Self-Speculative Decoding Scheme for Low-Memory GPUs Wei Zhong Manasa Bharadwaj 115 7 0 30 May 2024
ReflectionCoder: Learning from Reflection Sequence for Enhanced One-off Code Generation Houxing Ren Mingjie Zhan Zhongyuan Wu Aojun Zhou Junting Pan Hongsheng Li SyDa 125 7 0 27 May 2024
ZipCache: Accurate and Efficient KV Cache Quantization with Salient Token Identification Yefei He Luoming Zhang Weijia Wu Jing Liu Hong Zhou Bohan Zhuang MQ 118 34 0 23 May 2024
OpenRLHF: An Easy-to-use, Scalable and High-performance RLHF Framework Jian Hu Xibin Wu Weixun Wang OpenLLMAI Team Dehao Zhang ... Weikai Fang Xianyu Yu Cao Haotian Xu Yiming Liu VLM AI4CE 136 130 0 20 May 2024
Lean Attention: Hardware-Aware Scalable Attention Mechanism for the Decode-Phase of Transformers Rya Sanovar Srikant Bharadwaj Renée St. Amant Victor Rühle Saravan Rajmohan 164 7 0 17 May 2024
DEPTH: Discourse Education through Pre-Training Hierarchically Zachary Bamberger Ofek Glick Chaim Baskin Yonatan Belinkov 126 0 0 13 May 2024
vAttention: Dynamic Memory Management for Serving LLMs without PagedAttention Ramya Prabhu Ajay Nayak Jayashree Mohan Ramachandran Ramjee Ashish Panwar VLM 164 29 0 07 May 2024
Folded Context Condensation in Path Integral Formalism for Infinite Context Transformers Won-Gi Paeng Daesuk Kwon Kyungwon Jeong Honggyo Suh 164 0 0 07 May 2024
GS-LRM: Large Reconstruction Model for 3D Gaussian Splatting Kai Zhang Sai Bi Hao Tan Yuanbo Xiangli Nanxuan Zhao Kalyan Sunkavalli Zexiang Xu 3DGS 121 149 0 30 Apr 2024
Do Vision & Language Decoders use Images and Text equally? How Self-consistent are their Explanations? Letitia Parcalabescu Anette Frank MLLM CoGe VLM 164 6 0 29 Apr 2024
Continual Pre-Training for Cross-Lingual LLM Adaptation: Enhancing Japanese Language Capabilities Kazuki Fujii Taishi Nakamura Mengsay Loem Hiroki Iida Masanari Ohi Kakeru Hattori Hirai Shota Sakae Mizuki Rio Yokota Naoaki Okazaki CLL 131 73 0 27 Apr 2024
CORM: Cache Optimization with Recent Message for Large Language Model Inference Jincheng Dai Zhuowei Huang Haiyun Jiang Chen Chen Deng Cai Wei Bi Shuming Shi 109 3 0 24 Apr 2024
SnapKV: LLM Knows What You are Looking for Before Generation Yuhong Li Yingbing Huang Bowen Yang Bharat Venkitesh Acyr Locatelli Hanchen Ye Tianle Cai Patrick Lewis Deming Chen VLM 143 210 0 22 Apr 2024
Learn Your Reference Model for Real Good Alignment Alexey Gorbatovski Boris Shaposhnikov Alexey Malakhov Nikita Surnachev Yaroslav Aksenov Ian Maksimov Nikita Balagansky Daniil Gavrilov OffRL 129 35 0 15 Apr 2024
MemFlow: Optical Flow Estimation and Prediction with Memory Qiaole Dong Yanwei Fu 109 21 0 07 Apr 2024
Binary Classifier Optimization for Large Language Model Alignment Seungjae Jung Gunsoo Han D. W. Nam Kyoung-Woon On 82 25 0 06 Apr 2024
Linear Attention Sequence Parallelism Weigao Sun Zhen Qin Dong Li Xuyang Shen Yu Qiao Yiran Zhong 150 2 0 03 Apr 2024
PlainMamba: Improving Non-Hierarchical Mamba in Visual Recognition Chenhongyi Yang Zehui Chen Miguel Espinosa Linus Ericsson Zhenyu Wang Jiaming Liu Elliot J. Crowley Mamba 119 99 0 26 Mar 2024
Yi: Open Foundation Models by 01.AI 01. AI Alex Young 01.AI Alex Young Bei Chen Chao Li ... Yue Wang Yuxuan Cai Zhenyu Gu Zhiyuan Liu Zonghong Dai OSLM LRM 317 576 0 07 Mar 2024
Found in the Middle: How Language Models Use Long Contexts Better via Plug-and-Play Positional Encoding Zhenyu Zhang Runjin Chen Shiwei Liu Zhewei Yao Olatunji Ruwase Beidi Chen Xiaoxia Wu Zhangyang Wang 95 36 0 05 Mar 2024
RelayAttention for Efficient Large Language Model Serving with Long System Prompts Lei Zhu Xinjiang Wang Wayne Zhang Rynson W. H. Lau 88 8 0 22 Feb 2024
Analysing The Impact of Sequence Composition on Language Model Pre-Training Yu Zhao Yuanbin Qu Konrad Staniszewski Szymon Tworkowski Wei Liu Piotr Milo's Yuxiang Wu Pasquale Minervini 83 15 0 21 Feb 2024
CAMELoT: Towards Large Language Models with Training-Free Consolidated Associative Memory Zexue He Leonid Karlinsky Donghyun Kim Julian McAuley Dmitry Krotov Rogerio Feris KELM RALM 86 11 0 21 Feb 2024
Sequoia: Scalable, Robust, and Hardware-aware Speculative Decoding Zhuoming Chen Avner May Ruslan Svirschevski Yuhsun Huang Max Ryabinin Zhihao Jia Beidi Chen 104 52 0 19 Feb 2024
LEIA: Facilitating Cross-lingual Knowledge Transfer in Language Models with Entity-based Data Augmentation Ikuya Yamada Ryokan Ri KELM 78 1 0 18 Feb 2024