v1v2v3 (latest)

Marconi: Prefix Caching for the Era of Hybrid LLMs

28 November 2024

Papers citing "Marconi: Prefix Caching for the Era of Hybrid LLMs"

38 / 38 papers shown

Title
Taming the Titans: A Survey of Efficient LLM Inference Serving Ranran Zhen Junlin Li Yixin Ji Zhiyong Yang Tong Liu Qingrong Xia Xinyu Duan Zehao Wang Baoxing Huai Hao Fei LLMAG 173 0 0 28 Apr 2025
From Human Memory to AI Memory: A Survey on Memory Mechanisms in the Era of LLMs Yaxiong Wu Sheng Liang Chen Zhang Yucheng Wang Yanzhe Zhang Huifeng Guo Ruiming Tang Yong Liu KELM 134 7 0 22 Apr 2025
MiniMax-01: Scaling Foundation Models with Lightning Attention MiniMax Aonian Li Bangwei Gong Bo Yang Bo Shen ... Zhan Qin Zhenhua Fan Zhihang Yu Z. L. Jiang Zijia Wu MoE 171 42 0 14 Jan 2025
Jamba-1.5: Hybrid Transformer-Mamba Models at Scale Jamba Team Barak Lenz Alan Arazi Amir Bergman Avshalom Manevich ... Yehoshua Cohen Yonatan Belinkov Y. Globerson Yuval Peleg Levy Y. Shoham 114 33 0 22 Aug 2024
FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision Jay Shah Ganesh Bikshandi Ying Zhang Vijay Thakkar Pradeep Ramani Tri Dao 147 157 0 11 Jul 2024
B'MOJO: Hybrid State Space Realizations of Foundation Models with Eidetic and Fading Memory Luca Zancato Arjun Seshadri Yonatan Dukler Aditya Golatkar Yantao Shen Benjamin Bowman Matthew Trager Alessandro Achille Stefano Soatto 77 10 0 08 Jul 2024
Learning to (Learn at Test Time): RNNs with Expressive Hidden States Yu Sun Xinhao Li Karan Dalal Jiarui Xu Arjun Vikram ... Xinlei Chen Xiaolong Wang Sanmi Koyejo Tatsunori Hashimoto Carlos Guestrin 145 113 0 05 Jul 2024
InfiniGen: Efficient Generative Inference of Large Language Models with Dynamic KV Cache Management Wonbeom Lee Jungi Lee Junghwan Seo Jaewoong Sim RALM 87 96 0 28 Jun 2024
An Empirical Study of Mamba-based Language Models R. Waleffe Wonmin Byeon Duncan Riach Brandon Norick V. Korthikanti ... Vartika Singh Jared Casper Jan Kautz Mohammad Shoeybi Bryan Catanzaro 125 78 0 12 Jun 2024
Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling Liliang Ren Yang Liu Yadong Lu Yelong Shen Chen Liang Weizhu Chen Mamba 182 69 0 11 Jun 2024
Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality Tri Dao Albert Gu Mamba 145 542 0 31 May 2024
Zamba: A Compact 7B SSM Hybrid Model Paolo Glorioso Quentin G. Anthony Yury Tokpanov James Whittington Jonathan Pilault Adam Ibrahim Beren Millidge 91 49 0 26 May 2024
LoongServe: Efficiently Serving Long-context Large Language Models with Elastic Sequence Parallelism Bingya Wu Shengyu Liu Yinmin Zhong Peng Sun Xuanzhe Liu Xin Jin RALM 106 63 0 15 Apr 2024
RecurrentGemma: Moving Past Transformers for Efficient Open Language Models Aleksandar Botev Soham De Samuel L. Smith Anushan Fernando George-Christian Muraru ... Koray Kavukcuoglu Demis Hassabis R. Hadsell Yee Whye Teh Nando de Frietas VLM RALM 77 29 0 11 Apr 2024
Jamba: A Hybrid Transformer-Mamba Language Model Opher Lieber Barak Lenz Hofit Bata Gal Cohen Jhonathan Osin ... Nir Ratner N. Rozen Erez Shwartz Mor Zusman Y. Shoham 122 227 0 28 Mar 2024
Cost-Efficient Large Language Model Serving for Multi-turn Conversations with CachedAttention Bin Gao Zhuomin He Puru Sharma Qingxuan Kang Djordje Jevdjic Junbo Deng Xingkun Yang Zhou Yu Pengfei Zuo 137 56 0 23 Mar 2024
Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve Amey Agrawal Nitin Kedia Ashish Panwar Jayashree Mohan Nipun Kwatra Bhargav S. Gulavani Alexey Tumanov Ramachandran Ramjee 102 187 0 04 Mar 2024
Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models Soham De Samuel L. Smith Anushan Fernando Aleksandar Botev George-Christian Muraru ... David Budden Yee Whye Teh Razvan Pascanu Nando de Freitas Çağlar Gülçehre Mamba 130 135 0 29 Feb 2024
Can Mamba Learn How to Learn? A Comparative Study on In-Context Learning Tasks Jongho Park Jaeseung Park Zheyang Xiong Nayoung Lee Jaewoong Cho Samet Oymak Kangwook Lee Dimitris Papailiopoulos 103 75 0 06 Feb 2024
InferCept: Efficient Intercept Support for Augmented Large Language Model Inference Reyna Abhyankar Zijian He Vikranth Srivatsa Hao Zhang Yiying Zhang RALM 80 15 0 02 Feb 2024
Gated Linear Attention Transformers with Hardware-Efficient Training Aaron Courville Bailin Wang Songlin Yang Yikang Shen Yoon Kim 124 180 0 11 Dec 2023
Stateful Large Language Model Serving with Pensieve Lingfan Yu Jinyang Li RALM KELM LLMAG 77 14 0 09 Dec 2023
LooGLE: Can Long-Context Language Models Understand Long Contexts? Jiaqi Li Mengmeng Wang Zilong Zheng Muhan Zhang ELM RALM 93 134 0 08 Nov 2023
Prompt Cache: Modular Attention Reuse for Low-Latency Inference In Gim Guojun Chen Seung-seob Lee Nikhil Sarda Anurag Khandelwal Lin Zhong 121 88 0 07 Nov 2023
SWE-bench: Can Language Models Resolve Real-World GitHub Issues? Carlos E. Jimenez John Yang Alexander Wettig Shunyu Yao Kexin Pei Ofir Press Karthik Narasimhan ELM 141 647 0 10 Oct 2023
DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines Omar Khattab Arnav Singhvi Paridhi Maheshwari Zhiyuan Zhang Keshav Santhanam ... Thomas T. Joshi Hanna Moazam Heather Miller Matei A. Zaharia Christopher Potts RALM 106 283 0 05 Oct 2023
LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Dataset Lianmin Zheng Wei-Lin Chiang Ying Sheng Tianle Li Siyuan Zhuang ... Zi Lin Eric P. Xing Joseph E. Gonzalez Ion Stoica Haotong Zhang 126 221 0 21 Sep 2023
Efficient Memory Management for Large Language Model Serving with PagedAttention Woosuk Kwon Zhuohan Li Siyuan Zhuang Ying Sheng Lianmin Zheng Cody Hao Yu Joseph E. Gonzalez Haotong Zhang Ion Stoica VLM 206 2,338 0 12 Sep 2023
FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning Tri Dao LRM 122 1,336 0 17 Jul 2023
Retentive Network: A Successor to Transformer for Large Language Models Yutao Sun Li Dong Shaohan Huang Shuming Ma Yuqing Xia Jilong Xue Jianyong Wang Furu Wei LRM 198 348 0 17 Jul 2023
RWKV: Reinventing RNNs for the Transformer Era Bo Peng Eric Alcaide Quentin G. Anthony Alon Albalak Samuel Arcadinho ... Qihang Zhao P. Zhou Qinghua Zhou Jian Zhu Rui-Jie Zhu 240 614 0 22 May 2023
ReAct: Synergizing Reasoning and Acting in Language Models Shunyu Yao Jeffrey Zhao Dian Yu Nan Du Izhak Shafran Karthik Narasimhan Yuan Cao LLMAG ReLM LRM 473 2,998 0 06 Oct 2022
FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness Tri Dao Daniel Y. Fu Stefano Ermon Atri Rudra Christopher Ré VLM 397 2,297 0 27 May 2022
Self-Consistency Improves Chain of Thought Reasoning in Language Models Xuezhi Wang Jason W. Wei Dale Schuurmans Quoc Le Ed H. Chi Sharan Narang Aakanksha Chowdhery Denny Zhou ReLM BDL LRM AI4CE 592 3,757 0 21 Mar 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 1.0K 9,796 0 28 Jan 2022
Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing Pengfei Liu Weizhe Yuan Jinlan Fu Zhengbao Jiang Hiroaki Hayashi Graham Neubig VLM SyDa 283 4,044 0 28 Jul 2021
Measuring Massive Multitask Language Understanding Dan Hendrycks Collin Burns Steven Basart Andy Zou Mantas Mazeika Basel Alomair Jacob Steinhardt ELM RALM 212 4,582 0 07 Sep 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 1.1K 42,651 0 28 May 2020