Long-Context Inference with Retrieval-Augmented Speculative Decoding

Long-Context Inference with Retrieval-Augmented Speculative Decoding

27 February 2025

Papers citing "Long-Context Inference with Retrieval-Augmented Speculative Decoding"

12 / 12 papers shown

Title
Scaling Test-Time Inference with Policy-Optimized, Dynamic Retrieval-Augmented Generation via KV Caching and Decoding Sakhinana Sagar Srinivas Akash Das Shivam Gupta Venkataramana Runkana OffRL 80 1 0 02 Apr 2025
PromptDistill: Query-based Selective Token Retention in Intermediate Layers for Efficient Large Language Model Inference Weisheng Jin Maojia Song Tej Deep Pala Yew Ken Chia Amir Zadeh Chuan Li Soujanya Poria VLM 67 0 0 30 Mar 2025
GPU-Accelerated Motion Planning of an Underactuated Forestry Crane in Cluttered Environments M. Vu Gerald Ebmer Alexander Watcher Marc-Philip Ecker Giang Nguyen Tobias Glueck 90 3 0 18 Mar 2025
Long Context vs. RAG for LLMs: An Evaluation and Revisits Xinze Li Yixin Cao Yubo Ma Aixin Sun RALM 41 6 0 27 Dec 2024
Accelerating Inference of Networks in the Frequency Domain Chenqiu Zhao Guanfang Dong Anup Basu 75 13 0 06 Oct 2024
ZipCache: Accurate and Efficient KV Cache Quantization with Salient Token Identification Yefei He Luoming Zhang Weijia Wu Jing Liu Hong Zhou Bohan Zhuang MQ 89 31 0 23 May 2024
TriForce: Lossless Acceleration of Long Sequence Generation with Hierarchical Speculative Decoding Hanshi Sun Zhuoming Chen Xinyu Yang Yuandong Tian Beidi Chen 72 55 0 18 Apr 2024
GEAR: An Efficient KV Cache Compression Recipe for Near-Lossless Generative Inference of LLM Hao Kang Qingru Zhang Souvik Kundu Geonhwa Jeong Zaoxing Liu Tushar Krishna Tuo Zhao MQ 122 91 0 08 Mar 2024
H $_2$ O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models Zhenyu Zhang Ying Sheng Dinesh Manocha Tianlong Chen Lianmin Zheng ... Yuandong Tian Christopher Ré Clark W. Barrett Zhangyang Wang Beidi Chen VLM 108 275 0 24 Jun 2023
Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena Lianmin Zheng Wei-Lin Chiang Ying Sheng Siyuan Zhuang Zhanghao Wu ... Dacheng Li Eric Xing Haotong Zhang Joseph E. Gonzalez Ion Stoica ALM OSLM ELM 236 4,186 0 09 Jun 2023
Fast Inference from Transformers via Speculative Decoding Yaniv Leviathan Matan Kalman Yossi Matias LRM 81 663 0 30 Nov 2022
Contrastive Decoding: Open-ended Text Generation as Optimization Xiang Lisa Li Ari Holtzman Daniel Fried Percy Liang Jason Eisner Tatsunori Hashimoto Luke Zettlemoyer M. Lewis 70 348 0 27 Oct 2022