Towards Optimal Multi-draft Speculative Decoding

Towards Optimal Multi-draft Speculative Decoding

26 February 2025

Vignesh Viswanathan

ArXiv (abs)PDF HTML

Papers citing "Towards Optimal Multi-draft Speculative Decoding"

15 / 15 papers shown

Title
Beyond Decoder-only: Large Language Models Can be Good Encoders for Machine Translation Yingfeng Luo Tong Zheng Yongyu Mu Yangqiu Song Qinghong Zhang ... Ziqiang Xu Peinan Feng Xiaoqian Liu Tong Xiao Jingbo Zhu AI4CE 495 3 0 09 Mar 2025
Speculative Decoding and Beyond: An In-Depth Survey of Techniques Y. Hu Zining Liu Zhenyuan Dong Tianfan Peng Bradley McDanel Shanghang Zhang 160 0 0 27 Feb 2025
SpecHub: Provable Acceleration to Multi-Draft Speculative Decoding Ryan Sun Tianyi Zhou Xun Chen Lichao Sun 59 5 0 08 Nov 2024
TriForce: Lossless Acceleration of Long Sequence Generation with Hierarchical Speculative Decoding Hanshi Sun Zhuoming Chen Xinyu Yang Yuandong Tian Beidi Chen 99 65 0 18 Apr 2024
Recursive Speculative Decoding: Accelerating LLM Inference via Sampling Without Replacement Wonseok Jeon Mukul Gagrani Raghavv Goel Junyoung Park Mingu Lee Christopher Lott LRM 66 19 0 21 Feb 2024
EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty Yuhui Li Fangyun Wei Chao Zhang Hongyang R. Zhang 123 165 0 26 Jan 2024
Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads Tianle Cai Yuhong Li Zhengyang Geng Hongwu Peng Jason D. Lee De-huai Chen Tri Dao 149 313 0 19 Jan 2024
PaSS: Parallel Speculative Sampling Giovanni Monea Armand Joulin Edouard Grave MoE 70 38 0 22 Nov 2023
Accelerating LLM Inference with Staged Speculative Decoding Benjamin Spector Christal Re 69 112 0 08 Aug 2023
Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena Lianmin Zheng Wei-Lin Chiang Ying Sheng Siyuan Zhuang Zhanghao Wu ... Dacheng Li Eric Xing Haotong Zhang Joseph E. Gonzalez Ion Stoica ALM OSLM ELM 421 4,422 0 09 Jun 2023
Inference with Reference: Lossless Acceleration of Large Language Models Nan Yang Tao Ge Liang Wang Binxing Jiao Daxin Jiang Linjun Yang Rangan Majumder Furu Wei 47 63 0 10 Apr 2023
Fast Inference from Transformers via Speculative Decoding Yaniv Leviathan Matan Kalman Yossi Matias LRM 147 733 0 30 Nov 2022
Evaluating Large Language Models Trained on Code Mark Chen Jerry Tworek Heewoo Jun Qiming Yuan Henrique Pondé ... Bob McGrew Dario Amodei Sam McCandlish Ilya Sutskever Wojciech Zaremba ELM ALM 236 5,647 0 07 Jul 2021
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 874 42,379 0 28 May 2020
Teaching Machines to Read and Comprehend Karl Moritz Hermann Tomás Kociský Edward Grefenstette L. Espeholt W. Kay Mustafa Suleyman Phil Blunsom 349 3,552 0 10 Jun 2015

We use cookies and other tracking technologies to improve your browsing experience on our website, to show you personalized content and targeted ads, to analyze our website traffic, and to understand where our visitors are coming from. See our policy.