Cascade Speculative Drafting for Even Faster LLM Inference

Cascade Speculative Drafting for Even Faster LLM Inference

18 December 2023

Kevin Chen-Chuan Chang

Papers citing "Cascade Speculative Drafting for Even Faster LLM Inference"

13 / 13 papers shown

Title
SpecRouter: Adaptive Routing for Multi-Level Speculative Decoding in Large Language Models Hang Wu Jianian Zhu Yong Li Haojie Wang Biao Hou Jidong Zhai 45 0 0 12 May 2025
Accelerating Large Language Model Reasoning via Speculative Search Zhihai Wang Jie Wang Jilai Pan Xilin Xia Huiling Zhen M. Yuan Jianye Hao Feng Wu ReLM LRM 75 1 0 03 May 2025
Efficient Reasoning for LLMs through Speculative Chain-of-Thought Jikai Wang J. Li Lijun Wu Hao Fei LLMAG LRM 69 2 0 27 Apr 2025
GRIFFIN: Effective Token Alignment for Faster Speculative Decoding Shijing Hu Jingyang Li Xingyu Xie Zhihui Lu Kim-Chuan Toh Pan Zhou 48 0 0 16 Feb 2025
Falcon: Faster and Parallel Inference of Large Language Models through Enhanced Semi-Autoregressive Drafting and Custom-Designed Decoding Tree Xiangxiang Gao Weisheng Xie Yiwei Xiang Feng Ji 91 6 0 17 Dec 2024
Root Defence Strategies: Ensuring Safety of LLM at the Decoding Level Xinyi Zeng Yuying Shang Yutao Zhu Jingyuan Zhang Yu Tian AAML 211 2 0 09 Oct 2024
Efficient Inference for Large Language Model-based Generative Recommendation Xinyu Lin Chaoqun Yang Wenjie Wang Yongqi Li Cunxiao Du Fuli Feng See-Kiong Ng Tat-Seng Chua 70 4 0 07 Oct 2024
OPT-Tree: Speculative Decoding with Adaptive Draft Tree Structure Jikai Wang Yi Su Juntao Li Qingrong Xia Zi Ye Xinyu Duan Zhefeng Wang Min Zhang 46 14 0 25 Jun 2024
EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty Yuhui Li Fangyun Wei Chao Zhang Hongyang R. Zhang 52 128 0 26 Jan 2024
Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decoding Heming Xia Zhe Yang Qingxiu Dong Peiyi Wang Yongqi Li Tao Ge Tianyu Liu Wenjie Li Zhifang Sui LRM 38 105 0 15 Jan 2024
Online Speculative Decoding Xiaoxuan Liu Lanxiang Hu Peter Bailis Alvin Cheung Zhijie Deng Ion Stoica Hao Zhang 29 53 0 11 Oct 2023
Large Language Models are Zero-Shot Reasoners Takeshi Kojima S. Gu Machel Reid Yutaka Matsuo Yusuke Iwasawa ReLM LRM 328 4,142 0 24 May 2022
The Lottery Ticket Hypothesis for Pre-trained BERT Networks Tianlong Chen Jonathan Frankle Shiyu Chang Sijia Liu Yang Zhang Zhangyang Wang Michael Carbin 156 377 0 23 Jul 2020