Generation Meets Verification: Accelerating Large Language Model
Inference with Smart Parallel Auto-Correct Decoding

Generation Meets Verification: Accelerating Large Language Model Inference with Smart Parallel Auto-Correct Decoding

19 February 2024

Papers citing "Generation Meets Verification: Accelerating Large Language Model Inference with Smart Parallel Auto-Correct Decoding"

10 / 10 papers shown

Title
PipeDec: Low-Latency Pipeline-based Inference with Dynamic Speculative Decoding towards Large-scale Models Haofei Yin Mengbai Xiao Rouzhou Lu Xiao Zhang Dongxiao Yu Guanghui Zhang AI4CE 26 0 0 05 Apr 2025
EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test Yuhui Li Fangyun Wei Chao Zhang Hongyang R. Zhang 120 6 0 03 Mar 2025
Speculative Decoding and Beyond: An In-Depth Survey of Techniques Y. Hu Zining Liu Zhenyuan Dong Tianfan Peng Bradley McDanel S. Zhang 93 0 0 27 Feb 2025
SWIFT: On-the-Fly Self-Speculative Decoding for LLM Inference Acceleration Heming Xia Yongqi Li Jun Zhang Cunxiao Du Wenjie Li LRM 53 6 0 09 Oct 2024
ParallelSpec: Parallel Drafter for Efficient Speculative Decoding Zilin Xiao Hongming Zhang Tao Ge Siru Ouyang Vicente Ordonez Dong Yu 39 5 0 08 Oct 2024
S2D: Sorted Speculative Decoding For More Efficient Deployment of Nested Large Language Models Parsa Kavehzadeh Mohammadreza Pourreza Mojtaba Valipour Tinashu Zhu Haoli Bai Ali Ghodsi Boxing Chen Mehdi Rezagholizadeh 32 0 0 02 Jul 2024
EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees Yuhui Li Fangyun Wei Chao Zhang Hongyang R. Zhang 88 55 0 24 Jun 2024
A Comprehensive Survey of Accelerated Generation Techniques in Large Language Models Mahsa Khoshnoodi Vinija Jain Mingye Gao Malavika Srikanth Aman Chadha OffRL 33 1 0 15 May 2024
Clover: Regressive Lightweight Speculative Decoding with Sequential Knowledge Bin Xiao Chunan Shi Xiaonan Nie Fan Yang Xiangwei Deng Lei Su Weipeng Chen Bin Cui 31 8 0 01 May 2024
Break the Sequential Dependency of LLM Inference Using Lookahead Decoding Yichao Fu Peter Bailis Ion Stoica Hao Zhang 133 143 0 03 Feb 2024