PaSS: Parallel Speculative Sampling

22 November 2023

Papers citing "PaSS: Parallel Speculative Sampling"

28 / 28 papers shown

Title
Multi-Token Prediction Needs Registers Anastasios Gerontopoulos Spyros Gidaris N. Komodakis 29 0 0 15 May 2025
PARD: Accelerating LLM Inference with Low-Cost PARallel Draft Model Adaptation Zihao An Huajun Bai Ziqiang Liu Dong Li E. Barsoum 61 0 0 23 Apr 2025
Roll the dice & look before you leap: Going beyond the creative limits of next-token prediction Vaishnavh Nagarajan Chen Henry Wu Charles Ding Aditi Raghunathan 40 0 0 21 Apr 2025
Language Models, Graph Searching, and Supervision Adulteration: When More Supervision is Less and How to Make More More Arvid Frydenlund LRM 60 0 0 13 Mar 2025
EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test Yuhui Li Fangyun Wei Chao Zhang Hongyang R. Zhang 123 6 0 03 Mar 2025
Speculative Decoding and Beyond: An In-Depth Survey of Techniques Y. Hu Zining Liu Zhenyuan Dong Tianfan Peng Bradley McDanel S. Zhang 93 0 0 27 Feb 2025
From Hours to Minutes: Lossless Acceleration of Ultra Long Sequence Generation up to 100K Tokens Tong Wu Junzhe Shen Zixia Jia Yanjie Wang Zilong Zheng 85 0 0 26 Feb 2025
Towards Optimal Multi-draft Speculative Decoding Zhibo Hu Tong Zheng Vignesh Viswanathan Ziyi Chen Ryan Rossi Yihan Wu Dinesh Manocha Heng Huang 47 3 0 26 Feb 2025
Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planning Jiacheng Ye Jiahui Gao Shansan Gong Lin Zheng Xin Jiang Zhiyu Li Lingpeng Kong DiffM LRM 74 15 0 18 Oct 2024
The Mystery of the Pathological Path-star Task for Language Models Arvid Frydenlund LRM 27 4 0 17 Oct 2024
COrAL: Order-Agnostic Language Modeling for Efficient Iterative Refinement Yuxi Xie Anirudh Goyal Xiaobao Wu Xunjian Yin Xiao Xu Min-Yen Kan Liangming Pan William Yang Wang LRM 149 1 0 12 Oct 2024
ParallelSpec: Parallel Drafter for Efficient Speculative Decoding Zilin Xiao Hongming Zhang Tao Ge Siru Ouyang Vicente Ordonez Dong Yu 44 5 0 08 Oct 2024
Semformer: Transformer Language Models with Semantic Planning Yongjing Yin Junran Ding Kai Song Yue Zhang 45 4 0 17 Sep 2024
CREST: Effectively Compacting a Datastore For Retrieval-Based Speculative Decoding Sophia Ho Jinsol Park Patrick Wang 34 0 0 08 Aug 2024
EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees Yuhui Li Fangyun Wei Chao Zhang Hongyang R. Zhang 93 55 0 24 Jun 2024
SpecDec++: Boosting Speculative Decoding via Adaptive Candidate Lengths Kaixuan Huang Xudong Guo Mengdi Wang 47 20 0 30 May 2024
Faster Cascades via Speculative Decoding Harikrishna Narasimhan Wittawat Jitkrittum A. S. Rawat Seungyeon Kim Neha Gupta A. Menon Sanjiv Kumar LRM 44 6 0 29 May 2024
Hardware-Aware Parallel Prompt Decoding for Memory-Efficient Acceleration of LLM Inference Hao Mark Chen Wayne Luk Ka-Fai Cedric Yiu Rui Li Konstantin Mishchenko Stylianos I. Venieris Hongxiang Fan 47 7 0 28 May 2024
BlockLLM: Multi-tenant Finer-grained Serving for Large Language Models Jiamin Li Le Xu Hong-Yu Xu Aditya Akella 32 1 0 28 Apr 2024
Beyond the Speculative Game: A Survey of Speculative Execution in Large Language Models Chen Zhang Zhuorui Liu Dawei Song LRM 41 3 0 23 Apr 2024
A Survey on Efficient Inference for Large Language Models Zixuan Zhou Xuefei Ning Ke Hong Tianyu Fu Jiaming Xu ... Shengen Yan Guohao Dai Xiao-Ping Zhang Yuhan Dong Yu-Xiang Wang 46 86 0 22 Apr 2024
Exploring and Improving Drafts in Blockwise Parallel Decoding Taehyeon Kim A. Suresh Kishore Papineni Michael Riley Sanjiv Kumar Adrian Benton AI4TS 52 2 0 14 Apr 2024
The pitfalls of next-token prediction Gregor Bachmann Vaishnavh Nagarajan 37 63 0 11 Mar 2024
LLM Inference Unveiled: Survey and Roofline Model Insights Zhihang Yuan Yuzhang Shang Yang Zhou Zhen Dong Zhe Zhou ... Yong Jae Lee Yan Yan Beidi Chen Guangyu Sun Kurt Keutzer 56 81 0 26 Feb 2024
Chimera: A Lossless Decoding Method for Accelerating Large Language Models Inference by Fusing all Tokens Ziqian Zeng Jiahong Yu Qianshi Pang Zihao Wang Huiping Zhuang Cen Chen Xiaofeng Zou 38 4 0 24 Feb 2024
Ouroboros: Generating Longer Drafts Phrase by Phrase for Faster Speculative Decoding Weilin Zhao Yuxiang Huang Xu Han Wang Xu Chaojun Xiao Xinrong Zhang Yewei Fang Kaihuo Zhang Zhiyuan Liu Maosong Sun 43 11 0 21 Feb 2024
BiTA: Bi-Directional Tuning for Lossless Acceleration in Large Language Models Feng-Huei Lin Hanling Yi Hongbin Li Yifan Yang Xiaotian Yu Guangming Lu Rong Xiao 41 3 0 23 Jan 2024
Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decoding Heming Xia Zhe Yang Qingxiu Dong Peiyi Wang Yongqi Li Tao Ge Tianyu Liu Wenjie Li Zhifang Sui LRM 38 105 0 15 Jan 2024