BanditSpec: Adaptive Speculative Decoding via Bandit Algorithms

BanditSpec: Adaptive Speculative Decoding via Bandit Algorithms

21 May 2025

Vincent Y. F. Tan

ArXiv (abs)PDF HTML

Papers citing "BanditSpec: Adaptive Speculative Decoding via Bandit Algorithms"

15 / 15 papers shown

Title
Sparse-to-Dense: A Free Lunch for Lossless Acceleration of Video Understanding in LLMs Xuan Zhang Cunxiao Du Sicheng Yu Jiawei Wu Fengzhuo Zhang Wei Gao Qian Liu 61 0 0 25 May 2025
SAM Decoding: Speculative Decoding via Suffix Automaton Yuxuan Hu Ke Wang Jing Zhang Fanjin Zhang Cuiping Li Hong Chen Jing Zhang 138 5 0 16 Nov 2024
A Theoretical Perspective for Speculative Decoding Algorithm Ming Yin Minshuo Chen Kaixuan Huang Mengdi Wang 49 8 0 30 Oct 2024
TriForce: Lossless Acceleration of Long Sequence Generation with Hierarchical Speculative Decoding Hanshi Sun Zhuoming Chen Xinyu Yang Yuandong Tian Beidi Chen 117 65 0 18 Apr 2024
EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty Yuhui Li Fangyun Wei Chao Zhang Hongyang R. Zhang 144 165 0 26 Jan 2024
Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads Tianle Cai Yuhong Li Zhengyang Geng Hongwu Peng Jason D. Lee De-huai Chen Tri Dao 177 314 0 19 Jan 2024
Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decoding Heming Xia Zhe Yang Qingxiu Dong Peiyi Wang Chak Tou Leong Tao Ge Tianyu Liu Wenjie Li Zhifang Sui LRM 149 129 0 15 Jan 2024
Multiplier Bootstrap-based Exploration Runzhe Wan Haoyu Wei Branislav Kveton R. Song 47 3 0 03 Feb 2023
Fast Inference from Transformers via Speculative Decoding Yaniv Leviathan Matan Kalman Yossi Matias LRM 151 736 0 30 Nov 2022
Robust Stochastic Linear Contextual Bandits Under Adversarial Attacks Qin Ding Cho-Jui Hsieh James Sharpnack AAML 92 33 0 05 Jun 2021
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 917 42,520 0 28 May 2020
Garbage In, Reward Out: Bootstrapping Exploration in Multi-Armed Bandits Branislav Kveton Csaba Szepesvári Sharan Vaswani Zheng Wen Mohammad Ghavamzadeh Tor Lattimore 167 70 0 13 Nov 2018
Nearly Optimal Adaptive Procedure with Change Detection for Piecewise-Stationary Bandit Yang Cao Zheng Wen Branislav Kveton Yao Xie 90 96 0 11 Feb 2018
Efficient Contextual Bandits in Non-stationary Worlds Haipeng Luo Chen-Yu Wei Alekh Agarwal John Langford 91 133 0 05 Aug 2017
The KL-UCB Algorithm for Bounded Stochastic Bandits and Beyond Aurélien Garivier Olivier Cappé 206 616 0 12 Feb 2011