Speculative Decoding via Early-exiting for Faster LLM Inference with
Thompson Sampling Control Mechanism

Speculative Decoding via Early-exiting for Faster LLM Inference with Thompson Sampling Control Mechanism

6 June 2024

ArXiv (abs)PDF HTML

Papers citing "Speculative Decoding via Early-exiting for Faster LLM Inference with Thompson Sampling Control Mechanism"

10 / 10 papers shown

Title
COUNTDOWN: Contextually Sparse Activation Filtering Out Unnecessary Weights in Down Projection Jaewon Cheon Pilsung Kang 68 0 0 23 May 2025
Semi-Clairvoyant Scheduling of Speculative Decoding Requests to Minimize LLM Inference Latency Ruixiao Li Fahao Chen Peng Li 153 0 0 20 May 2025
Constrained Decoding with Speculative Lookaheads Nishanth Nakshatri Shamik Roy Rajarshi Das Suthee Chaidaroon Leonid Boytsov Rashmi Gangadharaiah 159 0 0 09 Dec 2024
Accelerating LLM Inference with Staged Speculative Decoding Benjamin Spector Christal Re 69 112 0 08 Aug 2023
Symbolic Chain-of-Thought Distillation: Small Models Can Also "Think" Step-by-Step Liunian Harold Li Jack Hessel Youngjae Yu Xiang Ren Kai-Wei Chang Yejin Choi LRM AI4CE ReLM 84 143 0 24 Jun 2023
Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena Lianmin Zheng Wei-Lin Chiang Ying Sheng Siyuan Zhuang Zhanghao Wu ... Dacheng Li Eric Xing Haotong Zhang Joseph E. Gonzalez Ion Stoica ALM OSLM ELM 405 4,422 0 09 Jun 2023
Fast Inference from Transformers via Speculative Decoding Yaniv Leviathan Matan Kalman Yossi Matias LRM 147 733 0 30 Nov 2022
PaLM: Scaling Language Modeling with Pathways Aakanksha Chowdhery Sharan Narang Jacob Devlin Maarten Bosma Gaurav Mishra ... Kathy Meier-Hellstern Douglas Eck J. Dean Slav Petrov Noah Fiedel PILM LRM 515 6,293 0 05 Apr 2022
Training Verifiers to Solve Math Word Problems K. Cobbe V. Kosaraju Mohammad Bavarian Mark Chen Heewoo Jun ... Jerry Tworek Jacob Hilton Reiichiro Nakano Christopher Hesse John Schulman ReLM OffRL LRM 326 4,569 0 27 Oct 2021
Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization Shashi Narayan Shay B. Cohen Mirella Lapata AILaw 146 1,682 0 27 Aug 2018

We use cookies and other tracking technologies to improve your browsing experience on our website, to show you personalized content and targeted ads, to analyze our website traffic, and to understand where our visitors are coming from. See our policy.