Ouroboros: Generating Longer Drafts Phrase by Phrase for Faster
Speculative Decoding

Ouroboros: Generating Longer Drafts Phrase by Phrase for Faster Speculative Decoding

21 February 2024

Weilin Zhao

Xu Han

Chaojun Xiao

Maosong Sun

Papers citing "Ouroboros: Generating Longer Drafts Phrase by Phrase for Faster Speculative Decoding"

5 / 5 papers shown

Title
PowerInfer: Fast Large Language Model Serving with a Consumer-grade GPU Yixin Song Zeyu Mi Haotong Xie Haibo Chen BDL 125 120 0 16 Dec 2023
Distilling Linguistic Context for Language Model Compression Geondo Park Gyeongman Kim Eunho Yang 45 38 0 17 Sep 2021
The Lottery Ticket Hypothesis for Pre-trained BERT Networks Tianlong Chen Jonathan Frankle Shiyu Chang Sijia Liu Yang Zhang Zhangyang Wang Michael Carbin 156 345 0 23 Jul 2020
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism M. Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 245 1,821 0 17 Sep 2019
Teaching Machines to Read and Comprehend Karl Moritz Hermann Tomás Kociský Edward Grefenstette L. Espeholt W. Kay Mustafa Suleyman Phil Blunsom 181 3,510 0 10 Jun 2015