Exploring and Improving Drafts in Blockwise Parallel Decoding

v1v2 (latest)

Exploring and Improving Drafts in Blockwise Parallel Decoding

14 April 2024

Kishore Papineni

Sanjiv Kumar

ArXiv (abs)PDF HTML

Papers citing "Exploring and Improving Drafts in Blockwise Parallel Decoding"

16 / 16 papers shown

Title
Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads Tianle Cai Yuhong Li Zhengyang Geng Hongwu Peng Jason D. Lee De-huai Chen Tri Dao 149 313 0 19 Jan 2024
PaSS: Parallel Speculative Sampling Giovanni Monea Armand Joulin Edouard Grave MoE 67 38 0 22 Nov 2023
Accelerating LLM Inference with Staged Speculative Decoding Benjamin Spector Christal Re 69 112 0 08 Aug 2023
Speculative Decoding with Big Little Decoder Sehoon Kim K. Mangalam Suhong Moon Jitendra Malik Michael W. Mahoney A. Gholami Kurt Keutzer MoE 97 112 0 15 Feb 2023
Fast Inference from Transformers via Speculative Decoding Yaniv Leviathan Matan Kalman Yossi Matias LRM 147 733 0 30 Nov 2022
Contrastive Decoding: Open-ended Text Generation as Optimization Xiang Lisa Li Ari Holtzman Daniel Fried Percy Liang Jason Eisner Tatsunori Hashimoto Luke Zettlemoyer M. Lewis 120 372 0 27 Oct 2022
Emergent Abilities of Large Language Models Jason W. Wei Yi Tay Rishi Bommasani Colin Raffel Barret Zoph ... Tatsunori Hashimoto Oriol Vinyals Percy Liang J. Dean W. Fedus ELM ReLM LRM 286 2,511 0 15 Jun 2022
ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers Z. Yao Reza Yazdani Aminabadi Minjia Zhang Xiaoxia Wu Conglong Li Yuxiong He VLM MQ 127 479 0 04 Jun 2022
PaLM: Scaling Language Modeling with Pathways Aakanksha Chowdhery Sharan Narang Jacob Devlin Maarten Bosma Gaurav Mishra ... Kathy Meier-Hellstern Douglas Eck J. Dean Slav Petrov Noah Fiedel PILM LRM 522 6,293 0 05 Apr 2022
SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarization Bogdan Gliwa Iwona Mochol M. Biesek A. Wawer 124 638 0 27 Nov 2019
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 470 20,317 0 23 Oct 2019
Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization Shashi Narayan Shay B. Cohen Mirella Lapata AILaw 146 1,683 0 27 Aug 2018
Newsroom: A Dataset of 1.3 Million Summaries with Diverse Extractive Strategies Max Grusky Mor Naaman Yoav Artzi 93 558 0 30 Apr 2018
Non-Autoregressive Neural Machine Translation Jiatao Gu James Bradbury Caiming Xiong Victor O.K. Li R. Socher 105 797 0 07 Nov 2017
SQuAD: 100,000+ Questions for Machine Comprehension of Text Pranav Rajpurkar Jian Zhang Konstantin Lopyrev Percy Liang RALM 314 8,169 0 16 Jun 2016
Teaching Machines to Read and Comprehend Karl Moritz Hermann Tomás Kociský Edward Grefenstette L. Espeholt W. Kay Mustafa Suleyman Phil Blunsom 349 3,552 0 10 Jun 2015

We use cookies and other tracking technologies to improve your browsing experience on our website, to show you personalized content and targeted ads, to analyze our website traffic, and to understand where our visitors are coming from. See our policy.