DeFT: Decoding with Flash Tree-attention for Efficient Tree-structured LLM Inference

DeFT: Decoding with Flash Tree-attention for Efficient Tree-structured LLM Inference

30 March 2024

Zeke Wang

Papers citing "DeFT: Decoding with Flash Tree-attention for Efficient Tree-structured LLM Inference"

8 / 8 papers shown

Title
Bifurcated Attention: Accelerating Massively Parallel Decoding with Shared Prefixes in LLMs Ben Athiwaratkun Sujan Kumar Gonugondla Sanjay Krishna Gouda Haifeng Qian Hantian Ding ... Liangfu Chen Parminder Bhatia Ramesh Nallapati Sudipta Sengupta Bing Xiang 59 4 0 13 Mar 2024
Hydragen: High-Throughput LLM Inference with Shared Prefixes Jordan Juravsky Bradley Brown Ryan Ehrlich Daniel Y. Fu Christopher Ré Azalia Mirhoseini 58 36 0 07 Feb 2024
APAR: LLMs Can Do Auto-Parallel Auto-Regressive Decoding Mingdao Liu Aohan Zeng Bowen Wang Peng Zhang Jie Tang Yuxiao Dong 72 8 0 12 Jan 2024
Lookahead: An Inference Acceleration Framework for Large Language Model with Lossless Generation Accuracy Yao-Min Zhao Zhitian Xie Chen Liang Chenyi Zhuang Jinjie Gu 61 11 0 20 Dec 2023
Self-Evaluation Guided Beam Search for Reasoning Yuxi Xie Kenji Kawaguchi Yiran Zhao Xu Zhao MingSung Kan Junxian He Qizhe Xie LRM 166 129 0 01 May 2023
Self-Consistency Improves Chain of Thought Reasoning in Language Models Xuezhi Wang Jason W. Wei Dale Schuurmans Quoc Le Ed H. Chi Sharan Narang Aakanksha Chowdhery Denny Zhou ReLM BDL LRM AI4CE 314 3,273 0 21 Mar 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 395 8,495 0 28 Jan 2022
Measuring Coding Challenge Competence With APPS Dan Hendrycks Steven Basart Saurav Kadavath Mantas Mazeika Akul Arora ... Collin Burns Samir Puranik Horace He D. Song Jacob Steinhardt ELM AIMat ALM 208 627 0 20 May 2021