ConsistentEE: A Consistent and Hardness-Guided Early Exiting Method for
Accelerating Language Models Inference

v1v2 (latest)

ConsistentEE: A Consistent and Hardness-Guided Early Exiting Method for Accelerating Language Models Inference

19 December 2023

ArXiv (abs)PDF HTML

Papers citing "ConsistentEE: A Consistent and Hardness-Guided Early Exiting Method for Accelerating Language Models Inference"

5 / 5 papers shown

Title
GREEN-CODE: Learning to Optimize Energy Efficiency in LLM-based Code Generation Shashikant Ilager Lukas Florian Briem Ivona Brandić 78 0 0 19 Jan 2025
COSEE: Consistency-Oriented Signal-Based Early Exiting via Calibrated Sample Weighting Mechanism Jianing He Qi Zhang Hongyun Zhang Xuanjing Huang Usman Naseem Duoqian Miao 136 1 0 17 Dec 2024
Dynamic layer selection in decoder-only transformers Theodore Glavas Joud Chataoui Florence Regol Wassim Jabbour Antonios Valkanas Boris N. Oreshkin Mark Coates AI4CE 83 1 0 26 Oct 2024
A Comprehensive Survey of Accelerated Generation Techniques in Large Language Models Mahsa Khoshnoodi Vinija Jain Mingye Gao Malavika Srikanth Aman Chadha OffRL 125 5 0 15 May 2024
LLM Inference Unveiled: Survey and Roofline Model Insights Zhihang Yuan Yuzhang Shang Yang Zhou Zhen Dong Zhe Zhou ... Yong Jae Lee Yan Yan Beidi Chen Guangyu Sun Kurt Keutzer 225 91 0 26 Feb 2024