Title
MindGuard: Towards Accessible and Sitgma-free Mental Health First Aid via Edge LLM Sijie Ji Xinzhe Zheng Jiawei Sun Renqi Chen Wei Gao Mani Srivastava AI4MH 62 4 0 16 Sep 2024
Foundations of Large Language Model Compression -- Part 1: Weight Quantization Sean I. Young MQ 83 1 0 03 Sep 2024
CHESS: Optimizing LLM Inference via Channel-Wise Thresholding and Selective Sparsification Junhui He Shangyu Wu Weidong Wen Chun Jason Xue Qingan Li 48 5 0 02 Sep 2024
AgentMonitor: A Plug-and-Play Framework for Predictive and Secure Multi-Agent Systems Chi-Min Chan Jianxuan Yu Weize Chen Chunyang Jiang Xinyu Liu Weijie Shi Zhiyuan Liu Wei Xue Yike Guo LLMAG 85 3 0 27 Aug 2024
On-Device Language Models: A Comprehensive Review Jiajun Xu Zhiyuan Li Wei Chen Qun Wang Xin Gao Qi Cai Ziyuan Ling 149 36 0 26 Aug 2024
A Tighter Complexity Analysis of SparseGPT Xiaoyu Li Yingyu Liang Zhenmei Shi Zhao Song 139 24 0 22 Aug 2024
Real-Time Video Generation with Pyramid Attention Broadcast Xuanlei Zhao Xiaolong Jin Kai Wang Yang You VGen DiffM 203 45 0 22 Aug 2024
ABQ-LLM: Arbitrary-Bit Quantized Inference Acceleration for Large Language Models Chao Zeng Songwei Liu Yusheng Xie Hong Liu Xiaojian Wang Miao Wei Shu Yang Fangmin Chen Xing Mei MQ 102 8 0 16 Aug 2024
Can Rule-Based Insights Enhance LLMs for Radiology Report Classification? Introducing the RadPrompt Methodology Panagiotis Fytas Anna Breger Ian Selby Simon Baker Shahab Shahipasand Anna Korhonen 64 0 0 07 Aug 2024
Designing Extremely Memory-Efficient CNNs for On-device Vision Tasks Jaewook Lee Yoel Park Seulki Lee VLM 63 1 0 07 Aug 2024
AgentsCoMerge: Large Language Model Empowered Collaborative Decision Making for Ramp Merging Senkang Hu Zhengru Fang Zihan Fang Yiqin Deng Xianhao Chen Yuguang Fang Sam Kwong 162 15 0 07 Aug 2024
ThinK: Thinner Key Cache by Query-Driven Pruning Yuhui Xu Zhanming Jie Hanze Dong Lei Wang Xudong Lu Aojun Zhou Amrita Saha Caiming Xiong Doyen Sahoo 180 23 0 30 Jul 2024
Apple Intelligence Foundation Language Models Tom Gunter Zirui Wang Chong-Jun Wang Ruoming Pang Andy Narayanan ... Xinwen Liu Yang Zhao Yin Xia Zhile Ren Zhongzheng Ren 150 40 0 29 Jul 2024
ScaleLLM: A Resource-Frugal LLM Serving Framework by Optimizing End-to-End Efficiency Yuhang Yao Han Jin Alay Dilipbhai Shah Shanshan Han Zijian Hu Yide Ran Dimitris Stripelis Zhaozhuo Xu Salman Avestimehr Chang D. Yoo 107 3 0 23 Jul 2024
PipeInfer: Accelerating LLM Inference using Asynchronous Pipelined Speculation Branden Butler Sixing Yu Arya Mazaheri Ali Jannesari LRM 130 7 0 16 Jul 2024
LeanQuant: Accurate Large Language Model Quantization with Loss-Error-Aware Grid Tianyi Zhang Anshumali Shrivastava MQ 70 6 0 14 Jul 2024
LLM-Collaboration on Automatic Science Journalism for the General Audience Gongyao Jiang Xinran Shi Qiong Luo 80 3 0 13 Jul 2024
Accuracy is Not All You Need Abhinav Dutta Sanjeev Krishnan Nipun Kwatra Ramachandran Ramjee 108 4 0 12 Jul 2024
RoLoRA: Fine-tuning Rotated Outlier-free LLMs for Effective Weight-Activation Quantization Xijie Huang Zechun Liu Shih-yang Liu Kwang-Ting Cheng MQ 93 9 0 10 Jul 2024
EfficientQAT: Efficient Quantization-Aware Training for Large Language Models Mengzhao Chen Wenqi Shao Peng Xu Jiahao Wang Peng Gao Kaipeng Zhang Ping Luo MQ 171 35 0 10 Jul 2024
On the Limitations of Compute Thresholds as a Governance Strategy Sara Hooker 140 19 0 08 Jul 2024
SpikeLLM: Scaling up Spiking Neural Network to Large Language Models via Saliency-based Spiking Xingrun Xing Boyan Gao Zheng Zhang David A. Clifton Shitao Xiao Li Du Guoqi Li Jiajun Zhang 172 6 0 05 Jul 2024
FoldGPT: Simple and Effective Large Language Model Compression Scheme Songwei Liu Chao Zeng Lianqiang Li Chenqian Yan Lean Fu Xing Mei Fangmin Chen 88 5 0 01 Jul 2024
VcLLM: Video Codecs are Secretly Tensor Codecs Ceyu Xu Yongji Wu Xinyu Yang Beidi Chen Matthew Lentz Danyang Zhuo Lisa Wu Wills 102 0 0 29 Jun 2024
OutlierTune: Efficient Channel-Wise Quantization for Large Language Models Jinguang Wang Yuexi Yin Haifeng Sun Qi Qi Jingyu Wang Zirui Zhuang Tingting Yang Jianxin Liao 81 2 0 27 Jun 2024
Layer-Wise Quantization: A Pragmatic and Effective Method for Quantizing LLMs Beyond Integer Bit-Levels Razvan-Gabriel Dumitru Vikas Yadav Rishabh Maheshwary Paul-Ioan Clotan Sathwik Tejaswi Madhusudhan Mihai Surdeanu MQ 127 2 0 25 Jun 2024
BitNet b1.58 Reloaded: State-of-the-art Performance Also on Smaller Networks Jacob Nielsen Peter Schneider-Kamp MQ 74 5 0 24 Jun 2024
From Decoding to Meta-Generation: Inference-time Algorithms for Large Language Models Sean Welleck Amanda Bertsch Matthew Finlayson Hailey Schoelkopf Alex Xie Graham Neubig Ilia Kulikov Zaid Harchaoui 161 77 0 24 Jun 2024
MoA: Mixture of Sparse Attention for Automatic Large Language Model Compression Tianyu Fu Haofeng Huang Xuefei Ning Genghan Zhang Boju Chen ... Shiyao Li Shengen Yan Guohao Dai Huazhong Yang Yu Wang MQ 129 24 0 21 Jun 2024
Optimizing Speculative Decoding for Serving Large Language Models Using Goodput Xiaoxuan Liu Cade Daniel Langxiang Hu Woosuk Kwon Zhuohan Li Xiangxi Mo Alvin Cheung Zhijie Deng Ion Stoica Hao Zhang 108 23 0 20 Jun 2024
Slice-Level Scheduling for High Throughput and Load Balanced LLM Serving Ke Cheng Wen Hu Zhi Wang Hongen Peng Jianguo Li Sheng Zhang 73 9 0 19 Jun 2024
Mixture of Scales: Memory-Efficient Token-Adaptive Binarization for Large Language Models Dongwon Jo Taesu Kim Yulhwa Kim Jae-Joon Kim 137 5 0 18 Jun 2024
Prefixing Attention Sinks can Mitigate Activation Outliers for Large Language Model Quantization Seungwoo Son Wonpyo Park Woohyun Han Kyuyeun Kim Jaeho Lee MQ 78 13 0 17 Jun 2024
Tender: Accelerating Large Language Models via Tensor Decomposition and Runtime Requantization Jungi Lee Wonbeom Lee Jaewoong Sim MQ 108 14 0 16 Jun 2024
Evaluating the Generalization Ability of Quantized LLMs: Benchmark, Analysis, and Toolbox Yijun Liu Yuan Meng Fang Wu Shenhao Peng Hang Yao Chaoyu Guan Chen Tang Xinzhu Ma Zhi Wang Wenwu Zhu MQ 115 8 0 15 Jun 2024
ME-Switch: A Memory-Efficient Expert Switching Framework for Large Language Models Jing Liu Ruihao Gong Mingyang Zhang Yefei He Jianfei Cai Bohan Zhuang MoE 78 0 0 13 Jun 2024
TernaryLLM: Ternarized Large Language Model Tianqi Chen Zhe Li Weixiang Xu Zeyu Zhu Dong Li Lu Tian E. Barsoum Peisong Wang Jian Cheng 77 7 0 11 Jun 2024
Low-Rank Quantization-Aware Training for LLMs Yelysei Bondarenko Riccardo Del Chiaro Markus Nagel MQ 79 14 0 10 Jun 2024
PowerInfer-2: Fast Large Language Model Inference on a Smartphone Zhenliang Xue Yixin Song Zeyu Mi Le Chen Yubin Xia Haibo Chen 129 52 0 10 Jun 2024
Turbo Sparse: Achieving LLM SOTA Performance with Minimal Activated Parameters Yixin Song Haotong Xie Zhengyan Zhang Bo Wen Li Ma Zeyu Mi Haibo Chen MoE 173 25 0 10 Jun 2024
BitsFusion: 1.99 bits Weight Quantization of Diffusion Model Yang Sui Yanyu Li Anil Kag Yerlan Idelbayev Junli Cao Ju Hu Dhritiman Sagar Bo Yuan Sergey Tulyakov Jian Ren MQ 94 22 0 06 Jun 2024
Empirical Guidelines for Deploying LLMs onto Resource-constrained Edge Devices Ruiyang Qin Dancheng Liu Zheyu Yan Zhaoxuan Tan Zixuan Pan Zhenge Jia Meng Jiang Ahmed Abbasi Jinjun Xiong Yiyu Shi 103 15 0 06 Jun 2024
Llumnix: Dynamic Scheduling for Large Language Model Serving Biao Sun Ziming Huang Hanyu Zhao Wencong Xiao Xinyi Zhang Yong Li Wei Lin 93 57 0 05 Jun 2024
Pruner-Zero: Evolving Symbolic Pruning Metric from scratch for Large Language Models Peijie Dong Lujun Li Zhenheng Tang Xiang Liu Xinglin Pan Qiang-qiang Wang Xiaowen Chu 156 33 0 05 Jun 2024
Seed-TTS: A Family of High-Quality Versatile Speech Generation Models Philip Anastassiou Jiawei Chen Jingshu Chen Yuanzhe Chen Zhuo Chen ... Wenjie Zhang Yanzhe Zhang Zilin Zhao Dejian Zhong Xiaobin Zhuang 121 106 0 04 Jun 2024
ViDiT-Q: Efficient and Accurate Quantization of Diffusion Transformers for Image and Video Generation Tianchen Zhao Tongcheng Fang Haofeng Huang Enshu Liu Widyadewi Soedarmadji ... Shengen Yan Huazhong Yang Xuefei Ning Xuefei Ning Yu Wang MQ VGen 197 35 0 04 Jun 2024
Achieving Sparse Activation in Small Language Models Jifeng Song Kai Huang Xiangyu Yin Boyuan Yang Wei Gao 89 4 0 03 Jun 2024
MagR: Weight Magnitude Reduction for Enhancing Post-Training Quantization Aozhong Zhang Naigang Wang Yanxia Deng Xin Li Zi Yang Penghang Yin MQ 86 8 0 02 Jun 2024
Outliers and Calibration Sets have Diminishing Effect on Quantization of Modern LLMs Davide Paglieri Saurabh Dash Tim Rocktaschel Jack Parker-Holder MQ 77 6 0 31 May 2024
One QuantLLM for ALL: Fine-tuning Quantized LLMs Once for Efficient Deployments Ke Yi Yuhui Xu Heng Chang Chen Tang Yuan Meng Tong Zhang Jia Li MQ 88 2 0 30 May 2024