LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models

9 October 2023

Huiqiang Jiang

Qianhui Wu

Chin-Yew Lin

Yuqing Yang

Lili Qiu

ArXiv PDF HTML

Papers citing "LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models"

23 / 73 papers shown

Title
GeoLLM-Engine: A Realistic Environment for Building Geospatial Copilots Simranjit Singh Michael Fore Dimitrios Stamoulis LLMAG 32 12 0 23 Apr 2024
Rethinking LLM Memorization through the Lens of Adversarial Compression Avi Schwarzschild Zhili Feng Pratyush Maini Zachary Chase Lipton J. Zico Kolter 47 42 0 23 Apr 2024
From Matching to Generation: A Survey on Generative Information Retrieval Xiaoxi Li Jiajie Jin Yujia Zhou Yuyao Zhang Peitian Zhang Yutao Zhu Zhicheng Dou 3DV 84 46 0 23 Apr 2024
A Survey on Efficient Inference for Large Language Models Zixuan Zhou Xuefei Ning Ke Hong Tianyu Fu Jiaming Xu ... Shengen Yan Guohao Dai Xiao-Ping Zhang Yuhan Dong Yu-Xiang Wang 46 83 0 22 Apr 2024
LongEmbed: Extending Embedding Models for Long Context Retrieval Dawei Zhu Liang Wang Nan Yang Yifan Song Wenhao Wu Furu Wei Sujian Li RALM 43 21 0 18 Apr 2024
Efficient Prompting Methods for Large Language Models: A Survey Kaiyan Chang Songcheng Xu Chenglong Wang Yingfeng Luo Tong Xiao Jingbo Zhu LRM 45 32 0 01 Apr 2024
PROMPT-SAW: Leveraging Relation-Aware Graphs for Textual Prompt Compression Muhammad Asif Ali Zhengping Li Shu Yang Keyuan Cheng Yang Cao Tianhao Huang Lijie Hu Lu Yu Di Wang VLM RALM 40 9 0 30 Mar 2024
PYRA: Parallel Yielding Re-Activation for Training-Inference Efficient Task Adaptation Yizhe Xiong Hui Chen Tianxiang Hao Zijia Lin Jungong Han Yuesong Zhang Guoxin Wang Yongjun Bao Guiguang Ding 51 17 0 14 Mar 2024
Retrieval-Augmented Generation for AI-Generated Content: A Survey Penghao Zhao Hailin Zhang Qinhan Yu Zhengren Wang Yunteng Geng Fangcheng Fu Ling Yang Wentao Zhang Jie Jiang Bin Cui 3DV 126 232 0 29 Feb 2024
FhGenie: A Custom, Confidentiality-preserving Chat AI for Corporate and Scientific Use Ingo Weber Hendrik Linka Daniel Mertens Tamara Muryshkin Heinrich Opgenoorth Stefan Langer SILM 30 3 0 29 Feb 2024
Beyond the Limits: A Survey of Techniques to Extend the Context Length in Large Language Models Xindi Wang Mahsa Salmani Parsa Omidi Xiangyu Ren Mehdi Rezagholizadeh A. Eshaghi LRM 34 35 0 03 Feb 2024
LoMA: Lossless Compressed Memory Attention Yumeng Wang Zhenyang Xiao 16 3 0 16 Jan 2024
Flexibly Scaling Large Language Models Contexts Through Extensible Tokenization Ninglu Shao Shitao Xiao Zheng Liu Peitian Zhang 28 4 0 15 Jan 2024
Retrieval-Augmented Generation for Large Language Models: A Survey Yunfan Gao Yun Xiong Xinyu Gao Kangxiang Jia Jinliu Pan Yuxi Bi Yi Dai Jiawei Sun Meng Wang Haofen Wang 3DV RALM 61 1,530 1 18 Dec 2023
A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions Lei Huang Weijiang Yu Weitao Ma Weihong Zhong Zhangyin Feng ... Qianglong Chen Weihua Peng Xiaocheng Feng Bing Qin Ting Liu LRM HILM 44 732 0 09 Nov 2023
PromptAgent: Strategic Planning with Language Models Enables Expert-level Prompt Optimization Xinyuan Wang Chenxi Li Zhen Wang Fan Bai Haotian Luo Jiayou Zhang Nebojsa Jojic Eric P. Xing Zhiting Hu 28 102 0 25 Oct 2023
CacheGen: KV Cache Compression and Streaming for Fast Language Model Serving Yuhan Liu Hanchen Li Yihua Cheng Siddhant Ray Yuyang Huang ... Ganesh Ananthanarayanan Michael Maire Henry Hoffmann Ari Holtzman Junchen Jiang 50 41 0 11 Oct 2023
LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression Huiqiang Jiang Qianhui Wu Xufang Luo Dongsheng Li Chin-Yew Lin Yuqing Yang Lili Qiu RALM 121 186 0 10 Oct 2023
LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Composition Chengsong Huang Qian Liu Bill Yuchen Lin Tianyu Pang Chao Du Min-Bin Lin MoMe 38 185 0 25 Jul 2023
In-context Autoencoder for Context Compression in a Large Language Model Tao Ge Jing Hu Lei Wang Xun Wang Si-Qing Chen Furu Wei RALM 40 68 0 13 Jul 2023
Complexity-Based Prompting for Multi-Step Reasoning Yao Fu Hao-Chun Peng Ashish Sabharwal Peter Clark Tushar Khot ReLM LRM 162 414 0 03 Oct 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 398 8,559 0 28 Jan 2022
Types of Out-of-Distribution Texts and How to Detect Them Udit Arora William Huang He He OODD 225 97 0 14 Sep 2021