Adapting Language Models to Compress Contexts

24 May 2023

Alexander Wettig

Papers citing "Adapting Language Models to Compress Contexts"

35 / 35 papers shown

Title
PIS: Linking Importance Sampling and Attention Mechanisms for Efficient Prompt Compression Lizhe Chen Binjia Zhou Yuyao Ge Jiayi Chen Shiguang NI 142 1 0 23 Apr 2025
M2IV: Towards Efficient and Fine-grained Multimodal In-Context Learning in Large Vision-Language Models Yanshu Li Hongyang He Yi Cao Qisen Cheng Xiang Fu Ruixiang Tang VLM 40 0 0 06 Apr 2025
Cognitive Memory in Large Language Models Lianlei Shan Shixian Luo Zezhou Zhu Yu Yuan Yong Wu LLMAG KELM 160 1 0 03 Apr 2025
DAST: Context-Aware Compression in LLMs via Dynamic Allocation of Soft Tokens Shaoshen Chen Yangning Li Zishan Xu Hai-Tao Zheng Xin Su Zifei Shan Hai-Tao Zheng 53 0 0 17 Feb 2025
Following the Autoregressive Nature of LLM Embeddings via Compression and Alignment Jingcheng Deng Zhongtao Jiang Liang Pang Liwei Chen Kun Xu Zihao Wei Huawei Shen Xueqi Cheng 54 1 0 17 Feb 2025
LV-XAttn: Distributed Cross-Attention for Long Visual Inputs in Multimodal Large Language Models Tzu-Tao Chang Shivaram Venkataraman VLM 175 0 0 04 Feb 2025
Vision-centric Token Compression in Large Language Model Ling Xing Alex Jinpeng Wang Rui Yan Xiangbo Shu Jinhui Tang VLM 62 0 0 02 Feb 2025
ELITR-Bench: A Meeting Assistant Benchmark for Long-Context Language Models Thibaut Thonet Jos Rozen Laurent Besacier RALM 137 2 0 20 Jan 2025
Better Prompt Compression Without Multi-Layer Perceptrons Edouardo Honig Andrew Lizarraga Zijun Zhang Ying Nian Wu MQ 149 1 0 12 Jan 2025
From Reading to Compressing: Exploring the Multi-document Reader for Prompt Compression Eunseong Choi Sunkyung Lee Minjin Choi June Park Jongwuk Lee 65 1 0 03 Jan 2025
Generative Adapter: Contextualizing Language Models in Parameters with A Single Forward Pass Tong Chen Hao Fang Patrick Xia Xiaodong Liu Benjamin Van Durme Luke Zettlemoyer Jianfeng Gao Hao Cheng KELM 53 2 0 08 Nov 2024
Interacting Large Language Model Agents. Interpretable Models and Social Learning Adit Jain Vikram Krishnamurthy LLMAG 37 0 0 02 Nov 2024
BRIEF: Bridging Retrieval and Inference for Multi-hop Reasoning via Compression Yuankai Li Jia-Chen Gu Di Wu Kai-Wei Chang Nanyun Peng RALM MQ 23 0 0 20 Oct 2024
Coarse-to-Fine Highlighting: Reducing Knowledge Hallucination in Large Language Models Qitan Lv Jie Wang Hanzhu Chen Bin Li Yongdong Zhang Feng Wu HILM 28 3 0 19 Oct 2024
Style-Compress: An LLM-Based Prompt Compression Framework Considering Task-Specific Styles Xiao Pu Tianxing He Xiaojun Wan VLM 40 3 0 17 Oct 2024
HART: Efficient Visual Generation with Hybrid Autoregressive Transformer Haotian Tang Yecheng Wu Shang Yang Enze Xie Junsong Chen Junyu Chen Zhuoyang Zhang Han Cai Yaojie Lu Song Han 71 33 0 14 Oct 2024
How to Train Long-Context Language Models (Effectively) Tianyu Gao Alexander Wettig Howard Yen Danqi Chen RALM 72 38 0 03 Oct 2024
The Compressor-Retriever Architecture for Language Model OS Yuan Yang Siheng Xiong Ehsan Shareghi Faramarz Fekri RALM KELM 32 1 0 02 Sep 2024
KV Cache Compression, But What Must We Give in Return? A Comprehensive Benchmark of Long Context Capable Approaches Jiayi Yuan Hongyi Liu Shaochen Zhong Yu-Neng Chuang ... Hongye Jin V. Chaudhary Zhaozhuo Xu Zirui Liu Xia Hu 43 17 0 01 Jul 2024
Leveraging Passage Embeddings for Efficient Listwise Reranking with Large Language Models Qi Liu Bo Wang Nan Wang Jiaxin Mao RALM 80 3 0 21 Jun 2024
DeciMamba: Exploring the Length Extrapolation Potential of Mamba Assaf Ben-Kish Itamar Zimerman Shady Abu Hussein Nadav Cohen Amir Globerson Lior Wolf Raja Giryes Mamba 77 13 0 20 Jun 2024
VoCo-LLaMA: Towards Vision Compression with Large Language Models Xubing Ye Yukang Gan Xiaoke Huang Yixiao Ge Yansong Tang MLLM VLM 43 22 0 18 Jun 2024
BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack Yuri Kuratov Aydar Bulatov Petr Anokhin Ivan Rodkin Dmitry Sorokin Artyom Sorokin Andrey Kravchenko RALM ALM LRM ReLM ELM 49 59 0 14 Jun 2024
Implicit In-context Learning Zhuowei Li Zihao Xu Ligong Han Yunhe Gao Song Wen Di Liu Hao Wang Dimitris N. Metaxas 38 1 0 23 May 2024
Unlocking Data-free Low-bit Quantization with Matrix Decomposition for KV Cache Compression Peiyu Liu Zeming Gao Wayne Xin Zhao Yipeng Ma Tao Wang Ji-Rong Wen MQ 37 5 0 21 May 2024
A Survey on Large Language Model-Based Game Agents Sihao Hu Tiansheng Huang Gaowen Liu Ramana Rao Kompella Gaowen Liu Selim Furkan Tekin Yichang Xu Zachary Yahn Ling Liu LLMAG LM&Ro AI4CE LM&MA 71 51 0 02 Apr 2024
NovelQA: Benchmarking Question Answering on Documents Exceeding 200K Tokens Cunxiang Wang Ruoxi Ning Boqi Pan Tonghui Wu Qipeng Guo ... Guangsheng Bao Xiangkun Hu Zheng Zhang Qian Wang Yue Zhang RALM 106 4 0 18 Mar 2024
User-LLM: Efficient LLM Contextualization with User Embeddings Lin Ning Luyang Liu Jiaxing Wu Neo Wu D. Berlowitz Sushant Prakash Bradley Green S. O’Banion Jun Xie 55 33 0 21 Feb 2024
Sentiment Analysis through LLM Negotiations Xiaofei Sun Xiaoya Li Shengyu Zhang Shuhe Wang Fei Wu Jiwei Li Tianwei Zhang Guoyin Wang 32 16 0 03 Nov 2023
LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models Huiqiang Jiang Qianhui Wu Chin-Yew Lin Yuqing Yang Lili Qiu 34 101 0 09 Oct 2023
Learning by Distilling Context Charles Burton Snell Dan Klein Ruiqi Zhong ReLM LRM 168 44 0 30 Sep 2022
Training Language Models with Memory Augmentation Zexuan Zhong Tao Lei Danqi Chen RALM 239 128 0 25 May 2022
Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation Ofir Press Noah A. Smith M. Lewis 253 695 0 27 Aug 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 280 3,848 0 18 Apr 2021
The Pile: An 800GB Dataset of Diverse Text for Language Modeling Leo Gao Stella Biderman Sid Black Laurence Golding Travis Hoppe ... Horace He Anish Thite Noa Nabeshima Shawn Presser Connor Leahy AIMat 279 1,996 0 31 Dec 2020