v1v2 (latest)

Adapting Language Models to Compress Contexts

24 May 2023

Alexander Wettig

Papers citing "Adapting Language Models to Compress Contexts"

41 / 41 papers shown

Title
LoRA-Gen: Specializing Large Language Model via Online LoRA Generation Yicheng Xiao Lin Song Rui Yang Cheng Cheng Yixiao Ge Xiu Li Y. Shan OffRL 24 0 0 13 Jun 2025
Evaluating and Improving Robustness in Large Language Models: A Survey and Future Directions Kun Zhang Le Wu Kui Yu Guangyi Lv Dacao Zhang AAML ELM 36 0 0 08 Jun 2025
Proactive Assistant Dialogue Generation from Streaming Egocentric Videos Yichi Zhang Xin Luna Dong Zhaojiang Lin Andrea Madotto Anuj Kumar Babak Damavandi J. Chai Seungwhan Moon 68 0 0 06 Jun 2025
Lossless Token Sequence Compression via Meta-Tokens John Harvill Ziwei Fan Hao Wang Yizhou Sun Hao Ding Luke Huan Anoop Deoras 28 0 0 30 May 2025
Select, Read, and Write: A Multi-Agent Framework of Full-Text-based Related Work Generation Xiaochuan Liu Ruihua Song Xiting Wang Xu Chen 65 0 0 26 May 2025
GMSA: Enhancing Context Compression via Group Merging and Layer Semantic Alignment Jiwei Tang Zhicheng Zhang Shunlong Wu Jingheng Ye Lichen Bai ... Tingwei Lu Jiaqi Chen Lin Hai Hai-Tao Zheng Hong-Gee Kim 61 0 0 18 May 2025
Cognitive Memory in Large Language Models Lianlei Shan Shixian Luo Zezhou Zhu Yu Yuan Yong Wu LLMAG KELM 525 3 0 03 Apr 2025
InfiniteICL: Breaking the Limit of Context Window Size via Long Short-term Memory Transformation Bowen Cao Deng Cai W. Lam CLL 101 1 0 02 Apr 2025
A Survey on Transformer Context Extension: Approaches and Evaluation Yijun Liu Jinzheng Yu Yang Xu Zhongyang Li Qingfu Zhu LLMAG 128 3 0 17 Mar 2025
RAPID: Long-Context Inference with Retrieval-Augmented Speculative Decoding Guanzheng Chen Qilong Feng Jinjie Ni Xin Li Michael Shieh RALM 117 3 0 27 Feb 2025
Detecting Knowledge Boundary of Vision Large Language Models by Sampling-Based Inference Zhuo Chen Xinyu Wang Yong Jiang Zhen Zhang Xinyu Geng Pengjun Xie Fei Huang Kewei Tu 176 1 0 25 Feb 2025
Cramming 1568 Tokens into a Single Vector and Back Again: Exploring the Limits of Embedding Space Capacity Yuri Kuratov M. Arkhipov Aydar Bulatov Andrey Kravchenko 139 3 0 18 Feb 2025
Following the Autoregressive Nature of LLM Embeddings via Compression and Alignment Jingcheng Deng Zhongtao Jiang Liang Pang Liwei Chen Kun Xu Zihao Wei Huawei Shen Xueqi Cheng 112 3 0 17 Feb 2025
DAST: Context-Aware Compression in LLMs via Dynamic Allocation of Soft Tokens Shaoshen Chen Yangning Li Zishan Xu Hai-Tao Zheng Xin Su Zifei Shan Hai-Tao Zheng 68 0 0 17 Feb 2025
LCIRC: A Recurrent Compression Approach for Efficient Long-form Context and Query Dependent Modeling in LLMs Sumin An Junyoung Sung Wonpyo Park Chanjun Park Paul Hongsuck Seo 232 0 0 10 Feb 2025
LV-XAttn: Distributed Cross-Attention for Long Visual Inputs in Multimodal Large Language Models Tzu-Tao Chang Shivaram Venkataraman VLM 561 0 0 04 Feb 2025
Can LLMs Maintain Fundamental Abilities under KV Cache Compression? Xiang Liu Zhenheng Tang Hong Chen Peijie Dong Zeyu Li Xiuze Zhou Bo Li Xuming Hu Xiaowen Chu 477 7 0 04 Feb 2025
Vision-centric Token Compression in Large Language Model Ling Xing Alex Jinpeng Wang Rui Yan Xiangbo Shu Jinhui Tang VLM 159 0 0 02 Feb 2025
Efficient Prompt Compression with Evaluator Heads for Long-Context Transformer Inference Weizhi Fei Xueyan Niu Guoqing Xie Yingqing Liu Bo Bai Wei Han 130 1 0 22 Jan 2025
ELITR-Bench: A Meeting Assistant Benchmark for Long-Context Language Models Thibaut Thonet Jos Rozen Laurent Besacier RALM 227 3 0 20 Jan 2025
Better Prompt Compression Without Multi-Layer Perceptrons Edouardo Honig Andrew Lizarraga Zijun Zhang Ying Nian Wu MQ 432 1 0 12 Jan 2025
From Reading to Compressing: Exploring the Multi-document Reader for Prompt Compression Eunseong Choi Sunkyung Lee Minjin Choi June Park Jongwuk Lee 158 2 0 03 Jan 2025
Attention Entropy is a Key Factor: An Analysis of Parallel Context Encoding with Full-attention-based Pre-trained Language Models Zhisong Zhang Yan Wang Xinting Huang Tianqing Fang Han Zhang Chenlong Deng Shuaiyi Li Dong Yu 150 6 0 21 Dec 2024
Interacting Large Language Model Agents. Interpretable Models and Social Learning Adit Jain Vikram Krishnamurthy LLMAG 150 0 0 02 Nov 2024
Long Sequence Modeling with Attention Tensorization: From Sequence to Tensor Learning Aosong Feng Rex Ying Leandros Tassiulas 58 2 0 28 Oct 2024
BRIEF: Bridging Retrieval and Inference for Multi-hop Reasoning via Compression Yuankai Li Jia-Chen Gu Di Wu Kai-Wei Chang Nanyun Peng RALM MQ 72 0 0 20 Oct 2024
HART: Efficient Visual Generation with Hybrid Autoregressive Transformer Haotian Tang Yecheng Wu Shang Yang Enze Xie Junsong Chen Junyu Chen Zhuoyang Zhang Han Cai Yaojie Lu Song Han 222 48 0 14 Oct 2024
How to Train Long-Context Language Models (Effectively) Tianyu Gao Alexander Wettig Howard Yen Danqi Chen RALM 204 48 0 03 Oct 2024
HiAgent: Hierarchical Working Memory Management for Solving Long-Horizon Agent Tasks with Large Language Model Mengkang Hu Tianxing Chen Qiguang Chen Yao Mu Wenqi Shao Ping Luo LM&Ro LLMAG RALM 88 6 0 18 Aug 2024
Retrieval-Augmented Generation for Natural Language Processing: A Survey Shangyu Wu Ying Xiong Yufei Cui Haolun Wu Can Chen ... Lianming Huang Xue Liu Tei-Wei Kuo Nan Guan Chun Jason Xue 3DV RALM 147 39 0 18 Jul 2024
UIO-LLMs: Unbiased Incremental Optimization for Long-Context LLMs Wenhao Li Mingbao Lin Mingliang Xu Shuicheng Yan Rongrong Ji 71 0 0 26 Jun 2024
Leveraging Passage Embeddings for Efficient Listwise Reranking with Large Language Models Qi Liu Bo Wang Nan Wang Jiaxin Mao RALM 140 4 0 21 Jun 2024
DeciMamba: Exploring the Length Extrapolation Potential of Mamba Assaf Ben-Kish Itamar Zimerman Shady Abu Hussein Nadav Cohen Amir Globerson Lior Wolf Raja Giryes Mamba 206 20 0 20 Jun 2024
VoCo-LLaMA: Towards Vision Compression with Large Language Models Xubing Ye Yukang Gan Xiaoke Huang Yixiao Ge Yansong Tang MLLM VLM 130 28 0 18 Jun 2024
Hierarchical Compression of Text-Rich Graphs via Large Language Models Shichang Zhang Da Zheng Jiani Zhang Qi Zhu Xiang Song Soji Adeshina Christos Faloutsos George Karypis Yizhou Sun VLM 92 1 0 13 Jun 2024
Implicit In-context Learning Zhuowei Li Zihao Xu Ligong Han Yunhe Gao Song Wen Di Liu Hao Wang Dimitris N. Metaxas 149 3 0 23 May 2024
A Survey on Large Language Model-Based Game Agents Sihao Hu Tiansheng Huang Gaowen Liu Ramana Rao Kompella Gaowen Liu Selim Furkan Tekin Yichang Xu Zachary Yahn Ling Liu LLMAG LM&Ro AI4CE LM&MA 231 58 0 02 Apr 2024
NovelQA: Benchmarking Question Answering on Documents Exceeding 200K Tokens Cunxiang Wang Ruoxi Ning Boqi Pan Tonghui Wu Qipeng Guo ... Guangsheng Bao Xiangkun Hu Zheng Zhang Qian Wang Yue Zhang RALM 237 11 0 18 Mar 2024
CAMELoT: Towards Large Language Models with Training-Free Consolidated Associative Memory Zexue He Leonid Karlinsky Donghyun Kim Julian McAuley Dmitry Krotov Rogerio Feris KELM RALM 86 11 0 21 Feb 2024
Fewer is More: Boosting LLM Reasoning with Reinforced Context Pruning Xijie Huang Li Lyna Zhang Kwang-Ting Cheng Fan Yang Mao Yang LRM ReLM 94 13 0 14 Dec 2023
Walking Down the Memory Maze: Beyond Context Limit through Interactive Reading Howard Chen Ramakanth Pasunuru Jason Weston Asli Celikyilmaz RALM 148 86 0 08 Oct 2023