v1v2v3 (latest)

Learning to Compress Prompts with Gist Tokens

17 April 2023

Xiang Lisa Li

Papers citing "Learning to Compress Prompts with Gist Tokens"

45 / 45 papers shown

Title
SFT-GO: Supervised Fine-Tuning with Group Optimization for Large Language Models Gyuhak Kim Sumiran Thakur Su Min Park Wei Wei Yujia Bao 17 0 0 17 Jun 2025
LoRA-Gen: Specializing Large Language Model via Online LoRA Generation Yicheng Xiao Lin Song Rui Yang Cheng Cheng Yixiao Ge Xiu Li Y. Shan OffRL 24 0 0 13 Jun 2025
Projectable Models: One-Shot Generation of Small Specialized Transformers from Large Ones A. Zhmoginov Jihwan Lee Mark Sandler 44 0 0 06 Jun 2025
Text-to-LoRA: Instant Transformer Adaption Rujikorn Charakorn Edoardo Cetin Yujin Tang Robert Tjarko Lange AI4CE 56 0 0 06 Jun 2025
Lossless Token Sequence Compression via Meta-Tokens John Harvill Ziwei Fan Hao Wang Yizhou Sun Hao Ding Luke Huan Anoop Deoras 28 0 0 30 May 2025
GMSA: Enhancing Context Compression via Group Merging and Layer Semantic Alignment Jiwei Tang Zhicheng Zhang Shunlong Wu Jingheng Ye Lichen Bai ... Tingwei Lu Jiaqi Chen Lin Hai Hai-Tao Zheng Hong-Gee Kim 59 0 0 18 May 2025
RAVU: Retrieval Augmented Video Understanding with Compositional Reasoning over Graph Sameer Malik Moyuru Yamada Ayush Singh Dishank Aggarwal 442 0 0 06 May 2025
InfiniteICL: Breaking the Limit of Context Window Size via Long Short-term Memory Transformation Bowen Cao Deng Cai W. Lam CLL 101 1 0 02 Apr 2025
A Survey on Transformer Context Extension: Approaches and Evaluation Yijun Liu Jinzheng Yu Yang Xu Zhongyang Li Qingfu Zhu LLMAG 128 3 0 17 Mar 2025
reWordBench: Benchmarking and Improving the Robustness of Reward Models with Transformed Inputs Zhaofeng Wu Michihiro Yasunaga Andrew Cohen Yoon Kim Asli Celikyilmaz Marjan Ghazvininejad 90 3 0 14 Mar 2025
Context-aware Biases for Length Extrapolation Ali Veisi Hamidreza Amirzadeh Amir Mansourian 165 1 0 11 Mar 2025
RankCoT: Refining Knowledge for Retrieval-Augmented Generation through Ranking Chain-of-Thoughts Mingyan Wu Zhenghao Liu Yukun Yan Xinze Li S. Yu Zheni Zeng Yu Gu Ge Yu RALM AI4TS LRM 87 2 0 25 Feb 2025
A generative approach to LLM harmfulness detection with special red flag tokens Sophie Xhonneux David Dobre Mehrnaz Mohfakhami Leo Schwinn Gauthier Gidel 184 2 0 22 Feb 2025
Mitigating Lost-in-Retrieval Problems in Retrieval Augmented Multi-Hop Question Answering Rongzhi Zhu Xiangyu Liu Zequn Sun Yiwei Wang Wei Hu RALM KELM LRM 251 2 0 20 Feb 2025
Cramming 1568 Tokens into a Single Vector and Back Again: Exploring the Limits of Embedding Space Capacity Yuri Kuratov M. Arkhipov Aydar Bulatov Andrey Kravchenko 139 3 0 18 Feb 2025
Following the Autoregressive Nature of LLM Embeddings via Compression and Alignment Jingcheng Deng Zhongtao Jiang Liang Pang Liwei Chen Kun Xu Zihao Wei Huawei Shen Xueqi Cheng 112 3 0 17 Feb 2025
LCIRC: A Recurrent Compression Approach for Efficient Long-form Context and Query Dependent Modeling in LLMs Sumin An Junyoung Sung Wonpyo Park Chanjun Park Paul Hongsuck Seo 228 0 0 10 Feb 2025
Vision-centric Token Compression in Large Language Model Ling Xing Alex Jinpeng Wang Rui Yan Xiangbo Shu Jinhui Tang VLM 155 0 0 02 Feb 2025
Efficient Prompt Compression with Evaluator Heads for Long-Context Transformer Inference Weizhi Fei Xueyan Niu Guoqing Xie Yingqing Liu Bo Bai Wei Han 130 1 0 22 Jan 2025
A Survey of Research in Large Language Models for Electronic Design Automation Jingyu Pan Guanglei Zhou Chen-Chia Chang Isaac Jacobson Jiang Hu Yuxiao Chen 133 5 0 17 Jan 2025
Better Prompt Compression Without Multi-Layer Perceptrons Edouardo Honig Andrew Lizarraga Zijun Zhang Ying Nian Wu MQ 432 1 0 12 Jan 2025
Registering Source Tokens to Target Language Spaces in Multilingual Neural Machine Translation Zhi Qu Yiran Wang Jiannan Mao Chenchen Ding Hideki Tanaka Masao Utiyama Taro Watanabe LRM 124 0 0 06 Jan 2025
From Reading to Compressing: Exploring the Multi-document Reader for Prompt Compression Eunseong Choi Sunkyung Lee Minjin Choi June Park Jongwuk Lee 158 2 0 03 Jan 2025
Attention Entropy is a Key Factor: An Analysis of Parallel Context Encoding with Full-attention-based Pre-trained Language Models Zhisong Zhang Yan Wang Xinting Huang Tianqing Fang Han Zhang Chenlong Deng Shuaiyi Li Dong Yu 150 6 0 21 Dec 2024
BRIEF: Bridging Retrieval and Inference for Multi-hop Reasoning via Compression Yuankai Li Jia-Chen Gu Di Wu Kai-Wei Chang Nanyun Peng RALM MQ 72 0 0 20 Oct 2024
HART: Efficient Visual Generation with Hybrid Autoregressive Transformer Haotian Tang Yecheng Wu Shang Yang Enze Xie Junsong Chen Junyu Chen Zhuoyang Zhang Han Cai Yaojie Lu Song Han 222 48 0 14 Oct 2024
ELICIT: LLM Augmentation via External In-Context Capability Futing Wang Jianhao Yan Yue Zhang Tao Lin 137 1 0 12 Oct 2024
Selective Attention Improves Transformer Yaniv Leviathan Matan Kalman Yossi Matias 119 12 0 03 Oct 2024
Locret: Enhancing Eviction in Long-Context LLM Inference with Trained Retaining Heads on Consumer-Grade Devices Yuxiang Huang Binhang Yuan Xu Han Chaojun Xiao Zhiyuan Liu RALM 168 3 0 02 Oct 2024
UIO-LLMs: Unbiased Incremental Optimization for Long-Context LLMs Wenhao Li Mingbao Lin Mingliang Xu Shuicheng Yan Rongrong Ji 71 0 0 26 Jun 2024
Leveraging Passage Embeddings for Efficient Listwise Reranking with Large Language Models Qi Liu Bo Wang Nan Wang Jiaxin Mao RALM 140 4 0 21 Jun 2024
DeciMamba: Exploring the Length Extrapolation Potential of Mamba Assaf Ben-Kish Itamar Zimerman Shady Abu Hussein Nadav Cohen Amir Globerson Lior Wolf Raja Giryes Mamba 206 19 0 20 Jun 2024
VoCo-LLaMA: Towards Vision Compression with Large Language Models Xubing Ye Yukang Gan Xiaoke Huang Yixiao Ge Yansong Tang MLLM VLM 130 28 0 18 Jun 2024
In-Context Editing: Learning Knowledge from Self-Induced Distributions Siyuan Qi Bangcheng Yang Kailin Jiang Xiaobo Wang Jiaqi Li Yifan Zhong Yaodong Yang Zilong Zheng KELM 183 10 0 17 Jun 2024
Lifelong Knowledge Editing for LLMs with Retrieval-Augmented Continuous Prompt Learning Qizhou Chen Taolin Zhang Xiaofeng He Dongyang Li Chengyu Wang Longtao Huang Hui Xue CLL KELM 112 15 0 06 May 2024
A Survey on Large Language Model-Based Game Agents Sihao Hu Tiansheng Huang Gaowen Liu Ramana Rao Kompella Gaowen Liu Selim Furkan Tekin Yichang Xu Zachary Yahn Ling Liu LLMAG LM&Ro AI4CE LM&MA 226 57 0 02 Apr 2024
CAMELoT: Towards Large Language Models with Training-Free Consolidated Associative Memory Zexue He Leonid Karlinsky Donghyun Kim Julian McAuley Dmitry Krotov Rogerio Feris KELM RALM 84 11 0 21 Feb 2024
DoraemonGPT: Toward Understanding Dynamic Scenes with Large Language Models (Exemplified as A Video Agent) Zongxin Yang Guikun Chen Xiaodi Li Wenguan Wang Yi Yang LM&Ro LLMAG 181 41 0 16 Jan 2024
Fewer is More: Boosting LLM Reasoning with Reinforced Context Pruning Xijie Huang Li Lyna Zhang Kwang-Ting Cheng Fan Yang Mao Yang LRM ReLM 94 13 0 14 Dec 2023
Function Vectors in Large Language Models Eric Todd Millicent Li Arnab Sen Sharma Aaron Mueller Byron C. Wallace David Bau 57 124 0 23 Oct 2023
HyperDreamBooth: HyperNetworks for Fast Personalization of Text-to-Image Models Nataniel Ruiz Yuanzhen Li Varun Jampani Wei Wei Tingbo Hou Yael Pritch Neal Wadhwa Michael Rubinstein Kfir Aberman DiffM 101 183 0 13 Jul 2023
H $_2$ O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models Zhenyu Zhang Ying Sheng Dinesh Manocha Tianlong Chen Lianmin Zheng ... Yuandong Tian Christopher Ré Clark W. Barrett Zhangyang Wang Beidi Chen VLM 180 314 0 24 Jun 2023
Identifying Informational Sources in News Articles Alexander Spangher Nanyun Peng Jonathan May Emilio Ferrara 62 5 0 24 May 2023
Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection Kai Greshake Sahar Abdelnabi Shailesh Mishra C. Endres Thorsten Holz Mario Fritz SILM 186 503 0 23 Feb 2023
Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors Thomas Hartvigsen S. Sankaranarayanan Hamid Palangi Yoon Kim Marzyeh Ghassemi KELM 155 177 0 20 Nov 2022