Retaining Key Information under High Compression Ratios: Query-Guided
Compressor for LLMs

v1v2 (latest)

Retaining Key Information under High Compression Ratios: Query-Guided Compressor for LLMs

4 June 2024

Yu Lu

ArXiv (abs)PDF HTML

Papers citing "Retaining Key Information under High Compression Ratios: Query-Guided Compressor for LLMs"

13 / 13 papers shown

Title
Rethinking Memory in AI: Taxonomy, Operations, Topics, and Future Directions Yiming Du Wenyu Huang Danna Zheng Zhaowei Wang Sébastien Montella Mirella Lapata Kam-Fai Wong Jeff Z. Pan KELM MU 223 5 0 01 May 2025
BRIEF: Bridging Retrieval and Inference for Multi-hop Reasoning via Compression Yuankai Li Jia-Chen Gu Di Wu Kai-Wei Chang Nanyun Peng RALM MQ 63 0 0 20 Oct 2024
RECOMP: Improving Retrieval-Augmented LMs with Compression and Selective Augmentation Fangyuan Xu Weijia Shi Eunsol Choi RALM 97 165 0 06 Oct 2023
RAVEN: In-Context Learning with Retrieval-Augmented Encoder-Decoder Language Models Jie Huang Ming-Yu Liu Peng Xu Mohammad Shoeybi Kevin Chen-Chuan Chang Bryan Catanzaro RALM 107 35 0 15 Aug 2023
Unlimiformer: Long-Range Transformers with Unlimited Length Input Amanda Bertsch Uri Alon Graham Neubig Matthew R. Gormley RALM 197 129 0 02 May 2023
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 888 13,207 0 04 Mar 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 850 9,714 0 28 Jan 2022
Unsupervised Dense Information Retrieval with Contrastive Learning Gautier Izacard Mathilde Caron Lucas Hosseini Sebastian Riedel Piotr Bojanowski Armand Joulin Edouard Grave RALM 221 923 0 16 Dec 2021
MoEfication: Transformer Feed-forward Layers are Mixtures of Experts Zhengyan Zhang Yankai Lin Zhiyuan Liu Peng Li Maosong Sun Jie Zhou MoE 96 128 0 05 Oct 2021
Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation Ofir Press Noah A. Smith M. Lewis 339 775 0 27 Aug 2021
Rider: Reader-Guided Passage Reranking for Open-Domain Question Answering Yuning Mao Pengcheng He Xiaodong Liu Yelong Shen Jianfeng Gao Jiawei Han Weizhu Chen OOD LRM 183 37 0 01 Jan 2021
Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation Nils Reimers Iryna Gurevych 104 1,030 0 21 Apr 2020
HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering Zhilin Yang Peng Qi Saizheng Zhang Yoshua Bengio William W. Cohen Ruslan Salakhutdinov Christopher D. Manning RALM 206 2,700 0 25 Sep 2018