v1v2 (latest)

Generalization through Memorization: Nearest Neighbor Language Models

1 November 2019

Urvashi Khandelwal

Omer Levy

Dan Jurafsky

Luke Zettlemoyer

M. Lewis

RALM

ArXiv (abs)PDF HTML

Papers citing "Generalization through Memorization: Nearest Neighbor Language Models"

50 / 597 papers shown

Title
From One to Many: Expanding the Scope of Toxicity Mitigation in Language Models Luiza Amador Pozzobon Patrick Lewis Sara Hooker Beyza Ermis 103 12 0 06 Mar 2024
MeaCap: Memory-Augmented Zero-shot Image Captioning Zequn Zeng Yan Xie Hao Zhang Chiyu Chen Zhengjue Wang Boli Chen VLM 86 15 0 06 Mar 2024
Reliable, Adaptable, and Attributable Language Models with Retrieval Akari Asai Zexuan Zhong Danqi Chen Pang Wei Koh Luke Zettlemoyer Hanna Hajishirzi Wen-tau Yih KELM RALM 119 63 0 05 Mar 2024
Retrieval-Augmented Generation for AI-Generated Content: A Survey Penghao Zhao Hailin Zhang Qinhan Yu Zhengren Wang Yunteng Geng Fangcheng Fu Ling Yang Wentao Zhang Jie Jiang Tengjiao Wang 3DV 290 286 0 29 Feb 2024
Memory-Augmented Generative Adversarial Transformers Stephan Raaijmakers Roos Bakker Anita Cremers R. D. Kleijn Tom Kouwenhoven Tessa Verhoef 86 0 0 29 Feb 2024
Follow My Instruction and Spill the Beans: Scalable Data Extraction from Retrieval-Augmented Generation Systems Zhenting Qi Hanlin Zhang Eric Xing Sham Kakade Hima Lakkaraju SILM 128 25 0 27 Feb 2024
Retrieval is Accurate Generation Bowen Cao Deng Cai Leyang Cui Xuxin Cheng Wei Bi Yuexian Zou Shuming Shi 133 8 0 27 Feb 2024
The Good and The Bad: Exploring Privacy Issues in Retrieval-Augmented Generation (RAG) Shenglai Zeng Jiankun Zhang Pengfei He Yue Xing Yiding Liu ... Jie Ren Shuaiqiang Wang D. Yin Yi Chang Jiliang Tang SILM 109 84 0 23 Feb 2024
Transformer-based Causal Language Models Perform Clustering Xinbo Wu Lav Varshney 78 6 0 19 Feb 2024
Unveiling the Magic: Investigating Attention Distillation in Retrieval-augmented Generation Zizhong Li Haopeng Zhang Jiawei Zhang RALM 94 1 0 19 Feb 2024
Metacognitive Retrieval-Augmented Large Language Models Yujia Zhou Zheng Liu Jiajie Jin Jian-yun Nie Zhicheng Dou RALM KELM AIFin LRM 55 21 0 18 Feb 2024
EVEDIT: Event-based Knowledge Editing with Deductive Editing Boundaries Jiateng Liu Pengfei Yu Yuji Zhang Sha Li Zixuan Zhang Heng Ji KELM 84 17 0 17 Feb 2024
In Search of Needles in a 11M Haystack: Recurrent Memory Finds What LLMs Miss Yuri Kuratov Aydar Bulatov Petr Anokhin Dmitry Sorokin Artyom Sorokin Andrey Kravchenko RALM 182 35 0 16 Feb 2024
Grounding Language Model with Chunking-Free In-Context Retrieval Hongjin Qian Zheng Liu Kelong Mao Yujia Zhou Zhicheng Dou 100 20 0 15 Feb 2024
Nearest Neighbour Score Estimators for Diffusion Generative Models Matthew Niedoba Dylan Green Saeid Naderiparizi Vasileios Lioutas J. Lavington ... Ke Zhang Setareh Dabiri Adam Scibior Berend Zwartsenberg Frank Wood DiffM 87 2 0 12 Feb 2024
ProtIR: Iterative Refinement between Retrievers and Predictors for Protein Function Annotation Zuobai Zhang Jiarui Lu Vijil Chenthamarakshan Aurélie C. Lozano Payel Das Jian Tang 55 1 0 10 Feb 2024
Let Your Graph Do the Talking: Encoding Structured Data for LLMs Bryan Perozzi Bahare Fatemi Dustin Zelle Anton Tsitsulin Mehran Kazemi Rami Al-Rfou Jonathan J. Halcrow GNN 89 69 0 08 Feb 2024
MEMORYLLM: Towards Self-Updatable Large Language Models Yu Wang Yifan Gao Xiusi Chen Haoming Jiang Shiyang Li ... Zheng Li Xian Li Bing Yin Jingbo Shang Julian McAuley KELM 108 19 0 07 Feb 2024
InfLLM: Training-Free Long-Context Extrapolation for LLMs with an Efficient Context Memory Chaojun Xiao Pengle Zhang Xu Han Guangxuan Xiao Yankai Lin Zhengyan Zhang Zhiyuan Liu Maosong Sun LLMAG 132 55 0 07 Feb 2024
Training Language Models to Generate Text with Citations via Fine-grained Rewards Chengyu Huang Zeqiu Wu Yushi Hu Wenya Wang HILM LRM 138 30 0 06 Feb 2024
LV-Eval: A Balanced Long-Context Benchmark with 5 Length Levels Up to 256K Tao Yuan Xuefei Ning Dong Zhou Zhijie Yang Shiyao Li ... Dahua Lin Boxun Li Guohao Dai Shengen Yan Yu Wang ALM 97 47 0 06 Feb 2024
Retrieve to Explain: Evidence-driven Predictions for Explainable Drug Target Identification Ravi Patel Angus Brayne Rogier E Hintzen Daniel Jaroslawicz Georgiana Neculae Dane S. Corneil 81 2 0 06 Feb 2024
UniMem: Towards a Unified View of Long-Context Large Language Models Junjie Fang Likai Tang Hongzhe Bi Yujia Qin Si Sun ... Xiaodong Shi Sen Song Yankai Lin Zhiyuan Liu Maosong Sun 46 3 0 05 Feb 2024
A Survey on Transformer Compression Yehui Tang Yunhe Wang Jianyuan Guo Zhijun Tu Kai Han Hailin Hu Dacheng Tao 158 35 0 05 Feb 2024
Efficient Prompt Caching via Embedding Similarity Hanlin Zhu Banghua Zhu Jiantao Jiao RALM 86 9 0 02 Feb 2024
Non-Exchangeable Conformal Language Generation with Nearest Neighbors Dennis Ulmer Chrysoula Zerva André F. T. Martins 209 13 0 01 Feb 2024
NNOSE: Nearest Neighbor Occupational Skill Extraction Mike Zhang Rob van der Goot Min-Yen Kan Barbara Plank 94 3 0 30 Jan 2024
Infini-gram: Scaling Unbounded n-gram Language Models to a Trillion Tokens Jiacheng Liu Sewon Min Luke Zettlemoyer Yejin Choi Hannaneh Hajishirzi 142 60 0 30 Jan 2024
When Large Language Models Meet Vector Databases: A Survey Zhi Jing Yongye Su Yikun Han Bo Yuan Haiyun Xu Chunjiang Liu Kehai Chen Min Zhang 144 38 0 30 Jan 2024
Accelerating Retrieval-Augmented Language Model Serving with Speculation Zhihao Zhang Alan Zhu Lijie Yang Yihua Xu Lanting Li P. Phothilimthana Zhihao Jia RALM KELM 111 18 0 25 Jan 2024
Automated Fact-Checking of Climate Change Claims with Large Language Models Markus Leippold S. Vaghefi Dominik Stammbach V. Muccione J. Bingler ... Tobias Schimanski Glen Gostlow Ting Yu Juerg Luterbacher C. Huggel 54 9 0 23 Jan 2024
Efficient Data Shapley for Weighted Nearest Neighbor Algorithms Jiachen T. Wang Prateek Mittal Ruoxi Jia TDI 136 5 0 20 Jan 2024
Communication-Efficient Personalized Federated Learning for Speech-to-Text Tasks Yichao Du Zhirui Zhang Linan Yue Xu Huang Yuqing Zhang Tong Xu Linli Xu Enhong Chen FedML 150 7 0 18 Jan 2024
Leveraging Biases in Large Language Models: "bias-kNN'' for Effective Few-Shot Learning Yong Zhang Hanzhang Li Zhitao Li Ning Cheng Ming Li Jing Xiao Jianzong Wang 85 3 0 18 Jan 2024
The Faiss library Matthijs Douze Alexandr Guzhva Chengqi Deng Jeff Johnson Gergely Szilvasy Pierre-Emmanuel Mazaré Maria Lomeli Lucas Hosseini Hervé Jégou 212 189 0 16 Jan 2024
The What, Why, and How of Context Length Extension Techniques in Large Language Models -- A Detailed Survey Saurav Pawar S.M. Towhidul Islam Tonmoy S. M. M. Zaman Vinija Jain Aman Chadha Amitava Das 68 29 0 15 Jan 2024
Bridging the Preference Gap between Retrievers and LLMs Zixuan Ke Weize Kong Cheng-rong Li Mingyang Zhang Qiaozhu Mei Michael Bendersky RALM 94 37 0 13 Jan 2024
Human-AI Collaborative Essay Scoring: A Dual-Process Framework with LLMs Changrong Xiao Wenxing Ma Qingping Song Sean Xin Xu Kunpeng Zhang Yufang Wang Qi Fu AI4Ed 54 19 0 12 Jan 2024
High-precision Voice Search Query Correction via Retrievable Speech-text Embedings Christopher Li Gary Wang Kyle Kastner Heng Su Allen Chen ... Zelin Wu L. Velikovich Pat Rondon D. Caseiro Petar S. Aleksic 65 1 0 08 Jan 2024
CaMML: Context-Aware Multimodal Learner for Large Models Yixin Chen Shuai Zhang Boran Han Tong He Bo Li VLM 117 4 0 06 Jan 2024
ReFusion: Improving Natural Language Understanding with Computation-Efficient Retrieval Representation Fusion Shangyu Wu Ying Xiong Yufei Cui Xue Liu Buzhou Tang Tei-Wei Kuo Chun Jason Xue 96 2 0 04 Jan 2024
A Comprehensive Study of Knowledge Editing for Large Language Models Ningyu Zhang Yunzhi Yao Bo Tian Peng Wang Shumin Deng ... Lei Liang Qing Cui Xiao-Jun Zhu Jun Zhou Huajun Chen KELM 176 89 0 02 Jan 2024
Adapting Large Language Models for Education: Foundational Capabilities, Potentials, and Challenges Qingyao Li Lingyue Fu Weiming Zhang Xianyu Chen Jingwei Yu Wei Xia Weinan Zhang Ruiming Tang Yong Yu AI4Ed ELM 146 24 0 27 Dec 2023
kNN-CTC: Enhancing ASR via Retrieval of CTC Pseudo Labels Jiaming Zhou Shiwan Zhao Yaqi Liu Wenjia Zeng Yong Chen Yong Qin 96 11 0 21 Dec 2023
Cached Transformers: Improving Transformers with Differentiable Memory Cache Zhaoyang Zhang Wenqi Shao Yixiao Ge Xiaogang Wang Liang Feng Ping Luo 65 3 0 20 Dec 2023
Retrieval-Augmented Generation for Large Language Models: A Survey Yunfan Gao Yun Xiong Xinyu Gao Kangxiang Jia Jinliu Pan Yuxi Bi Yi Dai Jiawei Sun Meng Wang Haofen Wang 3DV RALM 344 1,846 1 18 Dec 2023
kNN-ICL: Compositional Task-Oriented Parsing Generalization with Nearest Neighbor In-Context Learning Wenting Zhao Ye Liu Yao Wan Yibo Wang Qingyang Wu Zhongfen Deng Jiangshu Du Shuaiqi Liu Yunlong Xu Philip S. Yu 76 8 0 17 Dec 2023
DRAFT: Dense Retrieval Augmented Few-shot Topic classifier Framework Keonwoo Kim Younggun Lee VLM 57 1 0 05 Dec 2023
PEFA: Parameter-Free Adapters for Large-scale Embedding-based Retrieval Models Wei-Cheng Chang Jyun-Yu Jiang Jiong Zhang Mutasem Al-Darabsah C. Teo Cho-Jui Hsieh Hsiang-Fu Yu S. Vishwanathan RALM 68 3 0 05 Dec 2023
Advancing State of the Art in Language Modeling David Herel Tomas Mikolov 110 1 0 28 Nov 2023