Generalization through Memorization: Nearest Neighbor Language Models

1 November 2019

Urvashi Khandelwal

Omer Levy

Dan Jurafsky

Luke Zettlemoyer

M. Lewis

RALM

ArXiv PDF HTML

Papers citing "Generalization through Memorization: Nearest Neighbor Language Models"

50 / 582 papers shown

Title
ProtIR: Iterative Refinement between Retrievers and Predictors for Protein Function Annotation Zuobai Zhang Jiarui Lu Vijil Chenthamarakshan Aurélie C. Lozano Payel Das Jian Tang 26 1 0 10 Feb 2024
Let Your Graph Do the Talking: Encoding Structured Data for LLMs Bryan Perozzi Bahare Fatemi Dustin Zelle Anton Tsitsulin Mehran Kazemi Rami Al-Rfou Jonathan J. Halcrow GNN 42 57 0 08 Feb 2024
MEMORYLLM: Towards Self-Updatable Large Language Models Yu-Xiang Wang Yifan Gao Xiusi Chen Haoming Jiang Shiyang Li ... Zheng Li Xian Li Bing Yin Jingbo Shang Julian McAuley KELM 35 17 0 07 Feb 2024
InfLLM: Training-Free Long-Context Extrapolation for LLMs with an Efficient Context Memory Chaojun Xiao Pengle Zhang Xu Han Guangxuan Xiao Yankai Lin Zhengyan Zhang Zhiyuan Liu Maosong Sun LLMAG 47 35 0 07 Feb 2024
Training Language Models to Generate Text with Citations via Fine-grained Rewards Chengyu Huang Zeqiu Wu Yushi Hu Wenya Wang HILM LRM 79 27 0 06 Feb 2024
Retrieve to Explain: Evidence-driven Predictions with Language Models Ravi Patel Angus Brayne Rogier E Hintzen Daniel Jaroslawicz Georgiana Neculae Dane S. Corneil 24 2 0 06 Feb 2024
LV-Eval: A Balanced Long-Context Benchmark with 5 Length Levels Up to 256K Tao Yuan Xuefei Ning Dong Zhou Zhijie Yang Shiyao Li ... Dahua Lin Boxun Li Guohao Dai Shengen Yan Yu-Xiang Wang ALM 38 34 0 06 Feb 2024
UniMem: Towards a Unified View of Long-Context Large Language Models Junjie Fang Likai Tang Hongzhe Bi Yujia Qin Si Sun ... Xiaodong Shi Sen Song Yankai Lin Zhiyuan Liu Maosong Sun 27 3 0 05 Feb 2024
A Survey on Transformer Compression Yehui Tang Yunhe Wang Jianyuan Guo Zhijun Tu Kai Han Hailin Hu Dacheng Tao 41 29 0 05 Feb 2024
Efficient Prompt Caching via Embedding Similarity Hanlin Zhu Banghua Zhu Jiantao Jiao RALM 28 9 0 02 Feb 2024
Non-Exchangeable Conformal Language Generation with Nearest Neighbors Dennis Ulmer Chrysoula Zerva André F. T. Martins 38 11 0 01 Feb 2024
When Large Language Models Meet Vector Databases: A Survey Zhi Jing Yongye Su Yikun Han Bo Yuan Haiyun Xu Chunjiang Liu Kehai Chen Min Zhang 61 36 0 30 Jan 2024
NNOSE: Nearest Neighbor Occupational Skill Extraction Mike Zhang Rob van der Goot Min-Yen Kan Barbara Plank 37 3 0 30 Jan 2024
Infini-gram: Scaling Unbounded n-gram Language Models to a Trillion Tokens Jiacheng Liu Sewon Min Luke Zettlemoyer Yejin Choi Hannaneh Hajishirzi 51 52 0 30 Jan 2024
Accelerating Retrieval-Augmented Language Model Serving with Speculation Zhihao Zhang Alan Zhu Lijie Yang Yihua Xu Lanting Li P. Phothilimthana Zhihao Jia RALM KELM 56 16 0 25 Jan 2024
Automated Fact-Checking of Climate Change Claims with Large Language Models Markus Leippold S. Vaghefi Dominik Stammbach V. Muccione J. Bingler ... Tobias Schimanski Glen Gostlow Ting Yu Juerg Luterbacher C. Huggel 28 9 0 23 Jan 2024
Efficient Data Shapley for Weighted Nearest Neighbor Algorithms Jiachen T. Wang Prateek Mittal Ruoxi Jia TDI 51 5 0 20 Jan 2024
Communication-Efficient Personalized Federated Learning for Speech-to-Text Tasks Yichao Du Zhirui Zhang Linan Yue Xu Huang Yuqing Zhang Tong Xu Linli Xu Enhong Chen FedML 75 5 0 18 Jan 2024
Leveraging Biases in Large Language Models: "bias-kNN'' for Effective Few-Shot Learning Yong Zhang Hanzhang Li Zhitao Li Ning Cheng Ming Li Jing Xiao Jianzong Wang 28 3 0 18 Jan 2024
The Faiss library Matthijs Douze Alexandr Guzhva Chengqi Deng Jeff Johnson Gergely Szilvasy Pierre-Emmanuel Mazaré Maria Lomeli Lucas Hosseini Hervé Jégou 44 147 0 16 Jan 2024
The What, Why, and How of Context Length Extension Techniques in Large Language Models -- A Detailed Survey Saurav Pawar S.M. Towhidul Islam Tonmoy S. M. M. Zaman Vinija Jain Aman Chadha Amitava Das 37 28 0 15 Jan 2024
Bridging the Preference Gap between Retrievers and LLMs Zixuan Ke Weize Kong Cheng-rong Li Mingyang Zhang Qiaozhu Mei Michael Bendersky RALM 28 29 0 13 Jan 2024
Human-AI Collaborative Essay Scoring: A Dual-Process Framework with LLMs Changrong Xiao Wenxing Ma Qingping Song Sean Xin Xu Kunpeng Zhang Yufang Wang Qi Fu AI4Ed 29 16 0 12 Jan 2024
High-precision Voice Search Query Correction via Retrievable Speech-text Embedings Christopher Li Gary Wang Kyle Kastner Heng Su Allen Chen ... Zelin Wu L. Velikovich Pat Rondon D. Caseiro Petar S. Aleksic 34 1 0 08 Jan 2024
CaMML: Context-Aware Multimodal Learner for Large Models Yixin Chen Shuai Zhang Boran Han Tong He Bo Li VLM 32 4 0 06 Jan 2024
ReFusion: Improving Natural Language Understanding with Computation-Efficient Retrieval Representation Fusion Shangyu Wu Ying Xiong Yufei Cui Xue Liu Buzhou Tang Tei-Wei Kuo Chun Jason Xue 37 2 0 04 Jan 2024
A Comprehensive Study of Knowledge Editing for Large Language Models Ningyu Zhang Yunzhi Yao Bo Tian Peng Wang Shumin Deng ... Lei Liang Qing Cui Xiao-Jun Zhu Jun Zhou Huajun Chen KELM 55 77 0 02 Jan 2024
Adapting Large Language Models for Education: Foundational Capabilities, Potentials, and Challenges Qingyao Li Lingyue Fu Weiming Zhang Xianyu Chen Jingwei Yu Wei Xia Weinan Zhang Ruiming Tang Yong Yu AI4Ed ELM 46 18 0 27 Dec 2023
kNN-CTC: Enhancing ASR via Retrieval of CTC Pseudo Labels Jiaming Zhou Shiwan Zhao Yaqi Liu Wenjia Zeng Yong Chen Yong Qin 37 9 0 21 Dec 2023
Cached Transformers: Improving Transformers with Differentiable Memory Cache Zhaoyang Zhang Wenqi Shao Yixiao Ge Xiaogang Wang Liang Feng Ping Luo 19 2 0 20 Dec 2023
Retrieval-Augmented Generation for Large Language Models: A Survey Yunfan Gao Yun Xiong Xinyu Gao Kangxiang Jia Jinliu Pan Yuxi Bi Yi Dai Jiawei Sun Meng Wang Haofen Wang 3DV RALM 64 1,552 1 18 Dec 2023
kNN-ICL: Compositional Task-Oriented Parsing Generalization with Nearest Neighbor In-Context Learning Wenting Zhao Ye Liu Yao Wan Yibo Wang Qingyang Wu Zhongfen Deng Jiangshu Du Shuaiqi Liu Yunlong Xu Philip S. Yu 52 7 0 17 Dec 2023
DRAFT: Dense Retrieval Augmented Few-shot Topic classifier Framework Keonwoo Kim Younggun Lee VLM 36 0 0 05 Dec 2023
PEFA: Parameter-Free Adapters for Large-scale Embedding-based Retrieval Models Wei-Cheng Chang Jyun-Yu Jiang Jiong Zhang Mutasem Al-Darabsah C. Teo Cho-Jui Hsieh Hsiang-Fu Yu S. Vishwanathan RALM 33 2 0 05 Dec 2023
Advancing State of the Art in Language Modeling David Herel Tomáš Mikolov 34 1 0 28 Nov 2023
Rethinking Privacy in Machine Learning Pipelines from an Information Flow Control Perspective Lukas Wutschitz Boris Köpf Andrew J. Paverd Saravan Rajmohan Ahmed Salem Shruti Tople Santiago Zanella Béguelin Menglin Xia Victor Rühle 48 13 0 27 Nov 2023
Augmenting Unsupervised Reinforcement Learning with Self-Reference Andrew Zhao Erle Zhu Rui Lu Matthieu Lin Yong-Jin Liu Gao Huang SSL 34 1 0 16 Nov 2023
On Retrieval Augmentation and the Limitations of Language Model Training Ting-Rui Chiang Xinyan Velocity Yu Joshua Robinson Ollie Liu Isabelle G. Lee Dani Yogatama RALM 28 0 0 16 Nov 2023
Chain-of-Note: Enhancing Robustness in Retrieval-Augmented Language Models Wenhao Yu Hongming Zhang Xiaoman Pan Kaixin Ma Hongwei Wang Dong Yu KELM RALM LRM 70 105 0 15 Nov 2023
PEMA: An Offsite-Tunable Plug-in External Memory Adaptation for Language Models HyunJin Kim Young Jin Kim Jinyeong Bak KELM 14 1 0 14 Nov 2023
Learning to Filter Context for Retrieval-Augmented Generation Zhiruo Wang Jun Araki Zhengbao Jiang Md. Rizwan Parvez Graham Neubig RALM 42 50 0 14 Nov 2023
REST: Retrieval-Based Speculative Decoding Zhenyu He Zexuan Zhong Tianle Cai Jason D. Lee Di He RALM 28 80 0 14 Nov 2023
MetaReVision: Meta-Learning with Retrieval for Visually Grounded Compositional Concept Acquisition Guangyue Xu Parisa Kordjamshidi Joyce Chai 27 2 0 02 Nov 2023
Learn to Refuse: Making Large Language Models More Controllable and Reliable through Knowledge Scope Limitation and Refusal Mechanism Lang Cao 26 13 0 02 Nov 2023
Style Locality for Controllable Generation with kNN Language Models Gilles Nawezi Lucie Flek Charles F Welch RALM 24 0 0 01 Nov 2023
Defining a New NLP Playground Sha Li Chi Han Pengfei Yu Carl Edwards Manling Li ... Yi R. Fung Charles Yu Joel R. Tetreault Eduard H. Hovy Heng Ji 41 5 0 31 Oct 2023
DIVKNOWQA: Assessing the Reasoning Ability of LLMs via Open-Domain Question Answering over Knowledge Base and Text Wenting Zhao Ye Liu Tong Niu Yao Wan Philip S. Yu Chenyu You Yingbo Zhou Semih Yavuz LRM 27 6 0 31 Oct 2023
Knowledge Corpus Error in Question Answering Yejoon Lee Philhoon Oh James Thorne 26 2 0 27 Oct 2023
Automatically generating Riddles aiding Concept Attainment Niharika Sri Parasa Chaitali Diwan Srinath Srinivasa 21 2 0 27 Oct 2023
Nearest Neighbor Search over Vectorized Lexico-Syntactic Patterns for Relation Extraction from Financial Documents P. Rajpoot Ankur P. Parikh 34 0 0 26 Oct 2023