Generalization through Memorization: Nearest Neighbor Language Models

1 November 2019

Urvashi Khandelwal

Omer Levy

Dan Jurafsky

Luke Zettlemoyer

M. Lewis

RALM

ArXiv PDF HTML

Papers citing "Generalization through Memorization: Nearest Neighbor Language Models"

50 / 582 papers shown

Title
Long-range Language Modeling with Self-retrieval Ohad Rubin Jonathan Berant RALM KELM 36 18 0 23 Jun 2023
RepoFusion: Training Code Models to Understand Your Repository Disha Shrivastava Denis Kocetkov H. D. Vries Dzmitry Bahdanau Torsten Scholak 89 28 0 19 Jun 2023
Guiding Language Models of Code with Global Context using Monitors Lakshya A Agrawal Aditya Kanade Navin Goyal Shuvendu K. Lahiri S. Rajamani 40 23 0 19 Jun 2023
GLIMMER: generalized late-interaction memory reranker Michiel de Jong Yury Zemlyanskiy Nicholas FitzGerald Sumit Sanghai William W. Cohen Joshua Ainslie RALM 51 4 0 17 Jun 2023
Neural Priming for Sample-Efficient Adaptation Matthew Wallingford Vivek Ramanujan Alex Fang Aditya Kusupati Roozbeh Mottaghi Aniruddha Kembhavi Ludwig Schmidt Ali Farhadi VLM 110 13 0 16 Jun 2023
Encyclopedic VQA: Visual questions about detailed properties of fine-grained categories Thomas Mensink J. Uijlings Lluis Castrejon A. Goel Felipe Cadar Howard Zhou Fei Sha A. Araújo V. Ferrari 42 38 0 15 Jun 2023
Retrieval-Enhanced Contrastive Vision-Text Models Ahmet Iscen Mathilde Caron Alireza Fathi Cordelia Schmid CLIP VLM 31 26 0 12 Jun 2023
Retrosynthesis Prediction with Local Template Retrieval Shufang Xie Rui Yan Junliang Guo Yingce Xia Lijun Wu Tao Qin 31 12 0 07 Jun 2023
Information Flow Control in Machine Learning through Modular Model Architecture Trishita Tiwari Suchin Gururangan Chuan Guo Weizhe Hua Sanjay Kariyappa Udit Gupta Wenjie Xiong Kiwan Maeng Hsien-Hsin S. Lee G. E. Suh 26 6 0 05 Jun 2023
Retrieval-Enhanced Visual Prompt Learning for Few-shot Classification Jintao Rong Hao Chen Tianrun Chen Linlin Ou Xinyi Yu Yifan Liu VLM VPVLM 15 6 0 04 Jun 2023
The Information Pathways Hypothesis: Transformers are Dynamic Self-Ensembles Md Shamim Hussain Mohammed J Zaki D. Subramanian 45 3 0 02 Jun 2023
Exposing Attention Glitches with Flip-Flop Language Modeling Bingbin Liu Jordan T. Ash Surbhi Goel A. Krishnamurthy Cyril Zhang LRM 35 47 0 01 Jun 2023
LMCap: Few-shot Multilingual Image Captioning by Retrieval Augmented Language Model Prompting R. Ramos Bruno Martins Desmond Elliott VLM 18 16 0 31 May 2023
Information Association for Language Model Updating by Mitigating LM-Logical Discrepancy Pengfei Yu Heng Ji KELM 44 9 0 29 May 2023
Test-Time Training on Nearest Neighbors for Large Language Models Moritz Hardt Yu Sun VLM RALM 35 21 0 29 May 2023
Prompt-Guided Retrieval Augmentation for Non-Knowledge-Intensive Tasks Zhicheng Guo Sijie Cheng Yile Wang Peng Li Yang Liu RALM 30 18 0 28 May 2023
Landmark Attention: Random-Access Infinite Context Length for Transformers Amirkeivan Mohtashami Martin Jaggi LLMAG 27 150 0 25 May 2023
Surface-Based Retrieval Reduces Perplexity of Retrieval-Augmented Language Models Ehsan Doostmohammadi Tobias Norlund Marco Kuhlmann Richard Johansson RALM 35 8 0 25 May 2023
Enhancing Grammatical Error Correction Systems with Explanations Yuejiao Fei Leyang Cui Sen Yang Wai Lam Zhenzhong Lan Shuming Shi 28 13 0 25 May 2023
Aligning Language Models to User Opinions EunJeong Hwang Bodhisattwa Prasad Majumder Niket Tandon 29 62 0 24 May 2023
Privacy Implications of Retrieval-Based Language Models Yangsibo Huang Samyak Gupta Zexuan Zhong Keqin Li Danqi Chen RALM 33 29 0 24 May 2023
Adapting Language Models to Compress Contexts Alexis Chevalier Alexander Wettig Anirudh Ajith Danqi Chen LLMAG 16 175 0 24 May 2023
Trusting Your Evidence: Hallucinate Less with Context-aware Decoding Weijia Shi Xiaochuang Han M. Lewis Yulia Tsvetkov Luke Zettlemoyer Scott Yih HILM 27 192 0 24 May 2023
Enabling Large Language Models to Generate Text with Citations Tianyu Gao Howard Yen Jiatong Yu Danqi Chen LM&MA HILM 40 319 0 24 May 2023
KNN-LM Does Not Improve Open-ended Text Generation Shufan Wang Yixiao Song Andrew Drozdov Aparna Garimella Varun Manjunatha Mohit Iyyer RALM 50 8 0 24 May 2023
Accessing Higher Dimensions for Unsupervised Word Translation Sida I. Wang 30 0 0 23 May 2023
To Copy Rather Than Memorize: A Vertical Learning Paradigm for Knowledge Graph Completion Rui Li Xu Chen Chaozhuo Li Yanming Shen Jianan Zhao ... Weihao Han Hao Sun Weiwei Deng Qi Zhang Xing Xie 26 7 0 23 May 2023
Non-parametric, Nearest-neighbor-assisted Fine-tuning for Neural Machine Translation Jiayi Wang Ke Min Wang Yuqi Zhang Yu Zhao Pontus Stenetorp 21 0 0 23 May 2023
Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflicts Jian Xie Kai Zhang Jiangjie Chen Renze Lou Yu-Chuan Su RALM 227 159 0 22 May 2023
LM vs LM: Detecting Factual Errors via Cross Examination Roi Cohen May Hamri Mor Geva Amir Globerson HILM 41 120 0 22 May 2023
Nearest Neighbor Machine Translation is Meta-Optimizer on Output Projection Layer R. Gao Zhirui Zhang Yichao Du Lemao Liu Rui Wang 29 2 0 22 May 2023
Has It All Been Solved? Open NLP Research Questions Not Solved by Large Language Models Oana Ignat Zhijing Jin Artem Abzaliev Laura Biester Santiago Castro ... Verónica Pérez-Rosas Siqi Shen Zekun Wang Winston Wu Rada Mihalcea LRM 41 6 0 21 May 2023
Description-Based Text Similarity Shauli Ravfogel Valentina Pyatkin Amir D. N. Cohen Avshalom Manevich Yoav Goldberg 28 5 0 21 May 2023
Complex Claim Verification with Evidence Retrieved in the Wild Jifan Chen Grace Kim Aniruddh Sriram Greg Durrett Eunsol Choi HILM 41 72 0 19 May 2023
CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing Zhibin Gou Zhihong Shao Yeyun Gong Yelong Shen Yujiu Yang Nan Duan Weizhu Chen KELM LRM 36 360 0 19 May 2023
Decouple knowledge from parameters for plug-and-play language modeling Xin Cheng Yankai Lin Preslav Nakov Dongyan Zhao Rui Yan KELM 32 2 0 19 May 2023
ReGen: Zero-Shot Text Classification via Training Data Generation with Progressive Dense Retrieval Yue Yu Yuchen Zhuang Rongzhi Zhang Yu Meng Jiaming Shen Chao Zhang VLM 43 33 0 18 May 2023
Knowledge Card: Filling LLMs' Knowledge Gaps with Plug-in Specialized Language Models Shangbin Feng Weijia Shi Yuyang Bai Vidhisha Balachandran Tianxing He Yulia Tsvetkov KELM 55 31 0 17 May 2023
Memorization for Good: Encryption with Autoregressive Language Models Samuel Stevens Yung-Chun Su 28 0 0 15 May 2023
Synergistic Interplay between Search and Large Language Models for Information Retrieval Jiazhan Feng Chongyang Tao Xiubo Geng Tao Shen Can Xu Guodong Long Dongyan Zhao Daxin Jiang KELM 65 5 0 12 May 2023
Active Retrieval Augmented Generation Zhengbao Jiang Frank F. Xu Luyu Gao Zhiqing Sun Qian Liu Jane Dwivedi-Yu Yiming Yang Jamie Callan Graham Neubig RALM 30 256 0 11 May 2023
HistAlign: Improving Context Dependency in Language Generation by Aligning with History David Wan Shiyue Zhang Joey Tianyi Zhou AI4TS 40 6 0 08 May 2023
DAMO-NLP at SemEval-2023 Task 2: A Unified Retrieval-augmented System for Multilingual Named Entity Recognition Zeqi Tan Shen Huang Zixia Jia Jiong Cai Hai-Tao Zheng ... Yueting Zhuang Kewei Tu Pengjun Xie Fei Huang Yong-jia Jiang 49 8 0 05 May 2023
Now It Sounds Like You: Learning Personalized Vocabulary On Device Sida Wang Ashish Shenoy P. Chuang John Nguyen VLM 41 2 0 05 May 2023
Re $^3$ Dial: Retrieve, Reorganize and Rescale Dialogue Corpus for Long-Turn Open-Domain Dialogue Pre-training Jiaxin Wen Hao Zhou Jian Guan Minlie Huang 36 0 0 04 May 2023
Lift Yourself Up: Retrieval-augmented Text Generation with Self Memory Xin Cheng Di Luo Preslav Nakov Lemao Liu Dongyan Zhao Rui Yan RALM 153 91 0 03 May 2023
Unlimiformer: Long-Range Transformers with Unlimited Length Input Amanda Bertsch Uri Alon Graham Neubig Matthew R. Gormley RALM 116 123 0 02 May 2023
Speak, Memory: An Archaeology of Books Known to ChatGPT/GPT-4 Kent K. Chang Mackenzie Cramer Sandeep Soni David Bamman RALM 152 114 0 28 Apr 2023
Analogy-Forming Transformers for Few-Shot 3D Parsing N. Gkanatsios M. Singh Zhaoyuan Fang Shubham Tulsiani Katerina Fragkiadaki 3DPC 3DV 21 2 0 27 Apr 2023
Revisiting k-NN for Fine-tuning Pre-trained Language Models Lei Li Jing Chen Bo Tian Ning Zhang 29 1 0 18 Apr 2023