Generalization through Memorization: Nearest Neighbor Language Models

1 November 2019

Urvashi Khandelwal

Omer Levy

Dan Jurafsky

Luke Zettlemoyer

M. Lewis

RALM

ArXiv PDF HTML

Papers citing "Generalization through Memorization: Nearest Neighbor Language Models"

50 / 582 papers shown

Title
Nonparametric Masked Language Modeling Sewon Min Weijia Shi M. Lewis Xilun Chen Wen-tau Yih Hannaneh Hajishirzi Luke Zettlemoyer RALM 50 48 0 02 Dec 2022
Data-Efficient Finetuning Using Cross-Task Nearest Neighbors Hamish Ivison Noah A. Smith Hannaneh Hajishirzi Pradeep Dasigi 38 20 0 01 Dec 2022
Task-Specific Embeddings for Ante-Hoc Explainable Text Classification Kishaloy Halder Josip Krapac Alan Akbik Anthony Brew Matti Lyra 34 0 0 30 Nov 2022
Retrieval-Augmented Multimodal Language Modeling Michihiro Yasunaga Armen Aghajanyan Weijia Shi Rich James J. Leskovec Percy Liang M. Lewis Luke Zettlemoyer Wen-tau Yih RALM 22 95 0 22 Nov 2022
Token Turing Machines Michael S. Ryoo K. Gopalakrishnan Kumara Kahatapitiya Ted Xiao Kanishka Rao Austin Stone Yao Lu Julian Ibarz Anurag Arnab 27 21 0 16 Nov 2022
Error-Robust Retrieval for Chinese Spelling Check Xunjian Yin Xinyu Hu Jin Jiang Xiao-Yi Wan 35 3 0 15 Nov 2022
A Survey of Knowledge Enhanced Pre-trained Language Models Linmei Hu Zeyi Liu Ziwang Zhao Lei Hou Liqiang Nie Juanzi Li KELM VLM 28 124 0 11 Nov 2022
Suffix Retrieval-Augmented Language Modeling Zecheng Wang Yik-Cheung Tam RALM 18 1 0 06 Nov 2022
Exemplar Guided Deep Neural Network for Spatial Transcriptomics Analysis of Gene Expression Prediction Yan Yang Md Zakir Hossain Eric A. Stone Shafin Rahman AI4TS 31 15 0 30 Oct 2022
Knowledge-in-Context: Towards Knowledgeable Semi-Parametric Language Models Xiaoman Pan Wenlin Yao Hongming Zhang Dian Yu Dong Yu Jianshu Chen KELM 201 23 0 28 Oct 2022
You can't pick your neighbors, or can you? When and how to rely on retrieval in the $k$ NN-LM Andrew Drozdov Shufan Wang Razieh Rahimi Andrew McCallum Hamed Zamani Mohit Iyyer RALM 122 17 0 28 Oct 2022
Nearest Neighbor Language Models for Stylistic Controllable Generation Severino Trotta Lucie Flek Charles F Welch 31 4 0 27 Oct 2022
EUREKA: EUphemism Recognition Enhanced through Knn-based methods and Augmentation Sedrick Scott Keh Rohit K Bharadwaj Emmy Liu Simone Tedeschi Varun Gangal Roberto Navigli 23 7 0 23 Oct 2022
Generative Knowledge Graph Construction: A Review Hongbin Ye Ningyu Zhang Hui Chen Huajun Chen 56 70 0 23 Oct 2022
Cross-domain Generalization for AMR Parsing Xuefeng Bai Sen Yang Leyang Cui Linfeng Song Yue Zhang 49 2 0 22 Oct 2022
Enhancing Tabular Reasoning with Pattern Exploiting Training Abhilash Shankarampeta Vivek Gupta Shuo Zhang LMTD RALM ReLM 68 6 0 21 Oct 2022
Rescue Implicit and Long-tail Cases: Nearest Neighbor Relation Extraction Zhen Wan Qianying Liu Zhuoyuan Mao Fei Cheng Sadao Kurohashi Jiwei Li 35 8 0 21 Oct 2022
Schema-aware Reference as Prompt Improves Data-Efficient Knowledge Graph Construction Yunzhi Yao Shengyu Mao Ningyu Zhang Xiangnan Chen Shumin Deng Xi Chen Huajun Chen 28 9 0 19 Oct 2022
Transferring Knowledge via Neighborhood-Aware Optimal Transport for Low-Resource Hate Speech Detection Tulika Bose Irina Illina Dominique Fohr 29 0 0 17 Oct 2022
RARR: Researching and Revising What Language Models Say, Using Language Models Luyu Gao Zhuyun Dai Panupong Pasupat Anthony Chen Arun Tejasvi Chaganty ... Vincent Zhao Ni Lao Hongrae Lee Da-Cheng Juan Kelvin Guu HILM KELM 41 258 0 17 Oct 2022
Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey Sachin Kumar Vidhisha Balachandran Lucille Njoo Antonios Anastasopoulos Yulia Tsvetkov ELM 81 86 0 14 Oct 2022
Decoupled Context Processing for Context Augmented Language Modeling Zonglin Li Ruiqi Guo Surinder Kumar RALM KELM 21 23 0 11 Oct 2022
Measuring and Narrowing the Compositionality Gap in Language Models Ofir Press Muru Zhang Sewon Min Ludwig Schmidt Noah A. Smith M. Lewis ReLM KELM LRM 54 565 0 07 Oct 2022
MuRAG: Multimodal Retrieval-Augmented Generator for Open Question Answering over Images and Text Wenhu Chen Hexiang Hu Xi Chen Pat Verga William W. Cohen RALM 16 145 0 06 Oct 2022
Nonparametric Decoding for Generative Retrieval Hyunji Lee Jaeyoung Kim Hoyeon Chang Hanseok Oh Sohee Yang Vladimir Karpukhin Yi Lu Minjoon Seo RALM 19 5 0 05 Oct 2022
Memory in humans and deep language models: Linking hypotheses for model augmentation Omri Raccah Pheobe Chen Ted Willke David Poeppel Vy A. Vo RALM 23 1 0 04 Oct 2022
Zemi: Learning Zero-Shot Semi-Parametric Language Models from Multiple Tasks Zhenhailong Wang Xiaoman Pan Dian Yu Dong Yu Jianshu Chen Heng Ji VLM 46 9 0 01 Oct 2022
Re-Imagen: Retrieval-Augmented Text-to-Image Generator Wenhu Chen Hexiang Hu Chitwan Saharia William W. Cohen VLM 131 164 0 29 Sep 2022
Non-Parametric Temporal Adaptation for Social Media Topic Classification Fatemehsadat Mireshghallah Nikolai Vogler Junxian He Omar U. Florez Ahmed El-Kishky Taylor Berg-Kirkpatrick TTA 24 0 0 13 Sep 2022
A Review of Sparse Expert Models in Deep Learning W. Fedus J. Dean Barret Zoph MoE 25 144 0 04 Sep 2022
Efficient Methods for Natural Language Processing: A Survey Marcos Vinícius Treviso Ji-Ung Lee Tianchu Ji Betty van Aken Qingqing Cao ... Emma Strubell Niranjan Balasubramanian Leon Derczynski Iryna Gurevych Roy Schwartz 35 109 0 31 Aug 2022
Fusing Sentence Embeddings Into LSTM-based Autoregressive Language Models Vilém Zouhar Marius Mosbach Dietrich Klakow 29 1 0 04 Aug 2022
Retrieval-Augmented Transformer for Image Captioning Sara Sarto Marcella Cornia Lorenzo Baraldi Rita Cucchiara 29 57 0 26 Jul 2022
Text-Guided Synthesis of Artistic Images with Retrieval-Augmented Diffusion Models Robin Rombach A. Blattmann Bjorn Ommer DiffM 23 70 0 26 Jul 2022
MQRetNN: Multi-Horizon Time Series Forecasting with Retrieval Augmentation Sitan Yang Carson Eisenach Dhruv Madeka AI4TS 35 7 0 21 Jul 2022
Tip-Adapter: Training-free Adaption of CLIP for Few-shot Classification Renrui Zhang Zhang Wei Rongyao Fang Peng Gao Kunchang Li Jifeng Dai Yu Qiao Hongsheng Li VLM 35 299 0 19 Jul 2022
N-Grammer: Augmenting Transformers with latent n-grams Aurko Roy Rohan Anil Guangda Lai Benjamin Lee Jeffrey Zhao ... Yu Phuong Dao Christopher Fifty Zhehuai Chen Yonghui Wu 29 7 0 13 Jul 2022
Repository-Level Prompt Generation for Large Language Models of Code Disha Shrivastava Hugo Larochelle Daniel Tarlow 30 137 0 26 Jun 2022
Memory-Based Model Editing at Scale E. Mitchell Charles Lin Antoine Bosselut Christopher D. Manning Chelsea Finn KELM 37 324 0 13 Jun 2022
Annotation Error Detection: Analyzing the Past and Present for a More Coherent Future Jan-Christoph Klie Bonnie Webber Iryna Gurevych 42 43 0 05 Jun 2022
Dict-TTS: Learning to Pronounce with Prior Dictionary Knowledge for Text-to-Speech Ziyue Jiang Zhe Su Zhou Zhao Qian Yang Yi Ren Jinglin Liu Zhe Ye 26 4 0 05 Jun 2022
Decoupling Knowledge from Memorization: Retrieval-augmented Prompt Learning Xiang Chen Lei Li Ningyu Zhang Xiaozhuan Liang Shumin Deng Chuanqi Tan Fei Huang Luo Si Huajun Chen VLM 30 52 0 29 May 2022
kNN-Prompt: Nearest Neighbor Zero-Shot Inference Weijia Shi Julian Michael Suchin Gururangan Luke Zettlemoyer RALM VLM 29 32 0 27 May 2022
Tranception: protein fitness prediction with autoregressive transformers and inference-time retrieval Pascal Notin M. Dias J. Frazer Javier Marchena-Hurtado Aidan Gomez D. Marks Y. Gal 60 180 0 27 May 2022
Training Language Models with Memory Augmentation Zexuan Zhong Tao Lei Danqi Chen RALM 249 128 0 25 May 2022
ORCA: Interpreting Prompted Language Models via Locating Supporting Data Evidence in the Ocean of Pretraining Data Xiaochuang Han Yulia Tsvetkov 24 29 0 25 May 2022
Chunk-based Nearest Neighbor Machine Translation Pedro Henrique Martins Zita Marinho André F.T. Martins RALM 90 28 0 24 May 2022
StreamingQA: A Benchmark for Adaptation to New Knowledge over Time in Question Answering Models Adam Livska Tomávs Kovciský E. Gribovskaya Tayfun Terzi Eren Sezener ... Susannah Young Ellen Gilsenan-McMahon Sophia Austin Phil Blunsom Angeliki Lazaridou KELM 240 92 0 23 May 2022
Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models Kushal Tirumala Aram H. Markosyan Luke Zettlemoyer Armen Aghajanyan TDI 31 187 0 22 May 2022
RankGen: Improving Text Generation with Large Ranking Models Kalpesh Krishna Yapei Chang John Wieting Mohit Iyyer AIMat 24 68 0 19 May 2022