Generalization through Memorization: Nearest Neighbor Language Models

1 November 2019

Urvashi Khandelwal

Omer Levy

Dan Jurafsky

Luke Zettlemoyer

M. Lewis

RALM

ArXiv PDF HTML

Papers citing "Generalization through Memorization: Nearest Neighbor Language Models"

32 / 582 papers shown

Title
Subformer: Exploring Weight Sharing for Parameter Efficiency in Generative Transformers Machel Reid Edison Marrese-Taylor Y. Matsuo MoE 19 48 0 01 Jan 2021
Shortformer: Better Language Modeling using Shorter Inputs Ofir Press Noah A. Smith M. Lewis 230 89 0 31 Dec 2020
FastIF: Scalable Influence Functions for Efficient Model Interpretation and Debugging Han Guo Nazneen Rajani Peter Hase Joey Tianyi Zhou Caiming Xiong TDI 41 102 0 31 Dec 2020
Modifying Memories in Transformer Models Chen Zhu A. S. Rawat Manzil Zaheer Srinadh Bhojanapalli Daliang Li Felix X. Yu Sanjiv Kumar KELM 32 192 0 01 Dec 2020
Cross-Domain Generalization Through Memorization: A Study of Nearest Neighbors in Neural Duplicate Question Detection Yadollah Yaghoobzadeh Alexandre Rochette Timothy J. Hazen OOD 14 1 0 22 Nov 2020
Language Models are Open Knowledge Graphs Chenguang Wang Xiao Liu D. Song SSL KELM 26 135 0 22 Oct 2020
Limitations of Autoregressive Models and Their Alternatives Chu-cheng Lin Aaron Jaech Xin Li Matthew R. Gormley Jason Eisner 29 58 0 22 Oct 2020
Explaining and Improving Model Behavior with k Nearest Neighbor Representations Nazneen Rajani Ben Krause Wengpeng Yin Tong Niu R. Socher Caiming Xiong FAtt 19 32 0 18 Oct 2020
Example-Driven Intent Prediction with Observers Shikib Mehri Mihail Eric 33 39 0 17 Oct 2020
Large Product Key Memory for Pretrained Language Models Gyuwan Kim Tae-Hwan Jung VLM KELM 23 2 0 08 Oct 2020
Learning to Recombine and Resample Data for Compositional Generalization Ekin Akyürek Afra Feyza Akyürek Jacob Andreas 29 79 0 08 Oct 2020
Efficient Meta Lifelong-Learning with Limited Memory Zirui Wang Sanket Vaibhav Mehta Barnabás Póczós J. Carbonell CLL KELM 27 74 0 06 Oct 2020
Nearest Neighbor Machine Translation Urvashi Khandelwal Angela Fan Dan Jurafsky Luke Zettlemoyer M. Lewis RALM 18 282 0 01 Oct 2020
Case-Based Abductive Natural Language Inference Marco Valentino Mokanarangan Thayaparan André Freitas 23 5 0 30 Sep 2020
Controllable Text Generation with Focused Variation Lei Shu Alexandros Papangelis Yi-Chia Wang Gokhan Tur Hu Xu Zhaleh Feizollahi Bing-Quan Liu Piero Molino 19 10 0 25 Sep 2020
Grounded Compositional Outputs for Adaptive Language Modeling Nikolaos Pappas Phoebe Mulcaire Noah A. Smith KELM 33 7 0 24 Sep 2020
Taking Notes on the Fly Helps BERT Pre-training Qiyu Wu Chen Xing Yatao Li Guolin Ke Di He Tie-Yan Liu 13 10 0 04 Aug 2020
Neural Language Generation: Formulation, Methods, and Evaluation Cristina Garbacea Qiaozhu Mei 45 30 0 31 Jul 2020
Neural Composition: Learning to Generate from Multiple Models Denis Filimonov R. Gadde Ariya Rastrow 26 3 0 10 Jul 2020
Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval Lee Xiong Chenyan Xiong Ye Li Kwok-Fung Tang Jialin Liu Paul N. Bennett Junaid Ahmed Arnold Overwijk 25 1,182 0 01 Jul 2020
Learning Sparse Prototypes for Text Generation Junxian He Taylor Berg-Kirkpatrick Graham Neubig 27 23 0 29 Jun 2020
Train and You'll Miss It: Interactive Model Iteration with Weak Supervision and Pre-Trained Embeddings Mayee F. Chen Daniel Y. Fu Frederic Sala Sen Wu Ravi Teja Mullapudi Fait Poms Kayvon Fatahalian Christopher Ré 27 10 0 26 Jun 2020
Pre-training via Paraphrasing M. Lewis Marjan Ghazvininejad Gargi Ghosh Armen Aghajanyan Sida I. Wang Luke Zettlemoyer AIMat 30 159 0 26 Jun 2020
A Simple Approach to Case-Based Reasoning in Knowledge Bases Rajarshi Das Ameya Godbole S. Dhuliawala Manzil Zaheer Andrew McCallum 21 24 0 25 Jun 2020
Cross-lingual Retrieval for Iterative Self-Supervised Training C. Tran Y. Tang Xian Li Jiatao Gu RALM 28 73 0 16 Jun 2020
BERT-kNN: Adding a kNN Search Component to Pretrained Language Models for Better QA Nora Kassner Hinrich Schütze RALM 24 68 0 02 May 2020
Augmenting Transformers with KNN-Based Composite Memory for Dialogue Angela Fan Claire Gardent Chloé Braud Antoine Bordes RALM 47 75 0 27 Apr 2020
Don't Stop Pretraining: Adapt Language Models to Domains and Tasks Suchin Gururangan Ana Marasović Swabha Swayamdipta Kyle Lo Iz Beltagy Doug Downey Noah A. Smith VLM AI4CE CLL 75 2,360 0 23 Apr 2020
Exemplar VAE: Linking Generative Models, Nearest Neighbor Retrieval, and Data Augmentation Sajad Norouzi David J. Fleet Mohammad Norouzi VLM DRL 16 3 0 09 Apr 2020
REALM: Retrieval-Augmented Language Model Pre-Training Kelvin Guu Kenton Lee Zora Tung Panupong Pasupat Ming-Wei Chang RALM 48 2,006 0 10 Feb 2020
Improving Transformer Models by Reordering their Sublayers Ofir Press Noah A. Smith Omer Levy 22 87 0 10 Nov 2019
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism M. Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 245 1,836 0 17 Sep 2019