Generalization through Memorization: Nearest Neighbor Language Models

1 November 2019

Urvashi Khandelwal

Omer Levy

Dan Jurafsky

Luke Zettlemoyer

M. Lewis

RALM

ArXiv PDF HTML

Papers citing "Generalization through Memorization: Nearest Neighbor Language Models"

50 / 582 papers shown

Title
Non-Parametric Online Learning from Human Feedback for Neural Machine Translation Dongqi Wang Hao-Ran Wei Zhirui Zhang Shujian Huang Jun Xie Jiajun Chen OffRL 57 15 0 23 Sep 2021
RETRONLU: Retrieval Augmented Task-Oriented Semantic Parsing Vivek Gupta Akshat Shrivastava Adithya Sagar Armen Aghajanyan Denis Savenkov RALM 40 21 0 21 Sep 2021
ConvFiT: Conversational Fine-Tuning of Pretrained Language Models Ivan Vulić Pei-hao Su Sam Coope D. Gerz Paweł Budzianowski I. Casanueva Nikola Mrkvsić Tsung-Hsien Wen 27 36 0 21 Sep 2021
Regularized Training of Nearest Neighbor Language Models Jean-François Ton Walter A. Talbott Shuangfei Zhai J. Susskind RALM 25 3 0 16 Sep 2021
Remember the context! ASR slot error correction through memorization Dhanush Bekal Ashish Shenoy Monica Sunkara S. Bodapati Katrin Kirchhoff KELM 23 12 0 10 Sep 2021
Efficient Nearest Neighbor Language Models Junxian He Graham Neubig Taylor Berg-Kirkpatrick RALM 195 103 0 09 Sep 2021
Nearest Neighbour Few-Shot Learning for Cross-lingual Classification M Saiful Bari Batool Haider Saab Mansour VLM 19 13 0 06 Sep 2021
Combining Transformers with Natural Language Explanations Federico Ruggeri Marco Lippi Paolo Torroni 25 1 0 02 Sep 2021
$$\infty$-former: Infinite Memory Transformer$ $\infty$ -former: Infinite Memory Transformer Pedro Henrique Martins Zita Marinho André F. T. Martins 36 11 0 01 Sep 2021
Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation Ofir Press Noah A. Smith M. Lewis 253 701 0 27 Aug 2021
Towards Continual Entity Learning in Language Models for Conversational Agents R. Gadde I. Bulyko KELM 27 1 0 30 Jul 2021
Pointer Value Retrieval: A new benchmark for understanding the limits of neural network generalization Chiyuan Zhang M. Raghu Jon M. Kleinberg Samy Bengio OOD 32 30 0 27 Jul 2021
Internet-Augmented Dialogue Generation M. Komeili Kurt Shuster Jason Weston RALM 244 281 0 15 Jul 2021
On Training Instance Selection for Few-Shot Neural Text Generation Ernie Chang Xiaoyu Shen Hui-Syuan Yeh Vera Demberg 38 40 0 07 Jul 2021
Ascent Similarity Caching with Approximate Indexes T. Si Salem Giovanni Neglia D. Carra 20 7 0 02 Jul 2021
Memorization and Generalization in Neural Code Intelligence Models Md Rafiqul Islam Rabin Aftab Hussain Mohammad Amin Alipour Vincent J. Hellendoorn TDI 43 40 0 16 Jun 2021
End-to-End Training of Multi-Document Reader and Retriever for Open-Domain Question Answering Devendra Singh Sachan Siva Reddy William L. Hamilton Chris Dyer Dani Yogatama OOD RALM 37 160 0 09 Jun 2021
Improving Automated Evaluation of Open Domain Dialog via Diverse Reference Augmentation Varun Gangal Harsh Jhamtani Eduard H. Hovy Taylor Berg-Kirkpatrick 22 8 0 05 Jun 2021
Ember: No-Code Context Enrichment via Similarity-Based Keyless Joins S. Suri Ihab F. Ilyas Christopher Ré Theodoros Rekatsinas 33 21 0 02 Jun 2021
MOLEMAN: Mention-Only Linking of Entities with a Mention Annotation Network Nicholas FitzGerald Jan A. Botha D. Gillick Daniel M. Bikel Tom Kwiatkowski Andrew McCallum 37 15 0 02 Jun 2021
Fast Nearest Neighbor Machine Translation Yuxian Meng Xiaoya Li Xiayu Zheng Fei Wu Xiaofei Sun Tianwei Zhang Jiwei Li LRM 19 49 0 30 May 2021
Good for Misconceived Reasons: An Empirical Revisiting on the Need for Visual Context in Multimodal Machine Translation Zhiyong Wu Lingpeng Kong W. Bi Xiang Li B. Kao LRM 23 77 0 30 May 2021
Towards mental time travel: a hierarchical memory for reinforcement learning agents Andrew Kyle Lampinen Stephanie C. Y. Chan Andrea Banino Felix Hill 24 47 0 28 May 2021
Not Far Away, Not So Close: Sample Efficient Nearest Neighbour Data Augmentation via MiniMax Ehsan Kamalloo Mehdi Rezagholizadeh Peyman Passban Ali Ghodsi AAML 20 17 0 28 May 2021
Adaptive Nearest Neighbor Machine Translation Xin Zheng Zhirui Zhang Junliang Guo Shujian Huang Boxing Chen Weihua Luo Jiajun Chen 24 94 0 27 May 2021
Neural Machine Translation with Monolingual Translation Memory Deng Cai Yan Wang Huayang Li Wai Lam Lemao Liu 27 101 0 24 May 2021
Retrieval-Augmented Transformer-XL for Close-Domain Dialog Generation Giovanni Bonetta R. Cancelliere Ding Liu Paul Vozila RALM 24 16 0 19 May 2021
RetGen: A Joint framework for Retrieval and Grounded Text Generation Modeling Yizhe Zhang Siqi Sun Xiang Gao Yuwei Fang Chris Brockett Michel Galley Jianfeng Gao Bill Dolan RALM 38 30 0 14 May 2021
Paraphrastic Representations at Scale John Wieting Kevin Gimpel Graham Neubig Taylor Berg-Kirkpatrick 24 19 0 30 Apr 2021
Case-based Reasoning for Natural Language Queries over Knowledge Bases Rajarshi Das Manzil Zaheer Dung Ngoc Thai Ameya Godbole Ethan Perez Jay Yoon Lee Lizhen Tan L. Polymenakos Andrew McCallum 36 163 0 18 Apr 2021
Go Forth and Prosper: Language Modeling with Ancient Textual History Rik Koncel-Kedziorski Noah A. Smith KELM 14 0 0 18 Apr 2021
Generating Related Work Darsh J. Shah Regina Barzilay 36 3 0 18 Apr 2021
Cross-Modal Retrieval Augmentation for Multi-Modal Classification Shir Gur Natalia Neverova C. Stauffer Ser-Nam Lim Douwe Kiela A. Reiter 22 26 0 16 Apr 2021
Retrieval Augmentation Reduces Hallucination in Conversation Kurt Shuster Spencer Poff Moya Chen Douwe Kiela Jason Weston HILM 58 696 0 15 Apr 2021
Few-shot Intent Classification and Slot Filling with Retrieved Examples Dian Yu Luheng He Yuan Zhang Xinya Du Panupong Pasupat Qi Li VLM 33 50 0 12 Apr 2021
Lookup-Table Recurrent Language Models for Long Tail Speech Recognition Yifan Jiang Tara N. Sainath Cal Peyser Shankar Kumar David Rybach Trevor Strohman RALM LMTD 36 5 0 09 Apr 2021
Revisiting Simple Neural Probabilistic Language Models Simeng Sun Mohit Iyyer 24 14 0 08 Apr 2021
Perspective, Survey and Trends: Public Driving Datasets and Toolsets for Autonomous Driving Virtual Test Pengliang Ji Li Ruan Yunzhi Xue Limin Xiao Qian Dong 39 8 0 01 Apr 2021
A Neighbourhood Framework for Resource-Lean Content Flagging Sheikh Muhammad Sarwar Dimitrina Zlatkova Momchil Hardalov Yoan Dinkov Isabelle Augenstein Preslav Nakov 24 5 0 31 Mar 2021
BASE Layers: Simplifying Training of Large, Sparse Models M. Lewis Shruti Bhosale Tim Dettmers Naman Goyal Luke Zettlemoyer MoE 47 274 0 30 Mar 2021
Structure Inducing Pre-Training Matthew B. A. McDermott Brendan Yap Peter Szolovits Marinka Zitnik 42 18 0 18 Mar 2021
Retrieval Augmentation for Deep Neural Networks R. Ramos Patrícia Pereira Helena Moniz Joao Paulo Carvalho Bruno Martins VLM 19 0 0 25 Feb 2021
When Attention Meets Fast Recurrence: Training Language Models with Reduced Compute Tao Lei RALM VLM 59 47 0 24 Feb 2021
Leveraging Reinforcement Learning for evaluating Robustness of KNN Search Algorithms Pramod Vadiraja Christoph Balada OOD 29 1 0 10 Feb 2021
Adaptive Semiparametric Language Models Dani Yogatama Cyprien de Masson dÁutume Lingpeng Kong KELM RALM 43 99 0 04 Feb 2021
Mind the Gap: Assessing Temporal Generalization in Neural Language Models Angeliki Lazaridou A. Kuncoro E. Gribovskaya Devang Agrawal Adam Liska ... Sebastian Ruder Dani Yogatama Kris Cao Susannah Young Phil Blunsom VLM 41 207 0 03 Feb 2021
CNN with large memory layers R. Karimov Yury Malkov Karim Iskakov Victor Lempitsky 27 0 0 27 Jan 2021
Data-to-text Generation by Splicing Together Nearest Neighbors Sam Wiseman A. Backurs K. Stratos 27 9 0 20 Jan 2021
Diagnostic Captioning: A Survey John Pavlopoulos Vasiliki Kougia Ion Androutsopoulos D. Papamichail 3DV MedIm 91 26 0 18 Jan 2021
What Makes Good In-Context Examples for GPT- $3$ ? Jiachang Liu Dinghan Shen Yizhe Zhang Bill Dolan Lawrence Carin Weizhu Chen AAML RALM 275 1,315 0 17 Jan 2021