v1v2 (latest)

Generalization through Memorization: Nearest Neighbor Language Models

1 November 2019

Urvashi Khandelwal

Omer Levy

Dan Jurafsky

Luke Zettlemoyer

M. Lewis

RALM

ArXiv (abs)PDF HTML

Papers citing "Generalization through Memorization: Nearest Neighbor Language Models"

50 / 597 papers shown

Title
How much do language models copy from their training data? Evaluating linguistic novelty in text generation using RAVEN R. Thomas McCoy P. Smolensky Tal Linzen Jianfeng Gao Asli Celikyilmaz SyDa 98 124 0 18 Nov 2021
Personalized Federated Learning through Local Memorization Othmane Marfoq Giovanni Neglia Laetitia Kameni Richard Vidal FedML 128 95 0 17 Nov 2021
NeurInt : Learning to Interpolate through Neural ODEs Avinandan Bose Aniket Das Yatin Dandi P. Rai DiffM DRL 55 0 0 07 Nov 2021
Tip-Adapter: Training-free CLIP-Adapter for Better Vision-Language Modeling Renrui Zhang Rongyao Fang Wei Zhang Peng Gao Kunchang Li Jifeng Dai Yu Qiao Hongsheng Li VLM 292 405 0 06 Nov 2021
GNN-LM: Language Modeling based on Global Contexts via GNN Yuxian Meng Shi Zong Xiaoya Li Xiaofei Sun Tianwei Zhang Leilei Gan Jiwei Li LRM 127 39 0 17 Oct 2021
Controllable Semantic Parsing via Retrieval Augmentation Panupong Pasupat Yuan Zhang Kelvin Guu 206 48 0 16 Oct 2021
Hindsight: Posterior-guided training of retrievers for improved open-ended generation Ashwin Paranjape Omar Khattab Christopher Potts Matei A. Zaharia Christopher D. Manning RALM 126 44 0 14 Oct 2021
The Inductive Bias of In-Context Learning: Rethinking Pretraining Example Design Yoav Levine Noam Wies Daniel Jannai D. Navon Yedid Hoshen Amnon Shashua AI4CE 116 37 0 09 Oct 2021
Capturing Structural Locality in Non-parametric Language Models Frank F. Xu Junxian He Graham Neubig Vincent J. Hellendoorn 112 14 0 06 Oct 2021
How BPE Affects Memorization in Transformers Eugene Kharitonov Marco Baroni Dieuwke Hupkes 247 33 0 06 Oct 2021
KNN-BERT: Fine-Tuning Pre-Trained Models with KNN Classifier Linyang Li Demin Song Ruotian Ma Xipeng Qiu Xuanjing Huang 110 21 0 06 Oct 2021
Data Augmentation Approaches in Natural Language Processing: A Survey Bohan Li Yutai Hou Wanxiang Che 219 284 0 05 Oct 2021
VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understanding Hu Xu Gargi Ghosh Po-Yao (Bernie) Huang Dmytro Okhonko Armen Aghajanyan Florian Metze Luke Zettlemoyer Florian Metze Luke Zettlemoyer Christoph Feichtenhofer CLIP VLM 335 584 0 28 Sep 2021
Instance-Based Neural Dependency Parsing Hiroki Ouchi Jun Suzuki Sosuke Kobayashi Sho Yokoi Tatsuki Kuribayashi Masashi Yoshikawa Kentaro Inui 69 3 0 28 Sep 2021
Non-Parametric Online Learning from Human Feedback for Neural Machine Translation Dongqi Wang Hao-Ran Wei Zhirui Zhang Shujian Huang Jun Xie Jiajun Chen OffRL 107 15 0 23 Sep 2021
RETRONLU: Retrieval Augmented Task-Oriented Semantic Parsing Vivek Gupta Akshat Shrivastava Adithya Sagar Armen Aghajanyan Denis Savenkov RALM 87 23 0 21 Sep 2021
ConvFiT: Conversational Fine-Tuning of Pretrained Language Models Ivan Vulić Pei-hao Su Sam Coope D. Gerz Paweł Budzianowski I. Casanueva Nikola Mrkvsić Tsung-Hsien Wen 103 37 0 21 Sep 2021
Regularized Training of Nearest Neighbor Language Models Jean-François Ton Walter A. Talbott Shuangfei Zhai J. Susskind RALM 27 3 0 16 Sep 2021
Remember the context! ASR slot error correction through memorization Dhanush Bekal Ashish Shenoy Monica Sunkara S. Bodapati Katrin Kirchhoff KELM 57 12 0 10 Sep 2021
Efficient Nearest Neighbor Language Models Junxian He Graham Neubig Taylor Berg-Kirkpatrick RALM 278 106 0 09 Sep 2021
Nearest Neighbour Few-Shot Learning for Cross-lingual Classification M Saiful Bari Batool Haider Saab Mansour VLM 41 14 0 06 Sep 2021
Combining Transformers with Natural Language Explanations Federico Ruggeri Marco Lippi Paolo Torroni 93 1 0 02 Sep 2021
$$\infty$-former: Infinite Memory Transformer$ $\infty$ -former: Infinite Memory Transformer Pedro Henrique Martins Zita Marinho André F. T. Martins 111 11 0 01 Sep 2021
Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation Ofir Press Noah A. Smith M. Lewis 352 779 0 27 Aug 2021
Towards Continual Entity Learning in Language Models for Conversational Agents R. Gadde I. Bulyko KELM 76 1 0 30 Jul 2021
Pointer Value Retrieval: A new benchmark for understanding the limits of neural network generalization Chiyuan Zhang M. Raghu Jon M. Kleinberg Samy Bengio OOD 111 32 0 27 Jul 2021
Internet-Augmented Dialogue Generation M. Komeili Kurt Shuster Jason Weston RALM 332 291 0 15 Jul 2021
On Training Instance Selection for Few-Shot Neural Text Generation Ernie Chang Xiaoyu Shen Hui-Syuan Yeh Vera Demberg 88 42 0 07 Jul 2021
Ascent Similarity Caching with Approximate Indexes T. Si Salem Giovanni Neglia D. Carra 47 7 0 02 Jul 2021
Memorization and Generalization in Neural Code Intelligence Models Md Rafiqul Islam Rabin Aftab Hussain Mohammad Amin Alipour Vincent J. Hellendoorn TDI 87 43 0 16 Jun 2021
End-to-End Training of Multi-Document Reader and Retriever for Open-Domain Question Answering Devendra Singh Sachan Siva Reddy William L. Hamilton Chris Dyer Dani Yogatama OOD RALM 99 170 0 09 Jun 2021
Improving Automated Evaluation of Open Domain Dialog via Diverse Reference Augmentation Varun Gangal Harsh Jhamtani Eduard H. Hovy Taylor Berg-Kirkpatrick 57 9 0 05 Jun 2021
Ember: No-Code Context Enrichment via Similarity-Based Keyless Joins S. Suri Ihab F. Ilyas Christopher Ré Theodoros Rekatsinas 51 22 0 02 Jun 2021
MOLEMAN: Mention-Only Linking of Entities with a Mention Annotation Network Nicholas FitzGerald Jan A. Botha D. Gillick Daniel M. Bikel Tom Kwiatkowski Andrew McCallum 86 15 0 02 Jun 2021
Fast Nearest Neighbor Machine Translation Yuxian Meng Xiaoya Li Xiayu Zheng Leilei Gan Xiaofei Sun Tianwei Zhang Jiwei Li LRM 85 49 0 30 May 2021
Good for Misconceived Reasons: An Empirical Revisiting on the Need for Visual Context in Multimodal Machine Translation Zhiyong Wu Lingpeng Kong W. Bi Xiang Li B. Kao LRM 74 81 0 30 May 2021
Towards mental time travel: a hierarchical memory for reinforcement learning agents Andrew Kyle Lampinen Stephanie C. Y. Chan Andrea Banino Felix Hill 96 47 0 28 May 2021
Not Far Away, Not So Close: Sample Efficient Nearest Neighbour Data Augmentation via MiniMax Ehsan Kamalloo Mehdi Rezagholizadeh Peyman Passban Ali Ghodsi AAML 71 17 0 28 May 2021
Adaptive Nearest Neighbor Machine Translation Xin Zheng Zhirui Zhang Junliang Guo Shujian Huang Boxing Chen Weihua Luo Jiajun Chen 79 95 0 27 May 2021
Neural Machine Translation with Monolingual Translation Memory Deng Cai Yan Wang Huayang Li Wai Lam Lemao Liu 99 103 0 24 May 2021
Retrieval-Augmented Transformer-XL for Close-Domain Dialog Generation Giovanni Bonetta R. Cancelliere Ding Liu Paul Vozila RALM 30 17 0 19 May 2021
RetGen: A Joint framework for Retrieval and Grounded Text Generation Modeling Yizhe Zhang Siqi Sun Xiang Gao Yuwei Fang Chris Brockett Michel Galley Jianfeng Gao Bill Dolan RALM 112 34 0 14 May 2021
Paraphrastic Representations at Scale John Wieting Kevin Gimpel Graham Neubig Taylor Berg-Kirkpatrick 139 19 0 30 Apr 2021
Case-based Reasoning for Natural Language Queries over Knowledge Bases Rajarshi Das Manzil Zaheer Dung Ngoc Thai Ameya Godbole Ethan Perez Jay Yoon Lee Lizhen Tan L. Polymenakos Andrew McCallum 111 168 0 18 Apr 2021
Go Forth and Prosper: Language Modeling with Ancient Textual History Rik Koncel-Kedziorski Noah A. Smith KELM 40 0 0 18 Apr 2021
Generating Related Work Darsh J. Shah Regina Barzilay 67 3 0 18 Apr 2021
Cross-Modal Retrieval Augmentation for Multi-Modal Classification Shir Gur Natalia Neverova C. Stauffer Ser-Nam Lim Douwe Kiela A. Reiter 147 30 0 16 Apr 2021
Retrieval Augmentation Reduces Hallucination in Conversation Kurt Shuster Spencer Poff Moya Chen Douwe Kiela Jason Weston HILM 127 753 0 15 Apr 2021
Few-shot Intent Classification and Slot Filling with Retrieved Examples Dian Yu Luheng He Yuan Zhang Xinya Du Panupong Pasupat Qi Li VLM 69 54 0 12 Apr 2021
Lookup-Table Recurrent Language Models for Long Tail Speech Recognition Wenjie Huang Tara N. Sainath Cal Peyser Shankar Kumar David Rybach Trevor Strohman RALM LMTD 97 6 0 09 Apr 2021