Generalization through Memorization: Nearest Neighbor Language Models

1 November 2019

Urvashi Khandelwal

Omer Levy

Dan Jurafsky

Luke Zettlemoyer

M. Lewis

RALM

ArXiv PDF HTML

Papers citing "Generalization through Memorization: Nearest Neighbor Language Models"

50 / 582 papers shown

Title
Long-term Control for Dialogue Generation: Methods and Evaluation Ramya Ramakrishnan H. Narangodage M. Schilman Kilian Q. Weinberger Ryan T. McDonald 27 8 0 15 May 2022
Near-Negative Distinction: Giving a Second Life to Human Evaluation Datasets Philippe Laban Chien-Sheng Wu Wenhao Liu Caiming Xiong 41 5 0 13 May 2022
kNN-Embed: Locally Smoothed Embedding Mixtures For Multi-interest Candidate Retrieval Ahmed El-Kishky Thomas Markovich Kenny Leung Frank Portman A. Haghighi Ying Xiao 16 12 0 12 May 2022
Few-shot Mining of Naturally Occurring Inputs and Outputs Mandar Joshi Terra Blevins M. Lewis Daniel S. Weld Luke Zettlemoyer 33 1 0 09 May 2022
Relation Extraction as Open-book Examination: Retrieval-enhanced Prompt Tuning Xiang Chen Lei Li Ningyu Zhang Chuanqi Tan Fei Huang Luo Si Huajun Chen RALM VLM 24 36 0 04 May 2022
Retrieval-Enhanced Machine Learning Hamed Zamani Fernando Diaz Mostafa Dehghani Donald Metzler Michael Bendersky 27 49 0 02 May 2022
Can deep learning match the efficiency of human visual long-term memory in storing object details? Emin Orhan VLM OCL 28 0 0 27 Apr 2022
Semi-Parametric Neural Image Synthesis A. Blattmann Robin Rombach Kaan Oktay Jonas Muller Bjorn Ommer DiffM 36 28 0 25 Apr 2022
METRO: Efficient Denoising Pretraining of Large Scale Autoencoding Language Models with Model Generated Signals Payal Bajaj Chenyan Xiong Guolin Ke Xiaodong Liu Di He Saurabh Tiwary Tie-Yan Liu Paul N. Bennett Xia Song Jianfeng Gao 52 32 0 13 Apr 2022
Augmenting Pre-trained Language Models with QA-Memory for Open-Domain Question Answering Wenhu Chen Pat Verga Michiel de Jong John Wieting William W. Cohen RALM KELM 37 26 0 10 Apr 2022
Knowledge Infused Decoding Ruibo Liu Guoqing Zheng Shashank Gupta Radhika Gaonkar Chongyang Gao Soroush Vosoughi Milad Shokouhi Ahmed Hassan Awadallah KELM 28 14 0 06 Apr 2022
Knowledge Base Index Compression via Dimensionality and Precision Reduction Vilém Zouhar Marius Mosbach Miaoran Zhang Dietrich Klakow 29 2 0 06 Apr 2022
$k$ NN-NER: Named Entity Recognition with Nearest Neighbor Search Shuhe Wang Xiaoya Li Yuxian Meng Tianwei Zhang Rongbin Ouyang Jiwei Li Guoyin Wang 34 11 0 31 Mar 2022
Shoring Up the Foundations: Fusing Model Embeddings and Weak Supervision Mayee F. Chen Daniel Y. Fu Dyah Adila Michael Zhang Frederic Sala Kayvon Fatahalian Christopher Ré 35 20 0 24 Mar 2022
Language Models that Seek for Knowledge: Modular Search & Generation for Dialogue and Prompt Completion Kurt Shuster M. Komeili Leonard Adolphs Stephen Roller Arthur Szlam Jason Weston KELM 45 122 0 24 Mar 2022
An Empirical Study of Memorization in NLP Xiaosen Zheng Jing Jiang TDI 17 24 1 23 Mar 2022
Better Language Model with Hypernym Class Prediction Richard He Bai Tong Wang Alessandro Sordoni Peng Shi 84 15 0 21 Mar 2022
Memorizing Transformers Yuhuai Wu M. Rabe DeLesley S. Hutchins Christian Szegedy RALM 30 173 0 16 Mar 2022
Interpretability for Language Learners Using Example-Based Grammatical Error Correction Masahiro Kaneko Sho Takase Ayana Niwa Naoaki Okazaki 41 26 0 14 Mar 2022
Internet-augmented language models through few-shot prompting for open-domain question answering Angeliki Lazaridou E. Gribovskaya Wojciech Stokowiec N. Grigorev KELM LRM 20 132 0 10 Mar 2022
Retrieval Augmented Classification for Long-Tail Visual Recognition Alex Long Wei Yin Thalaiyasingam Ajanthan Vu-Linh Nguyen Pulak Purkait Ravi Garg Alan Blair Chunhua Shen Anton Van Den Hengel 21 107 0 22 Feb 2022
Knowledge Base Question Answering by Case-based Reasoning over Subgraphs Rajarshi Das Ameya Godbole Ankita Rajaram Naik Elliot Tower Robin Jia Manzil Zaheer Hannaneh Hajishirzi Andrew McCallum 25 53 0 22 Feb 2022
What Does it Mean for a Language Model to Preserve Privacy? Hannah Brown Katherine Lee Fatemehsadat Mireshghallah Reza Shokri Florian Tramèr PILM 55 232 0 11 Feb 2022
A Survey on Retrieval-Augmented Text Generation Huayang Li Yixuan Su Deng Cai Yan Wang Lemao Liu RALM 69 198 0 02 Feb 2022
Neuro-Symbolic Language Modeling with Automaton-augmented Retrieval Uri Alon Frank F. Xu Junxian He Sudipta Sengupta Dan Roth Graham Neubig RALM 77 63 0 28 Jan 2022
Relational Memory Augmented Language Models Qi Liu Dani Yogatama Phil Blunsom KELM RALM 69 32 0 24 Jan 2022
Artefact Retrieval: Overview of NLP Models with Knowledge Base Access Vilém Zouhar Marius Mosbach Debanjali Biswas Dietrich Klakow KELM 34 4 0 24 Jan 2022
LaMDA: Language Models for Dialog Applications R. Thoppilan Daniel De Freitas Jamie Hall Noam M. Shazeer Apoorv Kulshreshtha ... Blaise Aguera-Arcas Claire Cui M. Croak Ed H. Chi Quoc Le ALM 50 1,561 0 20 Jan 2022
Memory-assisted prompt editing to improve GPT-3 after deployment Aman Madaan Niket Tandon Peter Clark Yiming Yang KELM 42 0 0 16 Jan 2022
Reasoning Through Memorization: Nearest Neighbor Knowledge Graph Embeddings Peng Wang Xin Xie Xiaohan Wang Ningyu Zhang RALM 35 16 0 14 Jan 2022
DREAM: Improving Situational QA by First Elaborating the Situation Yuling Gu Bhavana Dalvi Peter Clark 21 18 0 16 Dec 2021
Learning To Retrieve Prompts for In-Context Learning Ohad Rubin Jonathan Herzig Jonathan Berant VPVLM RALM 14 667 0 16 Dec 2021
Rethinking Nearest Neighbors for Visual Classification Menglin Jia Bor-Chun Chen Zuxuan Wu Claire Cardie Serge Belongie Ser-Nam Lim SSL 41 10 0 15 Dec 2021
Faster Nearest Neighbor Machine Translation Shuhe Wang Jiwei Li Yuxian Meng Rongbin Ouyang Guoyin Wang Xiaoya Li Tianwei Zhang Shi Zong 22 12 0 15 Dec 2021
Improving and Diagnosing Knowledge-Based Visual Question Answering via Entity Enhanced Knowledge Injection Diego Garcia-Olano Yasumasa Onoe Joydeep Ghosh 21 18 0 13 Dec 2021
Improving language models by retrieving from trillions of tokens Sebastian Borgeaud A. Mensch Jordan Hoffmann Trevor Cai Eliza Rutherford ... Simon Osindero Karen Simonyan Jack W. Rae Erich Elsen Laurent Sifre KELM RALM 90 1,027 0 08 Dec 2021
How much do language models copy from their training data? Evaluating linguistic novelty in text generation using RAVEN R. Thomas McCoy P. Smolensky Tal Linzen Jianfeng Gao Asli Celikyilmaz SyDa 25 119 0 18 Nov 2021
Personalized Federated Learning through Local Memorization Othmane Marfoq Giovanni Neglia Laetitia Kameni Richard Vidal FedML 43 88 0 17 Nov 2021
NeurInt : Learning to Interpolate through Neural ODEs Avinandan Bose Aniket Das Yatin Dandi P. Rai DiffM DRL 25 0 0 07 Nov 2021
Tip-Adapter: Training-free CLIP-Adapter for Better Vision-Language Modeling Renrui Zhang Rongyao Fang Wei Zhang Peng Gao Kunchang Li Jifeng Dai Yu Qiao Hongsheng Li VLM 194 387 0 06 Nov 2021
GNN-LM: Language Modeling based on Global Contexts via GNN Yuxian Meng Shi Zong Xiaoya Li Xiaofei Sun Tianwei Zhang Fei Wu Jiwei Li LRM 29 37 0 17 Oct 2021
Controllable Semantic Parsing via Retrieval Augmentation Panupong Pasupat Yuan Zhang Kelvin Guu 132 47 0 16 Oct 2021
Hindsight: Posterior-guided training of retrievers for improved open-ended generation Ashwin Paranjape Omar Khattab Christopher Potts Matei A. Zaharia Christopher D. Manning RALM 72 43 0 14 Oct 2021
The Inductive Bias of In-Context Learning: Rethinking Pretraining Example Design Yoav Levine Noam Wies Daniel Jannai D. Navon Yedid Hoshen Amnon Shashua AI4CE 35 36 0 09 Oct 2021
Capturing Structural Locality in Non-parametric Language Models Frank F. Xu Junxian He Graham Neubig Vincent J. Hellendoorn 27 14 0 06 Oct 2021
How BPE Affects Memorization in Transformers Eugene Kharitonov Marco Baroni Dieuwke Hupkes 177 32 0 06 Oct 2021
KNN-BERT: Fine-Tuning Pre-Trained Models with KNN Classifier Linyang Li Demin Song Ruotian Ma Xipeng Qiu Xuanjing Huang 29 21 0 06 Oct 2021
Data Augmentation Approaches in Natural Language Processing: A Survey Bohan Li Yutai Hou Wanxiang Che 132 274 0 05 Oct 2021
VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understanding Hu Xu Gargi Ghosh Po-Yao (Bernie) Huang Dmytro Okhonko Armen Aghajanyan Florian Metze Luke Zettlemoyer Florian Metze Luke Zettlemoyer Christoph Feichtenhofer CLIP VLM 259 561 0 28 Sep 2021
Instance-Based Neural Dependency Parsing Hiroki Ouchi Jun Suzuki Sosuke Kobayashi Sho Yokoi Tatsuki Kuribayashi Masashi Yoshikawa Kentaro Inui 37 3 0 28 Sep 2021