v1v2 (latest)

Generalization through Memorization: Nearest Neighbor Language Models

1 November 2019

Urvashi Khandelwal

Omer Levy

Dan Jurafsky

Luke Zettlemoyer

M. Lewis

RALM

ArXiv (abs)PDF HTML

Papers citing "Generalization through Memorization: Nearest Neighbor Language Models"

50 / 597 papers shown

Title
N-Grammer: Augmenting Transformers with latent n-grams Aurko Roy Rohan Anil Guangda Lai Benjamin Lee Jeffrey Zhao ... Yu Phuong Dao Christopher Fifty Zhiwen Chen Yonghui Wu 77 8 0 13 Jul 2022
Repository-Level Prompt Generation for Large Language Models of Code Disha Shrivastava Hugo Larochelle Daniel Tarlow 103 144 0 26 Jun 2022
Memory-Based Model Editing at Scale E. Mitchell Charles Lin Antoine Bosselut Christopher D. Manning Chelsea Finn KELM 126 362 0 13 Jun 2022
Annotation Error Detection: Analyzing the Past and Present for a More Coherent Future Jan-Christoph Klie Bonnie Webber Iryna Gurevych 135 47 0 05 Jun 2022
Dict-TTS: Learning to Pronounce with Prior Dictionary Knowledge for Text-to-Speech Ziyue Jiang Zhe Su Zhou Zhao Qian Yang Yi Ren Jinglin Liu Zhe Ye 73 5 0 05 Jun 2022
Decoupling Knowledge from Memorization: Retrieval-augmented Prompt Learning Xiang Chen Lei Li Ningyu Zhang Xiaozhuan Liang Shumin Deng Chuanqi Tan Fei Huang Luo Si Huajun Chen VLM 155 55 0 29 May 2022
kNN-Prompt: Nearest Neighbor Zero-Shot Inference Weijia Shi Julian Michael Suchin Gururangan Luke Zettlemoyer RALM VLM 90 32 0 27 May 2022
Tranception: protein fitness prediction with autoregressive transformers and inference-time retrieval Pascal Notin M. Dias J. Frazer Javier Marchena-Hurtado Aidan Gomez D. Marks Y. Gal 129 189 0 27 May 2022
Training Language Models with Memory Augmentation Zexuan Zhong Tao Lei Danqi Chen RALM 345 133 0 25 May 2022
ORCA: Interpreting Prompted Language Models via Locating Supporting Data Evidence in the Ocean of Pretraining Data Xiaochuang Han Yulia Tsvetkov 97 31 0 25 May 2022
Chunk-based Nearest Neighbor Machine Translation Pedro Henrique Martins Zita Marinho André F.T. Martins RALM 137 28 0 24 May 2022
StreamingQA: A Benchmark for Adaptation to New Knowledge over Time in Question Answering Models Adam Livska Tomávs Kovciský E. Gribovskaya Tayfun Terzi Eren Sezener ... Susannah Young Ellen Gilsenan-McMahon Sophia Austin Phil Blunsom Angeliki Lazaridou KELM 315 104 0 23 May 2022
Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models Kushal Tirumala Aram H. Markosyan Luke Zettlemoyer Armen Aghajanyan TDI 138 197 0 22 May 2022
RankGen: Improving Text Generation with Large Ranking Models Kalpesh Krishna Yapei Chang John Wieting Mohit Iyyer AIMat 90 69 0 19 May 2022
Long-term Control for Dialogue Generation: Methods and Evaluation Ramya Ramakrishnan H. Narangodage M. Schilman Kilian Q. Weinberger Ryan T. McDonald 70 8 0 15 May 2022
Near-Negative Distinction: Giving a Second Life to Human Evaluation Datasets Philippe Laban Chien-Sheng Wu Wenhao Liu Caiming Xiong 78 5 0 13 May 2022
kNN-Embed: Locally Smoothed Embedding Mixtures For Multi-interest Candidate Retrieval Ahmed El-Kishky Thomas Markovich Kenny Leung Frank Portman A. Haghighi Ying Xiao 70 12 0 12 May 2022
Few-shot Mining of Naturally Occurring Inputs and Outputs Mandar Joshi Terra Blevins M. Lewis Daniel S. Weld Luke Zettlemoyer 97 1 0 09 May 2022
Relation Extraction as Open-book Examination: Retrieval-enhanced Prompt Tuning Xiang Chen Lei Li Ningyu Zhang Chuanqi Tan Fei Huang Luo Si Huajun Chen RALM VLM 83 40 0 04 May 2022
Retrieval-Enhanced Machine Learning Hamed Zamani Fernando Diaz Mostafa Dehghani Donald Metzler Michael Bendersky 70 51 0 02 May 2022
Can deep learning match the efficiency of human visual long-term memory in storing object details? Emin Orhan VLM OCL 124 0 0 27 Apr 2022
Semi-Parametric Neural Image Synthesis A. Blattmann Robin Rombach Kaan Oktay Jonas Muller Bjorn Ommer DiffM 102 31 0 25 Apr 2022
METRO: Efficient Denoising Pretraining of Large Scale Autoencoding Language Models with Model Generated Signals Payal Bajaj Chenyan Xiong Guolin Ke Xiaodong Liu Di He Saurabh Tiwary Tie-Yan Liu Paul N. Bennett Xia Song Jianfeng Gao 121 32 0 13 Apr 2022
Augmenting Pre-trained Language Models with QA-Memory for Open-Domain Question Answering Wenhu Chen Pat Verga Michiel de Jong John Wieting William W. Cohen RALM KELM 90 27 0 10 Apr 2022
Knowledge Infused Decoding Ruibo Liu Guoqing Zheng Shashank Gupta Radhika Gaonkar Chongyang Gao Soroush Vosoughi Milad Shokouhi Ahmed Hassan Awadallah KELM 90 14 0 06 Apr 2022
Knowledge Base Index Compression via Dimensionality and Precision Reduction Vilém Zouhar Marius Mosbach Miaoran Zhang Dietrich Klakow 86 2 0 06 Apr 2022
$k$ NN-NER: Named Entity Recognition with Nearest Neighbor Search Shuhe Wang Xiaoya Li Yuxian Meng Tianwei Zhang Rongbin Ouyang Jiwei Li Guoyin Wang 72 11 0 31 Mar 2022
Shoring Up the Foundations: Fusing Model Embeddings and Weak Supervision Mayee F. Chen Daniel Y. Fu Dyah Adila Michael Zhang Frederic Sala Kayvon Fatahalian Christopher Ré 78 20 0 24 Mar 2022
Language Models that Seek for Knowledge: Modular Search & Generation for Dialogue and Prompt Completion Kurt Shuster M. Komeili Leonard Adolphs Stephen Roller Arthur Szlam Jason Weston KELM 110 129 0 24 Mar 2022
An Empirical Study of Memorization in NLP Xiaosen Zheng Jing Jiang TDI 88 25 1 23 Mar 2022
Better Language Model with Hypernym Class Prediction Richard He Bai Tong Wang Alessandro Sordoni Peng Shi 139 16 0 21 Mar 2022
Memorizing Transformers Yuhuai Wu M. Rabe DeLesley S. Hutchins Christian Szegedy RALM 109 179 0 16 Mar 2022
Interpretability for Language Learners Using Example-Based Grammatical Error Correction Masahiro Kaneko Sho Takase Ayana Niwa Naoaki Okazaki 70 26 0 14 Mar 2022
Internet-augmented language models through few-shot prompting for open-domain question answering Angeliki Lazaridou E. Gribovskaya Wojciech Stokowiec N. Grigorev KELM LRM 76 139 0 10 Mar 2022
Retrieval Augmented Classification for Long-Tail Visual Recognition Alex Long Wei Yin Thalaiyasingam Ajanthan Vu-Linh Nguyen Pulak Purkait Ravi Garg Alan Blair Chunhua Shen Anton Van Den Hengel 75 114 0 22 Feb 2022
Knowledge Base Question Answering by Case-based Reasoning over Subgraphs Rajarshi Das Ameya Godbole Ankita Rajaram Naik Elliot Tower Robin Jia Manzil Zaheer Hannaneh Hajishirzi Andrew McCallum 83 54 0 22 Feb 2022
What Does it Mean for a Language Model to Preserve Privacy? Hannah Brown Katherine Lee Fatemehsadat Mireshghallah Reza Shokri Florian Tramèr PILM 111 243 0 11 Feb 2022
A Survey on Retrieval-Augmented Text Generation Huayang Li Yixuan Su Deng Cai Yan Wang Lemao Liu RALM 149 212 0 02 Feb 2022
Neuro-Symbolic Language Modeling with Automaton-augmented Retrieval Uri Alon Frank F. Xu Junxian He Sudipta Sengupta Dan Roth Graham Neubig RALM 152 64 0 28 Jan 2022
Relational Memory Augmented Language Models Qi Liu Dani Yogatama Phil Blunsom KELM RALM 134 32 0 24 Jan 2022
Artefact Retrieval: Overview of NLP Models with Knowledge Base Access Vilém Zouhar Marius Mosbach Debanjali Biswas Dietrich Klakow KELM 90 4 0 24 Jan 2022
LaMDA: Language Models for Dialog Applications R. Thoppilan Daniel De Freitas Jamie Hall Noam M. Shazeer Apoorv Kulshreshtha ... Blaise Aguera-Arcas Claire Cui M. Croak Ed H. Chi Quoc Le ALM 158 1,606 0 20 Jan 2022
Memory-assisted prompt editing to improve GPT-3 after deployment Aman Madaan Niket Tandon Peter Clark Yiming Yang KELM 124 0 0 16 Jan 2022
Reasoning Through Memorization: Nearest Neighbor Knowledge Graph Embeddings Peng Wang Xin Xie Xiaohan Wang Ningyu Zhang RALM 131 18 0 14 Jan 2022
DREAM: Improving Situational QA by First Elaborating the Situation Yuling Gu Bhavana Dalvi Peter Clark 119 18 0 16 Dec 2021
Learning To Retrieve Prompts for In-Context Learning Ohad Rubin Jonathan Herzig Jonathan Berant VPVLM RALM 105 710 0 16 Dec 2021
Rethinking Nearest Neighbors for Visual Classification Menglin Jia Bor-Chun Chen Zuxuan Wu Claire Cardie Serge Belongie Ser-Nam Lim SSL 92 10 0 15 Dec 2021
Faster Nearest Neighbor Machine Translation Shuhe Wang Jiwei Li Yuxian Meng Rongbin Ouyang Guoyin Wang Xiaoya Li Tianwei Zhang Shi Zong 45 12 0 15 Dec 2021
Improving and Diagnosing Knowledge-Based Visual Question Answering via Entity Enhanced Knowledge Injection Diego Garcia-Olano Yasumasa Onoe Joydeep Ghosh 69 18 0 13 Dec 2021
Improving language models by retrieving from trillions of tokens Sebastian Borgeaud A. Mensch Jordan Hoffmann Trevor Cai Eliza Rutherford ... Simon Osindero Karen Simonyan Jack W. Rae Erich Elsen Laurent Sifre KELM RALM 303 1,109 0 08 Dec 2021