Generalization through Memorization: Nearest Neighbor Language Models

1 November 2019

Urvashi Khandelwal

Omer Levy

Dan Jurafsky

Luke Zettlemoyer

M. Lewis

RALM

ArXiv PDF HTML

Papers citing "Generalization through Memorization: Nearest Neighbor Language Models"

50 / 582 papers shown

Title
Tool Learning with Foundation Models Yujia Qin Shengding Hu Yankai Lin Weize Chen Ning Ding ... Cheng Yang Tongshuang Wu Heng Ji Zhiyuan Liu Maosong Sun 48 200 0 17 Apr 2023
Shall We Pretrain Autoregressive Language Models with Retrieval? A Comprehensive Study Wei Ping Ming-Yu Liu P. Xu Lawrence C. McAfee Zihan Liu ... Oleksii Kuchaiev Bo-wen Li Chaowei Xiao Anima Anandkumar Bryan Catanzaro RALM 46 56 0 13 Apr 2023
Improving Image Recognition by Retrieving from Web-Scale Image-Text Data Ahmet Iscen Alireza Fathi Cordelia Schmid VLM 3DV 38 25 0 11 Apr 2023
Multilingual Machine Translation with Large Language Models: Empirical Results and Analysis Wenhao Zhu Hongyi Liu Qingxiu Dong Jingjing Xu Shujian Huang Lingpeng Kong Jiajun Chen Lei Li LRM 45 142 0 10 Apr 2023
LADER: Log-Augmented DEnse Retrieval for Biomedical Literature Search Qiao Jin Andrew Shin Zhiyong Lu RALM 32 7 0 10 Apr 2023
ConceptEVA: Concept-Based Interactive Exploration and Customization of Document Summaries Xiaoyu Zhang J. Li Po-Wei Chi Senthil K. Chandrasegaran Kwan-Liu Ma 25 22 0 31 Mar 2023
MoViT: Memorizing Vision Transformers for Medical Image Analysis Yiqing Shen Pengfei Guo Jinpu Wu Qi Huang Nhat Le Jinyuan Zhou Shanshan Jiang Mathias Unberath ViT MedIm 34 10 0 27 Mar 2023
Scaling Expert Language Models with Unsupervised Domain Discovery Suchin Gururangan Margaret Li M. Lewis Weijia Shi Tim Althoff Noah A. Smith Luke Zettlemoyer MoE 30 46 0 24 Mar 2023
$k$ NN Prompting: Beyond-Context Learning with Calibration-Free Nearest Neighbor Inference Benfeng Xu Quan Wang Zhendong Mao Yajuan Lyu Qiaoqiao She Yongdong Zhang 104 175 0 24 Mar 2023
Retrieval-Augmented Classification with Decoupled Representation Xinnian Liang Shuangzhi Wu Hui Huang Jiaqi Bai Chao Bian Zhoujun Li 29 0 0 23 Mar 2023
Language Model Behavior: A Comprehensive Survey Tyler A. Chang Benjamin Bergen VLM LRM LM&MA 32 103 0 20 Mar 2023
On-the-fly Text Retrieval for End-to-End ASR Adaptation Bolaji Yusuf Aditya Gourav Ankur Gandhe I. Bulyko KELM RALM 43 4 0 20 Mar 2023
Text-to-image Diffusion Models in Generative AI: A Survey Chenshuang Zhang Chaoning Zhang Mengchun Zhang In So Kweon VLM 51 267 0 14 Mar 2023
MetaTroll: Few-shot Detection of State-Sponsored Trolls with Transformer Adapters Lin Tian Xiuzhen Zhang Jey Han Lau 32 10 0 13 Mar 2023
A Theoretical Analysis Of Nearest Neighbor Search On Approximate Near Neighbor Graph Anshumali Shrivastava Zhao Song Zhaozhuo Xu GNN 39 8 0 10 Mar 2023
Semiparametric Language Models Are Scalable Continual Learners Guangyue Peng Tao Ge Si-Qing Chen Furu Wei Houfeng Wang KELM 52 11 0 02 Mar 2023
Nearest Neighbors Meet Deep Neural Networks for Point Cloud Analysis Renrui Zhang Liuhui Wang Ziyu Guo Jianbo Shi 3DPC 32 10 0 01 Mar 2023
Retrieved Sequence Augmentation for Protein Representation Learning Chang Ma Haiteng Zhao Lin Zheng Jiayi Xin Qintong Li Lijun Wu Zhihong Deng Yang Lu Qi Liu Lingpeng Kong AI4TS 30 9 0 24 Feb 2023
Federated Nearest Neighbor Machine Translation Yichao Du Zhirui Zhang Bingzhe Wu Lemao Liu Tong Xu Enhong Chen FedML 42 6 0 23 Feb 2023
Simple and Scalable Nearest Neighbor Machine Translation Yu-Hsiu Dai Zhirui Zhang Qiuzhi Liu Qu Cui Wei-Hong Li Yichao Du Tong Xu 37 16 0 23 Feb 2023
On the Generalization Ability of Retrieval-Enhanced Transformers Tobias Norlund Ehsan Doostmohammadi Richard Johansson Marco Kuhlmann RALM 35 6 0 23 Feb 2023
$k$ NN-Adapter: Efficient Domain Adaptation for Black-Box Language Models Yangsibo Huang Daogao Liu Zexuan Zhong Weijia Shi Y. Lee RALM ALM 43 14 0 21 Feb 2023
Retrieval-augmented Image Captioning R. Ramos Desmond Elliott Bruno Martins VLM 34 29 0 16 Feb 2023
Augmented Language Models: a Survey Grégoire Mialon Roberto Dessì Maria Lomeli Christoforos Nalmpantis Ramakanth Pasunuru ... Jane Dwivedi-Yu Asli Celikyilmaz Edouard Grave Yann LeCun Thomas Scialom LRM KELM 47 370 0 15 Feb 2023
Augmenting Zero-Shot Dense Retrievers with Plug-in Mixture-of-Memories Suyu Ge Chenyan Xiong Corby Rosset Arnold Overwijk Jiawei Han Paul N. Bennett VLM 41 6 0 07 Feb 2023
ResMem: Learn what you can and memorize the rest Zitong Yang Michal Lukasik Vaishnavh Nagarajan Zong-xiao Li A. S. Rawat Manzil Zaheer A. Menon Surinder Kumar VLM 43 8 0 03 Feb 2023
In-Context Retrieval-Augmented Language Models Ori Ram Yoav Levine Itay Dalmedigos Dor Muhlgay Amnon Shashua Kevin Leyton-Brown Y. Shoham KELM RALM LRM 31 548 0 31 Jan 2023
N-Gram Nearest Neighbor Machine Translation Rui Lv Junliang Guo Rui Wang Xu Tan Qi Liu Tao Qin 23 2 0 30 Jan 2023
REPLUG: Retrieval-Augmented Black-Box Language Models Weijia Shi Sewon Min Michihiro Yasunaga Minjoon Seo Rich James M. Lewis Luke Zettlemoyer Wen-tau Yih RALM VLM KELM 85 586 0 30 Jan 2023
ProtST: Multi-Modality Learning of Protein Sequences and Biomedical Texts Minghao Xu Xinyu Yuan Santiago Miret Jian Tang AI4TS 37 95 0 28 Jan 2023
Semi-Parametric Video-Grounded Text Generation Sungdong Kim Jin-Hwa Kim Jiyoung Lee Minjoon Seo VGen 32 14 0 27 Jan 2023
Pre-computed memory or on-the-fly encoding? A hybrid approach to retrieval augmentation makes the most of your compute Michiel de Jong Yury Zemlyanskiy Nicholas FitzGerald Joshua Ainslie Sumit Sanghai Fei Sha William W. Cohen RALM 40 16 0 25 Jan 2023
Learning Customized Visual Models with Retrieval-Augmented Knowledge Haotian Liu Kilho Son Jianwei Yang Ce Liu Jianfeng Gao Yong Jae Lee Chunyuan Li VLM 40 54 0 17 Jan 2023
Structured Case-based Reasoning for Inference-time Adaptation of Text-to-SQL parsers Abhijeet Awasthi Soumen Chakrabarti Sunita Sarawagi 33 5 0 10 Jan 2023
Why do Nearest Neighbor Language Models Work? Frank F. Xu Uri Alon Graham Neubig RALM 30 22 0 07 Jan 2023
Automating Nearest Neighbor Search Configuration with Constrained Optimization Philip Sun Ruiqi Guo Surinder Kumar 31 7 0 04 Jan 2023
Analogical Inference Enhanced Knowledge Graph Embedding Zhen Yao Wen Zhang Mingyang Chen Yufen Huang Yezhou Yang Hua-zeng Chen 41 12 0 03 Jan 2023
Rethinking with Retrieval: Faithful Large Language Model Inference Hangfeng He Hongming Zhang Dan Roth KELM LRM 146 161 0 31 Dec 2022
Continual Contrastive Finetuning Improves Low-Resource Relation Extraction Wenxuan Zhou Sheng Zhang Tristan Naumann Muhao Chen Hoifung Poon 54 6 0 21 Dec 2022
When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories Alex Troy Mallen Akari Asai Victor Zhong Rajarshi Das Daniel Khashabi Hannaneh Hajishirzi RALM HILM KELM 64 529 0 20 Dec 2022
Empowering Sentence Encoders with Prompting and Label Retrieval for Zero-shot Text Classification Jimin Hong Jungsoo Park Daeyoung Kim Seongjae Choi Bokyung Son Jaewoo Kang 24 3 0 20 Dec 2022
CoCoMIC: Code Completion By Jointly Modeling In-file and Cross-file Context Yangruibo Ding Zijian Wang Wasi Uddin Ahmad M. K. Ramanathan Ramesh Nallapati Parminder Bhatia Dan Roth Bing Xiang 24 70 0 20 Dec 2022
Training Trajectories of Language Models Across Scales Mengzhou Xia Mikel Artetxe Chunting Zhou Xi Lin Ramakanth Pasunuru Danqi Chen Luke Zettlemoyer Ves Stoyanov AIFin LRM 39 55 0 19 Dec 2022
Can Retriever-Augmented Language Models Reason? The Blame Game Between the Retriever and the Language Model Parishad BehnamGhader Santiago Miret Siva Reddy ReLM LRM 27 36 0 18 Dec 2022
Evaluating Step-by-Step Reasoning through Symbolic Verification Yi-Fan Zhang Hanlin Zhang Li Erran Li Eric P. Xing ReLM LRM 19 8 0 16 Dec 2022
Memories are One-to-Many Mapping Alleviators in Talking Face Generation Anni Tang Tianyu He Xuejiao Tan Jun Ling Liang Song CVBM 26 23 0 09 Dec 2022
Document-Level Abstractive Summarization Gonçalo Raposo Afonso Raposo Ana Sofia Carmo 27 1 0 06 Dec 2022
Meta-Learning Fast Weight Language Models Kevin Clark Kelvin Guu Ming-Wei Chang Panupong Pasupat Geoffrey E. Hinton Mohammad Norouzi KELM 32 13 0 05 Dec 2022
Retrieval as Attention: End-to-end Learning of Retrieval and Reading within a Single Transformer Zhengbao Jiang Luyu Gao Jun Araki Haibo Ding Zhiruo Wang Jamie Callan Graham Neubig RALM 32 40 0 05 Dec 2022
GNN-SL: Sequence Labeling Based on Nearest Examples via GNN Shuhe Wang Yuxian Meng Rongbin Ouyang Jiwei Li Tianwei Zhang Lingjuan Lyu Guoyin Wang 32 9 0 05 Dec 2022