Generalization through Memorization: Nearest Neighbor Language Models

1 November 2019

Urvashi Khandelwal

Omer Levy

Dan Jurafsky

Luke Zettlemoyer

M. Lewis

RALM

ArXiv PDF HTML

Papers citing "Generalization through Memorization: Nearest Neighbor Language Models"

50 / 582 papers shown

Title
SoK: Memorization in General-Purpose Large Language Models Valentin Hartmann Anshuman Suri Vincent Bindschaedler David Evans Shruti Tople Robert West KELM LLMAG 29 20 0 24 Oct 2023
TRAMS: Training-free Memory Selection for Long-range Language Modeling Haofei Yu Cunxiang Wang Yue Zhang Wei Bi RALM 46 6 0 24 Oct 2023
Multilingual k-Nearest-Neighbor Machine Translation David Stap Christof Monz 24 3 0 23 Oct 2023
Large Search Model: Redefining Search Stack in the Era of LLMs Liang Wang Nan Yang Xiaolong Huang Linjun Yang Rangan Majumder Furu Wei LRM KELM 47 13 0 23 Oct 2023
From Interpolation to Extrapolation: Complete Length Generalization for Arithmetic Transformers Shaoxiong Duan Yining Shi Wei Xu 28 8 0 18 Oct 2023
Emptying the Ocean with a Spoon: Should We Edit Models? Yuval Pinter Michael Elhadad KELM 27 26 0 18 Oct 2023
Heterogenous Memory Augmented Neural Networks Zihan Qiu Zhen Liu Shuicheng Yan Shanghang Zhang Jie Fu 20 0 0 17 Oct 2023
RegaVAE: A Retrieval-Augmented Gaussian Mixture Variational Auto-Encoder for Language Modeling Jingcheng Deng Liang Pang Huawei Shen Xueqi Cheng RALM 36 10 0 16 Oct 2023
Chameleon: a Heterogeneous and Disaggregated Accelerator System for Retrieval-Augmented Language Models Wenqi Jiang Marco Zeller R. Waleffe Torsten Hoefler Gustavo Alonso 54 14 0 15 Oct 2023
InstructRetro: Instruction Tuning post Retrieval-Augmented Pretraining Wei Ping Ming-Yu Liu Lawrence C. McAfee Peng Xu Bo Li M. Shoeybi Bryan Catanzaro RALM 16 47 0 11 Oct 2023
Goodtriever: Adaptive Toxicity Mitigation with Retrieval-augmented Models Luiza Amador Pozzobon Beyza Ermis Patrick Lewis Sara Hooker 36 20 0 11 Oct 2023
How Do Large Language Models Capture the Ever-changing World Knowledge? A Review of Recent Advances Zihan Zhang Meng Fang Lingxi Chen Mohammad-Reza Namazi-Rad Jun Wang KELM 29 21 0 11 Oct 2023
A Meta-Learning Perspective on Transformers for Causal Language Modeling Xinbo Wu L. Varshney 37 6 0 09 Oct 2023
What do larger image classifiers memorise? Michal Lukasik Vaishnavh Nagarajan A. S. Rawat A. Menon Sanjiv Kumar 38 5 0 09 Oct 2023
Toolink: Linking Toolkit Creation and Using through Chain-of-Solving on Open-Source Model Cheng Qian Chenyan Xiong Zhenghao Liu Zhiyuan Liu LRM 34 12 0 08 Oct 2023
Self-Knowledge Guided Retrieval Augmentation for Large Language Models Yile Wang Peng Li Maosong Sun Yang Liu RALM KELM 29 42 0 08 Oct 2023
RECOMP: Improving Retrieval-Augmented LMs with Compression and Selective Augmentation Fangyuan Xu Weijia Shi Eunsol Choi RALM 35 147 0 06 Oct 2023
Retrieval meets Long Context Large Language Models Peng Xu Ming-Yu Liu Xianchao Wu Lawrence C. McAfee Chen Zhu Zihan Liu Sandeep Subramanian Evelina Bakhturina M. Shoeybi Bryan Catanzaro RALM LRM 14 82 0 04 Oct 2023
Dodo: Dynamic Contextual Compression for Decoder-only LMs Guanghui Qin Corby Rosset Ethan C. Chau Nikhil Rao Benjamin Van Durme 27 8 0 03 Oct 2023
OceanGPT: A Large Language Model for Ocean Science Tasks Zhen Bi Ningyu Zhang Yida Xue Yixin Ou Daxiong Ji Guozhou Zheng Huajun Chen ALM LLMAG 34 28 0 03 Oct 2023
RA-DIT: Retrieval-Augmented Dual Instruction Tuning Xi Lin Xilun Chen Mingda Chen Weijia Shi Maria Lomeli ... Jacob Kahn Gergely Szilvasy Mike Lewis Luke Zettlemoyer Scott Yih RALM 47 133 0 02 Oct 2023
Noise-Tolerant Unsupervised Adapter for Vision-Language Models Eman Ali Dayan Guan Muhammad Haris Khan Abdulmotaleb Elsaddik VLM 24 0 0 26 Sep 2023
Ragas: Automated Evaluation of Retrieval Augmented Generation ES Shahul Jithin James Luis Espinosa-Anke Steven Schockaert 91 177 0 26 Sep 2023
Can Whisper perform speech-based in-context learning? Siyin Wang Chao-Han Huck Yang Ji Wu Chao Zhang 32 26 0 13 Sep 2023
Towards Reliable and Fluent Large Language Models: Incorporating Feedback Learning Loops in QA Systems Dongyub Lee Taesun Whang Chanhee Lee Heuiseok Lim KELM 24 9 0 08 Sep 2023
ImageBind-LLM: Multi-modality Instruction Tuning Jiaming Han Renrui Zhang Wenqi Shao Peng Gao Peng Xu ... Yafei Wen Xiaoxin Chen Xiangyu Yue Hongsheng Li Yu Qiao MLLM 51 117 0 07 Sep 2023
RenAIssance: A Survey into AI Text-to-Image Generation in the Era of Large Model Fengxiang Bie Yibo Yang Zhongzhu Zhou Adam Ghanem Minjia Zhang ... Pareesa Ameneh Golnari David A. Clifton Yuxiong He Dacheng Tao Shuaiwen Leon Song EGVM 36 20 0 02 Sep 2023
RAMP: Retrieval-Augmented MOS Prediction via Confidence-based Dynamic Weighting Haibo Wang Shiwan Zhao Xiguang Zheng Yong Qin 34 12 0 31 Aug 2023
Supervised Contrastive Learning with Nearest Neighbor Search for Speech Emotion Recognition Xuechen Wang Shiwan Zhao Yong Qin 22 6 0 31 Aug 2023
LM-Infinite: Zero-Shot Extreme Length Generalization for Large Language Models Chi Han Qifan Wang Hao Peng Wenhan Xiong Yu Chen Heng Ji Sinong Wang 50 50 0 30 Aug 2023
Cross-Modal Retrieval Meets Inference:Improving Zero-Shot Classification with Cross-Modal Retrieval Seong-Hoon Eom Namgyu Ho Jaehoon Oh Se-Young Yun CLIP VLM 38 0 0 29 Aug 2023
CAGRA: Highly Parallel Graph Construction and Approximate Nearest Neighbor Search for GPUs Hiroyuki Ootomo Akira Naruse Corey J. Nolet Ray Wang Tamas B. Fehér Yuanbo Wang GNN 36 20 0 29 Aug 2023
MEMORY-VQ: Compression for Tractable Internet-Scale Memory Yury Zemlyanskiy Michiel de Jong Luke Vilnis Santiago Ontañón William W. Cohen Sumit Sanghai Joshua Ainslie RALM MQ 35 0 0 28 Aug 2023
LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding Yushi Bai Xin Lv Jiajie Zhang Hong Lyu Jiankai Tang ... Aohan Zeng Lei Hou Yuxiao Dong Jie Tang Juanzi Li LLMAG RALM 31 507 0 28 Aug 2023
With a Little Help from your own Past: Prototypical Memory Networks for Image Captioning Manuele Barraco Sara Sarto Marcella Cornia Lorenzo Baraldi Rita Cucchiara VLM 60 19 0 23 Aug 2023
RSpell: Retrieval-augmented Framework for Domain Adaptive Chinese Spelling Check Siqi Song Qi Lv Lei Geng Ziqiang Cao Guohong Fu 27 5 0 16 Aug 2023
RAVEN: In-Context Learning with Retrieval-Augmented Encoder-Decoder Language Models Jie Huang Ming-Yu Liu Peng Xu M. Shoeybi Kevin Chen-Chuan Chang Bryan Catanzaro RALM 37 33 0 15 Aug 2023
WeaverBird: Empowering Financial Decision-Making with Large Language Model, Knowledge Base, and Search Engine Siqiao Xue Fan Zhou Y. Xu Ming Jin Qingsong Wen ... Jun Zhou Shuo Xie D. Xiu James Y. Zhang Hongyuan Mei RALM AIFin 31 15 0 10 Aug 2023
SILO Language Models: Isolating Legal Risk In a Nonparametric Datastore Sewon Min Suchin Gururangan Eric Wallace Hannaneh Hajishirzi Noah A. Smith Luke Zettlemoyer AILaw 28 63 0 08 Aug 2023
TabR: Tabular Deep Learning Meets Nearest Neighbors in 2023 Yu. V. Gorishniy Ivan Rubachev Nikolay Kartashev Daniil Shlenskii Akim Kotelnikov Artem Babenko OOD LMTD 27 15 0 26 Jul 2023
Benchmarking and Analyzing Generative Data for Visual Recognition Bo-wen Li Haotian Liu Liangyu Chen Yong Jae Lee C. Li Ziwei Liu EGVM VLM 18 4 0 25 Jul 2023
Learning to Retrieve In-Context Examples for Large Language Models Liang Wang Nan Yang Furu Wei RALM 44 37 0 14 Jul 2023
Generating Benchmarks for Factuality Evaluation of Language Models Dor Muhlgay Ori Ram Inbal Magar Yoav Levine Nir Ratner Yonatan Belinkov Omri Abend Kevin Leyton-Brown Amnon Shashua Y. Shoham HILM 33 91 0 13 Jul 2023
Copy Is All You Need Tian Lan Deng Cai Yan Wang Heyan Huang Xian-Ling Mao 35 27 0 13 Jul 2023
Pluggable Neural Machine Translation Models via Memory-augmented Adapters Yuzhuang Xu Shuo Wang Peng Li Xuebo Liu Xiaolong Wang Weidong Liu Yang Liu 50 1 0 12 Jul 2023
ReLoRA: High-Rank Training Through Low-Rank Updates Vladislav Lialin Namrata Shivagunde Sherin Muckatira Anna Rumshisky BDL 37 95 0 11 Jul 2023
Linear Alignment of Vision-language Models for Image Captioning Fabian Paischer M. Hofmarcher Sepp Hochreiter Thomas Adler CLIP VLM 53 0 0 10 Jul 2023
Focused Transformer: Contrastive Training for Context Scaling Szymon Tworkowski Konrad Staniszewski Mikolaj Pacek Yuhuai Wu Henryk Michalewski Piotr Milo's 36 136 0 06 Jul 2023
Multimodal Prompt Retrieval for Generative Visual Question Answering Timothy Ossowski Junjie Hu 23 1 0 30 Jun 2023
LeanDojo: Theorem Proving with Retrieval-Augmented Language Models Kaiyu Yang Aidan M. Swope Alex Gu Rahul Chalamala Peiyang Song Shixing Yu Saad Godil R. Prenger Anima Anandkumar RALM 29 216 0 27 Jun 2023