Generalization through Memorization: Nearest Neighbor Language Models

1 November 2019

Urvashi Khandelwal

Omer Levy

Dan Jurafsky

Luke Zettlemoyer

M. Lewis

RALM

ArXiv PDF HTML

Papers citing "Generalization through Memorization: Nearest Neighbor Language Models"

50 / 581 papers shown

Title
Retrieval-Enhanced Machine Learning: Synthesis and Opportunities To Eun Kim Alireza Salemi Andrew Drozdov Fernando Diaz Hamed Zamani 56 7 0 17 Jul 2024
Goldfish: Vision-Language Understanding of Arbitrarily Long Videos Kirolos Ataallah Xiaoqian Shen Eslam Abdelrahman Essam Sleiman Mingchen Zhuge Jian Ding Deyao Zhu Jürgen Schmidhuber Mohamed Elhoseiny VLM 30 17 0 17 Jul 2024
Mitigating Entity-Level Hallucination in Large Language Models Weihang Su Yichen Tang Qingyao Ai Changyue Wang Zhijing Wu Yiqun Liu HILM 47 6 0 12 Jul 2024
CompAct: Compressing Retrieved Documents Actively for Question Answering Chanwoong Yoon Taewhoo Lee Hyeon Hwang Minbyul Jeong Jaewoo Kang KELM RALM MQ 37 12 0 12 Jul 2024
Detect, Investigate, Judge and Determine: A Knowledge-guided Framework for Few-shot Fake News Detection Ye Liu Jiajun Zhu Kai Zhang Haoyu Tang Yanghai Zhang Xukai Liu Qi Liu Enhong Chen 61 0 0 12 Jul 2024
Operationalizing the Blueprint for an AI Bill of Rights: Recommendations for Practitioners, Researchers, and Policy Makers Alex Oesterling Usha Bhalla Suresh Venkatasubramanian Himabindu Lakkaraju 48 1 0 11 Jul 2024
Speculative RAG: Enhancing Retrieval Augmented Generation through Drafting Zilong Wang Zifeng Wang Long Le Huaixiu Steven Zheng Swaroop Mishra ... Anush Mattapalli Ankur Taly Jingbo Shang Chen-Yu Lee Tomas Pfister RALM 85 34 0 11 Jul 2024
Scaling Retrieval-Based Language Models with a Trillion-Token Datastore Rulin Shao Jacqueline He Akari Asai Weijia Shi Tim Dettmers Sewon Min Luke Zettlemoyer Pang Wei Koh RALM 29 22 0 09 Jul 2024
VRSD: Rethinking Similarity and Diversity for Retrieval in Large Language Models Hang Gao Yongfeng Zhang 43 2 0 05 Jul 2024
Mixture of A Million Experts Xu Owen He MoE 44 26 0 04 Jul 2024
Do Generalised Classifiers really work on Human Drawn Sketches? Hmrishav Bandyopadhyay Pinaki Nath Chowdhury Aneeshan Sain Subhadeep Koley Tao Xiang A. Bhunia Yi-Zhe Song VLM 33 2 0 04 Jul 2024
DSLR: Document Refinement with Sentence-Level Re-ranking and Reconstruction to Enhance Retrieval-Augmented Generation Taeho Hwang Soyeong Jeong Sukmin Cho SeungYoon Han Jong C. Park RALM 43 1 0 04 Jul 2024
Contrast then Memorize: Semantic Neighbor Retrieval-Enhanced Inductive Multimodal Knowledge Graph Completion Yu Zhao Ying Zhang Baohang Zhou Xinying Qian Kehui Song Xiangrui Cai 32 2 0 03 Jul 2024
Efficient Nearest Neighbor based Uncertainty Estimation for Natural Language Processing Tasks Wataru Hashimoto Hidetaka Kamigaito Taro Watanabe 60 0 0 02 Jul 2024
SeaKR: Self-aware Knowledge Retrieval for Adaptive Retrieval Augmented Generation Zijun Yao Weijian Qi Liangming Pan S. Cao Linmei Hu Weichuan Liu Lei Hou Juanzi Li RALM 46 6 0 27 Jun 2024
RAVEN: Multitask Retrieval Augmented Vision-Language Learning Varun Nagaraj Rao Siddharth Choudhary Aditya Deshpande R. Satzoda Srikar Appalaraju RALM VLM 58 4 0 27 Jun 2024
Banishing LLM Hallucinations Requires Rethinking Generalization Johnny Li Saksham Consul Eda Zhou James Wong Naila Farooqui ... Zhuxiaona Wei Tian Wu Ben Echols Sharon Zhou Gregory Diamos LRM 34 10 0 25 Jun 2024
Mitigating Hallucination in Fictional Character Role-Play Nafis Sadeq Zhouhang Xie Byungkyu Kang Prarit Lamba Xiang Gao Julian McAuley HILM 48 7 0 25 Jun 2024
Ragnarök: A Reusable RAG Framework and Baselines for TREC 2024 Retrieval-Augmented Generation Track Ronak Pradeep Nandan Thakur Sahel Sharifymoghaddam Eric Zhang Ryan Nguyen Daniel Campos Nick Craswell Jimmy Lin 48 12 0 24 Jun 2024
Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization Cheng-Yu Hsieh Yung-Sung Chuang Chun-Liang Li Zifeng Wang Long T. Le ... James R. Glass Alexander Ratner Chen-Yu Lee Ranjay Krishna Tomas Pfister 48 33 0 23 Jun 2024
Memorizing Documents with Guidance in Large Language Models Bumjin Park Jaesik Choi KELM RALM 36 1 0 23 Jun 2024
GraphReader: Building Graph-based Agent to Enhance Long-Context Abilities of Large Language Models Shilong Li Yancheng He Hangyu Guo Xingyuan Bu Ge Bai ... Xingwei Qu Yangguang Li Wanli Ouyang Wenbo Su Bo Zheng RALM LLMAG 51 7 0 20 Jun 2024
InstructRAG: Instructing Retrieval-Augmented Generation via Self-Synthesized Rationales Zhepei Wei Wei-Lin Chen Yu Meng RALM 69 17 0 19 Jun 2024
Evaluating $n$ -Gram Novelty of Language Models Using Rusty-DAWG William Merrill Noah A. Smith Yanai Elazar ELM TDI 50 9 0 18 Jun 2024
Think-then-Act: A Dual-Angle Evaluated Retrieval-Augmented Generation Yige Shen Hao Jiang Hua Qu Jihong Zhao RALM LRM 42 1 0 18 Jun 2024
PlanRAG: A Plan-then-Retrieval Augmented Generation for Generative Large Language Models as Decision Makers Myeonghwa Lee Seonho An Min-Soo Kim 3DV RALM 44 18 0 18 Jun 2024
Iterative Utility Judgment Framework via LLMs Inspired by Relevance in Philosophy Hengran Zhang Keping Bi Jiafeng Guo Xueqi Cheng 42 1 0 17 Jun 2024
Refiner: Restructure Retrieval Content Efficiently to Advance Question-Answering Capabilities Zhonghao Li Xuming Hu Aiwei Liu Kening Zheng S. Huang Hui Xiong RALM 115 8 0 17 Jun 2024
BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack Yuri Kuratov Aydar Bulatov Petr Anokhin Ivan Rodkin Dmitry Sorokin Artyom Sorokin Andrey Kravchenko RALM ALM LRM ReLM ELM 51 61 0 14 Jun 2024
Integrating Large Language Models with Graph-based Reasoning for Conversational Question Answering Parag Jain Mirella Lapata 41 0 0 14 Jun 2024
Multi-Modal Retrieval For Large Language Model Based Speech Recognition J. Kolehmainen Aditya Gourav Prashanth Gurunath Shivakumar Yile Gu Ankur Gandhe Ariya Rastrow Grant P. Strimel I. Bulyko 40 4 0 13 Jun 2024
Action2Sound: Ambient-Aware Generation of Action Sounds from Egocentric Videos Changan Chen Puyuan Peng Ami Baid Zihui Xue Wei-Ning Hsu David Harwath Kristen Grauman VGen 42 8 0 13 Jun 2024
Supportiveness-based Knowledge Rewriting for Retrieval-augmented Language Modeling Zile Qiao Wei Ye Yong Jiang Tong Mo Pengjun Xie Weiping Li Fei Huang Shikun Zhang KELM 36 4 0 12 Jun 2024
QuickLLaMA: Query-aware Inference Acceleration for Large Language Models Jingyao Li Han Shi Xin Jiang Zhenguo Li Hong Xu Jiaya Jia LRM 35 2 0 11 Jun 2024
Efficient k-Nearest-Neighbor Machine Translation with Dynamic Retrieval Yan Gao Zhiwei Cao Zhongjian Miao Baosong Yang Shiyu Liu Min Zhang Jinsong Su 42 0 0 10 Jun 2024
Retrieval & Fine-Tuning for In-Context Tabular Models Valentin Thomas Junwei Ma Rasa Hosseinzadeh Keyvan Golestan Guangwei Yu M. Volkovs Anthony L. Caterini 41 12 0 07 Jun 2024
Speaker-Smoothed kNN Speaker Adaptation for End-to-End ASR Shaojun Li Daimeng Wei Hengchao Shang Jiaxin Guo Zongyao Li Zhanglin Wu Zhiqiang Rao Yuanchang Luo Xianghui He Hao Yang 26 1 0 07 Jun 2024
MATTER: Memory-Augmented Transformer Using Heterogeneous Knowledge Sources Dongkyu Lee Chandana Satya Prakash Jack G. M. FitzGerald Jens Lehmann RALM 41 2 0 07 Jun 2024
Memorization in deep learning: A survey Jiaheng Wei Yanjun Zhang Leo Yu Zhang Ming Ding Chao Chen Kok-Leong Ong Jun Zhang Yang Xiang 47 7 0 06 Jun 2024
XL-HeadTags: Leveraging Multimodal Retrieval Augmentation for the Multilingual Generation of News Headlines and Tags Faisal Tareque Shohan Mir Tafseer Nayeem Samsul Islam Abu Ubaida Akash Chenyu You 42 2 0 06 Jun 2024
Extended Mind Transformers Phoebe Klett Thomas Ahle RALM 29 0 0 04 Jun 2024
CtrlA: Adaptive Retrieval-Augmented Generation via Probe-Guided Control Huanshuo Liu Hao Zhang Zhijiang Guo Kuicai Dong Xiangyang Li Yi Quan Lee Cong Zhang Yong-jin Liu 3DV 41 1 0 29 May 2024
Nearest Neighbor Speculative Decoding for LLM Generation and Attribution Minghan Li Xilun Chen Ari Holtzman Beidi Chen Jimmy Lin Wen-tau Yih Xi Lin RALM BDL 108 10 0 29 May 2024
Superposed Decoding: Multiple Generations from a Single Autoregressive Inference Pass Ethan Shen Alan Fan Sarah M Pratt Jae Sung Park Matthew Wallingford Sham Kakade Ari Holtzman Ranjay Krishna Ali Farhadi Aditya Kusupati 53 2 0 28 May 2024
Accelerating Inference of Retrieval-Augmented Generation via Sparse Context Selection Yun Zhu Jia-Chen Gu Caitlin Sikora Ho Ko Yinxiao Liu ... Lei Shu Liangchen Luo Lei Meng Bang Liu Jindong Chen RALM 30 14 0 25 May 2024
Mixture of In-Context Prompters for Tabular PFNs Derek Xu Olcay Cirit Reza Asadi Yizhou Sun Wei Wang 41 9 0 25 May 2024
Perception of Knowledge Boundary for Large Language Models through Semi-open-ended Question Answering Zhihua Wen Zhiliang Tian Z. Jian Zhen Huang Pei Ke Yifu Gao Minlie Huang Dongsheng Li 46 9 0 23 May 2024
xRAG: Extreme Context Compression for Retrieval-augmented Generation with One Token Xin Cheng Xun Wang Xingxing Zhang Tao Ge Si-Qing Chen Furu Wei Huishuai Zhang Dongyan Zhao 67 30 0 22 May 2024
Automated Evaluation of Retrieval-Augmented Language Models with Task-Specific Exam Generation Gauthier Guinet Behrooz Omidvar-Tehrani Anoop Deoras Laurent Callot RALM 73 17 0 22 May 2024
Towards Retrieval-Augmented Architectures for Image Captioning Sara Sarto Marcella Cornia Lorenzo Baraldi Alessandro Nicolosi Rita Cucchiara VLM 32 10 0 21 May 2024