MEMORY-VQ: Compression for Tractable Internet-Scale Memory

MEMORY-VQ: Compression for Tractable Internet-Scale Memory

28 August 2023

Yury Zemlyanskiy

Michiel de Jong

Santiago Ontañón

William W. Cohen

Sumit Sanghai

Joshua Ainslie

Papers citing "MEMORY-VQ: Compression for Tractable Internet-Scale Memory"

17 / 17 papers shown

Title
LAIT: Efficient Multi-Segment Encoding in Transformers with Layer-Adjustable Interaction Jeremiah Milbauer Annie Louis Mohammad Javad Hosseini Alex Fabrikant Donald Metzler Tal Schuster 90 9 0 31 May 2023
Decoupled Context Processing for Context Augmented Language Modeling Zonglin Li Ruiqi Guo Surinder Kumar RALM KELM 41 24 0 11 Oct 2022
Training Language Models with Memory Augmentation Zexuan Zhong Tao Lei Danqi Chen RALM 279 131 0 25 May 2022
Memorizing Transformers Yuhuai Wu M. Rabe DeLesley S. Hutchins Christian Szegedy RALM 68 177 0 16 Mar 2022
Large Dual Encoders Are Generalizable Retrievers Jianmo Ni Chen Qu Jing Lu Zhuyun Dai Gustavo Hernández Ábrego ... Vincent Zhao Yi Luan Keith B. Hall Ming-Wei Chang Yinfei Yang DML 138 450 0 15 Dec 2021
Improving language models by retrieving from trillions of tokens Sebastian Borgeaud A. Mensch Jordan Hoffmann Trevor Cai Eliza Rutherford ... Simon Osindero Karen Simonyan Jack W. Rae Erich Elsen Laurent Sifre KELM RALM 210 1,083 0 08 Dec 2021
ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction Keshav Santhanam Omar Khattab Jon Saad-Falcon Christopher Potts Matei A. Zaharia 81 405 0 02 Dec 2021
Mention Memory: incorporating textual knowledge into Transformers through entity mention attention Michiel de Jong Yury Zemlyanskiy Nicholas FitzGerald Fei Sha William W. Cohen RALM 68 47 0 12 Oct 2021
REALM: Retrieval-Augmented Language Model Pre-Training Kelvin Guu Kenton Lee Zora Tung Panupong Pasupat Ming-Wei Chang RALM 103 2,090 0 10 Feb 2020
Generalization through Memorization: Nearest Neighbor Language Models Urvashi Khandelwal Omer Levy Dan Jurafsky Luke Zettlemoyer M. Lewis RALM 146 837 0 01 Nov 2019
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 369 20,053 0 23 Oct 2019
HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering Zhilin Yang Peng Qi Saizheng Zhang Yoshua Bengio William W. Cohen Ruslan Salakhutdinov Christopher D. Manning RALM 147 2,635 0 25 Sep 2018
Adafactor: Adaptive Learning Rates with Sublinear Memory Cost Noam M. Shazeer Mitchell Stern ODL 72 1,043 0 11 Apr 2018
FEVER: a large-scale dataset for Fact Extraction and VERification James Thorne Andreas Vlachos Christos Christodoulopoulos Arpit Mittal HILM 121 1,646 0 14 Mar 2018
Neural Discrete Representation Learning Aaron van den Oord Oriol Vinyals Koray Kavukcuoglu BDL SSL OCL 208 4,989 0 02 Nov 2017
TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension Mandar Joshi Eunsol Choi Daniel S. Weld Luke Zettlemoyer RALM 195 2,636 0 09 May 2017
Billion-scale similarity search with GPUs Jeff Johnson Matthijs Douze Hervé Jégou 232 3,717 0 28 Feb 2017