ResearchTrend.AI
  • Papers
  • Communities
  • Events
  • Blog
  • Pricing
Papers
Communities
Social Events
Terms and Conditions
Pricing
Parameter LabParameter LabTwitterGitHubLinkedInBlueskyYoutube

© 2025 ResearchTrend.AI, All rights reserved.

  1. Home
  2. Papers
  3. 2112.04426
  4. Cited By
Improving language models by retrieving from trillions of tokens

Improving language models by retrieving from trillions of tokens

8 December 2021
Sebastian Borgeaud
A. Mensch
Jordan Hoffmann
Trevor Cai
Eliza Rutherford
Katie Millican
George van den Driessche
Jean-Baptiste Lespiau
Bogdan Damoc
Aidan Clark
Diego de Las Casas
Aurelia Guy
Jacob Menick
Roman Ring
Tom Hennigan
Saffron Huang
Lorenzo Maggiore
Chris Jones
Albin Cassirer
Andy Brock
Michela Paganini
G. Irving
Oriol Vinyals
Simon Osindero
Karen Simonyan
Jack W. Rae
Erich Elsen
Laurent Sifre
    KELM
    RALM
ArXivPDFHTML

Papers citing "Improving language models by retrieving from trillions of tokens"

50 / 722 papers shown
Title
Domain-Specific NER via Retrieving Correlated Samples
Domain-Specific NER via Retrieving Correlated Samples
Xin Zhang
Yong-jia Jiang
Xiaobin Wang
Xuming Hu
Yueheng Sun
Pengjun Xie
Meishan Zhang
30
14
0
27 Aug 2022
PEER: A Collaborative Language Model
PEER: A Collaborative Language Model
Timo Schick
Jane Dwivedi-Yu
Zhengbao Jiang
Fabio Petroni
Patrick Lewis
Gautier Izacard
Qingfei You
Christoforos Nalmpantis
Edouard Grave
Sebastian Riedel
ALM
50
93
0
24 Aug 2022
Retrieval-based Controllable Molecule Generation
Retrieval-based Controllable Molecule Generation
Zichao Wang
Weili Nie
Zhuoran Qiao
Chaowei Xiao
Richard Baraniuk
Anima Anandkumar
24
36
0
23 Aug 2022
Ered: Enhanced Text Representations with Entities and Descriptions
Ered: Enhanced Text Representations with Entities and Descriptions
Qinghua Zhao
Shuai Ma
Yu Lei
24
1
0
18 Aug 2022
Retrieval-Augmented Transformer for Image Captioning
Retrieval-Augmented Transformer for Image Captioning
Sara Sarto
Marcella Cornia
Lorenzo Baraldi
Rita Cucchiara
24
57
0
26 Jul 2022
Text-Guided Synthesis of Artistic Images with Retrieval-Augmented
  Diffusion Models
Text-Guided Synthesis of Artistic Images with Retrieval-Augmented Diffusion Models
Robin Rombach
A. Blattmann
Bjorn Ommer
DiffM
18
70
0
26 Jul 2022
Discrete Key-Value Bottleneck
Discrete Key-Value Bottleneck
Frederik Trauble
Anirudh Goyal
Nasim Rahaman
Michael C. Mozer
Kenji Kawaguchi
Yoshua Bengio
Bernhard Schölkopf
CLL
26
22
0
22 Jul 2022
MQRetNN: Multi-Horizon Time Series Forecasting with Retrieval
  Augmentation
MQRetNN: Multi-Horizon Time Series Forecasting with Retrieval Augmentation
Sitan Yang
Carson Eisenach
Dhruv Madeka
AI4TS
22
7
0
21 Jul 2022
Can large language models reason about medical questions?
Can large language models reason about medical questions?
Valentin Liévin
C. Hother
Andreas Geert Motzfeldt
Ole Winther
ELM
LM&MA
AI4MH
LRM
24
299
0
17 Jul 2022
Recent Developments in AI and USPTO Open Data
Recent Developments in AI and USPTO Open Data
Scott Beliveau
Jerry Ma
22
1
0
12 Jul 2022
TPU-KNN: K Nearest Neighbor Search at Peak FLOP/s
TPU-KNN: K Nearest Neighbor Search at Peak FLOP/s
Felix Chern
Blake A. Hechtman
Andy Davis
Ruiqi Guo
David Majnemer
Surinder Kumar
94
22
0
28 Jun 2022
ProGen2: Exploring the Boundaries of Protein Language Models
ProGen2: Exploring the Boundaries of Protein Language Models
Erik Nijkamp
Jeffrey A. Ruffolo
Eli N. Weinstein
Nikhil Naik
Ali Madani
AI4TS
22
282
0
27 Jun 2022
Emergent Abilities of Large Language Models
Emergent Abilities of Large Language Models
Jason W. Wei
Yi Tay
Rishi Bommasani
Colin Raffel
Barret Zoph
...
Tatsunori Hashimoto
Oriol Vinyals
Percy Liang
J. Dean
W. Fedus
ELM
ReLM
LRM
48
2,344
0
15 Jun 2022
LIFT: Language-Interfaced Fine-Tuning for Non-Language Machine Learning
  Tasks
LIFT: Language-Interfaced Fine-Tuning for Non-Language Machine Learning Tasks
Tuan Dinh
Yuchen Zeng
Ruisu Zhang
Ziqian Lin
Michael Gira
Shashank Rajput
Jy-yong Sohn
Dimitris Papailiopoulos
Kangwook Lee
LMTD
32
126
0
14 Jun 2022
Large-Scale Retrieval for Reinforcement Learning
Large-Scale Retrieval for Reinforcement Learning
Peter C. Humphreys
A. Guez
O. Tieleman
Laurent Sifre
T. Weber
Timothy Lillicrap
RALM
OffRL
37
26
0
10 Jun 2022
Improving Contrastive Learning of Sentence Embeddings with
  Case-Augmented Positives and Retrieved Negatives
Improving Contrastive Learning of Sentence Embeddings with Case-Augmented Positives and Retrieved Negatives
Wei Wang
Liangzhu Ge
Jingqiao Zhang
Cheng Yang
20
20
0
06 Jun 2022
Decoupling Knowledge from Memorization: Retrieval-augmented Prompt
  Learning
Decoupling Knowledge from Memorization: Retrieval-augmented Prompt Learning
Xiang Chen
Lei Li
Ningyu Zhang
Xiaozhuan Liang
Shumin Deng
Chuanqi Tan
Fei Huang
Luo Si
Huajun Chen
VLM
25
52
0
29 May 2022
Learning to Automate Follow-up Question Generation using Process
  Knowledge for Depression Triage on Reddit Posts
Learning to Automate Follow-up Question Generation using Process Knowledge for Depression Triage on Reddit Posts
Shrey Gupta
Anmol Agarwal
Manas Gaur
Kaushik Roy
Vignesh Narayanan
Ponnurangam Kumaraguru
Amit P. Sheth
AI4MH
14
34
0
27 May 2022
kNN-Prompt: Nearest Neighbor Zero-Shot Inference
kNN-Prompt: Nearest Neighbor Zero-Shot Inference
Weijia Shi
Julian Michael
Suchin Gururangan
Luke Zettlemoyer
RALM
VLM
15
32
0
27 May 2022
Tranception: protein fitness prediction with autoregressive transformers
  and inference-time retrieval
Tranception: protein fitness prediction with autoregressive transformers and inference-time retrieval
Pascal Notin
M. Dias
J. Frazer
Javier Marchena-Hurtado
Aidan N. Gomez
D. Marks
Y. Gal
55
177
0
27 May 2022
Training Language Models with Memory Augmentation
Training Language Models with Memory Augmentation
Zexuan Zhong
Tao Lei
Danqi Chen
RALM
234
128
0
25 May 2022
TALM: Tool Augmented Language Models
TALM: Tool Augmented Language Models
Aaron T Parisi
Yao-Min Zhao
Noah Fiedel
KELM
RALM
LLMAG
27
144
0
24 May 2022
Chunk-based Nearest Neighbor Machine Translation
Chunk-based Nearest Neighbor Machine Translation
Pedro Henrique Martins
Zita Marinho
André F.T. Martins
RALM
80
28
0
24 May 2022
Memorization Without Overfitting: Analyzing the Training Dynamics of
  Large Language Models
Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models
Kushal Tirumala
Aram H. Markosyan
Luke Zettlemoyer
Armen Aghajanyan
TDI
29
185
0
22 May 2022
Visually-Augmented Language Modeling
Visually-Augmented Language Modeling
Weizhi Wang
Li Dong
Hao Cheng
Haoyu Song
Xiaodong Liu
Xifeng Yan
Jianfeng Gao
Furu Wei
VLM
28
18
0
20 May 2022
A Generalist Agent
A Generalist Agent
Scott E. Reed
Konrad Zolna
Emilio Parisotto
Sergio Gomez Colmenarejo
Alexander Novikov
...
Yutian Chen
R. Hadsell
Oriol Vinyals
Mahyar Bordbar
Nando de Freitas
LM&Ro
LLMAG
AI4CE
54
785
0
12 May 2022
Asking for Knowledge: Training RL Agents to Query External Knowledge
  Using Language
Asking for Knowledge: Training RL Agents to Query External Knowledge Using Language
Iou-Jen Liu
Xingdi Yuan
Marc-Alexandre Côté
Pierre-Yves Oudeyer
A. Schwing
RALM
19
12
0
12 May 2022
Retrieval-Enhanced Machine Learning
Retrieval-Enhanced Machine Learning
Hamed Zamani
Fernando Diaz
Mostafa Dehghani
Donald Metzler
Michael Bendersky
19
49
0
02 May 2022
OPT: Open Pre-trained Transformer Language Models
OPT: Open Pre-trained Transformer Language Models
Susan Zhang
Stephen Roller
Naman Goyal
Mikel Artetxe
Moya Chen
...
Daniel Simig
Punit Singh Koura
Anjali Sridhar
Tianlu Wang
Luke Zettlemoyer
VLM
OSLM
AI4CE
54
3,488
0
02 May 2022
TemporalWiki: A Lifelong Benchmark for Training and Evaluating
  Ever-Evolving Language Models
TemporalWiki: A Lifelong Benchmark for Training and Evaluating Ever-Evolving Language Models
Joel Jang
Seonghyeon Ye
Changho Lee
Sohee Yang
Joongbo Shin
Janghoon Han
Gyeonghun Kim
Minjoon Seo
CLL
KELM
27
91
0
29 Apr 2022
Can deep learning match the efficiency of human visual long-term memory
  in storing object details?
Can deep learning match the efficiency of human visual long-term memory in storing object details?
Emin Orhan
VLM
OCL
22
0
0
27 Apr 2022
Semi-Parametric Neural Image Synthesis
Semi-Parametric Neural Image Synthesis
A. Blattmann
Robin Rombach
Kaan Oktay
Jonas Muller
Bjorn Ommer
DiffM
28
28
0
25 Apr 2022
ChapterBreak: A Challenge Dataset for Long-Range Language Models
ChapterBreak: A Challenge Dataset for Long-Range Language Models
Simeng Sun
Katherine Thai
Mohit Iyyer
10
19
0
22 Apr 2022
Standing on the Shoulders of Giant Frozen Language Models
Standing on the Shoulders of Giant Frozen Language Models
Yoav Levine
Itay Dalmedigos
Ori Ram
Yoel Zeldes
Daniel Jannai
...
Barak Lenz
Shai Shalev-Shwartz
Amnon Shashua
Kevin Leyton-Brown
Y. Shoham
VLM
27
49
0
21 Apr 2022
K-LITE: Learning Transferable Visual Models with External Knowledge
K-LITE: Learning Transferable Visual Models with External Knowledge
Sheng Shen
Chunyuan Li
Xiaowei Hu
Jianwei Yang
Yujia Xie
...
Ce Liu
Kurt Keutzer
Trevor Darrell
Anna Rohrbach
Jianfeng Gao
CLIP
VLM
33
83
0
20 Apr 2022
METRO: Efficient Denoising Pretraining of Large Scale Autoencoding
  Language Models with Model Generated Signals
METRO: Efficient Denoising Pretraining of Large Scale Autoencoding Language Models with Model Generated Signals
Payal Bajaj
Chenyan Xiong
Guolin Ke
Xiaodong Liu
Di He
Saurabh Tiwary
Tie-Yan Liu
Paul N. Bennett
Xia Song
Jianfeng Gao
44
32
0
13 Apr 2022
Training a Helpful and Harmless Assistant with Reinforcement Learning
  from Human Feedback
Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
Yuntao Bai
Andy Jones
Kamal Ndousse
Amanda Askell
Anna Chen
...
Jack Clark
Sam McCandlish
C. Olah
Benjamin Mann
Jared Kaplan
72
2,318
0
12 Apr 2022
Augmenting Pre-trained Language Models with QA-Memory for Open-Domain
  Question Answering
Augmenting Pre-trained Language Models with QA-Memory for Open-Domain Question Answering
Wenhu Chen
Pat Verga
Michiel de Jong
John Wieting
William W. Cohen
RALM
KELM
24
26
0
10 Apr 2022
Knowledge Base Index Compression via Dimensionality and Precision
  Reduction
Knowledge Base Index Compression via Dimensionality and Precision Reduction
Vilém Zouhar
Marius Mosbach
Miaoran Zhang
Dietrich Klakow
16
2
0
06 Apr 2022
KNN-Diffusion: Image Generation via Large-Scale Retrieval
KNN-Diffusion: Image Generation via Large-Scale Retrieval
Shelly Sheynin
Oron Ashual
Adam Polyak
Uriel Singer
Oran Gafni
Eliya Nachmani
Yaniv Taigman
VLM
SyDa
DiffM
19
113
0
06 Apr 2022
PaLM: Scaling Language Modeling with Pathways
PaLM: Scaling Language Modeling with Pathways
Aakanksha Chowdhery
Sharan Narang
Jacob Devlin
Maarten Bosma
Gaurav Mishra
...
Kathy Meier-Hellstern
Douglas Eck
J. Dean
Slav Petrov
Noah Fiedel
PILM
LRM
89
6,004
0
05 Apr 2022
Revisiting a kNN-based Image Classification System with High-capacity
  Storage
Revisiting a kNN-based Image Classification System with High-capacity Storage
K. Nakata
Youyang Ng
Daisuke Miyashita
A. Maki
Yu Lin
J. Deguchi
24
26
0
03 Apr 2022
PanGu-Bot: Efficient Generative Dialogue Pre-training from Pre-trained
  Language Model
PanGu-Bot: Efficient Generative Dialogue Pre-training from Pre-trained Language Model
Fei Mi
Yitong Li
Yulong Zeng
Jingyan Zhou
Yasheng Wang
Chuanfei Xu
Lifeng Shang
Xin Jiang
Shiqi Zhao
Qun Liu
ALM
37
18
0
31 Mar 2022
Training Compute-Optimal Large Language Models
Training Compute-Optimal Large Language Models
Jordan Hoffmann
Sebastian Borgeaud
A. Mensch
Elena Buchatskaya
Trevor Cai
...
Karen Simonyan
Erich Elsen
Jack W. Rae
Oriol Vinyals
Laurent Sifre
AI4TS
37
1,832
0
29 Mar 2022
Diagonal State Spaces are as Effective as Structured State Spaces
Diagonal State Spaces are as Effective as Structured State Spaces
Ankit Gupta
Albert Gu
Jonathan Berant
37
290
0
27 Mar 2022
Language Models that Seek for Knowledge: Modular Search & Generation for
  Dialogue and Prompt Completion
Language Models that Seek for Knowledge: Modular Search & Generation for Dialogue and Prompt Completion
Kurt Shuster
M. Komeili
Leonard Adolphs
Stephen Roller
Arthur Szlam
Jason Weston
KELM
37
121
0
24 Mar 2022
Teaching language models to support answers with verified quotes
Teaching language models to support answers with verified quotes
Jacob Menick
Maja Trebacz
Vladimir Mikulik
John Aslanides
Francis Song
...
Mia Glaese
Susannah Young
Lucy Campbell-Gillingham
G. Irving
Nat McAleese
ELM
RALM
240
257
0
21 Mar 2022
Reasoning over Public and Private Data in Retrieval-Based Systems
Reasoning over Public and Private Data in Retrieval-Based Systems
Simran Arora
Patrick Lewis
Angela Fan
Jacob Kahn
Christopher Ré
15
23
0
14 Mar 2022
Internet-augmented language models through few-shot prompting for
  open-domain question answering
Internet-augmented language models through few-shot prompting for open-domain question answering
Angeliki Lazaridou
E. Gribovskaya
Wojciech Stokowiec
N. Grigorev
KELM
LRM
20
131
0
10 Mar 2022
Finite-Sum Coupled Compositional Stochastic Optimization: Theory and
  Applications
Finite-Sum Coupled Compositional Stochastic Optimization: Theory and Applications
Bokun Wang
Tianbao Yang
38
31
0
24 Feb 2022
Previous
123...131415
Next