FiDO: Fusion-in-Decoder optimized for stronger performance and faster
inference

FiDO: Fusion-in-Decoder optimized for stronger performance and faster inference

15 December 2022

Michiel de Jong

Yury Zemlyanskiy

Joshua Ainslie

Nicholas FitzGerald

Sumit Sanghai

William W. Cohen

Papers citing "FiDO: Fusion-in-Decoder optimized for stronger performance and faster inference"

17 / 17 papers shown

Title
Yi: Open Foundation Models by 01.AI 01. AI Alex Young 01.AI Alex Young Bei Chen Chao Li ... Yue Wang Yuxuan Cai Zhenyu Gu Zhiyuan Liu Zonghong Dai OSLM LRM 233 550 0 07 Mar 2024
Can Open-Domain QA Reader Utilize External Knowledge Efficiently like Humans? Neeraj Varshney Man Luo Chitta Baral RALM 38 12 0 23 Nov 2022
Decoupled Context Processing for Context Augmented Language Modeling Zonglin Li Ruiqi Guo Surinder Kumar RALM KELM 41 24 0 11 Oct 2022
GLM-130B: An Open Bilingual Pre-trained Model Aohan Zeng Xiao Liu Zhengxiao Du Zihan Wang Hanyu Lai ... Jidong Zhai Wenguang Chen Peng Zhang Yuxiao Dong Jie Tang BDL LRM 342 1,090 0 05 Oct 2022
Generate rather than Retrieve: Large Language Models are Strong Context Generators Wenhao Yu Dan Iter Shuohang Wang Yichong Xu Mingxuan Ju Soumya Sanyal Chenguang Zhu Michael Zeng Meng Jiang RALM AIMat 294 335 0 21 Sep 2022
Memorizing Transformers Yuhuai Wu M. Rabe DeLesley S. Hutchins Christian Szegedy RALM 68 177 0 16 Mar 2022
LongT5: Efficient Text-To-Text Transformer for Long Sequences Mandy Guo Joshua Ainslie David C. Uthus Santiago Ontanon Jianmo Ni Yun-hsuan Sung Yinfei Yang VLM 55 312 0 15 Dec 2021
Improving language models by retrieving from trillions of tokens Sebastian Borgeaud A. Mensch Jordan Hoffmann Trevor Cai Eliza Rutherford ... Simon Osindero Karen Simonyan Jack W. Rae Erich Elsen Laurent Sifre KELM RALM 208 1,082 0 08 Dec 2021
Mention Memory: incorporating textual knowledge into Transformers through entity mention attention Michiel de Jong Yury Zemlyanskiy Nicholas FitzGerald Fei Sha William W. Cohen RALM 66 47 0 12 Oct 2021
KG-FiD: Infusing Knowledge Graph in Fusion-in-Decoder for Open-Domain Question Answering Donghan Yu Chenguang Zhu Yuwei Fang Wenhao Yu Shuohang Wang Yichong Xu Xiang Ren Yiming Yang Michael Zeng 50 90 0 08 Oct 2021
Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus Jesse Dodge Maarten Sap Ana Marasović William Agnew Gabriel Ilharco Dirk Groeneveld Margaret Mitchell Matt Gardner AILaw 96 443 0 18 Apr 2021
How Much Knowledge Can You Pack Into the Parameters of a Language Model? Adam Roberts Colin Raffel Noam M. Shazeer KELM 104 888 0 10 Feb 2020
REALM: Retrieval-Augmented Language Model Pre-Training Kelvin Guu Kenton Lee Zora Tung Panupong Pasupat Ming-Wei Chang RALM 103 2,090 0 10 Feb 2020
Fast Transformer Decoding: One Write-Head is All You Need Noam M. Shazeer 133 457 0 06 Nov 2019
Adafactor: Adaptive Learning Rates with Sublinear Memory Cost Noam M. Shazeer Mitchell Stern ODL 72 1,043 0 11 Apr 2018
TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension Mandar Joshi Eunsol Choi Daniel S. Weld Luke Zettlemoyer RALM 195 2,636 0 09 May 2017
Sequence Transduction with Recurrent Neural Networks Alex Graves 169 1,866 0 14 Nov 2012

We use cookies and other tracking technologies to improve your browsing experience on our website, to show you personalized content and targeted ads, to analyze our website traffic, and to understand where our visitors are coming from. See our policy.