REALM: Retrieval-Augmented Language Model Pre-Training

10 February 2020

Papers citing "REALM: Retrieval-Augmented Language Model Pre-Training"

50 / 505 papers shown

Title
An Empirical Study of Retrieval-enhanced Graph Neural Networks Dingmin Wang Shengchao Liu Hanchen Wang Bernardo Cuenca Grau Linfeng Song Jian Tang Song Le Qi Liu 21 0 0 01 Jun 2022
Neural Retriever and Go Beyond: A Thesis Proposal Man Luo 35 1 0 31 May 2022
Fast and Light-Weight Answer Text Retrieval in Dialogue Systems H. Wan S. Patel J. William Murdock Saloni Potdar Sachindra Joshi 32 1 0 27 May 2022
Learning to Automate Follow-up Question Generation using Process Knowledge for Depression Triage on Reddit Posts Shrey Gupta Anmol Agarwal Manas Gaur Kaushik Roy Vignesh Narayanan Ponnurangam Kumaraguru Amit P. Sheth AI4MH 22 34 0 27 May 2022
kNN-Prompt: Nearest Neighbor Zero-Shot Inference Weijia Shi Julian Michael Suchin Gururangan Luke Zettlemoyer RALM VLM 26 32 0 27 May 2022
Optimizing Test-Time Query Representations for Dense Retrieval Mujeen Sung Jungsoo Park Jaewoo Kang Danqi Chen Jinhyuk Lee 33 5 0 25 May 2022
Training Language Models with Memory Augmentation Zexuan Zhong Tao Lei Danqi Chen RALM 242 128 0 25 May 2022
TALM: Tool Augmented Language Models Aaron T Parisi Yao-Min Zhao Noah Fiedel KELM RALM LLMAG 38 144 0 24 May 2022
RetroMAE: Pre-Training Retrieval-oriented Language Models Via Masked Auto-Encoder Shitao Xiao Zheng Liu Yingxia Shao Bo Zhao RALM 120 110 0 24 May 2022
Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models Kushal Tirumala Aram H. Markosyan Luke Zettlemoyer Armen Aghajanyan TDI 29 187 0 22 May 2022
CORAL: Contextual Response Retrievability Loss Function for Training Dialog Generation Models Bishal Santra Ravi Ghadia Manish Gupta Pawan Goyal OffRL 23 0 0 21 May 2022
Retrieval-Augmented Multilingual Keyphrase Generation with Retriever-Generator Iterative Training Yifan Gao Qingyu Yin Zheng Li Rui Meng Tong Zhao Bing Yin Irwin King M. Lyu RALM 36 20 0 21 May 2022
Can Foundation Models Wrangle Your Data? A. Narayan Ines Chami Laurel J. Orr Simran Arora Christopher Ré LMTD AI4CE 181 214 0 20 May 2022
RankGen: Improving Text Generation with Large Ranking Models Kalpesh Krishna Yapei Chang John Wieting Mohit Iyyer AIMat 24 68 0 19 May 2022
Modeling Exemplification in Long-form Question Answering via Retrieval Shufan Wang Fangyuan Xu Laure Thompson Eunsol Choi Mohit Iyyer 41 10 0 19 May 2022
SKILL: Structured Knowledge Infusion for Large Language Models Fedor Moiseev Zhe Dong Enrique Alfonseca Martin Jaggi KELM 58 58 0 17 May 2022
POLITICS: Pretraining with Same-story Article Comparison for Ideology Prediction and Stance Detection Yujian Liu Xinliang Frederick Zhang David Wegsman Nick Beauchamp Lu Wang 34 71 0 02 May 2022
TemporalWiki: A Lifelong Benchmark for Training and Evaluating Ever-Evolving Language Models Joel Jang Seonghyeon Ye Changho Lee Sohee Yang Joongbo Shin Janghoon Han Gyeonghun Kim Minjoon Seo CLL KELM 27 92 0 29 Apr 2022
Faithful to the Document or to the World? Mitigating Hallucinations via Entity-linked Knowledge in Abstractive Summarization Yue Dong John Wieting Pat Verga HILM 24 24 0 28 Apr 2022
Synthetic Target Domain Supervision for Open Retrieval QA Revanth Reddy Gangi Reddy Bhavani Iyer Md Arafat Sultan Rong Zhang Avirup Sil Vittorio Castelli Radu Florian Salim Roukos OOD 36 12 0 20 Apr 2022
Modality-Balanced Embedding for Video Retrieval Xun Wang Bingqing Ke Xuanping Li Fangyu Liu Mingyu Zhang Xiao Liang Qi-En Xiao Cheng Luo Yue Yu 27 9 0 18 Apr 2022
Knowledgeable Salient Span Mask for Enhancing Language Models as Knowledge Base Cunxiang Wang Fuli Luo Yanyang Li Runxin Xu Fei Huang Yue Zhang KELM 36 2 0 17 Apr 2022
Unsupervised Cross-Task Generalization via Retrieval Augmentation Bill Yuchen Lin Kangmin Tan Chris Miller Beiwen Tian Xiang Ren LRM RALM 27 48 0 17 Apr 2022
Improving Passage Retrieval with Zero-Shot Question Generation Devendra Singh Sachan M. Lewis Mandar Joshi Armen Aghajanyan Wen-tau Yih J. Pineau Luke Zettlemoyer OOD LRM 33 157 0 15 Apr 2022
METRO: Efficient Denoising Pretraining of Large Scale Autoencoding Language Models with Model Generated Signals Payal Bajaj Chenyan Xiong Guolin Ke Xiaodong Liu Di He Saurabh Tiwary Tie-Yan Liu Paul N. Bennett Xia Song Jianfeng Gao 50 32 0 13 Apr 2022
Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback Yuntao Bai Andy Jones Kamal Ndousse Amanda Askell Anna Chen ... Jack Clark Sam McCandlish C. Olah Benjamin Mann Jared Kaplan 77 2,341 0 12 Apr 2022
What Language Model Architecture and Pretraining Objective Work Best for Zero-Shot Generalization? Thomas Wang Adam Roberts Daniel Hesslow Teven Le Scao Hyung Won Chung Iz Beltagy Julien Launay Colin Raffel 39 167 0 12 Apr 2022
Knowledge Infused Decoding Ruibo Liu Guoqing Zheng Shashank Gupta Radhika Gaonkar Chongyang Gao Soroush Vosoughi Milad Shokouhi Ahmed Hassan Awadallah KELM 25 14 0 06 Apr 2022
$k$ NN-NER: Named Entity Recognition with Nearest Neighbor Search Shuhe Wang Xiaoya Li Yuxian Meng Tianwei Zhang Rongbin Ouyang Jiwei Li Guoyin Wang 29 11 0 31 Mar 2022
Entity-driven Fact-aware Abstractive Summarization of Biomedical Literature Amanuel Alambo Tanvi Banerjee K. Thirunarayan M. Raymer MedIm 18 7 0 30 Mar 2022
LinkBERT: Pretraining Language Models with Document Links Michihiro Yasunaga J. Leskovec Percy Liang KELM 29 353 0 29 Mar 2022
Training Compute-Optimal Large Language Models Jordan Hoffmann Sebastian Borgeaud A. Mensch Elena Buchatskaya Trevor Cai ... Karen Simonyan Erich Elsen Jack W. Rae Oriol Vinyals Laurent Sifre AI4TS 69 1,846 0 29 Mar 2022
Fine-tuning Image Transformers using Learnable Memory Mark Sandler A. Zhmoginov Max Vladymyrov Andrew Jackson ViT 29 47 0 29 Mar 2022
ANNA: Enhanced Language Representation for Question Answering Changwook Jun Hansol Jang Myoseop Sim Hyun Kim Jooyoung Choi Kyungkoo Min Kyunghoon Bae 31 6 0 28 Mar 2022
Towards Abstractive Grounded Summarization of Podcast Transcripts Kaiqiang Song Chen Li Xiaoyang Wang Dong Yu Fei Liu 35 9 0 22 Mar 2022
RELIC: Retrieving Evidence for Literary Claims Katherine Thai Yapei Chang Kalpesh Krishna Mohit Iyyer RALM 23 16 0 18 Mar 2022
elBERto: Self-supervised Commonsense Learning for Question Answering Xunlin Zhan Yuan Li Xiao Dong Xiaodan Liang Zhiting Hu Lawrence Carin SSL RALM LRM 24 7 0 17 Mar 2022
C-MORE: Pretraining to Answer Open-Domain Questions by Consulting Millions of References Xiang Yue Xiaoman Pan Wenlin Yao Dian Yu Dong Yu Jianshu Chen RALM LRM 8 6 0 16 Mar 2022
Memorizing Transformers Yuhuai Wu M. Rabe DeLesley S. Hutchins Christian Szegedy RALM 30 173 0 16 Mar 2022
Representation Learning for Resource-Constrained Keyphrase Generation Di Wu Wasi Uddin Ahmad Sunipa Dev Kai-Wei Chang 41 17 0 15 Mar 2022
Interpretability for Language Learners Using Example-Based Grammatical Error Correction Masahiro Kaneko Sho Takase Ayana Niwa Naoaki Okazaki 41 26 0 14 Mar 2022
Reasoning over Public and Private Data in Retrieval-Based Systems Simran Arora Patrick Lewis Angela Fan Jacob Kahn Christopher Ré 23 23 0 14 Mar 2022
LaPraDoR: Unsupervised Pretrained Dense Retriever for Zero-Shot Text Retrieval Canwen Xu Daya Guo Nan Duan Julian McAuley RALM VLM 24 46 0 11 Mar 2022
Internet-augmented language models through few-shot prompting for open-domain question answering Angeliki Lazaridou E. Gribovskaya Wojciech Stokowiec N. Grigorev KELM LRM 20 131 0 10 Mar 2022
Read before Generate! Faithful Long Form Question Answering with Machine Reading Dan Su Xiaoguang Li Jindi Zhang Lifeng Shang Xin Jiang Qun Liu Pascale Fung HILM 19 59 0 01 Mar 2022
A Mutually Reinforced Framework for Pretrained Sentence Embeddings Junhan Yang Zheng Liu Shitao Xiao Jianxun Lian Lijun Wu Defu Lian Guangzhong Sun Xing Xie SSL 16 2 0 28 Feb 2022
Subgraph Retrieval Enhanced Model for Multi-hop Knowledge Base Question Answering Jing Zhang Xiaokang Zhang Jifan Yu Jian Tang Jie Tang Cuiping Li Hong Chen 25 118 0 27 Feb 2022
Finite-Sum Coupled Compositional Stochastic Optimization: Theory and Applications Bokun Wang Tianbao Yang 48 31 0 24 Feb 2022
Retrieval Augmented Classification for Long-Tail Visual Recognition Alex Long Wei Yin Thalaiyasingam Ajanthan Vu-Linh Nguyen Pulak Purkait Ravi Garg Alan Blair Chunhua Shen Anton Van Den Hengel 21 107 0 22 Feb 2022
Retrieval-Augmented Reinforcement Learning Anirudh Goyal A. Friesen Andrea Banino T. Weber Nan Rosemary Ke ... Michal Valko Simon Osindero Timothy Lillicrap N. Heess Charles Blundell OffRL 32 53 0 17 Feb 2022