REALM: Retrieval-Augmented Language Model Pre-Training

10 February 2020

Papers citing "REALM: Retrieval-Augmented Language Model Pre-Training"

50 / 488 papers shown

Title
Bridging the Gap between Language Model and Reading Comprehension: Unsupervised MRC via Self-Supervision Ning Bian Xianpei Han Bo Chen Hongyu Lin Xianpei Han Le Sun SSL LRM 29 5 0 19 Jul 2021
Noise Stability Regularization for Improving BERT Fine-tuning Hang Hua Xingjian Li Dejing Dou Chengzhong Xu Jiebo Luo 19 43 0 10 Jul 2021
Time-Aware Language Models as Temporal Knowledge Bases Bhuwan Dhingra Jeremy R. Cole Julian Martin Eisenschlos D. Gillick Jacob Eisenstein William W. Cohen KELM 28 264 0 29 Jun 2021
Multimodal Few-Shot Learning with Frozen Language Models Maria Tsimpoukelli Jacob Menick Serkan Cabi S. M. Ali Eslami Oriol Vinyals Felix Hill MLLM 55 749 0 25 Jun 2021
Extreme Multi-label Learning for Semantic Matching in Product Search Wei-Cheng Chang Daniel Jiang Hsiang-Fu Yu C. Teo Jiong Zhang ... Qie Hu Nikhil Shandilya Vyacheslav Ievgrafov Japinder Singh Inderjit S. Dhillon 45 59 0 23 Jun 2021
Weakly Supervised Pre-Training for Multi-Hop Retriever Yeon Seonwoo Sang-Woo Lee Ji-Hoon Kim Jung-Woo Ha Alice H. Oh RALM 16 8 0 18 Jun 2021
Pre-Trained Models: Past, Present and Future Xu Han Zhengyan Zhang Ning Ding Yuxian Gu Xiao Liu ... Jie Tang Ji-Rong Wen Jinhui Yuan Wayne Xin Zhao Jun Zhu AIFin MQ AI4MH 40 815 0 14 Jun 2021
Can Generative Pre-trained Language Models Serve as Knowledge Bases for Closed-book QA? Cunxiang Wang Pai Liu Yue Zhang RALM 25 80 0 03 Jun 2021
Good for Misconceived Reasons: An Empirical Revisiting on the Need for Visual Context in Multimodal Machine Translation Zhiyong Wu Lingpeng Kong W. Bi Xiang Li B. Kao LRM 20 77 0 30 May 2021
Neural Machine Translation with Monolingual Translation Memory Deng Cai Yan Wang Huayang Li Wai Lam Lemao Liu 21 101 0 24 May 2021
Retrieval-Augmented Transformer-XL for Close-Domain Dialog Generation Giovanni Bonetta R. Cancelliere Ding Liu Paul Vozila RALM 21 16 0 19 May 2021
RetGen: A Joint framework for Retrieval and Grounded Text Generation Modeling Yizhe Zhang Siqi Sun Xiang Gao Yuwei Fang Chris Brockett Michel Galley Jianfeng Gao Bill Dolan RALM 38 30 0 14 May 2021
REPT: Bridging Language Models and Machine Reading Comprehension via Retrieval-Based Pre-training Fangkai Jiao Yangyang Guo Yilin Niu Feng Ji Feng-Lin Li Liqiang Nie LRM 28 12 0 10 May 2021
D2S: Document-to-Slide Generation Via Query-Based Text Summarization Edward Sun Yufang Hou Dakuo Wang Yunfeng Zhang N. Wang 27 34 0 08 May 2021
A Dataset of Information-Seeking Questions and Answers Anchored in Research Papers Pradeep Dasigi Kyle Lo Iz Beltagy Arman Cohan Noah A. Smith Matt Gardner RALM 36 279 0 07 May 2021
Hybrid Encoder: Towards Efficient and Precise Native AdsRecommendation via Hybrid Transformer Encoding Networks Junhan Yang Zheng Liu Bowen Jin Jianxun Lian Defu Lian Akshay Soni Eun Yong Kang Yajun Wang Guangzhong Sun Xing Xie 43 1 0 22 Apr 2021
Efficient Retrieval Optimized Multi-task Learning He Fun S. Gandhi Sujith Ravi RALM 18 6 0 20 Apr 2021
Case-based Reasoning for Natural Language Queries over Knowledge Bases Rajarshi Das Manzil Zaheer Dung Ngoc Thai Ameya Godbole Ethan Perez Jay Yoon Lee Lizhen Tan L. Polymenakos Andrew McCallum 28 162 0 18 Apr 2021
Can NLI Models Verify QA Systems' Predictions? Jifan Chen Eunsol Choi Greg Durrett 25 54 0 18 Apr 2021
BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models Nandan Thakur Nils Reimers Andreas Rucklé Abhishek Srivastava Iryna Gurevych VLM 237 971 0 17 Apr 2021
Joint Passage Ranking for Diverse Multi-Answer Retrieval Sewon Min Kenton Lee Ming-Wei Chang Kristina Toutanova Hannaneh Hajishirzi 21 39 0 17 Apr 2021
Cross-Modal Retrieval Augmentation for Multi-Modal Classification Shir Gur Natalia Neverova C. Stauffer Ser-Nam Lim Douwe Kiela A. Reiter 16 26 0 16 Apr 2021
Towards Robust Neural Retrieval Models with Synthetic Pre-Training Revanth Reddy Gangi Reddy Vikas Yadav Md Arafat Sultan M. Franz Vittorio Castelli Heng Ji Avirup Sil 18 14 0 15 Apr 2021
Semantic maps and metrics for science Semantic maps and metrics for science using deep transformer encoders Brendan Chambers James A. Evans MedIm 13 0 0 13 Apr 2021
Relational World Knowledge Representation in Contextual Language Models: A Review Tara Safavi Danai Koutra KELM 35 51 0 12 Apr 2021
What's in a Summary? Laying the Groundwork for Advances in Hospital-Course Summarization Griffin Adams Emily Alsentzer Mert Ketenci Jason Zucker Noémie Elhadad 50 47 0 12 Apr 2021
Few-shot Intent Classification and Slot Filling with Retrieved Examples Dian Yu Luheng He Yuan Zhang Xinya Du Panupong Pasupat Qi Li VLM 23 50 0 12 Apr 2021
Action-Based Conversations Dataset: A Corpus for Building More In-Depth Task-Oriented Dialogue Systems Derek Chen Howard Chen Yi Yang A. Lin Zhou Yu 17 65 0 01 Apr 2021
A Neighbourhood Framework for Resource-Lean Content Flagging Sheikh Muhammad Sarwar Dimitrina Zlatkova Momchil Hardalov Yoan Dinkov Isabelle Augenstein Preslav Nakov 21 5 0 31 Mar 2021
The NLP Cookbook: Modern Recipes for Transformer based Deep Learning Architectures Sushant Singh A. Mahmood AI4TS 60 92 0 23 Mar 2021
Open Domain Question Answering over Tables via Dense Retrieval Jonathan Herzig Thomas Müller Syrine Krichene Julian Martin Eisenschlos LMTD VLM RALM 41 99 0 22 Mar 2021
LightningDOT: Pre-training Visual-Semantic Embeddings for Real-Time Image-Text Retrieval Siqi Sun Yen-Chun Chen Linjie Li Shuohang Wang Yuwei Fang Jingjing Liu VLM 38 82 0 16 Mar 2021
Open-book Video Captioning with Retrieve-Copy-Generate Network Ziqi Zhang Zhongang Qi Chun Yuan Ying Shan Bing Li Ying Deng Weiming Hu 31 92 0 09 Mar 2021
Pruning the Index Contents for Memory Efficient Open-Domain QA Martin Fajcik Martin Docekal Karel Ondrej Pavel Smrz 23 8 0 21 Feb 2021
Going Full-TILT Boogie on Document Understanding with Text-Image-Layout Transformer Rafal Powalski Łukasz Borchmann Dawid Jurkiewicz Tomasz Dwojak Michal Pietruszka Gabriela Pałka ViT 36 157 0 18 Feb 2021
Training Large-Scale News Recommenders with Pretrained Language Models in the Loop Shitao Xiao Zheng Liu Yingxia Shao Tao Di Xing Xie VLM AIFin 127 41 0 18 Feb 2021
Reasoning Over Virtual Knowledge Bases With Open Predicate Relations Haitian Sun Pat Verga Bhuwan Dhingra Ruslan Salakhutdinov William W. Cohen LRM 39 26 0 14 Feb 2021
PAQ: 65 Million Probably-Asked Questions and What You Can Do With Them Patrick Lewis Yuxiang Wu Linqing Liu Pasquale Minervini Heinrich Küttler Aleksandra Piktus Pontus Stenetorp Sebastian Riedel RALM 40 229 0 13 Feb 2021
When Can Models Learn From Explanations? A Formal Framework for Understanding the Roles of Explanation Data Peter Hase Joey Tianyi Zhou XAI 25 87 0 03 Feb 2021
Mind the Gap: Assessing Temporal Generalization in Neural Language Models Angeliki Lazaridou A. Kuncoro E. Gribovskaya Devang Agrawal Adam Liska ... Sebastian Ruder Dani Yogatama Kris Cao Susannah Young Phil Blunsom VLM 41 207 0 03 Feb 2021
Multilingual LAMA: Investigating Knowledge in Multilingual Pretrained Language Models Nora Kassner Philipp Dufter Hinrich Schütze 21 132 0 01 Feb 2021
Can We Automate Scientific Reviewing? Weizhe Yuan Pengfei Liu Graham Neubig 83 84 0 30 Jan 2021
Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity W. Fedus Barret Zoph Noam M. Shazeer MoE 11 2,075 0 11 Jan 2021
Retrieving and Reading: A Comprehensive Survey on Open-domain Question Answering Fengbin Zhu Wenqiang Lei Chao Wang Jianming Zheng Soujanya Poria Tat-Seng Chua RALM 213 252 0 04 Jan 2021
Few-Shot Question Answering by Pretraining Span Selection Ori Ram Yuval Kirstain Jonathan Berant Amir Globerson Omer Levy 30 97 0 02 Jan 2021
Baleen: Robust Multi-Hop Reasoning at Scale via Condensed Retrieval Omar Khattab Christopher Potts Matei A. Zaharia RALM LRM 27 55 0 02 Jan 2021
Studying Strategically: Learning to Mask for Closed-book QA Qinyuan Ye Belinda Z. Li Sinong Wang Benjamin Bolte Hao Ma Wen-tau Yih Xiang Ren Madian Khabsa OffRL 21 11 0 31 Dec 2020
Evidence-based Factual Error Correction James Thorne Andreas Vlachos KELM OffRL 24 54 0 31 Dec 2020
UniK-QA: Unified Representations of Structured and Unstructured Knowledge for Open-Domain Question Answering Barlas Oğuz Xilun Chen Vladimir Karpukhin Stanislav Peshterliev Dmytro Okhonko M. Schlichtkrull Sonal Gupta Yashar Mehdad Scott Yih 211 87 0 29 Dec 2020
The Curse of Dense Low-Dimensional Information Retrieval for Large Index Sizes Nils Reimers Iryna Gurevych 18 58 0 28 Dec 2020