REALM: Retrieval-Augmented Language Model Pre-Training

10 February 2020

Papers citing "REALM: Retrieval-Augmented Language Model Pre-Training"

50 / 488 papers shown

Title
Improving language models by retrieving from trillions of tokens Sebastian Borgeaud A. Mensch Jordan Hoffmann Trevor Cai Eliza Rutherford ... Simon Osindero Karen Simonyan Jack W. Rae Erich Elsen Laurent Sifre KELM RALM 90 1,024 0 08 Dec 2021
Human Parity on CommonsenseQA: Augmenting Self-Attention with External Attention Yichong Xu Chenguang Zhu Shuohang Wang Siqi Sun Hao Cheng Xiaodong Liu Jianfeng Gao Pengcheng He Michael Zeng Xuedong Huang LRM 254 55 0 06 Dec 2021
ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction Keshav Santhanam Omar Khattab Jon Saad-Falcon Christopher Potts Matei A. Zaharia 39 383 0 02 Dec 2021
Refined Commonsense Knowledge from Large-Scale Web Contents Tuan-Phong Nguyen Simon Razniewski Julien Romero Gerhard Weikum 39 32 0 30 Nov 2021
Building a Question Answering System for the Manufacturing Domain Liu Xingguang Cheng Zhenbo Shen Zhengyuan Zhang Haoxin Meng Hangcheng Xu Xuesong Xiao Gang 32 9 0 19 Nov 2021
Achieving Human Parity on Visual Question Answering Ming Yan Haiyang Xu Chenliang Li Junfeng Tian Bin Bi ... Ji Zhang Songfang Huang Fei Huang Luo Si Rong Jin 32 12 0 17 Nov 2021
Recent Advances in Automated Question Answering In Biomedical Domain K. D. Baksi 25 0 0 10 Nov 2021
DEEPAGÉ: Answering Questions in Portuguese about the Brazilian Environment Flávio N. Caccao M. M. José A. Oliveira Stefano Spindola A. H. R. Costa Fabio Gagliardi Cozman 28 6 0 19 Oct 2021
GNN-LM: Language Modeling based on Global Contexts via GNN Yuxian Meng Shi Zong Xiaoya Li Xiaofei Sun Tianwei Zhang Fei Wu Jiwei Li LRM 24 37 0 17 Oct 2021
Attacking Open-domain Question Answering by Injecting Misinformation Liangming Pan Wenhu Chen Min-Yen Kan Luu Anh Tuan HILM AAML 201 22 0 15 Oct 2021
Hindsight: Posterior-guided training of retrievers for improved open-ended generation Ashwin Paranjape Omar Khattab Christopher Potts Matei A. Zaharia Christopher D. Manning RALM 72 43 0 14 Oct 2021
Retrieval-guided Counterfactual Generation for QA Bhargavi Paranjape Matthew Lamm Ian Tenney 33 31 0 14 Oct 2021
Zero-Shot Dense Retrieval with Momentum Adversarial Domain Invariant Representations Ji Xin Chenyan Xiong A. Srinivasan Ankita Sharma Damien Jose Paul N. Bennett VLM 83 41 0 14 Oct 2021
RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-ranking Ruiyang Ren Yingqi Qu Jing Liu Wayne Xin Zhao Qiaoqiao She Hua Wu Haifeng Wang Ji-Rong Wen 141 249 0 14 Oct 2021
Building Chinese Biomedical Language Models via Multi-Level Text Discrimination Quan Wang Songtai Dai Benfeng Xu Yajuan Lyu Yong Zhu Hua Wu Haifeng Wang 71 14 0 14 Oct 2021
Cross-Lingual Open-Domain Question Answering with Answer Sentence Generation Benjamin Muller Luca Soldaini Rik Koncel-Kedziorski Eric Lind Alessandro Moschitti LRM 18 7 0 14 Oct 2021
ConditionalQA: A Complex Reading Comprehension Dataset with Conditional Answers Haitian Sun William W. Cohen Ruslan Salakhutdinov 67 33 0 13 Oct 2021
Mention Memory: incorporating textual knowledge into Transformers through entity mention attention Michiel de Jong Yury Zemlyanskiy Nicholas FitzGerald Fei Sha William W. Cohen RALM 29 46 0 12 Oct 2021
Distantly-Supervised Evidence Retrieval Enables Question Answering without Evidence Annotation Chen Zhao Chenyan Xiong Jordan L. Boyd-Graber Hal Daumé RALM 21 8 0 10 Oct 2021
The Inductive Bias of In-Context Learning: Rethinking Pretraining Example Design Yoav Levine Noam Wies Daniel Jannai D. Navon Yedid Hoshen Amnon Shashua AI4CE 35 36 0 09 Oct 2021
KG-FiD: Infusing Knowledge Graph in Fusion-in-Decoder for Open-Domain Question Answering Donghan Yu Chenguang Zhu Yuwei Fang W. Yu Shuohang Wang Yichong Xu Xiang Ren Yiming Yang Michael Zeng 27 88 0 08 Oct 2021
Towards Continual Knowledge Learning of Language Models Joel Jang Seonghyeon Ye Sohee Yang Joongbo Shin Janghoon Han Gyeonghun Kim Stanley Jungkyu Choi Minjoon Seo CLL KELM 230 151 0 07 Oct 2021
Encoder Adaptation of Dense Passage Retrieval for Open-Domain Question Answering Minghan Li Jimmy J. Lin AI4CE 25 9 0 04 Oct 2021
Perhaps PTLMs Should Go to School -- A Task to Assess Open Book and Closed Book QA Manuel R. Ciosici Joe Cecil Alex Hedges Dong-Ho Lee Marjorie Freedman R. Weischedel 27 9 0 04 Oct 2021
BeliefBank: Adding Memory to a Pre-Trained Language Model for a Systematic Notion of Belief Nora Kassner Oyvind Tafjord Hinrich Schütze Peter Clark KELM LRM 245 64 0 29 Sep 2021
Instance-Based Neural Dependency Parsing Hiroki Ouchi Jun Suzuki Sosuke Kobayashi Sho Yokoi Tatsuki Kuribayashi Masashi Yoshikawa Kentaro Inui 34 3 0 28 Sep 2021
MultiDoc2Dial: Modeling Dialogues Grounded in Multiple Documents Song Feng S. Patel H. Wan Sachindra Joshi 57 66 0 26 Sep 2021
More Than Reading Comprehension: A Survey on Datasets and Metrics of Textual Question Answering Yang Bai D. Wang 93 10 0 25 Sep 2021
Towards Universal Dense Retrieval for Open-domain Question Answering Christopher Sciavolino RALM 8 1 0 23 Sep 2021
RETRONLU: Retrieval Augmented Task-Oriented Semantic Parsing Vivek Gupta Akshat Shrivastava Adithya Sagar Armen Aghajanyan Denis Savenkov RALM 40 21 0 21 Sep 2021
Relation-Guided Pre-Training for Open-Domain Question Answering Ziniu Hu Yizhou Sun Kai-Wei Chang RALM OnRL 28 6 0 21 Sep 2021
ConvFiT: Conversational Fine-Tuning of Pretrained Language Models Ivan Vulić Pei-hao Su Sam Coope D. Gerz Paweł Budzianowski I. Casanueva Nikola Mrkvsić Tsung-Hsien Wen 27 36 0 21 Sep 2021
SPLADE v2: Sparse Lexical and Expansion Model for Information Retrieval Thibault Formal Carlos Lassance Benjamin Piwowarski S. Clinchant 202 186 0 21 Sep 2021
Slot Filling for Biomedical Information Extraction Yannis Papanikolaou Marlene Staib Justin Grace Francine Bennett MedIm 35 12 0 17 Sep 2021
Simple Entity-Centric Questions Challenge Dense Retrievers Christopher Sciavolino Zexuan Zhong Jinhyuk Lee Danqi Chen RALM 24 159 0 17 Sep 2021
A Bag of Tricks for Dialogue Summarization Muhammad Khalifa Miguel Ballesteros Kathleen McKeown 27 27 0 16 Sep 2021
Phrase Retrieval Learns Passage Retrieval, Too Jinhyuk Lee Alexander Wettig Danqi Chen RALM DML 27 46 0 16 Sep 2021
Summarize-then-Answer: Generating Concise Explanations for Multi-hop Reading Comprehension Naoya Inoue H. Trivedi Steven K. Sinha Niranjan Balasubramanian Kentaro Inui 58 14 0 14 Sep 2021
Adaptive Information Seeking for Open-Domain Question Answering Yunchang Zhu Liang Pang Yanyan Lan Huawei Shen Xueqi Cheng RALM 33 39 0 14 Sep 2021
InceptionXML: A Lightweight Framework with Synchronized Negative Sampling for Short Text Extreme Classification Siddhant Kharbanda Atmadeep Banerjee Devaansh Gupta Akash Palrecha Rohit Babbar 32 9 0 13 Sep 2021
SituatedQA: Incorporating Extra-Linguistic Contexts into QA Michael J.Q. Zhang Eunsol Choi RALM 32 136 0 13 Sep 2021
Question Answering over Electronic Devices: A New Benchmark Dataset and a Multi-Task Learning based QA Framework Abhilash Nandy Soumya Sharma Shubham Maddhashiya K. Sachdeva Pawan Goyal Niloy Ganguly 22 16 0 13 Sep 2021
Entity-Based Knowledge Conflicts in Question Answering Shayne Longpre Kartik Perisetla Anthony Chen Nikhil Ramesh Chris DuBois Sameer Singh HILM 245 237 0 10 Sep 2021
Enhancing Natural Language Representation with Large-Scale Out-of-Domain Commonsense Wanyun Cui Xingran Chen 22 6 0 06 Sep 2021
Robust Retrieval Augmented Generation for Zero-shot Slot Filling Michael R. Glass Gaetano Rossiello Md. Faisal Mahbub Chowdhury A. Gliozzo RALM 36 32 0 31 Aug 2021
PAIR: Leveraging Passage-Centric Similarity Relation for Improving Dense Passage Retrieval Ruiyang Ren Shangwen Lv Yingqi Qu Jing Liu Wayne Xin Zhao Qiaoqiao She Hua Wu Haifeng Wang Ji-Rong Wen 128 91 0 13 Aug 2021
Knowledge-intensive Language Understanding for Explainable AI A. Sheth Manas Gaur Kaushik Roy Keyur Faldu 19 48 0 02 Aug 2021
Domain-matched Pre-training Tasks for Dense Retrieval Barlas Oğuz Kushal Lakhotia Anchit Gupta Patrick Lewis Vladimir Karpukhin ... Xilun Chen Sebastian Riedel Wen-tau Yih Sonal Gupta Yashar Mehdad RALM 23 66 0 28 Jul 2021
Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing Pengfei Liu Weizhe Yuan Jinlan Fu Zhengbao Jiang Hiroaki Hayashi Graham Neubig VLM SyDa 40 3,838 0 28 Jul 2021
One Question Answering Model for Many Languages with Cross-lingual Dense Passage Retrieval Akari Asai Xinyan Velocity Yu Jungo Kasai Hannaneh Hajishirzi RALM LRM 24 70 0 26 Jul 2021