Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing

28 July 2021

Graham Neubig

Papers citing "Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing"

50 / 231 papers shown

Title
Constrained Language Models Yield Few-Shot Semantic Parsers Richard Shin C. H. Lin Sam Thomson Charles C. Chen Subhro Roy Emmanouil Antonios Platanios Adam Pauls Dan Klein J. Eisner Benjamin Van Durme 391 206 0 18 Apr 2021
MT6: Multilingual Pretrained Text-to-Text Transformer with Translation Pairs Zewen Chi Li Dong Shuming Ma Shaohan Huang Xian-Ling Mao Heyan Huang Furu Wei LRM 106 74 0 18 Apr 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 589 4,093 0 18 Apr 2021
Surface Form Competition: Why the Highest Probability Answer Isn't Always Right Ari Holtzman Peter West Vered Schwartz Yejin Choi Luke Zettlemoyer LRM 126 239 0 16 Apr 2021
KnowPrompt: Knowledge-aware Prompt-tuning with Synergistic Optimization for Relation Extraction Xiang Chen Ningyu Zhang Xin Xie Shumin Deng Yunzhi Yao Chuanqi Tan Fei Huang Luo Si Huajun Chen 141 416 0 15 Apr 2021
Generating Datasets with Pretrained Language Models Timo Schick Hinrich Schütze 156 235 0 15 Apr 2021
RefSum: Refactoring Neural Summarization Yixin Liu Zi-Yi Dou Pengfei Liu 57 40 0 15 Apr 2021
Learning How to Ask: Querying LMs with Mixtures of Soft Prompts Guanghui Qin J. Eisner 63 549 0 14 Apr 2021
Factual Probing Is [MASK]: Learning vs. Learning to Recall Zexuan Zhong Dan Friedman Danqi Chen 56 411 0 12 Apr 2021
Improving and Simplifying Pattern Exploiting Training Derek Tam Rakesh R Menon Joey Tianyi Zhou Shashank Srivastava Colin Raffel 74 151 0 22 Mar 2021
GPT Understands, Too Xiao Liu Yanan Zheng Zhengxiao Du Ming Ding Yujie Qian Zhilin Yang Jie Tang VLM 168 1,182 0 18 Mar 2021
How Many Data Points is a Prompt Worth? Teven Le Scao Alexander M. Rush VLM 164 303 0 15 Mar 2021
BERTese: Learning to Speak to BERT Adi Haviv Jonathan Berant Amir Globerson 77 124 0 09 Mar 2021
Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLP Timo Schick Sahana Udupa Hinrich Schütze 313 387 0 28 Feb 2021
PADA: Example-based Prompt Learning for on-the-fly Adaptation to Unseen Domains Eyal Ben-David Nadav Oved Roi Reichart VLM OOD 61 94 0 24 Feb 2021
Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm Laria Reynolds Kyle McDonell 114 918 0 15 Feb 2021
Can We Automate Scientific Reviewing? Weizhe Yuan Pengfei Liu Graham Neubig 145 89 0 30 Jan 2021
What Makes Good In-Context Examples for GPT- $3$ ? Jiachang Liu Dinghan Shen Yizhe Zhang Bill Dolan Lawrence Carin Weizhu Chen AAML RALM 390 1,390 0 17 Jan 2021
Prefix-Tuning: Optimizing Continuous Prompts for Generation Xiang Lisa Li Percy Liang 252 4,305 0 01 Jan 2021
WARP: Word-level Adversarial ReProgramming Karen Hambardzumyan Hrant Khachatrian Jonathan May AAML 334 353 0 01 Jan 2021
Making Pre-trained Language Models Better Few-shot Learners Tianyu Gao Adam Fisch Danqi Chen 404 1,975 0 31 Dec 2020
ERNIE-M: Enhanced Multilingual Representation by Aligning Cross-lingual Semantics with Monolingual Corpora Ouyang Xuan Shuohuan Wang Chao Pang Yu Sun Hao Tian Hua Wu Haifeng Wang 125 102 0 31 Dec 2020
Few-Shot Text Generation with Pattern-Exploiting Training Timo Schick Hinrich Schütze 96 148 0 22 Dec 2020
Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learning Zeyuan Allen-Zhu Yuanzhi Li FedML 151 375 0 17 Dec 2020
CTRLsum: Towards Generic Controllable Text Summarization Junxian He Wojciech Kry'sciñski Bryan McCann Nazneen Rajani Caiming Xiong 258 142 0 08 Dec 2020
How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answering Zhengbao Jiang Jun Araki Haibo Ding Graham Neubig UQCV 60 437 0 02 Dec 2020
CPM: A Large-scale Generative Chinese Pre-trained Language Model Zhengyan Zhang Xu Han Hao Zhou Pei Ke Yuxian Gu ... Wentao Han Jie Tang Juan-Zi Li Xiaoyan Zhu Maosong Sun 68 118 0 01 Dec 2020
AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts Taylor Shin Yasaman Razeghi Robert L Logan IV Eric Wallace Sameer Singh KELM 89 402 0 29 Oct 2020
Automatically Identifying Words That Can Serve as Labels for Few-Shot Text Classification Timo Schick Helmut Schmid Hinrich Schütze VLM 82 208 0 26 Oct 2020
ERNIE-Gram: Pre-Training with Explicitly N-Gram Masked Language Modeling for Natural Language Understanding Dongling Xiao Yukun Li Han Zhang Yu Sun Hao Tian Hua Wu Haifeng Wang 29 39 0 23 Oct 2020
mT5: A massively multilingual pre-trained text-to-text transformer Linting Xue Noah Constant Adam Roberts Mihir Kale Rami Al-Rfou Aditya Siddhant Aditya Barua Colin Raffel 148 2,560 0 22 Oct 2020
GSum: A General Framework for Guided Neural Abstractive Summarization Zi-Yi Dou Pengfei Liu Hiroaki Hayashi Zhengbao Jiang Graham Neubig 66 259 0 15 Oct 2020
Improving Natural Language Processing Tasks with Human Gaze-Guided Neural Attention Ekta Sood Simon Tannert Philipp Mueller Andreas Bulling 79 73 0 15 Oct 2020
A Wrong Answer or a Wrong Question? An Intricate Relationship between Question Reformulation and Answer Selection in Conversational Question Answering Svitlana Vakulenko Shayne Longpre Zhucheng Tu R. Anantha 48 13 0 13 Oct 2020
X-FACTR: Multilingual Factual Knowledge Retrieval from Pretrained Language Models Zhengbao Jiang Antonios Anastasopoulos Jun Araki Haibo Ding Graham Neubig HILM KELM 76 144 0 13 Oct 2020
A Survey of Knowledge-Enhanced Text Generation Wenhao Yu Chenguang Zhu Zaitang Li Zhiting Hu Qingyun Wang Heng Ji Meng Jiang 109 288 0 09 Oct 2020
A Mathematical Exploration of Why Language Models Help Solve Downstream Tasks Nikunj Saunshi Sadhika Malladi Sanjeev Arora 85 89 0 07 Oct 2020
RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models Samuel Gehman Suchin Gururangan Maarten Sap Yejin Choi Noah A. Smith 168 1,221 0 24 Sep 2020
It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners Timo Schick Hinrich Schütze 132 976 0 15 Sep 2020
DeBERTa: Decoding-enhanced BERT with Disentangled Attention Pengcheng He Xiaodong Liu Jianfeng Gao Weizhu Chen AAML 165 2,754 0 05 Jun 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 889 42,463 0 28 May 2020
TaBERT: Pretraining for Joint Understanding of Textual and Tabular Data Pengcheng Yin Graham Neubig Wen-tau Yih Sebastian Riedel RALM LMTD 99 606 0 17 May 2020
How Context Affects Language Models' Factual Predictions Fabio Petroni Patrick Lewis Aleksandra Piktus Tim Rocktaschel Yuxiang Wu Alexander H. Miller Sebastian Riedel KELM 62 239 0 10 May 2020
UnifiedQA: Crossing Format Boundaries With a Single QA System Daniel Khashabi Sewon Min Tushar Khot Ashish Sabharwal Oyvind Tafjord Peter Clark Hannaneh Hajishirzi 150 742 0 02 May 2020
XCOPA: A Multilingual Dataset for Causal Commonsense Reasoning Edoardo Ponti Goran Glavaš Olga Majewska Qianchu Liu Ivan Vulić Anna Korhonen LRM 86 328 0 01 May 2020
Abstractive Summarization with Combination of Pre-trained Sequence-to-Sequence and Saliency Models Itsumi Saito Kyosuke Nishida Kosuke Nishida J. Tomita 65 29 0 29 Mar 2020
Pre-trained Models for Natural Language Processing: A Survey Xipeng Qiu Tianxiang Sun Yige Xu Yunfan Shao Ning Dai Xuanjing Huang LM&MA VLM 377 1,489 0 18 Mar 2020
UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training Hangbo Bao Li Dong Furu Wei Wenhui Wang Nan Yang ... Yu Wang Songhao Piao Jianfeng Gao Ming Zhou H. Hon AI4CE 88 394 0 28 Feb 2020
Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stopping Jesse Dodge Gabriel Ilharco Roy Schwartz Ali Farhadi Hannaneh Hajishirzi Noah A. Smith 101 597 0 15 Feb 2020
REALM: Retrieval-Augmented Language Model Pre-Training Kelvin Guu Kenton Lee Zora Tung Panupong Pasupat Ming-Wei Chang RALM 145 2,118 0 10 Feb 2020