Decoupling Knowledge from Memorization: Retrieval-augmented Prompt Learning

29 May 2022

Xiang Chen

Lei Li

Ningyu Zhang

Xiaozhuan Liang

Shumin Deng

Chuanqi Tan

Fei Huang

Luo Si

Huajun Chen

VLM

ArXiv PDF HTML

Papers citing "Decoupling Knowledge from Memorization: Retrieval-augmented Prompt Learning"

50 / 50 papers shown

Title
An Empirical Study of Memorization in NLP Xiaosen Zheng Jing Jiang TDI 65 24 1 23 Mar 2022
Do Prompts Solve NLP Tasks Using Natural Language? Sen Yang Yunchen Zhang Leyang Cui Yue Zhang LRM 57 4 0 02 Mar 2022
Neuro-Symbolic Language Modeling with Automaton-augmented Retrieval Uri Alon Frank F. Xu Junxian He Sudipta Sengupta Dan Roth Graham Neubig RALM 104 63 0 28 Jan 2022
Learning To Retrieve Prompts for In-Context Learning Ohad Rubin Jonathan Herzig Jonathan Berant VPVLM RALM 79 701 0 16 Dec 2021
Improving language models by retrieving from trillions of tokens Sebastian Borgeaud A. Mensch Jordan Hoffmann Trevor Cai Eliza Rutherford ... Simon Osindero Karen Simonyan Jack W. Rae Erich Elsen Laurent Sifre KELM RALM 230 1,083 0 08 Dec 2021
GNN-LM: Language Modeling based on Global Contexts via GNN Yuxian Meng Shi Zong Xiaoya Li Xiaofei Sun Tianwei Zhang Leilei Gan Jiwei Li LRM 48 38 0 17 Oct 2021
Good Examples Make A Faster Learner: Simple Demonstration-based Learning for Low-resource NER Dong-Ho Lee Akshen Kadakia Kangmin Tan Mahak Agarwal Xinyu Feng Takashi Shibuya Ryosuke Mitani Toshiyuki Sekiya Jay Pujara Xiang Ren 55 87 0 16 Oct 2021
Multitask Prompted Training Enables Zero-Shot Task Generalization Victor Sanh Albert Webson Colin Raffel Stephen H. Bach Lintang Sutawika ... T. Bers Stella Biderman Leo Gao Thomas Wolf Alexander M. Rush LRM 337 1,700 0 15 Oct 2021
MSP: Multi-Stage Prompting for Making Pre-trained Language Models Better Translators Zhixing Tan Xiangwen Zhang Shuo Wang Yang Liu VLM LRM 234 51 0 13 Oct 2021
Learning to Ask for Data-Efficient Event Argument Extraction Hongbin Ye Ningyu Zhang Zhen Bi Shumin Deng Chuanqi Tan Hui Chen Fei Huang Huajun Chen RALM 50 11 0 01 Oct 2021
Template-free Prompt Tuning for Few-shot NER Ruotian Ma Xin Zhou Tao Gui Y. Tan Linyang Li Qi Zhang Xuanjing Huang VLM 184 181 0 28 Sep 2021
Efficient Nearest Neighbor Language Models Junxian He Graham Neubig Taylor Berg-Kirkpatrick RALM 217 104 0 09 Sep 2021
Finetuned Language Models Are Zero-Shot Learners Jason W. Wei Maarten Bosma Vincent Zhao Kelvin Guu Adams Wei Yu Brian Lester Nan Du Andrew M. Dai Quoc V. Le ALM UQCV 120 3,742 0 03 Sep 2021
Differentiable Prompt Makes Pre-trained Language Models Better Few-shot Learners Ningyu Zhang Luoqiu Li Xiang Chen Shumin Deng Zhen Bi Chuanqi Tan Fei Huang Huajun Chen VLM 103 179 0 30 Aug 2021
Prompt-Learning for Fine-Grained Entity Typing Ning Ding Yulin Chen Xu Han Guangwei Xu Pengjun Xie Haitao Zheng Zhiyuan Liu Juan-Zi Li Hong-Gee Kim 72 159 0 24 Aug 2021
Knowledgeable Prompt-tuning: Incorporating Knowledge into Prompt Verbalizer for Text Classification Shengding Hu Ning Ding Huadong Wang Zhiyuan Liu Jingang Wang Juan-Zi Li Wei Wu Maosong Sun VLM 82 367 0 04 Aug 2021
Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing Pengfei Liu Weizhe Yuan Jinlan Fu Zhengbao Jiang Hiroaki Hayashi Graham Neubig VLM SyDa 187 3,964 0 28 Jul 2021
Reordering Examples Helps during Priming-based Few-Shot Learning Sawan Kumar Partha P. Talukdar 47 58 0 03 Jun 2021
PTR: Prompt Tuning with Rules for Text Classification Xu Han Weilin Zhao Ning Ding Zhiyuan Liu Maosong Sun VLM 96 524 0 24 May 2021
Few-NERD: A Few-Shot Named Entity Recognition Dataset Ning Ding Guangwei Xu Yulin Chen Xiaobin Wang Xu Han Pengjun Xie Haitao Zheng Zhiyuan Liu 68 234 0 16 May 2021
Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity Yao Lu Max Bartolo Alastair Moore Sebastian Riedel Pontus Stenetorp AILaw LRM 387 1,181 0 18 Apr 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 517 4,032 0 18 Apr 2021
Memorisation versus Generalisation in Pre-trained Language Models Michael Tänzer Sebastian Ruder Marek Rei 106 51 0 16 Apr 2021
KnowPrompt: Knowledge-aware Prompt-tuning with Synergistic Optimization for Relation Extraction Xiang Chen Ningyu Zhang Xin Xie Shumin Deng Yunzhi Yao Chuanqi Tan Fei Huang Luo Si Huajun Chen 89 413 0 15 Apr 2021
GPT Understands, Too Xiao Liu Yanan Zheng Zhengxiao Du Ming Ding Yujie Qian Zhilin Yang Jie Tang VLM 155 1,173 0 18 Mar 2021
Memorization vs. Generalization: Quantifying Data Leakage in NLP Performance Evaluation Aparna Elangovan Jiayuan He Karin Verspoor TDI FedML 201 93 0 03 Feb 2021
What Makes Good In-Context Examples for GPT- $3$ ? Jiachang Liu Dinghan Shen Yizhe Zhang Bill Dolan Lawrence Carin Weizhu Chen AAML RALM 364 1,374 0 17 Jan 2021
Prefix-Tuning: Optimizing Continuous Prompts for Generation Xiang Lisa Li Percy Liang 215 4,247 0 01 Jan 2021
Making Pre-trained Language Models Better Few-shot Learners Tianyu Gao Adam Fisch Danqi Chen 364 1,964 0 31 Dec 2020
Automatically Identifying Words That Can Serve as Labels for Few-Shot Text Classification Timo Schick Helmut Schmid Hinrich Schütze VLM 75 208 0 26 Oct 2020
Nearest Neighbor Machine Translation Urvashi Khandelwal Angela Fan Dan Jurafsky Luke Zettlemoyer M. Lewis RALM 59 285 0 01 Oct 2020
What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimation Vitaly Feldman Chiyuan Zhang TDI 140 462 0 09 Aug 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 713 41,894 0 28 May 2020
BERT-kNN: Adding a kNN Search Component to Pretrained Language Models for Better QA Nora Kassner Hinrich Schütze RALM 72 68 0 02 May 2020
Dense Passage Retrieval for Open-Domain Question Answering Vladimir Karpukhin Barlas Oğuz Sewon Min Patrick Lewis Ledell Yu Wu Sergey Edunov Danqi Chen Wen-tau Yih RALM 154 3,749 0 10 Apr 2020
REALM: Retrieval-Augmented Language Model Pre-Training Kelvin Guu Kenton Lee Zora Tung Panupong Pasupat Ming-Wei Chang RALM 103 2,093 0 10 Feb 2020
PyTorch: An Imperative Style, High-Performance Deep Learning Library Adam Paszke Sam Gross Francisco Massa Adam Lerer James Bradbury ... Sasank Chilamkurthy Benoit Steiner Lu Fang Junjie Bai Soumith Chintala ODL 408 42,393 0 03 Dec 2019
Generalization through Memorization: Nearest Neighbor Language Models Urvashi Khandelwal Omer Levy Dan Jurafsky Luke Zettlemoyer M. Lewis RALM 149 837 0 01 Nov 2019
BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension M. Lewis Yinhan Liu Naman Goyal Marjan Ghazvininejad Abdel-rahman Mohamed Omer Levy Veselin Stoyanov Luke Zettlemoyer AIMat VLM 229 10,815 0 29 Oct 2019
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks Nils Reimers Iryna Gurevych 1.2K 12,181 0 27 Aug 2019
RoBERTa: A Robustly Optimized BERT Pretraining Approach Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy M. Lewis Luke Zettlemoyer Veselin Stoyanov AIMat 545 24,422 0 26 Jul 2019
SpanBERT: Improving Pre-training by Representing and Predicting Spans Mandar Joshi Danqi Chen Yinhan Liu Daniel S. Weld Luke Zettlemoyer Omer Levy 133 1,964 0 24 Jul 2019
Does Learning Require Memorization? A Short Tale about a Long Tail Vitaly Feldman TDI 119 494 0 12 Jun 2019
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.7K 94,729 0 11 Oct 2018
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 1.1K 7,152 0 20 Apr 2018
Unbounded cache model for online language modeling with open vocabulary Edouard Grave Moustapha Cissé Armand Joulin KELM CLL 41 63 0 07 Nov 2017
A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference Adina Williams Nikita Nangia Samuel R. Bowman 515 4,476 0 18 Apr 2017
Understanding Black-box Predictions via Influence Functions Pang Wei Koh Percy Liang TDI 169 2,882 0 14 Mar 2017
Billion-scale similarity search with GPUs Jeff Johnson Matthijs Douze Hervé Jégou 253 3,720 0 28 Feb 2017
SQuAD: 100,000+ Questions for Machine Comprehension of Text Pranav Rajpurkar Jian Zhang Konstantin Lopyrev Percy Liang RALM 256 8,124 0 16 Jun 2016