oLMpics -- On what Language Model Pre-training Captures

31 December 2019

Papers citing "oLMpics -- On what Language Model Pre-training Captures"

47 / 197 papers shown

Title
Measuring and Improving Consistency in Pretrained Language Models Yanai Elazar Nora Kassner Shauli Ravfogel Abhilasha Ravichander Eduard H. Hovy Hinrich Schütze Yoav Goldberg HILM 269 346 0 01 Feb 2021
Making Pre-trained Language Models Better Few-shot Learners Tianyu Gao Adam Fisch Danqi Chen 243 1,924 0 31 Dec 2020
Few-Shot Text Generation with Pattern-Exploiting Training Timo Schick Hinrich Schütze 31 145 0 22 Dec 2020
Learning from Mistakes: Using Mis-predictions as Harm Alerts in Language Pre-Training Chen Xing Wenhao Liu Caiming Xiong 26 0 0 16 Dec 2020
KgPLM: Knowledge-guided Language Model Pre-training via Generative and Discriminative Learning Bin He Xin Jiang Jinghui Xiao Qun Liu 37 18 0 07 Dec 2020
How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answering Zhengbao Jiang Jun Araki Haibo Ding Graham Neubig UQCV 31 410 0 02 Dec 2020
Automatically Identifying Words That Can Serve as Labels for Few-Shot Text Classification Timo Schick Helmut Schmid Hinrich Schütze VLM 11 206 0 26 Oct 2020
Rethinking embedding coupling in pre-trained language models Hyung Won Chung Thibault Févry Henry Tsai Melvin Johnson Sebastian Ruder 95 142 0 24 Oct 2020
It's not Greek to mBERT: Inducing Word-Level Translations from Multilingual BERT Hila Gonen Shauli Ravfogel Yanai Elazar Yoav Goldberg 26 50 0 16 Oct 2020
Formalizing Trust in Artificial Intelligence: Prerequisites, Causes and Goals of Human Trust in AI Alon Jacovi Ana Marasović Tim Miller Yoav Goldberg 255 426 0 15 Oct 2020
A Mathematical Exploration of Why Language Models Help Solve Downstream Tasks Nikunj Saunshi Sadhika Malladi Sanjeev Arora 20 86 0 07 Oct 2020
PRover: Proof Generation for Interpretable Reasoning over Rules Swarnadeep Saha Sayan Ghosh Shashank Srivastava Joey Tianyi Zhou ReLM LRM 34 77 0 06 Oct 2020
BERT Knows Punta Cana is not just beautiful, it's gorgeous: Ranking Scalar Adjectives with Contextualised Representations Aina Garí Soler Marianna Apidianaki 6 19 0 06 Oct 2020
On the Interplay Between Fine-tuning and Sentence-level Probing for Linguistic Knowledge in Pre-trained Transformers Marius Mosbach A. Khokhlova Michael A. Hedderich Dietrich Klakow 22 44 0 06 Oct 2020
Discern: Discourse-Aware Entailment Reasoning Network for Conversational Machine Reading Yifan Gao Chien-Sheng Wu Jingjing Li Chenyu You Guosheng Lin Caiming Xiong Irwin King M. Lyu FedML 25 51 0 05 Oct 2020
JAKET: Joint Pre-training of Knowledge Graph and Language Understanding Donghan Yu Chenguang Zhu Yiming Yang Michael Zeng KELM 11 142 0 02 Oct 2020
What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Exams Di Jin Eileen Pan Nassim Oufattole W. Weng Hanyi Fang Peter Szolovits FaML ELM LM&MA 17 697 0 28 Sep 2020
Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamics Swabha Swayamdipta Roy Schwartz Nicholas Lourie Yizhong Wang Hannaneh Hajishirzi Noah A. Smith Yejin Choi 44 429 0 22 Sep 2020
Compositional and Lexical Semantics in RoBERTa, BERT and DistilBERT: A Case Study on CoQA Ieva Staliunaite Ignacio Iacobacci 6 25 0 17 Sep 2020
Evaluating representations by the complexity of learning low-loss predictors William F. Whitney M. Song David Brandfonbrener Jaan Altosaar Kyunghyun Cho 25 23 0 15 Sep 2020
Critical Thinking for Language Models Gregor Betz Christian Voigt Kyle Richardson SyDa ReLM LRM AI4CE 20 35 0 15 Sep 2020
It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners Timo Schick Hinrich Schütze 36 955 0 15 Sep 2020
Neural Language Generation: Formulation, Methods, and Evaluation Cristina Garbacea Qiaozhu Mei 45 30 0 31 Jul 2020
Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering Gautier Izacard Edouard Grave RALM 22 1,108 0 02 Jul 2020
Are Pretrained Language Models Symbolic Reasoners Over Knowledge? Nora Kassner Benno Kroje Hinrich Schütze ReLM LRM 6 5 0 18 Jun 2020
Leap-Of-Thought: Teaching Pre-Trained Models to Systematically Reason Over Implicit Knowledge Alon Talmor Oyvind Tafjord Peter Clark Yoav Goldberg Jonathan Berant ReLM LRM 33 39 0 11 Jun 2020
Amnesic Probing: Behavioral Explanation with Amnesic Counterfactuals Yanai Elazar Shauli Ravfogel Alon Jacovi Yoav Goldberg 20 25 0 01 Jun 2020
Behind the Scene: Revealing the Secrets of Pre-trained Vision-and-Language Models Jize Cao Zhe Gan Yu Cheng Licheng Yu Yen-Chun Chen Jingjing Liu VLM 19 127 0 15 May 2020
Commonsense Evidence Generation and Injection in Reading Comprehension Ye Liu Tao Yang Zeyu You Wei Fan Philip S. Yu 33 14 0 11 May 2020
How Context Affects Language Models' Factual Predictions Fabio Petroni Patrick Lewis Aleksandra Piktus Tim Rocktaschel Yuxiang Wu Alexander H. Miller Sebastian Riedel KELM 19 228 0 10 May 2020
DQI: Measuring Data Quality in NLP Swaroop Mishra Anjana Arunkumar Bhavdeep Singh Sachdeva Chris Bryan Chitta Baral 36 30 0 02 May 2020
RICA: Evaluating Robust Inference Capabilities Based on Commonsense Axioms Pei Zhou Rahul Khanna Seyeon Lee Bill Yuchen Lin Daniel E. Ho Jay Pujara Xiang Ren ReLM 19 36 0 02 May 2020
Birds have four legs?! NumerSense: Probing Numerical Commonsense Knowledge of Pre-trained Language Models Bill Yuchen Lin Seyeon Lee Rahul Khanna Xiang Ren AIMat 8 154 0 02 May 2020
Probing Contextual Language Models for Common Ground with Visual Representations Gabriel Ilharco Rowan Zellers Ali Farhadi Hannaneh Hajishirzi 30 14 0 01 May 2020
Learning Music Helps You Read: Using Transfer to Study Linguistic Structure in Language Models Isabel Papadimitriou Dan Jurafsky 24 9 0 30 Apr 2020
Asking without Telling: Exploring Latent Ontologies in Contextual Representations Julian Michael Jan A. Botha Ian Tenney 31 42 0 29 Apr 2020
Explaining Question Answering Models through Text Generation Veronica Latcinnik Jonathan Berant LRM 13 51 0 12 Apr 2020
Probing Neural Language Models for Human Tacit Assumptions Nathaniel Weir Adam Poliak Benjamin Van Durme 21 6 0 10 Apr 2020
Information-Theoretic Probing with Minimum Description Length Elena Voita Ivan Titov 21 270 0 27 Mar 2020
A Primer in BERTology: What we know about how BERT works Anna Rogers Olga Kovaleva Anna Rumshisky OffRL 35 1,458 0 27 Feb 2020
CodeBERT: A Pre-Trained Model for Programming and Natural Languages Zhangyin Feng Daya Guo Duyu Tang Nan Duan Xiaocheng Feng ... Linjun Shou Bing Qin Ting Liu Daxin Jiang Ming Zhou 45 2,532 0 19 Feb 2020
Transformers as Soft Reasoners over Language Peter Clark Oyvind Tafjord Kyle Richardson ReLM OffRL LRM 31 346 0 14 Feb 2020
How Much Knowledge Can You Pack Into the Parameters of a Language Model? Adam Roberts Colin Raffel Noam M. Shazeer KELM 8 862 0 10 Feb 2020
K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapters Ruize Wang Duyu Tang Nan Duan Zhongyu Wei Xuanjing Huang Jianshu Ji Guihong Cao Daxin Jiang Ming Zhou KELM 19 545 0 05 Feb 2020
Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference Timo Schick Hinrich Schütze 258 1,589 0 21 Jan 2020
What Does My QA Model Know? Devising Controlled Probes using Expert Knowledge Kyle Richardson Ashish Sabharwal 25 45 0 31 Dec 2019
Language Models as Knowledge Bases? Fabio Petroni Tim Rocktaschel Patrick Lewis A. Bakhtin Yuxiang Wu Alexander H. Miller Sebastian Riedel KELM AI4MH 419 2,588 0 03 Sep 2019