Pre-Training Transformers as Energy-Based Cloze Models

Pre-Training Transformers as Energy-Based Cloze Models

15 December 2020

Minh-Thang Luong

Christopher D. Manning

Papers citing "Pre-Training Transformers as Energy-Based Cloze Models"

19 / 19 papers shown

Title
Entangled Relations: Leveraging NLI and Meta-analysis to Enhance Biomedical Relation Extraction William Hogan Jingbo Shang 18 0 0 31 May 2024
Controlled Text Generation for Black-box Language Models via Score-based Progressive Editor Sangwon Yu Changmin Lee Hojin Lee Sungroh Yoon 29 0 0 13 Nov 2023
Refashioning Emotion Recognition Modelling: The Advent of Generalised Large Models Zixing Zhang Liyizhe Peng Tao Pang Jing Han Huan Zhao Bjorn W. Schuller 40 13 0 21 Aug 2023
Pseudo Outlier Exposure for Out-of-Distribution Detection using Pretrained Transformers Jaeyoung Kim Kyuheon Jung Dongbin Na Sion Jang Eunbin Park Sungchul Choi OODD 38 6 0 18 Jul 2023
Exploiting Abstract Meaning Representation for Open-Domain Question Answering Cunxiang Wang Zhikun Xu Qipeng Guo Xiangkun Hu Xuefeng Bai Zheng-Wei Zhang Yue Zhang 25 3 0 26 May 2023
Exploring Energy-based Language Models with Different Architectures and Training Methods for Speech Recognition Hong Liu Z. Lv Zhijian Ou Wenbo Zhao Qing Xiao 24 0 0 22 May 2023
An Overview on Language Models: Recent Developments and Outlook Chengwei Wei Yun Cheng Wang Bin Wang C.-C. Jay Kuo 30 42 0 10 Mar 2023
Unsupervised Term Extraction for Highly Technical Domains Francesco Fusco Peter W. J. Staar Diego Antognini 25 4 0 24 Oct 2022
Robustifying Sentiment Classification by Maximally Exploiting Few Counterfactuals Maarten De Raedt Fréderic Godin Chris Develder Thomas Demeester 13 1 0 21 Oct 2022
On Reinforcement Learning and Distribution Matching for Fine-Tuning Language Models with no Catastrophic Forgetting Tomasz Korbak Hady ElSahar Germán Kruszewski Marc Dymetman CLL 25 51 0 01 Jun 2022
Representation Learning by Detecting Incorrect Location Embeddings Sepehr Sameni Simon Jenni Paolo Favaro ViT 34 4 0 10 Apr 2022
Lacuna Reconstruction: Self-supervised Pre-training for Low-Resource Historical Document Transcription Nikolai Vogler J. Allen M. Miller Taylor Berg-Kirkpatrick 32 5 0 16 Dec 2021
Contrastive Document Representation Learning with Graph Attention Networks Peng Xu Xinchi Chen Xiaofei Ma Zhiheng Huang Bing Xiang 14 9 0 20 Oct 2021
Frustratingly Simple Pretraining Alternatives to Masked Language Modeling Atsuki Yamaguchi G. Chrysostomou Katerina Margatina Nikolaos Aletras 27 25 0 04 Sep 2021
FewCLUE: A Chinese Few-shot Learning Evaluation Benchmark Liang Xu Xiaojing Lu Chenyang Yuan Xuanwei Zhang Huilin Xu ... Guoao Wei X. Pan Xin Tian Libo Qin Hai Hu ELM 24 56 0 15 Jul 2021
How to Train BERT with an Academic Budget Peter Izsak Moshe Berchansky Omer Levy 23 113 0 15 Apr 2021
A Primer on Contrastive Pretraining in Language Processing: Methods, Lessons Learned and Perspectives Nils Rethmeier Isabelle Augenstein SSL VLM 94 91 0 25 Feb 2021
WARP: Word-level Adversarial ReProgramming Karen Hambardzumyan Hrant Khachatrian Jonathan May AAML 254 342 0 01 Jan 2021
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 299 6,984 0 20 Apr 2018