ALBERT: A Lite BERT for Self-supervised Learning of Language Representations

26 September 2019

Papers citing "ALBERT: A Lite BERT for Self-supervised Learning of Language Representations"

50 / 2,913 papers shown

Title
New Vietnamese Corpus for Machine Reading Comprehension of Health News Articles Kiet Van Nguyen Tin Van Huynh Duc-Vu Nguyen A. Nguyen Ngan Luu-Thuy Nguyen 21 40 0 19 Jun 2020
Neural Parameter Allocation Search Bryan A. Plummer Nikoli Dryden Julius Frost Torsten Hoefler Kate Saenko 22 16 0 18 Jun 2020
Self-supervised Learning for Speech Enhancement Yuchun Wang Shrikant Venkataramani Paris Smaragdis SSL 19 31 0 18 Jun 2020
I-BERT: Inductive Generalization of Transformer to Arbitrary Context Lengths Hyoungwook Nam S. Seo Vikram Sharma Malithody Noor Michael Lang Li 24 1 0 18 Jun 2020
PERL: Pivot-based Domain Adaptation for Pre-trained Deep Contextualized Embedding Models Eyal Ben-David Carmel Rabinovitz Roi Reichart SSL 58 61 0 16 Jun 2020
Self-supervised Learning: Generative or Contrastive Xiao Liu Fanjin Zhang Zhenyu Hou Zhaoyu Wang Li Mian Jing Zhang Jie Tang SSL 52 1,587 0 15 Jun 2020
How to Avoid Being Eaten by a Grue: Structured Exploration Strategies for Textual Worlds Prithviraj Ammanabrolu Ethan Tien Matthew J. Hausknecht Mark O. Riedl LLMAG 24 50 0 12 Jun 2020
SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020) Marcos Zampieri Preslav Nakov Sara Rosenthal Pepa Atanasova Georgi Karadzhov Hamdy Mubarak Leon Derczynski Zeses Pitenis cCaugri cColtekin 30 482 0 12 Jun 2020
A Practical Sparse Approximation for Real Time Recurrent Learning Jacob Menick Erich Elsen Utku Evci Simon Osindero Karen Simonyan Alex Graves 21 31 0 12 Jun 2020
NanoFlow: Scalable Normalizing Flows with Sublinear Parameter Complexity Sang-gil Lee Sungwon Kim Sungroh Yoon 24 17 0 11 Jun 2020
A Monolingual Approach to Contextualized Word Embeddings for Mid-Resource Languages Pedro Ortiz Suarez Laurent Romary Benoît Sagot 28 227 0 11 Jun 2020
Revisiting Few-sample BERT Fine-tuning Tianyi Zhang Felix Wu Arzoo Katiyar Kilian Q. Weinberger Yoav Artzi 41 441 0 10 Jun 2020
MC-BERT: Efficient Language Pre-Training via a Meta Controller Zhenhui Xu Linyuan Gong Guolin Ke Di He Shuxin Zheng Liwei Wang Jiang Bian Tie-Yan Liu BDL 19 18 0 10 Jun 2020
On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselines Marius Mosbach Maksym Andriushchenko Dietrich Klakow 31 354 0 08 Jun 2020
Pre-training Polish Transformer-based Language Models at Scale Slawomir Dadas Michal Perelkiewicz Rafal Poswiata 27 38 0 07 Jun 2020
BERT Loses Patience: Fast and Robust Inference with Early Exit Wangchunshu Zhou Canwen Xu Tao Ge Julian McAuley Ke Xu Furu Wei 11 334 0 07 Jun 2020
An Overview of Neural Network Compression James OÑeill AI4CE 45 98 0 05 Jun 2020
DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representations John Giorgi Osvald Nitski Bo Wang Gary D. Bader SSL 39 490 0 05 Jun 2020
DeBERTa: Decoding-enhanced BERT with Disentangled Attention Pengcheng He Xiaodong Liu Jianfeng Gao Weizhu Chen AAML 64 2,626 0 05 Jun 2020
GMAT: Global Memory Augmentation for Transformers Ankit Gupta Jonathan Berant RALM 13 49 0 05 Jun 2020
Understanding Self-Attention of Self-Supervised Audio Transformers Shu-Wen Yang Andy T. Liu Hung-yi Lee 22 27 0 05 Jun 2020
Funnel-Transformer: Filtering out Sequential Redundancy for Efficient Language Processing Zihang Dai Guokun Lai Yiming Yang Quoc V. Le 48 230 0 05 Jun 2020
Position Masking for Language Models Andy Wagner T. Mitra Mrinal Iyer Godfrey Da Costa Marc Tremblay 12 5 0 02 Jun 2020
Subjective Question Answering: Deciphering the inner workings of Transformers in the realm of subjectivity Lukas Muttenthaler 20 3 0 02 Jun 2020
WikiBERT models: deep transfer learning for many languages S. Pyysalo Jenna Kanerva Antti Virtanen Filip Ginter KELM 36 38 0 02 Jun 2020
Question Answering on Scholarly Knowledge Graphs M. Y. Jaradeh M. Stocker Sören Auer LMTD RALM 6 12 0 02 Jun 2020
Careful analysis of XRD patterns with Attention Koichi Kano T. Segi H. Ozono 23 0 0 02 Jun 2020
A Pairwise Probe for Understanding BERT Fine-Tuning on Machine Reading Comprehension Jie Cai Zhengzhou Zhu Ping Nie Qian Liu AAML 21 7 0 02 Jun 2020
BERT-based Ensembles for Modeling Disclosure and Support in Conversational Social Media Text Tanvi Dadu Kartikey Pant R. Mamidi 12 9 0 01 Jun 2020
Emergence of Separable Manifolds in Deep Language Representations Jonathan Mamou Hang Le Miguel Angel del Rio Cory Stephenson Hanlin Tang Yoon Kim SueYeon Chung AAML AI4CE 22 38 0 01 Jun 2020
Conversational Machine Comprehension: a Literature Review Somil Gupta Bhanu Pratap Singh Rawat Hong Yu 11 22 0 01 Jun 2020
A Survey on Transfer Learning in Natural Language Processing Zaid Alyafeai Maged S. Alshaibani Irfan Ahmad 30 72 0 31 May 2020
LRG at SemEval-2020 Task 7: Assessing the Ability of BERT and Derivative Models to Perform Short-Edits based Humor Grading Siddhant Mahurkar Rajaswa Patil 6 7 0 31 May 2020
Beyond Leaderboards: A survey of methods for revealing weaknesses in Natural Language Inference data and models Viktor Schlegel Goran Nenadic R. Batista-Navarro ELM 33 18 0 29 May 2020
ValueNet: A Natural Language-to-SQL System that Learns from Database Information Ursin Brunner Kurt Stockinger 6 10 0 29 May 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 97 40,302 0 28 May 2020
Language Representation Models for Fine-Grained Sentiment Classification Brian Cheang Bailey Wei David Kogan H. Qiu Masud Ahmed AI4MH 6 8 0 27 May 2020
Syntactic Structure Distillation Pretraining For Bidirectional Encoders A. Kuncoro Lingpeng Kong Daniel Fried Dani Yogatama Laura Rimell Chris Dyer Phil Blunsom 31 33 0 27 May 2020
GECToR -- Grammatical Error Correction: Tag, Not Rewrite Kostiantyn Omelianchuk Vitaliy Atrasevych Artem Chernodub Oleksandr Skurzhanskyi 36 304 0 26 May 2020
ParsBERT: Transformer-based Model for Persian Language Understanding Mehrdad Farahani Mohammad Gharachorloo Marzieh Farahani Mohammad Manthouri 16 199 0 26 May 2020
An Audio-enriched BERT-based Framework for Spoken Multiple-choice Question Answering Chia-Chih Kuo Shang-Bao Luo Kuan-Yu Chen 20 17 0 25 May 2020
NILE : Natural Language Inference with Faithful Natural Language Explanations Sawan Kumar Partha P. Talukdar XAI LRM 16 160 0 25 May 2020
KaLM at SemEval-2020 Task 4: Knowledge-aware Language Models for Comprehension And Generation Jiajing Wan Xinting Huang LRM 27 5 0 24 May 2020
Transformer-based Context-aware Sarcasm Detection in Conversation Threads from Social Media Xiangjue Dong Changmao Li Jinho Choi 24 25 0 22 May 2020
Open-Retrieval Conversational Question Answering Chen Qu Liu Yang Cen Chen Minghui Qiu W. Bruce Croft Mohit Iyyer RALM 19 172 0 22 May 2020
Comparative Study of Machine Learning Models and BERT on SQuAD Devshree Patel Param Raval Ratnam Parikh Yesha Shastri 8 7 0 22 May 2020
PruneNet: Channel Pruning via Global Importance A. Khetan Zohar Karnin 18 11 0 22 May 2020
Med-BERT: pre-trained contextualized embeddings on large-scale structured electronic health records for disease prediction L. Rasmy Yang Xiang Z. Xie Cui Tao Degui Zhi AI4MH LM&MA 24 657 0 22 May 2020
Pretraining with Contrastive Sentence Objectives Improves Discourse Performance of Language Models Dan Iter Kelvin Guu L. Lansing Dan Jurafsky 6 77 0 20 May 2020
BiQGEMM: Matrix Multiplication with Lookup Table For Binary-Coding-based Quantized DNNs Yongkweon Jeon Baeseong Park S. Kwon Byeongwook Kim Jeongin Yun Dongsoo Lee MQ 33 30 0 20 May 2020