ALBERT: A Lite BERT for Self-supervised Learning of Language Representations

26 September 2019

Papers citing "ALBERT: A Lite BERT for Self-supervised Learning of Language Representations"

50 / 2,913 papers shown

Title
Pretraining and Fine-Tuning Strategies for Sentiment Analysis of Latvian Tweets Gaurish Thakkar Marcis Pinnis 70 9 0 23 Oct 2020
ERNIE-Gram: Pre-Training with Explicitly N-Gram Masked Language Modeling for Natural Language Understanding Dongling Xiao Yukun Li Han Zhang Yu Sun Hao Tian Hua Wu Haifeng Wang 27 38 0 23 Oct 2020
Language Models are Open Knowledge Graphs Chenguang Wang Xiao Liu D. Song SSL KELM 26 135 0 22 Oct 2020
AdapterDrop: On the Efficiency of Adapters in Transformers Andreas Rucklé Gregor Geigle Max Glockner Tilman Beck Jonas Pfeiffer Nils Reimers Iryna Gurevych 57 255 0 22 Oct 2020
Contrastive Self-Supervised Learning for Wireless Power Control Navid Naderializadeh SSL 28 6 0 22 Oct 2020
Towards Fully Bilingual Deep Language Modeling Li-Hsin Chang S. Pyysalo Jenna Kanerva Filip Ginter 34 3 0 22 Oct 2020
Calibrated Language Model Fine-Tuning for In- and Out-of-Distribution Data Lingkai Kong Haoming Jiang Yuchen Zhuang Jie Lyu T. Zhao Chao Zhang OODD 30 26 0 22 Oct 2020
Detection of COVID-19 informative tweets using RoBERTa Sirigireddy Dhanalaxmi Rohit Agarwal Aman Sinha 17 7 0 21 Oct 2020
Knowledge Distillation for Improved Accuracy in Spoken Question Answering Chenyu You Nuo Chen Yuexian Zou 25 49 0 21 Oct 2020
Contextualized Attention-based Knowledge Transfer for Spoken Conversational Question Answering Chenyu You Nuo Chen Yuexian Zou 16 36 0 21 Oct 2020
Complaint Identification in Social Media with Transformer Networks Mali Jin Nikolaos Aletras 17 16 0 21 Oct 2020
German's Next Language Model Branden Chan Stefan Schweter Timo Möller 36 265 0 21 Oct 2020
Learning to Embed Categorical Features without Embedding Tables for Recommendation Wang-Cheng Kang D. Cheng Tiansheng Yao Xinyang Yi Ting-Li Chen Lichan Hong Ed H. Chi LMTD CML DML 50 68 0 21 Oct 2020
Self-supervised Graph Learning for Recommendation Jiancan Wu Xiang Wang Fuli Feng Xiangnan He Liang Chen Jianxun Lian Xing Xie SSL 33 1,121 0 21 Oct 2020
An Empirical Investigation of Contextualized Number Prediction Daniel M. Spokoyny Taylor Berg-Kirkpatrick AI4TS 27 34 0 20 Oct 2020
Bayesian Attention Modules Xinjie Fan Shujian Zhang Bo Chen Mingyuan Zhou 117 59 0 20 Oct 2020
Optimal Subarchitecture Extraction For BERT Adrian de Wynter Daniel J. Perry MQ 56 18 0 20 Oct 2020
BERT2DNN: BERT Distillation with Massive Unlabeled Data for Online E-Commerce Search Yunjiang Jiang Yue Shang Ziyang Liu Hongwei Shen Yun Xiao Wei Xiong Sulong Xu Weipeng P. Yan Di Jin 31 17 0 20 Oct 2020
CharacterBERT: Reconciling ELMo and BERT for Word-Level Open-Vocabulary Representations From Characters Hicham El Boukkouri Olivier Ferret Thomas Lavergne Hiroshi Noji Pierre Zweigenbaum Junichi Tsujii 77 156 0 20 Oct 2020
A Benchmark for Lease Contract Review Spyretta Leivaditi Julien Rossi Evangelos Kanoulas AILaw 111 36 0 20 Oct 2020
Bi-directional Cognitive Thinking Network for Machine Reading Comprehension Wei Peng Yue Hu Luxi Xing Yuqiang Xie Jing Yu Yajing Sun Xiangpeng Wei 16 7 0 20 Oct 2020
CoRT: Complementary Rankings from Transformers Marco Wrzalik D. Krechel 25 9 0 20 Oct 2020
ChemBERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Prediction Seyone Chithrananda Gabriel Grand Bharath Ramsundar AI4CE 37 389 0 19 Oct 2020
Technical Question Answering across Tasks and Domains Wenhao Yu Lingfei Wu Yu Deng Qingkai Zeng R. Mahindru S. Guven Meng Jiang 36 8 0 19 Oct 2020
An Empirical Study for Vietnamese Constituency Parsing with Pre-training Tuan-Vi Tran Xuan-Thien Pham Duc-Vu Nguyen Kiet Van Nguyen Ngan Luu-Thuy Nguyen 44 4 0 19 Oct 2020
Better Distractions: Transformer-based Distractor Generation and Multiple Choice Question Filtering J. Offerijns Suzan Verberne Tessa Verhoef 26 26 0 19 Oct 2020
Knowledge-guided Open Attribute Value Extraction with Reinforcement Learning Ye Liu Sheng Zhang Rui Song Suo Feng Yanghua Xiao 32 8 0 19 Oct 2020
Towards Interpreting BERT for Reading Comprehension Based QA Sahana Ramnath Preksha Nema Deep Sahni Mitesh M. Khapra 42 30 0 18 Oct 2020
Towards Data Distillation for End-to-end Spoken Conversational Question Answering Chenyu You Nuo Chen Fenglin Liu Dongchao Yang Yuexian Zou 22 45 0 18 Oct 2020
HABERTOR: An Efficient and Effective Deep Hatespeech Detector T. Tran Yifan Hu Changwei Hu Kevin Yen Fei Tan Kyumin Lee Serim Park VLM 34 32 0 17 Oct 2020
Hierarchical Multitask Learning Approach for BERT Çagla Aksoy Alper Ahmetoglu Tunga Güngör SSL 10 5 0 17 Oct 2020
TweetBERT: A Pretrained Language Representation Model for Twitter Text Analysis Mohiuddin Md Abdul Qudar Vijay K. Mago SSeg 28 35 0 17 Oct 2020
Delaying Interaction Layers in Transformer-based Encoders for Efficient Open Domain Question Answering W. Siblini Mohamed Challal Charlotte Pasqual 21 3 0 16 Oct 2020
Detecting ESG topics using domain-specific language models and data augmentation approaches Timothy Nugent N. Stelea Jochen L. Leidner 39 13 0 16 Oct 2020
A Transformer Based Pitch Sequence Autoencoder with MIDI Augmentation Mingshuo Ding Yi Ma 20 1 0 15 Oct 2020
TopicBERT for Energy Efficient Document Classification Yatin Chaudhary Pankaj Gupta Khushbu Saxena Vivek Kulkarni Thomas Runkler Hinrich Schütze 24 21 0 15 Oct 2020
Text Classification Using Label Names Only: A Language Model Self-Training Approach Yu Meng Yunyi Zhang Jiaxin Huang Chenyan Xiong Heng Ji Chao Zhang Jiawei Han VLM 55 75 0 14 Oct 2020
An Investigation on Different Underlying Quantization Schemes for Pre-trained Language Models Zihan Zhao Yuncong Liu Lu Chen Qi Liu Rao Ma Kai Yu MQ 24 12 0 14 Oct 2020
Weight Squeezing: Reparameterization for Knowledge Transfer and Model Compression Artem Chumachenko Daniil Gavrilov Nikita Balagansky Pavel Kalaidin 16 0 0 14 Oct 2020
Vokenization: Improving Language Understanding with Contextualized, Visual-Grounded Supervision Hao Tan Joey Tianyi Zhou CLIP 22 120 0 14 Oct 2020
With Little Power Comes Great Responsibility Dallas Card Peter Henderson Urvashi Khandelwal Robin Jia Kyle Mahowald Dan Jurafsky 230 115 0 13 Oct 2020
Pretrained Transformers for Text Ranking: BERT and Beyond Jimmy J. Lin Rodrigo Nogueira Andrew Yates VLM 244 612 0 13 Oct 2020
A Wrong Answer or a Wrong Question? An Intricate Relationship between Question Reformulation and Answer Selection in Conversational Question Answering Svitlana Vakulenko Shayne Longpre Zhucheng Tu R. Anantha 24 12 0 13 Oct 2020
Improving Text Generation Evaluation with Batch Centering and Tempered Word Mover Distance Xi Chen Nan Ding Tomer Levinboim Radu Soricut 14 5 0 13 Oct 2020
Oort: Efficient Federated Learning via Guided Participant Selection Fan Lai Xiangfeng Zhu H. Madhyastha Mosharaf Chowdhury FedML OODD 29 226 0 12 Oct 2020
Zero-shot Entity Linking with Efficient Long Range Sequence Modeling Zonghai Yao Liangliang Cao Huapu Pan VLM 36 21 0 12 Oct 2020
Improving Self-supervised Pre-training via a Fully-Explored Masked Language Model Ming Zheng Dinghan Shen Yelong Shen Weizhu Chen Lin Xiao SSL 24 4 0 12 Oct 2020
Measuring and Reducing Gendered Correlations in Pre-trained Models Kellie Webster Xuezhi Wang Ian Tenney Alex Beutel Emily Pitler Ellie Pavlick Jilin Chen Ed Chi Slav Petrov FaML 18 250 0 12 Oct 2020
EFSG: Evolutionary Fooling Sentences Generator Marco Di Giovanni Marco Brambilla AAML 35 2 0 12 Oct 2020
A BERT-based Distractor Generation Scheme with Multi-tasking and Negative Answer Training Strategies Ho-Lam Chung Ying-Hong Chan Yao-Chung Fan 41 41 0 12 Oct 2020