v1v2v3v4v5v6 (latest)

ALBERT: A Lite BERT for Self-supervised Learning of Language Representations

26 September 2019

ArXiv (abs)PDF HTML Github (3271★)

Papers citing "ALBERT: A Lite BERT for Self-supervised Learning of Language Representations"

50 / 2,935 papers shown

Title
ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generation Yu Sun Shuohuan Wang Shikun Feng Siyu Ding Chao Pang ... Ouyang Xuan Dianhai Yu Hao Tian Hua Wu Haifeng Wang 114 475 0 05 Jul 2021
Doing Good or Doing Right? Exploring the Weakness of Commonsense Causal Reasoning Models Mingyue Han Yinglin Wang LRM 72 11 0 05 Jul 2021
Neural-Symbolic Solver for Math Word Problems with Auxiliary Tasks Jinghui Qin Xiaodan Liang Yining Hong Jianheng Tang Liang Lin AIMat AAML 105 57 0 03 Jul 2021
R2D2: Recursive Transformer based on Differentiable Tree for Interpretable Hierarchical Language Modeling Xiang Hu Haitao Mi Zujie Wen Yafang Wang Yi Su Jing Zheng Gerard de Melo 67 23 0 02 Jul 2021
Learned Token Pruning for Transformers Sehoon Kim Sheng Shen D. Thorsley A. Gholami Woosuk Kwon Joseph Hassoun Kurt Keutzer 86 157 0 02 Jul 2021
AutoFormer: Searching Transformers for Visual Recognition Minghao Chen Houwen Peng Jianlong Fu Haibin Ling ViT 104 268 0 01 Jul 2021
Pretext Tasks selection for multitask self-supervised speech representation learning Salah Zaiem Titouan Parcollet S. Essid Abdel Heba SSL 89 13 0 01 Jul 2021
Elbert: Fast Albert with Confidence-Window Based Early Exit Keli Xie Siyuan Lu Meiqi Wang Zhongfeng Wang 54 20 0 01 Jul 2021
ChineseBERT: Chinese Pretraining Enhanced by Glyph and Pinyin Information Zijun Sun Xiaoya Li Xiaofei Sun Yuxian Meng Xiang Ao Qing He Leilei Gan Jiwei Li SSeg 144 191 0 30 Jun 2021
HySPA: Hybrid Span Generation for Scalable Text-to-Graph Extraction Liliang Ren Chenkai Sun Heng Ji Julia Hockenmaier 80 14 0 30 Jun 2021
New Arabic Medical Dataset for Diseases Classification Jaafar Hammoud A. Vatian N. Dobrenko N. Vedernikov A. Shalyto N. Gusarova OOD 66 6 0 29 Jun 2021
SCARF: Self-Supervised Contrastive Learning using Random Feature Corruption Dara Bahri Heinrich Jiang Yi Tay Donald Metzler SSL 74 178 0 29 Jun 2021
TWAG: A Topic-Guided Wikipedia Abstract Generator Fangwei Zhu Shangqing Tu Jiaxin Shi Juan-Zi Li Lei Hou Tong Cui 36 11 0 29 Jun 2021
A Knowledge-Grounded Dialog System Based on Pre-Trained Language Models Weijie Zhang Jiaoxuan Chen Haipang Wu Sanhui Wan Gongfeng Li 51 4 0 28 Jun 2021
A Closer Look at How Fine-tuning Changes BERT Yichu Zhou Vivek Srikumar 82 68 0 27 Jun 2021
Improving Sequential Recommendation Consistency with Self-Supervised Imitation Xu Yuan Hongshen Chen Yonghao Song Xiaofang Zhao Zhuoye Ding Zhen He Bo Long 57 22 0 26 Jun 2021
Answering Chinese Elementary School Social Study Multiple Choice Questions Daniel Lee Chao-Chun Liang Keh-Yih Su 29 1 0 26 Jun 2021
Benchmarking Differential Privacy and Federated Learning for BERT Models Priya Basu Tiasa Singha Roy Rakshit Naidu Zumrut Muftuoglu Sahib Singh Fatemehsadat Mireshghallah FedML AI4MH 78 51 0 26 Jun 2021
Knowledge-Grounded Self-Rationalization via Extractive and Natural Language Explanations Bodhisattwa Prasad Majumder Oana-Maria Camburu Thomas Lukasiewicz Julian McAuley 98 36 0 25 Jun 2021
Learning to Sample Replacements for ELECTRA Pre-Training Y. Hao Li Dong Hangbo Bao Ke Xu Furu Wei MU 45 12 0 25 Jun 2021
aiSTROM -- A roadmap for developing a successful AI strategy Dorien Herremans 37 9 0 25 Jun 2021
ParaLaw Nets -- Cross-lingual Sentence-level Pretraining for Legal Text Processing Nguyen Ha Thanh Vu Tran Phuong Minh Nguyen Thi-Hai-Yen Vuong Quan Minh Bui Chau Nguyen Binh Dang Minh Le Nguyen Kenji Satoh AILaw 52 10 0 25 Jun 2021
Cutting Down on Prompts and Parameters: Simple Few-Shot Learning with Language Models Robert L Logan IV Ivana Balavzević Eric Wallace Fabio Petroni Sameer Singh Sebastian Riedel VPVLM 106 212 0 24 Jun 2021
Physics perception in sloshing scenes with guaranteed thermodynamic consistency B. Moya Alberto Badías D. González Francisco Chinesta Elías Cueto 76 14 0 24 Jun 2021
PALRACE: Reading Comprehension Dataset with Human Data and Labeled Rationales Jiajie Zou Yuran Zhang Peiqing Jin Cheng Luo Xunyi Pan Nai Ding FaML 113 5 0 23 Jun 2021
LegoFormer: Transformers for Block-by-Block Multi-view 3D Reconstruction Farid Yagubbayli Yida Wang A. Tonioni Federico Tombari ViT 50 35 0 23 Jun 2021
LV-BERT: Exploiting Layer Variety for BERT Weihao Yu Zihang Jiang Fei Chen Qibin Hou Jiashi Feng MQ 51 0 0 22 Jun 2021
Towards Long-Form Video Understanding Chaoxia Wu Philipp Krahenbuhl VLM ViT 119 170 0 21 Jun 2021
Secure Distributed Training at Scale Eduard A. Gorbunov Alexander Borzunov Michael Diskin Max Ryabinin FedML 90 15 0 21 Jun 2021
VIMPAC: Video Pre-Training via Masked Token Prediction and Contrastive Learning Hao Tan Jie Lei Thomas Wolf Joey Tianyi Zhou 118 67 0 21 Jun 2021
Iterative Network Pruning with Uncertainty Regularization for Lifelong Sentiment Classification Binzong Geng Min Yang Fajie Yuan Shupeng Wang Xiang Ao Ruifeng Xu CLL 77 19 0 21 Jun 2021
Distributed Deep Learning in Open Collaborations Michael Diskin Alexey Bukhtiyarov Max Ryabinin Lucile Saulnier Quentin Lhoest ... Denis Mazur Ilia Kobelev Yacine Jernite Thomas Wolf Gennady Pekhimenko FedML 129 59 0 18 Jun 2021
Anomaly Detection in Dynamic Graphs via Transformer Yixin Liu Shirui Pan Yu Guang Wang Fei Xiong Liang Wang Qingfeng Chen V. C. Lee 76 98 0 18 Jun 2021
Application-driven Design Exploration for Dense Ferroelectric Embedded Non-volatile Memories Mohammad Mehdi Shariﬁ †∞ LillianPentecost R. Rajaei Arman Kazemi Qiuwen Lou ... David Brooks Kai Ni Sharon Hu Michael Niemier M. Donato 20 5 0 18 Jun 2021
Learning Knowledge Graph-based World Models of Textual Environments Prithviraj Ammanabrolu Mark O. Riedl 3DV 102 32 0 17 Jun 2021
Classifying vaccine sentiment tweets by modelling domain-specific representation and commonsense knowledge into context-aware attentive GRU Usman Naseem Matloob Khushi Jinman Kim A. Dunn 54 12 0 17 Jun 2021
Modeling Worlds in Text Prithviraj Ammanabrolu Mark O. Riedl VGen LM&Ro 63 14 0 17 Jun 2021
DravidianCodeMix: Sentiment Analysis and Offensive Language Identification Dataset for Dravidian Languages in Code-Mixed Text Bharathi Raja Chakravarthi R. Priyadharshini Vigneshwaran Muralidaran Navya Jose Shardul Suryawanshi E. Sherly John P. Mccrae 62 107 0 17 Jun 2021
Named Entity Recognition with Small Strongly Labeled and Large Weakly Labeled Data Haoming Jiang Danqing Zhang Tianyu Cao Bing Yin T. Zhao NoLa 80 46 0 16 Jun 2021
Direction is what you need: Improving Word Embedding Compression in Large Language Models Klaudia Bałazy Mohammadreza Banaei R. Lebret Jacek Tabor Karl Aberer 55 7 0 15 Jun 2021
CBLUE: A Chinese Biomedical Language Understanding Evaluation Benchmark Ningyu Zhang Mosha Chen Zhen Bi Xiaozhuan Liang Lei Li ... Jun Yan Hongying Zan Kunli Zhang Buzhou Tang Qingcai Chen LM&MA ELM 91 193 0 15 Jun 2021
Unsupervised Abstractive Opinion Summarization by Generating Sentences with Tree-Structured Topic Guidance Masaru Isonuma Junichiro Mori Danushka Bollegala Ichiro Sakata 49 27 0 15 Jun 2021
Incorporating Word Sense Disambiguation in Neural Language Models Jan Philip Wahle Terry Ruas Norman Meuschke Bela Gipp 67 11 0 15 Jun 2021
Bilateral Personalized Dialogue Generation with Contrastive Learning Bin Li Hanjun Deng 88 9 0 15 Jun 2021
Pre-Trained Models: Past, Present and Future Xu Han Zhengyan Zhang Ning Ding Yuxian Gu Xiao Liu ... Jie Tang Ji-Rong Wen Jinhui Yuan Wayne Xin Zhao Jun Zhu AIFin MQ AI4MH 177 863 0 14 Jun 2021
InfoBehavior: Self-supervised Representation Learning for Ultra-long Behavior Sequence via Hierarchical Grouping Runshi Liu Pengda Qin Yuhong Li Weigao Wen Dong Li Kefeng Deng Qiang Wu AI4TS 56 0 0 13 Jun 2021
Can Transformer Language Models Predict Psychometric Properties? Antonio Laverghetta Animesh Nighojkar Jamshidbek Mirzakhalov John Licato LM&MA 71 14 0 12 Jun 2021
Incorporating External POS Tagger for Punctuation Restoration Ning Shi Wei Wang Wei Ping Jinfeng Li Xiangyu Liu Zhouhan Lin KELM 61 10 0 12 Jun 2021
A Discussion on Building Practical NLP Leaderboards: The Case of Machine Translation Sebastin Santy Prasanta Bhattacharya LLMAG 86 3 0 11 Jun 2021
Hybrid Generative-Contrastive Representation Learning Saehoon Kim Sungwoong Kim Juho Lee SSL 68 11 0 11 Jun 2021