ALBERT: A Lite BERT for Self-supervised Learning of Language Representations

26 September 2019

Papers citing "ALBERT: A Lite BERT for Self-supervised Learning of Language Representations"

50 / 2,913 papers shown

Title
Transient Chaos in BERT Katsuma Inoue Soh Ohara Yasuo Kuniyoshi Kohei Nakajima 29 3 0 06 Jun 2021
MergeDistill: Merging Pre-trained Language Models using Distillation Simran Khanuja Melvin Johnson Partha P. Talukdar 35 16 0 05 Jun 2021
Meta-Learning with Fewer Tasks through Task Interpolation Huaxiu Yao Linjun Zhang Chelsea Finn 49 54 0 04 Jun 2021
BERT-Based Sentiment Analysis: A Software Engineering Perspective Himanshu Batra Narinder Singh Punn S. K. Sonbhadra Sonali Agarwal 10 34 0 04 Jun 2021
You Only Compress Once: Towards Effective and Elastic BERT Compression via Exploit-Explore Stochastic Nature Gradient Shaokun Zhang Xiawu Zheng Chenyi Yang Yuchao Li Yan Wang Rongrong Ji Mengdi Wang Shen Li Jun Yang Rongrong Ji MQ 26 22 0 04 Jun 2021
ERNIE-Tiny : A Progressive Distillation Framework for Pretrained Transformer Compression Weiyue Su Xuyi Chen Shi Feng Jiaxiang Liu Weixin Liu Yu Sun Hao Tian Hua Wu Haifeng Wang 34 13 0 04 Jun 2021
Enabling Lightweight Fine-tuning for Pre-trained Language Model Compression based on Matrix Product Operators Peiyu Liu Ze-Feng Gao Wayne Xin Zhao Z. Xie Zhong-Yi Lu Ji-Rong Wen 23 29 0 04 Jun 2021
Self-supervised Dialogue Learning for Spoken Conversational Question Answering Nuo Chen Chenyu You Yuexian Zou SSL 28 33 0 04 Jun 2021
The Case for Translation-Invariant Self-Attention in Transformer-Based Language Models Ulme Wennberg G. Henter MILM 40 21 0 03 Jun 2021
E2E-VLP: End-to-End Vision-Language Pre-training Enhanced by Visual Learning Haiyang Xu Ming Yan Chenliang Li Bin Bi Songfang Huang Wenming Xiao Fei Huang VLM 31 118 0 03 Jun 2021
Transformers are Deep Infinite-Dimensional Non-Mercer Binary Kernel Machines Matthew A. Wright Joseph E. Gonzalez 42 20 0 02 Jun 2021
Towards Deeper Deep Reinforcement Learning with Spectral Normalization Johan Bjorck Carla P. Gomes Kilian Q. Weinberger 19 23 0 02 Jun 2021
A Multi-Level Attention Model for Evidence-Based Fact Checking Canasai Kruengkrai Junichi Yamagishi Xin Wang GNN 21 25 0 02 Jun 2021
Conversational Question Answering: A Survey Munazza Zaib Wei Emma Zhang Quan Z. Sheng A. Mahmood Yang Zhang 48 88 0 02 Jun 2021
Claim Matching Beyond English to Scale Global Fact-Checking Ashkan Kazemi Kiran Garimella Devin Gaffney Scott A. Hale 30 58 0 01 Jun 2021
Comparing Test Sets with Item Response Theory Clara Vania Phu Mon Htut William Huang Dhara Mungra Richard Yuanzhe Pang Jason Phang Haokun Liu Kyunghyun Cho Sam Bowman 27 40 0 01 Jun 2021
What Ingredients Make for an Effective Crowdsourcing Protocol for Difficult NLU Data Collection Tasks? Nikita Nangia Saku Sugawara H. Trivedi Alex Warstadt Clara Vania Sam Bowman 30 35 0 01 Jun 2021
Dialogue-oriented Pre-training Yi Xu Hai Zhao 28 14 0 01 Jun 2021
Sub-Character Tokenization for Chinese Pretrained Language Models Chenglei Si Zhengyan Zhang Yingfa Chen Fanchao Qi Xiaozhi Wang Zhiyuan Liu Yasheng Wang Qun Liu Maosong Sun 27 9 0 01 Jun 2021
Improving the Adversarial Robustness for Speaker Verification by Self-Supervised Learning Haibin Wu Xu Li Andy T. Liu Zhiyong Wu Helen Meng Hung-yi Lee AAML SSL 55 29 0 01 Jun 2021
Adversarial VQA: A New Benchmark for Evaluating the Robustness of VQA Models Linjie Li Jie Lei Zhe Gan Jingjing Liu AAML VLM 28 70 0 01 Jun 2021
Choose a Transformer: Fourier or Galerkin Shuhao Cao 47 228 0 31 May 2021
SemEval-2021 Task 4: Reading Comprehension of Abstract Meaning Boyuan Zheng Xiaoyu Yang Yu-Ping Ruan Zhen-Hua Ling Quan Liu Si Wei Xiao-Dan Zhu ELM 25 13 0 31 May 2021
LEAP: Learnable Pruning for Transformer-based Models Z. Yao Xiaoxia Wu Linjian Ma Sheng Shen Kurt Keutzer Michael W. Mahoney Yuxiong He 30 7 0 30 May 2021
Neural Models for Offensive Language Detection Ehab Hamdy 22 4 0 30 May 2021
Pre-training Universal Language Representation Yian Li Hai Zhao SSL 35 8 0 30 May 2021
NAS-BERT: Task-Agnostic and Adaptive-Size BERT Compression with Neural Architecture Search Jin Xu Xu Tan Renqian Luo Kaitao Song Jian Li Tao Qin Tie-Yan Liu MQ 23 78 0 30 May 2021
Sentiment analysis in tweets: an assessment study from classical to modern text representation models Sérgio Barreto Ricardo Moura Jonnathan Carvalho A. Paes A. Plastino 23 14 0 29 May 2021
NeuralLog: Natural Language Inference with Joint Neural and Logical Reasoning Zeming Chen Qiyue Gao Lawrence S. Moss FedML NAI 13 41 0 29 May 2021
Cisco at SemEval-2021 Task 5: What's Toxic?: Leveraging Transformers for Multiple Toxic Span Extraction from Online Comments Sreyan Ghosh Sonal Kumar 38 8 0 28 May 2021
Knowledge Inheritance for Pre-trained Language Models Yujia Qin Yankai Lin Jing Yi Jiajie Zhang Xu Han ... Yusheng Su Zhiyuan Liu Peng Li Maosong Sun Jie Zhou VLM 34 49 0 28 May 2021
Accelerating BERT Inference for Sequence Labeling via Early-Exit Xiaonan Li Yunfan Shao Tianxiang Sun Hang Yan Xipeng Qiu Xuanjing Huang 24 40 0 28 May 2021
Lightweight Cross-Lingual Sentence Representation Learning Zhuoyuan Mao Prakhar Gupta Pei Wang Chenhui Chu Martin Jaggi Sadao Kurohashi VLM 30 8 0 28 May 2021
Early Exiting with Ensemble Internal Classifiers Tianxiang Sun Yunhua Zhou Xiangyang Liu Xinyu Zhang Hao Jiang Bo Zhao Xuanjing Huang Xipeng Qiu 32 30 0 28 May 2021
An Explanatory Query-Based Framework for Exploring Academic Expertise O. Cocarascu A. McLean Paul French Francesca Toni 16 0 0 28 May 2021
Hierarchical Transformer Encoders for Vietnamese Spelling Correction H. Tran C. Dinh Long Phan S. T. Nguyen 31 12 0 28 May 2021
Inspecting the concept knowledge graph encoded by modern language models Carlos Aspillaga Marcelo Mendoza Alvaro Soto 27 13 0 27 May 2021
Verb Sense Clustering using Contextualized Word Representations for Semantic Frame Induction Kosuke Yamada Ryohei Sasano Koichi Takeda 30 7 0 27 May 2021
A Full-Stack Search Technique for Domain Optimized Deep Learning Accelerators Dan Zhang Safeen Huda Ebrahim M. Songhori Kartik Prabhu Quoc V. Le Anna Goldie Azalia Mirhoseini 34 51 0 26 May 2021
TreeBERT: A Tree-Based Pre-Trained Model for Programming Language Xue Jiang Zhuoran Zheng Chen Lyu Liang Li Lei Lyu 27 91 0 26 May 2021
LMMS Reloaded: Transformer-based Sense Embeddings for Disambiguation and Beyond Daniel Loureiro A. Jorge Jose Camacho-Collados 37 26 0 26 May 2021
NEUer at SemEval-2021 Task 4: Complete Summary Representation by Filling Answers into Question for Matching Reading Comprehension Zhixiang Chen Yikun Lei Pai Liu G. Guo 23 0 0 25 May 2021
Dynamic Semantic Graph Construction and Reasoning for Explainable Multi-hop Science Question Answering Weiwen Xu Huihui Zhang Deng Cai Wai Lam 39 34 0 25 May 2021
True Few-Shot Learning with Language Models Ethan Perez Douwe Kiela Kyunghyun Cho 23 428 0 24 May 2021
Using Adversarial Attacks to Reveal the Statistical Bias in Machine Reading Comprehension Models Jieyu Lin Jiajie Zou Nai Ding AAML 18 42 0 24 May 2021
One4all User Representation for Recommender Systems in E-commerce Kyuyong Shin Hanock Kwak KyungHyun Kim Minkyu Kim Young-Jin Park Jisu Jeong Seungjae Jung 36 28 0 24 May 2021
Structural Pre-training for Dialogue Comprehension Zhuosheng Zhang Hai Zhao 29 31 0 23 May 2021
Pretrained Language Models for Text Generation: A Survey Junyi Li Tianyi Tang Wayne Xin Zhao Ji-Rong Wen LM&MA VLM SyDa 30 185 0 21 May 2021
Dynaboard: An Evaluation-As-A-Service Platform for Holistic Next-Generation Benchmarking Zhiyi Ma Kawin Ethayarajh Tristan Thrush Somya Jain Ledell Yu Wu Robin Jia Christopher Potts Adina Williams Douwe Kiela ELM 35 57 0 21 May 2021
Boosting Span-based Joint Entity and Relation Extraction via Squence Tagging Mechanism Bin Ji Shasha Li Jie Yu Jun Ma Huijun Liu 27 4 0 21 May 2021