ALBERT: A Lite BERT for Self-supervised Learning of Language Representations

26 September 2019

Papers citing "ALBERT: A Lite BERT for Self-supervised Learning of Language Representations"

50 / 2,913 papers shown

Title
Detecting Harmful Content On Online Platforms: What Platforms Need Vs. Where Research Efforts Go Arnav Arora Preslav Nakov Momchil Hardalov Sheikh Muhammad Sarwar Vibha Nayak ... Dimitrina Zlatkova Kyle Dent Ameya Bhatawdekar Guillaume Bouchard Isabelle Augenstein 38 46 0 27 Feb 2021
Automated essay scoring using efficient transformer-based language models C. Ormerod Akanksha Malhotra Amir Jafari 29 30 0 25 Feb 2021
ZJUKLAB at SemEval-2021 Task 4: Negative Augmentation with Language Model for Reading Comprehension of Abstract Meaning Xin Xie Xiangnan Chen Xiang Chen Yong Wang Ningyu Zhang Shumin Deng Huajun Chen 44 2 0 25 Feb 2021
LazyFormer: Self Attention with Lazy Update Chengxuan Ying Guolin Ke Di He Tie-Yan Liu 25 15 0 25 Feb 2021
When Attention Meets Fast Recurrence: Training Language Models with Reduced Compute Tao Lei RALM VLM 59 47 0 24 Feb 2021
LRG at SemEval-2021 Task 4: Improving Reading Comprehension with Abstract Words using Augmentation, Linguistic Features and Voting Abheesht Sharma Harshit Pandey Gunjan Chhablani Yash Bhartia T. Dash 23 1 0 24 Feb 2021
Hopeful_Men@LT-EDI-EACL2021: Hope Speech Detection Using Indic Transliteration and Transformers I. S. Upadhyay E. Nikhil Anshul Wadhawan R. Mamidi 11 14 0 24 Feb 2021
Do Transformer Modifications Transfer Across Implementations and Applications? Sharan Narang Hyung Won Chung Yi Tay W. Fedus Thibault Févry ... Wei Li Nan Ding Jake Marcus Adam Roberts Colin Raffel 33 126 0 23 Feb 2021
LogME: Practical Assessment of Pre-trained Models for Transfer Learning Kaichao You Yong Liu Jianmin Wang Mingsheng Long 35 178 0 22 Feb 2021
Using Prior Knowledge to Guide BERT's Attention in Semantic Textual Matching Tasks Tingyu Xia Yue Wang Yuan Tian Yi-Ju Chang 30 51 0 22 Feb 2021
UniT: Multimodal Multitask Learning with a Unified Transformer Ronghang Hu Amanpreet Singh ViT 30 296 0 22 Feb 2021
VisualGPT: Data-efficient Adaptation of Pretrained Language Models for Image Captioning Jun Chen Han Guo Kai Yi Boyang Albert Li Mohamed Elhoseiny VLM 31 219 0 20 Feb 2021
Multilingual Answer Sentence Reranking via Automatically Translated Data Thuy Vu Alessandro Moschitti 32 5 0 20 Feb 2021
Analyzing Curriculum Learning for Sentiment Analysis along Task Difficulty, Pacing and Visualization Axes Anvesh Rao Vijjini Kaveri Anuranjana R. Mamidi 38 3 0 19 Feb 2021
Towards Emotion Recognition in Hindi-English Code-Mixed Data: A Transformer Based Approach Anshul Wadhawan Akshita Aggarwal 22 29 0 19 Feb 2021
MUDES: Multilingual Detection of Offensive Spans Tharindu Ranasinghe Marcos Zampieri 29 41 0 18 Feb 2021
Highly Fast Text Segmentation With Pairwise Markov Chains E. Azeraf E. Monfrini Emmanuel Vignon W. Pieczynski 8 5 0 17 Feb 2021
Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts Soravit Changpinyo P. Sharma Nan Ding Radu Soricut VLM 323 1,086 0 17 Feb 2021
COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretraining Yu Meng Chenyan Xiong Payal Bajaj Saurabh Tiwary Paul N. Bennett Jiawei Han Xia Song 127 203 0 16 Feb 2021
Conversations Gone Alright: Quantifying and Predicting Prosocial Outcomes in Online Conversations Jiajun Bao J. Wu Yiming Zhang Eshwar Chandrasekharan David Jurgens 48 45 0 16 Feb 2021
Exploring Transformers in Natural Language Generation: GPT, BERT, and XLNet M. O. Topal Anil Bas Imke van Heerden LLMAG AI4CE 26 88 0 16 Feb 2021
Improving speech recognition models with small samples for air traffic control systems Yi Lin Qin Li Bo Yang Zhen Yan Huachun Tan Zhengmao Chen 42 32 0 16 Feb 2021
Have Attention Heads in BERT Learned Constituency Grammar? Ziyang Luo 29 6 0 16 Feb 2021
Improved Customer Transaction Classification using Semi-Supervised Knowledge Distillation Rohan Sukumaran 22 2 0 15 Feb 2021
PAQ: 65 Million Probably-Asked Questions and What You Can Do With Them Patrick Lewis Yuxiang Wu Linqing Liu Pasquale Minervini Heinrich Küttler Aleksandra Piktus Pontus Stenetorp Sebastian Riedel RALM 45 229 0 13 Feb 2021
Capturing Label Distribution: A Case Study in NLI Shujian Zhang Chengyue Gong Eunsol Choi 46 8 0 13 Feb 2021
Optimizing Inference Performance of Transformers on CPUs D. Dice Alex Kogan 19 15 0 12 Feb 2021
Less is More: ClipBERT for Video-and-Language Learning via Sparse Sampling Jie Lei Linjie Li Luowei Zhou Zhe Gan Tamara L. Berg Joey Tianyi Zhou Jingjing Liu CLIP 46 648 0 11 Feb 2021
Text Compression-aided Transformer Encoding Z. Li ZhuoSheng Zhang Hai Zhao Rui Wang Kehai Chen Masao Utiyama Eiichiro Sumita AI4CE 30 45 0 11 Feb 2021
Customizing Contextualized Language Models forLegal Document Reviews Shohreh Shaghaghian Luna Feng Feng Borna Jafarpour Nicolai Pogrebnyakov AILaw 28 19 0 10 Feb 2021
Self-supervised learning for fast and scalable time series hyper-parameter tuning Peiyi Zhang Xiaodong Jiang Ginger m Holt N. Laptev C. Komurlu Peng Gao Yang Yu AI4TS 32 5 0 10 Feb 2021
Multi-turn Dialogue Reading Comprehension with Pivot Turns and Knowledge ZhuoSheng Zhang Junlong Li Hai Zhao 42 23 0 10 Feb 2021
User Engagement Prediction for Clarification in Search Ivan Sekulić Mohammad Aliannejadi Fabio Crestani 25 25 0 08 Feb 2021
Nyströmformer: A Nyström-Based Algorithm for Approximating Self-Attention Yunyang Xiong Zhanpeng Zeng Rudrasis Chakraborty Mingxing Tan G. Fung Yin Li Vikas Singh 47 508 0 07 Feb 2021
Memory Augmented Sequential Paragraph Retrieval for Multi-hop Question Answering Nan Shao Yiming Cui Ting Liu Shijin Wang Guoping Hu KELM 18 5 0 07 Feb 2021
Unifying Vision-and-Language Tasks via Text Generation Jaemin Cho Jie Lei Hao Tan Joey Tianyi Zhou MLLM 277 525 0 04 Feb 2021
Learning to Select External Knowledge with Multi-Scale Negative Sampling H. He Hua Lu Siqi Bao Fan Wang Hua Wu Zhengyu Niu Haifeng Wang 24 32 0 03 Feb 2021
AutoFreeze: Automatically Freezing Model Blocks to Accelerate Fine-tuning Yuhan Liu Saurabh Agarwal Shivaram Venkataraman OffRL 22 54 0 02 Feb 2021
Do Question Answering Modeling Improvements Hold Across Benchmarks? Nelson F. Liu Tony Lee Robin Jia Percy Liang 28 13 0 01 Feb 2021
Measuring and Improving Consistency in Pretrained Language Models Yanai Elazar Nora Kassner Shauli Ravfogel Abhilasha Ravichander Eduard H. Hovy Hinrich Schütze Yoav Goldberg HILM 272 347 0 01 Feb 2021
Scaling Federated Learning for Fine-tuning of Large Language Models Agrin Hilmkil Sebastian Callh Matteo Barbieri L. R. Sütfeld Edvin Listo Zec Olof Mogren FedML 22 47 0 01 Feb 2021
Many Hands Make Light Work: Using Essay Traits to Automatically Score Essays Rahul Kumar Sandeep Albert Mathias S. Saha P. Bhattacharyya 35 26 0 01 Feb 2021
Speech Recognition by Simply Fine-tuning BERT Wen-Chin Huang Chia-Hua Wu Shang-Bao Luo Kuan-Yu Chen Hsin-Min Wang Tomoki Toda 79 28 0 30 Jan 2021
A transformer based approach for fighting COVID-19 fake news S. M. S. Shifath Mohammad Faiyaz Khan Md. Saiful Islam MedIm 36 23 0 28 Jan 2021
BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generation Jwala Dhamala Tony Sun Varun Kumar Satyapriya Krishna Yada Pruksachatkun Kai-Wei Chang Rahul Gupta 25 376 0 27 Jan 2021
KoreALBERT: Pretraining a Lite BERT Model for Korean Language Understanding HyunJae Lee Jaewoong Yoon Bonggyu Hwang Seongho Joe Seungjai Min Youngjune Gwon SSeg 31 16 0 27 Jan 2021
Neural Sentence Ordering Based on Constraint Graphs Yutao Zhu Kun Zhou J. Nie Shengchao Liu Zhicheng Dou NAI 23 23 0 27 Jan 2021
A Case Study of Deep Learning Based Multi-Modal Methods for Predicting the Age-Suitability Rating of Movie Trailers Mahsa Shafaei C. Smailis I. Kakadiaris Thamar Solorio 221 1 0 26 Jan 2021
Evaluation of BERT and ALBERT Sentence Embedding Performance on Downstream NLP Tasks Hyunjin Choi Judong Kim Seongho Joe Youngjune Gwon SSeg 19 101 0 26 Jan 2021
Stereotype and Skew: Quantifying Gender Bias in Pre-trained and Fine-tuned Language Models Daniel de Vassimon Manela D. Errington Thomas Fisher B. V. Breugel Pasquale Minervini 14 88 0 24 Jan 2021