v1v2v3v4v5v6 (latest)

ALBERT: A Lite BERT for Self-supervised Learning of Language Representations

26 September 2019

ArXiv (abs)PDF HTML Github (3271★)

Papers citing "ALBERT: A Lite BERT for Self-supervised Learning of Language Representations"

50 / 2,935 papers shown

Title
How Far Can It Go?: On Intrinsic Gender Bias Mitigation for Text Classification E. Tokpo Pieter Delobelle Bettina Berendt T. Calders 88 8 0 30 Jan 2023
Towards Vision Transformer Unrolling Fixed-Point Algorithm: a Case Study on Image Restoration Peng Qiao Sidun Liu Tao Sun Ke Yang Y. Dou ViT 81 1 0 29 Jan 2023
Comparing Intrinsic Gender Bias Evaluation Measures without using Human Annotated Examples Masahiro Kaneko Danushka Bollegala Naoaki Okazaki 60 10 0 28 Jan 2023
Understanding INT4 Quantization for Transformer Models: Latency Speedup, Composability, and Failure Cases Xiaoxia Wu Cheng-rong Li Reza Yazdani Aminabadi Z. Yao Yuxiong He MQ 74 25 0 27 Jan 2023
Improved knowledge distillation by utilizing backward pass knowledge in neural networks A. Jafari Mehdi Rezagholizadeh A. Ghodsi 37 1 0 27 Jan 2023
SWARM Parallelism: Training Large Models Can Be Surprisingly Communication-Efficient Max Ryabinin Tim Dettmers Michael Diskin Alexander Borzunov MoE 111 38 0 27 Jan 2023
Open Problems in Applied Deep Learning M. Raissi AI4CE 115 2 0 26 Jan 2023
A benchmark for toxic comment classification on Civil Comments dataset Corentin Duchene Henri Jamet Pierre Guillaume Reda Dehak 74 9 0 26 Jan 2023
Story Shaping: Teaching Agents Human-like Behavior with Stories Xiangyu Peng Christopher Cui Wei Zhou Renee Jia Mark O. Riedl 82 6 0 24 Jan 2023
A Stability Analysis of Fine-Tuning a Pre-Trained Model Z. Fu Anthony Man-Cho So Nigel Collier 62 3 0 24 Jan 2023
AttMEMO : Accelerating Transformers with Memoization on Big Memory Systems Yuan Feng Hyeran Jeon F. Blagojevic Cyril Guyot Qing Li Dong Li GNN 65 3 0 23 Jan 2023
An Empirical Study of Metrics to Measure Representational Harms in Pre-Trained Language Models Saghar Hosseini Hamid Palangi Ahmed Hassan Awadallah 58 24 0 22 Jan 2023
SPEC5G: A Dataset for 5G Cellular Network Protocol Analysis Imtiaz Karim Kazi Samin Mubasshir Mirza Masfiqur Rahman Elisa Bertino 48 25 0 22 Jan 2023
Unifying Structure Reasoning and Language Model Pre-training for Complex Reasoning Siyuan Wang Zhongyu Wei Jiarong Xu Taishan Li Zhihao Fan LRM 80 5 0 21 Jan 2023
Phoneme-Level BERT for Enhanced Prosody of Text-to-Speech with Grapheme Predictions Yinghao Aaron Li Cong Han Xilin Jiang N. Mesgarani 64 24 0 20 Jan 2023
Towards a Holistic Understanding of Mathematical Questions with Contrastive Pre-training Yuting Ning Zhenya Huang Xin Lin Enhong Chen Shiwei Tong Zheng Gong Shijin Wang AIMat 73 7 0 18 Jan 2023
Ankh: Optimized Protein Language Model Unlocks General-Purpose Modelling Ahmed Elnaggar Hazem Essam Wafaa Salah-Eldin Walid Moustafa Mohamed Elkerdawy Charlotte Rochereau B. Rost 238 102 0 16 Jan 2023
Toward Building General Foundation Models for Language, Vision, and Vision-Language Understanding Tasks Xinsong Zhang Yan Zeng Jipeng Zhang Hang Li VLM AI4CE LRM 117 17 0 12 Jan 2023
Everyone's Voice Matters: Quantifying Annotation Disagreement Using Demographic Information Ruyuan Wan Jaehyung Kim Dongyeop Kang 57 38 0 12 Jan 2023
EXIF as Language: Learning Cross-Modal Associations Between Images and Camera Metadata Chenhao Zheng Ayush Shrivastava Andrew Owens VLM 139 12 0 11 Jan 2023
Counteracts: Testing Stereotypical Representation in Pre-trained Language Models Damin Zhang Julia Taylor Rayz Romila Pradhan 79 2 0 11 Jan 2023
Channel-aware Decoupling Network for Multi-turn Dialogue Comprehension Zhuosheng Zhang Hai Zhao Longxiang Liu BDL 93 2 0 10 Jan 2023
Cross-Model Comparative Loss for Enhancing Neuronal Utility in Language Understanding Yunchang Zhu Liang Pang Kangxi Wu Yanyan Lan Huawei Shen Xueqi Cheng AAML ELM 57 2 0 10 Jan 2023
Learning Trajectory-Word Alignments for Video-Language Tasks Xu Yang Zhang Li Haiyang Xu Hanwang Zhang Qinghao Ye Chenliang Li Ming Yan Yu Zhang Fei Huang Songfang Huang 80 7 0 05 Jan 2023
MessageNet: Message Classification using Natural Language Processing and Meta-data Adar Kahana Oren Elisha 26 0 0 04 Jan 2023
Integrating Semantic Information into Sketchy Reading Module of Retro-Reader for Vietnamese Machine Reading Comprehension Hang Le Viet-Duc Ho Duc-Vu Nguyen Ngan Luu-Thuy Nguyen 72 2 0 01 Jan 2023
Floods Relevancy and Identification of Location from Twitter Posts using NLP Techniques M. Suleman Muhammad Asif Ayub Tayyab Zamir Ayaz Mehmood Jebran Khan Nasir Ahmad Kashif Ahmad 35 3 0 01 Jan 2023
Transformers in Action Recognition: A Review on Temporal Modeling Elham Shabaninia Hossein Nezamabadi-pour Fatemeh Shafizadegan ViT 67 9 0 29 Dec 2022
On Transforming Reinforcement Learning by Transformer: The Development Trajectory Shengchao Hu Li Shen Ya Zhang Yixin Chen Dacheng Tao OffRL 148 30 0 29 Dec 2022
Cramming: Training a Language Model on a Single GPU in One Day Jonas Geiping Tom Goldstein MoE 117 91 0 28 Dec 2022
OVO: One-shot Vision Transformer Search with Online distillation Zimian Wei H. Pan Xin-Yi Niu Dongsheng Li ViT 67 1 0 28 Dec 2022
A Survey on Knowledge-Enhanced Pre-trained Language Models Chaoqi Zhen Yanlei Shang Xiangyu Liu Yifei Li Yong Chen Dell Zhang VLM KELM 92 3 0 27 Dec 2022
Optimizing Deep Transformers for Chinese-Thai Low-Resource Translation Wenjie Hao Hongfei Xu Lingling Mu Hongying Zan MoE 97 4 0 24 Dec 2022
STRUDEL: Structured Dialogue Summarization for Dialogue Comprehension Borui Wang Chengcheng Feng Arjun Nair Madelyn Mao Jai Desai Asli Celikyilmaz Haoran Li Yashar Mehdad Dragomir R. Radev 83 1 0 24 Dec 2022
Content Rating Classification for Fan Fiction Yu Qiao James Pope 16 1 0 23 Dec 2022
Automatic Emotion Modelling in Written Stories Lukas Christ Shahin Amiriparian M. Milling Ilhan Aslan Björn W. Schuller 58 2 0 21 Dec 2022
Mining User-aware Multi-relations for Fake News Detection in Large Scale Online Social Networks Xing Su Jian Yang Hongzhi Zhang Yuchen Zhang GNN 65 19 0 21 Dec 2022
ORCA: A Challenging Benchmark for Arabic Language Understanding AbdelRahim Elmadany El Moatez Billah Nagoudi Muhammad Abdul-Mageed ELM 109 45 0 21 Dec 2022
MoralDial: A Framework to Train and Evaluate Moral Dialogue Systems via Moral Discussions Hao Sun Zhexin Zhang Fei Mi Yasheng Wang Wen Liu Jianwei Cui Bin Wang Qun Liu Minlie Huang 92 20 0 21 Dec 2022
A Survey of Deep Learning for Mathematical Reasoning Pan Lu Liang Qiu Wenhao Yu Sean Welleck Kai-Wei Chang ReLM LRM 133 150 0 20 Dec 2022
Localising In-Domain Adaptation of Transformer-Based Biomedical Language Models T. M. Buonocore Claudio Crema A. Redolfi Riccardo Bellazzi Enea Parimbelli LM&MA 36 22 0 20 Dec 2022
Quirk or Palmer: A Comparative Study of Modal Verb Frameworks with Annotated Datasets Risako Owan Maria Gini Dongyeop Kang 34 1 0 20 Dec 2022
Do CoNLL-2003 Named Entity Taggers Still Work Well in 2023? Shuheng Liu Alan Ritter AI4TS 87 13 0 19 Dec 2022
Source-Free Domain Adaptation for Question Answering with Masked Self-training M. Yin B. Wang Yue Dong Charles Ling OOD 98 4 0 19 Dec 2022
Enriching Relation Extraction with OpenIE Alessandro Temperoni M. Biryukov Martin Theobald 51 1 0 19 Dec 2022
TextGrad: Advancing Robustness Evaluation in NLP by Gradient-Driven Optimization Bairu Hou Jinghan Jia Yihua Zhang Guanhua Zhang Yang Zhang Sijia Liu Shiyu Chang SILM AAML 63 24 0 19 Dec 2022
BEATs: Audio Pre-Training with Acoustic Tokenizers Sanyuan Chen Yu-Huan Wu Chengyi Wang Shujie Liu Daniel C. Tompkins Zhuo Chen Furu Wei 124 299 0 18 Dec 2022
Modeling Instance Interactions for Joint Information Extraction with Neural High-Order Conditional Random Field Zixia Jia Zhaohui Yan Wenjuan Han Zilong Zheng Kewei Tu BDL 64 1 0 17 Dec 2022
Rarely a problem? Language models exhibit inverse scaling in their predictions following few-type quantifiers J. Michaelov Benjamin Bergen 44 17 0 16 Dec 2022
Context-aware Fine-tuning of Self-supervised Speech Models Suwon Shon Felix Wu Kwangyoun Kim Prashant Sridhar Karen Livescu Shinji Watanabe 74 9 0 16 Dec 2022