v1v2v3v4v5v6 (latest)

ALBERT: A Lite BERT for Self-supervised Learning of Language Representations

26 September 2019

ArXiv (abs)PDF HTML Github (3271★)

Papers citing "ALBERT: A Lite BERT for Self-supervised Learning of Language Representations"

50 / 2,935 papers shown

Title
Gradient-Free Structured Pruning with Unlabeled Data Azade Nova H. Dai Dale Schuurmans SyDa 84 22 0 07 Mar 2023
Adaptive Knowledge Distillation between Text and Speech Pre-trained Models Jinjie Ni Yukun Ma Wen Wang Qian Chen Dianwen Ng Han Lei Trung Hieu Nguyen Chong Zhang B. Ma Min Zhang 31 2 0 07 Mar 2023
FedML Parrot: A Scalable Federated Learning System via Heterogeneity-aware Scheduling on Sequential and Hierarchical Training Zhenheng Tang Xiaowen Chu Ryan Yide Ran Sunwoo Lee Shaoshuai Shi Yonggang Zhang Yuxin Wang Alex Liang A. Avestimehr Chaoyang He FedML 70 10 0 03 Mar 2023
WiCE: Real-World Entailment for Claims in Wikipedia Ryo Kamoi Tanya Goyal Juan Diego Rodriguez Greg Durrett 103 92 0 02 Mar 2023
Document Provenance and Authentication through Authorship Classification Muhammad Tayyab Zamir Muhammad Asif Ayub Jebran Khan Muhammad Jawad Ikram Nasir Ahmad Kashif Ahmad 16 2 0 02 Mar 2023
BPT: Binary Point Cloud Transformer for Place Recognition Zhixing Hou Yuzhang Shang Tian Gao Yan Yan MQ ViT 71 3 0 02 Mar 2023
How Robust is GPT-3.5 to Predecessors? A Comprehensive Study on Language Understanding Tasks Xuanting Chen Junjie Ye Can Zu Nuo Xu Rui Zheng Minlong Peng Jie Zhou Tao Gui Qi Zhang Xuanjing Huang AI4MH ELM 67 83 0 01 Mar 2023
Deep learning for COVID-19 topic modelling via Twitter: Alpha, Delta and Omicron Janhavi Lande Arti Pillay Rohitash Chandra 53 9 0 28 Feb 2023
HugNLP: A Unified and Comprehensive Library for Natural Language Processing Jiadong Wang Nuo Chen Qiushi Sun Wenkang Huang Chengyu Wang Ming Gao 69 4 0 28 Feb 2023
Weighted Sampling for Masked Language Modeling Linhan Zhang Qian Chen Wen Wang Chong Deng Xin Cao Kongzhang Hao Yuxin Jiang Wen Wang 70 2 0 28 Feb 2023
Full Stack Optimization of Transformer Inference: a Survey Sehoon Kim Coleman Hooper Thanakul Wattanawong Minwoo Kang Ruohan Yan ... Qijing Huang Kurt Keutzer Michael W. Mahoney Y. Shao A. Gholami MQ 163 106 0 27 Feb 2023
Elementwise Language Representation Du-Yeong Kim Jeeeun Kim 67 0 0 27 Feb 2023
Deep Learning for Video-Text Retrieval: a Review Cunjuan Zhu Qi Jia Wei Chen Yanming Guo Yu Liu 75 18 0 24 Feb 2023
KHAN: Knowledge-Aware Hierarchical Attention Networks for Accurate Political Stance Prediction Yunyong Ko Seongeun Ryu Soeun Han Youngseung Jeon Jaehoon Kim Sohyun Park Kyungsik Han Hanghang Tong Sang-Wook Kim 115 15 0 23 Feb 2023
VoxSRC 2022: The Fourth VoxCeleb Speaker Recognition Challenge Jaesung Huh A. Brown Jee-weon Jung Joon Son Chung Arsha Nagrani D. Garcia-Romero Andrew Zisserman 99 26 0 20 Feb 2023
A Novel Collaborative Self-Supervised Learning Method for Radiomic Data Zhiyuan Li Hailong Li Anca L. Ralescu Jonathan R. Dillman N. Parikh Lili He 60 9 0 20 Feb 2023
SanskritShala: A Neural Sanskrit NLP Toolkit with Web-Based Interface for Pedagogical and Annotation Purposes Jivnesh Sandhan Anshul Agarwal Laxmidhar Behera Tushar Sandhan Pawan Goyal 67 4 0 19 Feb 2023
Few-shot Multimodal Multitask Multilingual Learning Aman Chadha Vinija Jain 111 0 0 19 Feb 2023
Learning Language Representations with Logical Inductive Bias Jianshu Chen NAI AI4CE LRM 51 3 0 19 Feb 2023
Complex QA and language models hybrid architectures, Survey Xavier Daull P. Bellot Emmanuel Bruno Vincent Martin Elisabeth Murisasco ELM 212 16 0 17 Feb 2023
A Comprehensive Review and a Taxonomy of Edge Machine Learning: Requirements, Paradigms, and Techniques Wenbin Li Hakim Hacid Ebtesam Almazrouei Merouane Debbah 89 13 0 16 Feb 2023
Auto-Parallelizing Large Models with Rhino: A Systematic Approach on Production AI Platform Shiwei Zhang Lansong Diao Siyu Wang Zongyan Cao Yiliang Gu Chang Si Ziji Shi Zhen Zheng Chuan Wu W. Lin AI4CE 54 4 0 16 Feb 2023
Do We Still Need Clinical Language Models? Eric P. Lehman Evan Hernandez Diwakar Mahajan Jonas Wulff Micah J. Smith Zachary M. Ziegler Daniel Nadler Peter Szolovits Alistair E. W. Johnson Emily Alsentzer LM&MA AI4MH 96 142 0 16 Feb 2023
Platform-Independent and Curriculum-Oriented Intelligent Assistant for Higher Education Ramteja Sajja Y. Sermet David M. Cwiertny Ibrahim Demir 60 68 0 15 Feb 2023
Speculative Decoding with Big Little Decoder Sehoon Kim K. Mangalam Suhong Moon Jitendra Malik Michael W. Mahoney A. Gholami Kurt Keutzer MoE 147 112 0 15 Feb 2023
Measuring the Instability of Fine-Tuning Yupei Du D. Nguyen 69 4 0 15 Feb 2023
ForceFormer: Exploring Social Force and Transformer for Pedestrian Trajectory Prediction Wei-chao Zhang Hao Cheng Fatema-Tuj-Johora Monika Sester 86 10 0 15 Feb 2023
Energy Transformer Benjamin Hoover Yuchen Liang Bao Pham Yikang Shen Hendrik Strobelt Duen Horng Chau Mohammed J Zaki Dmitry Krotov ViT 94 49 0 14 Feb 2023
Distinguishability Calibration to In-Context Learning Hongjing Li Hanqi Yan Yanran Li Li Qian Yulan He Lin Gui 77 2 0 13 Feb 2023
The Framework Tax: Disparities Between Inference Efficiency in NLP Research and Deployment Jared Fernandez Jacob Kahn Clara Na Yonatan Bisk Emma Strubell FedML 89 11 0 13 Feb 2023
NYCU-TWO at Memotion 3: Good Foundation, Good Teacher, then you have Good Meme Analysis Yu-Chien Tang Kuang-Da Wang Ting-Yun Ou Wenjie Peng 32 2 0 13 Feb 2023
TextDefense: Adversarial Text Detection based on Word Importance Entropy Lujia Shen Xuhong Zhang S. Ji Yuwen Pu Chunpeng Ge Xing Yang Yanghe Feng AAML 59 8 0 12 Feb 2023
Transformer models: an introduction and catalog X. Amatriain Ananth Sankar Jie Bing Praveen Kumar Bodigutla Timothy J. Hazen Michaeel Kazi 117 53 0 12 Feb 2023
Lightweight Transformers for Clinical Natural Language Processing Omid Rohanian Mohammadmahdi Nouriborji Hannah Jauncey Samaneh Kouchaki Isaric Clinical Characterisation Group Lei A. Clifton L. Merson David Clifton MedIm LM&MA 77 12 0 09 Feb 2023
A Large-Scale Analysis of Persian Tweets Regarding Covid-19 Vaccination Taha ShabaniMirzaei Houmaan Chamani Amirhossein Abaskohi Zhivar Sourati Hassan Zadeh B. Bahrak 18 1 0 09 Feb 2023
CRL+: A Novel Semi-Supervised Deep Active Contrastive Representation Learning-Based Text Classification Model for Insurance Data Amir Namavar Jahromi Ebrahim Pourjafari H. Karimipour Amit Satpathy Lovell Hodge 50 3 0 08 Feb 2023
Training-free Lexical Backdoor Attacks on Language Models Yujin Huang Terry Yue Zhuo Xingliang Yuan Han Hu Lizhen Qu Chunyang Chen SILM 92 46 0 08 Feb 2023
Revisiting Offline Compression: Going Beyond Factorization-based Methods for Transformer Language Models Mohammadreza Banaei Klaudia Bałazy Artur Kasymov R. Lebret Jacek Tabor Karl Aberer OffRL 45 0 0 08 Feb 2023
Leveraging Summary Guidance on Medical Report Summarization Yunqi Zhu Xuebing Yang Yuanyuan Wu Wensheng Zhang 55 11 0 08 Feb 2023
EvoText: Enhancing Natural Language Generation Models via Self-Escalation Learning for Up-to-Date Knowledge and Improved Performance Zheng Yuan HU Xue Chuxu Zhang Yongming Liu VLM 66 0 0 08 Feb 2023
SLaM: Student-Label Mixing for Distillation with Unlabeled Examples Vasilis Kontonis Fotis Iliopoulos Khoa Trinh Cenk Baykal Gaurav Menghani Erik Vee 85 8 0 08 Feb 2023
A Survey on Arabic Named Entity Recognition: Past, Recent Advances, and Future Trends Xiaoye Qu Yingjie Gu Qingrong Xia Zechang Li Zhefeng Wang Baoxing Huai 92 20 0 07 Feb 2023
Computation vs. Communication Scaling for Future Transformers on Future Hardware Suchita Pati Shaizeen Aga Mahzabeen Islam Nuwan Jayasena Matthew D. Sinclair 51 10 0 06 Feb 2023
Towards energy-efficient Deep Learning: An overview of energy-efficient approaches along the Deep Learning Lifecycle Vanessa Mehlin Sigurd Schacht Carsten Lanquillon HAI MedIm 127 20 0 05 Feb 2023
Representation Deficiency in Masked Language Modeling Yu Meng Jitin Krishnan Sinong Wang Qifan Wang Yuning Mao Han Fang Marjan Ghazvininejad Jiawei Han Luke Zettlemoyer 140 7 0 04 Feb 2023
Entity-Agnostic Representation Learning for Parameter-Efficient Knowledge Graph Embedding Yin Hua Wen Zhang Zhen Yao Yushan Zhu Yang Gao Jeff Z. Pan Hua-zeng Chen 55 10 0 03 Feb 2023
A Survey on Efficient Training of Transformers Bohan Zhuang Jing Liu Zizheng Pan Haoyu He Yuetian Weng Chunhua Shen 128 49 0 02 Feb 2023
Multimodality Representation Learning: A Survey on Evolution, Pretraining and Its Applications Muhammad Arslan Manzoor S. Albarri Ziting Xian Zaiqiao Meng Preslav Nakov Shangsong Liang AI4TS 101 32 0 01 Feb 2023
UPop: Unified and Progressive Pruning for Compressing Vision-Language Transformers Dachuan Shi Chaofan Tao Ying Jin Zhendong Yang Chun Yuan Jiaqi Wang VLM ViT 116 39 0 31 Jan 2023
ContCommRTD: A Distributed Content-based Misinformation-aware Community Detection System for Real-Time Disaster Reporting Elena Simona Apostol Ciprian-Octavian Truică Adrian Paschke 77 20 0 30 Jan 2023