v1v2v3v4v5v6 (latest)

ALBERT: A Lite BERT for Self-supervised Learning of Language Representations

26 September 2019

ArXiv (abs)PDF HTML Github (3271★)

Papers citing "ALBERT: A Lite BERT for Self-supervised Learning of Language Representations"

50 / 2,935 papers shown

Title
Improving self-supervised representation learning via sequential adversarial masking Dylan Sam Min Bai Tristan McKinney Li Erran Li SSL 88 0 0 16 Dec 2022
The Effects of In-domain Corpus Size on pre-training BERT Chris Sanchez Zheyu Zhang AI4CE 23 4 0 15 Dec 2022
Gradient-based Intra-attention Pruning on Pre-trained Language Models Ziqing Yang Yiming Cui Xin Yao Shijin Wang VLM 71 12 0 15 Dec 2022
Efficient Self-supervised Learning with Contextualized Target Representations for Vision, Speech and Language Alexei Baevski Arun Babu Wei-Ning Hsu Michael Auli VLM SSL 129 97 0 14 Dec 2022
Pre-trained Language Models Can be Fully Zero-Shot Learners Xuandong Zhao Siqi Ouyang Zhiguo Yu Ming-li Wu Lei Li VLM LRM 103 34 0 14 Dec 2022
Paraphrase Identification with Deep Learning: A Review of Datasets and Methods Chao Zhou Cheng Qiu Daniel Ernesto Acuna 127 26 0 13 Dec 2022
TencentPretrain: A Scalable and Flexible Toolkit for Pre-training Models of Different Modalities Zhe Zhao Yudong Li Cheng-An Hou Jing-xin Zhao Rong Tian ... Xingwu Sun Zhanhui Kang Xiaoyong Du Linlin Shen Kimmo Yan VLM 106 24 0 13 Dec 2022
Federated Few-Shot Learning for Mobile NLP Dongqi Cai Shangguang Wang Yaozong Wu F. Lin Mengwei Xu FedML 93 12 0 12 Dec 2022
Ensembling Transformers for Cross-domain Automatic Term Extraction T. Hanh Matej Martinc Andraz Pelicon Antoine Doucet Senja Pollak 42 6 0 12 Dec 2022
Towards Leaving No Indic Language Behind: Building Monolingual Corpora, Benchmark and Models for Indic Languages Sumanth Doddapaneni Rahul Aralikatte Gowtham Ramesh Shreyansh Goyal Mitesh M. Khapra Anoop Kunchukuttan Pratyush Kumar ELM 107 86 0 11 Dec 2022
Topic-Aware Response Generation in Task-Oriented Dialogue with Unstructured Knowledge Access Yue Feng Gerasimos Lampouras Ignacio Iacobacci 56 4 0 10 Dec 2022
Multi-task Learning for Personal Health Mention Detection on Social Media O. Aduragba Jialin Yu Alexandra I. Cristea 41 0 0 09 Dec 2022
Sparse Upcycling: Training Mixture-of-Experts from Dense Checkpoints Aran Komatsuzaki J. Puigcerver James Lee-Thorp Carlos Riquelme Ruiz Basil Mustafa Joshua Ainslie Yi Tay Mostafa Dehghani N. Houlsby MoMe MoE 106 124 0 09 Dec 2022
From Cloze to Comprehension: Retrofitting Pre-trained Masked Language Model to Pre-trained Machine Reader Weiwen Xu Xin Li Wenxuan Zhang Meng Zhou W. Lam Luo Si Lidong Bing 86 2 0 09 Dec 2022
Alleviating neighbor bias: augmenting graph self-supervise learning with structural equivalent positive samples Jiawei Zhu Mei Hong R. Du Haifeng Li 76 3 0 08 Dec 2022
Hierarchical multimodal transformers for Multi-Page DocVQA Rubèn Pérez Tito Dimosthenis Karatzas Ernest Valveny 92 61 0 07 Dec 2022
LawngNLI: A Long-Premise Benchmark for In-Domain Generalization from Short to Long Contexts and for Implication-Based Retrieval William F. Bruno Dan Roth ELM AILaw 51 7 0 06 Dec 2022
FacT: Factor-Tuning for Lightweight Adaptation on Vision Transformer Shibo Jie Zhi-Hong Deng 80 137 0 06 Dec 2022
Transformers for End-to-End InfoSec Tasks: A Feasibility Study Ethan M. Rudd Mohammad Saidur Rahman Philip Tully 80 5 0 05 Dec 2022
Grounded Keys-to-Text Generation: Towards Factual Open-Ended Generation Faeze Brahman Baolin Peng Michel Galley Sudha Rao Bill Dolan Snigdha Chaturvedi Jianfeng Gao HILM 69 5 0 04 Dec 2022
MiLMo:Minority Multilingual Pre-trained Language Model Sisi Liu Hanru Shi Xinhe Yu Wugedele Bao Yuan Sun Xiaobing Zhao 81 0 0 04 Dec 2022
IRRGN: An Implicit Relational Reasoning Graph Network for Multi-turn Response Selection Jingcheng Deng Hengwei Dai Xuewei Guo Yuanchen Ju Wei Peng LRM 70 2 0 01 Dec 2022
Language Model Pre-training on True Negatives Zhuosheng Zhang Hai Zhao Masao Utiyama Eiichiro Sumita 71 2 0 01 Dec 2022
A Commonsense-Infused Language-Agnostic Learning Framework for Enhancing Prediction of Political Polarity in Multilingual News Headlines Swati Swati Adrian Mladenic Grobelnik Dunja Mladenić M. Grobelnik 77 3 0 01 Dec 2022
Towards Practical Few-shot Federated NLP Dongqi Cai Yaozong Wu Haitao Yuan Shangguang Wang F. Lin Mengwei Xu FedML 84 6 0 01 Dec 2022
A Pipeline for Generating, Annotating and Employing Synthetic Data for Real World Question Answering Matthew Maufe James Ravenscroft Rob Procter Maria Liakata 51 3 0 30 Nov 2022
Transformers are Short Text Classifiers: A Study of Inductive Short Text Classifiers on Benchmarks and Real-world Datasets Fabian Karl A. Scherp VLM 74 20 0 30 Nov 2022
Revisiting text decomposition methods for NLI-based factuality scoring of summaries John Glover Federico Fancellu V. Jagannathan Matthew R. Gormley Thomas Schaaf HILM 87 17 0 30 Nov 2022
Protein Language Models and Structure Prediction: Connection and Progression Bozhen Hu Jun Xia Jiangbin Zheng Cheng Tan Yufei Huang Yongjie Xu Stan Z. Li 70 41 0 30 Nov 2022
Improving Commonsense in Vision-Language Models via Knowledge Graph Riddles Shuquan Ye Yujia Xie Dongdong Chen Yichong Xu Lu Yuan Chenguang Zhu Jing Liao VLM 66 12 0 29 Nov 2022
Survey on Self-Supervised Multimodal Representation Learning and Foundation Models Sushil Thapa AI4TS SSL 48 1 0 29 Nov 2022
On the Effectiveness of Parameter-Efficient Fine-Tuning Z. Fu Haoran Yang Anthony Man-Cho So Wai Lam Lidong Bing Nigel Collier 76 162 0 28 Nov 2022
Arguments to Key Points Mapping with Prompt-based Learning Ahnaf Mozib Samin Behrooz Nikandish Jingyan Chen AAML 48 2 0 28 Nov 2022
ESIE-BERT: Enriching Sub-words Information Explicitly with BERT for Joint Intent Classification and SlotFilling Yutian Guo Zhilong Xie Xingyan Chen Huangen Chen Leilei Wang Huaming Du Shaopeng Wei Yu Zhao Qing Li Ganglu Wu 102 10 0 27 Nov 2022
A Survey of Text Representation Methods and Their Genealogy Philipp Siebers Christian Janiesch Patrick Zschech AI4TS 33 9 0 26 Nov 2022
DBA: Efficient Transformer with Dynamic Bilinear Low-Rank Attention Bosheng Qin Juncheng Li Siliang Tang Yueting Zhuang 52 2 0 24 Nov 2022
This is the way: designing and compiling LEPISZCZE, a comprehensive NLP benchmark for Polish Lukasz Augustyniak Kamil Tagowski Albert Sawczyn Denis Janiak Roman Bartusiak ... Arkadiusz Janz Piotr Szymañski M. Morzy Tomasz Kajdanowicz Maciej Piasecki 62 12 0 23 Nov 2022
Word-Level Representation From Bytes For Language Modeling Chul Lee Qipeng Guo Xipeng Qiu 72 1 0 23 Nov 2022
Linear Interpolation In Parameter Space is Good Enough for Fine-Tuned Language Models Mark Rofin Nikita Balagansky Daniil Gavrilov MoMe KELM 94 7 0 22 Nov 2022
A Survey on Backdoor Attack and Defense in Natural Language Processing Xuan Sheng Zhaoyang Han Piji Li Xiangmao Chang SILM 71 21 0 22 Nov 2022
Evaluating the Knowledge Dependency of Questions Hyeongdon Moon Yoonseok Yang Jamin Shin Hangyeol Yu Seunghyun Lee Myeongho Jeong Juneyoung Park Minsam Kim Seungtaek Choi AI4Ed 63 11 0 21 Nov 2022
Enhancing Self-Consistency and Performance of Pre-Trained Language Models through Natural Language Inference E. Mitchell Joseph J. Noh Siyan Li William S. Armstrong Ananth Agarwal Patrick Liu Chelsea Finn Christopher D. Manning 84 35 0 21 Nov 2022
Perceiver-VL: Efficient Vision-and-Language Modeling with Iterative Latent Attention Zineng Tang Jaemin Cho Jie Lei Joey Tianyi Zhou VLM 77 9 0 21 Nov 2022
AF Adapter: Continual Pretraining for Building Chinese Biomedical Language Model Yongyu Yan Kui Xue Xiaoming Shi Qi Ye Jingping Liu Tong Ruan CLL 71 2 0 21 Nov 2022
Artificial Interrogation for Attributing Language Models Farhan Dhanani Muhammad Rafi 34 1 0 20 Nov 2022
Combining State-of-the-Art Models with Maximal Marginal Relevance for Few-Shot and Zero-Shot Multi-Document Summarization David Adams Gandharv Suri Yllias Chali VLM 57 3 0 19 Nov 2022
Entity-Assisted Language Models for Identifying Check-worthy Sentences Ting-Han Su Craig Macdonald I. Ounis 32 0 0 19 Nov 2022
GENIUS: Sketch-based Language Model Pre-training via Extreme and Selective Masking for Text Generation and Augmentation Biyang Guo Yeyun Gong Yelong Shen Songqiao Han Hailiang Huang Nan Duan Weizhu Chen VLM 80 19 0 18 Nov 2022
Vision Transformers in Medical Imaging: A Review Emerald U. Henry Onyeka Emebob C. Omonhinmin ViT MedIm 88 36 0 18 Nov 2022
3d human motion generation from the text via gesture action classification and the autoregressive model Gwantae Kim Youngsuk Ryu Junyeop Lee D. Han Jeongmin Bae Hanseok Ko 34 2 0 18 Nov 2022