CLOWER: A Pre-trained Language Model with Contrastive Learning over Word
and Character Representations

v1v2 (latest)

CLOWER: A Pre-trained Language Model with Contrastive Learning over Word and Character Representations

23 August 2022

Kai Zhang

ArXiv (abs)PDF HTML

Papers citing "CLOWER: A Pre-trained Language Model with Contrastive Learning over Word and Character Representations"

18 / 18 papers shown

Title
CLINE: Contrastive Learning with Semantic Negative Examples for Natural Language Understanding Dong Wang Ning Ding Pijian Li Haitao Zheng AAML 55 117 0 01 Jul 2021
Charformer: Fast Character Transformers via Gradient-based Subword Tokenization Yi Tay Vinh Q. Tran Sebastian Ruder Jai Gupta Hyung Won Chung Dara Bahri Zhen Qin Simon Baumgartner Cong Yu Donald Metzler 114 159 0 23 Jun 2021
ByT5: Towards a token-free future with pre-trained byte-to-byte models Linting Xue Aditya Barua Noah Constant Rami Al-Rfou Sharan Narang Mihir Kale Adam Roberts Colin Raffel 98 504 0 28 May 2021
ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfer Yuanmeng Yan Rumei Li Sirui Wang Fuzheng Zhang Wei Wu Weiran Xu SSL 119 557 0 25 May 2021
SimCSE: Simple Contrastive Learning of Sentence Embeddings Tianyu Gao Xingcheng Yao Danqi Chen AILaw SSL 261 3,396 0 18 Apr 2021
COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretraining Yu Meng Chenyan Xiong Payal Bajaj Saurabh Tiwary Paul N. Bennett Jiawei Han Xia Song 164 205 0 16 Feb 2021
CLEAR: Contrastive Learning for Sentence Representation Zhuofeng Wu Sinong Wang Jiatao Gu Madian Khabsa Fei Sun Hao Ma SSL 71 323 0 31 Dec 2020
OCNLI: Original Chinese Natural Language Inference Hai Hu Kyle Richardson Liang Xu Lu Li Sandra Kübler L. Moss 88 118 0 12 Oct 2020
AMBERT: A Pre-trained Language Model with Multi-Grained Tokenization Xinsong Zhang Pengshuai Li Hang Li 50 52 0 27 Aug 2020
Supervised Contrastive Learning Prannay Khosla Piotr Teterwak Chen Wang Aaron Sarna Yonglong Tian Phillip Isola Aaron Maschinot Ce Liu Dilip Krishnan SSL 151 4,547 0 23 Apr 2020
CLUE: A Chinese Language Understanding Evaluation Benchmark Liang Xu Hai Hu Xuanwei Zhang Lu Li Chenjie Cao ... Cong Yue Xinrui Zhang Zhen-Yi Yang Kyle Richardson Zhenzhong Lan ELM 87 386 0 13 Apr 2020
Momentum Contrast for Unsupervised Visual Representation Learning Kaiming He Haoqi Fan Yuxin Wu Saining Xie Ross B. Girshick SSL 204 12,085 0 13 Nov 2019
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations Zhenzhong Lan Mingda Chen Sebastian Goodman Kevin Gimpel Piyush Sharma Radu Soricut SSL AIMat 368 6,455 0 26 Sep 2019
ERNIE 2.0: A Continual Pre-training Framework for Language Understanding Yu Sun Shuohuan Wang Yukun Li Shikun Feng Hao Tian Hua Wu Haifeng Wang CLL 87 810 0 29 Jul 2019
SpanBERT: Improving Pre-training by Representing and Predicting Spans Mandar Joshi Danqi Chen Yinhan Liu Daniel S. Weld Luke Zettlemoyer Omer Levy 147 1,965 0 24 Jul 2019
XLNet: Generalized Autoregressive Pretraining for Language Understanding Zhilin Yang Zihang Dai Yiming Yang J. Carbonell Ruslan Salakhutdinov Quoc V. Le AI4CE 232 8,433 0 19 Jun 2019
ERNIE: Enhanced Representation through Knowledge Integration Yu Sun Shuohuan Wang Yukun Li Shikun Feng Xuyi Chen Han Zhang Xin Tian Danxiang Zhu Hao Tian Hua Wu 121 901 0 19 Apr 2019
DRCD: a Chinese Machine Reading Comprehension Dataset C. Shao Trois Liu Yuting Lai Yiying Tseng Sam S. Tsai 64 128 0 04 Jun 2018