MC-BERT: Efficient Language Pre-Training via a Meta Controller

MC-BERT: Efficient Language Pre-Training via a Meta Controller

10 June 2020

Jiang Bian

Papers citing "MC-BERT: Efficient Language Pre-Training via a Meta Controller"

8 / 8 papers shown

Title
MATES: Model-Aware Data Selection for Efficient Pretraining with Data Influence Models Zichun Yu Spandan Das Chenyan Xiong 53 29 0 10 Jun 2024
Enhancing Context Through Contrast Kshitij Ambilduke Aneesh Shetye Diksha Bagade Rishika Bhagwatkar Khurshed Fitter P. Vagdargi Shital S. Chiddarwar 31 0 0 06 Jan 2024
Pre-training Language Model as a Multi-perspective Course Learner Beiduo Chen Shaohan Huang Zi-qiang Zhang Wu Guo Zhen-Hua Ling Haizhen Huang Furu Wei Weiwei Deng Qi Zhang 39 0 0 06 May 2023
METRO: Efficient Denoising Pretraining of Large Scale Autoencoding Language Models with Model Generated Signals Payal Bajaj Chenyan Xiong Guolin Ke Xiaodong Liu Di He Saurabh Tiwary Tie-Yan Liu Paul N. Bennett Xia Song Jianfeng Gao 52 32 0 13 Apr 2022
Building Chinese Biomedical Language Models via Multi-Level Text Discrimination Quan Wang Songtai Dai Benfeng Xu Yajuan Lyu Yong Zhu Hua Wu Haifeng Wang 71 14 0 14 Oct 2021
SAS: Self-Augmentation Strategy for Language Model Pre-training Yifei Xu Jingqiao Zhang Ru He Liangzhu Ge Chao Yang Cheng Yang Ying Wu 42 1 0 14 Jun 2021
COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretraining Yu Meng Chenyan Xiong Payal Bajaj Saurabh Tiwary Paul N. Bennett Jiawei Han Xia Song 127 203 0 16 Feb 2021
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 304 7,005 0 20 Apr 2018