ERNIE 2.0: A Continual Pre-training Framework for Language Understanding

29 July 2019

Papers citing "ERNIE 2.0: A Continual Pre-training Framework for Language Understanding"

50 / 109 papers shown

Title
RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-ranking Ruiyang Ren Yingqi Qu Jing Liu Wayne Xin Zhao Qiaoqiao She Hua Wu Haifeng Wang Ji-Rong Wen 141 249 0 14 Oct 2021
Building Chinese Biomedical Language Models via Multi-Level Text Discrimination Quan Wang Songtai Dai Benfeng Xu Yajuan Lyu Yong Zhu Hua Wu Haifeng Wang 71 14 0 14 Oct 2021
Adversarial Retriever-Ranker for dense text retrieval Hang Zhang Yeyun Gong Yelong Shen Jiancheng Lv Nan Duan Weizhu Chen VLM RALM 48 115 0 07 Oct 2021
A Survey On Neural Word Embeddings Erhan Sezerer Selma Tekir AI4TS 26 12 0 05 Oct 2021
Analysing the Effect of Masking Length Distribution of MLM: An Evaluation Framework and Case Study on Chinese MRC Datasets Changchang Zeng Shaobo Li 24 6 0 29 Sep 2021
Question Answering over Electronic Devices: A New Benchmark Dataset and a Multi-Task Learning based QA Framework Abhilash Nandy Soumya Sharma Shubham Maddhashiya K. Sachdeva Pawan Goyal Niloy Ganguly 30 17 0 13 Sep 2021
CPT: A Pre-Trained Unbalanced Transformer for Both Chinese Language Understanding and Generation Yunfan Shao Zhichao Geng Yitao Liu Junqi Dai Hang Yan Fei Yang Li Zhe Hujun Bao Xipeng Qiu MedIm 70 149 0 13 Sep 2021
Frustratingly Simple Pretraining Alternatives to Masked Language Modeling Atsuki Yamaguchi G. Chrysostomou Katerina Margatina Nikolaos Aletras 27 25 0 04 Sep 2021
NoiER: An Approach for Training more Reliable Fine-TunedDownstream Task Models Myeongjun Jang Thomas Lukasiewicz 24 4 0 29 Aug 2021
Regularizing Transformers With Deep Probabilistic Layers Aurora Cobo Aguilera Pablo Martínez Olmos Antonio Artés-Rodríguez Fernando Pérez-Cruz 41 7 0 23 Aug 2021
PAIR: Leveraging Passage-Centric Similarity Relation for Improving Dense Passage Retrieval Ruiyang Ren Shangwen Lv Yingqi Qu Jing Liu Wayne Xin Zhao Qiaoqiao She Hua Wu Haifeng Wang Ji-Rong Wen 130 92 0 13 Aug 2021
Unsupervised Corpus Aware Language Model Pre-training for Dense Passage Retrieval Luyu Gao Jamie Callan RALM 175 330 0 12 Aug 2021
Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing Pengfei Liu Weizhe Yuan Jinlan Fu Zhengbao Jiang Hiroaki Hayashi Graham Neubig VLM SyDa 73 3,853 0 28 Jul 2021
An Evaluation of Generative Pre-Training Model-based Therapy Chatbot for Caregivers Lu Wang Munif Ishad Mujib Jake Williams G. Demiris Jina Huh-Yoo AI4MH 32 32 0 28 Jul 2021
ChineseBERT: Chinese Pretraining Enhanced by Glyph and Pinyin Information Zijun Sun Xiaoya Li Xiaofei Sun Yuxian Meng Xiang Ao Qing He Fei Wu Jiwei Li SSeg 57 184 0 30 Jun 2021
Alzheimer's Disease Detection from Spontaneous Speech through Combining Linguistic Complexity and (Dis)Fluency Features with Pretrained Language Models Yu Qiao Xuefeng Yin Daniel Wiechmann E. Kerz 25 22 0 16 Jun 2021
Pre-Trained Models: Past, Present and Future Xu Han Zhengyan Zhang Ning Ding Yuxian Gu Xiao Liu ... Jie Tang Ji-Rong Wen Jinhui Yuan Wayne Xin Zhao Jun Zhu AIFin MQ AI4MH 58 816 0 14 Jun 2021
Linguistically Informed Masking for Representation Learning in the Patent Domain Sophia Althammer Mark Buckley Sebastian Hofstatter Allan Hanbury 45 11 0 10 Jun 2021
ERNIE-Tiny : A Progressive Distillation Framework for Pretrained Transformer Compression Weiyue Su Xuyi Chen Shi Feng Jiaxiang Liu Weixin Liu Yu Sun Hao Tian Hua Wu Haifeng Wang 34 13 0 04 Jun 2021
Read, Listen, and See: Leveraging Multimodal Information Helps Chinese Spell Checking Heng-Da Xu Zhongli Li Qingyu Zhou Chao Li Zizhen Wang Yunbo Cao Heyan Huang Xian-Ling Mao 46 94 0 26 May 2021
REPT: Bridging Language Models and Machine Reading Comprehension via Retrieval-Based Pre-training Fangkai Jiao Yangyang Guo Yilin Niu Feng Ji Feng-Lin Li Liqiang Nie LRM 34 12 0 10 May 2021
Efficient Retrieval Optimized Multi-task Learning He Fun S. Gandhi Sujith Ravi RALM 26 6 0 20 Apr 2021
Understanding Chinese Video and Language via Contrastive Multimodal Pre-Training Chenyi Lei Shixian Luo Yong-jin Liu Wanggui He Jiamang Wang Guoxin Wang Haihong Tang Chunyan Miao Houqiang Li 30 41 0 19 Apr 2021
Editing Factual Knowledge in Language Models Nicola De Cao Wilker Aziz Ivan Titov KELM 68 478 0 16 Apr 2021
Relational World Knowledge Representation in Contextual Language Models: A Review Tara Safavi Danai Koutra KELM 38 51 0 12 Apr 2021
A Heuristic-driven Uncertainty based Ensemble Framework for Fake News Detection in Tweets and News Articles Sourya Dipta Das Ayan Basak S. Dutta 45 47 0 05 Apr 2021
Machine Learning Meets Natural Language Processing -- The story so far N. Galanis P. Vafiadis K.-G. Mirzaev G. Papakostas 38 7 0 27 Mar 2021
Boosting Low-Resource Biomedical QA via Entity-Aware Masking Strategies Gabriele Pergola E. Kochkina Lin Gui Maria Liakata Yulan He 88 31 0 16 Feb 2021
Combining pre-trained language models and structured knowledge Pedro Colon-Hernandez Catherine Havasi Jason B. Alonso Matthew Huggins C. Breazeal KELM 38 48 0 28 Jan 2021
Exploring Text-transformers in AAAI 2021 Shared Task: COVID-19 Fake News Detection in English Xiangyang Li Yu Xia Xiang Long Zheng Li Sujian Li 226 37 0 07 Jan 2021
Learning to Emphasize: Dataset and Shared Task Models for Selecting Emphasis in Presentation Slides Amirreza Shirani Gia-Lac Tran Hieu Trinh Franck Dernoncourt Nedim Lipka P. Asente J. Echevarria Thamar Solorio 184 1 0 02 Jan 2021
ERNIE-Doc: A Retrospective Long-Document Modeling Transformer Siyu Ding Junyuan Shang Shuohuan Wang Yu Sun Hao Tian Hua Wu Haifeng Wang 73 52 0 31 Dec 2020
Improving BERT with Syntax-aware Local Attention Zhongli Li Qingyu Zhou Chao Li Ke Xu Yunbo Cao 63 44 0 30 Dec 2020
KgPLM: Knowledge-guided Language Model Pre-training via Generative and Discriminative Learning Bin He Xin Jiang Jinghui Xiao Qun Liu 37 18 0 07 Dec 2020
CalibreNet: Calibration Networks for Multilingual Sequence Labeling Shining Liang Linjun Shou J. Pei Ming Gong Wanli Zuo Daxin Jiang 31 8 0 11 Nov 2020
A Comprehensive Survey on Word Representation Models: From Classical to State-Of-The-Art Word Representation Language Models Usman Naseem Imran Razzak S. Khan M. Prasad 14 156 0 28 Oct 2020
ERNIE-Gram: Pre-Training with Explicitly N-Gram Masked Language Modeling for Natural Language Understanding Dongling Xiao Yukun Li Han Zhang Yu Sun Hao Tian Hua Wu Haifeng Wang 27 38 0 23 Oct 2020
Multi-Style Transfer with Discriminative Feedback on Disjoint Corpus Navita Goyal Balaji Vasan Srinivasan Anandhavelu Natarajan Abhilasha Sancheti 36 20 0 22 Oct 2020
CharacterBERT: Reconciling ELMo and BERT for Word-Level Open-Vocabulary Representations From Characters Hicham El Boukkouri Olivier Ferret Thomas Lavergne Hiroshi Noji Pierre Zweigenbaum Junichi Tsujii 77 156 0 20 Oct 2020
RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answering Yingqi Qu Yuchen Ding Jing Liu Kai Liu Ruiyang Ren Xin Zhao Daxiang Dong Hua Wu Haifeng Wang RALM OffRL 214 595 0 16 Oct 2020
HUJI-KU at MRP~2020: Two Transition-based Neural Parsers Ofir Arviv Ruixiang Cui Daniel Hershcovich 42 10 0 12 Oct 2020
On Losses for Modern Language Models Stephane Aroca-Ouellette Frank Rudzicz 22 32 0 04 Oct 2020
kk2018 at SemEval-2020 Task 9: Adversarial Training for Code-Mixing Sentiment Classification Jiaxiang Liu Xuyi Chen Shikun Feng Shuohuan Wang Ouyang Xuan Yu Sun Zhengjie Huang Weiyue Su 32 19 0 08 Sep 2020
Continual BERT: Continual Learning for Adaptive Extractive Summarization of COVID-19 Literature Jongjin Park CLL 33 15 0 07 Jul 2020
ERNIE-ViL: Knowledge Enhanced Vision-Language Representations Through Scene Graph Fei Yu Jiji Tang Weichong Yin Yu Sun Hao Tian Hua Wu Haifeng Wang 31 376 0 30 Jun 2020
Evaluation of Text Generation: A Survey Asli Celikyilmaz Elizabeth Clark Jianfeng Gao ELM LM&MA 19 377 0 26 Jun 2020
SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020) Marcos Zampieri Preslav Nakov Sara Rosenthal Pepa Atanasova Georgi Karadzhov Hamdy Mubarak Leon Derczynski Zeses Pitenis cCaugri cColtekin 30 482 0 12 Jun 2020
Syntactic Structure Distillation Pretraining For Bidirectional Encoders A. Kuncoro Lingpeng Kong Daniel Fried Dani Yogatama Laura Rimell Chris Dyer Phil Blunsom 31 33 0 27 May 2020
BiQGEMM: Matrix Multiplication with Lookup Table For Binary-Coding-based Quantized DNNs Yongkweon Jeon Baeseong Park S. Kwon Byeongwook Kim Jeongin Yun Dongsoo Lee MQ 33 30 0 20 May 2020
Exploiting Structured Knowledge in Text via Graph-Guided Representation Learning Tao Shen Yi Mao Pengcheng He Guodong Long Adam Trischler Weizhu Chen 18 63 0 29 Apr 2020