ERNIE 2.0: A Continual Pre-training Framework for Language Understanding

29 July 2019

Papers citing "ERNIE 2.0: A Continual Pre-training Framework for Language Understanding"

50 / 108 papers shown

Title
Enriching Patent Claim Generation with European Patent Dataset Lekang Jiang Chengzu Li Stephan Goetz 7 0 0 18 May 2025
Alchemist: Towards the Design of Efficient Online Continual Learning System Yuyang Huang Yuhan Liu Haryadi S. Gunawi Beibin Li Changho Hwang CLL OnRL 106 0 0 03 Mar 2025
A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethics Kai He Rui Mao Qika Lin Yucheng Ruan Xiang Lan Mengling Feng Min Zhang LM&MA AILaw 104 155 0 28 Jan 2025
A Contrastive Pretrain Model with Prompt Tuning for Multi-center Medication Recommendation Qiang Liu Zhaopeng Qiu Xiangyu Zhao X. Wu Zijian Zhang Tong Xu Feng Tian 41 0 0 31 Dec 2024
Comprehensive benchmarking of large language models for RNA secondary structure prediction L. I. Zablocki L. A. Bugnon M. Gerard L. Di Persia G. Stegmayer D. H. Milone AI4TS 31 3 0 21 Oct 2024
Towards LifeSpan Cognitive Systems Yu Wang Chi Han Tongtong Wu Xiaoxin He Wangchunshu Zhou ... Zexue He Wei Wang Gholamreza Haffari Heng Ji Julian McAuley KELM CLL 215 1 0 20 Sep 2024
From Prediction to Application: Language Model-based Code Knowledge Tracing with Domain Adaptive Pre-Training and Automatic Feedback System with Pedagogical Prompting for Comprehensive Programming Education Unggi Lee Jiyeong Bae Yeonji Jung Minji Kang Gyuri Byun ... Sookbun Lee Jaekwon Park Taekyung Ahn Gunho Lee Hyeoncheol Kim AI4Ed KELM 39 1 0 31 Aug 2024
An Investigation of Warning Erroneous Chat Translations in Cross-lingual Communication Yunmeng Li Jun Suzuki Makoto Morishita Kaori Abe Kentaro Inui 65 1 0 28 Aug 2024
DEPTH: Discourse Education through Pre-Training Hierarchically Zachary Bamberger Ofek Glick Chaim Baskin Yonatan Belinkov 67 0 0 13 May 2024
CRE-LLM: A Domain-Specific Chinese Relation Extraction Framework with Fine-tuned Large Language Model Zhengpeng Shi Haoran Luo LRM ALM 38 2 0 28 Apr 2024
From Matching to Generation: A Survey on Generative Information Retrieval Xiaoxi Li Jiajie Jin Yujia Zhou Yuyao Zhang Peitian Zhang Yutao Zhu Zhicheng Dou 3DV 84 47 0 23 Apr 2024
Simple and Scalable Strategies to Continually Pre-train Large Language Models Adam Ibrahim Benjamin Thérien Kshitij Gupta Mats L. Richter Quentin Anthony Timothée Lesort Eugene Belilovsky Irina Rish KELM CLL 44 54 0 13 Mar 2024
GraphEdit: Large Language Models for Graph Structure Learning Zirui Guo Lianghao Xia Yanhua Yu Yuling Wang Zixuan Yang Zhiyong Huang Chao Huang 55 19 0 23 Feb 2024
GeoDecoder: Empowering Multimodal Map Understanding Feng Qi Mian Dai Zixian Zheng Chao Wang 42 1 0 26 Jan 2024
Sentiment Analysis through LLM Negotiations Xiaofei Sun Xiaoya Li Shengyu Zhang Shuhe Wang Fei Wu Jiwei Li Tianwei Zhang Guoyin Wang 35 16 0 03 Nov 2023
The Skipped Beat: A Study of Sociopragmatic Understanding in LLMs for 64 Languages Chiyu Zhang Khai Duy Doan Qisheng Liao Muhammad Abdul-Mageed 36 6 0 23 Oct 2023
Knowledgeable In-Context Tuning: Exploring and Exploiting Factual Knowledge for In-Context Learning Jie Wang Chengyu Wang Chuanqi Tan Jun Huang Ming Gao KELM 34 4 0 26 Sep 2023
Prompt-Based Length Controlled Generation with Reinforcement Learning Renlong Jie Xiaojun Meng Lifeng Shang Xin Jiang Qun Liu 22 8 0 23 Aug 2023
A Novel Correlation-optimized Deep Learning Method for Wind Speed Forecast Yang Yang Jin Lang Jian Wu Yanyan Zhang Xiangman Song 35 7 0 03 Jun 2023
It's about Time: Rethinking Evaluation on Rumor Detection Benchmarks using Chronological Splits Yida Mu Kalina Bontcheva Nikolaos Aletras 25 19 0 06 Feb 2023
Multipath agents for modular multitask ML systems Andrea Gesmundo 28 1 0 06 Feb 2023
Modeling Sequential Sentence Relation to Improve Cross-lingual Dense Retrieval Shunyu Zhang Yaobo Liang Ming Gong Daxin Jiang Nan Duan 25 4 0 03 Feb 2023
InPars-Light: Cost-Effective Unsupervised Training of Efficient Rankers Leonid Boytsov Preksha Patel Vivek Sourabh Riddhi Nisar Sayan Kundu R. Ramanathan Eric Nyberg 34 19 0 08 Jan 2023
APOLLO: A Simple Approach for Adaptive Pretraining of Language Models for Logical Reasoning Soumya Sanyal Yichong Xu Shuohang Wang Ziyi Yang Reid Pryzant Wenhao Yu Chenguang Zhu Xiang Ren ReLM LRM 35 8 0 19 Dec 2022
CAPSTONE: Curriculum Sampling for Dense Retrieval with Document Expansion Xingwei He Yeyun Gong Alex Jin Hang Zhang Anlei Dong Jian Jiao Siu-Ming Yiu Nan Duan RALM 54 3 0 18 Dec 2022
TencentPretrain: A Scalable and Flexible Toolkit for Pre-training Models of Different Modalities Zhe Zhao Yudong Li Cheng-An Hou Jing-xin Zhao Rong Tian ... Xingwu Sun Zhanhui Kang Xiaoyong Du Linlin Shen Kimmo Yan VLM 41 23 0 13 Dec 2022
Transformers are Short Text Classifiers: A Study of Inductive Short Text Classifiers on Benchmarks and Real-world Datasets Fabian Karl A. Scherp VLM 24 20 0 30 Nov 2022
Exploiting Transformer-based Multitask Learning for the Detection of Media Bias in News Articles Timo Spinde Jan-David Krieger Terry Ruas Jelena Mitrović Franz Götz-Hahn Akiko Aizawa Bela Gipp 35 27 0 07 Nov 2022
UPainting: Unified Text-to-Image Diffusion Generation with Cross-modal Guidance Wei Li Xue Xu Xinyan Xiao Jiacheng Liu Hu Yang ... Zhanpeng Wang Zhifan Feng Qiaoqiao She Yajuan Lyu Hua Wu 121 29 0 28 Oct 2022
SimANS: Simple Ambiguous Negatives Sampling for Dense Text Retrieval Kun Zhou Yeyun Gong Xiao Liu Wayne Xin Zhao Yelong Shen ... Jing Lu Rangan Majumder Ji-Rong Wen Nan Duan Weizhu Chen 44 33 0 21 Oct 2022
Multi-CLS BERT: An Efficient Alternative to Traditional Ensembling Haw-Shiuan Chang Ruei-Yao Sun Kathryn Ricci Andrew McCallum 43 15 0 10 Oct 2022
ERNIE-ViL 2.0: Multi-view Contrastive Learning for Image-Text Pre-training Bin Shan Weichong Yin Yu Sun Hao Tian Hua Wu Haifeng Wang VLM 27 19 0 30 Sep 2022
Show, Interpret and Tell: Entity-aware Contextualised Image Captioning in Wikipedia K. Nguyen Ali Furkan Biten Andrés Mafla Lluís Gómez Dimosthenis Karatzas 36 10 0 21 Sep 2022
Design of the topology for contrastive visual-textual alignment Zhun Sun 32 1 0 05 Sep 2022
CLOWER: A Pre-trained Language Model with Contrastive Learning over Word and Character Representations Borun Chen Hongyin Tang Jiahao Bu Kai Zhang Jingang Wang Qifan Wang Haitao Zheng Wei Wu Liqian Yu VLM 27 1 0 23 Aug 2022
BSpell: A CNN-Blended BERT Based Bangla Spell Checker C. R. Rahman Md. Hasibur Rahman S. Zakir Mohammad Rafsan Mohammed Eunus Ali 41 4 0 20 Aug 2022
Deep Multi-Task Models for Misogyny Identification and Categorization on Arabic Social Media Abdelkader El Mahdaouy Abdellah El Mekki Ahmed Oumar El-Shangiti H. Mousannif Ismail Berrada 16 5 0 16 Jun 2022
E2S2: Encoding-Enhanced Sequence-to-Sequence Pretraining for Language Understanding and Generation Qihuang Zhong Liang Ding Juhua Liu Bo Du Dacheng Tao 54 27 0 30 May 2022
Can Foundation Models Wrangle Your Data? A. Narayan Ines Chami Laurel J. Orr Simran Arora Christopher Ré LMTD AI4CE 181 214 0 20 May 2022
Lexical Knowledge Internalization for Neural Dialog Generation Zhiyong Wu Wei Bi Xiang Li Lingpeng Kong B. Kao 21 2 0 04 May 2022
Knowledge Infused Decoding Ruibo Liu Guoqing Zheng Shashank Gupta Radhika Gaonkar Chongyang Gao Soroush Vosoughi Milad Shokouhi Ahmed Hassan Awadallah KELM 25 14 0 06 Apr 2022
ERNIE-SPARSE: Learning Hierarchical Efficient Transformer Through Regularized Self-Attention Yang Liu Jiaxiang Liu L. Chen Yuxiang Lu Shi Feng Zhida Feng Yu Sun Hao Tian Huancheng Wu Hai-feng Wang 31 9 0 23 Mar 2022
Parallel Instance Query Network for Named Entity Recognition Yongliang Shen Xiaobin Wang Zeqi Tan Guangwei Xu Pengjun Xie Fei Huang Weiming Lu Yueting Zhuang 24 57 0 20 Mar 2022
Geographic Adaptation of Pretrained Language Models Valentin Hofmann Goran Glavavs Nikola Ljubevsić J. Pierrehumbert Hinrich Schütze VLM 21 16 0 16 Mar 2022
Long Time No See! Open-Domain Conversation with Long-Term Persona Memory Xinchao Xu Zhibin Gou Wenquan Wu Zheng-Yu Niu Hua Wu Haifeng Wang Shihang Wang RALM 27 110 0 11 Mar 2022
OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Framework Peng Wang An Yang Rui Men Junyang Lin Shuai Bai Zhikang Li Jianxin Ma Chang Zhou Jingren Zhou Hongxia Yang MLLM ObjD 74 850 0 07 Feb 2022
Linguistically-driven Multi-task Pre-training for Low-resource Neural Machine Translation Zhuoyuan Mao Chenhui Chu Sadao Kurohashi 17 6 0 20 Jan 2022
ERNIE 3.0 Titan: Exploring Larger-scale Knowledge Enhanced Pre-training for Language Understanding and Generation Shuohuan Wang Yu Sun Yang Xiang Zhihua Wu Siyu Ding ... Tian Wu Wei Zeng Ge Li Wen Gao Haifeng Wang ELM 39 79 0 23 Dec 2021
HeterPS: Distributed Deep Learning With Reinforcement Learning Based Scheduling in Heterogeneous Environments Ji Liu Zhihua Wu Dianhai Yu Yanjun Ma Danlei Feng Minxu Zhang Xinxuan Wu Xuefeng Yao Dejing Dou 18 44 0 20 Nov 2021
Response Generation with Context-Aware Prompt Learning X. Gu Kang Min Yoo Sang-Woo Lee 30 25 0 04 Nov 2021