StructBERT: Incorporating Language Structures into Pre-training for Deep Language Understanding

13 August 2019

Papers citing "StructBERT: Incorporating Language Structures into Pre-training for Deep Language Understanding"

50 / 50 papers shown

Title
Beyond Semantics: Learning a Behavior Augmented Relevance Model with Self-supervised Learning Ze-jie Chen Wei Chen Jia Xu Zhongyi Liu Wei Zhang RALM 23 4 0 10 Aug 2023
On the (In)Effectiveness of Large Language Models for Chinese Text Correction Hai-Tao Zheng Haojing Huang Shirong Ma Yong-jia Jiang Yong Li F. Zhou Haitao Zheng Qingyu Zhou 36 43 0 18 Jul 2023
ScrollTimes: Tracing the Provenance of Paintings as a Window into History Wei Zhang Jason K. Wong Yitian Chen Ailing Jia Luwei Wang Jianwei Zhang Lechao Cheng Huamin Qu Wei Chen 29 7 0 15 Jun 2023
QUERT: Continual Pre-training of Language Model for Query Understanding in Travel Domain Search Jian Xie Yidan Liang Jingping Liu Yanghua Xiao Baohua Wu Shenghua Ni VLM LRM 38 8 0 11 Jun 2023
Zero-Shot Text Classification via Self-Supervised Tuning Chaoqun Liu Wenxuan Zhang Guizhen Chen Xiaobao Wu A. Luu Chip Hong Chang Lidong Bing VLM 55 11 0 19 May 2023
GeoGLUE: A GeoGraphic Language Understanding Evaluation Benchmark Dongyang Li Ruixue Ding Qiang-Wei Zhang Zheng Li Boli Chen ... Yao Xu Xin Li Ning Guo Fei Huang Xiaofeng He ELM VLM 34 5 0 11 May 2023
Going beyond research datasets: Novel intent discovery in the industry setting Aleksandra Chrabrowa Tsimur Hadeliya D. Kajtoch Robert Mroczkowski Piotr Rybak 24 2 0 09 May 2023
Interpretable multimodal sentiment analysis based on textual modality descriptions by using large-scale language models Sixia Li S. Okada 32 3 0 07 May 2023
Overview of the ICASSP 2023 General Meeting Understanding and Generation Challenge (MUG) Qinglin Zhang Chong Deng Jiaqing Liu Hai Yu Qian Chen Wen Wang Zhijie Yan Jinglin Liu Yi Ren Zhou Zhao 46 0 0 24 Mar 2023
A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPT Yihan Cao Siyu Li Yixin Liu Zhiling Yan Yutong Dai Philip S. Yu Lichao Sun 38 509 0 07 Mar 2023
Can ChatGPT Understand Too? A Comparative Study on ChatGPT and Fine-tuned BERT Qihuang Zhong Liang Ding Juhua Liu Bo Du Dacheng Tao AI4MH 66 238 0 19 Feb 2023
A Concept Knowledge Graph for User Next Intent Prediction at Alipay Yacheng He Qianghuai Jia Lin Yuan Ruopeng Li Yixin Ou Ningyu Zhang 22 5 0 02 Jan 2023
Paraphrase Identification with Deep Learning: A Review of Datasets and Methods Chao Zhou Cheng Qiu Daniel Ernesto Acuna 37 25 0 13 Dec 2022
Language Model Pre-training on True Negatives ZhuoSheng Zhang Hai Zhao Masao Utiyama Eiichiro Sumita 36 2 0 01 Dec 2022
FCGEC: Fine-Grained Corpus for Chinese Grammatical Error Correction Lvxiaowei Xu Jian Wu Jiawei Peng Jiayu Fu Ming Cai 32 13 0 22 Oct 2022
Linguistic Rules-Based Corpus Generation for Native Chinese Grammatical Error Correction Shirong Ma Hai-Tao Zheng Rongyi Sun Qingyu Zhou Shulin Huang ... Ruiyang Liu Zhongli Li Yunbo Cao Haitao Zheng Ying Shen 29 26 0 19 Oct 2022
Knowing Where and What: Unified Word Block Pretraining for Document Understanding Song Tao Zijian Wang Tiantian Fan Canjie Luo Can Huang SSL 40 2 0 28 Jul 2022
Attention Mechanism in Neural Networks: Where it Comes and Where it Goes Derya Soydaner 3DV 44 150 0 27 Apr 2022
MuCGEC: a Multi-Reference Multi-Source Evaluation Dataset for Chinese Grammatical Error Correction Yue Zhang Zhenghua Li Zuyi Bao Jiacheng Li Bo Zhang Chen Li Fei Huang Min Zhang ELM 30 50 0 23 Apr 2022
Cross-Domain Generalization and Knowledge Transfer in Transformers Trained on Legal Data Jaromír Šavelka Hannes Westermann Karim Benyekhlef 28 15 0 15 Dec 2021
Linking-Enhanced Pre-Training for Table Semantic Parsing Bowen Qin Lihan Wang Binyuan Hui Ruiying Geng Zhen Cao Min Yang Jian Sun Yongbin Li 35 1 0 18 Nov 2021
Achieving Human Parity on Visual Question Answering Ming Yan Haiyang Xu Chenliang Li Junfeng Tian Bin Bi ... Ji Zhang Songfang Huang Fei Huang Luo Si Rong Jin 32 12 0 17 Nov 2021
ICDAR 2021 Competition on Document VisualQuestion Answering Rubèn Pérez Tito Minesh Mathew C. V. Jawahar Ernest Valveny Dimosthenis Karatzas 40 23 0 10 Nov 2021
MNet-Sim: A Multi-layered Semantic Similarity Network to Evaluate Sentence Similarity Manuela Nayantara Jeyaraj D. Kasthurirathna 13 3 0 09 Nov 2021
Shaking Syntactic Trees on the Sesame Street: Multilingual Probing with Controllable Perturbations Ekaterina Taktasheva Vladislav Mikhailov Ekaterina Artemova 24 13 0 28 Sep 2021
Monolingual and Cross-Lingual Acceptability Judgments with the Italian CoLA corpus Daniela Trotta R. Guarasci Elisa Leonardelli Sara Tonelli 44 30 0 24 Sep 2021
K-AID: Enhancing Pre-trained Language Models with Domain Knowledge for Question Answering Fu Sun Feng-Lin Li Ruize Wang Qianglong Chen Xingyi Cheng Ji Zhang VLM KELM 41 4 0 22 Sep 2021
AliMe MKG: A Multi-modal Knowledge Graph for Live-streaming E-commerce Guohai Xu Hehong Chen Feng-Lin Li Fu Sun Yunzhou Shi Zhixiong Zeng Wei Zhou Zhongzhou Zhao Ji Zhang 24 16 0 13 Sep 2021
Frustratingly Simple Pretraining Alternatives to Masked Language Modeling Atsuki Yamaguchi G. Chrysostomou Katerina Margatina Nikolaos Aletras 27 25 0 04 Sep 2021
How to Query Language Models? Leonard Adolphs S. Dhuliawala Thomas Hofmann KELM 24 15 0 04 Aug 2021
Memorization in Deep Neural Networks: Does the Loss Function matter? Deep Patel P. Sastry TDI 27 8 0 21 Jul 2021
HerBERT: Efficiently Pretrained Transformer-based Language Model for Polish Robert Mroczkowski Piotr Rybak Alina Wróblewska Ireneusz Gawlik 36 81 0 04 May 2021
Syntactic Perturbations Reveal Representational Correlates of Hierarchical Phrase Structure in Pretrained Language Models Matteo Alleman J. Mamou Miguel Rio Hanlin Tang Yoon Kim SueYeon Chung NAI 38 17 0 15 Apr 2021
SemVLP: Vision-Language Pre-training by Aligning Semantics at Multiple Levels Chenliang Li Ming Yan Haiyang Xu Fuli Luo Wei Wang Bin Bi Songfang Huang VLM 34 36 0 14 Mar 2021
COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretraining Yu Meng Chenyan Xiong Payal Bajaj Saurabh Tiwary Paul N. Bennett Jiawei Han Xia Song 125 203 0 16 Feb 2021
Out of Order: How Important Is The Sequential Order of Words in a Sentence in Natural Language Understanding Tasks? Thang M. Pham Trung Bui Long Mai Anh Totti Nguyen 220 122 0 30 Dec 2020
Progressively Stacking 2.0: A Multi-stage Layerwise Training Method for BERT Training Speedup Cheng Yang Shengnan Wang Chao Yang Yuechuan Li Ru He Jingqiao Zhang 32 25 0 27 Nov 2020
A Comprehensive Survey on Word Representation Models: From Classical to State-Of-The-Art Word Representation Language Models Usman Naseem Imran Razzak S. Khan M. Prasad 14 156 0 28 Oct 2020
CAPT: Contrastive Pre-Training for Learning Denoised Sequence Representations Fuli Luo Pengcheng Yang Shicheng Li Xuancheng Ren Xu Sun VLM SSL 21 16 0 13 Oct 2020
On Losses for Modern Language Models Stephane Aroca-Ouellette Frank Rudzicz 22 32 0 04 Oct 2020
DeBERTa: Decoding-enhanced BERT with Disentangled Attention Pengcheng He Xiaodong Liu Jianfeng Gao Weizhu Chen AAML 64 2,631 0 05 Jun 2020
Syntactic Structure Distillation Pretraining For Bidirectional Encoders A. Kuncoro Lingpeng Kong Daniel Fried Dani Yogatama Laura Rimell Chris Dyer Phil Blunsom 31 33 0 27 May 2020
Pre-trained Models for Natural Language Processing: A Survey Xipeng Qiu Tianxiang Sun Yige Xu Yunfan Shao Ning Dai Xuanjing Huang LM&MA VLM 246 1,454 0 18 Mar 2020
A Survey on Contextual Embeddings Qi Liu Matt J. Kusner Phil Blunsom 225 146 0 16 Mar 2020
Sentence Meta-Embeddings for Unsupervised Semantic Textual Similarity Nina Poerner Ulli Waltinger Hinrich Schütze AI4TS 27 20 0 09 Nov 2019
E-BERT: Efficient-Yet-Effective Entity Embeddings for BERT Nina Poerner Ulli Waltinger Hinrich Schütze 15 156 0 09 Nov 2019
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 132 19,529 0 23 Oct 2019
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations Zhenzhong Lan Mingda Chen Sebastian Goodman Kevin Gimpel Piyush Sharma Radu Soricut SSL AIMat 112 6,380 0 26 Sep 2019
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 304 6,996 0 20 Apr 2018
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation Yonghui Wu M. Schuster Zhehuai Chen Quoc V. Le Mohammad Norouzi ... Alex Rudnick Oriol Vinyals G. Corrado Macduff Hughes J. Dean AIMat 718 6,750 0 26 Sep 2016