ResearchTrend.AI
  • Papers
  • Communities
  • Events
  • Blog
  • Pricing
Papers
Communities
Social Events
Terms and Conditions
Pricing
Parameter LabParameter LabTwitterGitHubLinkedInBlueskyYoutube

© 2025 ResearchTrend.AI, All rights reserved.

  1. Home
  2. Papers
  3. 2002.12804
  4. Cited By
UniLMv2: Pseudo-Masked Language Models for Unified Language Model
  Pre-Training

UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training

28 February 2020
Hangbo Bao
Li Dong
Furu Wei
Wenhui Wang
Nan Yang
Xiaodong Liu
Yu-Chiang Frank Wang
Songhao Piao
Jianfeng Gao
Ming Zhou
H. Hon
    AI4CE
ArXivPDFHTML

Papers citing "UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training"

50 / 219 papers shown
Title
Exploring Phonetic Context-Aware Lip-Sync For Talking Face Generation
Exploring Phonetic Context-Aware Lip-Sync For Talking Face Generation
Se Jin Park
Minsu Kim
J. Choi
Y. Ro
CVBM
29
4
0
31 May 2023
LayoutMask: Enhance Text-Layout Interaction in Multi-modal Pre-training
  for Document Understanding
LayoutMask: Enhance Text-Layout Interaction in Multi-modal Pre-training for Document Understanding
Yi Tu
Ya Guo
Huan Chen
Jinyang Tang
31
15
0
30 May 2023
Diagnosing Transformers: Illuminating Feature Spaces for Clinical
  Decision-Making
Diagnosing Transformers: Illuminating Feature Spaces for Clinical Decision-Making
Aliyah R. Hsu
Yeshwanth Cherapanamjeri
Briton Park
Tristan Naumann
A. Odisho
Bin-Xia Yu
MedIm
29
0
0
27 May 2023
Neural Summarization of Electronic Health Records
Neural Summarization of Electronic Health Records
Koyena Pal
Seyed Ali Bahrainian
Laura Y. Mercurio
Carsten Eickhoff
25
3
0
24 May 2023
Segmented Recurrent Transformer: An Efficient Sequence-to-Sequence Model
Segmented Recurrent Transformer: An Efficient Sequence-to-Sequence Model
Yinghan Long
Sayeed Shafayet Chowdhury
Kaushik Roy
40
1
0
24 May 2023
When Giant Language Brains Just Aren't Enough! Domain Pizzazz with
  Knowledge Sparkle Dust
When Giant Language Brains Just Aren't Enough! Domain Pizzazz with Knowledge Sparkle Dust
Minh Le Nguyen
Duy-Hung Nguyen
Shahab Sabahi
Hung Le
Jeffrey Yang
Hajime Hotta
33
1
0
12 May 2023
FormNetV2: Multimodal Graph Contrastive Learning for Form Document
  Information Extraction
FormNetV2: Multimodal Graph Contrastive Learning for Form Document Information Extraction
Nils Loose
Chun-Liang Li
Hao Zhang
Timothy Dozat
Felix Mächtle
...
Shangbang Long
Siyang Qin
Yasuhisa Fujii
Nan Hua
T. Eisenbarth
SSL
48
17
0
04 May 2023
Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond
Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond
Jingfeng Yang
Hongye Jin
Ruixiang Tang
Xiaotian Han
Qizhang Feng
Haoming Jiang
Bing Yin
Xia Hu
LM&MA
139
626
0
26 Apr 2023
A Cheaper and Better Diffusion Language Model with Soft-Masked Noise
A Cheaper and Better Diffusion Language Model with Soft-Masked Noise
Jiaao Chen
Aston Zhang
Mu Li
Alexander J. Smola
Diyi Yang
DiffM
32
17
0
10 Apr 2023
WebBrain: Learning to Generate Factually Correct Articles for Queries by
  Grounding on Large Web Corpus
WebBrain: Learning to Generate Factually Correct Articles for Queries by Grounding on Large Web Corpus
Hongjing Qian
Yutao Zhu
Zhicheng Dou
Haoqi Gu
Xinyu Zhang
Zheng Liu
Ruofei Lai
Bo Zhao
J. Nie
Ji-Rong Wen
38
25
0
10 Apr 2023
Using Semantic Similarity and Text Embedding to Measure the Social Media
  Echo of Strategic Communications
Using Semantic Similarity and Text Embedding to Measure the Social Media Echo of Strategic Communications
Tristan J. B. Cann
Ben Dennes
Travis G. Coan
S. OÑeill
Hywel T. P. Williams
11
0
0
29 Mar 2023
CoBIT: A Contrastive Bi-directional Image-Text Generation Model
CoBIT: A Contrastive Bi-directional Image-Text Generation Model
Haoxuan You
Mandy Guo
Zhecan Wang
Kai-Wei Chang
Jason Baldridge
Jiahui Yu
DiffM
49
13
0
23 Mar 2023
Traj-MAE: Masked Autoencoders for Trajectory Prediction
Traj-MAE: Masked Autoencoders for Trajectory Prediction
Hao Chen
Jiaze Wang
Kun Shao
Furui Liu
Jianye Hao
Chenyong Guan
Guangyong Chen
Pheng-Ann Heng
72
38
0
12 Mar 2023
Learning Language Representations with Logical Inductive Bias
Learning Language Representations with Logical Inductive Bias
Jianshu Chen
NAI
AI4CE
LRM
34
2
0
19 Feb 2023
Representation Deficiency in Masked Language Modeling
Representation Deficiency in Masked Language Modeling
Yu Meng
Jitin Krishnan
Sinong Wang
Qifan Wang
Yuning Mao
Han Fang
Marjan Ghazvininejad
Jiawei Han
Luke Zettlemoyer
90
7
0
04 Feb 2023
Parameter-Efficient Fine-Tuning Design Spaces
Parameter-Efficient Fine-Tuning Design Spaces
Jiaao Chen
Aston Zhang
Xingjian Shi
Mu Li
Alexander J. Smola
Diyi Yang
42
59
0
04 Jan 2023
A comprehensive review of automatic text summarization techniques:
  method, data, evaluation and coding
A comprehensive review of automatic text summarization techniques: method, data, evaluation and coding
D. Cajueiro
A. G. Nery
Igor Tavares
Maísa Kely de Melo
Silvia A. dos Reis
Weigang Li
V. R. R. Celestino
33
15
0
04 Jan 2023
Local Learning on Transformers via Feature Reconstruction
Local Learning on Transformers via Feature Reconstruction
P. Pathak
Jingwei Zhang
Dimitris Samaras
ViT
24
5
0
29 Dec 2022
Cramming: Training a Language Model on a Single GPU in One Day
Cramming: Training a Language Model on a Single GPU in One Day
Jonas Geiping
Tom Goldstein
MoE
30
85
0
28 Dec 2022
Pre-trained Language Models for Keyphrase Generation: A Thorough
  Empirical Study
Pre-trained Language Models for Keyphrase Generation: A Thorough Empirical Study
Di Wu
Wasi Uddin Ahmad
Kai-Wei Chang
29
17
0
20 Dec 2022
GanLM: Encoder-Decoder Pre-training with an Auxiliary Discriminator
GanLM: Encoder-Decoder Pre-training with an Auxiliary Discriminator
Jian Yang
Shuming Ma
Li Dong
Shaohan Huang
Haoyang Huang
Yuwei Yin
Dongdong Zhang
Liqun Yang
Furu Wei
Zhoujun Li
SyDa
AI4CE
32
25
0
20 Dec 2022
Wukong-Reader: Multi-modal Pre-training for Fine-grained Visual Document
  Understanding
Wukong-Reader: Multi-modal Pre-training for Fine-grained Visual Document Understanding
Haoli Bai
Zhiguang Liu
Xiaojun Meng
Wentao Li
Shuangning Liu
...
Liangwei Wang
Lu Hou
Jiansheng Wei
Xin Jiang
Qun Liu
ViT
35
11
0
19 Dec 2022
Multimodal Vision Transformers with Forced Attention for Behavior
  Analysis
Multimodal Vision Transformers with Forced Attention for Behavior Analysis
Tanay Agrawal
Michal Balazia
Philippe Muller
Franccois Brémond
ViT
23
9
0
07 Dec 2022
Protein Language Models and Structure Prediction: Connection and
  Progression
Protein Language Models and Structure Prediction: Connection and Progression
Bozhen Hu
Jun Xia
Jiangbin Zheng
Cheng Tan
Yufei Huang
Yongjie Xu
Stan Z. Li
27
40
0
30 Nov 2022
Unified Multimodal Model with Unlikelihood Training for Visual Dialog
Unified Multimodal Model with Unlikelihood Training for Visual Dialog
Zihao Wang
Junli Wang
Changjun Jiang
MLLM
29
10
0
23 Nov 2022
Proactive Detractor Detection Framework Based on Message-Wise Sentiment
  Analysis Over Customer Support Interactions
Proactive Detractor Detection Framework Based on Message-Wise Sentiment Analysis Over Customer Support Interactions
J. S. Gallo
Jesus Solano
Javier A. García
David Zarruk-Valencia
Alejandro Correa-Bahnsen
19
1
0
08 Nov 2022
ViT-LSLA: Vision Transformer with Light Self-Limited-Attention
ViT-LSLA: Vision Transformer with Light Self-Limited-Attention
Zhenzhe Hechen
Wei Huang
Yixin Zhao
ViT
38
6
0
31 Oct 2022
$N$-gram Is Back: Residual Learning of Neural Text Generation with
  $n$-gram Language Model
NNN-gram Is Back: Residual Learning of Neural Text Generation with nnn-gram Language Model
Huayang Li
Deng Cai
J. Xu
Taro Watanabe
VLM
37
1
0
26 Oct 2022
Towards Better Few-Shot and Finetuning Performance with Forgetful Causal
  Language Models
Towards Better Few-Shot and Finetuning Performance with Forgetful Causal Language Models
Hao Liu
Xinyang Geng
Lisa Lee
Igor Mordatch
Sergey Levine
Sharan Narang
Pieter Abbeel
KELM
CLL
35
2
0
24 Oct 2022
PATS: Sensitivity-aware Noisy Learning for Pretrained Language Models
PATS: Sensitivity-aware Noisy Learning for Pretrained Language Models
Yupeng Zhang
Hongzhi Zhang
Sirui Wang
Wei Wu
Zhoujun Li
AAML
37
1
0
22 Oct 2022
S2WAT: Image Style Transfer via Hierarchical Vision Transformer using
  Strips Window Attention
S2WAT: Image Style Transfer via Hierarchical Vision Transformer using Strips Window Attention
Chi Zhang
Lu Zhou
Lei Wang
Zaiyan Dai
Jun Yang
ViT
34
23
0
22 Oct 2022
Metric-guided Distillation: Distilling Knowledge from the Metric to
  Ranker and Retriever for Generative Commonsense Reasoning
Metric-guided Distillation: Distilling Knowledge from the Metric to Ranker and Retriever for Generative Commonsense Reasoning
Xingwei He
Yeyun Gong
Alex Jin
Weizhen Qi
Hang Zhang
Jian Jiao
Bartuer Zhou
Biao Cheng
Sm Yiu
Nan Duan
38
11
0
21 Oct 2022
A Unified View of Masked Image Modeling
A Unified View of Masked Image Modeling
Zhiliang Peng
Li Dong
Hangbo Bao
QiXiang Ye
Furu Wei
VLM
56
35
0
19 Oct 2022
ERNIE-Layout: Layout Knowledge Enhanced Pre-training for Visually-rich
  Document Understanding
ERNIE-Layout: Layout Knowledge Enhanced Pre-training for Visually-rich Document Understanding
Qiming Peng
Yinxu Pan
Wenjin Wang
Bin Luo
Zhenyu Zhang
...
Shi Feng
Yu Sun
Hao Tian
Hua Wu
Haifeng Wang
13
83
0
12 Oct 2022
Generative Language Models for Paragraph-Level Question Generation
Generative Language Models for Paragraph-Level Question Generation
Asahi Ushio
Fernando Alva-Manchego
Jose Camacho-Collados
ELM
13
45
0
08 Oct 2022
XDoc: Unified Pre-training for Cross-Format Document Understanding
XDoc: Unified Pre-training for Cross-Format Document Understanding
Jingye Chen
Tengchao Lv
Lei Cui
Changrong Zhang
Furu Wei
50
13
0
06 Oct 2022
Calibrating Sequence likelihood Improves Conditional Language Generation
Calibrating Sequence likelihood Improves Conditional Language Generation
Yao-Min Zhao
Misha Khalman
Rishabh Joshi
Shashi Narayan
Mohammad Saleh
Peter J. Liu
UQLM
31
119
0
30 Sep 2022
Multiple-Choice Question Generation: Towards an Automated Assessment
  Framework
Multiple-Choice Question Generation: Towards an Automated Assessment Framework
Vatsal Raina
Mark Gales
AI4Ed
ELM
34
32
0
23 Sep 2022
Selecting Better Samples from Pre-trained LLMs: A Case Study on Question
  Generation
Selecting Better Samples from Pre-trained LLMs: A Case Study on Question Generation
Xingdi Yuan
Tong Wang
Yen-Hsiang Wang
Emery Fine
Rania Abdelghani
Pauline Lucas
Hélene Sauzéon
Pierre-Yves Oudeyer
30
29
0
22 Sep 2022
Understanding the Tricks of Deep Learning in Medical Image Segmentation:
  Challenges and Future Directions
Understanding the Tricks of Deep Learning in Medical Image Segmentation: Challenges and Future Directions
Dong-Ming Zhang
Yi-Mou Lin
Hao Chen
Zhuotao Tian
Xin Yang
Jinhui Tang
Kwang-Ting Cheng
VLM
35
11
0
21 Sep 2022
ERNIE-mmLayout: Multi-grained MultiModal Transformer for Document
  Understanding
ERNIE-mmLayout: Multi-grained MultiModal Transformer for Document Understanding
Wenjin Wang
Zhengjie Huang
Bin Luo
Qianglong Chen
Qiming Peng
...
Weichong Yin
Shi Feng
Yu Sun
Dianhai Yu
Yin Zhang
ViT
35
11
0
18 Sep 2022
Enhancing Pre-trained Models with Text Structure Knowledge for Question
  Generation
Enhancing Pre-trained Models with Text Structure Knowledge for Question Generation
Zichen Wu
Xin Jia
Fanyi Qu
Yunfang Wu
21
4
0
09 Sep 2022
TransPolymer: a Transformer-based language model for polymer property
  predictions
TransPolymer: a Transformer-based language model for polymer property predictions
Changwen Xu
Yuyang Wang
A. Farimani
27
86
0
03 Sep 2022
Learning Better Masking for Better Language Model Pre-training
Learning Better Masking for Better Language Model Pre-training
Dongjie Yang
Zhuosheng Zhang
Hai Zhao
37
15
0
23 Aug 2022
Image as a Foreign Language: BEiT Pretraining for All Vision and
  Vision-Language Tasks
Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Tasks
Wenhui Wang
Hangbo Bao
Li Dong
Johan Bjorck
Zhiliang Peng
...
Kriti Aggarwal
O. Mohammed
Saksham Singhal
Subhojit Som
Furu Wei
MLLM
VLM
ViT
54
629
0
22 Aug 2022
BEiT v2: Masked Image Modeling with Vector-Quantized Visual Tokenizers
BEiT v2: Masked Image Modeling with Vector-Quantized Visual Tokenizers
Zhiliang Peng
Li Dong
Hangbo Bao
QiXiang Ye
Furu Wei
29
306
0
12 Aug 2022
DeepGen: Diverse Search Ad Generation and Real-Time Customization
DeepGen: Diverse Search Ad Generation and Real-Time Customization
Konstantin Golobokov
Junyi Chai
Victor Ye Dong
Mandy Gu
Bingyu Chi
Jie Cao
Yulan Yan
Yi Liu
31
8
0
06 Aug 2022
AlexaTM 20B: Few-Shot Learning Using a Large-Scale Multilingual Seq2Seq
  Model
AlexaTM 20B: Few-Shot Learning Using a Large-Scale Multilingual Seq2Seq Model
Saleh Soltan
Shankar Ananthakrishnan
Jack G. M. FitzGerald
Rahul Gupta
Wael Hamza
...
Mukund Sridhar
Fabian Triefenbach
Apurv Verma
Gokhan Tur
Premkumar Natarajan
56
82
0
02 Aug 2022
MAR: Masked Autoencoders for Efficient Action Recognition
MAR: Masked Autoencoders for Efficient Action Recognition
Zhiwu Qing
Shiwei Zhang
Ziyuan Huang
Xiang Wang
Yuehuang Wang
Yiliang Lv
Changxin Gao
Nong Sang
32
42
0
24 Jul 2022
PanGu-Coder: Program Synthesis with Function-Level Language Modeling
PanGu-Coder: Program Synthesis with Function-Level Language Modeling
Fenia Christopoulou
Gerasimos Lampouras
Milan Gritta
Guchun Zhang
Yinpeng Guo
...
Guangtai Liang
Jia Wei
Xin Jiang
Qianxiang Wang
Qun Liu
ELM
SyDa
ALM
45
74
0
22 Jul 2022
Previous
12345
Next