Title
CPT: A Pre-Trained Unbalanced Transformer for Both Chinese Language Understanding and Generation Yunfan Shao Zhichao Geng Yitao Liu Junqi Dai Hang Yan Fei Yang Li Zhe Hujun Bao Xipeng Qiu MedIm 70 148 0 13 Sep 2021
NSP-BERT: A Prompt-based Few-Shot Learner Through an Original Pre-training Task--Next Sentence Prediction Yi Sun Yu Zheng Chao Hao Hangping Qiu VLM 41 37 0 08 Sep 2021
DialogLM: Pre-trained Model for Long Dialogue Understanding and Summarization Ming Zhong Yang Liu Yichong Xu Chenguang Zhu Michael Zeng VLM AI4CE 46 125 0 06 Sep 2021
FewshotQA: A simple framework for few-shot learning of question answering tasks using pre-trained text-to-text models Rakesh Chada P. Natarajan 36 45 0 04 Sep 2021
FedKD: Communication Efficient Federated Learning via Knowledge Distillation Chuhan Wu Fangzhao Wu Lingjuan Lyu Yongfeng Huang Xing Xie FedML 27 373 0 30 Aug 2021
Differentiable Prompt Makes Pre-trained Language Models Better Few-shot Learners Ningyu Zhang Luoqiu Li Xiang Chen Shumin Deng Zhen Bi Chuanqi Tan Fei Huang Huajun Chen VLM 36 171 0 30 Aug 2021
Generating Answer Candidates for Quizzes and Answer-Aware Question Generators Kristiyan Vachev Momchil Hardalov Georgi Karadzhov Georgi Georgiev Ivan Koychev Preslav Nakov AI4Ed 31 5 0 29 Aug 2021
Fastformer: Additive Attention Can Be All You Need Chuhan Wu Fangzhao Wu Tao Qi Yongfeng Huang Xing Xie 46 117 0 20 Aug 2021
Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing Pengfei Liu Weizhe Yuan Jinlan Fu Zhengbao Jiang Hiroaki Hayashi Graham Neubig VLM SyDa 68 3,853 0 28 Jul 2021
TAPEX: Table Pre-training via Learning a Neural SQL Executor Qian Liu Bei Chen Jiaqi Guo Morteza Ziyadi Zeqi Lin Weizhu Chen Jian-Guang Lou LMTD 39 259 0 16 Jul 2021
Rethinking Positional Encoding Jianqiao Zheng Sameera Ramasinghe Simon Lucey 27 51 0 06 Jul 2021
Improving Factual Consistency of Abstractive Summarization on Customer Feedback Yang Liu Yifei Sun Vincent Gao HILM 18 6 0 30 Jun 2021
XLM-E: Cross-lingual Language Model Pre-training via ELECTRA Zewen Chi Shaohan Huang Li Dong Shuming Ma Bo Zheng ... Payal Bajaj Xia Song Xian-Ling Mao Heyan Huang Furu Wei 53 118 0 30 Jun 2021
ChineseBERT: Chinese Pretraining Enhanced by Glyph and Pinyin Information Zijun Sun Xiaoya Li Xiaofei Sun Yuxian Meng Xiang Ao Qing He Fei Wu Jiwei Li SSeg 57 184 0 30 Jun 2021
SCARF: Self-Supervised Contrastive Learning using Random Feature Corruption Dara Bahri Heinrich Jiang Yi Tay Donald Metzler SSL 21 164 0 29 Jun 2021
Automatic Construction of Enterprise Knowledge Base Junyi Chai Yujie He H. Hashemi Bing Li Daraksha Parveen Ranganath Kondapally Wenjin Xu 24 5 0 29 Jun 2021
Learning to Sample Replacements for ELECTRA Pre-Training Y. Hao Li Dong Hangbo Bao Ke Xu Furu Wei MU 11 11 0 25 Jun 2021
Video Swin Transformer Ze Liu Jia Ning Yue Cao Yixuan Wei Zheng-Wei Zhang Stephen Lin Han Hu ViT 44 1,446 0 24 Jun 2021
Stable, Fast and Accurate: Kernelized Attention with Relative Positional Encoding Shengjie Luo Shanda Li Tianle Cai Di He Dinglan Peng Shuxin Zheng Guolin Ke Liwei Wang Tie-Yan Liu 35 50 0 23 Jun 2021
DocFormer: End-to-End Transformer for Document Understanding Srikar Appalaraju Bhavan A. Jasani Bhargava Urala Kota Yusheng Xie R. Manmatha ViT 41 271 0 22 Jun 2021
BARTScore: Evaluating Generated Text as Text Generation Weizhe Yuan Graham Neubig Pengfei Liu 50 809 0 22 Jun 2021
A Condense-then-Select Strategy for Text Summarization Hou Pong Chan Irwin King 17 13 0 19 Jun 2021
Enhancing Question Generation with Commonsense Knowledge Xin Jia Hao Wang D. Yin Yunfang Wu 27 6 0 19 Jun 2021
BEiT: BERT Pre-Training of Image Transformers Hangbo Bao Li Dong Songhao Piao Furu Wei ViT 68 2,749 0 15 Jun 2021
CAT: Cross Attention in Vision Transformer Hezheng Lin Xingyi Cheng Xiangyu Wu Fan Yang Dong Shen Zhongyuan Wang Qing Song Wei Yuan ViT 35 149 0 10 Jun 2021
VT-SSum: A Benchmark Dataset for Video Transcript Segmentation and Summarization Tengchao Lv Lei Cui M. Vasilijevic Furu Wei 44 7 0 10 Jun 2021
One Teacher is Enough? Pre-trained Language Model Distillation from Multiple Teachers Chuhan Wu Fangzhao Wu Yongfeng Huang 26 63 0 02 Jun 2021
HiddenCut: Simple Data Augmentation for Natural Language Understanding with Better Generalization Jiaao Chen Dinghan Shen Weizhu Chen Diyi Yang BDL 24 47 0 31 May 2021
Retrieval Enhanced Model for Commonsense Generation Han Wang Yang Liu Chenguang Zhu Linjun Shou Ming Gong Yichong Xu Michael Zeng RALM LRM 31 32 0 24 May 2021
DialogSum: A Real-Life Scenario Dialogue Summarization Dataset Yulong Chen Yang Liu Liang Chen Yue Zhang 42 224 0 14 May 2021
GraphFormers: GNN-nested Transformers for Representation Learning on Textual Graph Junhan Yang Zheng Liu Shitao Xiao Chaozhuo Li Defu Lian Sanjay Agrawal Amit Singh Guangzhong Sun Xing Xie AI4CE 11 148 0 06 May 2021
Knowledge Neurons in Pretrained Transformers Damai Dai Li Dong Y. Hao Zhifang Sui Baobao Chang Furu Wei KELM MU 28 418 0 18 Apr 2021
MT6: Multilingual Pretrained Text-to-Text Transformer with Translation Pairs Zewen Chi Li Dong Shuming Ma Shaohan Huang Xian-Ling Mao Heyan Huang Furu Wei LRM 53 72 0 18 Apr 2021
Multi-Scale Vision Longformer: A New Vision Transformer for High-Resolution Image Encoding Pengchuan Zhang Xiyang Dai Jianwei Yang Bin Xiao Lu Yuan Lei Zhang Jianfeng Gao ViT 29 330 0 29 Mar 2021
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows Ze Liu Yutong Lin Yue Cao Han Hu Yixuan Wei Zheng-Wei Zhang Stephen Lin B. Guo ViT 151 20,774 0 25 Mar 2021
An Approach to Improve Robustness of NLP Systems against ASR Errors Tong Cui Jinghui Xiao Liangyou Li Xin Jiang Qun Liu 27 11 0 25 Mar 2021
GLM: General Language Model Pretraining with Autoregressive Blank Infilling Zhengxiao Du Yujie Qian Xiao Liu Ming Ding J. Qiu Zhilin Yang Jie Tang BDL AI4CE 33 1,492 0 18 Mar 2021
Constrained Text Generation with Global Guidance -- Case Study on CommonGen Yixian Liu Liwen Zhang Wenjuan Han Yue Zhang Kewei Tu 36 9 0 12 Mar 2021
TAG: Gradient Attack on Transformer-based Language Models Jieren Deng Yijue Wang Ji Li Chao Shang Hang Liu Sanguthevar Rajasekaran Caiwen Ding FedML PILM 16 73 0 11 Mar 2021
M6: A Chinese Multimodal Pretrainer Junyang Lin Rui Men An Yang Chan Zhou Ming Ding ... Yong Li Wei Lin Jingren Zhou J. Tang Hongxia Yang VLM MoE 37 132 0 01 Mar 2021
Training Large-Scale News Recommenders with Pretrained Language Models in the Loop Shitao Xiao Zheng Liu Yingxia Shao Tao Di Xing Xie VLM AIFin 127 41 0 18 Feb 2021
Quiz-Style Question Generation for News Stories Á. Lelkes Vinh Q. Tran Cong Yu 35 42 0 18 Feb 2021
COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretraining Yu Meng Chenyan Xiong Payal Bajaj Saurabh Tiwary Paul N. Bennett Jiawei Han Xia Song 125 203 0 16 Feb 2021
NewsBERT: Distilling Pre-trained Language Model for Intelligent News Application Chuhan Wu Fangzhao Wu Yang Yu Tao Qi Yongfeng Huang Qi Liu VLM 11 44 0 09 Feb 2021
Improving Sequence-to-Sequence Pre-training via Sequence Span Rewriting Wangchunshu Zhou Tao Ge Canwen Xu Ke Xu Furu Wei LRM 16 15 0 02 Jan 2021
Intent Classification and Slot Filling for Privacy Policies Wasi Uddin Ahmad Jianfeng Chi Tu Le Thomas B. Norton Yuan Tian Kai-Wei Chang 18 23 0 01 Jan 2021
MiniLMv2: Multi-Head Self-Attention Relation Distillation for Compressing Pretrained Transformers Wenhui Wang Hangbo Bao Shaohan Huang Li Dong Furu Wei MQ 24 257 0 31 Dec 2020
XLM-T: Scaling up Multilingual Machine Translation with Pretrained Cross-lingual Transformer Encoders Shuming Ma Jian Yang Haoyang Huang Zewen Chi Li Dong ... Akiko Eriguchi Saksham Singhal Xia Song Arul Menezes Furu Wei LRM 26 33 0 31 Dec 2020
Generating Query Focused Summaries from Query-Free Resources Yumo Xu Mirella Lapata AI4TS 15 34 0 29 Dec 2020
LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding Yang Xu Yiheng Xu Tengchao Lv Lei Cui Furu Wei ... D. Florêncio Cha Zhang Wanxiang Che Min Zhang Lidong Zhou ViT MLLM 153 501 0 29 Dec 2020