XLNet: Generalized Autoregressive Pretraining for Language Understanding

19 June 2019

Papers citing "XLNet: Generalized Autoregressive Pretraining for Language Understanding"

50 / 1,408 papers shown

Title
GPS: Genetic Prompt Search for Efficient Few-shot Learning Hanwei Xu Yujun Chen Yulun Du Nan Shao Yanggang Wang Haiyu Li Zhilin Yang VLM 14 28 0 31 Oct 2022
Character-level White-Box Adversarial Attacks against Transformers via Attachable Subwords Substitution Aiwei Liu Honghai Yu Xuming Hu Shuang Li Li Lin Fukun Ma Yawen Yang Lijie Wen 36 33 0 31 Oct 2022
Parameter-Efficient Tuning Makes a Good Classification Head Zhuoyi Yang Ming Ding Yanhui Guo Qingsong Lv Jie Tang VLM 58 14 0 30 Oct 2022
token2vec: A Joint Self-Supervised Pre-training Framework Using Unpaired Speech and Text Xianghu Yue Junyi Ao Xiaoxue Gao Haizhou Li SSL 26 8 0 30 Oct 2022
Probing for targeted syntactic knowledge through grammatical error detection Christopher Davis Christopher Bryant Andrew Caines Marek Rei P. Buttery 22 3 0 28 Oct 2022
COST-EFF: Collaborative Optimization of Spatial and Temporal Efficiency with Slenderized Multi-exit Language Models Bowen Shen Zheng Lin Yuanxin Liu Zhengxiao Liu Lei Wang Weiping Wang VLM 47 4 0 27 Oct 2022
Dictionary-Assisted Supervised Contrastive Learning Patrick Y. Wu Richard Bonneau Joshua A. Tucker Jonathan Nagler CLIP 33 0 0 27 Oct 2022
Privately Fine-Tuning Large Language Models with Differential Privacy R. Behnia Mohammadreza Ebrahimi Jason L. Pacheco B. Padmanabhan 32 44 0 26 Oct 2022
Learning on Large-scale Text-attributed Graphs via Variational Inference Jianan Zhao Meng Qu Chaozhuo Li Hao Yan Qian Liu Rui Li Xing Xie Jian Tang VLM 37 133 0 26 Oct 2022
Uncertainty Sentence Sampling by Virtual Adversarial Perturbation Han Zhang Zhen Zhang Hongfei Jiang Yang Song 28 0 0 26 Oct 2022
Leveraging Affirmative Interpretations from Negation Improves Natural Language Understanding Md Mosharaf Hossain Eduardo Blanco 50 4 0 26 Oct 2022
Revision for Concision: A Constrained Paraphrase Generation Task Wenchuan Mu Kwanin Lim 32 3 0 25 Oct 2022
Same Pre-training Loss, Better Downstream: Implicit Bias Matters for Language Models Hong Liu Sang Michael Xie Zhiyuan Li Tengyu Ma AI4CE 40 49 0 25 Oct 2022
PolyHope: Two-Level Hope Speech Detection from Tweets F. Balouchzahi Grigori Sidorov Alexander Gelbukh 11 48 0 25 Oct 2022
Audio MFCC-gram Transformers for respiratory insufficiency detection in COVID-19 M. Gauy Marcelo Finger 24 7 0 25 Oct 2022
Towards Better Few-Shot and Finetuning Performance with Forgetful Causal Language Models Hao Liu Xinyang Geng Lisa Lee Igor Mordatch Sergey Levine Sharan Narang Pieter Abbeel KELM CLL 35 2 0 24 Oct 2022
ELMER: A Non-Autoregressive Pre-trained Language Model for Efficient and Effective Text Generation Junyi Li Tianyi Tang Wayne Xin Zhao J. Nie Ji-Rong Wen 22 17 0 24 Oct 2022
DiscoSense: Commonsense Reasoning with Discourse Connectives Prajjwal Bhargava Vincent Ng LRM 170 4 0 22 Oct 2022
P $^3$ LM: Probabilistically Permuted Prophet Language Modeling for Generative Pre-Training Junwei Bao Yifan Wang Jiangyong Ying Yeyun Gong Jing Zhao Youzheng Wu Xiaodong He 37 1 0 22 Oct 2022
TCAB: A Large-Scale Text Classification Attack Benchmark Kalyani Asthana Zhouhang Xie Wencong You Adam Noack Jonathan Brophy Sameer Singh Daniel Lowd 39 3 0 21 Oct 2022
Transcending Scaling Laws with 0.1% Extra Compute Yi Tay Jason W. Wei Hyung Won Chung Vinh Q. Tran David R. So ... Donald Metzler Slav Petrov N. Houlsby Quoc V. Le Mostafa Dehghani LRM 47 68 0 20 Oct 2022
Late Prompt Tuning: A Late Prompt Could Be Better Than Many Prompts Xiangyang Liu Tianxiang Sun Xuanjing Huang Xipeng Qiu VLM 36 27 0 20 Oct 2022
Attribution and Obfuscation of Neural Text Authorship: A Data Mining Perspective Adaku Uchendu Thai Le Dongwon Lee DeLMO 32 41 0 19 Oct 2022
ELASTIC: Numerical Reasoning with Adaptive Symbolic Compiler Jiaxin Zhang Yashar Moshfeghi AIMat 27 18 0 18 Oct 2022
Tiny-Attention Adapter: Contexts Are More Important Than the Number of Parameters Hongyu Zhao Hao Tan Hongyuan Mei MoE 39 16 0 18 Oct 2022
Controllable Fake Document Infilling for Cyber Deception Yibo Hu Yu Lin Eric Parolin Latif Khan Kevin W. Hamlen 35 8 0 18 Oct 2022
Sentiment-Aware Word and Sentence Level Pre-training for Sentiment Analysis Shuai Fan Chen Lin Haonan Li Zheng-Wen Lin Jinsong Su Hang Zhang Yeyun Gong Jian Guo Nan Duan VLM 36 18 0 18 Oct 2022
Modelling Emotion Dynamics in Song Lyrics with State Space Models Yingjin Song Daniel Beck 48 5 0 17 Oct 2022
Flipped Classroom: Effective Teaching for Time Series Forecasting P. Teutsch Patrick Mäder AI4TS 23 8 0 17 Oct 2022
BERTScore is Unfair: On Social Bias in Language Model-Based Metrics for Text Generation Tianxiang Sun Junliang He Xipeng Qiu Xuanjing Huang 24 44 0 14 Oct 2022
StyLEx: Explaining Style Using Human Lexical Annotations Shirley Anugrah Hayati Kyumin Park Dheeraj Rajagopal Lyle Ungar Dongyeop Kang 28 3 0 14 Oct 2022
Spontaneous Emerging Preference in Two-tower Language Model Zhengqi He Taro Toyoizumi LRM 21 1 0 13 Oct 2022
A context-aware knowledge transferring strategy for CTC-based ASR Keda Lu Kuan-Yu Chen 15 15 0 12 Oct 2022
Decoupled Context Processing for Context Augmented Language Modeling Zonglin Li Ruiqi Guo Surinder Kumar RALM KELM 21 23 0 11 Oct 2022
Enriching Biomedical Knowledge for Low-resource Language Through Large-Scale Translation Long Phan Tai Dang H. Tran Trieu H. Trinh Vy Phan Lam D. Chau Minh-Thang Luong 15 8 0 11 Oct 2022
Multi-CLS BERT: An Efficient Alternative to Traditional Ensembling Haw-Shiuan Chang Ruei-Yao Sun Kathryn Ricci Andrew McCallum 43 14 0 10 Oct 2022
Uncertainty Quantification with Pre-trained Language Models: A Large-Scale Empirical Analysis Yuxin Xiao Paul Pu Liang Umang Bhatt Willie Neiswanger Ruslan Salakhutdinov Louis-Philippe Morency 175 86 0 10 Oct 2022
Parameter-Efficient Tuning with Special Token Adaptation Xiaoocong Yang James Y. Huang Wenxuan Zhou Muhao Chen 34 12 0 10 Oct 2022
SpeechUT: Bridging Speech and Text with Hidden-Unit for Encoder-Decoder Based Speech-Text Pre-training Zi-Hua Zhang Long Zhou Junyi Ao Shujie Liu Lirong Dai Jinyu Li Furu Wei 61 57 0 07 Oct 2022
Debiasing isn't enough! -- On the Effectiveness of Debiasing MLMs and their Social Biases in Downstream Tasks Masahiro Kaneko Danushka Bollegala Naoaki Okazaki 28 41 0 06 Oct 2022
XDoc: Unified Pre-training for Cross-Format Document Understanding Jingye Chen Tengchao Lv Lei Cui Changrong Zhang Furu Wei 50 13 0 06 Oct 2022
Melody Infilling with User-Provided Structural Context Chih-Pin Tan A. Su Yi-Hsuan Yang 36 3 0 06 Oct 2022
Transformer-based conditional generative adversarial network for multivariate time series generation Abdellah Madane M. Dilmi Florent Forest Hanane Azzag M. Lebbah J. Lacaille 30 10 0 05 Oct 2022
Understanding Prior Bias and Choice Paralysis in Transformer-based Language Representation Models through Four Experimental Probes Ke Shen Mayank Kejriwal 27 4 0 03 Oct 2022
Neural Media Bias Detection Using Distant Supervision With BABE -- Bias Annotations By Experts Timo Spinde Manuel Plank Jan-David Krieger Terry Ruas Bela Gipp Akiko Aizawa 27 68 0 29 Sep 2022
Downstream Datasets Make Surprisingly Good Pretraining Corpora Kundan Krishna Saurabh Garg Jeffrey P. Bigham Zachary Chase Lipton 50 30 0 28 Sep 2022
TVLT: Textless Vision-Language Transformer Zineng Tang Jaemin Cho Yixin Nie Joey Tianyi Zhou VLM 51 28 0 28 Sep 2022
Outlier Suppression: Pushing the Limit of Low-bit Transformer Language Models Xiuying Wei Yunchen Zhang Xiangguo Zhang Ruihao Gong Shanghang Zhang Qi Zhang F. Yu Xianglong Liu MQ 38 145 0 27 Sep 2022
Deep Lake: a Lakehouse for Deep Learning S. Hambardzumyan Abhina Tuli Levon Ghukasyan Fariz Rahman Hrant Topchyan ... Mark McQuade M. Harutyunyan Tatevik Hakobyan I. Stranic Davit Buniatyan 26 17 0 22 Sep 2022
Knowledge-Aware Bayesian Deep Topic Model Dongsheng Wang Yishi Xu Miaoge Li Zhibin Duan Chaojie Wang Bo Chen Mingyuan Zhou BDL 38 15 0 20 Sep 2022