XLNet: Generalized Autoregressive Pretraining for Language Understanding

19 June 2019

Papers citing "XLNet: Generalized Autoregressive Pretraining for Language Understanding"

50 / 1,487 papers shown

Title
Understanding and Overcoming the Challenges of Efficient Transformer Quantization Yelysei Bondarenko Markus Nagel Tijmen Blankevoort MQ 25 133 0 27 Sep 2021
Multiplicative Position-aware Transformer Models for Language Understanding Zhiheng Huang Davis Liang Peng Xu Bing Xiang 17 1 0 27 Sep 2021
Improving Question Answering Performance Using Knowledge Distillation and Active Learning Yasaman Boreshban Seyed Morteza Mirbostani Gholamreza Ghassem-Sani Seyed Abolghasem Mirroshandel Shahin Amiriparian 34 15 0 26 Sep 2021
Parallel Refinements for Lexically Constrained Text Generation with BART Xingwei He 34 39 0 26 Sep 2021
DziriBERT: a Pre-trained Language Model for the Algerian Dialect Amine Abdaoui Mohamed Berrimi Mourad Oussalah A. Moussaoui 37 43 0 25 Sep 2021
More Than Reading Comprehension: A Survey on Datasets and Metrics of Textual Question Answering Yang Bai D. Wang 96 10 0 25 Sep 2021
Pushing on Text Readability Assessment: A Transformer Meets Handcrafted Linguistic Features Bruce W. Lee Yoonna Jang J. Lee VLM 48 75 0 25 Sep 2021
Monolingual and Cross-Lingual Acceptability Judgments with the Italian CoLA corpus Daniela Trotta R. Guarasci Elisa Leonardelli Sara Tonelli 47 30 0 24 Sep 2021
Conditional Poisson Stochastic Beam Search Clara Meister Afra Amini Tim Vieira Ryan Cotterell 37 10 0 22 Sep 2021
Small-Bench NLP: Benchmark for small single GPU trained models in Natural Language Processing K. Kanakarajan Bhuvana Kundumani Malaikannan Sankarasubbu ALM MoE 19 5 0 22 Sep 2021
MiRANews: Dataset and Benchmarks for Multi-Resource-Assisted News Summarization Xinnuo Xu Ondrej Dusek Shashi Narayan Verena Rieser Ioannis Konstas HILM 28 6 0 22 Sep 2021
Digital Signal Processing Using Deep Neural Networks Brian Shevitski Y. Watkins Nicole Man Michael Girard AI4CE 33 4 0 21 Sep 2021
AutoGCL: Automated Graph Contrastive Learning via Learnable View Generators Yihang Yin Qingzhong Wang Siyu Huang Haoyi Xiong Xiang Zhang 56 145 0 21 Sep 2021
RAIL-KD: RAndom Intermediate Layer Mapping for Knowledge Distillation Md. Akmal Haidar Nithin Anchuri Mehdi Rezagholizadeh Abbas Ghaddar Philippe Langlais Pascal Poupart 33 22 0 21 Sep 2021
BERT Has Uncommon Sense: Similarity Ranking for Word Sense BERTology Luke Gessler Nathan Schneider 38 7 0 20 Sep 2021
DisCoDisCo at the DISRPT2021 Shared Task: A System for Discourse Segmentation, Classification, and Connective Detection Luke Gessler Shabnam Behzad Yang Liu Siyao Peng Yilun Zhu Amir Zeldes 40 32 0 20 Sep 2021
Towards Zero-Label Language Learning Zirui Wang Adams Wei Yu Orhan Firat Yuan Cao SyDa 192 102 0 19 Sep 2021
Primer: Searching for Efficient Transformers for Language Modeling David R. So Wojciech Mañke Hanxiao Liu Zihang Dai Noam M. Shazeer Quoc V. Le VLM 91 154 0 17 Sep 2021
MeLT: Message-Level Transformer with Masked Document Representations as Pre-Training for Stance Detection Matthew Matero Nikita Soni Niranjan Balasubramanian H. Andrew Schwartz 43 21 0 16 Sep 2021
Language Models are Few-shot Multilingual Learners Genta Indra Winata Andrea Madotto Zhaojiang Lin Rosanne Liu J. Yosinski Pascale Fung ELM LRM 38 133 0 16 Sep 2021
Unsupervised Keyphrase Extraction by Jointly Modeling Local and Global Context Xinnian Liang Shuangzhi Wu Mu Li Zhoujun Li 60 61 0 15 Sep 2021
EfficientBERT: Progressively Searching Multilayer Perceptron via Warm-up Knowledge Distillation Chenhe Dong Guangrun Wang Hang Xu Jiefeng Peng Xiaozhe Ren Xiaodan Liang 26 28 0 15 Sep 2021
Semantic Answer Type Prediction using BERT: IAI at the ISWC SMART Task 2020 Vinay Setty K. Balog 29 12 0 14 Sep 2021
KFCNet: Knowledge Filtering and Contrastive Learning Network for Generative Commonsense Reasoning Haonan Li Yeyun Gong Jian Jiao Ruofei Zhang Timothy Baldwin Nan Duan OffRL 60 6 0 14 Sep 2021
Different Strokes for Different Folks: Investigating Appropriate Further Pre-training Approaches for Diverse Dialogue Tasks Yao Qiu Jinchao Zhang Jie Zhou 25 5 0 14 Sep 2021
STraTA: Self-Training with Task Augmentation for Better Few-shot Learning Tu Vu Minh-Thang Luong Quoc V. Le Grady Simon Mohit Iyyer 131 61 0 13 Sep 2021
KroneckerBERT: Learning Kronecker Decomposition for Pre-trained Language Models via Knowledge Distillation Marzieh S. Tahaei Ella Charlaix V. Nia A. Ghodsi Mehdi Rezagholizadeh 46 22 0 13 Sep 2021
Not All Models Localize Linguistic Knowledge in the Same Place: A Layer-wise Probing on BERToids' Representations Mohsen Fayyaz Ehsan Aghazadeh Ali Modarressi Hosein Mohebbi Mohammad Taher Pilehvar 18 21 0 13 Sep 2021
Question Answering over Electronic Devices: A New Benchmark Dataset and a Multi-Task Learning based QA Framework Abhilash Nandy Soumya Sharma Shubham Maddhashiya K. Sachdeva Pawan Goyal Niloy Ganguly 30 17 0 13 Sep 2021
How to Select One Among All? An Extensive Empirical Study Towards the Robustness of Knowledge Distillation in Natural Language Understanding Tianda Li Ahmad Rashid A. Jafari Pranav Sharma A. Ghodsi Mehdi Rezagholizadeh AAML 38 5 0 13 Sep 2021
Raise a Child in Large Language Model: Towards Effective and Generalizable Fine-tuning Runxin Xu Fuli Luo Zhiyuan Zhang Chuanqi Tan Baobao Chang Songfang Huang Fei Huang LRM 151 178 0 13 Sep 2021
Asking Questions Like Educational Experts: Automatically Generating Question-Answer Pairs on Real-World Examination Data Fanyi Qu Xin Jia Hao Sun AI4Ed 93 24 0 11 Sep 2021
Does Pretraining for Summarization Require Knowledge Transfer? Kundan Krishna Jeffrey P. Bigham Zachary Chase Lipton 32 36 0 10 Sep 2021
Counterfactual Adversarial Learning with Representation Interpolation Wen Wang Wei Ping Ning Shi Jinfeng Li Bingyu Zhu Xiangyu Liu Rongxin Zhang AAML OOD CML 26 2 0 10 Sep 2021
EfficientCLIP: Efficient Cross-Modal Pre-training by Ensemble Confident Learning and Language Modeling Jue Wang Haofan Wang Jincan Deng Weijia Wu Debing Zhang VLM CLIP 67 18 0 10 Sep 2021
Augmenting BERT-style Models with Predictive Coding to Improve Discourse-level Representations Vladimir Araujo Andrés Villa Marcelo Mendoza Marie-Francine Moens Alvaro Soto 32 7 0 10 Sep 2021
BERT, mBERT, or BiBERT? A Study on Contextualized Embeddings for Neural Machine Translation Haoran Xu Benjamin Van Durme Kenton W. Murray 50 57 0 09 Sep 2021
MetaXT: Meta Cross-Task Transfer between Disparate Label Spaces Srinagesh Sharma Guoqing Zheng Ahmed Hassan Awadallah 27 1 0 09 Sep 2021
KELM: Knowledge Enhanced Pre-Trained Language Representations with Message Passing on Hierarchical Relational Graphs Yinquan Lu H. Lu Guirong Fu Qun Liu KELM 18 34 0 09 Sep 2021
Efficient Nearest Neighbor Language Models Junxian He Graham Neubig Taylor Berg-Kirkpatrick RALM 197 103 0 09 Sep 2021
Discrete and Soft Prompting for Multilingual Models Mengjie Zhao Hinrich Schütze LRM 18 71 0 08 Sep 2021
NSP-BERT: A Prompt-based Few-Shot Learner Through an Original Pre-training Task--Next Sentence Prediction Yi Sun Yu Zheng Chao Hao Hangping Qiu VLM 43 37 0 08 Sep 2021
Beyond Preserved Accuracy: Evaluating Loyalty and Robustness of BERT Compression Canwen Xu Wangchunshu Zhou Tao Ge Kelvin J. Xu Julian McAuley Furu Wei 21 41 0 07 Sep 2021
Sequential Attention Module for Natural Language Processing Mengyuan Zhou Jian Ma Haiqing Yang Lian-Xin Jiang Yang Mo AI4TS 27 2 0 07 Sep 2021
GPT-3 Models are Poor Few-Shot Learners in the Biomedical Domain M. Moradi Kathrin Blagec F. Haberl Matthias Samwald LM&MA AI4MH 42 65 0 06 Sep 2021
PermuteFormer: Efficient Relative Position Encoding for Long Sequences Peng-Jen Chen 36 21 0 06 Sep 2021
Frustratingly Simple Pretraining Alternatives to Masked Language Modeling Atsuki Yamaguchi G. Chrysostomou Katerina Margatina Nikolaos Aletras 27 25 0 04 Sep 2021
Hybrid Contrastive Learning of Tri-Modal Representation for Multimodal Sentiment Analysis Sijie Mai Ying Zeng Shuangjia Zheng Haifeng Hu 30 117 0 04 Sep 2021
CTAL: Pre-training Cross-modal Transformer for Audio-and-Language Representations Hang Li Yunxing Kang Tianqiao Liu Wenbiao Ding Zitao Liu 41 17 0 01 Sep 2021
Interactive Machine Comprehension with Dynamic Knowledge Graphs Xingdi Yuan 34 3 0 31 Aug 2021