DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing

18 November 2021

Papers citing "DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing"

50 / 664 papers shown

Title
On Feature Learning in the Presence of Spurious Correlations Pavel Izmailov Polina Kirichenko Nate Gruver A. Wilson 36 118 0 20 Oct 2022
Attribution and Obfuscation of Neural Text Authorship: A Data Mining Perspective Adaku Uchendu Thai Le Dongwon Lee DeLMO 32 41 0 19 Oct 2022
Leveraging a New Spanish Corpus for Multilingual and Crosslingual Metaphor Detection Elisa Sanchez-Bayona Rodrigo Agerri 14 10 0 19 Oct 2022
MiQA: A Benchmark for Inference on Metaphorical Questions Iulia Comsa Julian Martin Eisenschlos S. Narayanan 27 8 0 14 Oct 2022
Expose Backdoors on the Way: A Feature-Based Efficient Defense against Textual Backdoor Attacks Sishuo Chen Wenkai Yang Zhiyuan Zhang Xiaohan Bi Xu Sun SILM AAML 37 23 0 14 Oct 2022
Assessing Out-of-Domain Language Model Performance from Few Examples Prasann Singhal Jarad Forristal Xi Ye Greg Durrett LRM 25 5 0 13 Oct 2022
Task Compass: Scaling Multi-task Pre-training with Task Prefix Zhuosheng Zhang Shuohang Wang Yichong Xu Yuwei Fang Wenhao Yu Yang Liu Han Zhao Chenguang Zhu Michael Zeng SSL LRM 25 16 0 12 Oct 2022
MiDe22: An Annotated Multi-Event Tweet Dataset for Misinformation Detection Cagri Toraman Oguzhan Ozcelik Furkan Şahinuç Fazli Can 38 12 0 11 Oct 2022
Short Text Pre-training with Extended Token Classification for E-commerce Query Understanding Haoming Jiang Tianyu Cao Zheng Li Cheng-hsin Luo Xianfeng Tang Qingyu Yin Danqing Zhang R. Goutam Bing Yin RALM 35 11 0 08 Oct 2022
Improving Large-scale Paraphrase Acquisition and Generation Yao Dou Chao Jiang Wei-ping Xu 50 9 0 06 Oct 2022
A Distributional Lens for Multi-Aspect Controllable Text Generation Yuxuan Gu Xiaocheng Feng Sicheng Ma Lingyuan Zhang Heng Gong Bing Qin 112 36 0 06 Oct 2022
Less is More: Task-aware Layer-wise Distillation for Language Model Compression Chen Liang Simiao Zuo Qingru Zhang Pengcheng He Weizhu Chen Tuo Zhao VLM 42 68 0 04 Oct 2022
PART: Pre-trained Authorship Representation Transformer Javier Huertas-Tato Álvaro Huertas-García Alejandro Martín 29 8 0 30 Sep 2022
Using contradictions improves question answering systems Étienne Fortier-Dubois Domenic Rosati 15 0 0 28 Sep 2022
Scope of Pre-trained Language Models for Detecting Conflicting Health Information Josepho D. Gatto Madhusudan Basak S. Preum 32 7 0 22 Sep 2022
Unsupervised Lexical Substitution with Decontextualised Embeddings Takashi Wada Timothy Baldwin Yuji Matsumoto Jey Han Lau 88 6 0 17 Sep 2022
Possible Stories: Evaluating Situated Commonsense Reasoning under Multiple Possible Scenarios Mana Ashida Saku Sugawara 65 6 0 16 Sep 2022
IDIAPers @ Causal News Corpus 2022: Efficient Causal Relation Identification Through a Prompt-based Few-shot Approach Sergio Burdisso Juan Pablo Zuluaga Esaú Villatoro-Tello Martin Fajcik Muskaan Singh Pavel Smrz P. Motlícek 41 3 0 08 Sep 2022
SynSciPass: detecting appropriate uses of scientific text generation Domenic Rosati DeLMO 53 17 0 07 Sep 2022
Efficient Methods for Natural Language Processing: A Survey Marcos Vinícius Treviso Ji-Ung Lee Tianchu Ji Betty van Aken Qingqing Cao ... Emma Strubell Niranjan Balasubramanian Leon Derczynski Iryna Gurevych Roy Schwartz 33 109 0 31 Aug 2022
Generating Intermediate Steps for NLI with Next-Step Supervision Deepanway Ghosal Somak Aditya Monojit Choudhury LRM 35 1 0 31 Aug 2022
Predicting Query-Item Relationship using Adversarial Training and Robust Modeling Techniques Min Seok Kim 22 0 0 23 Aug 2022
Z-Code++: A Pre-trained Language Model Optimized for Abstractive Summarization Pengcheng He Baolin Peng Liyang Lu Song Wang Jie Mei ... Chenguang Zhu Wayne Xiong Michael Zeng Jianfeng Gao Xuedong Huang 28 47 0 21 Aug 2022
MENLI: Robust Evaluation Metrics from Natural Language Inference Yanran Chen Steffen Eger 32 16 0 15 Aug 2022
Domain-Specific Text Generation for Machine Translation Yasmin Moslem Rejwanul Haque John D. Kelleher Andy Way 21 16 0 11 Aug 2022
giMLPs: Gate with Inhibition Mechanism in MLPs Cheng Kang Jindich Prokop Lei Tong Huiyu Zhou Yong Hu Daneil Novak 29 0 0 01 Aug 2022
Some Practice for Improving the Search Results of E-commerce Fanyou Wu Yang Liu R. Gazo Benes Bedrich Xiaobo Qu 24 3 0 30 Jul 2022
Claim-Dissector: An Interpretable Fact-Checking System with Joint Re-ranking and Veracity Prediction Martin Fajcik P. Motlícek Pavel Smrz 33 18 0 28 Jul 2022
A Cognitive Study on Semantic Similarity Analysis of Large Corpora: A Transformer-based Approach Praneeth Nemani Satyanarayana Vollala 14 0 0 24 Jul 2022
Language Modelling with Pixels Phillip Rust Jonas F. Lotz Emanuele Bugliarello Elizabeth Salesky Miryam de Lhoneux Desmond Elliott VLM 38 46 0 14 Jul 2022
Scene-Aware Prompt for Multi-modal Dialogue Understanding and Generation Bin Li Yixuan Weng Ziyu Ma Bin Sun Shutao Li VLM 13 2 0 05 Jul 2022
Understanding Performance of Long-Document Ranking Models through Comprehensive Evaluation and Leaderboarding Leonid Boytsov David Akinpelu Tianyi Lin Fangwei Gao Yutian Zhao Jeffrey Huang Nipun Katyal Eric Nyberg 47 9 0 04 Jul 2022
A Zero-Shot Classification Approach for a Word-Guessing Challenge Nicos Isaak 6 1 0 27 Jun 2022
PLATON: Pruning Large Transformer Models with Upper Confidence Bound of Weight Importance Qingru Zhang Simiao Zuo Chen Liang Alexander Bukharin Pengcheng He Weizhu Chen T. Zhao 25 78 0 25 Jun 2022
SC-Ques: A Sentence Completion Question Dataset for English as a Second Language Learners Qiongqiong Liu Yaying Huang Zitao Liu Shuyan Huang Jiahao Chen Xiangyu Zhao Guimin Lin Yuyu Zhou Weiqing Luo 33 1 0 24 Jun 2022
BenchCLAMP: A Benchmark for Evaluating Language Models on Syntactic and Semantic Parsing Subhro Roy Sam Thomson Tongfei Chen Richard Shin Adam Pauls Jason Eisner Benjamin Van Durme ALM 33 12 0 21 Jun 2022
DIALOG-22 RuATD Generated Text Detection Narek Maloyan Bulat Nutfullin Eugene Ilyushin DeLMO 27 8 0 16 Jun 2022
Findings of the The RuATD Shared Task 2022 on Artificial Text Detection in Russian T. Shamardina Vladislav Mikhailov Daniil Chernianskii Alena Fenogenova Marat Saidov A. Valeeva Tatiana Shavrina I. Smurov E. Tutubalina Ekaterina Artemova DeLMO 16 30 0 03 Jun 2022
Acquiring and Modelling Abstract Commonsense Knowledge via Conceptualization Mutian He Tianqing Fang Weiqi Wang Yangqiu Song 37 29 0 03 Jun 2022
Detecting Label Errors by using Pre-Trained Language Models Derek Chong Jenny Hong Christopher D. Manning NoLa 55 21 0 25 May 2022
Automatic Rule Induction for Interpretable Semi-Supervised Learning Reid Pryzant Ziyi Yang Yichong Xu Chenguang Zhu Michael Zeng 41 9 0 18 May 2022
Adversarial Training for High-Stakes Reliability Daniel M. Ziegler Seraphina Nix Lawrence Chan Tim Bauman Peter Schmidt-Nielsen ... Noa Nabeshima Benjamin Weinstein-Raun D. Haas Buck Shlegeris Nate Thomas AAML 38 59 0 03 May 2022
Solution of DeBERTaV3 on CommonsenseQA Letian Peng Zuchao Li Hai Zhao 13 0 0 30 Apr 2022
On the Limitations of Dataset Balancing: The Lost Battle Against Spurious Correlations Roy Schwartz Gabriel Stanovsky 37 26 0 27 Apr 2022
MoEBERT: from BERT to Mixture-of-Experts via Importance-Guided Adaptation Simiao Zuo Qingru Zhang Chen Liang Pengcheng He T. Zhao Weizhu Chen MoE 24 38 0 15 Apr 2022
Adapting Pre-trained Language Models to African Languages via Multilingual Adaptive Fine-Tuning Jesujoba Oluwadara Alabi David Ifeoluwa Adelani Marius Mosbach Dietrich Klakow 39 148 0 13 Apr 2022
Nowruz at SemEval-2022 Task 7: Tackling Cloze Tests with Transformers and Ordinal Regression Mohammadmahdi Nouriborji Omid Rohanian David A. Clifton 24 1 0 01 Apr 2022
COOL, a Context Outlooker, and its Application to Question Answering and other Natural Language Processing Tasks Fangyi Zhu See-Kiong Ng S. Bressan LRM 22 1 0 01 Apr 2022
PACS: A Dataset for Physical Audiovisual CommonSense Reasoning Samuel Yu Peter Wu Paul Pu Liang Ruslan Salakhutdinov Louis-Philippe Morency LRM 36 13 0 21 Mar 2022
Towards Visual-Prompt Temporal Answering Grounding in Medical Instructional Video Bin Li Yixuan Weng Bin Sun Shutao Li 35 24 0 13 Mar 2022