DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing

18 November 2021

Papers citing "DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing"

50 / 665 papers shown

Title
Overview of AuTexTification at IberLEF 2023: Detection and Attribution of Machine-Generated Text in Multiple Domains A. Sarvazyan José Ángel González Marc Franco-Salvador Francisco Rangel Berta Chulvi Paolo Rosso DeLMO 38 61 0 20 Sep 2023
A Family of Pretrained Transformer Language Models for Russian Dmitry Zmitrovich Alexander Abramov Andrey Kalmykov Maria Tikhonova Ekaterina Taktasheva ... Vitalii Kadulin Sergey Markov Tatiana Shavrina Vladislav Mikhailov Alena Fenogenova 33 26 0 19 Sep 2023
Specializing Small Language Models towards Complex Style Transfer via Latent Attribute Pre-Training Ruiqi Xu Y. Huang Xin Chen Lin Zhang 24 3 0 19 Sep 2023
OpenMSD: Towards Multilingual Scientific Documents Similarity Measurement Yang Gao Ji Ma I. Korotkov Keith B. Hall Dana Alon Donald Metzler 18 0 0 19 Sep 2023
An Evaluation of GPT-4 on the ETHICS Dataset Sergey Rodionov Z. Goertzel Ben Goertzel 29 4 0 19 Sep 2023
Headless Language Models: Learning without Predicting with Contrastive Weight Tying Nathan Godey Eric Villemonte de la Clergerie Benoît Sagot 42 3 0 15 Sep 2023
How to Handle Different Types of Out-of-Distribution Scenarios in Computational Argumentation? A Comprehensive and Fine-Grained Field Study Andreas Waldis Yufang Hou Iryna Gurevych 30 2 0 15 Sep 2023
Leveraging Contextual Information for Effective Entity Salience Detection Rajarshi Bhowmik Marco Ponza Atharva Tendle Anant Gupta Rebecca Jiang Xingyu Lu Qian Zhao Daniel Preotiuc-Pietro 18 1 0 14 Sep 2023
Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructions Federico Bianchi Mirac Suzgun Giuseppe Attanasio Paul Röttger Dan Jurafsky Tatsunori Hashimoto James Zou ALM LM&MA LRM 34 183 0 14 Sep 2023
SIB-200: A Simple, Inclusive, and Big Evaluation Dataset for Topic Classification in 200+ Languages and Dialects David Ifeoluwa Adelani Hannah Liu Xiaoyu Shen Nikita Vassilyev Jesujoba Oluwadara Alabi Yanke Mao Haonan Gao Annie En-Shiun Lee ELM 38 63 0 14 Sep 2023
Gpachov at CheckThat! 2023: A Diverse Multi-Approach Ensemble for Subjectivity Detection in News Articles Georgi Pachov Dimitar Dimitrov Ivan Koychev Preslav Nakov 27 4 0 13 Sep 2023
MMHQA-ICL: Multimodal In-context Learning for Hybrid Question Answering over Text, Tables and Images Weihao Liu Fangyu Lei Tongxu Luo Jiahe Lei Shizhu He Jun Zhao Kang Liu LMTD 32 9 0 09 Sep 2023
From Base to Conversational: Japanese Instruction Dataset and Tuning Large Language Models Masahiro Suzuki Masanori Hirano Hiroki Sakaji 39 6 0 07 Sep 2023
Delta-LoRA: Fine-Tuning High-Rank Parameters with the Delta of Low-Rank Matrices Bojia Zi Xianbiao Qi Lingzhi Wang Jianan Wang Kam-Fai Wong Lei Zhang 34 42 0 05 Sep 2023
IncreLoRA: Incremental Parameter Allocation Method for Parameter-Efficient Fine-tuning Feiyu F. Zhang Liangzhi Li Jun-Cheng Chen Zhouqian Jiang Bowen Wang Yiming Qian 51 33 0 23 Aug 2023
A Survey on Fairness in Large Language Models Yingji Li Mengnan Du Rui Song Xin Wang Ying Wang ALM 52 60 0 20 Aug 2023
Chinese Spelling Correction as Rephrasing Language Model Linfeng Liu Hongqiu Wu Hai Zhao LRM 36 13 0 17 Aug 2023
Semantic Consistency for Assuring Reliability of Large Language Models Harsh Raj Vipul Gupta Domenic Rosati S. Majumdar HILM 110 14 0 17 Aug 2023
Foundation Model is Efficient Multimodal Multitask Model Selector Fanqing Meng Wenqi Shao Zhanglin Peng Chong Jiang Kaipeng Zhang Yu Qiao Ping Luo 30 13 0 11 Aug 2023
A Survey of Spanish Clinical Language Models Guillem García Subies Á. Jiménez Paloma Martínez LM&MA ELM LRM 29 0 0 04 Aug 2023
Specious Sites: Tracking the Spread and Sway of Spurious News Stories at Scale Hans W. A. Hanley Deepak Kumar Zakir Durumeric 42 8 0 03 Aug 2023
DoDo Learning: DOmain-DemOgraphic Transfer in Language Models for Detecting Abuse Targeted at Public Figures Angus R. Williams Hannah Rose Kirk L. Burke Yi-Ling Chung Ivan Debono Pica Johansson Francesca Stevens Jonathan Bright Scott A. Hale 50 1 0 31 Jul 2023
ARC-NLP at PAN 2023: Transition-Focused Natural Language Inference for Writing Style Detection Izzet Emre Kucukkaya Umitcan Sahin Cagri Toraman 11 4 0 27 Jul 2023
FinTree: Financial Dataset Pretrain Transformer Encoder for Relation Extraction Hyunjong Ok 22 2 0 26 Jul 2023
ARC-NLP at Multimodal Hate Speech Event Detection 2023: Multimodal Methods Boosted by Ensemble Learning, Syntactical and Entity Features Umitcan Sahin Izzet Emre Kucukkaya Oguzhan Ozcelik Cagri Toraman 41 10 0 25 Jul 2023
Automated Essay Scoring in Argumentative Writing: DeBERTeachingAssistant Yann Hicke Tonghua Tian Karan Jha Choong Hee Kim 21 2 0 09 Jul 2023
Your spouse needs professional help: Determining the Contextual Appropriateness of Messages through Modeling Social Relationships David Jurgens Agrima Seth Jack E. Sargent Athena Aghighi Michael Geraci 22 7 0 06 Jul 2023
SpaceNLI: Evaluating the Consistency of Predicting Inferences in Space Lasha Abzianidze J. Zwarts Yoad Winter 27 2 0 05 Jul 2023
Chain of Thought Prompting Elicits Knowledge Augmentation Di Wu Jing Zhang Xinmei Huang LRM 28 31 0 04 Jul 2023
Improving Language Plasticity via Pretraining with Active Forgetting Yihong Chen Kelly Marchisio Roberta Raileanu David Ifeoluwa Adelani Pontus Stenetorp Sebastian Riedel Mikel Artetx KELM AI4CE CLL 37 24 0 03 Jul 2023
Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Bias Yue Yu Yuchen Zhuang Jieyu Zhang Yu Meng Alexander Ratner Ranjay Krishna Jiaming Shen Chao Zhang ALM 44 207 0 28 Jun 2023
ARIES: A Corpus of Scientific Paper Edits Made in Response to Peer Reviews Mike DÁrcy Alexis Ross Erin Bransom Bailey Kuehl Jonathan Bragg Tom Hope Doug Downey KELM 32 21 0 21 Jun 2023
Towards Theory-based Moral AI: Moral AI with Aggregating Models Based on Normative Ethical Theory Masashi Takeshita Rafal Rzepka K. Araki 26 8 0 20 Jun 2023
LoSparse: Structured Compression of Large Language Models based on Low-Rank and Sparse Approximation Yixiao Li Yifan Yu Qingru Zhang Chen Liang Pengcheng He Weizhu Chen Tuo Zhao 44 69 0 20 Jun 2023
$RED$^{\rm FM}$: a Filtered and Multilingual Relation Extraction Dataset$ RED $^{\rm FM}$ : a Filtered and Multilingual Relation Extraction Dataset Pere-Lluís Huguet Cabot Simone Tedeschi A. N. Ngomo Roberto Navigli 23 12 0 16 Jun 2023
EaSyGuide : ESG Issue Identification Framework leveraging Abilities of Generative Large Language Models Hanwool Albert Lee Jonghyun Choi Sohyeon Kwon Sungbum Jung 25 3 0 11 Jun 2023
Towards a Robust Detection of Language Model Generated Text: Is ChatGPT that Easy to Detect? Wissam Antoun Virginie Mouilleron Benoît Sagot Djamé Seddah DeLMO 24 33 0 09 Jun 2023
LCT-1 at SemEval-2023 Task 10: Pre-training and Multi-task Learning for Sexism Detection and Classification K. Chernyshev E. Garanina Duygu Bayram Qiankun Zheng Lukas Edman 13 0 0 08 Jun 2023
From the One, Judge of the Whole: Typed Entailment Graph Construction with Predicate Generation Zhibin Chen Yansong Feng Dongyan Zhao 27 0 0 07 Jun 2023
A Unified One-Step Solution for Aspect Sentiment Quad Prediction Junxian Zhou Haiqin Yang Yuxuan He Hao Mou Junbo Yang 34 11 0 07 Jun 2023
CL-UZH at SemEval-2023 Task 10: Sexism Detection through Incremental Fine-Tuning and Multi-Task Learning with Label Descriptions Janis Goldzycher 18 1 0 06 Jun 2023
LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusion Dongfu Jiang Xiang Ren Bill Yuchen Lin ELM 22 275 0 05 Jun 2023
bgGLUE: A Bulgarian General Language Understanding Evaluation Benchmark Momchil Hardalov Pepa Atanasova Todor Mihaylov G. Angelova K. Simov P. Osenova Ves Stoyanov Ivan Koychev Preslav Nakov Dragomir R. Radev ELM FedML 36 4 0 04 Jun 2023
MultiLegalPile: A 689GB Multilingual Legal Corpus Joel Niklaus Veton Matoshi Matthias Sturmer Ilias Chalkidis Daniel E. Ho AILaw ELM 25 40 0 03 Jun 2023
A Simple yet Effective Self-Debiasing Framework for Transformer Models Xiaoyue Wang Lijie Wang Xin Liu Suhang Wu Jinsong Su Huasen Wu 39 3 0 02 Jun 2023
Distilling Efficient Language-Specific Models for Cross-Lingual Transfer Alan Ansell Edoardo Ponti Anna Korhonen Ivan Vulić 32 5 0 02 Jun 2023
Data-Efficient French Language Modeling with CamemBERTa Wissam Antoun Benoît Sagot Djamé Seddah 28 7 0 02 Jun 2023
Boosting the Performance of Transformer Architectures for Semantic Textual Similarity Ivan Rep V. Ceperic 14 0 0 01 Jun 2023
Measuring the Robustness of NLP Models to Domain Shifts Nitay Calderon Naveh Porat Eyal Ben-David Alexander Chapanin Zorik Gekhman Nadav Oved Vitaly Shalumov Roi Reichart 21 7 0 31 May 2023
Automatic Discrimination of Human and Neural Machine Translation in Multilingual Scenarios Mălina Chichirău Rik van Noord Antonio Toral 11 2 0 31 May 2023