DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing

18 November 2021

Papers citing "DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing"

50 / 79 papers shown

Title
RefLoRA: Refactored Low-Rank Adaptation for Efficient Fine-Tuning of Large Models Yilang Zhang Bingcong Li G. Giannakis 228 1 0 24 May 2025
Fann or Flop: A Multigenre, Multiera Benchmark for Arabic Poetry Understanding in LLMs Wafa Alghallabi Ritesh Thawkar Sara Ghaboura Ketan More Omkar Thawakar Hisham Cholakkal Salman Khan Rao Muhammad Anwer 139 0 0 23 May 2025
Unveiling the Hidden: Movie Genre and User Bias in Spoiler Detection Haokai Zhang Shengtao Zhang Zijian Cai Heng Wang Ruixuan Zhu Zinan Zeng Minnan Luo 109 0 0 24 Apr 2025
Accuracy is Not Agreement: Expert-Aligned Evaluation of Crash Narrative Classification Models S. Bhagat Ibne Farabi Shihab Anuj Sharma 169 0 0 17 Apr 2025
Robust and Fine-Grained Detection of AI Generated Texts Ram Mohan Rao Kadiyala Siddartha Pullakhandam Kanwal Mehreen Drishti Sharma Siddhant Gupta ... Arvind Reddy Bobbili Suraj Telugara Chandrashekhar Modabbir Adeeb Srinadh Vura Hamza Farooq DeLMO 93 0 0 16 Apr 2025
Resampling Benchmark for Efficient Comprehensive Evaluation of Large Vision-Language Models Teppei Suzuki Keisuke Ozawa VLM 153 0 0 14 Apr 2025
SemEval-2025 Task 5: LLMs4Subjects -- LLM-based Automated Subject Tagging for a National Technical Library's Open-Access Catalog Jennifer D’Souza Sameer Sadruddin Holger Israel Mathias Begoin Diana Slawig 187 5 0 09 Apr 2025
GLiNER-BioMed: A Suite of Efficient Models for Open Biomedical Named Entity Recognition Anthony Yazdani Ihor Stepanov Douglas Teodoro VLM AI4CE 103 0 0 01 Apr 2025
AfroXLMR-Social: Adapting Pre-trained Language Models for African Languages Social Media Text Tadesse Destaw Belay Israel Abebe Azime Ibrahim Said Ahmad David Ifeoluwa Adelani Idris Abdulmumin Abinew Ali Ayele Shamsuddeen Hassan Muhammad Seid Muhie Yimam 94 0 0 24 Mar 2025
EuroBERT: Scaling Multilingual Encoders for European Languages Nicolas Boizard Hippolyte Gisserot-Boukhlef Duarte M. Alves André F. T. Martins Ayoub Hammal ... Maxime Peyrard Nuno M. Guerreiro Patrick Fernandes Ricardo Rei Pierre Colombo 474 3 0 07 Mar 2025
Quantum-PEFT: Ultra parameter-efficient fine-tuning Toshiaki Koike-Akino F. Tonin Yongtao Wu Frank Zhengqing Wu Leyla Naz Candogan Volkan Cevher MQ 186 5 0 07 Mar 2025
Measuring and Benchmarking Large Language Models' Capabilities to Generate Persuasive Language Amalie Brogaard Pauli Isabelle Augenstein Ira Assent 97 8 0 24 Feb 2025
Machine-generated text detection prevents language model collapse George Drayson Emine Yilmaz Vasileios Lampos DeLMO 147 1 0 21 Feb 2025
BabyLM Turns 3: Call for papers for the 2025 BabyLM workshop Lucas Charpentier Leshem Choshen Ryan Cotterell Mustafa Omer Gul Michael Y. Hu ... Candace Ross Raj Sanjay Shah Alex Warstadt Ethan Gotlieb Wilcox Adina Williams 93 4 0 15 Feb 2025
AIMA at SemEval-2024 Task 3: Simple Yet Powerful Emotion Cause Pair Analysis Alireza Ghahramani Kure Mahshid Dehghani Mohammad Mahdi Abootorabi Nona Ghazizadeh Seyed Arshan Dalili Ehsaneddin Asgari 90 1 0 19 Jan 2025
RAGBench: Explainable Benchmark for Retrieval-Augmented Generation Systems Robert Friel Masha Belyi Atindriyo Sanyal 124 27 0 17 Jan 2025
KaSA: Knowledge-Aware Singular-Value Adaptation of Large Language Models Fan Wang Juyong Jiang Chansung Park Sunghun Kim Jing Tang 176 1 0 08 Dec 2024
Attention Tracker: Detecting Prompt Injection Attacks in LLMs Kuo-Han Hung Ching-Yun Ko Ambrish Rawat I-Hsin Chung Winston H. Hsu Pin-Yu Chen 99 8 0 01 Nov 2024
InjecGuard: Benchmarking and Mitigating Over-defense in Prompt Injection Guardrail Models Haoyang Li Xiaogeng Liu SILM 102 9 0 30 Oct 2024
Are AI Detectors Good Enough? A Survey on Quality of Datasets With Machine-Generated Texts German Gritsai Anastasia Voznyuk Andrey Grabovoy Yury Chekhovich DeLMO 109 2 0 18 Oct 2024
MIRAGE-Bench: Automatic Multilingual Benchmark Arena for Retrieval-Augmented Generation Systems Nandan Thakur Suleman Kazi Ge Luo Jimmy J. Lin Amin Ahmad VLM RALM 177 7 0 17 Oct 2024
On A Scale From 1 to 5: Quantifying Hallucination in Faithfulness Evaluation Xiaonan Jing Srinivas Billa Danny Godbout HILM 98 0 0 16 Oct 2024
On the Risk of Evidence Pollution for Malicious Social Text Detection in the Era of LLMs Herun Wan Minnan Luo Zhixiong Su Guang Dai Xiang Zhao DeLMO 93 0 0 16 Oct 2024
Yesterday's News: Benchmarking Multi-Dimensional Out-of-Distribution Generalization of Misinformation Detection Models Ivo Verhoeven Pushkar Mishra Ekaterina Shutova 84 0 0 12 Oct 2024
HarmAug: Effective Data Augmentation for Knowledge Distillation of Safety Guard Models Seanie Lee Haebin Seong Dong Bok Lee Minki Kang Xiaoyin Chen Dominik Wagner Yoshua Bengio Juho Lee Sung Ju Hwang 164 6 0 02 Oct 2024
A Bayesian Interpretation of Adaptive Low-Rank Adaptation Haolin Chen Philip N. Garner 114 1 0 16 Sep 2024
Algorithmic Behaviors Across Regions: A Geolocation Audit of YouTube Search for COVID-19 Misinformation Between the United States and South Africa Hayoung Jung Prerna Juneja Tanushree Mitra MLAU 102 1 0 16 Sep 2024
CLOCR-C: Context Leveraging OCR Correction with Pre-trained Language Models Jonathan Bourne 91 4 0 30 Aug 2024
ReLiK: Retrieve and LinK, Fast and Accurate Entity Linking and Relation Extraction on an Academic Budget Adam Gould Pere-Lluis Huguet-Cabot S. Dadhania Francesca Toni 156 9 0 31 Jul 2024
Efficient Nearest Neighbor based Uncertainty Estimation for Natural Language Processing Tasks Wataru Hashimoto Hidetaka Kamigaito Taro Watanabe 95 0 0 02 Jul 2024
SEACrowd: A Multilingual Multimodal Data Hub and Benchmark Suite for Southeast Asian Languages Holy Lovenia Rahmad Mahendra Salsabil Maulana Akbar Lester James V. Miranda Jennifer Santoso ... Genta Indra Winata Ruochen Zhang Fajri Koto Zheng-Xin Yong Samuel Cahyawijaya 169 14 0 14 Jun 2024
MARS: Benchmarking the Metaphysical Reasoning Abilities of Language Models with a Multi-task Evaluation Dataset Weiqi Wang Yangqiu Song LRM 111 10 0 04 Jun 2024
PiSSA: Principal Singular Values and Singular Vectors Adaptation of Large Language Models Fanxu Meng Zhaohui Wang Muhan Zhang VLM 128 97 0 03 Apr 2024
ELLEN: Extremely Lightly Supervised Learning For Efficient Named Entity Recognition Haris Riaz Razvan-Gabriel Dumitru Mihai Surdeanu MU 113 0 0 26 Mar 2024
Can We Verify Step by Step for Incorrect Answer Detection? Xin Xu Shizhe Diao Can Yang Yang Wang LRM 216 15 0 16 Feb 2024
Fast Word Error Rate Estimation Using Self-Supervised Representations for Speech and Text Chanho Park Chengsong Lu Mingjie Chen Thomas Hain 110 3 0 12 Oct 2023
Semantic Consistency for Assuring Reliability of Large Language Models Harsh Raj Vipul Gupta Domenic Rosati S. Majumdar HILM 132 14 0 17 Aug 2023
PART: Pre-trained Authorship Representation Transformer Javier Huertas-Tato Álvaro Huertas-García Alejandro Martín 111 9 0 30 Sep 2022
Small-Bench NLP: Benchmark for small single GPU trained models in Natural Language Processing K. Kanakarajan Bhuvana Kundumani Malaikannan Sankarasubbu ALM MoE 37 5 0 22 Sep 2021
XLM-E: Cross-lingual Language Model Pre-training via ELECTRA Zewen Chi Shaohan Huang Li Dong Shuming Ma Bo Zheng ... Payal Bajaj Xia Song Xian-Ling Mao Heyan Huang Furu Wei 80 120 0 30 Jun 2021
COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretraining Yu Meng Chenyan Xiong Payal Bajaj Saurabh Tiwary Paul N. Bennett Jiawei Han Xia Song 162 205 0 16 Feb 2021
Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity W. Fedus Barret Zoph Noam M. Shazeer MoE 85 2,187 0 11 Jan 2021
MiniLMv2: Multi-Head Self-Attention Relation Distillation for Compressing Pretrained Transformers Wenhui Wang Hangbo Bao Shaohan Huang Li Dong Furu Wei MQ 84 268 0 31 Dec 2020
mT5: A massively multilingual pre-trained text-to-text transformer Linting Xue Noah Constant Adam Roberts Mihir Kale Rami Al-Rfou Aditya Siddhant Aditya Barua Colin Raffel 138 2,555 0 22 Oct 2020
DeBERTa: Decoding-enhanced BERT with Disentangled Attention Pengcheng He Xiaodong Liu Jianfeng Gao Weizhu Chen AAML 159 2,737 0 05 Jun 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 762 42,055 0 28 May 2020
MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers Wenhui Wang Furu Wei Li Dong Hangbo Bao Nan Yang Ming Zhou VLM 145 1,267 0 25 Feb 2020
CodeBERT: A Pre-Trained Model for Programming and Natural Languages Zhangyin Feng Daya Guo Duyu Tang Nan Duan Xiaocheng Feng ... Linjun Shou Bing Qin Ting Liu Daxin Jiang Ming Zhou 162 2,633 0 19 Feb 2020
Unsupervised Cross-lingual Representation Learning at Scale Alexis Conneau Kartikay Khandelwal Naman Goyal Vishrav Chaudhary Guillaume Wenzek Francisco Guzmán Edouard Grave Myle Ott Luke Zettlemoyer Veselin Stoyanov 215 6,555 0 05 Nov 2019
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 424 20,181 0 23 Oct 2019