DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing

18 November 2021

Papers citing "DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing"

50 / 664 papers shown

Title
Large Language Models in the Workplace: A Case Study on Prompt Engineering for Job Type Classification Benjamin Clavié Alexandru Ciceu Frederick Naylor Guillaume Soulié Thomas Brightwell LLMAG 30 42 0 13 Mar 2023
SemEval-2023 Task 10: Explainable Detection of Online Sexism Hannah Rose Kirk Wenjie Yin Bertie Vidgen Paul Röttger 24 117 0 07 Mar 2023
UDAPDR: Unsupervised Domain Adaptation via LLM Prompting and Distillation of Rerankers Jon Saad-Falcon Omar Khattab Keshav Santhanam Radu Florian M. Franz Salim Roukos Avirup Sil Md Arafat Sultan Christopher Potts 29 41 0 01 Mar 2023
Tweets Under the Rubble: Detection of Messages Calling for Help in Earthquake Disaster Cagri Toraman Izzet Emre Kucukkaya Oguzhan Ozcelik Umitcan Sahin 18 9 0 26 Feb 2023
Automated Extraction of Fine-Grained Standardized Product Information from Unstructured Multilingual Web Data Alexander Flick Sebastian Jäger Ivana Trajanovska F. Biessmann 16 0 0 23 Feb 2023
ChatGPT: Jack of all trades, master of none Jan Kocoñ Igor Cichecki Oliwier Kaszyca Mateusz Kochanek Dominika Szydło ... Maciej Piasecki Lukasz Radliñski Konrad Wojtasik Stanislaw Wo'zniak Przemyslaw Kazienko AI4MH 37 528 0 21 Feb 2023
HomoDistil: Homotopic Task-Agnostic Distillation of Pre-trained Transformers Chen Liang Haoming Jiang Zheng Li Xianfeng Tang Bin Yin Tuo Zhao VLM 27 24 0 19 Feb 2023
AfriSenti: A Twitter Sentiment Analysis Benchmark for African Languages Shamsuddeen Hassan Muhammad Idris Abdulmumin A. Ayele N. Ousidhoum David Ifeoluwa Adelani ... Hailu Beshada Balcha S. Chala Hagos Tesfahun Gebremichael Bernard Opoku Steven Arthur 39 83 0 17 Feb 2023
Enhancing Model Performance in Multilingual Information Retrieval with Comprehensive Data Engineering Techniques Qi Zhang Zijian Yang Yi-Li Huang Ze Chen Zijian Cai Kangxu Wang Jiewen Zheng Jiarong He Jin Gao LRM VLM 26 1 0 14 Feb 2023
An Extended Sequence Tagging Vocabulary for Grammatical Error Correction Stuart Mesham Christopher Bryant Marek Rei Zheng Yuan 35 7 0 12 Feb 2023
Flexible, Model-Agnostic Method for Materials Data Extraction from Text Using General Purpose Language Models Maciej P. Polak S. Modi Anna Latosinska Jinming Zhang Ching-Wen Wang Shanonan Wang Ayan Deep Hazra Dane Morgan 27 21 0 09 Feb 2023
Representation Deficiency in Masked Language Modeling Yu Meng Jitin Krishnan Sinong Wang Qifan Wang Yuning Mao Han Fang Marjan Ghazvininejad Jiawei Han Luke Zettlemoyer 90 7 0 04 Feb 2023
ESC: Exploration with Soft Commonsense Constraints for Zero-shot Object Navigation KAI-QING Zhou Kai Zheng Connor Pryor Yilin Shen Hongxia Jin Lise Getoor Qing Guo 34 107 0 30 Jan 2023
LEXTREME: A Multi-Lingual and Multi-Task Benchmark for the Legal Domain Joel Niklaus Veton Matoshi Pooja Rani Andrea Galassi Matthias Sturmer Ilias Chalkidis ELM AILaw 19 55 0 30 Jan 2023
Theme-driven Keyphrase Extraction to Analyze Social Media Discourse William Romano Omar Sharif Madhusudan Basak Joseph Gatto S. Preum 19 6 0 27 Jan 2023
ViDeBERTa: A powerful pre-trained language model for Vietnamese Cong Dao Tran Nhut Huy Pham Anh-Viêt Nguyên Truong-Son Hy Tu Vu 27 14 0 25 Jan 2023
OPD@NL4Opt: An ensemble approach for the NER task of the optimization problem Kangxu Wang Ze Chen Jiewen Zheng 21 6 0 06 Jan 2023
Cramming: Training a Language Model on a Single GPU in One Day Jonas Geiping Tom Goldstein MoE 30 85 0 28 Dec 2022
ZEROTOP: Zero-Shot Task-Oriented Semantic Parsing using Large Language Models Dheeraj Mekala Jason Wolfe Subhro Roy 41 9 0 21 Dec 2022
SimpleStyle: An Adaptable Style Transfer Approach Elron Bandel Yoav Katz Noam Slonim L. Ein-Dor 23 0 0 20 Dec 2022
WeCheck: Strong Factual Consistency Checker via Weakly Supervised Learning Wenhao Wu Wei Li Xinyan Xiao Jiachen Liu Sujian Li Yajuan Lv HILM 28 4 0 20 Dec 2022
Dataless Knowledge Fusion by Merging Weights of Language Models Xisen Jin Xiang Ren Daniel Preotiuc-Pietro Pengxiang Cheng FedML MoMe 24 214 0 19 Dec 2022
Rethinking Label Smoothing on Multi-hop Question Answering Zhangyue Yin Yuxin Wang Xiannian Hu Yiguang Wu Hang Yan Xinyu Zhang Bo Zhao Xuanjing Huang Xipeng Qiu 26 9 0 19 Dec 2022
Latent Diffusion for Language Generation Justin Lovelace Varsha Kishore Chao-gang Wan Eliot Shekhtman Kilian Q. Weinberger DiffM 24 71 0 19 Dec 2022
APOLLO: A Simple Approach for Adaptive Pretraining of Language Models for Logical Reasoning Soumya Sanyal Yichong Xu Shuohang Wang Ziyi Yang Reid Pryzant Wenhao Yu Chenguang Zhu Xiang Ren ReLM LRM 35 8 0 19 Dec 2022
Lessons learned from the evaluation of Spanish Language Models Rodrigo Agerri Eneko Agirre ELM 30 15 0 16 Dec 2022
APOLLO: An Optimized Training Approach for Long-form Numerical Reasoning Jiashuo Sun Hang Zhang Chen Lin Nan Duan Yeyun Gong Jian Guo AIMat RALM 33 6 0 14 Dec 2022
VTCC-NLP at NL4Opt competition subtask 1: An Ensemble Pre-trained language models for Named Entity Recognition Xuan-Dung Doan 35 6 0 14 Dec 2022
Artificial Text Detection with Multiple Training Strategies Bin Li Yixuan Weng Qiya Song Hanjun Deng DeLMO 26 4 0 10 Dec 2022
BudgetLongformer: Can we Cheaply Pretrain a SotA Legal Language Model From Scratch? Joel Niklaus Daniele Giofré 33 11 0 30 Nov 2022
X-PuDu at SemEval-2022 Task 6: Multilingual Learning for English and Arabic Sarcasm Detection Yangchao Han Yekun Chai Shuohuan Wang Yu Sun Hongyi Huang Guanghao Chen Yitong Xu Yang Yang 27 8 0 30 Nov 2022
Contrastive Novelty-Augmented Learning: Anticipating Outliers with Large Language Models Albert Xu Xiang Ren Robin Jia OODD 35 2 0 28 Nov 2022
Frustratingly Easy Label Projection for Cross-lingual Transfer Yang Chen Chao Jiang Alan Ritter Wei-ping Xu 27 31 0 28 Nov 2022
Understanding BLOOM: An empirical study on diverse NLP tasks Parag Dakle Sai Krishna Rallabandi Preethi Raghavan AI4CE 39 3 0 27 Nov 2022
X-PuDu at SemEval-2022 Task 7: A Replaced Token Detection Task Pre-trained Model with Pattern-aware Ensembling for Identifying Plausible Clarifications Junyuan Shang Shuohuan Wang Yu Sun Yanjun Yu Yue Zhou Li Xiang Guixiu Yang 37 2 0 27 Nov 2022
A Report on the Euphemisms Detection Shared Task Patrick Lee Anna Feldman J. Peng 52 9 0 23 Nov 2022
FolkScope: Intention Knowledge Graph Construction for E-commerce Commonsense Discovery Changlong Yu Weiqi Wang Xin Liu Jiaxin Bai Yangqiu Song Zheng Li Yifan Gao Tianyu Cao Bing Yin 52 22 0 15 Nov 2022
A Universal Discriminator for Zero-Shot Generalization Haike Xu Zongyu Lin Jing Zhou Yanan Zheng Zhilin Yang AI4CE 21 14 0 15 Nov 2022
Measuring Reliability of Large Language Models through Semantic Consistency Harsh Raj Domenic Rosati S. Majumdar HILM 24 30 0 10 Nov 2022
Detecting Euphemisms with Literal Descriptions and Visual Imagery .Ilker Kesen Aykut Erdem Erkut Erdem Iacer Calixto 29 4 0 08 Nov 2022
SocioProbe: What, When, and Where Language Models Learn about Sociodemographics Anne Lauscher Federico Bianchi Samuel R. Bowman Dirk Hovy 32 7 0 08 Nov 2022
Using Deep Mixture-of-Experts to Detect Word Meaning Shift for TempoWiC Ze Chen Kangxu Wang Zijian Cai Jiewen Zheng Jiarong He Max Gao Jason Zhang MoE 14 3 0 07 Nov 2022
PASTA: Table-Operations Aware Fact Verification via Sentence-Table Cloze Pre-training Zihui Gu Ju Fan Nan Tang Preslav Nakov Xiaoman Zhao Xiaoyong Du LMTD 31 45 0 05 Nov 2022
CONDAQA: A Contrastive Reading Comprehension Dataset for Reasoning about Negation Abhilasha Ravichander Matt Gardner Ana Marasović 33 34 0 01 Nov 2022
RLET: A Reinforcement Learning Based Approach for Explainable QA with Entailment Trees Tengxiao Liu Qipeng Guo Xiangkun Hu Yue Zhang Xipeng Qiu Zheng-Wei Zhang LRM 41 14 0 31 Oct 2022
Improving Bilingual Lexicon Induction with Cross-Encoder Reranking Yaoyiran Li Fangyu Liu Ivan Vulić Anna Korhonen 34 10 0 30 Oct 2022
Two is Better than Many? Binary Classification as an Effective Approach to Multi-Choice Question Answering Deepanway Ghosal Navonil Majumder Rada Mihalcea Soujanya Poria 58 10 0 29 Oct 2022
Visual Answer Localization with Cross-modal Mutual Knowledge Transfer Yixuan Weng Bin Li 24 6 0 26 Oct 2022
Robots-Dont-Cry: Understanding Falsely Anthropomorphic Utterances in Dialog Systems David Gros Yu Li Zhou Yu 49 9 0 22 Oct 2022
MasakhaNER 2.0: Africa-centric Transfer Learning for Named Entity Recognition David Ifeoluwa Adelani Graham Neubig Sebastian Ruder Shruti Rijhwani Michael Beukman ... Idris Abdulmumin Odunayo Ogundepo Oreen Yousuf Tatiana Moteu Ngoli Dietrich Klakow 44 43 0 22 Oct 2022