DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing

18 November 2021

Papers citing "DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing"

50 / 664 papers shown

Title
On the Evaluation Practices in Multilingual NLP: Can Machine Translation Offer an Alternative to Human Translations? Rochelle Choenni Sara Rajaee Christof Monz Ekaterina Shutova 39 1 0 20 Jun 2024
Bayesian-LoRA: LoRA based Parameter Efficient Fine-Tuning using Optimal Quantization levels and Rank Values trough Differentiable Bayesian Gates Cristian Meo Ksenia Sycheva Anirudh Goyal Justin Dauwels MQ 29 4 0 18 Jun 2024
LaMDA: Large Model Fine-Tuning via Spectrally Decomposed Low-Dimensional Adaptation Seyedarmin Azizi Souvik Kundu Massoud Pedram 32 7 0 18 Jun 2024
Using LLMs to Aid Annotation and Collection of Clinically-Enriched Data in Bipolar Disorder and Schizophrenia Ankit Aich Avery Quynh Pamela Osseyi Amy Pinkham Philip Harvey Brenda L. Curtis Colin A. Depp Natalie Parde AI4MH 16 2 0 18 Jun 2024
MultiSocial: Multilingual Benchmark of Machine-Generated Text Detection of Social-Media Texts Dominik Macko Jakub Kopal Robert Moro Ivan Srba DeLMO 41 2 0 18 Jun 2024
SyncVSR: Data-Efficient Visual Speech Recognition with End-to-End Crossmodal Audio Token Synchronization Young Jin Ahn Jungwoo Park Sangha Park Jonghyun Choi Kee-Eung Kim 34 7 0 18 Jun 2024
Knowledge Fusion By Evolving Weights of Language Models Guodong Du Jing Li Hanting Liu Runhua Jiang Shuyang Yu Yifei Guo S. Goh Ho-Kin Tang MoMe 44 8 0 18 Jun 2024
Not Eliminate but Aggregate: Post-Hoc Control over Mixture-of-Experts to Address Shortcut Shifts in Natural Language Understanding Ukyo Honda Tatsushi Oka Peinan Zhang Masato Mita 52 1 0 17 Jun 2024
Self-training Large Language Models through Knowledge Detection Wei Jie Yeo Teddy Ferdinan Przemyslaw Kazienko Ranjan Satapathy Erik Cambria 41 9 0 17 Jun 2024
Scaling Synthetic Logical Reasoning Datasets with Context-Sensitive Declarative Grammars Damien Sileo LRM ReLM 47 3 0 16 Jun 2024
On the Role of Entity and Event Level Conceptualization in Generalizable Reasoning: A Survey of Tasks, Methods, Applications, and Future Directions Weiqi Wang Tianqing Fang Haochen Shi Baixuan Xu Wenxuan Ding ... Wei Fan Jiaxin Bai Haoran Li Xin Liu Yangqiu Song LRM 32 3 0 16 Jun 2024
GNOME: Generating Negotiations through Open-Domain Mapping of Exchanges Darshan Deshpande Shambhavi Sinha Anirudh Ravi Kumar Debaditya Pal Jonathan May AI4CE 57 0 0 16 Jun 2024
Mixture-of-Subspaces in Low-Rank Adaptation Taiqiang Wu Jiahao Wang Zhe Zhao Ngai Wong 49 22 0 16 Jun 2024
MIND: Multimodal Shopping Intention Distillation from Large Vision-language Models for E-commerce Purchase Understanding Baixuan Xu Weiqi Wang Haochen Shi Wenxuan Ding Huihao Jing Tianqing Fang Jiaxin Bai Long Chen Yangqiu Song 44 9 0 15 Jun 2024
Personalized Pieces: Efficient Personalized Large Language Models through Collaborative Efforts Zhaoxuan Tan Zheyuan Liu Meng Jiang 38 20 0 15 Jun 2024
IntentionQA: A Benchmark for Evaluating Purchase Intention Comprehension Abilities of Language Models in E-commerce Wenxuan Ding Weiqi Wang Sze Heng Douglas Kwok Minghao Liu Tianqing Fang Jiaxin Bai Junxian He Yangqiu Song RALM 44 7 0 14 Jun 2024
Datasets for Multilingual Answer Sentence Selection Matteo Gabburo S. Campese Federico Agostini Alessandro Moschitti 46 0 0 14 Jun 2024
GLiNER multi-task: Generalist Lightweight Model for Various Information Extraction Tasks Ihor Stepanov Mykhailo Shtopko 26 2 0 14 Jun 2024
Disentangling Dialect from Social Bias via Multitask Learning to Improve Fairness Maximilian Spliethover Sai Nikhil Menon Henning Wachsmuth 44 2 0 14 Jun 2024
Detecting Response Generation Not Requiring Factual Judgment Ryohei Kamei Daiki Shiono Reina Akama Jun Suzuki HILM 34 0 0 14 Jun 2024
SEACrowd: A Multilingual Multimodal Data Hub and Benchmark Suite for Southeast Asian Languages Holy Lovenia Rahmad Mahendra Salsabil Maulana Akbar Lester James V. Miranda Jennifer Santoso ... Genta Indra Winata Ruochen Zhang Fajri Koto Zheng-Xin Yong Samuel Cahyawijaya 95 9 0 14 Jun 2024
FouRA: Fourier Low Rank Adaptation Shubhankar Borse Shreya Kadambi N. Pandey Kartikeya Bhardwaj Viswanath Ganapathy Sweta Priyadarshi Risheek Garrepalli Rafael Esteves Munawar Hayat Fatih Porikli 42 6 0 13 Jun 2024
Scaling the Vocabulary of Non-autoregressive Models for Efficient Generative Retrieval Ravisri Valluri Akash Kumar Mohankumar Kushal Dave Amit Singh Jian Jiao Manik Varma Gaurav Sinha 61 1 0 10 Jun 2024
Curating Grounded Synthetic Data with Global Perspectives for Equitable AI Elin Törnquist R. Caulk SyDa 44 4 0 10 Jun 2024
SecureNet: A Comparative Study of DeBERTa and Large Language Models for Phishing Detection Sakshi Mahendru Tejul Pandit 33 1 0 10 Jun 2024
TTM-RE: Memory-Augmented Document-Level Relation Extraction Chufan Gao Xuan Wang Jimeng Sun 35 3 0 09 Jun 2024
Integrating Text and Image Pre-training for Multi-modal Algorithmic Reasoning Zijian Zhang Wei Liu 29 0 0 08 Jun 2024
HateDebias: On the Diversity and Variability of Hate Speech Debiasing Nankai Lin Hongyan Wu Zhengming Chen Zijian Li Lianxi Wang Shengyi Jiang Dong Zhou Aimin Yang 31 0 0 07 Jun 2024
mCSQA: Multilingual Commonsense Reasoning Dataset with Unified Creation Strategy by Language Models and Humans Yusuke Sakai Hidetaka Kamigaito Taro Watanabe LRM 46 3 0 06 Jun 2024
Pointer-Guided Pre-Training: Infusing Large Language Models with Paragraph-Level Contextual Awareness L. Hillebrand Prabhupad Pradhan Christian Bauckhage R. Sifa 21 0 0 06 Jun 2024
Measuring Retrieval Complexity in Question Answering Systems Matteo Gabburo Nicolaas Paul Jedema Siddhant Garg Leonardo F. R. Ribeiro Alessandro Moschitti 47 0 0 05 Jun 2024
Language Model Can Do Knowledge Tracing: Simple but Effective Method to Integrate Language Model and Knowledge Tracing Task Unggi Lee Jiyeong Bae Dohee Kim Sookbun Lee Jaekwon Park Taekyung Ahn Gunho Lee Damji Stratton Hyeoncheol Kim AI4Ed KELM 26 8 0 05 Jun 2024
HYDRA: Model Factorization Framework for Black-Box LLM Personalization Yuchen Zhuang Haotian Sun Yue Yu Rushi Qiang Qifan Wang Chao Zhang Bo Dai AAML 53 15 0 05 Jun 2024
Modeling Emotional Trajectories in Written Stories Utilizing Transformers and Weakly-Supervised Learning Lukas Christ Shahin Amiriparian M. Milling Ilhan Aslan B. Schuller 37 0 0 04 Jun 2024
MARS: Benchmarking the Metaphysical Reasoning Abilities of Language Models with a Multi-task Evaluation Dataset Weiqi Wang Yangqiu Song LRM 35 8 0 04 Jun 2024
Luna: An Evaluation Foundation Model to Catch Language Model Hallucinations with High Accuracy and Low Cost Masha Belyi Robert Friel Shuai Shao Atindriyo Sanyal HILM RALM 64 5 0 03 Jun 2024
Large Language Models for Relevance Judgment in Product Search Navid Mehrdad Hrushikesh Mohapatra Mossaab Bagdouri Prijith Chandran Alessandro Magnani ... Ajit Puthenputhussery Sachin Yadav Tony Lee Chengxiang Zhai Ciya Liao 29 5 0 01 Jun 2024
Entangled Relations: Leveraging NLI and Meta-analysis to Enhance Biomedical Relation Extraction William Hogan Jingbo Shang 18 0 0 31 May 2024
ETHER: Efficient Finetuning of Large-Scale Models with Hyperplane Reflections Massimo Bini Karsten Roth Zeynep Akata Anna Khoreva 37 4 0 30 May 2024
Heidelberg-Boston @ SIGTYP 2024 Shared Task: Enhancing Low-Resource Language Analysis With Character-Aware Hierarchical Transformers Frederick Riemenschneider Kevin Krahn 29 2 0 30 May 2024
SVFT: Parameter-Efficient Fine-Tuning with Singular Vectors Vijay Lingam Atula Tejaswi Aditya Vavre Aneesh Shetty Gautham Krishna Gudur Joydeep Ghosh Alexandros G. Dimakis Eunsol Choi Aleksandar Bojchevski Sujay Sanghavi 49 10 0 30 May 2024
Cross-Modal Safety Alignment: Is textual unlearning all you need? Trishna Chakraborty Erfan Shayegani Zikui Cai Nael B. Abu-Ghazaleh Ulugbek S. Kamilov Yue Dong A. Roy-Chowdhury Chengyu Song 41 16 0 27 May 2024
DoRA: Enhancing Parameter-Efficient Fine-Tuning with Dynamic Rank Distribution Yulong Mao Kaiyu Huang Changhao Guan Ganglin Bao Fengran Mo Jinan Xu 37 11 0 27 May 2024
Accurate and Nuanced Open-QA Evaluation Through Textual Entailment Peiran Yao Denilson Barbosa ELM 32 6 0 26 May 2024
LoQT: Low Rank Adapters for Quantized Training Sebastian Loeschcke M. Toftrup M. Kastoryano Serge Belongie Vésteinn Snæbjarnarson MQ 42 0 0 26 May 2024
Bridging The Gap between Low-rank and Orthogonal Adaptation via Householder Reflection Adaptation Shen Yuan Haotian Liu Hongteng Xu 44 2 0 24 May 2024
Synergizing In-context Learning with Hints for End-to-end Task-oriented Dialog Systems Vishal Vivek Saley Rocktim Jyoti Das Dinesh Raghu Mausam 29 1 0 24 May 2024
Data Augmentation Method Utilizing Template Sentences for Variable Definition Extraction Kotaro Nagayama Shota Kato Manabu Kano 30 1 0 23 May 2024
Maintaining Structural Integrity in Parameter Spaces for Parameter Efficient Fine-tuning Chongjie Si Xuehui Wang Xue Yang Zhengqin Xu Qingyun Li Jifeng Dai Yu Qiao Xiaokang Yang Wei Shen 31 8 0 23 May 2024
Distilling Instruction-following Abilities of Large Language Models with Task-aware Curriculum Planning Yuanhao Yue Chengyu Wang Jun Huang Peng Wang ALM 30 4 0 22 May 2024