ResearchTrend.AI
  • Papers
  • Communities
  • Events
  • Blog
  • Pricing
Papers
Communities
Social Events
Terms and Conditions
Pricing
Parameter LabParameter LabTwitterGitHubLinkedInBlueskyYoutube

© 2025 ResearchTrend.AI, All rights reserved.

  1. Home
  2. Papers
  3. 2506.00968
46
0

PolyBERT: Fine-Tuned Poly Encoder BERT-Based Model for Word Sense Disambiguation

1 June 2025
Linhan Xia
Mingzhan Yang
Guohui Yuan
Shengnan Tao
Yujing Qiu
Guo Yu
Kai Lei
ArXiv (abs)PDFHTML
Main:5 Pages
3 Figures
Bibliography:1 Pages
Abstract

Mainstream Word Sense Disambiguation (WSD) approaches have employed BERT to extract semantics from both context and definitions of senses to determine the most suitable sense of a target word, achieving notable performance. However, there are two limitations in these approaches. First, previous studies failed to balance the representation of token-level (local) and sequence-level (global) semantics during feature extraction, leading to insufficient semantic representation and a performance bottleneck. Second, these approaches incorporated all possible senses of each target word during the training phase, leading to unnecessary computational costs. To overcome these limitations, this paper introduces a poly-encoder BERT-based model with batch contrastive learning for WSD, named PolyBERT. Compared with previous WSD methods, PolyBERT has two improvements: (1) A poly-encoder with a multi-head attention mechanism is utilized to fuse token-level (local) and sequence-level (global) semantics, rather than focusing on just one. This approach enriches semantic representation by balancing local and global semantics. (2) To avoid redundant training inputs, Batch Contrastive Learning (BCL) is introduced. BCL utilizes the correct senses of other target words in the same batch as negative samples for the current target word, which reduces training inputs and computational cost. The experimental results demonstrate that PolyBERT outperforms baseline WSD methods such as Huang's GlossBERT and Blevins's BEM by 2\% in F1-score. In addition, PolyBERT with BCL reduces GPU hours by 37.6\% compared with PolyBERT without BCL.

View on arXiv
@article{xia2025_2506.00968,
  title={ PolyBERT: Fine-Tuned Poly Encoder BERT-Based Model for Word Sense Disambiguation },
  author={ Linhan Xia and Mingzhan Yang and Guohui Yuan and Shengnan Tao and Yujing Qiu and Guo Yu and Kai Lei },
  journal={arXiv preprint arXiv:2506.00968},
  year={ 2025 }
}
Comments on this paper