ResearchTrend.AI
  • Papers
  • Communities
  • Events
  • Blog
  • Pricing
Papers
Communities
Social Events
Terms and Conditions
Pricing
Parameter LabParameter LabTwitterGitHubLinkedInBlueskyYoutube

© 2025 ResearchTrend.AI, All rights reserved.

  1. Home
  2. Papers
  3. 1707.00117
33
0
v1v2v3 (latest)

SAM: Semantic Attribute Modulated Language Modeling

1 July 2017
Wenbo Hu
Lifeng Hua
Lei Li
Tian Wang
Tian Wang
Ning Chen
Bo Zhang
ArXiv (abs)PDFHTML
Abstract

As a fundamental task in the natural language processing field, language modeling aims to estimate the distribution of the word sequences. However, the most existing algorithms have focused on the main texts while often ignoring the vastly-accessible semantic attributes of the documents, e.g., titles, authors, sentiments and tags. To address this issue, we propose Semantic Attribute Modulated (SAM) language modeling, a novel language modeling framework that incorporates the various semantic attributes. Attributes are selected automatically with an attribute attention mechanism. We build three text datasets with a diversity of semantic attributes. On the three text datasets, we empirically examine the language model perplexities of several typical corpora, and then demonstrate the superiority of our model with the different combinations of the attributes. Extensive qualitative results, including word semantic analysis, attention values and an interesting lyric generation, further demonstrate the effectiveness of our SAM method.

View on arXiv
Comments on this paper