ResearchTrend.AI
  • Papers
  • Communities
  • Events
  • Blog
  • Pricing
Papers
Communities
Social Events
Terms and Conditions
Pricing
Parameter LabParameter LabTwitterGitHubLinkedInBlueskyYoutube

© 2025 ResearchTrend.AI, All rights reserved.

  1. Home
  2. Papers
  3. 1707.00117
40
0
v1v2v3 (latest)

SAM: Semantic Attribute Modulated Language Modeling

1 July 2017
Wenbo Hu
Lifeng Hua
Lei Li
Tian Wang
Tian Wang
Ning Chen
Bo Zhang
ArXiv (abs)PDFHTML
Abstract

As a fundamental task in natural language processing, language modeling aims to estimate the distribution of the word sequences. However, most existing algorithms have focused on the main text while often ignoring the vastly-accessible semantic attributes, e.g., titles, authors, sentiments and tags. To address this issue, we build three text datasets with a diversity of semantic attributes, and propose Semantic Attribute Modulated (SAM) language modeling, a novel language modeling framework that incorporates various attributes. Attributes are selected automatically with an attribute attention mechanism. We empirically examine the language model perplexities of several typical corpora, and demonstrate the superiority of our model with different combinations of the attributes. Lyric generation by taking both the title and author attributes into account further demonstrates the effectiveness of our method.

View on arXiv
Comments on this paper