ResearchTrend.AI
  • Papers
  • Communities
  • Events
  • Blog
  • Pricing
Papers
Communities
Social Events
Terms and Conditions
Pricing
Parameter LabParameter LabTwitterGitHubLinkedInBlueskyYoutube

© 2025 ResearchTrend.AI, All rights reserved.

  1. Home
  2. Papers
  3. 1809.00918
46
5
v1v2 (latest)

Segmentation-free Compositional nnn-gram Embedding

4 September 2018
Geewook Kim
Kazuki Fukui
Hidetoshi Shimodaira
    OCL
ArXiv (abs)PDFHTML
Abstract

We propose a new type of representation learning method that models words, phrases and sentences seamlessly. Our method does not depend on word segmentation and any human-annotated resources (e.g., word dictionaries), yet it is very effective for noisy corpora written in unsegmented languages such as Chinese and Japanese. The main idea of our method is to ignore word boundaries completely (i.e., segmentation-free), and construct representations for all character nnn-grams in a raw corpus with embeddings of compositional sub-nnn-grams. Although the idea is simple, our experiments on various benchmarks and real-world datasets show the efficacy of our proposal.

View on arXiv
Comments on this paper