ResearchTrend.AI
  • Papers
  • Communities
  • Events
  • Blog
  • Pricing
Papers
Communities
Social Events
Terms and Conditions
Pricing
Parameter LabParameter LabTwitterGitHubLinkedInBlueskyYoutube

© 2025 ResearchTrend.AI, All rights reserved.

  1. Home
  2. Papers
  3. 1606.03474
40
2
v1v2v3v4 (latest)

On degeneracy control in overcomplete ICA

10 June 2016
J. Livezey
Alejandro F. Bujan
Friedrich T. Sommer
ArXiv (abs)PDFHTML
Abstract

Understanding the effects of degeneracy control mechanisms when learning overcomplete representations is crucial for applying Independent Components Analysis (ICA) in machine learning and theoretical neuroscience. A number of approaches to degeneracy control have been proposed which can learn non-degenerate complete representations, however some of these methods can fall into bad local minima when extended to overcomplete ICA. Furthermore, they may have unintended side-effects on the distribution of learned basis elements, which may lead to a biased exploration of the data manifold. In this work, we identify and theoretically analyze the cause of these failures and propose a framework that can be used to evaluate arbitrary degeneracy control mechanisms. We evaluate different methods for degeneracy control in overcomplete ICA and suggest two novel approaches, one of which can learn highly orthonormal bases. Finally, we compare all methods on the task of estimating an overcomplete basis on natural images.

View on arXiv
Comments on this paper