Functional Nonlinear Sparse Models

Signal processing is rich in inherently continuous and often nonlinear applications, such as radar, spectral estimation, and super-resolution microscopy, in which sparsity plays a key role in obtaining state-of-the-art results. Coping with the infinite dimensionality and non-convexity of these estimation problems typically involves discretization and convex relaxations, e.g., using atomic norms. Although successful, these approaches are not without issues. Discretization often leads to high dimensional, potentially ill-conditioned optimization problems. Moreover, due to grid mismatch and other coherence issues, a sparse signal in the continuous domain need not be sparse when discretized. Finally, problems involving nonlinear measurements remain non-convex even after relaxing the sparsity objective. Even in the linear case, existing performance guarantees for atomic norm relaxations hold under assumptions that may be hard to meet in practice and cannot be checked. We propose to address these issues by directly tackling the continuous, nonlinear problem cast as a sparse functional optimization program. We prove that these problems have no duality gap and show that they can be solved efficiently using duality and (stochastic) subgradient ascent-type algorithms. We illustrate the wide range of applications for this new approach by formulating and solving problems from nonlinear spectral estimation and robust classification.