Convex Relaxation Regression: Efficient Optimization of Smooth Functions by Learning Their Convex Envelopes

Finding efficient and provable methods to solve non-convex optimization problems is an outstanding challenge in machine learning. A popular approach used to tackle non-convex problems is to use convex relaxation techniques to find a convex surrogate for the problem. Unfortunately, convex relaxations typically must be found on a problem-by-problem basis. Thus, providing a general-purpose strategy to estimate a convex relaxation would have a wide reaching impact. Here, we introduce Convex Relaxation Regression (CoRR), an approach for learning the convex relaxation of a wide class of smooth functions. The main idea behind our approach is to estimate the convex envelope of a function $f$ by evaluating $f$ at random points and then fitting a convex function to these function evaluations. We prove that, as the number $T$ of function evaluations grows, the solution of our algorithm converges to the global minimum of $f$ with a polynomial rate in $T$. Also our result scales polynomially with the dimension. Our approach enables the use of convex optimization tools to solve a broad class of non-convex optimization problems.