111
10

Perseus: A Simple and Optimal High-Order Method for Variational Inequalities

Abstract

We settle an open and challenging question pertaining to the design of simple and optimal high-order methods for solving smooth and monotone variational inequalities (VIs). A VI involves finding xXx^\star \in \mathcal{X} such that F(x),xx0\langle F(x), x - x^\star\rangle \geq 0 for all xXx \in \mathcal{X} and we consider the setting in which F:RdRdF: \mathbb{R}^d \mapsto \mathbb{R}^d is smooth with up to (p1)th(p-1)^{th}-order derivatives. For p=2p = 2, the cubic regularized Newton's method has been extended to VIs with a global rate of O(ϵ1)O(\epsilon^{-1}). An improved rate of O(ϵ2/3loglog(1/ϵ))O(\epsilon^{-2/3}\log\log(1/\epsilon)) can be obtained via an alternative second-order method, but this method requires a nontrivial line-search procedure as an inner loop. Similarly, high-order methods based on similar line-search procedures have been shown to achieve a rate of O(ϵ2/(p+1)loglog(1/ϵ))O(\epsilon^{-2/(p+1)}\log\log(1/\epsilon)), but the inner loop requires fine-tuning of parameters and can be computationally complex. As highlighted by Nesterov, it would be desirable to develop a simple high-order VI method that retains the optimality of the more complex methods. We propose a pthp^{th}-order method that does \textit{not} require any search procedure and provably converges to a weak solution at a rate of O(ϵ2/(p+1))O(\epsilon^{-2/(p+1)}). We prove that our pthp^{th}-order method is optimal in the monotone setting by establishing a lower bound of Ω(ϵ2/(p+1))\Omega(\epsilon^{-2/(p+1)}) under a linear span assumption. A version with restarting attains a global linear and local superlinear convergence rate for smooth and strongly monotone VIs. Furthermore, our method achieves a global rate of O(ϵ2/p)O(\epsilon^{-2/p}) for solving smooth and nonmonotone VIs satisfying the Minty condition. The restarted version again attains a global linear and local superlinear convergence rate if the strong Minty condition is satisfied.

View on arXiv
Comments on this paper

We use cookies and other tracking technologies to improve your browsing experience on our website, to show you personalized content and targeted ads, to analyze our website traffic, and to understand where our visitors are coming from. See our policy.