This paper focuses on the coordinate update method, which is useful for solving large-sized problems involving linear and nonlinear mappings, and smooth and nonsmooth functions. It decomposes a problem into simple subproblems, where each subproblem updates one, or a small block of, variables. The coordinate update method sits at a high level of abstraction and includes many special cases such as the Jacobi, Gauss-Seidel, alternated projection, as well as coordinate descent methods. They have found greatly many applications throughout computational sciences. In this paper, we abstract many problems to the fixed-point problem and study the favorable structures in operator that enable highly efficient coordinate updates: . Such updates can be carried out in the sequential, parallel, and async-parallel fashions. This study leads to new coordinate update algorithms for a variety of problems in machine learning, image processing, as well as sub-areas of optimization. The obtained algorithms are scalable to very large instances through parallel and even asynchronous computing. We present numerical examples to illustrate how effective these algorithms are.
View on arXiv