Page 1 of 44 Information Systems Research MATHEMATICS OF OPERATIONS RESEARCH INFORMS Vol. 00, No. 0, Xxxxx 0000, pp. 000{000 doi 10.1287/xxxx.0000.0000 0364-765X j 1526-5471 j 00 j 0000 j 0001 issn eissn c 0000 INFORMS Authors are encouraged to submit new papers to INFORMS journals by means of a style file template, which includes the journal title. However, use of a template does not certify that the paper has been accepted for publication in the named jour- nal. INFORMS journal templates are for the exclusive purpose of submitting to an INFORMS journal and should not be used to distribute the papers in print or online or to submit the papers to another publication. On the Efficiency of Random Permutation for ADMM and Coordinate Descent * Ruoyu Sun University of Illinois Urbana-Champaign,
[email protected] Zhi-Quan Luo Chinese University of Hong Kong (Shenzhen), University of Minnesota
[email protected] Yinyu Ye Stanford University,
[email protected] Random permutation is observed to be powerful for optimization algorithms: for multi-block ADMM (alter- nating direction method of multipliers), while the classical cyclic version divergence, the randomly permuted version converges in practice; for BCD (block coordinate descent), the randomly permuted version is typically faster than other versions. In this paper, we provide strong theoretical evidence that random permutation has positive effects on ADMM and BCD, by analyzing randomly permuted ADMM (RP-ADMM) for solving linear systems of equations, and randomly permuted BCD (RP-BCD) for solving unconstrained quadratic problems. First, we prove that RP-ADMM converges in expectation for solving systems of linear equations.