University of California Los Angeles
Total Page：16
File Type：pdf, Size：1020Kb
University of California Los Angeles Robust Communication and Optimization over Dynamic Networks A dissertation submitted in partial satisfaction of the requirements for the degree Doctor of Philosophy in Electrical Engineering by Can Karakus 2018 c Copyright by Can Karakus 2018 ABSTRACT OF THE DISSERTATION Robust Communication and Optimization over Dynamic Networks by Can Karakus Doctor of Philosophy in Electrical Engineering University of California, Los Angeles, 2018 Professor Suhas N. Diggavi, Chair Many types of communication and computation networks arising in modern systems have fundamentally dynamic, time-varying, and ultimately unreliably available resources. Specifi- cally, in wireless communication networks, such unreliability may manifest itself as variability in channel conditions, intermittent availability of undedicated resources (such as unlicensed spectrum), or collisions due to multiple-access. In distributed computing, and specifically in large-scale distributed optimization and machine learning, this phenomenon manifests itself in the form of communication bottlenecks, straggling or failed nodes, or running background processes which hamper or slow down the computational task. In this thesis, we develop information-theoretically-motivated approaches that make progress towards building robust and reliable communication and computation networks built upon unreliable resources. In the first part of the thesis, we focus on three problems in wireless networks which involve opportunistically harnessing time-varying resources while providing theoretical per- formance guarantees. First, we show that in full-duplex uplink-downlink cellular networks, a simple, low-overhead user scheduling scheme that exploits the variations in channel con- ditions can be used to optimally mitigate inter-user interference in the many-user regime. Next, we consider the use of intermittently available links over unlicensed spectral bands to enhance communication over the licensed cellular band. We show that channel output feedback over such links, combined with quantize-map-forward relaying, provides generalized- degrees-of-freedom gain in interference networks. We characterize the information-theoretic ii capacity region of this model to within a constant gap. We finally consider the use of such intermittent links in device-to-device cooperation to aid cellular downlink. We develop an optimal dynamic resource allocation algorithm for such networks using stochastic approxi- mation and graph theory techniques, and show that the resulting scheme results in up to 5-6x throughput gain for cell-edge users. In the second part, we consider the problem of distributed optimization and machine learning over large-scale, yet unreliable clusters. Focusing on a master-worker architecture, where large-scale datasets are distributed across worker nodes which communicate with a central parameter server to optimize a global objective, we develop a framework for embed- ding redundancy in the dataset to combat node failures and delays. This framework consists of an efficient linear transformation (coding) of the dataset that results in an overcomplete representation, combined with a coding-oblivious application of a distributed optimization algorithm. We show that if the linear transformation is designed to satisfy certain spectral properties resembling the restricted isometry property, nodes that fail or delay their com- putation can be dynamically left out of the computational process, while still converging to a reasonable solution with fast convergence rates, obviating the need for explicit fault- tolerance mechanisms and significantly speeding up overall computation. We implement the techniques on Amazon EC2 clusters to demonstrate the applicability of the proposed technique to various machine learning problems, such as logistic regression, support vector machine, ridge regression, and collaborative filtering; as well as several popular optimization algorithms including gradient descent, L-BFGS, coordinate descent and proximal gradient methods. iii The dissertation of Can Karakus is approved. Wotao Yin Gregory J. Pottie Richard D. Wesel Suhas N. Diggavi, Committee Chair University of California, Los Angeles 2018 iv To my parents, Cilen and Ilhami. v Table of Contents 1 Introduction :::::::::::::::::::::::::::::::::::::: 1 1.1 Dynamic Networks . 1 1.1.1 Wireless networks . 2 1.1.2 Distributed computing . 3 1.2 Thesis Outline . 4 2 Opportunistic Feedback for Interference Management ::::::::::: 6 2.1 Introduction . 6 2.2 Interference Channel with Intermittent Feedback . 7 2.3 Model and Formulation . 11 2.3.1 Linear deterministic model . 12 2.3.2 Gaussian model . 13 2.4 Insights from Linear Deterministic Model . 15 2.5 Capacity Results for Interference Channels with Intermittent Feedback . 18 2.5.1 Linear deterministic model . 19 2.5.2 Gaussian model . 20 2.5.3 Discussion of results . 24 2.6 Achievability . 26 2.6.1 Overview of the achievable strategy . 26 2.6.2 Codebook generation . 27 2.6.3 Encoding . 27 2.6.4 Decoding . 29 vi 2.6.5 Error analysis . 33 2.6.6 Rate region evaluation . 36 2.7 Converse . 38 2.7.1 Bounds on R1 and R2 .......................... 38 2.7.2 Bounds on R1 + R2, 2R1 + R2 and R1 + 2R2 . 39 2.8 Discussion and Extensions . 44 3 Opportunistic Scheduling in Full-duplex Cellular Networks :::::::: 46 3.1 Introduction . 46 3.2 Related Work and Contributions . 47 3.3 Model and Notation . 49 3.4 Opportunistic Scheduling for Homogeneous Networks . 51 3.4.1 Opportunistic scheduling . 51 3.4.2 Asymptotic sum capacity for fixed number of antennas . 53 3.4.3 Scaling the number of antennas . 54 3.5 D2D Cooperation for Clustered Full-Duplex Networks . 55 3.5.1 Heterogeneous model . 55 3.5.2 Sum rate upper bound and the gap from the decoupled system capacity 56 3.5.3 Potential for cooperation over side-channels . 58 4 Opportunistic D2D Cooperation in Cellular Networks ::::::::::: 60 4.1 Introduction . 60 4.2 Related Work and Contributions . 62 4.3 System Architecture and Model . 65 4.3.1 Overview of the architecture . 65 vii 4.3.2 D2D link model and conflict graph . 68 4.3.3 D2D transmission queues . 69 4.3.4 Problem formulation . 70 4.4 Physical-layer Cooperation . 71 4.4.1 Cooperation strategy . 72 4.4.2 Cooperation with MU-MIMO . 74 4.4.3 Scaling of SNR gain in clustered networks . 75 4.5 Scheduling for Dynamic Networks Under D2D Cooperation . 77 4.5.1 Utility maximization formulation . 79 4.5.2 Stability Region Structure . 80 4.5.3 Optimal scheduling . 81 4.5.4 Greedy implementation . 82 4.5.5 Choice of utility function . 83 4.5.6 Proof outline of Theorem 4.4 . 84 4.6 Numerical Results . 86 4.6.1 Simulation setup . 86 4.6.2 Throughput distribution for regular cells . 88 4.6.3 Throughput distribution for heterogeneous networks . 89 4.6.4 Relaying cost . 90 4.6.5 D2D link intermittence . 90 4.6.6 Co-existence with WiFi . 91 4.6.7 Number of streams scheduled . 93 4.6.8 Relaxing the stability constraint . 94 4.6.9 Effect of clustering . 95 viii 4.6.10 Co-existence with WiFi off-loading . 96 5 Encoded Distributed Optimization I: Node Failures ::::::::::::: 97 5.1 Introduction . 97 5.2 Model and Notation . 100 5.3 Encoded Distributed Convex Programs . 102 5.3.1 Equiangular tight frames . 102 5.3.2 Random coding . 106 5.4 Lower Bound for Unconstrained Optimization . 106 5.5 Numerical Results . 107 5.5.1 Ridge regression . 108 5.5.2 Binary SVM classification . 108 5.5.3 Low-rank approximation . 109 6 Encoded Distributed Optimization II: Node Delays ::::::::::::: 111 6.1 Introduction . 111 6.2 Related work . 113 6.3 Encoded Distributed Optimization for Straggler Mitigation . 115 6.3.1 Data parallelism . 117 6.3.2 Model parallelism . 121 6.4 Convergence Analysis . 124 6.4.1 A spectral condition . 124 6.4.2 Convergence of encoded gradient descent . 125 6.4.3 Convergence of encoded L-BFGS . 127 6.4.4 Convergence of encoded proximal gradient . 128 ix 6.4.5 Convergence of encoded block coordinate descent . 129 6.5 Code Design . 130 6.5.1 Block RIP condition and code design . 130 6.5.2 Efficient encoding . 133 6.5.3 Cost of encoding . 136 6.6 Numerical Results . 136 6.6.1 Ridge regression . 137 6.6.2 Matrix factorization . 138 6.6.3 Logistic regression . 140 6.6.4 LASSO . 142 7 Conclusions and Open Problems ::::::::::::::::::::::::: 144 A Proofs for Chapter 2 :::::::::::::::::::::::::::::::: 146 A.1 Proof of Lemma 2.1 . 146 A.2 Proofs of Lemmas 2.2, 2.3, 2.4 and 2.5 . 148 A.2.1 Notation . ..