Machine Learning Tips and Tricks for Power Line Communications Andrea M. Tonello, Senior Member, IEEE, Nunzio A. Letizia, Davide Righini and Francesco Marcuzzi Abstract—A great deal of attention has been recently given to afterwards with an empirical and top-down approach. The Machine Learning (ML) techniques in many different application immense contributions of I. Kant with his philosophy of fields. This paper provides a vision of what ML can do in Power transcendental aesthetics and logic [2], made a step further into Line Communications (PLC). We firstly and briefly describe classical formulations of ML, and distinguish deterministic from the understanding and the definition of knowledge: knowledge statistical learning models with relevance to communications. We of the structure of time and space and their relationships, is a then discuss ML applications in PLC for each layer, namely, for priori knowledge; knowledge acquired from observations is a characterization and modeling, for the development of physical posteriori knowledge; most of our knowledge comes from the layer algorithms, for media access control and networking. process of learning and observing phenomena, and without a Finally, other applications of PLC that can benefit from the usage of ML, as grid diagnostics, are analyzed. Illustrative numerical priori knowledge it is impossible to reach the true knowledge. examples are reported to serve the purpose of validating the In this respect, Machine Learning (ML) [3], [4], which is ideas and motivate future research endeavors in this stimulating the topic of this paper, can be considered an implementation signal/data processing field. by humans of techniques in machines to acquire knowledge Index Terms—Machine learning, Statistical learning, Com- from a posteriori observations of natural phenomena. The munications, Power line communications, Channel modeling, Physical layer, MAC layer, Network layer, Grid diagnostics. origin of success of ML relies on its ability to derive relations among phenomena and potentially discover the hidden (latent) true I. INTRODUCTION state of a system, i.e, potentially provide an intrinsic knowledge of the system. System identification and model Modern communication systems have reached a high degree based design through the aid of ML [5] constitute a first step of performance, meeting demanding requirements in numerous to find undiscovered system properties via a mixed a priori - application fields. Significant progress in the analysis and de- a posteriori learning approach, which, retrospectively, follows sign of communication systems has been rendered possible by Kant’s philosophical structure. the milestone work of C. Shannon [1] that provided a method- ML is indeed bringing new lymph in the domain of commu- ological approach to attack the challenge of reliably trans- nication systems modeling, design, optimization, and manage- mitting information through a given communication mean. ment. It provides a paradigm shift: rather than concentrating Shannon’s mathematical approach suggests to represent the on a physical bottom-up description of the communication system as a chain of blocks mathematically modeled, namely, scheme, ML aims to learn and capture information from a the transmitter, the channel, and the receiver. The transmitter collection of data, to derive the input-output relations of the can be further divided into a source coder, a channel coder and system, or of a given task in the system. We would argue a signal modulator. The channel is represented by a transfer that a miraculous solution of communications challenges with function (in most cases considered linear time invariant, or ML does not yet exist. In addition, what is learned via ML time variant) and an additive noise term. Three generations tools is not necessarily representative of the physical reality, of scientists and engineers grew up with this mathematical i.e., wrong believes about relations and dependencies among mindset which provided tools to acquire domain knowledge data may be generated. Consequently, the results have to be and use it to build a model for each block, so that the overall validated through the support of a probabilistic approach and arXiv:1904.11949v2 [eess.SP] 6 Jun 2019 behavior becomes known. Such a framework intrinsically has an understanding of the system physics. But the path has been the advantage that each block can be individually studied and mapped out: ML offers a great deal of opportunities to research optimized. We would refer to this approach as physical and and design communication systems. bottom-up. From an epistemological point of view, the mathematical A. But what are the domains of application of ML in commu- theory of communications is based on knowledge coming nications? from a priori justifications and relying on intuitions and The applications of ML in communications are a multitude the nature of these intuitions, which is intrinsically what and cover all three fundamental protocol stacks: the physical mathematics does. On the contrary, a posteriori knowledge is layer, the MAC layer and the network layer. More specifically, created by what is known from experience, therefore generated the same applies to Power Line Communications (PLC) which is the technology that exploits the existing power deliver The authors are with the University of Klagenfurt - Chair of Embedded Communication Systems, 9020 Klagenfurt, Austria. (e-mail: fandrea.tonello, infrastructure to convey information signals [6]. When things nunzio.letizia, davide.righini, [email protected]) become complex and a bottom-up model is difficult to derive 1 or has too many uncertainties, ML can help, no doubt. This is more in general system identification, data detection, resource particularly true in PLC since, despite the advances in channel allocation, network management etc. and noise modeling [7], the communication media is still In the following, we summarize learning methods and tools not fully understood and modeled especially when it comes and we report specific examples of ML based solutions already to noise and interference. Consequently, the transceiver tech- proposed in the literature using however a unified description niques designed so far might not be optimal [8]. Media access approach. control and resource allocation in massive PLC networks (as smart metering ones) is an extremely complex task that can A. Supervised Learning benefit from ML approaches [9]. The analysis of PLC signals 1) Preliminaries and Definitions: Let (xi; yi) ∼ p(x; y), exchanged among nodes can reveal properties of the grid status i = 1;:::;N, be samples collected into a training set D and detect anomalies in the cables, loads and generators, which belonging to the joint distribution (pdf) p(x; y). Supervised is relevant for grid predictive maintenance [10], [11]. learning, under a deterministic model, aims to find a mapping (x ; y ) B. Paper Contribution between all pairs of input-output vectors i i , thus, an inferred function F that element-wise satisfies y = F (x). The In this paper, we will discuss in detail the application of ideal scenario would map unseen samples ~x into the right, ML in PLC providing concrete insights (tips) of what can initially unknown, label/target ~y. As we want to address the be done and with what ML tools (tricks). Several numerical problem under a stochastic/probabilistic approach, we state examples are reported to validate the ideas and to stimulate that probabilistic supervised learning tries to predict y from x further work in this research domain. To better understand by estimating p(yjx) under a discriminative model or by esti- ML, we start our short journey into ML for PLC by proving mating the joint distribution p(x; y) under a generative model. a compact introduction to ML with focus on applications in Fig. 1 schematically distinguishes between deterministic and communications. This serves also the purpose of surveying, at probabilistic approaches in ML. the best of the authors knowledge, the existing literature on If the outputs are continuous variables, we consider it as the topic and the initial studies conducted. a regression problem, while if the targets are discrete, then In detail, the paper is organized as follows. In Sec. II, we have a classification problem. A standard way to proceed ML fundamentals for both supervised and unsupervised learn- during the learning process is to define a cost function C, ing are reported. Specific tools are described. They include namely a performance measure that evaluates the quality of artificial neural networks, and support vector machines (for our prediction ^y. In most applications, we can rely only on the supervised ML), and clustering, autoencoders, and generative observed dataset D and derive an empirical sample distribution networks (for unsupervised ML). Convolutional and Recurrent since we do not have knowledge of the true joint distribution Neural Networks as well as Reinforcement Learning are also p(x; y). In particular, the goal of the training process is to briefly discussed. In Sec. III, we focus on PLC and ML for the minimize characterization of the medium and its modeling through the C(^y) = [δ(y; ^y)] (1) use of a data driven, synthetic approach. In Sec. IV, ML for E(x;y)∼D physical layer PLC is discussed while the MAC and network where δ is a measure of distance between the wanted target y layers are considered in Sec. V. Other applications of ML for and the prediction ^y, and E denotes expectation. PLC are the topic of Sec. VI. The conclusions then follow. 2) Tools: Neural Networks: Neural Networks (NNs) are among the most popular tools in this field since they are II. MACHINE LEARNING BASICS FOR COMMUNICATIONS known being universal function approximators [14], they can Following the definition provided by Mitchell [12], ML al- be implemented in parallel on concurrent architectures and gorithms can be categorized according to the learning process most importantly, they can be trained by backpropagation [15]. (the kind of experience E the machine has), the specific task A feedforward neural network with L layers maps a given T and the performance measure P .