Machine Learning for Wireless Link Adaptation Supervised and Reinforcement Learning Theory and Algorithms VIDIT SAXENA Doctoral Thesis in Electrical Engineering Stockholm, Sweden, 2021 KTH Royal Institute of Technology School of Electrical Engineering and Computer Science Division of Information Science and Engineering TRITA-EECS-AVL-2021:35 SE-10044 Stockholm ISBN 978-91-7873-886-1 Sweden Akademisk avhandling som med tillst˚and av Kungl Tekniska h¨ogskolan framl¨agges till o↵entlig granskning f¨or avl¨aggande av Teknologie doktorexamen i elektroteknik torsdagen den 20 maj 2021 klockan 13.00 i Sal F3, Lindstedtsv¨agen 26, Kungliga Tekniska H¨ogskolan, Stockholm. Academic thesis which, with permission of the KTH Royal Institute of Technology, is submitted for public scrutiny for the completion of the Ph.D. in Electrical Engi- neering on Thursday May 20, 2021 at 13.00 in the lecture hall F3, Lindstedtsv¨agen 26, KTH Royal Institute of Technology, Stockholm. c Vidit Saxena, April 28, 2021 Tryck: Universitetsservice US AB i Abstract Wireless data communication is a complex phenomenon. Wireless links encounter random, time-varying, channel e↵ects that are challenging to pre- dict and compensate. Hence, to optimally utilize the channel, wireless links adapt the data transmission parameters in real time. This process, known as wireless link adaptation, can lead to large gains in link performance. Link adaptation is hence an integral part of state-of-the-art wireless deployments. Existing link adaptation schemes use simple heuristics that match the data transmission rate to the estimated channel. These schemes have proven to be useful for the ubiquitous wireless services of voice telephony and mo- bile broadband. However, as wireless networks increase in complexity and also evolve to support new service types, these link adaptation schemes are rapidly becoming inadequate. The reason for this change is threefold: first, in several operating scenarios, simple heuristics-based link adaptation does not fully exploit the available channel. Second, the heuristics are typically tuned empirically for good performance, which incurs additional expense and can be error-prone. Finally, traditional link adaptation does not naturally extend to applications beyond the traditional wireless services, for example to industrial control or vehicular communications. In this thesis, we address wireless link adaptation through machine learn- ing. Our proposed solutions efficiently navigate the link parameter space by learning from the available information. These solutions thus improve the link performance compared to the state-of-the-art, for example by doubling the link throughput. Further, we advance link adaptation support for new wireless services by optimizing the link for complex performance objectives. Finally, we also introduce mechanisms that autonomously tune the link adap- tation parameters with respect to the operating environment. Our schemes hence mitigate the dependence on empirical configurations adopted in current wireless networks. This thesis is composed of six technical papers. Based on these papers, there are three key contributions of this thesis: a neural link adaptation model (Paper I, Paper II,andPaper III), link adaptation under packet error rate constraints (Paper IV and Paper V), and efficient model-based link adaptation (Paper VI). In this thesis, we emphasise the theoretical underpinnings of our pro- posed machine learning schemes for link adaptation. We approach this goal in three ways: First, we make theoretically reasoned choices for machine learn- ing models and learning algorithms for link adaptation. Second, we extend these models for the specific problem formulations encountered in link adap- tation. For this, we develop rigorous problem formulations that are analyzed using classical techniques. Third, we develop theoretical results for the real- time behaviour of the proposed schemes. These bounds extend the machine learning state-of-the-art in terms of performance bounds for stochastic online optimization. The contributions of this thesis hence go beyond the realm of wireless optimization, and extend to new developments applicable to broader machine learning problems. ii Keywords: Wireless Communications, Reinforcement Learning, Multi-Armed Bandits, Thompson Sampling, Convex Optimization, Deep Learning. iii Sammanfattning Tr˚adl¨os datakommunikation ¨ar ett komplext fenomen. Tr˚adl¨osa l¨ankar st¨oter p˚aslumpm¨assiga och tidsvarierande kanale↵ekter som ¨ar utmanande att f¨oruts¨aga och kompensera f¨or. F¨or att optimalt utnyttja den tr˚adl¨osa kanalen anpassar d¨arf¨or kommunikationssystem data¨overf¨oringsparametrarna i realtid. Denna process, ¨aven kallad tr˚adl¨os l¨ankanpassning, kan leda till stora vinster i l¨ankprestanda. L¨ank-anpassning ¨ar d¨arf¨or en integrerad del av alla moderna kommunikationssystem. Befintliga metoder f¨or l¨ankanpassning anv¨ander enkla heuristiker som an- passar data¨overf¨oringshastigheten till den skattade tr˚adl¨osa kanalen. Dessa system har visat sig vara anv¨andbara f¨or de brett anv¨anda tr˚adl¨osa tj¨ansterna r¨osttelefoni och mobilt bredband. Eftersom tr˚adl¨osa n¨atverk ¨okar i komplexi- tet och ocks˚autvecklas f¨or att st¨odja nya tj¨anstetyper, blir dock dessa meto- der f¨or l¨ankanpassning snabbt otillr¨ackliga. Anledningen till detta ¨ar trefaldig: F¨or det f¨orsta s˚autnyttjar heuristikbaserad l¨ankanpassning i flera nya tj¨anster utnyttjar helt enkelt inte den tillg¨angliga kanalen till fullo. F¨or det andra s˚a ¨ar heuristiken vanligtvis anpassad empiriskt f¨or bra prestanda, vilket kan va- ra felben¨aget i nya scenarion och vilket medf¨or extra kostnader. Slutligen s˚a generaliserar traditionell l¨ankanpassning inte naturligt till till¨ampningar som g˚ar ut¨over de traditionella tr˚adl¨osa tj¨ansterna, till exempel till industriella reglersystem eller fordonskommunikation. Idennaavhandlingbehandlarvil¨ankanpassning genom maskininl¨arning. V˚ara f¨oreslagna system utforskar e↵ektivt l¨ankparameterutrymmet genom att l¨ara av tillg¨anglig information. De f¨oreslagna metoderna f¨orb¨attrar s˚aledes l¨ankprestandan j¨amf¨ort med den senaste tekniken, till exempel genom att f¨ordubbla l¨ankgenomstr¨omningen. Vidare utvecklar vi ocks˚al¨ankadaptationsst¨od f¨or nya tr˚adl¨osa tj¨anster genom att optimera l¨anken f¨or mer komplexa prestan- dam˚al. Slutligen s˚aintroducerar vi ocks˚amekanismer som autonomt justerar l¨ankanpassningsparametrarna baserat p˚adriftsmilj¨on. V˚ara system mildrar d¨armed beroendet p˚aempiriska konfigurationer som anv¨ands i nuvarande tr˚adl¨osa n¨atverk. Denna avhandling best˚ar av sex tekniska artiklar. Baserat p˚adessa artik- lar finns det tre viktiga bidrag fr˚an denna avhandling: En modell f¨or anpass- ning av neurala l¨ankar (Paper I, Paper II och Paper III), l¨ankanpassning under begr¨ansningar i paketfelfrekvensen (Paper IV och Paper V), och e↵ektiv modellbaserad l¨ankanpassning (Paper VI). I denna avhandling betonar vi den teoretiska grunden f¨or v˚ara f¨oreslagna maskininl¨arningsmetoder f¨or l¨ankanpassning. Vi n¨armar oss detta m˚al p˚atre s¨att: F¨or det f¨orsta g¨or vi teoretiskt motiverade val f¨or maskininl¨arningsmodeller och inl¨arningsalgoritmer f¨or l¨ankanpassning. F¨or det andra ut¨okar vi dessa modeller f¨or de specifika problemformuleringar som p˚atr¨a↵as vid l¨ankanpassning. F¨or detta utvecklar vi noggranna problemformuleringar som analyseras med klassiska tekniker. F¨or det tredje utvecklar vi teoretiska resultat f¨or de f¨oreslagna systemens realtidsbeteende. Dessa gr¨anser ut¨okar f¨altet maskininl¨arningen n¨ar det g¨aller prestationsgr¨anser f¨or stokastisk online-optimering. Bidragen fr˚an denna avhandling g˚ar allts˚aut¨over omr˚adet f¨or tr˚adl¨os kommunikation och str¨acker sig till nya till¨ampningsomr˚aden. iv सारांश वायरलसडे टाे सचारं एक जटल ूबया ह।ै वायरलसे कड़याँ (लस)ं अयविःथत और बम-रहत चनलै ूभाव का सामना करतीं ह, िजनक तपतू कर पाना चनौतीपु णू ह।ै अतः, चनलै का सवम उपयोग करने के लए, वायरलसे लसं वाःतवक समय म डटाे सचारणं मापदडं (परामीटस)ै को अनकु ूलत करते ह। इस ूबया को वायरलसे लकं अनकु ूलन के नाम से जाना जाता ह,ै जो अयाधनकु वायरलसे परनयोजन का एक अभ अगहं ।ै मौजदाू लकं अनकु ूलन योजनाएं अनभवु पर आधारत, सरल, अनमानु का उपयोग करती ह। आमतौर स,ये े योजनांए डटाे सचारणं दर का अनमानतु वायरलसचे नलै से मले कराती ह । पवकालू म, ये योजनाएं दरभाषू और मोबाइल ॄॉडबड क सवयापी वायरलससे वाओे ं के लए उपयोगी साबत हईु ह। कत,जु सै -जे सै े वायरलसने टवके जटल होते जा रहे ह , और नए ूकार क सचारणं -यवःथाएं वकसत हो रह ह, मौजदाू लकं अनकु ूलन योजनाएं भी तजीे से अपया होती चल जा रह ह। इस परवतन के यह तीन मयु कारण ह: पहला, कई परँय म, सरल लकं अनकु ूलन मौजदाू चनलै का परू तरह से उपयोग नहं कर पाता। दसरा,ू सचारणं मापदडं को सामायतः आनभावकु प से चनाु जाता ह,ै जो अतर सचरणं को बढ़ाता है और इसम ऽटयु क सभावना अधक होती ह।अै ततः,पारं ंपरक लकं अनकु ूलन नयी सवा-ूयोगे क ओर ःवाभावक प से वःतार नहं करता - उदाहरण के लए, औोगक नयऽणं अथवा वाहन-आधारत सचार।ं इस शोध ूबधं (थीसस) म, हम मशीन लनग के मायम से वायरलसे लकं अनकु ूलन का अनसु धानं करते ह। हमारे ूःतावत समाधान सामाय सचारणं जानकार से सीखकर, लकसं चरणं मापदडं का ःवतः और कुशलतापवकू सालन करते ह। अयाधनकु अनकु ूलन वधय क तलनाु म, हमारे समाधान लकं नंपादन (परफॉरमस) म सधारु करते ह, उदाहरण के लए लकं ूवाह मता (ापू ट)ु को दोगनाु करके । इसके अतर,हमारे समाधान जटल नंपादन उेँय के लए लकं को अनकु ूलत करके नई वायरलससे वाओे ं को लाभ पहचातंु े ह। अतं म, हम ऐसी तकनीक भी ूःततु करते ह जो वायरलसे वातावरण के आधार पर, लकं अनकु ूलन मापदडं को ःवतः सचालतं करती ह।ै इस ूकार, हमार ूःतावत योजनाएं आज के वायरलसने टवके क अनभवजयु नभरता कम करती ह। इस थीसस म छह तकनीक