Machine learning for long-distance quantum communication Julius Walln¨ofer,1, 2 Alexey A. Melnikov,2, 3, 4 Wolfgang D¨ur,2 and Hans J. Briegel2, 5 1Department of Physics, Freie Universit¨atBerlin, Arnimallee 14, 14195 Berlin, Germany 2Institute for Theoretical Physics, University of Innsbruck, Technikerstraße 21a, 6020 Innsbruck, Austria 3Department of Physics, University of Basel, Klingelbergstrasse 82, 4056 Basel, Switzerland 4Valiev Institute of Physics and Technology, Russian Academy of Sciences, Nakhimovskii prospekt 36/1, 117218 Moscow, Russia 5Department of Philosophy, University of Konstanz, Fach 17, 78457 Konstanz, Germany (Dated: September 18, 2020) Machine learning can help us in solving problems in the context big data analysis and classifica- tion, as well as in playing complex games such as Go. But can it also be used to find novel protocols and algorithms for applications such as large-scale quantum communication? Here we show that machine learning can be used to identify central quantum protocols, including teleportation, entan- glement purification and the quantum repeater. These schemes are of importance in long-distance quantum communication, and their discovery has shaped the field of quantum information pro- cessing. However, the usefulness of learning agents goes beyond the mere re-production of known protocols; the same approach allows one to find improved solutions to long-distance communication problems, in particular when dealing with asymmetric situations where channel noise and segment distance are non-uniform. Our findings are based on the use of projective simulation, a model of a learning agent that combines reinforcement learning and decision making in a physically motivated framework. The learning agent is provided with a universal gate set, and the desired task is specified via a reward scheme. From a technical perspective, the learning agent has to deal with stochastic environments and reactions. We utilize an idea reminiscent of hierarchical skill acquisition, where solutions to sub-problems are learned and re-used in the overall scheme. This is of particular im- portance in the development of long-distance communication schemes, and opens the way for using machine learning in the design and implementation of quantum networks. I. INTRODUCTION modern artificial intelligence [16{18]. By using projec- tive simulation (PS) [19], a physically motivated frame- work for RL, we show that teleportation, entanglement Humans have invented technologies with transform- swapping, and entanglement purification are found by a ing impact on society. One such example is the inter- PS agent. We equip the agent with a universal gate set, net, which significantly influences our everyday life. The and specify the desired task via a reward scheme. With quantum internet [1,2] could become the next genera- certain specifications of the structure of the action and tion of such a world-spanning network, and promises ap- percept spaces, RL then leads to the re-discovery of the plications that go beyond its classical counterpart. This desired protocols. Based on these elementary schemes, includes e.g. distributed quantum computation, secure we then show that such an artificial agent can also learn communication or distributed quantum sensing. Quan- more complex tasks and discover long-distance commu- tum technologies are now at the brink of being com- nication protocols, the so-called quantum repeaters [12]. mercially used, and the quantum internet is conceived The usage of elementary protocols learned previously is of as one of the key applications in this context. Such central importance in this case. We also equip the agent quantum technologies are based on the invention of a with the possibility to call sub-agents, thereby allowing number of central protocols and schemes, for instance for a design of a hierarchical scheme [20, 21] that offers quantum cryptography [3{7] and teleportation [8]. Ad- the flexibility to deal with various environmental situa- ditional schemes that solve fundamental problems such as arXiv:1904.10797v2 [quant-ph] 17 Sep 2020 tions. The proper combination of optimized block actions the accumulation of channel noise and decoherence have discovered by the sub-agents is the central element at been discovered and have also shaped future research. this learning stage, which allows the agent to find a scal- This includes e.g. entanglement purification [9{11] and able, efficient scheme for long-distance communication. the quantum repeater [12] that allow for scalable long- We are aware that we make use of existing knowledge distance quantum communication. These schemes are in the specific design of the challenges. Rediscovering considered key results whose discovery represent break- existing protocols under such guidance is naturally very throughs in the field of quantum information processing. different from the original achievement (by humans) of But to what extent are human minds required to find conceiving of and proposing them in the first place, an such schemes? essential part of which includes the identification of rele- Here we show that many of these central quantum pro- vant concepts and resources. However, the agent does not tocols can in fact be found using machine learning by only re-discover known protocols and schemes, but can phrasing the problem in a reinforcement learning (RL) go beyond known solutions. In particular, we find that in framework [13{15], the framework at the forefront of asymmetric situations, where channel noise and decoher- 2 ence are non-uniform, the schemes found by the agent II. PROJECTIVE SIMULATION FOR outperform human-designed schemes that are based on QUANTUM COMMUNICATION TASKS known solutions for symmetric cases. In this paper the process of designing quantum commu- From a technical perspective, the agent is situated in nication protocols is viewed as a reinforcement learning stochastic environments [13, 14, 22], as measurements (RL) problem. RL, and more generally machine learn- with random outcomes are central elements of some of ing (ML), is becoming increasingly more useful in au- the schemes considered. This requires to learn proper re- tomation of problem-solving in quantum information sci- actions to all measurement outcomes, e.g., the required ence [25{27]. First, ML has been shown to be capable of correction operations in a teleportation protocol depend- designing new quantum experiments [24, 28{30] and new ing on outcomes of (Bell) measurements. Additional ele- quantum algorithms [31, 32]. Next, by bridging knowl- ments are abort operations, as not all measurement out- edge about quantum algorithms with actual near-term comes lead to a situation where the resulting state can experimental capabilities, ML can be used to identify be further used. This happens for instance in entangle- problems in which quantum advantage over a classical ap- ment purification, where the process needs to be restarted proach can be obtained [33{35]. Then, ML is used to re- in some cases as the resulting state is no longer entan- alize these algorithms and protocols in quantum devices, gled. The overall scheme is thus probabilistic. These by autonomously learning how to control [36{38], error- are new challenges that have not been treated in pro- correct [39{42], and measure quantum devices [43]. Fi- jective simulation before, but the PS agent can in fact nally, given experimental data, ML can reconstruct quan- deal with such challenges. Another interesting element tum states of physical systems [44{46], learn a compact is the usage of block actions that have been learned representation of these states and characterize them [47{ previously. This is a mechanism similar to hierarchical 49]. skill learning in robotics [20, 21], and to clip composi- Here we propose learning quantum communication tion in PS [19, 23, 24], where previously learned tasks protocols by a trial and error process. This process is are used to solve more complex challenges and problems. visualized in Fig.1 as an interaction between an RL Here we use this concept for long-distance communica- agent and its environment: by trial and error the agent tion schemes. The initial situation is a quantum chan- is manipulating quantum states hence constructing com- nel that has been subdivided by multiple repeater sta- munication protocols. At each interaction step the RL tions that share entangled pairs with their neighboring agent perceives the current state of the protocol (envi- stations. Previously learned protocols, namely entangle- ronment) and chooses one of the available operations (ac- ment swapping and entanglement purification, are used tions). This action modifies the previous version of the as new primitives. Additionally, the agent is allowed to protocol and the interaction step ends. In addition to employ sub-agents that operate in the same way but deal the state of the protocol the agent gets feedback at each with a problem at a smaller scale, i.e. they find optimized interaction step. This feedback is specified by a reward block actions for shorter distances that the main agent function, which depends on the specific quantum com- can employ at the larger scale. This allows the agent munication task a)-d) in Fig.1. A reward is interpreted to deal with big systems, and re-discover the quantum by the RL agent and its memory is updated. repeater with its favorable scaling. The ability to del- The described RL approach is used for two reasons. egate is of special importance in asymmetric situations First, there is a similarity between a target quantum com- as such block actions need to be learned separately for munication protocol and a typical RL target. A target different initial states of the environment { in our case quantum communication protocol is a sequence of ele- the fidelity of the elementary pairs might vary drastically mentary operations leading to a desired quantum state, either because they correspond to segments with differ- whereas a target of an RL agent is a sequence of actions ent channel noise, or they are of different length.