Inference for Network Structure and Dynamics from Time Series Data Via Graph Neural Network
Total Page:16
File Type:pdf, Size:1020Kb
Inference for Network Structure and Dynamics from Time Series Data via Graph Neural Network Mengyuan Chen Jiang Zhang* Zhang Zhang Lun Du School of Systems Science School of Systems Science School of Systems Science Data Knowledge Intelligence Group Beijing Normal University Beijing Normal University Beijing Normal University Microsoft Research BeiJing, China BeiJing, China BeiJing, China BeiJing, China [email protected] [email protected] [email protected] [email protected] Qiao Hu Shuo Wang Jiaqi Zhu Technology Group School of Systems Scienc School of Systems Scienc Swarma Campus (Beijing) Technology Beijing Normal University Beijing Normal University BeiJing, China BeiJing, China BeiJing, China [email protected] [email protected] [email protected] Abstract—Network structures in various backgrounds play individual development for young scientists [6, 7]. However, important roles in social, technological, and biological systems. the data of network structure is always incomplete or even However, the observable network structures in real cases are unavailable either because measuring binary links is costly or often incomplete or unavailable due to the measurement errors or private protection issues. Therefore, inferring the complete the data of weak ties is missing [8, 9, 5]. Therefore, it is urgent network structure is useful for understanding complex systems. to find a way to infer the complete network structure according The existing studies have not fully solved the problem of inferring to non-structural information [10, 11]. network structure with partial or no information about connec- Link prediction, as the traditional task in network science, tions or nodes. In this paper, we tackle the problem by utilizing tries to infer the lost links in network structure according to time series data generated by network dynamics. We regard the network inference problem based on dynamical time series the linking patterns of existing connections [12, 13]. Although data as a problem of minimizing errors for predicting future numerous algorithms have been developed to complete the states and proposed a novel data-driven deep learning model missing links of a large network with high accuracy [14, 15], called Gumbel Graph Network (GGN) to solve the two kinds all of these approaches require the complete nodes information of network inference problems: Network Reconstruction and but it is always unavailable in practice [8, 16]. link prediction Network Completion. For the network reconstruction problem, the GGN framework includes two modules: the dynamics learner cannot solve the inference problem under the condition that and the network generator. For the network completion problem, the network contains unobservable nodes. In real cases, we GGN adds a new module called the States Learner to infer can either obtain nodes information of the only partial network missing parts of the network. We carried out experiments on or without any information about links [17, 18], as a result, discrete and continuous time series data. The experiments show conventional link prediction algorithms cannot work. that our method can reconstruct up to 100% network structure on the network reconstruction task. While the model can also Network completion methods have been developed in re- infer the unknown parts of the structure with up to 90% accuracy cent years trying to tackle the first problem we will discuss when some nodes are missing. And the accuracy decays with the in the paper, that is, to infer the missing connections on increase of the fractions of missing nodes. Our framework may unobserved nodes according to the linking patterns between have wide application areas where the network structure is hard arXiv:2001.06576v1 [cs.LG] 18 Jan 2020 observable nodes. The methods can be categorized into tradi- to obtained and the time series data is rich. Index Terms—Network Inference, Network Reconstruction, tional statistical-based methods and graph neural network liked Network Completion, Graph Network, Time Series methods. Give examples of traditional methods represented by expectation maximum(EM) algorithm, Myunghwan Kim I. INTRODUCTION and Jure Leskovec applied Kronecker graph model and EM A complex system is an integrated system with many parts, to complete the network according to the observed linking and the emergent behaviors of the entire system are determined patterns [19]. Although their algorithms obtain a relatively by how these parts interact [1], i.e., the network structure. For high accuracy of recovering missing links, there is an implicit example, the topology of a social network determines how requirement, the underlying network structure is required to fast the opinions or ideas can spread in a social media [2, 3]; follow the self-similar property as possible, which is violated the structure of a supply chain network between companies by some network [20]. Following the same expectation max- influences the safety of the whole market because risk may imum(EM) algorithm, in a recent NC work, Xue, Yuankun propagate along the links [4, 5]; the topology of the cooper- and Bogdan, Paul developed a causal inference method to ation network plays critical role for scientific innovation and recover the complete network structure from the adversarial interventions [21]. On the other hand, as the booming devel- are known or unknown. Both problems are formulated as opment of deep learning on graphs [22, 23, 24], researchers the same kind of optimization problem, which is to find an applied graph convolution network (GCN) liked models on optimized network structure and the approximator of the net- network completion problems. Da Xu et al regard the complete work dynamics such that the errors between the observed time network as the growth of the partial network [25], then they series and the generated time series according to the candidate train the GCN to learn the growing process with the partially network structure and dynamical rules is minimized. Both observed network and generalized to the complete unknown problems are solved in the same framework called Gumbel- network structure. Cong Tran, Won-Yong Shin, and Andreas Graph-Network (GGN) [38], which is a combination of the Spitz et al solve the problem by training a graph generating network generator based on Gumbel softmax sampling and the model to learn the connection patterns among a large set of dynamics learner based on Graph Network. Gumbel softmax similar graphs for training and applied the trained generating sampling is a technique to simulate the sampling process model to complete the missing information [16]. All of these with a differentiable computation process. Equipped with this network completion methods depend on a partially observed technique, we can train a network generator with a gradient network structure because they try to discover the latent descent method. Graph Neural Network (GNN) is a new patterns of the observed connections and to infer the unknown deep learning architecture on graph. By learning a bunch of structures. Nevertheless, in some cases the network structures functions defined on nodes and links representing propagation are totally implicit and only some signals of partial nodes and aggregation, GNN can recover the complex dynamical can be observed, such as biological network [26] and social process defined on a graph like rigid body movement [39], network [8]. How can we infer the whole network structure coupled oscillators [38], traffic flows [40], pollution spreading without any information of connection patterns? [41], etc. By deploying the powerful capability of graph neural In fact, time series data on nodes as another important infor- network in learning, we can simulate the underlying dynamics mation source [27, 28], which is always available in practice, based on the real network. is ignored by mentioned previous works. For example, in an online social network, we can only observe the discrete retweet II. NETWORK INFERENCE METHODOLOGIES events between a large set of users, neither their features In this paper, we focus on the network inference problems of like sex, education, etc. and their connection information is the structure and dynamic based on states evolution time series unavailable; in a stock market, all the information that we of all or partial nodes. According to the two different applica- can obtain is the prices of different stocks, the connections tion scenarios, we divide the problem into two sub-problems: between the stocks are unknown. Thus, can we develop a (1) Network reconstruction problem: inferring interconnected method to infer the network structure according to the time structure and dynamic of network evolution with all nodes series data representing the nodes’states? A large number of observable; (2) Network completion problem: recovering the methods have been proposed for reconstructing network from structure and nodes’ states of the entire network based on time series data. One class of them is based on the method of partial network structure and time series data of the observable statistical inference such as Granger causality [29, 30], and nodes. In this section, we first introduce the formal definitions correlation measurements [31, 32, 33]. Even though, these of the two sub-problems and then describe the specific models methods may fail to reveal the structural connection. Another to solve the problems separately. class of methods was developed for reconstructing structural Suppose our studied system has an interaction structure connections directly under certain assumptions. For example, described by a binary graph G = (V; E) with an adjacency methods such as driving response [34] or compressed sensing matrix A, where V = fv1; :::; vN g is the set of nodes, or [14, 35, 36, 37] either require the functional form of the interchangeably referred to as vertices, and N is the total differential equations, or the target-specific dynamics, or the number of nodes, E = feijg is the set of edges between the sparsity of time series data. However, getting this information nodes, and A is a binary matrix of which each entry equals 0 is very difficult.