Temporal Network Embedding with Micro- and Macro-Dynamics
Total Page:16
File Type:pdf, Size:1020Kb
Temporal Network Embedding with Micro- and Macro-dynamics Yuanfu Lu Xiao Wang Chuan Shi∗ Beijing University of Posts and Beijing University of Posts and Beijing University of Posts and Communications Communications Communications Beijing, China Beijing, China Beijing, China [email protected] [email protected] [email protected] Philip S. Yu Yanfang Ye University of Illinois at Chicago Case Western Reserve University IL, USA OH, USA [email protected] [email protected] ABSTRACT ACM Reference Format: Network embedding aims to embed nodes into a low-dimensional Yuanfu Lu, Xiao Wang, Chuan Shi, Philip S. Yu, and Yanfang Ye. 2019. Tem- space, while capturing the network structures and properties. Al- poral Network Embedding with Micro- and Macro-dynamics. In The 28th ACM International Conference on Information and Knowledge Management though quite a few promising network embedding methods have (CIKM ’19), November 3–7, 2019, Beijing, China. ACM, New York, NY, USA, been proposed, most of them focus on static networks. In fact, 10 pages. https://doi.org/10.1145/3357384.3357943 temporal networks, which usually evolve over time in terms of microscopic and macroscopic dynamics, are ubiquitous. The micro- 1 INTRODUCTION dynamics describe the formation process of network structures in a detailed manner, while the macro-dynamics refer to the evolution Network embedding has shed a light on network analysis due pattern of the network scale. Both micro- and macro-dynamics to its capability of encoding the structures and properties of net- are the key factors to network evolution; however, how to ele- works with latent representations [1, 3]. Though the state-of-the- gantly capture both of them for temporal network embedding, arts [5, 9, 21, 22, 25, 28] have achieved promising performance in especially macro-dynamics, has not yet been well studied. In this many data mining tasks, most of them focus on static networks with paper, we propose a novel temporal network embedding method fixed structures. In reality, the network usually exhibits complex with micro- and macro-dynamics, named M2DNE. Specifically, for temporal properties, meaning that the network structures are not micro-dynamics, we regard the establishments of edges as the oc- achieved overnight and usually evolve over time. In the so-called currences of chronological events and propose a temporal attention temporal networks [13], the establishments of edges between nodes point process to capture the formation process of network struc- are chronological and the network scale grows with some obvious tures in a fine-grained manner. For macro-dynamics, we define a distribution. For example, researchers collaborate with others in dif- general dynamics equation parameterized with network embed- ferent years, leading to sequential co-author events and continued dings to capture the inherent evolution pattern and impose con- growth of the network scale. Therefore, a temporal network natu- straints in a higher structural level on network embeddings. Mutual rally represents the evolution of a network, including not only the evolutions of micro- and macro-dynamics in a temporal network al- fine-grained network structure but also the macroscopic network ternately affect the process of learning node embeddings. Extensive scale. Embedding a temporal network with the latent representation experiments on three real-world temporal networks demonstrate space is of great importance for applications in practice. that M2DNE significantly outperforms the state-of-the-arts not Basically, one requirement for temporal network embedding is only in traditional tasks, e.g., network reconstruction, but also in that the learned embeddings should preserve the network structure temporal tendency-related tasks, e.g., scale prediction. and reflect its temporal evolution. The temporal network evolution usually follows two dynamics processes, i.e., the microscopic and arXiv:1909.04246v1 [cs.LG] 10 Sep 2019 KEYWORDS macroscopic dynamics. At the microscopic level, the temporal net- work structure is driven by the establishments of edges, which is Network Embedding, Temporal Network, Social Dynamics Analysis, actually a sequence of chronological events involving two nodes. Temporal Point Process Taking Figure 1 as an example, from time ti to tj , the formation ∗Corresponding author. of the structure can be described as f¹v ;v ; t º, ¹v ;v ; t º, · · · g ::: 3 1 i 3 2 i −−!f¹v3;v1; tj º; ¹v3;v4; tj º; · · · g, and here nodes v3 and v1 build a Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed link again at tj . Usually, an edge generated at time t is inevitably for profit or commercial advantage and that copies bear this notice and the full citation related with the historical neighbors before t, and the influence on the first page. Copyrights for components of this work owned by others than ACM of the neighbor structures on the edge formation comes from the must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a two-way nodes, not just a single one. Besides, different neighbor- fee. Request permissions from [email protected]. hoods may have distinct influences. For example, in Figure 1,the CIKM ’19, November 3–7, 2019, Beijing, China establishment of edge ¹v3;v4º at time tj should be influenced by © 2019 Association for Computing Machinery. ACM ISBN 978-1-4503-6976-3/19/11...$15.00 {¹v3;v1; ti º, ¹v3;v2; ti º; · · · g and f¹v4;v2; ti º; · · · g. Besides, the influ- https://doi.org/10.1145/3357384.3357943 ences of ¹v3;v2; ti º and ¹v4;v2; ti º on the event ¹v3;v4; tj º should CIKM ’19, November 3–7, 2019, Beijing, China Yuanfu Lu, Xiao Wang, Chuan Shi, Philip S. Yu, and Yanfang Ye Number of edges in a temporal network (i.e., micro-dynamics), we elaborately de- sign a temporal attention point process by parameterizing the condi- tional intensity function with node embeddings, which captures the fine-grained structural and temporal properties with a hierarchical temporal attention. To model the evolution pattern of the temporal network scale (i.e., macro-dynamics), we define a general dynamics 5 equation as a non-linear function of the network embedding, which 1 1 2 … 2 … … … imposes constraints on the network embedding at a high structural 3 4 3 4 level and well couples the dynamics analysis with representation Time learning on temporal networks. At last, we combine micro- and … … t … … <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> ti <latexit sha1_base64="(null)">(null)</latexit> j tk macro-dynamics preserved embedding and optimize them jointly. Figure 1: A toy example of temporal network with micro- As micro- and macro-dynamics mutually evolve and alternately and macro-dynamics. Micro-dynamics describe the forma- influence the process of learning node embeddings, the proposed 2 tion process of network structures (i.e., edge establish- M DNE has the capability to capture the formation process of topo- ments), while macro-dynamics refer to the evolution pat- logical structures and the evolutionary pattern of network scale in a tern of network scale (i.e., the number of edge). The red lines unified manner. We will make our code and data publicly available at website after the review. in the network mean new temporal edges (e.g., ¹v3;v4; tj º). The major contributions of this work can be summarized as follows: be larger than that of ¹v ;v ; t º, since nodes v ;v and v form 3 1 i 2 3 4 • For the first time, we study the important problem of incorpo- a closed triad [10, 34]. Such micro-dynamic evolution process de- rating the micro-dynamics and macro-dynamics into temporal scribes the edge formation between nodes at different timesteps in network embedding. detail and explains that “why the network evolves into such struc- • We propose a novel temporal network embedding method (M2DNE), tures at time t”. Modeling the micro-dynamics enables the learned which microscopically models the formation process of network node embeddings to capture the evolution of a temporal network structure with a temporal attention point process, and macro- more accurately, which will be beneficial for the downstream tem- scopically constrains the network structure to obey a certain poral network tasks. We notice that temporal network embedding evolutionary pattern with a dynamics equation. has been studied by some works [6, 26, 34, 37]. However, they either • We conduct comprehensive experiments to validate the benefits simplify the evolution process as a series of network snapshots, of M2DNE on the traditional applications (e.g., network recon- which cannot truly reveal the formation order of edges; or model struction and temporal link prediction), as well as some novel neighborhood structures using stochastic processes, which ignores applications related to temporal networks (e.g. scale prediction). the fine-grained structural and temporal properties. More importantly, at the macroscopic level, another salient prop- erty of the temporal network is that the network scale evolves with 2 RELATED WORK obvious distributions over time, e.g., S-shaped sigmoid curve [12] Recently, network embedding has attracted considerable attention or a power-law like pattern [32]. As shown in Figure 1, when the [3]. Inspired by word2vec [18], random walk based methods[9, network evolves over time, the edges are continuously being built 21] have been proposed to learn node embeddings by the skip- and form the network structures at each timestamp. Thus, the net- gram model. After that, [22, 30] are designed to better preserve work scale, i.e., the number of edges, grows with time and obeys a network properties, e.g.