An Online Deep Learning Framework for Mobile Big Data to Understand Human Mobility Patterns
Total Page:16
File Type:pdf, Size:1020Kb
JOURNAL OF LATEX CLASS FILES, VOL. 13, NO. 9, SEPTEMBER 2014 1 DeepSpace: An Online Deep Learning Framework for Mobile Big Data to Understand Human Mobility Patterns Xi Ouyang, Student Member, IEEE, Chaoyun Zhangy, Student Member, IEEE, Pan Zhou, Member, IEEE and Hao Jiang, Member, IEEE Abstract—In the recent years, the rapid spread of mobile device has create the vast amount of mobile data. However, some shallow-structure algorithms such as support vector machine (SVM) has difficulty dealing with high dimensional data with the development of mobile network. In this paper, we analyze the mobile big data to predict human trajectories in order to understand human mobility pattern via a deep-structure model called “DeepSpace”. To the best of out knowledge, it is the first time that the deep learning approach is applied to predict human trajectories. Furthermore, we develop the vanilla convolutional neural network (CNN) to be an online learning system, which can deal with the continuous mobile data stream. Meanwhile, “DeepSpace” consists of two different prediction models corresponding to different scales in space (the coarse prediction model and fine prediction models). The two prediction models constitute our model hierarchically, so our model can run in parrel. Finally, we test our model using the data usage detail records (UDRs) from the mobile cellular network in a city of southeastern China, rather the call detail records (CDRs) widely used by others as usual. The experiment results show that “DeepSpace” is promising in human trajectories prediction. Index Terms—Mobile big data, deep learning, CNN, online learning, spatial data. F 1 INTRODUCTION VER the course of the past decade, mobile big data controlling. For example, Uber drivers can reduce waiting O has a explosive growth due to the fast development time for passengers if the company can provide the precise of various mobile devices (e.g., iPhone, Android phone, trajectory information of customers. A research team finds Google Glass, iWatch etc.). Global mobile data traffic grew a 93% potential predictability for each individual mobility by 69% in 2014, according to Cisco Visual Networking Index across the their whole research users [1]. This conclusion (VNI). Meanwhile, the mobile data traffic created in last is based on the reach from Chaoming Song and Albert- years was nearly 30 times bigger in terms of size than Laszl´ o´ Barabasi´ who study the mobility patterns of many the size of the entire global Internet in 2000. It is amazing anonymized mobile phone users. Their experiments show that global mobile data traffic will increase nearly tenfold that there are few freewheeling style men in the crowd. Al- between 2014 and 2019 based on the Cisco VNI’s forecast. At though people may have significant differences in the mov- the same time, with the rapid development of the Internet- ing patterns, their moving patterns are predictable for each of-Things (IoT), much more data is automatically generated individual. Besides, this prediction is largely independent by millions of machine nodes with growing mobility, such of the distance human cover on a regular basis. Based on as sensors carried by moving objects or vehicles. As we shall the unique characteristics of human moving behaviour, we see, mobile big data has already penetrated to every aspects propose an effective deep-learning based model to predict in our lives. More and more people conduct their social the human trajectories. arXiv:1610.07009v2 [cs.CY] 15 Nov 2018 activities, watch videos, buy things via their mobile devices. Recently, deep learning has achieves marvelous results Mobile big data is a gold mine containing large amounts of in many field (e.g., computer vision, speech recognition, individual information. etc.). It shows its distinct advantages for the complex and Mobile data is naturally generated by various mobile massive data with the benefit of a deep nonlinear network device carried by human, which means that it involves structure. Mobile data has complex spatiotemporal relation the mobility patterns of each carrier. Understanding human and it is usually high-dimensional. The shallow-structure mobility pattern is valuable in many geographical applica- algorithms like Support Vector Machine (SVM) has difficulty tions, such as transportation planning and epidemic disease dealing with high dimensional data with the development of mobile network, because shallow-structure algorithms • Xi Ouyang and Pan Zhou are with the School of Electronic In- need to large amount of priori knowledge to extract feature formation and Communications, Huazhong University of Science manually. In contrast, deep learning just like the ”black and Technology, Wuhan 430074, China. Email: [email protected], magic”: it has been to shown to outperform state-of-the-art [email protected]. machine learning algorithm for such complex data. Consid- • Chaoyun Zhangy, Corresponding author of this paper, is with . Email: [email protected]. ering the power of deep learning, we develop our model • Hao Jiang is with the School of Electronics Information, Wuhan Univer- using convolutional neural network (CNN) which is one of sity, Wuhan 430074, China. E-mail: [email protected]. the popular and effective models in deep learning. There Manuscript received April 19, 2005; revised September 17, 2014. are two key advantages in when applying CNN to human JOURNAL OF LATEX CLASS FILES, VOL. 13, NO. 9, SEPTEMBER 2014 2 mobility pattern task: is non-random, which indicates that people always appear in several specific places. In spite of the diversity of behav- • Local Dependency: CNN is able to exploit local de- ior, human mobility always obeys the simple reproduction pendencies of nodes in the human trajectories. In the mode and a high degree of predictability [1] [7]. Sevtsuk field of computer vision, CNN can capture the local uses the erlang from base stations as indicators measures the dependency of visual information. There has strong population distribution, and reach a conclusion that human’ correlation between the nearby pixels in images. The activities have the universal law over different time period nearby positions in the human trajectories are also (e.g., hour, day, week, etc.) [8]. In a nutshell, human’ spatial likely to be relative. In a specific time interval, a hu- mobility is proved to have the following special features: 1) man can only move to the specific positions based on Regional. The probability of leaving a small area decreases the past positions rather than the random positions with time. 2) Strongly regular and predictability. There are because of the limited mobility and time. several frequent areas for individuals. • Scale Invariance: CNN ensures feature scale to be invariant. In image recognition, the training images Deep learning. Recently, deep learning make many might have different scales. When predicting hu- breakthroughs in different research fields, and has drawn man mobility pattern, a person can have similar great attention from both industrial and academic circles trajectories every day, but the time interval between [9]. Deep learning is proved to be very powerful in both positions and staying time in each position is often supervised and unsupervised learning fields [10]. In fact, the different. CNN can preserve spatial feature scale to artificial neural network, which is the predecessor of deep be invariant in different temporal paces. learning, can be traced back to the new perceptron proposed by Fukushima Kunihiko in 1980 [11]. In 1989, Yann et al. Furthermore, this model we design is a online learning [12] use a convolutional neural network to recognize the structure for the continuous mobile big data stream. We handwritten numeral. Although their model gets a great call our model “DeepSpace”, and the overall structure is accuracy in handwritten numerals dataset, it requites too shown in Fig. 1. Our model is a hierarchical framework, many computation and memory resources, which can run and the model completes the final predicting via two sub- for three days in the 1990s. However, after 2006, due to the architectures (the coarse prediction model and fine predic- excellent development of neural network [13] [14] [15], the tion models). We will describe this system in the following theory of deep-structure models have major breakthroughs of this paper in detail to explain how this system works as and solve the vanishing gradient problem [16]. Meanwhile, a online learning system and suiting the spatial character- accompanied by the development of hardware, the appli- istics of the mobile data. The contribution of this work are cations of deep learning boom since 2006. Especially in summarized as follows: the field of computer vision, Hinton and his students [17] • “DeepSpace” is a online learning model, which can greatly reduce the top-1 and top-5 error rates of ImageNet deal with the continuous mobile data stream in real via many efficient improvements in CNN. Applying CNN time. Meanwhile, we can run this model in paral- to non-image classification has also made rapid progress. lel, which is efficient even only limited computing In [18], Ming et al. propose an approach based on CNN resource is avaliable. to recognize human activities (e.g., “jogging”, “walking”, • To the best of our knowledge, “DeepSpace” is the “ascending stairs”, etc.). Their CNN-based approach is prac- first deep learning system that is redesigned to adjust tical and achieves higher accuracy than existing state-of-art to the spatial characteristics of the mobile data. methods. The remainder of the paper is organized as follows. In Online learning. There are three representative paper Section 2, we first introduce related work with respective [19] [20] [21] in transforming deep learning into a online to human mobility prediction, deep learning and online learning model. In [19], Zhou et al. use the denoising learning. Meanwhile, we discuss the motivation in Section autoencoder [22] as a building block for online learning.