Junmei Feng Recommending Multimedia Xiaoyi Feng Liming Deng Information in a Virtual Han School of Electronic and Information Chang’an City Roaming System Northwestern Polytechnical University Xi’an, PR China
Jinye Peng* School of Information Science and Abstract Technology Northwest University This article presents a roaming system of Han Chang’an City, with both virtual reality Xi’an, PR China (VR) technology and information recommendation technology. Nowadays, some new research issues in the cultural heritage domain can be achieved with the rapid develop- ment of VR technology. The ancient site of Han Chang’an City, as one of the most valuable and significant cultural heritages in China, attracts more and more attention around the world. To let more people understand Han Chang’an City and reproduce its beauty, in this article, we propose a virtual roaming system combined with informa- tion recommendation technology. First, Unity3D is selected as the three-dimensional platform to design the scenario model of Han Chang’ an City, and the virtual scene is reconstructed with VR technology, according to real historical data; then, the dynamic information recommendation module is designed to recommend hot topic informa- tion and personalized information. The former is obtained through web crawlers, including the latest released news related to Han Chang’an City for users. The latter is generated by the proposed hybrid recommendation algorithm, which combines explicit and implicit feedback. The performance of the proposed algorithm is validated on two datasets. Finally, we show some results of our system test. Our proposed sys- tem is released online now, and users can wander in the scene any time.
1 Introduction
1.1 Archeological Background
As the capital of the Western Han Dynasty (206 BC–25 AD), Han Chang’an City was the political, economic, and cultural center for two hundred years, and was also the eastern end of the Silk Road. Chang’an City was estab- lished as a capital in 202 BC by the first Han Emperor Gaozu. Figure 1 shows the plan of Han Chang’an City. Its area was about 34.39 square kilometers, and the length of the city wall was about 25.1 kilometers with three gates per side. The main buildings in Han Chang’an City included Weiyang Palace, Changle Palace, Mingguang Palace, and North Palace. Weiyang Palace, as the most important palace of the city, was the center of power of the empire and the place where the emperors lived and rested. Han Chang’an City was destroyed during the political upheaval at the end of the Tang dynasty in 904 AD, and an archeological site can be found on the site of Han Chang’an City (see Figure 2). Nowadays, the ancient site of Han Chang’an City is located ten kilometers northwest of today’s Xi’an City, Shaanxi Presence, Vol. 26, No. 3, Summer 2017, 322–336 doi:10.1162/PRES_a_00299 ª 2017 by the Massachusetts Institute of Technology *Correspondence to [email protected].
322 PRESENCE: VOLUME 26, NUMBER 3
Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/pres_a_00299 by guest on 25 September 2021 Feng et al. 323
Figure 1. Plan of Han Chang’an City. Reprinted with permission from J. Feng, X. Liu, X. Feng, J. Peng, X. Wang, & X. Peng (2016). A new method of virtual Han Chang’an City navigation system. Proceedings of Eighth International Conference on Digital Image Processing, copyright SPIE.
most representative and typical ‘‘national’’ sites in China, and it is still there as a silent witness of the Silk Road. It is too large to be physically reconstructed; hence, the low-budget virtual reconstruction of Han Chang’an City turns out to be a suitable way to play back the glory of the Western Han Dynasty. It is widely agreed that the focus of a cultural heritage exhibition has been a shift from static information to actively involved users. In the past, the exhibition focused more on cultural relics, but now it pays more attention to personalized services for users. According Figure 2. The site of Han Chang’an City. Photo was taken by to users’ preferences and cultural characteristics Chinese photographer Mu Ren. (Benouaret & Lenne, 2016; Wang et al., 2009), they Province, near the confluence of the Wei and Feng rivers, will be provided with digital information that they are China. Moreover, the site is the largest and the most interested in related to cultural heritage, such as images, completely preserved, with the richest relics, one of the text introduction, and video. In this way, users can fully
Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/pres_a_00299 by guest on 25 September 2021 324 PRESENCE: VOLUME 26, NUMBER 3
enjoy their cultural roaming trip. Considering this fact, this version is difficult to share. Although the web version we combine our virtual Chang’an City system with infor- canbereleasedtoshareonthenetwork,wehavenot mation recommendation technology to make our system implemented the connection with the VR glasses. At pres- more convenient and attractive. ent, the interactive method of the web version is to use the mouse and keyboard. To share our work with scholars, sci- entists, and students who prepare to carry out archeologi- 1.2 Virtual Reconstruction cal or historical research, this article is presented based on 1.2.1 VR Technology. VR (Kyriakou, Pan, & the released web version of virtual Han Chang’an City. Chrysanthou, 2016) is a technology that is used to simu- late a real environment, so that users can fully immerse 1.2.2 Data Collection. In order to reproduce the in the scene and interact with the virtual scene. There- beauty of Han Chang’an City, the first step is to collect fore, roaming in the scene is possible. In recent years, data related to the cultural heritage (Albanese, VR technology is widely used in daily life for its simple d’Acierno, Moscato, Persia, & Picariello, 2011) of Han operation and strong immersion, in areas such as educa- Chang’an City. As Han Chang’an City was seriously tion, entertainment (Ramanathan, Rangan, & Vin, damaged, the original geographic location data and the 2002), cultural heritage (Kim, Kesavadas, & Paley, information of historical relics should be collected, and 2014; Champion, 2015), and health (Pfandler, Lazaro- were obtained through online papers related to Han vici, Stefan, Wucherer, & Weigl, 2017). In terms of cul- Chang’an City (Han Chang’an Archeological Team, tural heritage, the display and protection of cultural 2012). These papers compared different maps, including relics can be improved to a new stage with the combina- the map of China, the map of Xi’an, and the aerial map tion of VR technology and network technology. First, of Han Chang’an City, by referring to archeological high precision and permanent preservation of the endan- findings and historical documents, visiting museums in gered heritage resources can be realized. Second, the Xi’an, consulting experts of Han Chang’an Archeologi- exhibition of cultural relics can be separated from geo- cal Team, and field measurement. The plan of Han graphical restrictions, realizing the sharing of resources. Chang’an City (see Figure 1) was drawn in AutoCAD Generally speaking, the use of VR technology can pro- based on the collected data, which was also imported to mote the cultural and trade industry to enter the infor- Unity3D as the buildings’ layout. The next work is mation age faster and realize the modernization of cul- deciding how to build the model of Han Chang’an City. tural relics display and protection. In this article, we choose 3ds Max (3D Studio Max) as Unity3D (Wang et al., 2010) is used as the development the tool of modeling (Shen & Zeng, 2011). platform for VR in our work. It is a cross-platform 3D game engine developed by Unity Technologies. As a fully 1.2.3 Building the Model. As mentioned above, integrated professional game engine, it reduces the diffi- the structure of Han Chang’an City is extremely compli- culty of modeling and improves the efficiency of the devel- cated, and composed of many famed historical architec- opment cycle. In the current VR interaction technology, tural sites such as Weiyang Palace, Changle Palace, the most used method is handle control. The advantages Mingguang Palace, North Palace, Gui Palace, West of this interaction are fast response and high sensitivity. Market, and East Market. As the realistic degree of the In this article, two versions of the virtual Han Chang’an model directly affects the VR experience, modeling is an City project are released, including PC (Personal Com- important part of the virtual system. puter) and Web Player. In the PC version, VR glasses are In this modeling, 3ds Max was used to design sophis- connected to our virtual scene to help users experience the ticated models such as palaces and cultural relics strictly virtual city, and a handle is used to navigate through the based on geometric data, and Unity3D was applied to multimedia information in the scene. One drawback of design easy models like city walls, roads, and trees to cre- this approach is that the released project is very large, and ate a realistic environment. Taking the model of Weiyang
Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/pres_a_00299 by guest on 25 September 2021 Feng et al. 325
Figure 3. Screenshots of the models built in 3ds Max: (a) a view of Weiyang Palace, (b) Front Hall, (c) Jiaofang Hall, (d) Han Emperor.
Palace, for example, it was the official center of govern- The structure of our article is organized as follows. ment and consisted of more than 40 halls, such as Front Section 2 describes the interactive technology used in Hall and Jiaofang Hall. We divided the model of the virtual Han Chang’an City roaming system. Section Weiyang Palace into models of halls to model separately 3 presents the proposed dynamic information recom- and saved them as .FBX format. After the models were mendation module, including hot topic recommenda- prepared, we imported the model files into Unity3D tion and personalized recommendation in detail. Section scene and moved to them to the corresponding positions 4 reports several experiments to test the performance of according to the buildings’ layout of our plan. Figure 3 our system and recommendation results of the system. shows the models of Weiyang Palace built in 3ds Max. Section 5 discusses the results and limitations of our sys- Figure 3(a) shows a view of Weiyang Palace, and the tem. Finally, Section 6 gives a high-level description of other three images are the close shot of small palaces in the conclusion and future work. Weiyang Palace. Front Hall was in the middle of Weiyang Palace, which was built with an incense wood rafter. Jiaofang Hall was the residence of the empress. 2 Design of System Interaction Additionally, considering the modeling efficiency, the modeling size should be taken into consideration. If the Human–computer interaction technology (Yang, project involves huge data, the running speed of the sys- Zhang, & Wu, 2009; Yang & Jie, 2014) is an efficient tem will be slowed down. Thus, reducing the size of the way to realize the dialog between people and computers model by minimizing the amount of data is a smart way with the input and output devices. In this article, inter- to optimize the system (Lin & Zhao, 2014). active technology is mainly used in the areas including
Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/pres_a_00299 by guest on 25 September 2021 326 PRESENCE: VOLUME 26, NUMBER 3
gation map window will be displayed in the upper left corner. This function will be detailed in the next section. The function of the button ‘‘Image Info’’ is to display some basic information of palaces in the form of images. ‘‘Instructions’’ gives users guidance on how to roam.
2.2 Navigation Map
In this article, the navigation map (Mousouris & Styliaras, 2014) is different from general two- dimensional maps with fixed sizes, which cannot show the detailed geographic information. As our roaming Figure 4. Original construction drawings for the screenshot of the system is released online, the designed map is a web- roaming mode selection interface. Reprinted with permission from based application combining Gaode map API (Applica- J. Feng, X. Feng, X. Liu, and J. Peng (2016), The Virtual Wandering tion Program Interface) with the Han Chang’an plan. System of Han Chang’an City Based on Information Recommendation. The city plan is added to the Gaode map API according Proceedings of 15th ACM SIGGRAPH Conference on Virtual-Reality to its longitude and latitude just as a layer covering. Continuum and its Applications in Industry; copyright Association for Hence, the higher the zoom level of the map, the more Computing Machinery, Inc. detailed the information displayed. As HTML can directly communicate with Unity3D scene roaming, navigation map, and multimedia infor- (Feng, Liu, et al., 2016), thus, the map can interact with mation display. the virtual scene online. A point mark is added to the map to display the position of the first person controller 2.1 Scene Roaming in the scene. Users cannot only know their specific posi- tion in the virtual Han Chang’an City scene, but also can With respect to roaming, two modes are estab- correspond to the geographical location in Xi’an. Our lished for users to choose: auto roaming and manual previous work (Feng, Feng, Liu, & Peng, 2016) shows roaming; the default mode is manual roaming. After one more details about this navigation map. determines the destination, auto roaming means that the first person controller moves along the path according to 2.3 Multimedia Information the A* search algorithm for pathfinding (Stamford, Khu- Display Module man, Carter, & Ahmadi, 2014). One of the main limita- tions of this mode is that one cannot artificially stop In this VR system, multimedia information (Barto- before reaching the destination. On the other hand, the lini et al., 2013) is obtained through the following inter- user who selects the manual roaming mode can control active ways: image, 3D models of cultural relics, voice the direction of movement and adjust the perspective explanation, and video. through the function keys on the GUI (Graphical User Images are used to show palaces, historical figures, Interface) which is shown in Figure 4. and historical events. In the virtual scene, round icons Figure 4 presents the screenshot of the roaming mode (as shown in Figure 4) are hidden information points; if selection interface. The four buttons in the lower right you click on an icon, the text image will be a pop-up. corner are function keys including ‘‘Panoramic View,’’ The format of the images in our system is BMP and JPG. ‘‘Map,’’ ‘‘Image Info,’’ and ‘‘Instructions.’’ If the button Models of cultural relics are shown in the GUI control of ‘‘Panoramic View’’ is pressed, one can overlook the interface; users can get more detailed information by whole Han Chang’an City scene accompanied by voice rotating or zooming. As shown in Figure 5, the cultural explanation. If one presses the ‘‘Map’’ button, the navi- relic of the green glaze run the beast bottle is displayed
Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/pres_a_00299 by guest on 25 September 2021 Feng et al. 327