Visualization Technology for the K computer Atsuji Ogasa Hiroyuki Maesaka Kiyotaka Sakamoto Sadanori Otagiri Visualization technology makes images or videos from the results of numerical computations performed by supercomputers. To visualize the results of parallel computation on a super-large scale of a few to a few tens of thousands of parallel processes, processing in which bulky computation result data are rendered at high speeds is required. The conventional method of transferring computation result data to a visualization server may give rise to many challenges that are difficult to meet such as reassembly and transfer of extensive amounts of computation result files. This paper presents technologies for solving these challenges to visualize super-large-scale computations performed by the K computer and the results of applying those technologies. 1. Introduction transferring the numerical data computed on The results of numerical computations numerical computation results obtained by performed by supercomputers are obtained a supercomputer to a visualization server, in as a series of digital numerical data on scales which visualization software is used, as shown that have recently come to exceed one billion. in Figure 1. The number of parallel processes Computer graphics (CG) can be used to render performed in a server is often one to about a these numerical data as images or videos to help dozen. In the past, numerical computations people intuitively understand them, which is a were usually performed in a supercomputer on technology called visualization. scales of a few tens to a few hundreds of parallel This paper presents solutions to challenges processes. And, for visualization, the obtained for visualizing super-large-scale computation data on numerical computation results were results with the K computernote)i and the reassembled into one to about a dozen files technologies applied. depending on the number of parallel processes performed by the visualization server. In the 2. Challenges in visualization K computer, however, numerical computation of large-scale parallel takes place on scales of a few to a few tens of computation thousands of parallel processes. Reassembling Generally, visualization is achieved by the resultant data for visualization takes a tremendous amount of time and trouble, and the note)i “K computer” is the English name that RIKEN has been using for the extensive size of the reassembled files give rise to supercomputer of this project since July challenges: 2010. “K” comes from the Japanese word 1) Too much transfer time is required to “Kei,” which means ten peta or 10 to the 16th power. move the computation result data to the 348 FUJITSU Sci. Tech. J., Vol. 48, No. 3, pp. 348–356 (July 2012) A. Ogasa et al.: Visualization Technology for the K computer visualization server. For example, the fi le 3. Visualization library of the K size of double precision scalar data with computer computational grid points of the cube of The fundamental cause of the challenges in 40963 is about 550 Gbytes, which requires visualizing the results of computations with the at least one hour for transfer even with a K computer is that computation result data are 1000BASE-T network. diffi cult to transfer and reassemble. The cause of 2) Even if the computation result data are these challenges can be removed by the following successfully transferred, the visualization approaches. server has insuffi cient memory to store the 1) Eliminating the need to transfer computation result data. For example, the computation result data by visualizing on a memory size of PC workstations, which are node of the K computer often used as visualization servers, is only 2) Eliminating the need to reassemble up to 192 Gbytes and the 550-Gbyte data computation result data by visualizing with mentioned above cannot be stored even if the same number of parallel processes as they are converted into single-precision the computation data. In general, visualization processing is 3) Thinning grids out from computation result often performed while interactively changing data at regular intervals so as to reduce the parameters as in the sweeping of a cross-section fi le size may cause important information position or iso-surface value. On a node of the K to be lost. Appropriate thinning requires computer, however, even visualization processing the characteristics of each type of data to be must be performed as a batch job for operational taken into consideration, and this cannot be reasons. For that reason, volume rendering,1) handled in a general manner. which is suitable for grasping at a glance the In this way, with large-scale parallel physical quantity distribution in the entire computations such as those performed by 3D computational space without interactive the K computer, there is a problem that processing such as cross-section sweeping, is the conventional visualization technique of employed as a visualization technique for the reassembling computation result data for present development. transfer to the visualization server cannot be Based on these approaches, we have employed. developed visualization software that uses Supercomputer Visualization server Data Visualization result transfer Results of numerical Visualization calculation software Figure 1 Concept of visualization. FUJITSU Sci. Tech. J., Vol. 48, No. 3 (July 2012) 349 A. Ogasa et al.: Visualization Technology for the K computer volume rendering, which effi ciently runs on a computation result data. node of the K computer even with the number of 2) To call an API from the numerical parallel processes on a scale exceeding thousands computation program. and a visualization library that includes tools One good point of the former is that for assisting use of the visualization software. visualization can be achieved without modifying Figure 2 shows the concept of visualization the numerical computation program. An with the K computer. Data on the results of example of program coding based on 1) is shown numerical computations are visualized on a node in Figure 3. of the K computer and the result of visualization The latter is advantageous in that the data is output as an image fi le in JPEG or some other in the memory can be directly visualized, which format. The size of the image fi le depends on eliminates the data transfer cost from reading the resolution of the image regardless of the fi les and makes it easier to visualize the process number of computational grid points, and is as of computation. small as a few hundred Kbytes to a few Mbytes in many cases, which means that the result of 3.2 File interface visualization can be easily transferred to the A fi le interface is intended for directly user’s terminal for viewing. reading the computation result data. File As interfaces for passing the computation interfaces are already prepared for some fi elds result data to the visualization library, we have of numerical computation performed by the K prepared program and fi le interfaces. computer. Simply by outputting computation A program interface calls an application result data in a format compatible with the programming interface (API) with C or fi le interface, the user can directly read the FORTRAN from the user program. Table 1 computation result data to the visualization shows the APIs and data structures of the library for visualization without having to use visualization library. A program interface can be APIs to create a program. used in two ways: An image of using the visualization library through program and fi le interfaces is shown in 3.1 Program interface Figure 4. 1) To call an API from the program to read the As tools to help people use visualization K computer User terminal Data transfer Visualization result image Visualization result (still image, video) Results of Visualization numerical calculation software Figure 2 Concept of visualization with the K computer. 350 FUJITSU Sci. Tech. J., Vol. 48, No. 3 (July 2012) A. Ogasa et al.: Visualization Technology for the K computer Table 1 APIs and data structures of visualization library. Structure for structured grid data: s_PBR_array_info Common API Description CPBR_Init Initializes PBVR Member name Type Description Specification of existence domain CPBR_Get_imageinfo Acquires composite image information extent float[2][3] of entire model CPBR_Run Runs PBVR datatype int Data type CPBR_Finalize Finalizes PBVR datasize int[3] Grid size Outputs visualization result as a JPEG CPBR_OutputJpeg file coordtype int Coordinate type Outputs visualization result as a PNG coord float * Coordinate data CPBR_OutputPng file data void * Visualization data API for structured Structure for unstructured grid data: s_PBR_cell_info Description grid data definition Member name Type Description Initializes a structure that stores CPBR_Init_array_info Specification of existence domain structured grid data parameters extent float[2][3] of entire model Registers a structure that stores node.n int Number of nodes CPBR_Set_array_info structured grid data parameters with PBVR node.datatype int Data type node.data void * Visualization data (node) API for unstructured node.coord float * Coordinate data Description grid data definition Data type (common to all cell.datatype int Initializes a structure that stores elements) CPBR_Init_ucd_info unstructured grid data parameters cell.n_tet int Number of tetrahedral elements Registers a structure that stores Tetrahedral element connection cell.c_tet int * CPBR_Set_ucd_info unstructured