System and Performance Monitoring on Diskless PC Clusters Chao-Tung Yang and Ping-I Chen High-Performance Computing Laboratory Department of Computer Science and Information Engineering Tunghai University, Taichung City, 40704 Taiwan, R.O.C. [email protected] [email protected] Abstract. In this paper, we introduce the experiments of monitoring on SLIM and DRBL diskless PC clusters. We constructed the performance experimentations by using 16 computing nodes and only one hard disk. We used LAM/MPI and PVM to run the matrix multiplication program in order to evaluate their performance. We also used spm2 and ntop to monitoring the system status during performance evaluation. As the result, we can find that the network loading of SLIM is less than DRBL even though the performance result of DRBL is better than SLIM. Maybe this is because the DRBL use NFS to mount everything which the nodes needed. We can realize that if we want to construct a high-performance diskless Linux cluster. We should take care of the free space of main memory and the mounting method of file systems. Keywords: Diskless, System monitoring, Performance monitoring, PC clusters, DBRL, SLIM. 1 Introduction The use of loosely coupled, powerful and low-cost commodity components (PCs or workstations, typically), especially without any hard disk drive, connected by high- speed networks has resulted in the widespread usage of a technology popularly called diskless cluster. Strictly speaking, it consists of one or more servers which provide not only bootstrap service but also related network services (such as DHCP, NIS, NFS servers, and etc) and many clients with no hard disk drive requesting for booting from the network. The availability of such clusters made maintain as easy as possible, and also reduced the waste in storage space. The diskless cluster differs from the traditional one in that a network is used to provide not only inter-processor communications but also a medium for booting and transmission for a live file system. Thus, each diskless node before booting can boot through a floppy disk or a NIC’s boot ROM with a small bootstrap program and even with a NIC’s PXE, which sends a broadcast packet to a DHCP server and is then assigned an IP address. After each node has been assigned a valid IP address, it sends a request to the TFTP server 2 Chao-Tung Yang and Ping-I Chen for getting the boot image, referred to the Linux Kernel, through TCP/IP protocol and starts the booting process. During the booting process, all the necessary system files are transmitted through the network. After the remote file system is mounted as root file system (NFS_ROOT), and the system initialization is done, the node is ready to work. In this paper, we introduce the experiments of monitoring on SLIM and DRBL diskless PC clusters. We constructed the performance experimentations by using 16 computing nodes and only one hard disk. We used LAM/MPI and PVM to run the matrix multiplication program in order to evaluate their performance. We also used spm2 and ntop to monitoring the system status during performance evaluation. As the result, we can find that the network loading of SLIM is less than DRBL even though the performance result of DRBL is better than SLIM. Maybe this is because the DRBL use NFS to mount everything which the nodes needed. We can realize that if we want to construct a high-performance diskless Linux cluster. We should take care of the main memory’s free space and the mounting method of file systems. We can realize that if we want to construct a high-performance Linux diskless PC cluster. We should take care of the free space of main memory and the mounting methods of file systems. Using NFS to mount everything can ease the nodes’ main memory loading. But it will cause the network loading very high. Using TFTP to download the whole server’s Linux image file will waste a lot of main memory space. Maybe we can combine the advantage of them to construct the brand-new high performance diskless Linux cluster. 2 Background Review 2.1 PXE PXE (Preboot eXecution Environment) is Intel’s loosely defined standard for booting PCs over the network. A PXE-capable BIOS or boot ROM can download bootstrapping code and load an operating system over the network. Booting Linux with PXE is a straightforward way of starting a diskless workstation or appliance in a closed network. PXE defines a method for the BIOS or NIC ROM to fetch a booting code over the network. It does this via standard Internet protocols. When the appliance is powered on, the BIOS or ROM makes a DHCP request. The DHCP server, recognizing the appliance as a network-booting client, returns instructions on the location of a TFTP server and the name of the name of the file that it should download from the server. First, it checks it's configuration files (located on the TFTP server) and then downloads the Linux kernel, passing it the necessary kernel arguments, including the IP information received from DHCP. When the kernel loads, it uses the IP information provided for it. The kernel can either be download an initial ramdisk (initrd) and use that as a root filesystem, or it may just connect to an NFS server. Once the kernel has gotten the root file system setup, the operating system can complete the boot process, and the system is ready for use.[1] System and Performance Monitoring on Diskless PC Clusters 3 2.2 TFTP TFTP is a simple protocol to transfer files, and therefore was named the Trivial File Transfer Protocol or TFTP. It has been implemented on top of the Internet User Datagram protocol (UDP or Datagram), so it may be used to move files between machines on different networks implementing UDP. It is designed to be small and easy to implement. Therefore, it lacks most of the features of a regular FTP. The only thing it can do is read and write files (or mail) from/to a remote server. It cannot list directories, and currently has no provisions for user authentication. In common with other Internet protocols, it passes 8-bit bytes of data. Three modes of transfer are currently supported: netascii (This is ASCII as defined in “USA Standard Code for Information Interchange” with the modifications specified in “Telnet Protocol Specification”.) Note that it is 8 bit ASCII. The term “netascii” will be used throughout this document to mean this particular version of ASCII.); octet (This replaces the “binary” mode of previous versions of this document.) raw 8 bit bytes; mail, netascii characters sent to a user rather than a file. (The mail mode is obsolete and should not be implemented or used.) Additional modes can be defined by pairs of cooperating hosts. [2] 2.3 NFS The Network File System (NFS) was developed to allow machines to mount a disk partition on a remote machine as if it were on a local hard drive. This allows for fast, seamless sharing of files across a network. It also gives the potential for unwanted people to access your hard drive over the network (and thereby possibly read your email and delete all your files as well as break into your system) if you set it up incorrectly. So please read the Security section of this document carefully if you intend to implement an NFS setup. [3] 3 Two Types of Diskless PC Cluster There are many ways to construct a Linux cluster. But dealing with differences with node file systems is time consuming and can be problematic. So, some research groups created this kind of cluster in order to construct the cluster in an efficient way. It also costs less money and we can manage the nodes easily. In our experiment, we only use two kinds of diskless cluster. One is SLIM, which is designed by University of Hong Kong. Another is DRBL, which is designed by National Center of High-performance Computing. 3.1 The SLIM Cluster SLIM stands for Single Linux Image Management. It holds the Linux OS image to be shared by all PCs via network booting. This solution is solely designed and 4 Chao-Tung Yang and Ping-I Chen implemented by people in Department of Computer Science, The University of Hong Kong. The SLIM server holds pre-installed Linux system image for sharing across the network. The system image is exported as read only by the NFS to all client PC to build their local root file system during booting up. One SLIM server may serve as many as OS images made by different Linux distributions. It also provides TFTP service to allow client PC to download network boot loader. It also holds OS boot images which are Linux kernel and initrd for network boot loader to download. [4] 3.2 The DRBL Cluster DRBL stands for Diskless Remote Boot in Linux. This solution is solely designed and implemented by people in National Center of High-performance Computing, Taiwan. DRBL uses PXE/etherboot, NFS, and NIS to provide services to client machines. Once the server is ready to be a DRBL server, then the client machines can boot via PXE/etherboot (diskless). “DRBL” does NOT touch the hard drive of the clients, so other Operating Systems (for example, M$ Windows) installed on your client machines will not be affected. This may be important in a phased deployment of GNU/Linux, where users still want to have the option of booting to Windows and running Office.