Moreno Baricevic CNR-INFM DEMOCRITOS Trieste, ITALY InstallationInstallation ProceduresProcedures forfor ClustersClusters PART 4 – Hands-on Laboratory AgendaAgenda Cluster Services Overview on Installation Procedures Configuration and Setup of a NETBOOT Environment Troubleshooting Cluster Management Tools Notes on Security Hands-onHands-on LaboratoryLaboratory SessionSession 2 Hands-onHands-on LaboratoryLaboratory –– SessionSession 11 Installation of a master node Post configuration of the master node Setting up NETBOOT services (DHCP, TFTP, PXE, NFS, package repository) Installing our first computing node Testing the cluster environment 3 Hands-onHands-on LaboratoryLaboratory –– SessionSession 22 Passwordless environment and c3 tools NFS shares Installing and configuring the resource management system (torque+maui) Serial job submission OpenMPI and parallel job submission Compilers and modules (optional) 4 Hands-onHands-on LaboratoryLaboratory –– SessionSession 33 Monitoring tools (ganglia) (*) Configuring DNS (bind) (*) kickstart %post: further configuration (*) NAT setup and IP forwarding (iptables) (*) 5 (*) optional Hands-onHands-on LaboratoryLaboratory SessionSession Let's do it! http://edu.escience-lab.org http://elearn.escience-lab.org 6 REFERENCESREFERENCES ANDAND USEFULUSEFUL LINKSLINKS Cluster Toolkits: Monitoring Tools: ● OSCAR – Open Source Cluster Application Resources ● Ganglia http://ganglia.sourceforge.net/ http://oscar.openclustergroup.org/ ● Nagios http://www.nagios.org/ ● NPACI Rocks ● Zabbix http://www.zabbix.org/ http://www.rocksclusters.org/ ● Scyld Beowulf Network traffic analyzer: http://www.beowulf.org/ ● tcpdump http://www.tcpdump.org ● CSM – IBM Cluster Systems Management ● wireshark http://www.wireshark.org http://www.ibm.com/servers/eserver/clusters/software/ ● xCAT – eXtreme Cluster Administration Toolkit UnionFS: http://www.xcat.org/ ● Hopeless, a system for building disk-less clusters ● Warewulf/PERCEUS http://www.evolware.org/chri/hopeless.html http://www.warewulf-cluster.org/ http://www.perceus.org/ ● UnionFS – A Stackable Unification File System http://www.unionfs.org Installation Software: http://www.fsl.cs.sunysb.edu/project-unionfs.html ● SystemImager http://www.systemimager.org/ ● FAI http://www.informatik.uni-koeln.de/fai/ RFC: (http://www.rfc.net) ● Anaconda/Kickstart http://fedoraproject.org/wiki/Anaconda/Kickstart ● RFC 1350 – The TFTP Protocol (Revision 2) http://www.rfc.net/rfc1350.html Management Tools: ● RFC 2131 – Dynamic Host Configuration Protocol ● openssh/openssl http://www.rfc.net/rfc2131.html http://www.openssh.com ● RFC 2132 – DHCP Options and BOOTP Vendor Extensions http://www.openssl.org http://www.rfc.net/rfc2132.html ● C3 tools – The Cluster Command and Control tool suite ● RFC 4578 – DHCP PXE Options http://www.csm.ornl.gov/torc/C3/ http://www.rfc.net/rfc4578.html ● PDSH – Parallel Distributed SHell ● RFC 4390 – DHCP over Infiniband https://computing.llnl.gov/linux/pdsh.html http://www.rfc.net/rfc4390.html ● DSH – Distributed SHell http://www.netfort.gr.jp/~dancer/software/dsh.html.en ● PXE specification ● ClusterSSH http://www.pix.net/software/pxeboot/archive/pxespec.pdf http://clusterssh.sourceforge.net/ ● SYSLINUX http://syslinux.zytor.com/ ● C4 tools – Cluster Command & Control Console http://gforge.escience-lab.org/projects/c-4/ 7 SomeSome acronyms...acronyms... ICTP – the Abdus Salam International Centre for Theoretical Physics DEMOCRITOS – Democritos Modeling Center for Research In aTOmistic Simulations INFM – Istituto Nazionale per la Fisica della Materia (Italian National Institute for the Physics of Matter) CNR – Consiglio Nazionale delle Ricerche (Italian National Research Council) HPC – High Performance Computing IP – Internet Protocol TCP – Transmission Control Protocol OS – Operating System UDP – User Datagram Protocol LINUX – LINUX is not UNIX DHCP – Dynamic Host Configuration Protocol GNU – GNU is not UNIX TFTP – Trivial File Transfer Protocol RPM – RPM Package Manager FTP – File Transfer Protocol HTTP – Hyper Text Transfer Protocol CLI – Command Line Interface NTP – Network Time Protocol BASH – Bourne Again SHell PERL – Practical Extraction and Report Language NIC – Network Interface Card/Controller MAC – Media Access Control PXE – Preboot Execution Environment OUI – Organizationally Unique Identifier INITRD – INITial RamDisk API – Application Program Interface NFS – Network File System UNDI – Universal Network Driver Interface SSH – Secure SHell PROM – Programmable Read-Only Memory LDAP – Lightweight Directory Access Protocol BIOS – Basic Input/Output System NIS – Network Information Service DNS – Domain Name System SNMP – Simple Network Management Protocol MIB – Management Information Base PAM – Pluggable Authentication Modules OID – Object IDentifier LAN – Local Area Network IPMI – Intelligent Platform Management Interface WAN – Wide Area Network LOM – Lights-Out Management RSA – IBM Remote Supervisor Adapter 8 BMC – Baseboard Management Controller.