Deployment of Compute Nodes for the WLCG with Cobbler, Ansible and Salt
Deployment of compute nodes for the WLCG with Cobbler, Ansible and Salt. Damien François, Olivier Mattelaer, Thomas Keutgen HEPiX meeting, October 20 !, "arcelona, #$ain Center for High Performance Computing and Mass Storage ')#M T2%"E%&'( Manneback+)ngrid cluster grows organically ; 1 to 10 machines at a time now 6000+ cores, Gb+10Gb 100 local users + CMS grid users, ~2 M jobs per year Manneback/Ingrid cluster -e started “manuall/0111 check-list shell script config. management make persistent make actionable make idempotent 111 and graduall/ im$rove, automation, and ,ocumentation1 -e settled on three tools 2or the provisioning o2 ne3 nodes Unboxing ● Label, rack, connect ● Choose Name, IP ● Gather MAC 1. Deploy 2. Integrate 3. Confgure Ready for jobs “Cobbler is a Linux installation server that allows for rapid setup of network installation environments.” http://cobbler.github.io Wrapper for PXE, TFTP, DHCP servers Manage OS images, machine profiles Install operating system Setup hardware-specifc confguration (disk partitions, NICs, IPMI, etc.) Setup minimal confguration (Admin SSH keys, Salt minion) “Ansible seamlessly unites workflow orchestration with configuration management, provisioning, and application deployment in one easy-to-use and deploy platform.” https://www.ansible.com Shell scripts on steroïds with builtin safety, idempotence, APIs One-off operations register to Zabbix, GLPI, Salt build files: - slurm.conf for Slurm, - /etc/hosts for dnsmasq, - /etc/ssh/ssh_known_hosts create CPU-specific directory for Easybuild “Scalable, flexible, intelligent IT orchestration and automation” https://saltstack.com Central configuration management server Daily management configure system: NTP, DNS, LDAP, Slurm, etc. install admin software, cvmfs, yum repos, etc. mount user filesystems, cvmfs, etc.
[Show full text]