Common Disaster Recovery Tools In Unix and Linux* © 2008 Dusan Baljevic The information contained herein is subject to change without notice Business Continuity Plans and Backups • People make IT support a complex issue! • Disaster Recovery must be based on Business Continuity Plans and meet the requirements as set in the following question: What is the cost of downtime per hour? February 1, 2016 Webinar 2 Loss of Data - Most Feared Threat Human error 35 Systems failure 31 s Supply chain disruption t n 29 e Virus, worm or other malicious attack on IT systems d n 28 o Employee malfeasance (e.g. theft or fraud) p s 25 e r Natural disasters, such as fires or floods f o 22 Unplanned downtime of online systems % 22 Terrorism 16 Power outage 13 Pandemic 13 Application failure 12 Industrial Action 8 February 1, 2016 Webinar 3 Bootable System Images in Unix and Linux Many tools available. For the sake of brevity, the following will be discussed: AIX mksysb, Network Installation Manager (NIM) HP make_tape_recovery/make_net_recovery, Dynamic Root Disk (DRD)* Linux Mondo Rescue, Clonezilla Solaris ufsdump, fssnap+ufsdump, flash/JumpStart Tru64 btcreate February 1, 2016 Webinar 4 Tape Drives Limitations inherent with tape media: • A tape drive must be available on each system to be archived. • Must remove old tapes and insert new ones for new backups. • If an archive exceeds the capacity of a tape, you must swap tapes for both creation and extraction. • Must check log files and run dummy restores to ensure data consistency. • Tape drives are more error-prone than a local network or CD- ROM and DVD. • Cost of tapes in large environment is significant. • Cost of managing tape loading and storage is significant. • Generally slower that disk or LAN. February 1, 2016 Webinar 5 AIX – mksysb(1) Creates a backup of the operating system (root volume group). The file system image is in backup-file format. The tape format includes a boot image, a bosinstall image, and an empty table of contents followed by the system backup (root volume group) image. The root volume group image is in backup-file format, starting with data files and then any optional map files. February 1, 2016 Webinar 6 AIX – mksysb(1) Pros highlights: • For managing a single or limited number of servers. • Can be used for system cloning. • Use when servers are not networked. • Allows backup to tape drive (CD-ROM and DVD typically done through mkcd(1)). February 1, 2016 Webinar 7 AIX – mksysb(1) Cons highlights: • Identical tape drive is needed for an off-site recovery. • Cannot back up files that are mounted from a remote server. • If /usr is remote-mounted, you cannot reinstall system from backup image. • Image does not include data on raw devices or in user- defined paging spaces. • It may not restore all device configurations for special features, such as /dev/netbios and some device drives not shipped with the product. • LC_ALL environment variable should be unset (if non-C value). • Does not have built-in error checking to minimize problems when backing up an active file system. • Format specific to AIX (backup-file).* February 1, 2016 Webinar 8 AIX – mksysb(1) Examples # mksysb –i /dev/rmt0* # mksysb –m /dev/rmt1** # mksysb –i –e /dev/rmt1*** # mksysb /dev/rmt0 –V # mkcd –U –d /dev/cd0 –P –V rootvg**** # mkdvd –d /dev/cd1 February 1, 2016 Webinar 9 AIX – mksysb(1) Recovery • Boot off the tape drive. • Select option 3 “Maintenance mode for system recovery”. • Access your devices. • Restore files. • Boot. February 1, 2016 Webinar 10 AIX – NIM(1) • Requires a NIM master and the number of client instances. • NIM depends on certain protocols (NFS, bootp or DHCP, and TFTP). Older versions of AIX also required RSH and other RCMD commands, but in AIX 5.3 and above you can use basic nimsh or openssl. • NIM master must be at the highest level of AIX that it is required to support. • Now supports Linux installations too. February 1, 2016 Webinar 11 HP-UX – make_tape_recovery(1) Pros highlights: • For managing a single or limited number of servers. • Can be used for system cloning. • Use when servers are not networked. • Suitable tape drive must exist. • Allows backup to tape drive, CD-ROM, or DVD. • Interactive and non-interactive. • Supports tar (default), cpio or pax formats. • Supports LVM and VxVM. • Multi-tape CLUI only. • Configurable. • Versions 7.x and above have ability to block particular paths and protocols during inventory (see instl_adm(4)). February 1, 2016 Webinar 12 HP-UX – make_tape_recovery(1) Cons highlights: • Identical tape drive is needed for an off-site recovery. • Cannot back up files that are mounted from a remote server. • Does not have built-in error checking to minimize problems when backing up an active file system. • LVM disk mirrors not restored. • LVM physical extents allocated to a logical volume may be in a different location on a disk than before (consider extending contiguous volumes). • Cannot use remote tape drive.* February 1, 2016 Webinar 13 HP-UX – make_tape_recovery(1) checks # check_tape_recovery # copy_boot_tape –u /dev/rmt/0mn –b –d /tmp # lifls –l /tmp/bootimage # mt –f /dev/rmt/0mn rew # mt –f /dev/rmt/0mn fsf 1 # tar tvf /dev/rmt/0mn* February 1, 2016 Webinar 14 HP-UX – make_net_recovery(1) Pros highlights: • For managing a large number of servers. • Can be used for system cloning. • Use when servers are networked. • No tape, CD-ROM, or DVD drives needed. • Interactive and non-interactive. • Supports tar (default), cpio or pax formats. • Highly configurable. February 1, 2016 Webinar 15 HP-UX – make_net_recovery(1) Cons highlights: • Requires large space if many clients are backed up. • Ignite-UX bundles must be at same version on server and clients. • NFS used to save data to Ignite server (firewall issues, especially older versions of NFS)*. • Requires Ignite server for recoveries. February 1, 2016 Webinar 16 HP-UX – make_net_recovery Protocols and Ports 67 bootpd UDP2 Bootstrap Protocol Server - This service should function only if the server is a BOOTP/DHCP server 68 bootpd UDP Bootstrap Protocol Client - This service should function only if the server is a BOOTP server 69 tftpd UDP Trivial File Transfer Protocol - Found on systems that have Ignite-UX installed. This service should function only if the host is being used as a TFTP server 111 portmap/sunrpc/ rpcbind TCP/UDP SUN Remote Procedure Call (RPC) February 1, 2016 Webinar 17 HP-UX – make_net_recovery Protocols and Ports (continued) 135 rpcd/dced TCP Distributed Computing Environment (DCE)-based RPC 514 shell TCP Remote Command, No Password Used 1067 instl_boots UDP Installation Bootstrap Protocol Server 1068 instl_bootc UDP Installation Bootstrap Protocol Client 2049 nfsd TCP/UDP NFS Remote File System February 1, 2016 Webinar 18 HP-UX – make_net_recovery Protocols and Ports (continued) 2121 swagentd TCP/UDP HP Software Distributor Daemon - Used for communication between systems for software installation, listing, or other sw commands 4000 - 4009 secure swagent ports TCP/UDP The swagent firewall configurable ports 49152 - 65535 Dynamic or Private Ports TCP/UDP Dynamic and Private Ports are used by many applications for dynamic port assignments. UDP ports in this range are often RPC ports February 1, 2016 Webinar 19 HP-UX – Ignite(1) Examples # make_tape_recovery -v -a /dev/rtape/tape3 -x \ inc_entire=vg00 -d “mysrv Ignite“ # make_net_recovery -s remsrv -n 3 -P s -x inc_entire=vg00 \ -x exclude=/tmp -x exclude=/var/tmp –d “mysrv Ignite" February 1, 2016 Webinar 20 HP-UX – Dynamic Root Disk • DRD (current release A.3.1.0 - February 2008) runs on both Integrity and PA platforms running the following operating systems: HP-UX 11i v2 (11.23) September 2004 or more recent HP-UX 11i v3 (11.31) • Root group being cloned can be managed by any release of LVM on an O/S release supported by DRD. In addition, the root group can be managed by VxVM 4.1 (HP-UX 11i v2 or 11i v3) or VxVM 5.0 (HP-UX 11i v2 only). February 1, 2016 Webinar 21 HP-UX DRD Benefit: Minimizing Planned Downtime Without DRD: Software management may require extended downtime With DRD: Install/remove software on the clone while applications continue running Install patches lvol1 lvol1 lvol1 lvol1 on the clone; lvol2 lvol2 lvol2 lvol2 applications lvol3 lvol3 lvol3 lvol3 remain running boot diskboot mirror clone clone mirror disk vg00 (active) cloned vg00 (inactive/patched) Activate the lvol1 lvol1 lvol1 lvol1 lvol2 lvol2 lvol2 lvol2 clone to make lvol3 lvol3 lvol3 lvol3 changes take effect boot diskboot mirror clone clone mirror disk vg00 (inactive) cloned vg00 (active/patched) February 1, 2016 Webinar 22 HP-UX – Dynamic Root Disk Pros highlights: • Fully supported by HP. • Full clone. • Complements other parts of total HP solution by reducing system downtime required to install and update patches and other software. • Copy operation is currently done by fbackup and frecover. • Kctune(1) command can be used to modify kernel parameters in the clone. • The ioconfig file and the entire /dev directory are copied by the DRD clone operation, so instance numbers will not change when the clone is booted.* • Supports nPars, vPars, and Integrity VMs. February 1, 2016 Webinar 23 HP-UX – Dynamic Root Disk Pros highlights: • No tape drive is needed. • No impact on network performance. • No security issues of transferring data across the network. • All DRD processes, including drd clone and drd runcmd, can be safely interrupted issuing Control-C (SIGINT) from the controlling terminal or by issuing kill -HUP<pid> (SIGHUP). This action causes DRD to abort processing and perform any necessary clean up. Do not interrupt DRD using the kill -9 <pid> command (SIGKILL), which fails to abort safely and does not perform cleanup.