Additional AIX Security Tools on IBM Pseries, IBM RS/6000, and SP/Cluster
Total Page:16
File Type:pdf, Size:1020Kb
Additional AIX Security Tools on IBM pSeries, IBM RS/6000, and SP/Cluster Customize the security of your pSeries systems Explore IBM, non-IBM, and freeware security tools Learn new approaches to security Abbas Farazdel Marc Genty Bruno Kerouanton Chune Keat Khor ibm.com/redbooks SG24-5971-00 International Technical Support Organization Additional AIX Security Tools on IBM ^ pSeries, IBM RS/6000, and SP/Cluster December 2000 Take Note! Before using this information and the product it supports, be sure to read the general information in Appendix D, “Special notices” on page 211. First Edition (December 2000) This edition applies to AIX 4.3.3. Comments may be addressed to: IBM Corporation, International Technical Support Organization Dept. JN9B Mail Station P099 2455 South Road Poughkeepsie, NY 12601-5400 When you send information to IBM, you grant IBM a non-exclusive right to use or distribute the information in any way it believes appropriate without incurring any obligation to you. © Copyright International Business Machines Corporation 2000. All rights reserved. Note to U.S Government Users – Documentation related to restricted rights – Use, duplication or disclosure is subject to restrictions set forth in GSA ADP Schedule Contract with IBM Corp. Contents Preface...................................................vii The team that wrote this redbook. ..................................viii Commentswelcome..............................................ix Chapter 1. Introduction ......................................1 1.1Securityframework-Overview...............................1 1.2 Security framework - Planning . ............................3 1.3Securityframework-Architecture.............................6 1.4Securityframework-Implementation..........................7 1.5Securityframework-Monitoring..............................8 1.6 Security framework - Incident response . ......................10 1.7Nextsteps.............................................12 Chapter 2. Firewalls ........................................13 2.1 Misconceptions about firewalls . ...........................13 2.2Typesoffirewalls........................................15 2.2.1Staticpacketfilterfirewalls.............................15 2.2.2Circuitlevelfirewalls..................................15 2.2.3Applicationlayer(proxy)firewalls........................16 2.2.4Dynamicpacketfilterfirewalls..........................16 2.2.5Comparisonbetweenthedifferenttypes...................16 2.3Firewalldesigns.........................................17 2.3.1Basicfirewalldesign..................................17 2.3.2 Firewalls with demilitarized zone (DMZ) . ................18 2.3.3Compartmentalizedfirewallenvironmentdesign.............21 2.4Securingfirewalls........................................24 2.5FirewallsonAIX.........................................24 Chapter 3. Check Point FireWall-1.............................25 3.1FireWall-1features.......................................25 3.2ComplementarysoftwareforFireWall-1.......................26 3.3SecuringFireWall-1......................................27 3.3.1 Closing vulnerabilities during system startup ...............28 3.3.2 Managing FireWall-1 logs . ...........................29 3.3.3SecuringFireWall-1defaultconfigurations.................29 3.3.4Creatingausefulrulebase.............................34 3.3.5 Viewing connections. .................................36 3.3.6 Enabling other defense mechanisms .....................37 3.4ListofportsthatCheckPointFireWall-1uses..................43 © Copyright IBM Corp. 2000 iii Chapter 4. IBM Secureway Firewall ............................47 4.1IBMSecurewayFirewallfeatures............................47 4.2ComplimentarysoftwareforIBMSecurewayFirewall.............48 4.3 Firewall hardening . ......................................49 4.4NetworkSecurityAuditor(NSA).............................49 4.4.1 Installing NSA. ......................................49 4.4.2UsingNSA.........................................50 4.4.3InterpretingNSAoutput...............................51 Chapter 5. Secure remote access .............................59 5.1SecureShell(ssh).......................................59 5.1.1ObtainingSSH......................................61 5.1.2DifferencebetweenSSH1andSSH2.....................62 5.1.3KeyconceptsofSSH.................................62 5.1.4 Installing OpenSSH on AIX. ...........................65 5.1.5 OpenSSH using SSH1 ................................68 5.1.6 OpenSSH using SSH2 ................................71 5.1.7 Other interesting SSH daemon configuration options .........75 5.1.8 SSH2 interoperability between OpenSSH and SSH.Com . .....76 5.1.9SSHclientsforthePC................................76 5.1.10ImplicationsofhavingSSH............................77 5.1.11AlternativestoSSH.................................77 5.2 TCP Wrapper ...........................................77 5.2.1 Obtaining and installing TCP Wrapper ....................78 5.2.2 Configuring TCP Wrapper. ...........................79 5.2.3 Additional TCP Wrapper security features . ................82 Chapter 6. Port and network scanning .........................83 6.1fping..................................................84 6.1.1 Obtaining and installing fping ...........................85 6.1.2Usingfping.........................................86 6.1.3 Protection against ping sweeps . ......................88 6.2 Network Mapper (NMAP) . .................................89 6.2.1 Obtaining and installing nmap...........................90 6.2.2Nmapusage........................................92 6.2.3 Protection against port scanners . ......................94 6.3SecurityAdministrator'sIntegratedNetworkTool(SAINT).........94 6.3.1 Obtaining and installing SAINT ..........................95 6.3.2UsingSAINT........................................98 6.4PortSentry.............................................98 6.4.1 Obtaining and installing PortSentry. ......................99 6.4.2DefenseprovidedbyPortSentry........................103 6.5ListOpenFiles(lsof)....................................103 iv Additional Security Tools for AIX Systems 6.5.1 Installing lsof . .....................................104 6.5.2Usinglsof.........................................105 6.6Intrusiondetection......................................106 Chapter 7. System and data integrity .........................109 7.1Tripwire..............................................110 7.1.1 Obtaining and installing Tripwire. .....................111 7.1.2ConfiguringandusingTripwire.........................112 7.1.3ConfiguringTripwire.................................114 7.1.4Commentsonconfiguration...........................118 7.1.5 When should Tripwire be run ..........................118 7.1.6AlternativestoTripwire...............................119 7.2 John the Ripper . .....................................119 7.2.1 Obtaining and installing John the Ripper . ...............120 7.2.2 Configuring John the Ripper . ..........................121 7.2.3 Using John the Ripper ...............................122 7.3PrettyGoodPrivacy(PGP)................................124 7.3.1PGPbasics.......................................124 7.3.2 Obtaining and installing PGP ..........................126 7.3.3UsingPGP........................................127 7.3.4Protectingyourprivatekey............................134 7.4MD5.................................................134 7.4.1 Ensuring the integrity of downloads .....................136 Chapter 8. Securing AIX....................................139 8.1Overview.............................................140 8.2 Step 1: Remove unnecessary services. .....................141 8.2.1Removingentriesfrom/etc/inittab......................142 8.2.2Removingentriesfrom/etc/rc.tcpip.....................144 8.2.3Removingentriesfrom/etc/inetd.conf....................148 8.3Step2:Tightenconfigurationsofremainingservices............153 8.3.1DomainNameSystem(DNS)..........................153 8.3.2NetworkFileSystemandNetworkInformationService.......168 8.3.3SimpleMailTransferProtocol(SMTP)...................175 8.3.4 Simple Network Management Protocol (SNMP) . ..........180 8.3.5TrivialFileTransferProtocol(TFTP).....................181 8.3.6SecuringX11......................................182 8.3.7FileTransferProtocol(ftp)............................184 8.3.8ProtectingTCPservicesusingSOCKS...................186 8.4 Step 3: Set proper network (no) options . .....................186 8.4.1SYNattackprotection................................187 8.4.2 Broadcast protection. ................................187 8.4.3IProutingoptions...................................188 v 8.5 Step 4: Tighten up user accounts . ..........................189 8.5.1 Removing unnecessary default accounts . ...............189 8.5.2Settinguserattributes...............................190 8.5.3Securingroot......................................192 8.5.4Otherattributes....................................193 8.6Step5:Setupstrongpasswordpolicy.......................194 8.6.1Modifyinguserpasswordattributes......................194 8.6.2Passwordcrackerutility..............................197 8.7 Step 6: Install additional security tools . .....................197 8.8 Step 7: Monitor logs, audit trails, and system behavior . ..........200 8.8.1Monitorsystemlogs.................................201 8.8.2 Enable auditing.....................................201 8.8.3Monitorfilesanddirectories...........................202 8.8.4Monitorcronandatjobs..............................203