D04 - Big Data - DB2 LUW Meta Data Repositories - A Field Guide for DBA's & Managers Speaker: A l e x a n d e r K o p a c Session Code: D04 Room: Sabine Date/Time: Monday, May 23, 2016 (05:00 PM - 06:00 PM) Platform: DB2 for LUW Renaissance Austin Hotel 9721 Arboretum Boulevard Austin, TX 78759, USA Photo by Steve from Austin, TX, USA Abstract For more than a decade, Big Data has been described by "Volume, Variety, and Velocity (or the Three V's)". DB2 database practitioners are in a position to help provide and sustain the fourth and fifth V's that are Veracity and Value. With the advent of IBM Big Data systems, there are many new DB2 LUW repository buzzwords including IIS, SPSS, C&DS, ADM to name a few that database practitioners will soon be familiar. These acronym heavy systems include Statistical Package for the Social Sciences (SPSS), IBM SPSS Modeler (SPSS), SPSS Collaboration and Decision Support (C&DS) Modeler, Analytical Decision Management (ADM), and IBM Infosphere Information Server (IIS) DataStage to name several. All of them require attention to detail. This session is designed to highlight many of the items that DBA's and managers care most to help reduce or eliminate the late night phone calls and keep your companies lights on. 2 Overall Session Objectives 1) Database system builds & configuration challenges 2) Security options discussion - root /non-root installation 3) Database maintenance, alerts, and utility operations 4) DB2 LUW Fixpack migration 'opportunities' 5) Summary 3 D04 - “Big Data - DB2 LUW Meta Data Repositories - A Field Guide for DBA's & Managers () Abstract Number: 1340 Track: DB2 for LUW - Track II Session Type: Podium Presentation Primary Presenter: A L E X A N D E R K O P A C a l e x a n d e r _ k o p a c @ h o t m a i l . c o m Room(s): Sabine Time(s): Monday, May 23, 2016 (05:00 PM - 06:00 PM) 4 D04 - “Big Data - DB2 LUW Meta Data Repositories - A Field Guide for DBA's & Managers” Sub-Title: What Big Data Padawans need to know . Background Info OS Components SuSE 11 WAS 8.5.5.4 SuSE 11 DB2 V10.5.* SuSE 11 IIS SuSE 11 CDC OCFS2 / NFS Shared file systems RedHat 6.5 WAS 8.5.5.4 RedHat 6.5 WAS 8.5.5.4 5 Presentation Legend This presentation is designed for beginners & for the 40% of folks who are visual learners; (visual repetitions & practice make for easier learning). The subject matter is complex; this presentation only scratches the surface. GENERICS used throughout this presentation: • Server name: serverxx, serverxy • Database name: sampdb • Instance Name: db2instx, samp01, samp01f • Tablespace Name: TBSPNAME • Table name: ACRONYNM • Schema name: MUSER • User name: MUSER01 6 Where Are We . 1) Introduction 2) Database system builds & configuration challenges 3) Security options discussion - root /non-root installation 4) Database maintenance, alerts, and utility operations 5) DB2 LUW Fixpack migration 'opportunities' 6) Summary 7 Introduction “Any sufficiently advanced technology is indistinguishable from magic.” Sir Arthur C. Clarke (1917 - 2008) 8 “Terms of Endearment” IBM Big Data Sstems Buzzwords 1. Statistical Package for the Social Sciences (SPSS) 2. IBM SPSS Modeler (SPSS) 3. SPSS Collaboration and Decision Support (C&DS) Modeler 4. Analytical Decision Management (ADM) 5. IBM Infosphere Information Server (IIS) DataStage 9 More Info - Redbooks Big Data • Information Governance Principles and Practices for a Big Data Landscape Published: 31 March 2014 - 280 pages http://www.redbooks.ibm.com/abstracts/sg248165.html?Open • Implementing IBM InfoSphere BigInsights on IBM System x • Published: March 27, 2014 http://www.redbooks.ibm.com/abstracts/sg248077.html?Open • Performance and Capacity Implications for Big Data Published: January, 2014 http://www.redbooks.ibm.com/redpapers/abstracts/redp5070.html?Open 10 Where Are We . 1) Introduction 2) Database system builds & configuration challenges 3) Security options discussion - root /non-root installation 4) Database maintenance, alerts, and utility operations 5) DB2 LUW Fixpack migration 'opportunities' 6) Summary 11 Redefinition needed in Webster's for knee jerk reactions to include … Blame the RDMS Database/DBA Knee jerk reaction Also found in: Thesaurus, Medical, Wikipedia1. Related to Knee jerk reaction: Knee jerk reflex Webster's Revised Unabridged Dictionary, published 1913 by C. & G. Merriam Co. an immediate unthinking emotional reaction produced by an event or statement to which the reacting person is highly sensitive; - in persons with strong feelings on a topic, it may be very predictable. 12 Pointing the finger away from DB2 • Train your WebSphere / Application folks to first choose to bounce the JVMs / Application when their log shows that they are unable to connect ! • App logs are normally generally not accessible on WebSphere / Application servers to the DBA Team. • TIP: Get read access to WebSphere / Application server logs at a minimum 13 HADR & TSAMP alarm perplexity amazement terror anxiety trepidation awe distraction bewilderment fear confusion muddle consternation shock dread stupefaction fright wonder horror muddlement panic trepidity 14 Avoid connection issues finger pointing with Application team • Place DB2 ‘client’ software on the WebSphere tier, Engine Tier, Application Tier, and C&DS Tier servers ; build an intermittent crontab script call to simply “connect” to the active database metadata repository servers & send txt msg or email alert when connect attempt fails. • db2pd -osinfo is a resource cheap alternative in a WebSphere tier, Engine Tier, Application Tier, and C&DS Tier servers script: Consider sending an alert from the WebSphere tier, Engine Tier, Application Tier, and C&DS Tier servers sent for FreeMem memory say less than 2000 mg to HADR Primary/Standby servers and for connect failures as well: • Physical Memory and Swap (Megabytes): Healthy Memory • TotalMem FreeMem AvailMem TotalSwap FreeSwap • 20480 6972 n/a 8192 8169 15 Know Location of the WebSphere/App Logs Obtain Read Access WebSphere Application Server IIS (IBM Information Server) SystemOut*.log & SystemErr*.logs Always sent/attached to PMR’S Send Server version.xml Send a description of the queries executed Send screenshots of errors coupled with timestamp of when the errors occurred 16 Sample location of Logs /tstapps/IBM/WebSphere/AppServer/profiles/AppSrv01/logs/serverxy/SystemOut*.log /tstapps/IBM/WebSphere/AppServer/profiles/AppSrv01/logs/serverxz/SystemErr*.log 17 Where Are We . 1) Introduction 2) Database system builds & configuration challenges 3) Security options discussion - root /non-root installation 4) Database maintenance, alerts, and utility operations 5) DB2 LUW Fixpack migration 'opportunities' 6) Summary 18 As a Matter of Fact . • Security Always Matters • Veracity Always Matters • Backups & Log Health Always Matters • Uptime Always Matters • Performance Always Matters • Value to the Business Always Matters 19 root /non-root installation • Avoid root installation – installs instance defaults :<( • OK for POC only – (no backup; backups are after thoughts) • Drop DB2 Instance / server as soon as reasonable ! • Non-root installation preferred :>) • Choose your both instance – fenced id names wisely! • Establish sudo access to install DB2 engine • Establish sudo access to perform specific DB2 functions • Establish sudo access to perform normal TSAMP functions • Establish sudo access to perform TSAMP getsadata traces 20 Refreshing/Copying data to Test Systems Become familiar with SSL Certificates Become familiar with the various hash algorithms Employ SHA-256; (not SHA-1) Root will normally install SSL Certificates as separation of duties task. 21 Change All Defaults • Change all defaults to be unique for your application • Do not share DB2 Instance or name for both Test & Production • Change the Default DB2 Instance Owner id & fenced-id name • from <db2inst1> & <db2fenc1> to something unique • Change the Default Port used for the DB2 Instance from 50000 • Employ sudo for DB2 Instance Owner id and for IBM TSAMP ‘getsadata’ logs and tools Attend Dave Beulke’s Session A14 - DB2 Security Best Practices: Protecting your system from the Legions of Doom Thu, May 26, 2016 (08:00 AM - 09:00 AM) 22 Change All Defaults Prior to DB2 instance installation echo "AIX" ; hostname ; date ; sudo /opt/IBM/db2/V10.5.7/instance/db2icrt -u db2fencx db2instx to echo "AIX" ; hostname ; date ; sudo /opt/IBM/db2/V10.5.7/instance/db2icrt -u samp01f samp01 echo "LINUX" ; hostname ; date ; sudo /opt/IBM/DB2/V10.5.7/instance/db2icrt -u db2fencx db2instx to echo "LINUX" ; hostname ; date ; sudo /opt/IBM/DB2/V10.5.7/instance/db2icrt -u samp01f samp01 23 Sample sudo -l (non-root installs) practicing injection prevention Mon Jan 25 06:15:31 EST 2016 serverxx samp01 SAMPDB No more wildcarding sudo /db2home/db2instx=> sudo -l User db2instx may run the following commands on this host: (ALL) NOPASSWD: /usr/bin/getsadata, /bin/rm -rf /tmp/*serverxx*, /bin/cat /tmp/db2icrt*, /bin/rm /tmp/db2icrt*, /usr/bin/addrgmbr, /usr/bin/addrpnode, /usr/bin/cfgsamadapter, /usr/sbin/rsct/bin/chequ, /usr/sbin/rsct/bin/chrel, /usr/sbin/rsct/bin/chrg, /usr/sbin/rsct/bin/chrgmbr, /usr/sbin/rsct/bin/chrsrc, /usr/sbin/rsct/bin/lsequ, /usr/sbin/rsct/bin/lsrel, /usr/sbin/rsct/bin/lsrg, /usr/sbin/rsct/bin/lsrgreq, /usr/sbin/rsct/bin/lsrpdomain, /usr/sbin/rsct/bin/lsrpnode, /usr/sbin/rsct/bin/lsrsrc, /usr/sbin/rsct/bin/lssam, /usr/sbin/rsct/bin/lssamctrl, /usr/bin/lssrc, /usr/sbin/rsct/bin/mkequ, /usr/sbin/rsct/bin/mkrel, /usr/sbin/rsct/bin/mkrg, /usr/sbin/rsct/bin/mkrpdomain,