Download in a Separate Pdf File
Total Page:16
File Type:pdf, Size:1020Kb
Note: Appendices are available for download in a separate pdf file. But we NEED to BE developers! DBAs frequently (constantly?) get called on to develop “quick” database scripts and programs, never mind that we don‟t have the time, the tools, or the training. From a quick, little runstats script in ksh, to a full blown application to load data or “fix a few rows”, it seems that if the data is in the database, it‟s my job to fix it! This presentation will present examples of manipulating DB2 data in several programming languages commonly used with DB2 LUW ( ksh, perl/dbi, SQL/PL, java/jdbc ) and discuss the plusses and minuses of each. Heavy emphasis will be placed on error handling, as this is one of areas that is most often overlooked or under developed. My favorite utility will be explained and sample code in perl and java will be discussed. 1 Next, I will share some of my favorite utilities, tools, tips and tricks to help the DBA get things done. 1 Show similar common DBA tasks in different languages, and give you some “tips and tricks” for each. Number 1 Depends on your DBA job description. But, for some of us… It is a novel idea that needs to be evangelized. 2 As a DBA, we are constantly tasked with fixing data, loading tables, and running utilities. It seems like, “If it lives in a database table, It‟s our responsibility to load, correct, and maintain.” We need to have and use the proper tools to do this. Many developers with LOTS of tools. One DBA, with…. Ksh and CLP. 3 This is the proper attitude for us to have. As long as we have a great set of tools to work with! However, UNLIKE Jeff Spicoli, we need to actually HAVE good tools and know how to use them. 4 Show similar common DBA task in three different languages, and give you some “tips and tricks” for each. Favorite bumper sticker – Think Geek used to sell them. Woodshop analogy: • If I want to help you get into woodworking: • Show you some cool tools: • Miter saw • Band saw • Planer • Jointer • All “cut” wood. All very cool. All useful. • Good in different ways and in different cases • Show you a cool “Chest of drawers” that you can build with it • That is what I am going to do with software tools 5 Show similar common DBA task in different languages, and give you some “tips and tricks” for each. Many times we are going into a fight with no weapons. (Or at least the wrong ones. ) Others? Python XML 6 Some good Manuals from the DB2 documents. http://www-01.ibm.com/support/docview.wss?uid=swg27038855 7 Note: Typeset –u – uppercase dbname ${1-sample} – default value for a parameter Backticks ` ` to run a command and capture it‟s output We are running db2 commands via the clp Five GLARING errors or Issues with this script: 1) No error checking What if you don‟t get something good back from db2? What if connect fails? 2) No way to check return codes from “underneath” tee command 3) No return code on the exit 4) “race condition” exists on select vs delete. The time between them is >0, so they may get different row count as rows may get old enough to be purged. 5) NO Intermediate COMMITS!!!!! Will this blow up your logs? What if it doesn‟t run for a while, then gets run? This is hard to fix in ksh / clp 8 Note: Trying to handle the “race” condition by using start_ts (which is, The time at which I start purging rows) Use count as the Number of rows to delete Using an error handling function: sql_error_handler Part of my standard library simply a case statement on rc 0 – ok 1 – no rows found 4 – warning 8 - error 9 Note: Looping until we delete enough rows. Committing every “chunk_sz” rows Using variable output to capture sqlca info from “db2 –a” purge_row is: number of rows from sqlca sqlcode: from sqlca Issues: Must rerun query to determine deletable rows over and over – no cursor. This could be really expensive! 10 This is an example of what “db2 –a” output looks like, so that you can see what I am “cut”ting out. This is somewhat fragile, should DB2 change this output. Not prod ready IMHO… Not sufficiently advanced technology - I can tell it from Magic 11 DB2 CLP is limited by the SQLCA - 32K buffer size “Panel” at idug in philly had a suggestion (that I haven‟t gotten to work, yet) WLM_SET_CLIENT_INFO procedure Have been forced to do some work in “O-rackle” lately. I MISS the CLP… 12 Need: A library of tools / functions. 13 See Appendix A Use of function in ksh Comments! They are GOOD! Check to see if the log file exists. If not, see if I can touch the log file and if so, can I write to it.? If not, write out reasonable errors and die with a RC 1. 14 Note: If both stdout (1) and stderr (2) are a tty, then I must be on a terminal. Basically, I am sending stdout and stderr to the log file, or if I‟m a terminal, I send it to tee and push it to the tty and the log file. use of print – more POSIX compliant vs echo. 15 See Appendix A This just shuts down the file descriptors using exec and makes sure to wait for the children to actually return. Maybe it should check to see if the logging is actually started, before it tries to stop it… 16 Note: “Source” in our .profile – big deal if running from cron Cron environment VERY different from your login environment. None of YOUR “stuff” .profile, .login pulled in. Cron environment is a “crippled” environment. Pull in our function library. Use of basename to get program name Fully qualified program names – good practice especially if running from cron ($PATH may not be your $PATH) Function call: Call to start_logging – passes in a log file name (This is verified in the function) stop_logging – no parameters Reading input from a file on a specific file descriptor – not commonly done in ksh. In this program, I also read from the terminal, so I need to use a separate “fd” for this input file. Continue statement { } – use – delimiter – reduces ambiguity – BEST practice! LOGFILE=${RECIPEFILE%.*}.log - strips anything to the right of the first „.‟ 17 Note: Good to have a heading with common stuff in it. Options for ksh: xtrace (-x) - debug nounset – will die if you don‟t have a param set errexit - can make error handling easier 18 Note: Co-worker who did this trick all the time. I had never seen it. Concern: What if you don‟t get exactly what you expect, when you read? Can you catch all possible weirdness??? - Buffer overflows, and SQL injections in this kind of thing, are very possible. 19 Note: Co-worker who did this trick all the time. I had never seen it. Concern: What if you don‟t get exactly what you expect, when you read? Can you catch all possible weirdness??? - Buffer overflows, and SQL injections in this kind of thing, are very possible. 20 From my backup script: All this just to subtract one from the log file name! Could do MUCH more easily in perl. 21 See Appendix B So, let‟s move on to Stored Procedures. 22 See Appendix B 23 Documentation implies that you have to do ENABLE to get the Output buffer to work. I Never have before. However, this time, I wrote more than the default 20,000 bytes and the proc would die with a SQL20511N. Yikes! This ENABLE Proc makes it “work”, but IMHO, it is still very fragile, if I log a bit too much, my proc will break. 24 25 26 27 He contends that this is much more efficient than the previous cursor based Stored proc. Serge knows more about SQL and the optimizer than I. I believe that it IS faster than the cursor version. But I notice that it does do the select multiple times. Which can be bad if the select is expensive. There are many variables, such as how expensive the select is, how the cursor is handled, i.e.: how many rows are buffered on each fetch, etc. 28 29 30 Here is a way to solve the “wrap a variable in single quotes but still let the shell inside the quotes to evaluate the variable” issue. I used a here document. (which adds a gratuitous newline). I used tr to chop the new line off. 31 When I started this presentation, I thought that I could create this little wrapper script quickly. I spent hours getting this „single quote‟ issue resolved. Still doesn‟t work in ksh/Cygwin on windows. ??? Anyone?? - superceded… Note: Solved!!!! TRIMMED_TS=`echo ${QUOTED_TS} | tr -d '\n‘ | tr –d ‘\r’` windows newline. I have never, BTW, gotten the perl/dbi working against db2 on Cygwin. Anyone? 32 The DBI is a library that provides database routines that are agnostic about the underlying database. The various DBDs provide routines to support dbi calls for a given database manager. 33 The DBI is a library that provides database routines that are agnostic about the underlying database. The various DBDs provide routines to support dbi calls for a given database manager. May need to recompile perl for AIX to get performance enhancements, Multi- threading, etc.