IBM Tivoli Workload Scheduler for z/OS
Customization and Tuning
Version8Release6SPE(RevisedDecember2012)
SC32-1265-07
IBM Tivoli Workload Scheduler for z/OS
Customization and Tuning
Version8Release6SPE(RevisedDecember2012)
SC32-1265-07 Note Before using this information and the product it supports, read the information in “Notices” on page 387.
This edition applies to version 8, release 6, modification level 0 SPE of Tivoli Workload Scheduler for z/OS (program number 5698-A17) and to all subsequent releases and modifications until otherwise indicated in new editions. © Copyright IBM Corporation 1999, 2012. US Government Users Restricted Rights – Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. Contents
Figures ...... vii RCLOPTS ...... 116 RCLSKIP ...... 123 Tables ...... ix RESOPTS...... 124 RESOURCE ...... 129 RODMOPTS...... 129 About this publication ...... xi ROUTOPTS ...... 133 What is new in this publication ...... xi Modifying the HTTP destinations .....137 Who should read this publication ...... xi SERVOPTS ...... 138 Publications ...... xii TCPOPTS ...... 143 Using LookAt to look up message explanations . . xii TOPOLOGY ...... 147 Accessibility ...... xiii TRGOPT ...... 147 Tivoli technical training ...... xiii TRROPTS ...... 149 Support information ...... xiii USRREC ...... 151 Conventions used in this publication .....xiii XCFOPTS ...... 151 How to read syntax diagrams ...... xiv Chapter 2. Identifying related Part 1. Customizing Tivoli Workload initialization-statement parameters . . 153 Scheduler for z/OS ...... 1 Configuration ...... 154 Security ...... 155 Chapter 1. Defining initialization Generating audit information (JT log data) . . . 156 statements ...... 3 Determining the success or failure of a job . . . 156 Recovery ...... 157 Specifying the statements ...... 3 Restart and cleanup ...... 158 Creating the statements ...... 3 Job log retrieval ...... 158 Storing the statements ...... 4 Automatic job recovery ...... 159 Overriding the EQQPARM statements .....4 Workload restart ...... 159 Choosing the appropriate statements .....4 Performance...... 160 ALERTS ...... 7 Reporting ...... 161 AROPTS ...... 11 RODM monitoring ...... 161 AUDIT ...... 13 Output processing...... 162 AUDITCP ...... 16 End-to-end scheduling with fault tolerance AUTHDEF ...... 16 capabilities ...... 162 BATCHOPT ...... 20 Network configuration ...... 163 CPUREC ...... 31 Job definitions ...... 164 DBCSOPTS ...... 32 Regional settings ...... 165 DBOPT ...... 33 WLM integration ...... 166 DOMREC ...... 36 External monitoring ...... 166 DSTOPTS ...... 37 DSTUTIL ...... 42 ERDROPTS ...... 44 Chapter 3. Implementing security . . . 169 EWTROPTS ...... 45 Planning security implementation ...... 169 EXITS ...... 50 How Tivoli Workload Scheduler for z/OS FLOPTS ...... 51 verifies access authority ...... 170 HTTPOPTS ...... 54 Identifying users ...... 171 INCLUDE ...... 59 Establishing naming conventions for IBM Tivoli INIT ...... 60 Workload Scheduler for z/OS resources . . . 172 INTFOPTS...... 64 Grouping RACF users and resources ....173 JCCOPTS ...... 66 General security considerations ...... 173 JTOPTS...... 69 Controlling access to Tivoli Workload Scheduler for MONOPTS ...... 93 z/OS ...... 174 MONPOL ...... 94 Controlling access to the Tivoli Workload NOERROR ...... 95 Scheduler for z/OS subsystem ...... 174 OPCOPTS ...... 100 Controlling access to Tivoli Workload Scheduler OUCOPTS ...... 114 for z/OS fixed resources...... 175 RCLDDP ...... 115 Controlling access to Tivoli Workload Scheduler RCLDSNP ...... 116 for z/OS subresources ...... 175
© Copyright IBM Corp. 1999, 2012 iii Controlling access to Tivoli Workload Scheduler Incident-record-create exit (EQQUX006) ....217 for z/OS from APPC ...... 177 Installing the exit ...... 218 Controlling access to Tivoli Workload Scheduler Interface to the exit ...... 218 for z/OS using Dynamic Workload Console . . 179 Operation-status-change exit (EQQUX007)....219 Controlling access through TSO commands . . 183 Installing the exit ...... 220 Functions and data that you can protect ....183 Interface to the exit ...... 220 Access requirements to fixed resources for dialog Operation-initiation exit (EQQUX009) .....223 users ...... 188 Installing the exit ...... 224 Examples of security strategies ...... 192 Interface to the exit ...... 224 A centralized security strategy...... 193 Job-tracking log write exit (EQQUX011) ....226 A decentralized security strategy ...... 194 Installing the exit ...... 226 Interface to the exit ...... 226 Chapter 4. Tivoli Workload Scheduler Job-tailoring event exit (EQQUX013)...... 227 for z/OS exits ...... 197 Installing the exit ...... 228 Interface to the exit ...... 228 Tivoli Workload Scheduler for z/OS user exits Time-dependent-operation exit (EQQUX014) . . . 230 invocation ...... 198 Installing the exit ...... 231 Start/stop exit (EQQUX000) ...... 198 Interface to the exit ...... 231 Job-submit exit (EQQUX001) ...... 198 EQQDELDS/EQQCLEAN Catalog exit Job-library-read exit (EQQUX002) .....199 (EQQUXCAT) ...... 233 Application-description-feedback exit Installing the exit ...... 233 (EQQUX003) ...... 199 Interface to the exit ...... 233 Event-filtering exit (EQQUX004) ...... 199 EQQCLEAN GDG Resolution exit (EQQUXGDG) 234 SYSOUT archiving exit (EQQUX005) ....199 DDPROT/DSNPROT interactions .....235 Incident-record-create exit (EQQUX006) . . . 199 Installing the exit ...... 235 Operation-status-change exit (EQQUX007). . . 199 Interface to the exit ...... 235 Operation-initiation exit (EQQUX009) ....199 JCL-imbed exit ...... 236 Job-tracking log write exit (EQQUX011) . . . 199 Installing the exit ...... 236 Job-tailoring prevention exit (EQQUX013) . . . 200 Interface to the exit ...... 236 Time-dependent-operation exit (EQQUX014) . . 200 Variable-substitution exit (on JCL or job definition JCL-imbed exit (on FETCH directive) ....200 variable) ...... 238 Variable-substitution exit (on JCL or job Installing the exit ...... 238 definition variable) ...... 200 Interface to the exit ...... 239 Automatic-job-recovery exit (on RECOVER Automatic-job-recovery exit ...... 241 statement) ...... 200 Installing the exit ...... 241 Daily-planning-report exit (EQQDPUE1) . . . 200 Interface to the exit ...... 241 EQQDELDS/EQQCLEAN catalog exit Daily-planning-report exit (EQQDPUE1) ....243 (EQQUXCAT) ...... 200 Installing the exit ...... 243 EQQCLEAN GDG resolution exit Interface to the exit ...... 243 (EQQUXGDG) ...... 200 Application-description-validation exit Application-description-validation (EQQUXPIF) 200 (EQQUXPIF) ...... 244 Daily-planning-scheduling environment exit Installing the exit ...... 244 EQQDPX01 ...... 201 Interface to the exit ...... 244 Tivoli Workload Scheduler for z/OS Start/Stop exit System Automation for z/OS user exit (EQQUX000) ...... 202 (EQQUXSAZ) ...... 245 Installing the exit ...... 202 Installing the exit ...... 246 Interface to the exit ...... 203 Interface to the exit ...... 246 Job-submit exit (EQQUX001) ...... 203 Installing the exit ...... 203 Interface to the exit ...... 204 Chapter 5. Open Systems Integration 249 Job-library-read exit (EQQUX002) ...... 207 Controlling heterogeneous systems ...... 249 Installing the exit ...... 208 Setting up the environment...... 250 Interface to the exit ...... 208 Submitting and tracking the workload ....250 Application-description-feedback exit (EQQUX003) 212 An alternate method for controlling VM processing 251 Installing the exit ...... 212 Method of use ...... 252 Interface to the exit ...... 213 Event-filtering exit (EQQUX004) ...... 213 Chapter 6. Reporting events to Tivoli Installing the exit ...... 214 Workload Scheduler for z/OS ....255 Interface to the exit ...... 214 Supplying event information to Tivoli Workload SYSOUT archiving exit (EQQUX005) .....215 Scheduler for z/OS ...... 255 Installing the exit ...... 215 Interface to the exit ...... 215 iv Tivoli Workload Scheduler for z/OS: Customization and Tuning: Customization and Tuning General information about Tivoli Workload Monitoring special resources through RODM. . . 303 Scheduler for z/OS subroutines ...... 256 Creating case-code-definition modules .....305 Using the general Tivoli Workload Scheduler for Invoking the data set deletion utility .....306 z/OS subroutine (EQQUSIN) ...... 257 Customizing Tivoli Workload Scheduler for Invocation requirements ...... 257 messages in end-to-end with fault tolerance EQQUSIN parameters ...... 257 capabilities environment...... 306 Specifying selection criteria...... 264 Specifying object fields to update...... 268 Part 2. Data integrity ...... 307 Return codes and reason codes generated by EQQUSIN ...... 271 Using individual Tivoli Workload Scheduler for Chapter 10. Backup and recovery of z/OS subroutines ...... 272 data sets ...... 309 Using EQQUSINB ...... 272 Backup procedures ...... 309 Using EQQUSINO...... 273 How Tivoli Workload Scheduler for z/OS manages Using EQQUSINS ...... 275 recovery of the current plan ...... 310 Using EQQUSINT ...... 276 Current-plan recovery principles ...... 310 Using EQQUSINW ...... 278 Current-plan recovery processing...... 312 Current-plan processing at Tivoli Workload Chapter 7. Using the Job Completion Scheduler for z/OS startup ...... 313 Checker...... 281 Avoiding corruption of the current plan backup 315 Restoring a damaged Tivoli Workload Scheduler JCC message tables ...... 281 for z/OS file from backup ...... 315 Incident logging function ...... 283 Restoring the Workstation Description (WS) data Defining message tables using EQQJCCT . . . 283 set...... 316 Restoring the Application Description (AD) data Chapter 8. Using the data store . . . 289 set...... 316 Overview...... 289 Restoring the Operator Instruction (OI) data set 316 Prerequisites...... 290 Restoring the Special Resource Description (RD) Installing the data store ...... 291 data set ...... 317 Running EQQJOBS to create installation samples 291 Restoring the Side Information (SI) data set . . 317 Estimating the size of data store VSAM data files 292 Restoring the Long-term Plan (LTP) data set . . 317 Data Files ...... 292 Restoring the JCL Repository (JS) data set . . . 318 Primary index ...... 293 Re-creating the current plan from the long-term Secondary index ...... 293 plan ...... 318 Characteristics of the local data store ....294 Re-creating the current plan from the Allocating data store VSAM ...... 294 new-current-plan and the JT archive log ....319 Data files ...... 294 Recovering from errors on the job-tracking log . . 320 Primary index ...... 294 Dual JT log data set problems ...... 320 Secondary index ...... 295 Recovering from errors on the JT archive log . . . 320 Initializing data store VSAM files...... 295 Recovering from errors on the checkpoint data set 321 Data files ...... 295 Recovering from errors on event data sets ....322 Primary index ...... 295 Recovering from errors on a submit/release data Secondary index ...... 296 set...... 322 Post-installation actions on data store VSAM files 296 Recovering from errors on the Configuring the data store ...... 297 current-plan-extension data set ...... 323 Data store initialization statements .....297 Automatic recovery from controller failures . . . 324 Set UP controller/tracker initialization Notification of controller failures ...... 324 statements ...... 298 Re-creating the symphony file from the current Considerations about RCLOPTS statements . . 298 plan ...... 324 Activating the data store ...... 298 Chapter 11. Cleanup and recovery of Chapter 9. Miscellaneous data store data sets ...... 325 customization ...... 299 Deleting data from the database ...... 325 Customizing Tivoli Workload Scheduler for z/OS Exporting data to a backup file ...... 326 messages ...... 299 Importing data from a backup file ...... 326 Customizing Tivoli Workload Scheduler for z/OS Recovering from failure ...... 327 panels ...... 300 What to do when data files fill up ...... 327 Customizing ended-in-error-list and ready-list Cleanup subtask ...... 328 default layouts ...... 301 Invoking hiperbatch support ...... 301 Customizing GMT clock...... 302
Contents v Chapter 12. Disaster recovery Tuning the TCP/IP server ...... 362 planning ...... 329 Restart and cleanup ...... 363 Designing your Tivoli Workload Scheduler for z/OS DRP ...... 329 Part 4. Appendixes ...... 365 Secondary-center options ...... 329 Standardizing the environment ...... 330 Appendix A. Tivoli Workload Implementing your Tivoli Workload Scheduler for Scheduler for z/OS macros .....367 z/OS DRP ...... 331 Recovery to start-of-day processing .....331 Recovery to a predefined recovery point . . . 334 Appendix B. Sample library Recovery to point-of-failure ...... 336 (SEQQSAMP) ...... 369 | Considerations about testing disaster recovery in EQQUSIN samples ...... 371 || an end-to-end environment...... 339 Tivoli Workload Scheduler for z/OS exits ....372 Start or stop exit ...... 372 Part 3. Tuning ...... 341 Job-submit exit ...... 372 Job-library-read exit ...... 373 Event-filtering exit...... 373 Chapter 13. Analyzing performance 343 SYSOUT archiving exit ...... 373 Setting performance objectives...... 343 Incident-record-create exit ...... 374 Measuring performance ...... 344 Operation-status-change exit ...... 374 Performance Reporter for MVS and Tivoli Operation-initiation exit ...... 374 Decision Support for OS/390 ...... 344 JCL-variable-substitution exit ...... 375 RMF...... 345 Job-tracking log write exit ...... 375 ACF/VTAM...... 346 EQQDELDS/EQQCLEAN catalog-exit ....375 VSAM...... 347 EQQCLEAN GDG resolution exit Tivoli Workload Scheduler for z/OS (EQQUXGDG) ...... 375 performance data ...... 347 Application-description-validation exit ....375 DP batch scheduling environment exit ....375 Chapter 14. Basic tuning activities 349 Job-tailoring prevention exit ...... 376 System resources ...... 349 Time Dependent Operation user exit ....376 I/O activity ...... 349 Open Systems integration ...... 376 Processor ...... 353 Tracker for VM ...... 376 Processor storage ...... 353 Tracker for OS/2 ...... 377 Common Storage Area allocated for user Tracker for AIX...... 378 connections ...... 354 Tivoli Workload Scheduler for z/OS audit package 379 Indicators for performance-related problems . . . 354 Viewing output from the ended-in-error list . . . 382 Preventing bottlenecks ...... 354 NetView samples ...... 383 Deadline WTO message ...... 383 Chapter 15. Tuning the controller, the Responding to WTO operations ...... 383 tracker, and the TCP/IP server ....357 Changing operation status from NetView . . . 383 z/OS hiperbatch support ...... 383 Tuning the controller ...... 357 Deleting data sets based on JCL disposition and Job submission ...... 357 catalog status ...... 384 Job tracking ...... 358 Miscellaneous samples ...... 385 Dialog response ...... 359 MASS update samples ...... 385 Background batch processing ...... 360 Recognizing the indicators ...... 360 Recommendations ...... 360 Notices ...... 387 Tuning the tracker ...... 361 Trademarks ...... 388 Event creation and communication .....361 Factors influencing performance ...... 361 Index ...... 391 JCC...... 362
vi Tivoli Workload Scheduler for z/OS: Customization and Tuning: Customization and Tuning Figures
1. Member RJJJJ in the Tivoli Workload 4. Controller, tracker, and data store - schematic Scheduler for z/OS JCL library .....252 view ...... 290 2. Using automatic-event reporting to control 5. EQQAUDIT report sample: header page 380 VM operations ...... 254 6. EQQAUDIT report sample: report pages 381 3. EQQUSIN buffer example ...... 258 7. EQQAUDIT report sample: summary page 382
© Copyright IBM Corp. 1999, 2012 vii viii Tivoli Workload Scheduler for z/OS: Customization and Tuning: Customization and Tuning Tables
1. Defining the appropriate initialization 25. Access requirements for IBM Tivoli Workload statements ...... 4 Scheduler for z/OS TSO commands ....183 2. Defining the initialization statements for 26. Protected fixed resources and subresources 184 end-to-end scheduling with fault tolerance 27. Access requirements to fixed resources for capabilities ...... 7 dialog users ...... 189 3. Limit for feedback examples ...... 78 28. Tivoli Workload Scheduler for z/OS exits 197 4. Smoothing factor examples ...... 84 29. CP_OPER_EVENT selection fields ....264 5. Initialization statements and related functions 153 30. CP_SR_EVENT selection fields...... 266 6. Configuration-related parameters .....154 31. CP_OPINFO_EVENT selection fields 266 7. Security-related parameters ...... 155 32. BACKUP_EVENT selection fields .....267 8. Auditing-related parameters ...... 156 33. CP_WS_EVENT selection fields .....267 9. Completion-checking-related parameters 156 34. Operation fields that you can update through 10. Restart and cleanup-related parameters 158 CP_OPER_EVENT ...... 268 11. Data store job log retrieval-related parameters 158 35. Special resource fields that you can update 12. Automatic-job-recovery-related parameters 159 through CP_SR_EVENT ...... 269 13. Workload-restart-related parameters ....159 36. Operation field that you can update through 14. Performance-related parameters .....160 CP_OPINFO_EVENT ...... 270 15. Reporting-related parameters ...... 161 37. Workstation fields that you can update 16. RODM-related parameters ...... 161 through CP_WS_EVENT ...... 270 17. Output-related parameters ...... 162 38. Valid RODM data types for value subfields 304 18. Network configuration-related parameters 163 39. Backup cycle for start-of-day DRP ....332 19. Job definition-related parameters .....164 40. Backup cycle for predefined recovery point 20. WLM integration-related parameters. 166 DRP...... 334 21. Parameters related to integration with 41. Backup cycle for point-of-failure DRP 337 external monitors ...... 166 42. SEQQSAMP library members for customizing 22. Security planning ...... 169 and tuning Tivoli Workload Scheduler for 23. Security implementation ...... 170 z/OS ...... 369 24. Relationship between security products and security selections ...... 180
© Copyright IBM Corp. 1999, 2012 ix x Tivoli Workload Scheduler for z/OS: Customization and Tuning: Customization and Tuning About this publication
IBM® Tivoli® Workload Scheduler for z/OS® Customization and Tuning shows you how to customize or tune IBM Tivoli Workload Scheduler for z/OS. You can use this publication for reference during installation.
Your workload can run on various platforms, but you control it from a central z/OS system that runs the Tivoli Workload Scheduler for z/OS controller.
The term scheduler, when used in this publication, refers to Tivoli Workload Scheduler for z/OS. The term DB2®, when used in this publication, refers to DATABASE 2 and DB2 Universal Database™.
What is new in this publication This section describes what has changed in this publication since version 8.5.1. Changed or added text is marked in the left margin with a vertical bar, except for: v Editorial changes v Release-dependent suffix changed from I (version 8.5.1) to J (version 8.6) for the modules EQQINITJ, EQQSSCMJ, and EQQMINOJ.
The following product enhancements are described in this publication: v Job logs can be configured to be automatically retrieved when a job ends in error. The automatic job log retrieval can be customized also for z-centric jobs that end in error. You can also configure to automatically retrieve the restart information for a restart and cleanup action. See the JOBLOGRETRIEVAL and RESTARTINFORETRIEVAL keywords in the “HTTPOPTS” on page 54 and “RCLOPTS” on page 116 statements. v New parameters added to the HTTPOPTS statement enable you take advantage of the variable substitution mechanism for the new job types with advanced options. See the VARTABLES, VARFAIL, VARSUB keywords in the “HTTPOPTS” on page 54 statement for more details. v Determine the permanence of external dependencies on a completed operation that belongs to occurrences that are still active when a daily plan job is submitted and either extended or replanned by specifying the KEEPCOMPDEPS keyword in the “BATCHOPT” on page 20 statement. This feature, if specified, facilitates a rerun operation because the external dependencies remain defined on the operation when the plan is extended and therefore there is no need to redefine them.
Who should read this publication This publication is intended for system programmers, security administrators, and other personnel who install, customize, or tune IBM Tivoli Workload Scheduler for z/OS.
To use this publication effectively, you need a working knowledge of z/OS and JES concepts and facilities. You should be familiar with the Interactive System Productivity Facility (ISPF), the Interactive System Productivity Facility/Program Development Facility (ISPF/PDF), and the Time-Sharing Option (TSO). A good working knowledge of Virtual Storage Access Method (VSAM) is desirable but not essential.
© Copyright IBM Corp. 1999, 2012 xi To implement security, you must know the Resource Access Control Facility (RACF®) or a similar product. To implement IBM Tivoli Workload Scheduler for z/OS exits or subroutines, you must know job control language (JCL) and have a good working knowledge of a programming language, for example, assembler or PL/I. You can use programming languages that support OS/390® linkage conventions and that can load and delete an assembler program.
Publications Full details of Tivoli Workload Scheduler publications can be found in Tivoli Workload Automation: Publications. This document also contains information about the conventions used in the publications.
A glossary of terms used in the product can be found in Tivoli Workload Automation: Glossary.
Both of these are in the Information Center as separate publications.
Using LookAt to look up message explanations LookAt is an online facility that lets you look up explanations for most of the IBM messages you encounter, as well as for some system abends (an abnormal end of a task) and codes. Using LookAt to find information is faster than a conventional search because in most cases LookAt goes directly to the message explanation.
You can use LookAt from the following locations to find IBM message explanations for z/OS elements and features, z/VM®, VSE/ESA, and Clusters for AIX® and Linux: v The Internet. You can access IBM message explanations directly from the LookAt website at http://www.ibm.com/eserver/zseries/zos/bkserv/lookat/. v Your z/OS TSO/E host system. You can install code on your z/OS system to access IBM message explanations, using LookAt from a TSO/E command line (for example, TSO/E prompt, ISPF, or z/OS UNIX System Services running OMVS). v Your Microsoft Windows workstation. You can install code to access IBM message explanations on the IBM Online Library z/OS Software Products Collection Kit (SK3T-4270), using LookAt from a Microsoft Windows DOS command line. v Your wireless handheld device. You can use the LookAt Mobile Edition with a handheld device that has wireless access and an Internet browser (for example, Internet Explorer for Pocket PCs, Blazer, or Eudora for Palm OS, or Opera for Linux handheld devices). Link to the LookAt Mobile Edition from the LookAt website.
You can obtain code to install LookAt on your host system or Microsoft Windows workstation from a disk on your IBM Online Library z/OS Software Products Collection Kit (SK3T-4270), or from the LookAt website (click Download, and select the platform, release, collection, and location that suit your needs). More information is available in the LOOKAT.ME files available during the download process.
xii Tivoli Workload Scheduler for z/OS: Customization and Tuning: Customization and Tuning Accessibility Accessibility features help users with a physical disability, such as restricted mobility or limited vision, to use software products successfully. With this product, you can use assistive technologies to hear and navigate the interface. You can also use the keyboard instead of the mouse to operate all features of the graphical user interface.
For full information with respect to the Dynamic Workload Console, see the Accessibility Appendix in the IBM Tivoli Workload Scheduler User’s Guide and Reference.
Tivoli technical training For Tivoli technical training information, refer to the following IBM Tivoli Education website:
http://www.ibm.com/software/tivoli/education
Support information If you have a problem with your IBM software, you want to resolve it quickly. IBM provides the following ways for you to obtain the support you need: v Searching knowledge bases: You can search across a large collection of known problems and workarounds, Technotes, and other information. v Obtaining fixes: You can locate the latest fixes that are already available for your product. v Contacting IBM Software Support: If you still cannot solve your problem, and you need to work with someone from IBM, you can use a variety of ways to contact IBM Software Support.
For more information about these three ways of resolving problems, see the appendix on support information in Tivoli Workload Scheduler: Troubleshooting Guide.
Conventions used in this publication The publication uses several typeface conventions for special terms and actions. Technical changes to the text are indicated by a vertical line to the left of the change. These conventions have the following meanings:
Information type Style convention Example Commands All capital letters CREATE References in the text to All capital letters QUANTITY fields on panels
File and directory names, Monospace MYAPPLICATION input you should type in panel fields First time new term Italics Application introduced, publication titles
About this publication xiii How to read syntax diagrams Throughout this publication, syntax is described in diagrams like the one shown here, which describes the SRSTAT TSO command: