C:\Program Files\Adobe\FrameMaker8\UniData 7.2\7.2rebranded\ADMINUNIX\ADMINUNIXTITLE.fm March 5, 2010 1:34 pm

Beta Beta Beta Beta Beta Beta Beta Beta Beta Beta Beta Beta Beta Beta Beta Beta

UniData

Administering UniData on UNIX Platforms

UDT-720-ADMU-1 C:\Program Files\Adobe\FrameMaker8\UniData 7.2\7.2rebranded\ADMINUNIX\ADMINUNIXTITLE.fm March 5, 2010 1:34 pm

Beta Beta Beta Beta Beta Beta Beta Beta Beta Beta Beta Beta Beta

Notices

Edition Publication date: July, 2008 Book number: UDT-720-ADMU-1 Product version: UniData 7.2

Copyright © Rocket Software, Inc. 1988-2010. All Rights Reserved.

Trademarks The following trademarks appear in this publication:

Trademark Trademark Owner

Rocket Software™ Rocket Software, Inc.

Dynamic Connect® Rocket Software, Inc.

RedBack® Rocket Software, Inc.

SystemBuilder™ Rocket Software, Inc.

UniData® Rocket Software, Inc.

UniVerse™ Rocket Software, Inc.

U2™ Rocket Software, Inc.

U2.NET™ Rocket Software, Inc.

U2 Web Development Environment™ Rocket Software, Inc.

wIntegrate® Rocket Software, Inc.

Microsoft® .NET Microsoft Corporation

Microsoft® Office Excel®, Outlook®, Word Microsoft Corporation

Windows® Microsoft Corporation

Windows® 7 Microsoft Corporation

Windows Vista® Microsoft Corporation

Java™ and all Java-based trademarks and logos Sun Microsystems, Inc.

UNIX® X/Open Company Limited

ii SB/XA Getting Started The above trademarks are property of the specified companies in the United States, other countries, or both. All other products or services mentioned in this document may be covered by the trademarks, service marks, or product names as designated by the companies who own or market them.

License agreement This software and the associated documentation are proprietary and confidential to Rocket Software, Inc., are furnished under license, and may be used and copied only in accordance with the terms of such license and with the inclusion of the copyright notice. This software and any copies thereof may not be provided or otherwise made available to any other person. No title to or ownership of the software and associated documentation is hereby transferred. Any unauthorized use or reproduction of this software or documentation may be subject to civil or criminal liability. The information in the software and documentation is subject to change and should not be construed as a commitment by Rocket Software, Inc. Restricted rights notice for license to the U.S. Government: Use, reproduction, or disclosure is subject to restrictions as stated in the “Rights in Technical Data- General” clause (alternate III), in FAR section 52.222-14. All title and ownership in this computer software remain with Rocket Software, Inc.

Note This product may contain encryption technology. Many countries prohibit or restrict the use, import, or export of encryption technologies, and current use, import, and export regulations should be followed when exporting this product. Please be aware: Any images or indications reflecting ownership or branding of the product(s) documented herein may or may not reflect the current legal ownership of the intellectual property rights associated with such product(s). All right and title to the product(s) documented herein belong solely to Rocket Software, Inc. and its subsidiaries, notwithstanding any notices (including screen captures) or any other indications to the contrary.

Contact information Rocket Software 275 Grove Street Suite 3-410 Newton, MA 02466-2272 USA Tel: (617) 614-4321 Fax: (617) 630-7100 Web Site: www.rocketsoftware.com

SB/XA Getting Started iii Beta Beta Beta Beta Beta Beta Beta Beta Beta Beta Beta Beta Beta Beta Beta Beta Table of Contents Table of Contents

Chapter 1 Introduction Introduction ...... 1-2 Audience ...... 1-3

Chapter 2 UniData and UNIX Security UNIX File Permissions ...... 2-3 Additional UNIX Access Modes ...... 2-5 UNIX umask ...... 2-7 UniData Default Permissions ...... 2-9 UniData Processes and root ...... 2-10

Chapter 3 UniData and the UNIX File System UniData Directories and Files ...... 3-3 Files, Pointers, and Links...... 3-5 Creating Files...... 3-5 Setting a UniData Pointer ...... 3-5 Setting an Environment Variable ...... 3-7 Setting a UNIX Link ...... 3-8 UniData Hashed Files ...... 3-10 Static Files...... 3-10 Dynamic Files ...... 3-11 Sequentially Hashed Files ...... 3-13 DIR-Type Files ...... 3-16 Multilevel Files ...... 3-16 Multilevel Directory Files ...... 3-18 Index Files and Index Log Files ...... 3-19 UniData and tmp Space ...... 3-21 Changing TMP in the udtconfig File ...... 3-22 Setting an Environment Variable ...... 3-22

:\Program Files\Adobe\FrameMaker8\UniData 7.2\7.2rebranded\ADMINUNIX\ADMINUNIXTOC.fm bookTOC template) Chapter 4 UniData and Daemons What Is a Daemon? ...... 4-3 Principal UniData Daemons ...... 4-4 Shared Basic Code Server (sbcs) ...... 4-4 Shared Memory Manager (smm) ...... 4-5 Clean Up (cleanupd)...... 4-6 UniRPC Service (unirpcd) ...... 4-6 sync Daemon ...... 4-7 Monitoring UniData Daemons ...... 4-8 showud Command ...... 4-8 Log Files ...... 4-8

Chapter 5 UniData and Memory UNIX and Shared Memory ...... 5-3 UniData and Shared Memory ...... 5-4 smm and Shared Memory ...... 5-4 sbcs and Shared Memory ...... 5-11 Self-Created Segments ...... 5-11 UniData and the UNIX Kernel ...... 5-12

Chapter 6 UniData and UNIX ipc Facilities Message Queues ...... 6-3 UniData and Message Queues...... 6-3 Semaphores ...... 6-5

Chapter 7 UniData and UNIX Devices UNIX Devices: Overview ...... 7-3 UniData and Terminal Devices ...... 7-4 UniData and Tape Devices ...... 7-5 UniData and Printers ...... 7-6 UniData and Serial Devices ...... 7-7

Chapter 8 Configuring Your UniData System Configuration Procedure ...... 8-3

Chapter 9 Starting, Stopping, and Pausing UniData Starting, Stopping, and Pausing UniData ...... 9-2 Normal Operation ...... 9-3 UniData Log Files ...... 9-3 Start UniData with startud ...... 9-4

Table of Contents v Stop UniData with stopud ...... 9-5 Pausing UniData ...... 9-6 The dbpause Command ...... 9-6 The dbpause_status Command ...... 9-7 Resuming Processing ...... 9-8 Additional Commands ...... 9-9 Listing Processes with showud ...... 9-10 Stopping a User Process with stopudt ...... 9-10 Stopping a User Process with deleteuser ...... 9-11 Displaying ipc Facilities with ipcstat ...... 9-12 Removing ipc Structures with udipcrm ...... 9-13 Stopping UniData with stopud -f ...... 9-13

Chapter 10 Managing UniData Accounts What Is a UniData Account? ...... 10-3 Creating a UniData Account ...... 10-4 Saving and Restoring Accounts ...... 10-8 Deleting an Account ...... 10-9 Clearing Space in UniData Accounts...... 10-10 CLEAR.ACCOUNT ...... 10-10

Chapter 11 Managing UniData Security Logins and Groups ...... 11-3 Adding a UNIX User ...... 11-3 Use Separate Logon IDs ...... 11-4 User Groups ...... 11-4 Home Directories ...... 11-5 Startup Scripts ...... 11-5 Customizing Permissions ...... 11-7 Customizing a VOC File ...... 11-11 Customizing UniData ...... 11-12 Remote Items ...... 11-14 The SETFILE Command ...... 11-16 LOGIN and LOGOUT Paragraphs ...... 11-17 UniData SQL Privileges...... 11-20 Field-Level Security for UniQuery ...... 11-21 Points to Remember about Field-Level Security ...... 11-21 The QUERY.PRIVILEGE File ...... 11-22 Turning on Field-Level Security ...... 11-24

vi Administering UniData on UNIX Chapter 12 Managing UniData Files UniData Hashed Files ...... 12-4 Static Hashed Files ...... 12-5 Dynamic Hashed Files...... 12-6 Dynamic Files and Overflow ...... 12-6 Dynamic Files, Part Files, and Part Tables ...... 12-9 When Dynamic Files Are Created ...... 12-12 Tips and Constraints for Creating a Dynamic File ...... 12-14 When Dynamic Files Expand ...... 12-15 Management Tools for Dynamic Files ...... 12-18 Dynamic Files and Disk Space ...... 12-20 Sequentially Hashed Files ...... 12-24 File-Handling Commands ...... 12-28 File Corruption ...... 12-31 What Causes File Corruption? ...... 12-31 Preventing File Corruption...... 12-31 UniData Detection Tools ...... 12-33 guide ...... 12-33 guide_ndx ...... 12-38 UniData Recovery Tools ...... 12-40 dumpgroup...... 12-40 fixgroup ...... 12-41 fixfile ...... 12-42 Detection and Repair Examples ...... 12-47 How to Use guide ...... 12-48 Error Messages ...... 12-51 File Access Messages ...... 12-51 Block Usage Messages ...... 12-51 Group Header Messages ...... 12-52 Header Key Messages ...... 12-52 Other Header Messages...... 12-52 Free Block Messages ...... 12-54 Long Record Messages ...... 12-54

Chapter 13 Managing UniData Locks The Global Lock Manager ...... 13-3 How GLM Works ...... 13-3 Locking in UniBasic ...... 13-5 How Locks Work...... 13-5 Locking Commands ...... 13-6

Table of Contents vii Resource Locks ...... 13-8 Listing Locks ...... 13-9 LIST.READU ...... 13-9 Parameters ...... 13-9 LIST.LOCKS ...... 13-11 LIST.QUEUE ...... 13-12 LIST.QUEUE Display...... 13-13 Commands for Clearing Locks ...... 13-17 SUPERCLEAR.LOCKS Command ...... 13-17 SUPERRELEASE Command ...... 13-19 Procedure for Clearing Locks ...... 13-20

Chapter 14 Managing UniData Users Adding Users ...... 14-3 Every User Needs a Logon ID ...... 14-3 Create Logon IDs at the UNIX Level ...... 14-3 Assign Users to Groups ...... 14-4 Monitoring User Processes ...... 14-5 UniData Commands ...... 14-5 Removing User Processes ...... 14-7 Using TIMEOUT ...... 14-8

Chapter 15 Managing Printers in UniData UniData and UNIX Spoolers ...... 15-3 Configuring the Spooler ...... 15-3 SETOSPRINTER Command ...... 15-6 Spooling from UniData ...... 15-7 UniData Printing Commands ...... 15-8 Configuring and Troubleshooting a Printer ...... 15-10 Physical Connection ...... 15-10 Spooler Definition ...... 15-10 Definition in UniData ...... 15-11 SETPTR ...... 15-12 Environment Variables ...... 15-15 Disabling Printer Validation ...... 15-15 Defining an Alternate Search Path ...... 15-15 Examples ...... 15-16 Redefining the Default UniData Print Unit ...... 15-16 Submitting Concurrent Print Jobs ...... 15-16 Printing to a UNIX Device ...... 15-17 viii Administering UniData on UNIX Passing Spooler Options to UNIX ...... 15-18 Decoding a UniData Print Statement ...... 15-19

Chapter 16 Accessing UNIX Devices UniData Tape Handling Commands ...... 16-3 SETTAPE ...... 16-5 Steps for Tape Device Use ...... 16-6 UniData LINE Commands ...... 16-9 Communicating with GET and SEND ...... 16-10 Dual-Terminal Debugging in UniBasic ...... 16-12 Setting Up Dual-Terminal Debugging ...... 16-13

Chapter 17 Managing Memory UniData Monitoring/Configuring Tools ...... 17-3 udtconf Main Display ...... 17-3 Calculating udtconfig Parameters ...... 17-4 Checking Configuration Parameters ...... 17-5 Saving Configuration Parameters...... 17-5 Recalculating the Size of the CTL ...... 17-6 Viewing Current and Suggested Settings ...... 17-6 Exiting udtconf ...... 17-7 Setting Shared Memory Parameters ...... 17-8 shmconf: Main Display ...... 17-8 shmconf: Viewing Current and Suggested Settings ...... 17-9 shmconf: Recalculating the Size of CTL ...... 17-10 shmconf: Recalculating Other Parameters ...... 17-10 Shared Memory and the Recoverable File System ...... 17-11 Analyzing UniData Configuration Parameters...... 17-11 Checking and Changing UniData Configuration Parameters . . . . 17-13 Checking Kernel Parameters ...... 17-15 sms ...... 17-15 Learning about Global Pages ...... 17-18 Learning about Local Control Tables ...... 17-19 UNIX Monitoring Tools ...... 17-21 UniData Configuration Parameters ...... 17-22 UNIX Kernel Parameters ...... 17-22 UniData Error Messages for smm ...... 17-23

Chapter 18 Managing ipc Facilities Message Queues, Shared Memory, and Semaphores ...... 18-3

Table of Contents ix UniData Log Files ...... 18-6 Removing ipc Structures ...... 18-7

Chapter 19 Managing Cataloged Programs UniBasic Source and Compiled Programs ...... 19-3 UniBasic Compiled Programs ...... 19-3 Cataloging UniBasic Programs ...... 19-4 Direct Cataloging ...... 19-4 Local Cataloging ...... 19-4 Global Cataloging ...... 19-5 Managing Global Catalogs ...... 19-7 Contents of a Global Catalog ...... 19-7 Verifying a Program Version...... 19-8 Listing Programs in Use...... 19-14 Creating an Alternate Global Catalog Space ...... 19-15 Files and Directories Created by newhome ...... 19-15 Procedure for Creating an Alternate Global Catalog Space . . . . 19-15

Chapter 20 CallBasic, CALLC, and makeudt Linking C Routines into UniData ...... 20-3 Overview...... 20-3 Requirements ...... 20-3 Rebuilding for Troubleshooting...... 20-4 makeudt...... 20-5 File Examples ...... 20-5 Creating cfuncdef_user ...... 20-9 Steps for Linking in C Functions ...... 20-10 Steps for Setting Up a Development Environment ...... 20-15 Accessing UniData from a C Program ...... 20-19 Requirements ...... 20-19 How CallBasic Works ...... 20-19 C Program Example ...... 20-21 UniBasic Subroutine Example ...... 20-24 Steps for CallBasic ...... 20-25

Chapter 21 General Troubleshooting Crashes and Restart Problems ...... 21-3 UniData Crashes ...... 21-3 UniData Cannot Start ...... 21-4 Response Problems in UniData ...... 21-5 x Administering UniData on UNIX UniData Consistently Slow ...... 21-5 UniData is Hung ...... 21-5 Error Messages ...... 21-6 Common Error Messages ...... 21-6

Chapter 22 Performance Monitoring and Tuning Performance Monitoring and Tuning ...... 22-2 UNIX Performance Considerations...... 22-3 UNIX Performance Monitoring ...... 22-4 Tools ...... 22-4 Tips ...... 22-4 UniData Performance Factors ...... 22-6 Database Design Considerations ...... 22-6 Using Alternate Key Indexes ...... 22-6 Sizing Static Hashed Files ...... 22-6 Sizing Dynamic Hashed Files ...... 22-7 UniBasic Coding Tips ...... 22-7 UniBasic Profiling ...... 22-10 1. Compile the Programs with -G...... 22-10 2. Execute the Programs with -G ...... 22-10 3. Review the Profile Output ...... 22-10 UniData Performance Monitoring: udtmon ...... 22-14 udtmon: UniData User Statistics ...... 22-17 udtmon: File I/O Statistics ...... 22-18 udtmon: Program Control Statistics ...... 22-20 udtmon: Dynamic Array Statistics ...... 22-22 udtmon: Lock Statistics ...... 22-24 udtmon: Data Conversion Statistics ...... 22-27 udtmon: Index Statistics ...... 22-28 udtmon: Mglm Performance ...... 22-30

Appendix A UniData Configuration Parameters

Appendix B Environment Variables for UniData

Appendix C Configuring SSL for Telnet Server Side Configuration: ...... C-1

Table of Contents xi Chapter Introduction 1

Audience ...... 1-3 Introduction The purpose of this manual is to collect, in a single book, as much information as possible about activities needed to administer a UniData installation on UNIX. This manual repeats some information presented elsewhere in the UniData documentation set. Certain command descriptions and examples have been amplified or modified in this manual to increase their usefulness to system administrators as opposed to end users.

Note: Before you try repeating the examples in this manual, make sure that you have set the environment variables UDTHOME and UDTBIN, and make sure that your PATH includes udtbin. See Chapter 8, “Configuring Your UniData System,” for information about setting these variables.

Many of the administrative functions you can execute from the command line are available through UniAdmin. For more information, see Using UniAdmin.

1-2

Audience

Administering UniData on UNIX is intended for people whose responsibilities include the following:

„ Tasks performed at the host level „ Reviewing and modifying kernel configuration „ Modifying file protections „ Adding UNIX users „ Creating directories „ Starting and stopping UniData „ Backing up UniData files „ Tasks performed within UniData „ Creating and managing UniData accounts „ Optimizing UniData configuration „ Customizing security „ Managing files „ Monitoring and accessing files, peripherals, and system resources

1-3 Administering UniData on UNIX Chapter UniData and UNIX Security 2

UNIX File Permissions ...... 2-3 Additional UNIX Access Modes ...... 2-5 UNIX umask ...... 2-7 UniData Default Permissions ...... 2-9 UniData Processes and root ...... 2-10

This chapter describes UNIX security mechanisms and outlines how these are used by UniData. It is important to understand UNIX security, because UNIX file permis- sions form the basis of UniData security.

2-2 Administering UniData on UNIX UNIX File Permissions

In UNIX, each file or directory has permissions set for the owner (called user), for members of the group owner (called group), and for all other users except root (called other). The root user has all access to all files on the system, regardless of their owners or group owners.

UniData uses UNIX permissions as a principal security mechanism. The following table shows what the UNIX permissions mean when applied to a file.

Permission Description

r (read) Read or copy a file

w (write) Modify a file

x (execute) Execute a script or program UNIX Permissions for Files

Note: Scripts or compiled programs are called executables throughout this manual.

The meaning of the permissions is somewhat different when they are applied to a directory, as shown in the following table.

Permission Description

r (read) List the directory’s contents

w (write) Add or remove files from the directory

x (search) cd to the directory, or include it in a path UNIX Permissions for Directories

2-3

The following screen shows a long listing for the contents of a UNIX directory:

% ls -l drwxrwxrwx 2 ump01 other 24 May 21 13:14 ACCOUNT -rw-rw-rw- 1 root sys 0 Apr 30 10:51 FileInfo drwxrwxrwx 12 root unisrc 4096 Apr 30 16:06 bin drwxrwxrwx 12 root unisrc 2048 May 17 18:40 demo drwxr-xr-x 2 root sys 1024 Apr 30 16:05 include drwxr-xr-x 2 root unisrc 1024 Apr 11 12:23 lib drwxrwxrwx 3 root sys 1024 Apr 17 11:55 logs drwxrwxrwx 4 root unisrc 1024 Apr 10 18:28 objcall drwxrwxrwx 5 us_admin users 1024 May 1 12:50 ods -rw-r--r-- 1 root sys 7 Apr 11 12:22 parttbl drwxrwxrwx 4 root unisrc 1024 Apr 10 19:12 sybase drwxrwxrwx 4 root unisrc 1024 May 20 19:31 sys drwxr-xr-x 2 root unisrc 1024 Apr 30 15:58 work Entries beginning with “d” are directories; entries beginning with “-” are files. Permissions for owner, group, and all others are shown in the next nine characters. For example, the directory ACCOUNT is owned by “ump01,” and the group owner is “other.” The owner, “ump01,” members of the group “other” and all other users all have read, write, and search permission on ACCOUNT.

To delete a file from a directory, a user needs write permission to the directory, but does not need any permissions to the file. On most UNIX versions, if the user does not have write permissions to the file, the system displays the octal format for the current permissions and asks for confirmation to override them, as shown on the following screen:

% rm testfile testfile: 644 mode ? (y/n) On some systems, you can set an additional access mode called the “sticky bit,” which prevents users from deleting a file unless they have write permission on that file. See “Additional UNIX Access Modes” on page 2-5 for more information.

Note: The UNIX commands ls, chmod, chown, and chgrp are used for viewing and modifying file ownership and permissions. Refer to your host operating system documentation for detailed information about these commands.

2-4 Administering UniData on UNIX Additional UNIX Access Modes

UNIX supports three additional file access modes, as shown in the following table.

Access Mode Description

t (sticky bit) Varies with UNIX version; when set on a directory, restricts delete access on files within the directory.

s (set UID or SUID) Set on executable files; allows a user to invoke the executable, which runs with the privileges of the owner of the file.

s (set GID or SGID) Set on executable files; allows a user to invoke the executable, which runs with the privileges of the owner of the group. Additional UNIX Access Modes

Warning: Setting SUID or SGID on executables allows users access they would not be granted based on the permissions. These access modes, if not used with caution, can compromise the security of your system. Also, these access modes behave somewhat differently in different UNIX versions. Review your host operating system documentation before setting SUID or SGID.

The following screens show how to set these access modes, and what permissions look like when each of them is set.

The first screen shows the sticky bit:

% ls -l total 2 drwxrwxrwx 2 ump01 unisrc 24 May 21 13:48 testdir % % chmod a+t testdir % ls -l total 2 drwxrwxrwt 2 ump01 unisrc 24 May 21 13:48 testdir % The next screen shows how to set SGID on a file called testfile:

% ls -l -rwxrwxrwx 1 ump01 unisrc 0 May 21 15:58 testfile % % chmod g+s testfile % ls -l -rwxrwsrwx 1 ump01 unisrc 0 May 21 15:58 testfile %

2-5

The group owner must have x (execute) permission on the file, and you must be logged in as the file owner or as root to set SGID.

The next screen shows how to set SUID on a file called newfile:

% ls -l -rwxrwxr-x 1 ump01 unisrc 0 May 21 16:03 newfile % % chmod u+s newfile % ls -l -rwsrwxr-x 1 ump01 unisrc 0 May 21 16:03 newfile % The owner of the file must have x (execute) permission on the file, and you must be logged in as the owner or as root to set SUID.

2-6 Administering UniData on UNIX UNIX umask

The UNIX umask command sets default permissions for file creation. umask allows you to specify permissions that apply when a file is created. To use umask, you need to know the octal values of the basic permissions (read, write, execute), and the UNIX default permissions for files and directories. The following table shows the octal values for the permissions and for common combinations.

Octal Permission Value read 1 write 2 execute (or search) 4 read+execute 5 read+write 6 write+execute 3 read+write+execute 7 UNIX Permissions (Octal Values)

The UNIX default permissions for file creation are shown in the next table.

Type UNIX Default Permission

File rw-rw-rw- (octal: 666)

Directory drwxrwxrwx (octal: 777) UNIX Default Permissions for File Creation

2-7

The value of umask is also expressed in octal format, and its effect is shown by subtracting the value from the UNIX default. For example, if you set umask to 002 and create a file, the permissions on that file are represented by the difference between the default (666) and umask (002), or 664. Permissions of 664 translate to rw-rw-r--. The following screen shows three umask settings and their effects:

% umask 022 % touch umask.tst1 % ls -l umask.tst1 -rw-r--r-- 1 ump01 unisrc 0 May 21 17:43 umask.tst1 % umask 222 % touch umask.tst2 % ls -l umask.tst2 -r--r--r-- 1 ump01 unisrc 0 May 21 17:43 umask.tst2 % umask 007 % mkdir umask.tst3 % ls -l drwxrwx--- 2 ump01 unisrc 24 May 21 17:43 umask.tst3 Notice that, in the example, the UNIX touch command creates empty files.

Note: When a user invokes the UniData ECL CREATE.FILE command, UniData sets file permissions in most cases according to the user’s current umask. The exceptions are the directories for dynamic files and multilevel files and directories. Permissions for these are set to 775 octal (rwxrwxr-x). Note: For security purposes, a system administrator can set a default value of umask in any user’s .login or .profile file. However, if users have access to the UNIX prompt, they can override the default before entering UniData.

Refer to your host operating system documentation for detailed information about the umask command.

2-8 Administering UniData on UNIX UniData Default Permissions

When you install UniData software on your system, the installation process sets the ownership of the files being installed to root. The installation process then prompts you to enter a group, which must be a valid group defined on your system. UniData then sets default permissions for all the files it installs. For each file, the owner permissions apply to root. The group permissions apply to all members of the group you specify in the installation procedure. The final set of permissions applies to all other users on your system. The following screen shows a long listing for the file /usr/ud72/include/udtconfig, illustrating the default permissions set when you install UniData.

% cd /usr/ud72/include % ls -l udtconfig -rw-r--r-- 1 root sys 809 Apr 30 16:05 udtconfig % In this case, the file is owned by root, and the installation process sets the group to sys. Root has read and write access to the file, and all other users have read access only.

If you log on as root and create a new UniData account with the newacct command, the system allows you to specify the owner and group for the account. The system sets the owner and group owner accordingly.

You can customize the file permissions to meet specific needs for your system. See Chapter 11, “Managing UniData Security,” for information about customizing file protections.

UniData also allows you to fine-tune your system security by customizing the VOC files in your UniData accounts and by granting specific privileges to UniData SQL users via the UniData SQL GRANT command. See Chapter 11, “Managing UniData Security,” for information about tuning UniData security.

Note: The ECL SETFILE command lets you set pointers in the UniData VOC file to allow files to be shared among accounts or distributed among file systems. For each file, the permissions that control access are those at the location where the file resides, which may be different from those in the directory containing the VOC file.

2-9

UniData Processes and root

Since the principal UniData daemons, smm, sbcs, unirpcd, and cleanupd must run as root, UniData must be started by root. Those daemons have all access to all files on your system. (If you are using the Recoverable File System (RFS), the RFS daemons also run as root.) For security reasons, UniData users should not have root privileges. When a user enters UniData, the user process (called a udt) runs under the UID of the user. Since the udt process drives all file access, users can perform only actions allowed to them by your system’s security, which consists of UNIX file permissions, the local VOC file, and SQL privileges.

2-10 Administering UniData on UNIX Chapter UniData and the UNIX File System 3

UniData Directories and Files...... 3-3 Files, Pointers, and Links ...... 3-5 Creating Files ...... 3-5 Setting a UniData Pointer...... 3-5 Setting an Environment Variable ...... 3-7 Setting a UNIX Link ...... 3-8 UniData Hashed Files ...... 3-10 Static Files ...... 3-10 Dynamic Files ...... 3-11 Sequentially Hashed Files ...... 3-13 DIR-Type Files ...... 3-16 Multilevel Files ...... 3-16 Multilevel Directory Files ...... 3-18 Index Files and Index Log Files...... 3-19 UniData and tmp Space...... 3-21 Changing TMP in the udtconfig File ...... 3-22 Setting an Environment Variable ...... 3-22

This chapter describes relationships between UniData file types and UNIX file types, and outlines the structures of various types of UniData files.

3-2 Administering UniData on UNIX UniData Directories and Files

UniData uses UNIX files, directories, and links to organize its database. A UniData account is, in simplest terms, a UNIX directory that contains a VOC file, its dictionary (D_VOC), and other files created by the newacct command. The VOC file serves as the central repository for information about the account. It contains direct information (such as commands and keywords you can use) and pointers to menus, data, dictionary files, and cataloged programs. See Chapter 10, “Managing UniData Accounts,” for information about the newacct command.

Note: The data and dictionary files referenced in a VOC file are not necessarily located in the same UNIX directory as the VOC file. You can list the files that are defined for a UniData account by listing VOC entries. It is normal for the resulting file list to differ from a UNIX or UniData listing (ls) of the files actually located in the directory.

In UniData, as in UNIX, a directory is treated as a type of file. The following table shows the relationships between UniData file types and UNIX file types.

UniData File Type UNIX File Type

Static hashed file Data file.

Dynamic hashed file UNIX directory that contains data files and overflow files (or UNIX symbolic links to these) and indexes (if any are built). The data, overflow, and index files are UNIX data files.

Sequentially hashed file UNIX directory that contains data files and overflow files and indexes whose records are stored in sequential order based on the @ID. A sequentially hashed file has a fixed modulo, just like a static hashed file.

Dictionary (DICT) file Data file that contains attribute and record definitions for a (A static hashed file) UniData file.

Alternate key index (for Data file located in the same directory at the same level as the static file) file being indexed.

Alternate key index (for Data file located in the dynamic file directory along with the dynamic file) data file and the overflow file. UniData and UNIX Files

UniData Directories and Files 3-3

UniData File Type UNIX File Type

DIR file UNIX directory. You can copy records from another file into a DIR file with the ECL COPY command; each such record is stored as a UNIX text file.

MULTIFILE UNIX directory that contains one or more UniData hashed (multilevel file) files. There is one dictionary for the MULTIFILE, which is shared by all hashed files in the directory. You can copy records from one file into any other file within the directory.

MULTIDIR UNIX directory that contains one or more subdirectories. (multilevel DIR file) There is one dictionary for the MULTIDIR, which is shared by each subdirectory. If you copy records from another file into one of the subdirectories, each record is stored as a UNIX text or data file. UniData and UNIX Files (continued) You can define links and pointers within UniData to reference files located in different UNIX file systems. You can also define environment variables for the paths of UniData accounts, and then use those variables when setting pointers in VOC entries.

3-4 Administering UniData on UNIX Files, Pointers, and Links

Creating Files

By default, the physical location of a UniData file is the UniData account directory where the file was created. The following screen shows the ECL CREATE.FILE command (creating a static file) and the ECL LS command (displaying the account directory).

UniData Release 7.2 Build: (3082) (c) Copyright IBM Corporation 2005. All rights reserved.

Current UniData home is /disk1/ud72/. Current working directory is /disk1/ud72/demo.

:CREATE.FILE STATIC.TST 5 Create file D_STATIC.TST, modulo/1,blocksize/1024 Hash type = 0 Create file STATIC.TST, modulo/5,blocksize/1024 Hash type = 0 Added "@ID", the default record for UniData to DICT STATIC.TST. :LS BP D_SAVEDLISTS D__REPORT _SAVEDLISTS _REPORT_ CTLG D_STATIC.TST D__SCREEN STATIC.TST _SCREEN_ D_BP D_VOC D__V__VIEW VOC __V__VIEW D_CTLG D__HOLD_ D_savedlists_HOLD_ savedlists D_MENUFILE D__PH_ MENUFILE _PH_

Setting a UniData Pointer

You can set a pointer in a UniData VOC file to a data file in another UniData account. This feature allows users working in different UniData accounts to share data files. There are two points to remember about setting a VOC pointer:

„ A VOC pointer is internal to UniData. It is not the same thing as a UNIX link. Because of this, even backup utilities that follow symbolic links do not automatically follow VOC pointers. See “Setting a UNIX Link” on page 3- 8 for more information about UNIX links. „ Setting a VOC pointer does not alter the physical location of the data file. Although you can access the file from the directory where the pointer resides, the physical location of the file and its indexes remains unchanged.

Files, Pointers, and Links 3-5

The following screen shows the ECL SETFILE command (setting a VOC pointer):

:SETFILE /usr/ud72/demo/INVENTORY INVENTORY Establish the file pointer Tree name /usr/ud72/demo/INVENTORY Voc name INVENTORY Dictionary name /usr/ud72/demo/D_INVENTORY Ok to establish pointer(Y/N) = Y SETFILE completed. :LIST INVENTORY WITH FEATURES LIKE "Portable..." FEATURES LIST INVENTORY WITH FEATURES LIKE "Portable..." FEATURES 18:56:11 May 21 2004 1 INVENTORY. Features......

39300 Portable, Sports Model 12006 Portable Color Ink Jet 39400 Portable Model 39000 Portable Sports Model 38000 Portable, 5-disk 52070 Portable Color, 3 ppm 52060 Portable Ink Jet, 5 ppm 10008 Portable, B/W, 6 ppm 30000 Portable Clock Radio 10009 Portable Color, 6 ppm 39100 Portable, Basic Functions 11 records listed :LS BP D_SAVEDLISTS D__REPORT _SAVEDLISTS _REPORT_ CTLG D_STATIC.TST D__SCREEN STATIC.TST _SCREEN_ D_BP D_VOC D__V__VIEW VOC __V__VIEW D_CTLG D__HOLD_ D_savedlists _HOLD_ savedlists D_MENUFILE D__PH_ MENUFILE _PH_ Notice that you can set the pointer and then access the file. However, the physical location of the file remains in /usr/ud72/demo, and the INVENTORY file is not included in the LS display.

Note: See the UniData Commands Reference manual and Using UniData for more information about creating and listing UniData files.

3-6 Administering UniData on UNIX Setting an Environment Variable

You can replace the “path” portion of a file reference in a VOC entry with a UNIX environment variable. This makes it easy to move a file or files when necessary without having to change each associated VOC entry. The following screen shows how to set an environment variable for the UniData demo account:

% setenv DEMO /usr/ud72/demo % printenv DEMO /usr/ud72/demo % cd $DEMO % pwd % /usr/ud72/demo % Notice that you can use the environment variable to access the demo database.

Note: The preceding example shows the C shell syntax for setting the environment variable. If you are using the Bourne or Korn shell, use the following syntax: DEMO=/usr/ud72/demo; export DEMO The following screen shows a VOC entry that uses the environment variable to identify a file in the demo database:

:WHERE /users/testacct :!printenv DEMO /usr/ud72/demo :CT VOC INVENTORY VOC:

INVENTORY: F @DEMO/INVENTORY @DEMO/D_INVENTORY : When users access the INVENTORY file, UniData uses the environment variable to locate the file. If you move the entire demo database, you can simply redefine the environment variable to the new path. Users can continue to access the files.

Note: If you use an environment variable in a VOC entry, precede the environment variable with the @ character as shown in the previous example. The @ character directs UniData to handle the path reference with the environment variable. Warning: You can use environment variables only in VOC entries for files. You cannot use them in other types of entries that include file paths (for instance, catalog pointer items).

Files, Pointers, and Links 3-7

Setting a UNIX Link

You can create a pointer to a file in another account directory by setting a symbolic link at the UNIX level. When you set a symbolic link, UNIX creates an entry in your current working directory that points to the location you linked to. The following screen shows how to set a symbolic link to a UniData file in another account:

% pwd /users/ump01/testacct % ln -s /usr/ud72/demo/ORDERS ORDERS % ln -s /usr/ud72/demo/D_ORDERS D_ORDERS :udt

UniData Release 6.1 Build: (3082) (c) Copyright IBM Corporation 2005. All rights reserved.

Current UniData home is /disk1/ud72/. Current working directory is /users/ump01/testacct. :LS BP D_ORDERS D__REPORT_ ORDERS _REPORT_ CTLG D_SAVEDLISTS D__SCREEN_ SAVEDLISTS _SCREEN_ D_BP D_VOC D___V__VIEW VOC __V__VIEW D_CTLG D__HOLD_ D_savedlists _HOLD_ savedlists D_MENUFILE D__PH_ MENUFILE _PH_ : Notice that ORDERS and D_ORDERS appear in the LS output. UNIX creates an entry in the current working directory for the link, although the ORDERS file remains physically located in /usr/ud72/demo.

3-8 Administering UniData on UNIX To access ORDERS from the current account, you must add a VOC entry for the file. You can use SETFILE (by entering SETFILE ORDERS ORDERS at the colon prompt), or you can use AE, as shown in the following example:

:LIST ORDERS WITH ORD_DATE LIKE "...1996" Not a filename : ORDERS :AE VOC ORDERS Top of New "ORDERS" in "VOC". *--: I 001= F 002= ORDERS 003= D_ORDERS *--: FI Filed "ORDERS" in file "VOC". LIST ORDERS WITH ORD_DATE LIKE "...1996" ORD_DATE LIST ORDERS WITH ORD_DATE LIKE "...1996" ORD_DATE 11:37:30 May 22 2004 1 Order ORDERS.... Date......

912 01/13/1996 941 01/14/1996 830 01/24/1996 970 01/15/1996 834 01/24/1996 Notice that even though ORDERS appeared in the LS output, you need to add a VOC entry to define the file to your current UniData account.

Note: Refer to your host operating system documentation for additional information about UNIX symbolic links. Refer to Using UniData for additional information about the VOC file.

Files, Pointers, and Links 3-9

UniData Hashed Files

Static Files

Hashed files are binary data files. They cannot be read by text editors external to UniData. Each UniData hashed file consists of a file header and one or more groups of data. UniData supports two proprietary hashing algorithms, which determine which data groups contain each record. Hashing allows direct access to a record by translating its record key into its location in a data file. The following screen shows some characteristics of a UniData static hashed file:

:LS AE_COMS D_BP D_VOC D_savedlists _HOLD_ AE_SCRATCH D_CTLG D__HOLD_ MENUFILE _PH_ BP D_MENUFILE D__PH_ ORDERS _REPORT_ CTLG D_ORDERS D__REPORT_ SAVEDLISTS _SCREEN_ D_AE_COMS D_SAVEDLISTS D__SCREEN_ STATIC.TEST __V__VIEW D_AE_SCRATCH D_STATIC.TEST D___V__VIEW VOC savedlists :!ls -l STATIC.TEST -rw-rw-rw- 1 claireg unisrc 4096 May 22 11:41 STATIC.TEST :!file STATIC.TEST STATIC.TEST: data When you create a static hashed file, you specify the modulo (number of groups) and the block size of the file. Static hashed files overflow if one or more groups cannot store all the data (level 1 overflow) or keys (level 2 overflow) of records hashed to the group. UniData adds overflow blocks to the file, but subsequent accessing and updating of records is then resource-intensive and performance suffers. UniData provides utilities to resize static files by adding more groups (changing the modulo) or by making the groups larger (changing the block size).

Points to Remember About Static Files

Remember the following points about static files:

„ A UniData static file is a binary data file. „ You define the size of a static file when you create the file, by specifying the number and size of groups in the file.

3-10 Administering UniData on UNIX „ When you add records to the file, each record is hashed to a group using a proprietary hashing algorithm. „ Static files can overflow, causing performance problems. „ A static hashed file cannot be larger than 2 GB. If a file exceeds 2 GB, you must make it a dynamic file.

See Chapter 12, “Managing UniData Files,” for more information about file management commands.

Dynamic Files

A dynamic file is a UNIX directory file, containing index, data, and overflow files (and/or symbolic links to these). UniData dynamic files can grow and shrink with respect to modulo (number of groups) as records are added and deleted. Dynamic files can also expand beyond the limits of a single UNIX file system. The following screen shows some characteristics of a simple dynamic file:

:CREATE.FILE DYNAMIC.TEST 1 DYNAMIC 1 is too small, modulo changed to 3. Create file D_DYNAMIC.TEST, modulo/1,blocksize/1024 Hash type = 0 Create dynamic file DYNAMIC.TEST, modulo/3,blocksize/1024 Hash type = 0 Split/Merge type = KEYONLY Added "@ID", the default record for UniData to DICT DYNAMIC.TEST. :LS BP D_DYNAMIC.TEST D__PH_ MENUFILE _REPORT_ CTLG D_MENUFILE D__REPORT_ SAVEDLISTS _SCREEN_ DYNAMIC.TEST D_SAVEDLISTS D__SCREEN_ VOC __V__VIEW D_BP D_VOC D___V__VIEW _HOLD_ savedlists D_CTLG D__HOLD_ D_savedlists _PH_ vocupgrade :!ls -l DYNAMIC.TEST total 10 -rw-rw-rw- 1 terric unisrc 4096 Jun 25 17:13 dat001 -rw-rw-rw- 1 terric unisrc 1024 Jun 25 17:13 over001 Notice that the UniData dynamic file is a UNIX directory, containing UNIX files dat001 and over001. The dat001 file is a UniData hashed file, and the blocks in over001 are linked to groups in the dat001 file.

UniData Hashed Files 3-11

The dat001 File

The dat001 file is also called the primary data file. As you add records to a dynamic file, UniData hashes the keys to groups in dat001. As the file fills up, UniData adds additional groups to the dat001 file. If the current file system fills up or if dat001 grows larger than a UniData limit, UniData creates a dat002 file. If dat002 is in another file system, UniData creates a UNIX link to the dat002 file in the original dynamic file.

The over001 File

As you add records to a dynamic file, whenever the space reserved for data in a group in the primary file gets too full, UniData writes the excess data into blocks in over001. Registers within UniData track how blocks in over001 are linked to groups in dat001. If over001 gets too large, UniData adds additional blocks to it. If the current file system becomes full or over001 grows larger than a UniData limit, UniData creates an over002 file. If the over002 file is in a file system different from the current one, UniData creates a UNIX link to over002 in the original dynamic file.

If you specify the OVERFLOW keyword with the CREATE.FILE command, UniData creates a dynamic file with an overflow file for each dat file. For example, over001 corresponds to dat001, over002 corresponds to dat002, and so on. When the file is cleared, UniData maintains this overflow structure.

Points to Remember About Dynamic Files

Remember the following points about dynamic files:

„ A UniData dynamic file is a UNIX directory. The directory contains files or UNIX links. „ Dynamic files expand and shrink with respect to modulo. Expansion and shrinking take place automatically during UniData processing. „ Dynamic files can expand across UNIX file systems. The original dynamic file contains UNIX links to any “part files” that are created on other file systems. „ Because the parts of a dynamic file are related by symbolic links, you need a backup utility that follows symbolic links to guarantee complete backups of dynamic files. Note: Chapter 12, “Managing UniData Files,” includes detailed information about the behavior of UniData dynamic files.

3-12 Administering UniData on UNIX Sequentially Hashed Files

A sequentially hashed file has the same structure as a dynamic file, but UniData stores all records sequentially based on the primary key. The modulo (number of groups) for a sequentially hashed file is fixed, it does not grow and shrink as records are added or deleted.

You create sequentially hashed files by converting from existing UniData static or dynamic files. You specify a percentage of the file that you want to remain empty to allow for growth. Although the structure for a sequentially hashed file is the same as a dynamic file, the modulo is fixed.

Use sequentially hashed files for files where the majority of access is based on the primary key.

The dat001 File

The dat001 file is also called the primary data file. As you add records to a sequen- tially hashed file, UniData hashes the keys, based on information in the gmekey file, to groups in dat001. If your data overflows the group (level 1 overflow), UniData writes the overflow data to the over001 file.

The over001 File

As you add records to a sequentially hashed file, whenever the space reserved for data in a group in the primary file gets too full, UniData writes the excess data into blocks in over001. Registers within UniData track how blocks in over001 are linked to groups in dat001. If over001 gets too large, UniData adds additional blocks to it. If the current file system becomes full, or over001 grows larger than a UniData limit, UniData creates an over002 file. If the over002 file resides in a different file system than the over001 file, UniData creates a link to over002 in the original sequentially hashed file.

If the sequentially hashed file has level 2 overflow, the file should be rebuilt using the shfbuild command.

UniData Hashed Files 3-13

The gmekey File

Each sequentially hashed file contains a static, read-only file called the gmekey file. This file is read into memory when you open a sequentially hashed file. The gmekey file contains information about the type of keys in the file (alpha or numeric), and controls which group a record is hashed to when it is written.

You create a sequentially hashed file by converting an existing dynamic or static file with the shfbuild command:

Syntax:

shfbuild [-a |-k] [-n | -t] [-f] [-e empty percent] [-m modulo] [-b block size multiplier] [-i infile] outfile

The following table describes the shfbuild options.

Parameter Description

-a Only rebuild the last group of the sequentially hashed file. UniData splits the last group into groups according to the records in the group. If you use this option, the outfile should be the name of the sequentially hashed file. Do not specify infile.

-k Build the gmekey file only. If you use this option, outfile should be the name of the sequentially hashed file. Do not specify infile. UniData rebuilds the gmekey file according to the keys in each group of outfile.

-n/-t Force the outfile to be in numeric or alphabetic order. By default, the order of outfile is determined by the infile primary key type. If infile is a sequen- tially hashed file, UniData uses the same order in outfile. If infile is not a sequentially hashed file, the order of outfile is determined by the justifi- cation of the @ID of the infile dictionary record. If it is right justified, it is numeric. Otherwise, it is alphabetic. If you use the -a or the -k option, these options have no effect.

-f Force outfile to truncate before being built.

-m Specifies the new modulo of outfile.

-e Empty percent. This is a number between 0 and 99 which indicates how much space in the rebuilt groups to reserve. UniData calculates the new modulo of the file from empty_percent and the number of records in the rebuilt groups. If you do not specify -e or -m, UniData rebuilds the sequen- tially hashed file according to the default empty percent of 20. shfbuild Parameters

3-14 Administering UniData on UNIX Parameter Description

-b Specifies the block size of the sequentially hashed file in kilobytes.

-i infile Load the contents from infile instead of outfile. infile can be any type of UniData file.

outfile The name of the output file. shfbuild Parameters (continued) To convert an existing file, execute the shfbuild command from the system level prompt, as shown in the following example:

% shfbuild -m 59 SEQUENTIAL 175 keys found from SEQUENTIAL. 175 records appended to SEQUENTIAL; current modulo is 59. After converting a file to a sequentially hashed file, you must manually enter a file pointer in the VOC file in order to access the sequentially hashed file, as shown in the following example:

:AE VOC SEQUENTIAL Top of New "SEQUENTIAL" in "VOC". *--: I 001= F 002= SEQUENTIAL 003= D_SEQUENTIAL *--: FI Filed "SEQUENTIAL" in file "VOC".

UniData Hashed Files 3-15

DIR-Type Files

A UniData DIR-type file is a UNIX directory that contains UNIX text or data files. Each UNIX text or data file is a UniData record. The BP file, a UniData file that stores UniBasic source files and compiled programs, is a DIR-type file. The following screen shows the structure of a sample BP file:

:LIST BP LIST BP 12:08:40 May 22 2004 1 BP......

MAINPROG _MAINPROG SUBR _SUBR 4 records listed In the example, MAINPROG and SUBR are UniBasic source files. _MAINPROG and _SUBR are compiled programs.

Multilevel Files

A UniData multilevel (LF-type) file is a UNIX directory that contains one or more UniData hashed files. All of the UniData hashed files share a common dictionary. To access a record, you must specify both the directory and the hashed file where the record is located. The following screen shows an example of a multilevel file:

:CT VOC MULTI1 VOC:

MULTI1: LF MULTI1 D_MULTI1 :!ls -l MULTI1 total 24 -rw-rw-rw- 1 claireg unisrc 4096 May 22 12:31 MULTI1 -rw-rw-rw- 1 claireg unisrc 4096 May 22 12:31 MULTI2 -rw-rw-rw- 1 claireg unisrc 4096 May 22 12:31 MULTI3 :LIST MULTI1,MULTI2 WITH F1 = PA LIST MULTI1,MULTI2 WITH F1 = PA 12:46:08 May 22 2002 1

ECLTYPE CP listdict CT SP.OPEN LISTDICT 6 records listed

3-16 Administering UniData on UNIX Note: If the subfile of a multilevel file has the same name as the directory, you can use the directory name only to access the subfile. For instance, LIST MULTI1 is correct syntax if the directory MULTI1 contains subfile MULTI1.

Points to Remember about Multilevel Files

Remember the following points about multilevel files:

„ A UniData multilevel file is a UNIX directory that contains UniData hashed files. „ Each multilevel file can contain a mixture of static and dynamic hashed files. „ All of the hashed files in a multilevel file share the same dictionary. „ UniData supports multilevel files to simplify conversion for legacy applica- tions. However, accessing and maintaining multilevel files is less efficient than accessing and maintaining ordinary static or dynamic files. The leveled structure requires more system resources to read and update these files. For this reason, we recommend using ordinary static or dynamic hashed files rather than multilevel files whenever possible. You can share a single dictionary among UniData files by modifying the VOC entries for each file to reference the same dictionary.

UniData Hashed Files 3-17

Multilevel Directory Files

A UniData multilevel directory (LD) file is a UNIX directory. The UNIX directory contains one or more UNIX subdirectories (UniData DIR type files). All of the DIR files share the same dictionary. To access a record, you must specify both the multi- level directory file and the DIR file where the record resides. The following screen shows some characteristics of a multilevel directory file:

:LS AE_COMS D_CTLG D_VOC MULTI1 _REPORT_ AE_SCRATCH D_DYNAMIC.TEST D__HOLD_ MULTID _SCREEN_ BP D_MENUFILE D__PH_ ORDERS __V__VIEW CTLG D_MULTI1 D__REPORT_ SAVEDLISTS savedlists DYNAMIC.TEST D_MULTID D__SCREEN_ STATIC.TEST D_AE_COMS D_ORDERS D___V__VIEW VOC D_AE_SCRATCH D_SAVEDLISTS D_savedlists _HOLD_ D_BP D_STATIC.TEST MENUFILE _PH_ :!ls -l MULTID total 4 drwxrwxr-x 2 claireg unisrc 24 May 22 12:49 TEST1 drwxrwxr-x 2 claireg unisrc 24 May 22 12:49 TEST2 :LIST MULTID,TEST1 LIST MULTID,TEST1 12:51:57 May 22 2005 1 MULTID....

MAINPROG _MAINPROG SUBR _SUBR 4 records listed Note: If a subdirectory of a multilevel directory file has the same name as the main directory, you can use the main directory name to access the subdirectory. For instance, LIST MULTID is correct syntax if the directory MULTID contains the subdirectory MULTID.

Points to Remember about Multilevel Directory Files

Remember the following points about multilevel directory files:

„ A UniData multilevel directory file is a UNIX directory that contains UniData DIR files (UNIX subdirectories). „ All of the DIR files in a multilevel file share the same dictionary. „ Each record in a multilevel directory is a UNIX file.

3-18 Administering UniData on UNIX „ UniData supports multilevel directory files to simplify conversion of legacy applications. However, accessing and maintaining multilevel directory files is less efficient than ordinary DIR files. The leveled structure means that more system resources are needed to read and update these files. For this reason, IBM recommends using ordinary DIR files rather than multilevel directory files whenever possible. You can share a single dictionary among UniData DIR files by modifying the VOC entries for each file to reference the same dictionary.

Index Files and Index Log Files

UniData creates an index file whenever a user creates the first alternate key index on a UniData hashed file. Index information is stored in B+ tree format. UniData index files are UNIX data files.

Note: Regardless how many alternate key indexes users create for a data file, UniData creates a single index file.

The ECL CREATE.INDEX command creates the index file. The ECL BUILD.INDEX command populates the index. DELETE.INDEX (with the ALL option) removes the index file.

By default, each time a UniData data file is updated, its associated indexes are updated. You can turn off automatic indexing on one or more data files (using the ECL DISABLE.INDEX command) to speed performance during periods of heavy activity on your system. If you turn off automatic indexing for a file, UniData creates a log file and stores all updates to it. The ECL UPDATE.INDEX command allows you to apply updates from index logs to indexes in batch mode, and the ECL ENABLE.INDEX command turns automatic updating back on. Either CLEAR.FILE or DELETE.INDEX (with the ALL option) removes the index log file.

Note: See the UniData Commands Reference for additional information about index handling commands.

UniData Hashed Files 3-19

Index-Related Files for a Static Hashed File

For a static hashed file, UniData creates both the index file and the index log file in the account directory with the data file. The following screen shows a sample account where a static file named STATIC.TEST has been indexed:

:LS AE_COMS D_CTLG D_VOC MULTI1 _PH_ AE_SCRATCH D_DYNAMIC.TEST D__HOLD_ MULTID _REPORT_ BP D_MENUFILE D__PH_ ORDERS _SCREEN_ CTLG D_MULTI1 D__REPORT_ SAVEDLISTS __V__VIEW DYNAMIC.TEST D_MULTID D__SCREEN_ STATIC.TEST savedlists D_AE_COMS D_ORDERS D___V__VIEW VOC x_STATIC.TEST D_AE_SCRATCH D_SAVEDLISTS D_savedlists X_STATIC.TEST D_BP D_STATIC.TEST MENUFILE _HOLD_ X_STATIC.TEST is the index file for the data file STATIC.TEST, and x_STATIC.TEST is the index log file.

Index-Related Files for a Dynamic Hashed or Sequentially Hashed File

For a dynamic hashed or sequentially hashed file, UniData creates both the index file and the index log file in the dynamic file directory. The following screen shows a sample account where a dynamic file named DYNAMIC.TST has been indexed:

:LS AE_COMS D_CTLG D_VOC MULTI1 _REPORT_ AE_SCRATCH D_DYNAMIC.TEST D__HOLD_ MULTID _SCREEN_ BP D_MENUFILE D__PH_ ORDERS __V__VIEW CTLG D_MULTI1 D__REPORT_ SAVEDLISTS savedlists DYNAMIC.TEST D_MULTID D__SCREEN_ STATIC.TEST D_AE_COMS D_ORDERS D___V__VIEW VOC D_AE_SCRATCH D_SAVEDLISTS D_savedlists _HOLD_ D_BP D_STATIC.TEST MENUFILE _PH_ :LS DYNAMIC.TEST dat001 idx001 over001 xlog001 Notice that the index and index log files are located in the dynamic file directory rather than in the account. The file idx001 is the index file, and xlog001 is the index log file.

3-20 Administering UniData on UNIX UniData and tmp Space

UniData uses temporary disk storage for a variety of purposes including:

„ Storing work files for UniQuery SORT and for sorting with the ORDER BY option in UniData SQL „ Building print files „ Using DELETE.FILE to delete UniData files „ Storing log and output files for layered products „ Storing work files for commands such as LIST.READU, listuser, BUILD.INDEX, UPDATE.INDEX, SP.EDIT „ Storing work files for file repair tools „ Storing work files for the UniBasic compiler

By default, UniData uses the UNIX partition /tmp for temporary disk storage. You can define an alternate temporary disk storage location by setting an environment variable called TMP, or by changing the TMP parameter in the udtconfig file, located in /usr/ud72/include. If both are set, the environment variable overrides the configu- ration parameter.

Note: You can override the default location for many UniData work files. However, there are some that cannot be overridden. Among these are working files used by SP.EDIT (copies of hold files you are editing), working files used by UniData SQL for sorting with the ORDER BY clause, and working files for the UniBasic compiler. UniData creates these files in /tmp regardless of any other setting.

In most cases, UniData removes its temporary work files when they are no longer needed. There are certain files that UniData does not remove, including output files it generates from filetools. Because the default /tmp is routinely cleared on many systems, you should consider defining alternate temporary storage if you know you are going to be repairing files, for example. Otherwise, you risk losing crucial data if the workfiles are removed before you are finished.

UniData and tmp Space 3-21

Changing TMP in the udtconfig File

The following screen shows a sample udtconfig file with the TMP parameter changed:

# # Unidata Configuration Parameters # # Section 1 Neutral parameters # These parameters are required by all Unidata installations. # # 1.1 System dependent parameters, they should not be changed. LOCKFIFO=1 SYS_PV=3

# 1.2 Changable parameters NFILES=60 NUSERS=40 WRITE_TO_CONSOLE=0 TMP=/users/tmp/

. . . Notice that the path name for TMP ends with the “/” character. This is required.

Setting an Environment Variable

You can set the environment variable TMP in individual users’ .login or .profile files to define alternate temporary disk storage for those users. A user with access to a UNIX prompt can set the environment variable as well.

In the C shell, use the following commands to set and display the TMP environment variable:

setenv TMP directory-name/ printenv TMP

In the Bourne or Korn shell, use the following commands to set and display the TMP environment variable:

TMP=directory-name/;export TMP printenv TMP

3-22 Administering UniData on UNIX Chapter UniData and Daemons 4

What Is a Daemon? ...... 4-3 Principal UniData Daemons ...... 4-4 Shared Basic Code Server (sbcs) ...... 4-4 Shared Memory Manager (smm) ...... 4-5 Clean Up (cleanupd) ...... 4-6 UniRPC Service (unirpcd) ...... 4-6 sync Daemon ...... 4-7 Monitoring UniData Daemons ...... 4-8 showud Command ...... 4-8 Log Files ...... 4-8

This chapter explains what UNIX daemons are, and describes daemons specific to UniData.

4-2 Administering UniData on UNIX What Is a Daemon?

A daemon is a background process that performs a specific task or set of tasks. Daemons wait in the background until they receive a request for their specific function. A number of standard UNIX daemons run on every UNIX platform to control system processes, schedule commands, handle print requests, and perform other similar functions. Refer to your host operating system documentation for detailed information about the UNIX daemons that run on your system.

What Is a Daemon? 4-3

Principal UniData Daemons

Three UniData daemons control your UniData environment. All three of these UniData daemons run as root. When a user starts a UniData session, the user’s process, called a udt, communicates with the daemons. The udt runs with the permis- sions valid for the user, preventing inappropriate file access by the UniData daemons.

„ Lock tracking — smm records all UniBasic locks and semaphore locks, identifying which UniData user holds each. „ Process cleanup — At periodic intervals, the cleanupd daemon checks the cleanupd daemon to see if terminated process flags have been set. If cleanupd detects a terminated process flag, it deletes the associated process from internal tables, removes any requests from the queue, and removes any messages sent to the terminated process. If cleanupd receives a message from a process, it checks to see if the message was sent from a terminated process. If so, it throws away the message.

Shared Basic Code Server (sbcs)

The shared basic code server, sbcs, manages shared memory used by globally cataloged UniBasic programs. UniData starts sbcs when you execute startud, and stops it when you execute stopud. The functions of sbcs include:

„ Loading and tracking globally cataloged programs—sbcs loads globally cataloged programs into shared memory as needed, and keeps track of the programs loaded and the number of processes executing each one. When you execute a globally cataloged program, sbcs checks in shared memory, then takes the following actions: „ If the program is already loaded, sbcs increments the counter for the number of users executing it, and tells the udt process which segment to attach to execute the program. „ If the program has not been loaded yet, sbcs loads the program into shared memory and starts a counter for it. „ Periodically sbcs checks shared memory and removes loaded programs that are no longer in use.

4-4 Administering UniData on UNIX „ Controlling shared memory—The sbcs daemon can attach up to 20 shared memory segments. (On some platforms sbcs cannot attach 20 segments because the operating system imposes a lower limit. For instance, AIX allows a process to attach only 10 shared memory segments.) „ The maximum size of each segment for sbcs is determined by the UniData configuration parameter SBCS_SHM_SIZE. sbcs attaches segments as it needs to load globally cataloged programs, and releases memory back to UNIX when it no longer needs the memory. „ Process cleanup — At periodic intervals, the sbcs process checks the cleanupd daemon to see if terminated process flags have been set. If sbcs detects a terminated process flag, it removes all messages sent for the process. If the terminated process is the only process using a program in shared memory, the program is removed from shared memory. sbcs uses the process ID to determine if a message it receives is from a terminated process. If so, sbcs discards the message. Note: See Chapter 19, “Managing Cataloged Programs,” for additional information about sbcs.

Shared Memory Manager (smm)

The shared memory manager, smm, builds and manages structures and tables within shared memory. UniData starts smm when you execute startud, and stops it when you execute stopud.

UniData processes (udt processes) communicate with smm to request and return shared memory. The UniData processes request shared memory from smm for the following tasks:

„ License control—The smm process tracks the number of users for which a site is licensed, and prevents more than that number of users from logging in to UniData. smm also displays warning messages when a license is about to expire. „ User process tracking — When a user logs on to UniData, smm assigns an internal tracking number to the user’s process and records information about the process in tables within UniData. „ Buffering program variables. „ Storing query records and intermediate results. „ Storing select lists.

Principal UniData Daemons 4-5

„ Storing expression buffers. „ Managing a current modulo table for dynamic files. „ Process cleanup—At periodic intervals, the smm process checks the cleanupd daemon to see if terminated process flags have been set. If smm detects a terminated process flag, it checks all ipc IDs. If one of the ipc IDs is invalid, smm exits, bringing down UniData. smm also checks all process groups to see if a group leader terminated abnormally. If so, smm removes all self-created shared memory pieces and reclaims all global pages occupied by the terminated group. smm also corrects any inconsistencies the global control tables (GCT) may have. An inconsistency could exist if the process was updating a GCT when it terminated.

The startud command starts smm, which creates a control table (CTL) in shared memory. The CTL tracks all information about the shared memory segments that smm manages. The size of the CTL is related to the number of users on the system and to a series of configuration parameters. See Chapter 5, “UniData and Memory,” and Chapter 7, “Managing Memory,” for more information about smm.

Clean Up (cleanupd)

The clean up daemon, cleanupd, detects terminated user processes at check time intervals. If cleanupd detects a terminated process, internal flags are set. The smm and sbcs daemons periodically check to see if cleanupd has set internal flags. If these daemons detect flags, each daemon performs the necessary cleanup and resets its own flag to zero. The cleanupd daemon performs clean up that is not handled by smm or sbcs. When the smm and sbcs daemons have reset their flags to zero, the cleanupd daemon resets its flag to zero, makes the user process ID available, and frees the local control table.

UniRPC Service (unirpcd)

The UniRPC service is used by UniAdmin, UniObjects, UniObjects for Java, UniData ODBC, UniOLEDB, and UCI to communicate with UniData on the server.

4-6 Administering UniData on UNIX sync Daemon

If you notice significant performance degradation during a checkpoint when running the Recoverable File System (RFS), you can start sync daemons by setting the udtconfig parameters N_SYNC and SYNC_TIME. Sync daemons periodically flush updated pages from the system buffer to the log files, reducing the amount of time it takes to complete a checkpoint.

N_SYNC determines the number of sync daemons UniData starts. SYNC_TIME defines, in seconds, the amount of time the sync daemons wait before scanning the system buffer for updated pages.

Note: The Recoverable File System creates and uses a group of additional UniData daemons. If you are using the Recoverable File System, refer to Administering the Recoverable File System for information about those daemons.

Principal UniData Daemons 4-7

Monitoring UniData Daemons

UniData provides a command and several log files to monitor the status of the daemons.

showud Command

The system-level command showud displays UniData daemons that are currently running. The following screen shows output from showud:

# showud UID PID TIME COMMAND root 19503 0:00 /disk1/ud72/bin/aimglog 0 27543 root 19504 0:00 /disk1/ud72/bin/aimglog 1 27543 root 19505 0:00 /disk1/ud72/bin/bimglog 2 27543 root 19506 0:00 /disk1/ud72/bin/bimglog 3 27543 root 19494 0:06 /disk1/ud72/bin/cleanupd -m 10 -t 20 root 19500 1:14 /disk1/ud72/bin/cm 27543 root 19490 0:00 /disk1/ud72/bin/sbcs -r root 19499 0:01 /disk1/ud72/bin/sm 60 10705 root 19483 0:05 /disk1/ud72/bin/smm -t 60 root 19525 0:00 /disk1/unishared/unirpc/unirpcd #

Log Files

The sbcs, cleanupd, and smm daemons each record messages in a pair of logs in the udtbin directory. In addition, the udt process writes messages to a log file, called udt.errlog, if a UniData process encounters file corruption in a data file. The following table lists these log files.

Daemon/Proce ss Routine Messages Error Messages

smm $UDTBIN/smm.log $UDTBIN/smm.errlog

sbcs $UDTBIN/sbcs.log $UDTBIN/sbsc.errlog

cleanupd $UDTBIN/cleanupd.log $UDTBIN/cleanupd.errlog

udt N/A $UDTBIN/udt.errlog Log Files for UniData Daemons and Processes

4-8 Administering UniData on UNIX Note: All messages written to the .errlog for a daemon are also written to the daemon log file. For example,if an error is written to the smm.errlog, the message also appears in the smm.log.

See Chapter 9, “Starting, Stopping, and Pausing UniData,” for additional information and examples.

The udt.errlog file

If a UniData process encounters file corruption in a data file during processing, the process writes a message to the udt.errlog in udtbin. System administrators can monitor this log and take corrective action for the specified file.

The following example illustrates errors printed to the udt.errlog when a SELECT statement is executed against a corrupt file:

udtno=1,pid=937,uid=1172,cwd=/home/claireg,Sep 12 12:44:46 1:grpno error in U_blkread for file 'TEST', key '', number=3

udtno=1,pid=937,uid=1172,cwd=/home/claireg,Sep 12 12:44:46 1:blkread error in U_read_group for file 'TEST', key '', number=3

udtno=1,pid=937,uid=1172,cwd=/home/claireg,Sep 12 12:44:46 1:read_all_block_in_group error in U_gen_read_group for file ' ', key ' ', number=0

Monitoring UniData Daemons 4-9 Chapter UniData and Memory 5

UNIX and Shared Memory ...... 5-3 UniData and Shared Memory ...... 5-4 smm and Shared Memory ...... 5-4 sbcs and Shared Memory ...... 5-11 Self-Created Segments ...... 5-11 UniData and the UNIX Kernel ...... 5-12 This chapter describes how UniData interacts with the UNIX kernel to configure, attach, and release shared memory.

5-2

UNIX and Shared Memory

Shared memory is a region of memory that one or more processes may access. Shared memory resides on a UNIX system outside the address space of any process. It is partitioned into segments, depending on the configuration of the system. As a process requires memory, the process attaches a segment to its own address space. Processes use UNIX system calls to create, attach, and release shared memory segments.

UNIX kernels define certain limits, such as the maximum and minimum size of a shared memory segment and the maximum number of shared memory segments the system supports. The names of these kernel parameters vary from system to system. The following table lists kernel parameters related to shared memory on an HP-UX system.

UNIX Parameter Description

shmmax The size, in bytes, of the largest shared memory segment the system supports.

shmmni The maximum number of shared memory segments the system supports.

shmseg The maximum number of shared memory segments the system can assign to one process. UNIX Parameters for Shared Memory

Note: Kernel configurations vary among UNIX versions. In some UNIX versions (AIX for example), all resources are allocated dynamically, and the system administrator has limited ability to affect the configuration. Some UNIX versions also have fixed limits on some parameters (for instance, AIX, where shmseg is 10). Other UNIX versions allow the system administrator to change parameter values, but procedures vary from system to system. Refer to your host operating system documentation for detailed information about your UNIX kernel. Note: You can distinguish between UNIX kernel parameter names and UniData configuration parameter names in this manual. UNIX kernel parameter names are in lowercase (for instance, shmmax) and UniData parameters are in uppercase (for instance, SHM_MAX_SIZE).

5-3 Administering UniData on UNIX UniData and Shared Memory

UniData interacts with UNIX shared memory by using UNIX system calls, UniData daemons, and UniData configuration parameters (within the limits imposed by the host UNIX system) to build its own structures in shared memory.

UniData, like UNIX, defines shared memory segments that can be attached by UniData processes. The sbcs daemon creates shared memory structures for storing active globally cataloged UniBasic programs.

See Chapter 19, “Managing Cataloged Programs,” for additional information about sbcs.

The smm daemon creates shared memory structures for internal tables required by UniData processes. UniData processes request memory for:

„ Buffering UniBasic variables „ Storing intermediate results „ Storing a current modulo table for dynamic files Note: The Recoverable File System (RFS) makes use of a specially allocated region of memory called the system buffer. If you are using RFS, refer to Administering the Recoverable File System for information about the system buffer. smm and Shared Memory

The smm daemon creates shared memory segments as needed. The size and charac- teristics of segments smm creates are determined by UniData configuration parameters. Whenever UniData is started, UniData reads the udtconfig file, located in /usr/ud61/include, and stores these values in shared memory. smm subdivides each of its segments into global pages, and subdivides each global page into local pages. smm also creates and maintains internal tables that track the use of the structures it creates. These internal tables, stored in a shared memory structure called CTL, allow smm to protect shared memory pages against accidental overwriting, and optimize the efficiency of memory use by releasing unneeded shared memory pages back to UNIX.

UniData and Shared Memory 5-4

Control Table List (CTL)

When you start UniData, smm creates one shared memory segment for the UniData control table list (CTL). The CTL remains in memory as long as UniData is running, and is returned to the operating system when you execute the stopud command.

CTL is subdivided into three regions, as shown in the following example:

Control Table List (CTL)

Virtual Memory Pool shared memory pool CTL shared memory segment

Control Table List local control GI GCT GCT GCT GCT LCT LCT table

general information global LCT control table Process Information Counter Table Memory Information Control Information local page

5-5 Administering UniData on UNIX The following table describes the structures in the CTL.

CTL Structure Description

GI Segment header; also called general information table.

GCTs (global control Each GCT records the use of global pages in a shared memory tables) segment. UniData determines the number of GCTs in the CTL by the configuration parameter SHM_GNTBLS. SHM_GNTBLS must not exceed the kernel parameter shmmni.

LCTs (local control Each LCT records the shared memory activity of a UniData tables) process group. UniData determines the number of LCTs in the CTL by the configuration parameter NUSERS. Each LCT comprises four subtables: PI, CT, MI, and CI. A process group is related to a process group leader, which can be a udt or SQL session, or a user executing UniData system-level commands from a UNIX prompt.

PI (process information) Each PI table registers all processes within a process group. table

CT (counter table) Each CT records information about the behavior of the process group.

MI (memory Each MI table records all global pages or self-created shared information) table memory segments used by the process group.

CI (control information) Each CI table records all blocks allocated from shared memory table for temporary buffers. Structures Within UniData’s CTL smm creates the CTL by using a series of configuration parameters. The following table lists the parameters smm uses to compute the size of CTL.

Parameter Description

SHM_GNTBLS The number of GCTs in the CTL, which is also the maximum number of shared memory segments that UniData can attach at a time.

NUSERS The number of LCTs in the CTL, which is also the maximum number of UniData process groups that can run at the same time.

SHM_GNPAGES The number of global pages in a shared memory segment. Configuration Parameters for CTL Structures

UniData and Shared Memory 5-6

Parameter Description

SHM_LPINENTS The number of entries in the PI table of each LCT, which is also the number of processes that can be included in a UniData process group.

SHM_LCINENTS The number of entries available in the CI table of each LCT; this is the maximum number of local memory segments that a process group can use at one time. A local segment is made up of one or more contiguous local pages.

SHM_LMINENTS The number of entries available in the MI table of each LCT; this is the maximum number of global pages or self-created memory segments a process group can use at one time. Configuration Parameters for CTL Structures (continued) The size of the CTL is the sum of the size required for the GI table, the GCTs, and the LCTs. You can calculate these by following these steps:

1. The size of the GI table is fixed at 69 bytes. 2. Compute the space (in bytes) needed for GCTs: ((2+SHM_GNPAGES)*4)*SHM_GNTBLS 3. Compute the space (in bytes) needed for LCTs: ((96+(4*SHM_LPINENTS)+ (12*LMINENTS)+(8*SHM_LCINENTS))*NUSERS 4. Add the values from the first three steps.

You can also use the UniData command shmconf to calculate the size of CTL. See Chapter 17, “Managing Memory,” for more information.

Note: The size of the shared memory segment reserved for CTL does not need to match the size of the segments smm manages. All the segments smm manages are the same size (computed by multiplying the number of global pages per segment by the size of a global page by 512), but they are not necessarily the same size as CTL.

5-7 Administering UniData on UNIX Creating and Assigning Memory Structures

When a UniData process requests memory, smm assigns one or more global pages, as requested, and updates the GCT (or GCTs, if global pages are assigned from more than one shared memory segment). smm also updates the information in the requesting processes’ LCT. When the requesting process has finished using the assigned memory, the process sends a message to smm, which releases the global page or pages and updates the GCTs and LCT.

The following figure illustrates smm’s shared memory structures:

Memory virtual memory pool CTL shared memory pool shared memory segment

shared memory segment

global page

global page

local page

UniData and Shared Memory 5-8

UniData determines the size and number of local pages per global page, and the size and number of global pages per segment, by configuration parameters. The following table lists these parameters and some of the relationships between them.

Parameter Description

SHM_LPAGESZ The size (in 512-byte blocks) of a single local page of shared memory.

SHM_GPAGESZ The size (in 512-byte blocks) of a global page of shared memory. SHM_GPAGESZ must be an integral multiple of SHM_LPAGESZ. Divide SHM_GPAGESZ by SHM_LPAGESZ to obtain the number of local pages in a global page.

SHM_GNPAGES The number of global pages in a shared memory segment. Compute the size, in bytes, of a shared memory segment by multiplying the size of a single global page (512*SHM_GPAGESZ) by the number of global pages per segment (SHM_GNPAGES). This total cannot exceed the maximum segment size defined in your UNIX kernel. Configuration Parameters for Memory Structure Sizes smm reserves some memory for requests and releases unused memory to the UNIX operating system. The following table describes UniData configuration parameters that smm uses to determine how much memory to reserve and how much to release.

Parameter Description

SHM_FREEPCT Percentage of global pages in a shared memory segment that should be kept free. If the percentage of free pages in a segment drops below this value, smm creates a new segment to handle requests.

SHM_NFREES Number of free shared memory segments that should be kept for UniData. If the number of free segments is larger than this value, smm releases the additional segments back to the operating system. If the number drops below this value, smm creates another segment. This parameter is usually set to one. Configuration Parameters for Creating Shared Memory Segments

5-9 Administering UniData on UNIX Displaying Parameter Settings

Use the UniData system-level command “sms -h” to display the current settings for configuration parameters related to shared memory. The following screen shows the output for this command for a system with a 32-user license:

% sms -h Shmid of CTL: 6201 ------IDs ------smm_pid smm_trace PtoM_msgqid MtoP_msgqid ct_semid (values) 17758 0 1400 1401 792 (1,1,1)

------GENERAL INFO ------SHM_GNTBLS = 50 (max 50 global segments / system) SHM_GNPAGES = 32 (32 global pages / global segment) SHM_GPAGESZ = 256 (128K bytes / global page)

NUSERS = 50 (max 50 process groups / system) SHM_LPINENTS = 10 (max 10 processes / group) SHM_LMINENTS = 32 (max 32 global pages / group) SHM_LCINENTS = 100 (max 100 control entries / group) SHM_LPAGESZ = 8 (4K bytes / local page)

SHM_FREEPCT = 25 SHM_NFREES = 1

SHM_FIL_CNT = 2048 JRNL_BUFSZ = 53248 Note: Refer to Appendix A, “UniData Configuration Parameters,” for further information about these parameters.

UniData and Shared Memory 5-10

sbcs and Shared Memory

sbcs creates structures in shared memory as needed for storing active globally cataloged UniBasic programs. The limits for structures created by sbcs are different from those for smm. The following table describes two udtconfig parameters that control the size of sbcs segments.

Parameter Description

SBCS_SHM_SIZE Size, in bytes, of shared memory segments sbcs creates. sbcs uses the segments to store globally cataloged programs. sbcs can attach a maximum of 20 segments. (On some UNIX versions, the kernel parameter shmseg constrains sbcs to 10 segments.)

MAX_OBJ_SIZE Maximum size, in bytes, of object code files that sbcs can load into shared memory. sbcs loads object code files larger than this size into the address space of the user instead of shared memory. Configuration Parameters for sbcs

Self-Created Segments

A UniData process can attach a segment of shared memory larger than a standard global page. UniData requires that a UniBasic variable it reads into memory be contained in a single global page. If a variable is larger than the size of a global page, smm creates a special segment in shared memory. These “self-created” segments, also called “indirect” segments, are attached to the requesting udt process. Some circumstances resulting in self-created segments are:

„ Editing a large report with AE. AE is a UniBasic program, and UniData reads a report in as a single variable. „ Reading a large array as a single variable.

smm creates a segment large enough to hold the variable. smm determines the maximum size by the UNIX kernel parameter shmmax. The self-created segment is counted as a global page used by the UniData process that created the segment.

5-11 Administering UniData on UNIX Warning: Creating these segments of memory is not an efficient resource use, and may result in poor performance or in thrashing. Use the system-level lstt command or the ipcstat command to determine if your application is using self-created segments on a regular basis. If so, analyze the sizes of variables the application uses. Consider increasing the value of SHM_GPAGESZ (the size of a global page) to handle the variables. Also, consider modifying the application to read arrays by element rather than as a single variable.

UniData and the UNIX Kernel

When optimizing configuration parameters for shared memory, certain changes may impact parameters in the UNIX kernel. Before implementing configuration changes, you should explore the impact of these changes on your kernel parameters, and determine if the kernel parameters should be changed. The following table describes relationships between UNIX kernel parameters and UniData.

UNIX Kernel Relationship to UniData

Maximum size of shared Must be larger than CTL, and larger than a shared memory memory segment segment created by smm or sbcs. UniData may not start if this (shmmax) kernel parameter is too low. If you change the configuration parameters that control the size of memory structures, you may need to adjust this kernel parameter.

Maximum number of Must be greater than SHM_GNTBLS, the number of GCTs in shared memory the control table. This parameter should be set high enough to segments (shmmni) accommodate all the GCTs, plus one segment for CTL, and one or more (up to 20) for sbcs.

Maximum number of Must be greater than SHM_LMINENTS, the number of entries shared memory available in the MI table for each LCT, which is the number of segments per process global pages that can be attached to a process. (shmseg) UniData and the UNIX Kernel

Note: If you are using RFS, UniData allocates a portion of shared memory as a system buffer for RFS processing. When you start UniData with RFS, UniData reserves this buffer, and it is therefore not available to smm or sbcs. See Administering the Recoverable File System for information about the system buffer.

UniData and Shared Memory 5-12 Chapter UniData and UNIX ipc Facilities 6

Message Queues ...... 6-3 UniData and Message Queues ...... 6-3 Semaphores ...... 6-5 Interprocess communication (ipc) facilities consist of message queues, shared memory segments, and semaphores. Chapter 5, “UniData and Memory,” describes shared memory handling. This chapter describes how UniData uses message queues and semaphores.

Note: The UNIX ipcs and ipcrm commands, and the UniData system-level ipcstat and udipcrm commands, are useful for tracking and managing ipc resources. Refer to your host operating system documentation for information about ipcs and ipcrm, and see Chapter 18, “Managing ipc Facilities,” for information about ipcstat and udipcrm.

6-2

Message Queues

A message queue is a system resource that can accept input from a number of processes. Processes can then retrieve messages from the queue, with some degree of selectivity. UNIX kernels generally include parameters that define the number of message queues, their size, and the number of outstanding messages the system can support.

The following table shows UNIX kernel parameters related to message queues and messages.

Parameter Description

msgmni The number of message queues the system can support.

msgmax The maximum size of a message, in bytes, allowed on the system.

msgmnb The maximum length, in bytes, of a message queue. This is the sum of the length of all messages in the queue.

msgmap Maximum number of entries in an internal table that UNIX uses to manage shared memory segments for holding messages.

msgseg Number of shared memory segments that UNIX reserves at kernel startup time for holding messages. UNIX Parameters for Message Queues Note: UNIX parameter names differ among versions of UNIX. These parameter names were read from a HP-UX kernel.

UniData and Message Queues

The smm and sbcs daemons use message queues for interprocess communication. When you start UniData, UniData initializes log files for each daemon, and lists the queues assigned to each daemon in its log.

6-3 Administering UniData on UNIX The following table describes the queues required for the UniData daemons.

UniData Daemon Queues

smm Two queues: one request queue and one reply queue.

sbcs Three queues: one request queue, one reply queue, and one “new version” queue, used to replace a cataloged program with a new version. UniData Message Queue Requirements

Note: If you are using RFS, you need additional message queues to handle communications for the RFS daemons. See Administering the Recoverable File System for information about RFS requirements for message queues. Tip: If one or more of your UniData daemons will not start, check the error logs for each daemon. Your UNIX kernel may not be configured with a sufficient number of message queues. Often, kernels are configured for a minimal number of queues. Refer to your host operating system documentation for information about kernel configuration.

Message Queues 6-4

Semaphores

UNIX system semaphores are used to control access to resources (like segments of shared memory) that can handle a limited number of simultaneous users. When a process acquires a semaphore, that process has access to the system resource the semaphore controls. When the process is finished with the resource, the process releases the semaphore.

A semaphore undo structure is a resource used to recover a semaphore if the process that acquired it terminates abnormally.

The following table lists UNIX kernel parameters that are important for the use of system semaphores.

UNIX Parameter Description

semmni The maximum number of semaphore identifiers systemwide.

semmnu The maximum number of semaphore “undo” structures systemwide.

semmns The maximum number of semaphores available systemwide. UNIX Parameters for Semaphores Note: UNIX parameter names differ between versions of UNIX. These parameter names were read from a HP-UX kernel.

UniData uses one system semaphore for smm. IBM also recommends that semmnu, the number of undo structures system-wide, be set to three times the number of licensed UniData users.

Note: If you are using RFS, you may need additional system semaphores. RFS semaphore operations may be handled at the UNIX system level, or by C or assembler instructions, depending on what method is most efficient for your UNIX version. See Administering the Recoverable File System for further information.

6-5 Administering UniData on UNIX Chapter UniData and UNIX Devices 7

UNIX Devices: Overview ...... 7-3 UniData and Terminal Devices ...... 7-4 UniData and Tape Devices...... 7-5 UniData and Printers ...... 7-6 UniData and Serial Devices ...... 7-7

This chapter briefly outlines how the UNIX operating system communicates with devices such as terminals, disk drives, tape drives, and printers. The chapter also outlines how UniData interacts with UNIX devices and device handling.

7-2 Administering UniData on UNIX UNIX Devices: Overview

The UNIX operating system uses device drivers to communicate with disk drives, tape drives, terminals, printers, and other hardware. Device drivers are programs that know how to communicate with each type of hardware. The UNIX kernel accesses these programs whenever a process requests interaction with a device.

The UNIX communication interface is set up so that any device can be viewed as a file. Data can be read from and written to a device file. The UNIX kernel translates actions on the device file into requests from the appropriate device driver program.

On most UNIX systems, device files reside in the /dev directory. This directory contains special files for serial devices, terminals of various sorts, disk devices, tape devices, and so forth. The names for these special device files vary among UNIX versions.

Note: Refer to your host operating system documentation for additional information about device files and device drivers on your system.

7-3

UniData and Terminal Devices

For virtually all terminal input/output, UniData relies on your host UNIX operating system to perform terminal emulation. UniData does not maintain its own terminal definition files. Different versions of UNIX handle terminal definitions differently. Some store terminal definition information in a termcap file while others use a terminfo database. Refer to your host operating system documentation for infor- mation about terminal definitions for your system.

Because terminal emulation in UniData happens at the UNIX level, terminal emulation problems that occur within UniData must be resolved at the UNIX level.

Note: There are a handful of UniData utilities that require a specific terminal definition file. For these utilities, your UniData product includes a file called vvtermcap, which is a termcap-like file with extensions. This file is located in udtbin. To execute the utilities that require it (which include confprod, udtconf, and shmconf) you must define either the UDTBIN or the VVTERMCAP environment variable. See Appendix B, “Environment Variables for UniData,” for further information.

7-4 Administering UniData on UNIX UniData and Tape Devices

Names for tape device files vary considerably between UNIX versions. The name of a tape device file often identifies a tape drive and an access method (such as rewind/no rewind).

UniData includes a number of commands that allow you to read data from or write data to UNIX tape devices. To use these commands effectively, you need to under- stand the tape device naming conventions on your system, because you need to specify device names to define them in UniData. Refer to your host operating system documentation for this information.

UniData uses a variety of proprietary methods, as well as some standard UNIX commands, to read/write data to tape devices. Tape devices must be defined in a UniData file before you can access them via UniData commands. See Chapter 16, “Accessing UNIX Devices,” and the UniData Commands Reference for more information.

7-5

UniData and Printers

UniData uses the UNIX spooler on each system to perform all printing. In order to print from within UniData to a UNIX printer, that printer must be connected to your system and defined within your spooler software. UniData provides commands and options that interface with your spooler and enable you to direct output to a printer or a class of printers.

Many printer problems that appear within UniData must be resolved outside of UniData, since the UniData commands do not interact directly with a printer device. You need to understand the printer configuration and spooler software on your system to troubleshoot UniData printing problems.

Since different UNIX versions use different spooler commands, UniData enables you to define the translation between UniData printing options and UNIX spooler options by defining a spooler configuration file. You can determine your current spooler selection with the ECL LIMIT or SETPTR command. For more information on defining spooler commands, refer to Chapter 15, “Managing Printers in UniData.”

7-6 Administering UniData on UNIX UniData and Serial Devices

UniData includes a group of commands that allow you to read data from or write data to a UNIX serial device. Although you can use these commands to access a terminal or printer device, they are most commonly used for communicating between UniData and a device, such as a bar code reader or scale. If such a device has a physical connection to your system, you can define it within UniData and communicate with it through UniBasic commands.

To make effective use of these UniData capabilities, you need to understand your configuration and device naming conventions. Refer to your host operating system documentation for this information. See Chapter 16, “Accessing UNIX Devices,” and Developing UniBasic Applications for information about communicating with serial devices.

7-7 Chapter Configuring Your UniData System 8

Configuration Procedure ...... 8-3 This chapter outlines configuration considerations that may be appropriate when you first implement UniData or when you make major changes to your system. (Major changes include adding or reconfiguring hardware, installing a new software appli- cation, or upgrading or relicensing UniData.)

This chapter does not present detailed information, but outlines the considerations and refers you to sources of additional information.

8-2

Configuration Procedure

If you are installing or upgrading UniData, see Installing and Licensing UniData Products for estimates for initial disk and memory needs for your system. These estimates should be sufficient to allow you to install and start UniData with a minimal configuration.

1. Identify Disk Requirements

Disk space and disk configuration are key factors that determine system perfor- mance. The initial estimates in Installing and Licensing UniData Products should allow you to install and run UniData. However, IBM recommends that you evaluate your system, including your operating system, your hardware configuration, and your application, to develop accurate disk space requirements. IBM offers the following suggestions.

„ Disk Requirements—Use the largest expected size for your data files to estimate how much disk space you need. In certain applications, such as financial applications, the number of records varies in a predictable way over time. Your system must have enough disk space to handle the maximum number of records without difficulty. „ Disk I/O—For best results, configure your disk system so that access is balanced across all devices. IBM suggests the following: „ /tmp directory—Locate your /tmp directory on a different physical device from your data for improved performance. „ UniData accounts—If your application uses more than one UniData account, locate the account directories on separate devices to distribute load. „ Logical volume management—Consider implementing a logical volume management protocol that includes disk striping. Various products and utilities are available; consult with your hardware vendor for recommendations. Striping, which allows you to create logical volumes that span physical devices, can provide significant performance improvements by balancing the load on the system. Tip: If your application frequently accesses data files that are larger than 300MB in size, IBM strongly recommends striping.

8-3 Administering UniData on UNIX 2. Identify Memory Requirements

The initial estimates in Installing and Licensing UniData Products should allow you to install and run UniData with a minimal configuration. However, memory require- ments can be platform dependent as well as application dependent. Estimate the memory required for the following components of your application:

„ The control table list (CTL) „ The memory segments controlled by smm „ The memory segments for sbcs „ If you use the Recoverable File System (RFS), the system buffer.

Refer to Chapter 5, “UniData and Memory,” for information about estimating memory needs.

3. Verify Version Compatibilities

If you are considering major upgrades to your hardware or to your operating system version, consult your IBM account representative early in your planning process to determine if your current UniData version is supported by the hardware or software you are considering.

4. Check/Reset UNIX Kernel Parameters

Depending on your version of UNIX, you may or may not be able to modify kernel parameters directly. The following categories of parameters may need adjustment when you first implement UniData:

„ Memory — Review parameters that limit the number of shared memory segments systemwide, the maximum and minimum size of a segment, and the maximum number of segments per process. See Chapter 5, “UniData and Memory,” and Chapter 17, “Managing Memory,” for additional information. „ Files and Users — Review parameters that define the maximum number of open files and file locks supported systemwide, the maximum number of files per user process, and the maximum number of user processes the system can support at one time.

Configuration Procedure 8-4

„ ipc Facilities — Review parameters that define the number and size of message queues your system supports, the number of semaphore sets and semaphores your system supports, and the number of semaphore undo struc- tures supported. Refer to Chapter 6, “UniData and UNIX ipc Facilities,” and Chapter 18, “Managing ipc Facilities,” for additional information. Warning: The number of systemwide semaphores, normally semmns, should be a minimum of NUSERS + 10. The number of semaphore identifiers, normally semmni, must be at least NUSERS/NSEM_PSET + 1. If either of these kernel parameters are not adequate for the number of licensed users, UniData displays an error message similar to “Exit: smm: cannot allocate semaphore for udtno xx errno 28. Exit: SMM can’t setup Control Table List,” and fails to start. Note: If you discover that you need to change both UNIX kernel parameters and UniData configuration parameters, identify all your requirements and then plan to rebuild the UNIX kernel first. If you attempt to start UniData with new parameters, and the UNIX kernel parameters are insufficient, UniData may not start.

5. Check/Reset UniData Configuration Parameters

The UniData udtconfig file, located in /usr/ud72/include, contains a set of parameters that are given default values when you install UniData. When you start a UniData session, UniData sets UNIX environment variables for each value in the udtconfig file.

Note: The udtconfig file is always located in /usr/udnn/include, where nn is the release number of UniData. For example, the udtconfig file for Release 7.2 is located in /usr/ud72/include, the udtconfig file for Release 6.1 is located in /usr/ud61/include, and so forth. Do not move the directory to another location, or UniData will not run.

The udtconfig file enables you to define values for each parameter that applies to your UniData environment. Most udtconfig parameters can be adjusted for your environment, but some should not be changed. Refer to Appendix A, “UniData Configuration Parameters,” for detailed information about each udtconfig parameter.

You must be logged on as root to modify udtconfig parameters.

8-5 Administering UniData on UNIX The following screen displays a sample udtconfig file:

# Unidata Configuration Parameters # # Section 1 Neutral parameters # These parameters are required by all Unidata installations. # # 1.1 System dependent parameters, they should not be changed. LOCKFIFO=1 SYS_PV=3

# 1.2 Changable parameters NFILES=55 NUSERS=40 WRITE_TO_CONSOLE=0 TMP=/tmp/ NVLMARK= FCNTL_ON=0 TOGGLE_NAP_TIME=91 NULL_FLAG=0 N_FILESYS=200 N_GLM_GLOBAL_BUCKET=101 N_GLM_SELF_BUCKET=23 GLM_MEM_SEGSZ=4194304 BGINPUTTIMEOUT=0 LB_FLAG=1 USE_DF=0

# 1.3 I18N related parameter UDT_LANGGRP=255/192/129 ZERO_CHAR=131

# # Section 2 Non-RFS related parameters # # 2.1 Shared memory related parameters SBCS_SHM_SIZE=1048576 SHM_MAX_SIZE=67108864 SHM_ATT_ADD=0 SHM_LBA=4096 SHM_MIN_NATT=4 SHM_GNTBLS=40 SHM_GNPAGES=32 SHM_GPAGESZ=256 SHM_LPINENTS=10 SHM_LMINENTS=32 SHM_LCINENTS=100 SHM_LPAGESZ=8 SHM_FREEPCT=25 SHM_NFREES=1

# 2.2 Size limitation parameters AVG_TUPLE_LEN=4

Configuration Procedure 8-6

EXPBLKSIZE=16 MAX_OBJ_SIZE=307200 MIN_MEMORY_TEMP=64

# 2.3 Dynamic file related parameters GRP_FREE_BLK=5 SHM_FIL_CNT=2048 SPLIT_LOAD=60 MERGE_LOAD=40 KEYDATA_SPLIT_LOAD=95 KEYDATA_MERGE_LOAD=40 MAX_FLENGTH=1073741824 PART_TBL=/liz1/ud60/parttbl

# 2.4 NFA/Telnet service related parameter EFS_LCKTIME=0 TSTIMEOUT=60 NFA_CONVERT_CHAR=0

# 2.5 Journal related parameters JRNL_MAX_PROCS=1 JRNL_MAX_FILES=400

# 2.6 UniBasic file related parameters MAX_OPEN_FILE=500 MAX_OPEN_SEQF=150 MAX_OPEN_OSF=100 MAX_DSFILES=1000

#2.7 UniBasic related parameters MAX_CAPT_LEVEL=2 MAX_RETN_LEVEL=2 COMPACTOR_POLICY=1 VARMEM_PCT=50

# 2.8 Number of semaphores per semaphore set NSEM_PSET=8

# 2.9 Index related parameters SETINDEX_BUFFER_KEYS=0 SETINDEX_VALIDATE_KEY=0

# 2.10 UPL/MGLM parameter MGLM_BUCKET_SIZE=50 UPL_LOGGING=0

# 2.11 Printer _HOLD_ file related parameters MAX_NEXT_HOLD_DIGITS=4 CHECK_HOLD_EXIST=0

# # Section 3 RFS related parameters # These parameters are only used for RFS which is turned by # setting SB_FLAG to a positive value.

8-7 Administering UniData on UNIX # # 3.1 RFS flag SB_FLAG=1

# 3.2 File related parameters BPF_NFILES=80 N_PARTFILE=500

# 3.3 AFT related parameters N_AFT=200 N_AFT_SECTION=1 N_AFT_BUCKET=101 N_AFT_MLF_BUCKET=23 N_TMAFT_BUCKET=19

# 3.4 Archive related parameters ARCH_FLAG=0 N_ARCH=2 ARCHIVE_TO_TAPE=0 ARCH_WRITE_SZ=0

# 3.5 System buffer parameters N_BIG=233 N_PUT=8192

# 3.6 TM message queue related parameters N_PGQ=10 N_TMQ=10

# 3.7 After/before image related parameters N_AIMG=2 N_BIMG=2 AIMG_BUFSZ=102400 BIMG_BUFSZ=102400 AIMG_MIN_BLKS=10 BIMG_MIN_BLKS=10 AIMG_FLUSH_BLKS=2 BIMG_FLUSH_BLKS=2

# 3.8 Flushing interval related parameters CHKPNT_TIME=300 GRPCMT_TIME=5

# 3.9 Sync Daemon related parameters N_SYNC=0 SYNC_TIME=0

# # Section 6 Century Pivot Date # CENTURY_PIVOT=1930

# # Section 7 Repliation parameters

Configuration Procedure 8-8

# REP_FLAG=1 TCA_SIZE=128 MAX_LRF_FILESIZE=134217728 N_REP_OPEN_FILE=8 MAX_REP_DISTRIB=1 MAX_REP_SHMSZ=67108864 UDR_CONVERT_CHAR=1

# # Euro data handling symbols # CONVERT_EURO=0 SYSTEM_EURO=164 TERM_EURO=164 LOG_OVRFLO=/liz1/ud72/log/log_overflow_dir REP_LOG_PATH=/liz1/ud72/log/replog #

6. Define Peripherals within UniData

You need to define tape devices, printers, and line devices need to within UniData before they can be accessed from UniData. Before defining a device within UniData, make certain that it is properly installed and functioning in your UNIX environment. Refer to your host operating system documentation for information about setting up peripherals on your system.

Use the ECL SETTAPE, SETLINE, and SETPTR commands to define your periph- erals to UniData. See Chapter 7, “UniData and UNIX Devices,” Chapter 15, “Managing Printers in UniData,” and Chapter 16, “Accessing UNIX Devices,” for additional information.

7. Create UniData Accounts

When you implement UniData, you may need to create one or more UniData accounts for your application. A UniData account is a UNIX directory that contains a UniData VOC file and its dictionary. The VOC file identifies commands, paragraphs, and all data files in the UniData account. The data files may be in the same UNIX directory as the VOC file, or the VOC file may contain pointers to data files in other UNIX file systems. Your system may have a single UniData account or several, depending on your application.

8-9 Administering UniData on UNIX Note: A UNIX account (login, password) is not the same as a UniData account. Every UniData user should have a separate UNIX account (login and password), but many users can access the same UniData account.

Use the UNIX mkdir command and the UniData system-level newacct command to create UniData accounts. Refer to your host operating system documentation for information about the mkdir command, and see Chapter 10, “Managing UniData Accounts,” for information about the newacct command.

8. Add UNIX Users

Accessing UniData requires each user to have a login and password on your UNIX system. IBM recommends you create a separate UNIX login (UNIX account) for each UniData user. Refer to your host operating system documentation for detailed information on creating UNIX accounts. See Chapter 11, “Managing UniData Security,” for UniData-specific information.

9. Set Environment Variables

A user wishing to access UniData must have two environment variables set: UDTHOME and UDTBIN. The settings you assign for these variables depend on whether you performed a basic or an advanced UniData installation.

Note: See Installing and Licensing UniData Products for information about installation types. UDTHOME — This variable identifies the absolute path of the UniData “home” directory. The home directory contains subdirectories UniData needs for processing. UDTBIN — This variable identifies the path for the directory that contains UniData executables. By default, udtbin is a subdirectory under udthome.

Configuration Procedure 8-10

Setting UDTHOME and UDTBIN

Each user needs UDTHOME and UDTBIN set to access UniData. You can add commands to set these environment variables to each user’s .login or .profile, or a user can set them at the UNIX prompt. If you are using the C shell, use the following commands to set these variables:

setenv UDTHOME /directory-name setenv UDTBIN /directory-name

If you are using the Bourne or Korn shell, use these commands:

UDTHOME=/directory-name;export UDTHOME UDTBIN=/directory-name;export UDTBIN

Setting PATH

Each user’s PATH should include the directory corresponding to UDTBIN. If you are using the C shell, use the following command:

set path = ($path $UDTBIN)

Use the following command for Bourne or Korn shells:

PATH=$PATH:$UDTBIN;export PATH

Setting Additional Environment Variables

Appendix B, “Environment Variables for UniData,” lists an additional set of variables that are significant for UniData users. These can be set in a user’s login script or defined from a UNIX prompt before entering UniData.

Note: While you are in a UniData session, you cannot change environment variables for that session. Even if you execute setenv, for instance, from the ! prompt, the new setting affects only processes started from the ! prompt. The new settings do not affect the current UniData session.

8-11 Administering UniData on UNIX 10. Review UniData Security

Depending on your application, you may need to implement additional measures to ensure data security and separation of duties. Review your application and implement any or all of the following:

„ Default Permissions—Modify the default permissions for udthome and its contents that were set when you installed UniData. „ Users and groups—Assign UniData users to separate UNIX groups, and set permissions on your database so that each group of users has access to the data they need. „ VOC file—Customize your VOC file to limit access to powerful commands. „ Remote entries—Use remote pointers to files and commands to allow more fine-grained protection. „ SQL Privileges—For SQL access, use the UniData SQL GRANT and UniData SQL REVOKE commands to customize access to tables. „ Query Privileges—For UniQuery access, use the QUERY.PRIVILEGE file.

Refer to Chapter 11, “Managing UniData Security,” for additional information.

11. Convert Data Files

Depending upon the nature of your system change, you may need to perform some conversion of UniData hashed files. Consider the following:

„ Recoverable files—If you are implementing UniData’s Recoverable File System, you need to create recoverable files and/or convert existing hashed files to recoverable files. See Administering the Recoverable File System for detailed information. „ Schema—If you are implementing UniData ODBC or UniOLEDB, you may need to make UniData files accessible to UniData SQL, and you may also need to utilize the Schema API to incorporate ODBC compliance for field and attribute names. See Using VSG and the Schema API for detailed information.

Configuration Procedure 8-12

„ File characteristics—UniData also offers you the capability to convert files from static to dynamic, change file characteristics such as block size and modulo, change hashing algorithm for a static file, and change file format between high-byte and low-byte formats. Refer to Chapter 12, “Managing UniData Files,” and to the UniData Commands Reference for additional information.

12. Perform makeudt

UniData provides the capability to invoke C functions from within UniBasic programs. It is necessary to rebuild the UniData executable (the binary file udtbin/udt) whenever you wish to link in additional C functions.

13. Review Backup Procedures

Special considerations are needed to back up UniData accounts. Make certain your backup procedures have the following capabilities:

„ Subdirectories—Your backup procedure should be able to back up at least three levels of subdirectories to support UniData MULTIDIR and MULTIFILE files. „ Symbolic links—Your backup procedure should be able to follow UNIX symbolic links to support large dynamic files. „ Backing up selected files—Your backup procedure should allow you to input a list of files to back up to support full backups of UniData accounts. Simply backing up the directory that contains the VOC file may be insuffi- cient, since data files may not be located in the same UNIX directory as the VOC file. The ECL SETFILE command creates VOC entries with pointers to files in other locations. However, backup utilities do not follow these SETFILE pointers like they do UNIX symbolic links. To create a complete backup of an account, make sure you back up and verify each physical file associated with the account.

8-13 Administering UniData on UNIX Chapter Starting, Stopping, and Pausing UniData 9

Normal Operation ...... 9-3 UniData Log Files ...... 9-3 Start UniData with startud ...... 9-4 Stop UniData with stopud ...... 9-5 Pausing UniData ...... 9-6 The dbpause Command ...... 9-6 The dbpause_status Command ...... 9-7 Resuming Processing ...... 9-8 Additional Commands ...... 9-9 Listing Processes with showud ...... 9-10 Stopping a User Process with stopudt ...... 9-10 Stopping a User Process with deleteuser ...... 9-11 Displaying ipc Facilities with ipcstat ...... 9-12 Removing ipc Structures with udipcrm ...... 9-13 Stopping UniData with stopud -f ...... 9-13

Starting, Stopping, and Pausing UniData This chapter describes procedures for normal startup and shutdown of UniData, and also describes commands used to log out users, stop processes, and remove ipc facil- ities if needed. These commands are also documented in the UniData Commands Reference.

9-2 Administering UniData on UNIX Normal Operation

Use the UniData startud and stopud commands, respectively, for normal startup and shutdown. These commands start and stop the sbcs, cleanupd, and smm daemons, in the correct order.

Note: You must be logged on as root to execute startud or stopud. Make sure you have defined the environment variables UDTHOME and UDTBIN, and make sure your PATH includes udtbin. If you are running more than one version of UniData, make sure that these environment variables are set for the version of UniData you want to start and stop.

UniData Log Files startud makes entries in the log files (smm.log, sbcs.log, and cleanupd.log) that identify the system resources used by the daemons. If you are using the Recoverable File System (RFS), startud and stopud also start the sm daemon and record infor- mation in its sm.log.

The following example shows a sample sbcs.log:

% pg sbcs.log

The next example shows a sample smm.log:

# pg sbcs.log

SBCS version: 7.2

BEGSMALL = 1 BEGMED = 5 BEGLARGE = 20 BEGHUGE = 45 Begsmall = 0 Begmed = 163 Beglarge = 490 Beghuge = 981 Beginning of emergency area = 1638, size = 410 Recover = 1 Debugmode = 0

Shm Attach Address: 0 Shm Max. Size.....: 1048576 SBCS process id...: 2474

IPC facilities:

q - 205 (request queue) q - 206 (reply queue) q - 207 (new version queue) m - 408

Normal Operation 9-3

The next example shows a sample smm.log:

% pg smm.log SMM version: 7.2

Number of users...... : 32 SMM checking interval: 60 seconds SMM process id...... : 2469

IPC facilities:

q - 203 (request queue) q - 204 (reply queue) m - 1405 (ctl) m - 406 (glm) m - 407 (shmbuf) s - 140 (ctl) s - 141 (journal) s - 142 (SUPERRELEASE/SUPERCLEAR.LOCKS) s - 135 (latch) s - 136 (latch) s - 137 (latch) s - 138 (latch) s - 139 (latch) The next example shows a sample cleanupd.log:

% pg cleanupd.log

CLEANUPD daemon: CLEANUPD checking interval: 20 seconds CLEANUPD minimum interval: 10 seconds CLEANUPD process id.....: 880

Start UniData with startud

The following screen shows the normal output from the startud command:

# startud Using UDTBIN=/disk1/ud72/bin

All output and error logs have been saved to ./saved_logs directory.

SMM is started. SBCS is started. CLEANUPD is started. SM is started. Unirpcd is started

UniData R7.2 has been started. #

9-4 Administering UniData on UNIX Note: You can configure your system so that UniData starts up automatically when you boot your computer. You need to add or modify a startup script to accomplish this. Refer to your host operating system documentation for detailed information about startup scripts. Warning: If you are using RFS, IBM recommends that you not start UniData automatically at reboot. If your system is rebooting because of a machine failure, and one or more file systems has been damaged, UniData failure recovery will not complete successfully.

Stop UniData with stopud

For normal operation, use the stopud command with no options. You need to make sure users are logged out of UniData before you execute this command.

The following screen shows the expected response to the stopud command:

# stopud Using UDTBIN=/disk1/ud72/bin

SM stopped successfully. CLEANUPD stopped successfully. SBCS stopped successfully. SMM stopped successfully. Unirpcd has already been stopped

Unidata R7.2 has been shut down. #

Normal Operation 9-5

Pausing UniData

The dbpause Command

UniData includes a system-level command that enables you to temporarily block updates to the database. You can use this feature to perform some tasks that require UniData to be stopped, such as backing up your data.

Syntax:

dbpause

UniData starts a new archive file when you resume processing.

Note: You must log on as root to issue the dbpause command.

dbpause blocks most updates to the database that are made within UniData. UniData completes writes or transactions in process when you issue the dbpause command before dbpause takes effect. Updates are blocked until the system administrator executes the dbresume command.

UniData does not block UNIX commands, such as cp or mv. In addition, UniData does not block updates to the _HOLD_ file and the _PH_ file, and does not interrupt report printing. If you execute dbpause while running RFS, UniData forces a check- point, flushes the after image logs to the archive files (if archiving is enabled), and marks the next available logical sequence number in the archive file for use after the backup. UniData displays this information on the screen from which you execute dbpause, and writes it to udtbin/sm.log.

Note: Some UniData system-level commands, such as cntl_install, require that UniData not be running. These commands do not execute successfully with dbpause in effect. You cannot stop UniData with dbpause in effect.

The following ECL commands are not blocked when dbpause is in effect:

„ ACCT.SAVE, T.ATT, T.DUMP „ BLOCK.PRINT, BLOCK.TERM „ CHECKOVER, dumpgroup, fixfile, fixgroup, guide „ CLEAR.ACCOUNT, CLEARDATA, CLR „ COMO

9-6 Administering UniData on UNIX „ CONFIGURE.FILE, HASH.TEST „ LIST.TRIGGER „ DATE.FORMAT „ CLEAR.LOCKS, DELETE.LOCKED.ACTION, LOCK, SUPER- CLEAR.LOCKS, SUPERRELEASE „ BLIST, VCATALOG „ deleteuser, ipcstat, makeudt, stopudt, updatevoc „ ECLTYPE, UDT.OPTIONS „ FLOAT.PRECISION „ HELP „ LINE.ATT „ LIST.INDEX „ LOGTO (unless a login paragraph exists in the account you are logging to, and an action is defined in the login paragraph that is paused) „ MIN.MEMORY „ SET.DEC, SET.MONEY, SET.THOUS, SET.WIDEZERO „ SETOSPRINTER, SETPTR, SP.ASSIGN, SP.EDIT „ TERM „ WHAT

The following example illustrates the output from the dbpause command:

# dbpause DBpause successful. # If you are running RFS, it is important that you synchronize your archive files with your backup files when using the dbpause command. For more information about using dbpause with RFS, see Administering the Recoverable File System.

The dbpause_status Command

The UniData system-level dbpause_status command returns information about the status of dbpause.

Pausing UniData 9-7

Syntax:

dbpause_status

The following example illustrates the output from the dbpause_status command when dbpause is in effect:

% dbpause_status DBpause is ON. %

Resuming Processing

To resume processing after issuing the dbpause command, issue the dbresume command. User processes resume, and writes that were blocked when the dbpause command was issued complete.

Syntax:

dbresume

You must log on as root to issue the dbresume command.

The following screen illustrates the output from the dbresume command:

# dbresume DBresume successful. #

9-8 Administering UniData on UNIX Additional Commands

UniData provides a number of system-level commands to assist you in clearing users, processes, and system resources from UniData, if necessary. These commands are intended for removing hung processes, clearing ipc facilities for a clean start of UniData, or logging users and resources off for an emergency shutdown. These commands are listed in the following table.

UniData Command Function

listuser Lists all current UniData users.

showud Lists all UniData daemons.

stopudt Logs a user out of UniData; a current write completes, but subse- quent operations for that udt do not take place. stopudt executes the UNIX kill -15 command.

deleteuser deleteuser first issues the UNIX kill -15 command. If kill -15 is not successful after 5 seconds, deleteuser issues the UNIX kill - 9 command. Kill -9 may interrupt a write in progress, causing inconsistent data and file corruption.

ipcstat Lists all ipc structures in use on the system; identifies those used by UniData daemons.

udipcrm Removes all ipc facilities associated with UniData (queues, shared memory segments, semaphores). This command shuts UniData down and may corrupt files; use the command only if a daemon has been killed and has left ipc structures behind.

stopud -f Forces all UniData daemons to stop regardless of system activity. UniData System-Level Commands

Warning: Notice that stopudt, deleteuser, udipcrm, and stopud -f all carry the risk of disturbing the integrity of your data. They should never be used as a routine substitute for normal user logoff.

Additional Commands 9-9

Listing Processes with showud

The showud command lists all UniData processes that are currently running. The following screen shows an example of showud output:

# showud UID PID TIME COMMAND root 20376 0:00 /disk1/ud72/bin/aimglog 0 14553 root 20377 0:00 /disk1/ud72/bin/aimglog 1 14553 root 20378 0:00 /disk1/ud72/bin/bimglog 2 14553 root 20379 0:00 /disk1/ud72/bin/bimglog 3 14553 root 20367 0:00 /disk1/ud72/bin/cleanupd -m 10 -t 20 root 20373 0:00 /disk1/ud72/bin/cm 14553 root 20363 0:00 /disk1/ud72/bin/sbcs -r root 20372 0:00 /disk1/ud72/bin/sm 60 30482 root 20356 0:00 /disk1/ud72/bin/smm -t 60 root 20393 0:00 /disk1/unishared/unirpc/unirpcd #

Stopping a User Process with stopudt

The stopudt command logs out a user, as opposed to stopud, which stops UniData.

Syntax:

stopudt pid

The argument pid is a UNIX process ID.

Use this command if you need to log out a user you cannot reach, or to clear a hung user process. The following screen shows the action of stopudt.

% listuser Max Number of Users UDT SQL TOTAL ~~~~~~~~~~~~~~~~~~~ ~~~ ~~~ ~~~~~ 32 3 0 3 UDTNO USRNBR UID USRNAME USRTYPE TTY TIME DATE 1 645 1182 miker udt ttyp2 09:06:46 Jun 27 2002 2 3717 1172 claireg udt pts/3 10:31:14 Jun 27 2002 3 670 1179 joand udt ttyp3 09:16:24 Jun 27 2002

# stopudt 3717 # listuser Max Number of Users UDT SQL TOTAL ~~~~~~~~~~~~~~~~~~~ ~~~ ~~~ ~~~~~ 32 2 0 2 UDTNO USRNBR UID USRNAME USRTYPE TTY TIME DATE 1 645 1182 miker udt ttyp2 09:06:46 Jun 27 2002 3 670 1179 joand udt ttyp3 09:16:24 Jun 27 2002 #

9-10 Administering UniData on UNIX You must log on as root to execute stopudt. If you run listuser immediately after stopudt, the user may still be included in the display. This is because the cleanupd process performs its checking and cleanup routines at a predefined interval.

Note: The argument for stopudt is the process ID (pid) associated with the udt process you are removing. If you use the UNIX ps command to get the number, the pid is clearly labeled. If you use the UniData listuser command, as shown in the preceding screen, the pid is called USRNBR.

Stopping a User Process with deleteuser

The deleteuser command first tries to kill the user process with the UNIX kill -15 command. If the kill -15 is unsuccessful after five seconds, a kill -9 is issued.

Syntax:

deleteuser pid

The argument pid is the UNIX process ID.

Warning: Because deleteuser may execute the UNIX kill -9 command, it is particularly important that you verify the pid carefully.

The following screen shows an example of the deleteuser command:

% listuser

Max Number of Users UDT SQL TOTAL ~~~~~~~~~~~~~~~~~~~ ~~~ ~~~ ~~~~~ 32 4 0 4

UDTNO USRNBR UID USRNAME USRTYPE TTY TIME DATE 1 645 1182 miker udt ttyp2 09:06:46 Jun 27 2002 2 3742 1179 joand udt ttyp3 10:47:14 Jun 27 2002 3 3930 1283 carolw udt pts/2 12:47:07 Jun 27 2002 5 3953 1172 claireg udt pts/3 13:32:01 Jun 27 2002

# deleteuser 3953 # listuser

Max Number of Users UDT SQL TOTAL ~~~~~~~~~~~~~~~~~~~ ~~~ ~~~ ~~~~~ 32 3 0 3

UDTNO USRNBR UID USRNAME USRTYPE TTY TIME DATE 1 645 1182 miker udt ttyp2 09:06:46 Jun 27 2002 2 3742 1179 joand udt ttyp3 10:47:14 Jun 27 2002 3 3930 1283 carolw udt pts/2 12:47:07 Jun 27 2002 Note: You must log on as root to execute deleteuser.

Additional Commands 9-11

Displaying ipc Facilities with ipcstat

The ipcstat command displays a list of all ipc facilities (message queues, semaphores, and shared memory segments) in use on a system, and identifies those facilities used by UniData processes. You do not need to log on as root to execute ipcstat.

Syntax:

ipcstat [-q] [-m] [-s]

The following screen shows an example of ipcstat output:

% ipcstat -m IPC status from /dev/kmem as of Tue Jun 22 14:48:14 2002 T ID KEY MODE OWNER GROUP Shared Memory: m 0 0x00000000 D-rw------root sys -> unknown m 1 0x411c031b --rw-rw-rw- root root -> unknown m 2 0x4e0c0002 --rw-rw-rw- root root -> unknown m 3 0x41201102 --rw-rw-rw- root root -> unknown m 2404 0x431cdb10 --rw-rw-rw- root sys -> unknown m 1405 0x00000000 --rw-rw-rw- root sys -> unknown m 406 0x00000000 --rw-rw-rw- root sys -> unknown m 407 0x00000000 --rw-r--r-- root sys -> unknown m 208 0x631cdb10 --rw-rw-rw- root sys -> unknown m 820 0x00000000 D-rw-rw-rw- root sys -> unknown m 1021 0x441c063f --rw-rw-rw- root sys -> smm R5.2 (ctl) m 422 0x00000000 -Crw-rw-rw- root sys -> smm R6.1 (glm) m 423 0x00000000 -Crw-rw-rw- root sys -> smm R6.1 (shmbuf) m 424 0x00000000 --rw-r--r-- root sys -> sbcs R6.1 m 225 0x641c063f --rw-rw-rw- root sys -> sm R5.2 (ctl) m 226 0x00000000 --rw-rw-rw- root sys -> sm R6.1 (sysbuf) # # Notice that, because the -m option was specified, the output lists shared memory segments only. Use ipcstat -q to display message queues, ipcstat -s to list semaphores, and ipcstat with no options to list all ipc facilities.

Note: UniData does not use all the ipc facilities on the system. The output from ipcstat indicates which ones are used by UniData processes. The ones that corre- spond to “unknown” are not associated with UniData daemons.

9-12 Administering UniData on UNIX Removing ipc Structures with udipcrm

The udipcrm command uses the UNIX ipcrm command to remove any and all ipc facilities associated with any UniData process. After an abnormal shutdown, you may be unable to start UniData because some ipc facilities did not stop cleanly. You can use either the UNIX ipcrm command or udipcrm to remove them.

Syntax:

udipcrm

The udipcrm command is related to the ipcstat command. ipcstat lists all ipc facilities currently in use on a system, and identifies which ones are used by UniData processes. udipcrm only removes the ones associated with UniData.

Warning: Do not use udipcrm to shut down UniData. Use this command only if UniData is down, you cannot restart UniData, and there are ipc facilities that did not stop normally. Use the system-level command showud to verify that the UniData daemons are not running, and use ipcstat to identify ipc facilities that did not stop normally. See Chapter 18, “Managing ipc Facilities,” for further information.

Stopping UniData with stopud -f

This command stops all daemons and UniData processes regardless of activity on the system. Use this only if your system is hung and stopud fails to work.

Syntax:

stopud -f

Additional Commands 9-13

The following screen shows the expected output from stopud -f:

# stopud -f Using UDTBIN=/disk1/ud72/bin

WARNING: 'stopud -f' will stop the Unidata System with force. This may not guarantee the consistency of the database files.

Would you like to continue?(y/n) [n] y SM stopped successfully. CLEANUPD stopped successfully. SBCS stopped successfully. SMM stopped successfully. Unirpcd has already been stopped

Unidata R7.2 has been shut down.

# You must log on as root to run stopud -f.

Warning: stopud -f may leave your database in an inconsistent state; use it as a last resort.

9-14 Administering UniData on UNIX Chapter Managing UniData Accounts 10

What Is a UniData Account? ...... 10-3 Creating a UniData Account ...... 10-4 Saving and Restoring Accounts ...... 10-8 Deleting an Account...... 10-9 Clearing Space in UniData Accounts ...... 10-10 CLEAR.ACCOUNT ...... 10-10

This chapter describes UniData accounts, and describes procedures used to create, save, and clear the accounts.

10-2 Administering UniData on UNIX What Is a UniData Account?

A UniData account is a UNIX directory that contains a default set of UniData files, including a VOC file and its dictionary.

Note: The default files UniData requires for an account are created by the UniData system-level newacct command.

The VOC file identifies commands, paragraphs, and all data files that are used in the UniData account. The data files may be in the same UNIX directory as the VOC file, or the VOC file may contain pointers to data files in other UNIX directories. Your system may have a single UniData account, or several, depending on your application.

Note: A UNIX account typically means a logon ID, its associated password, and its default directory. There is no direct relationship between UniData accounts and UNIX accounts (logons). Many UNIX users may access any UniData account. A UNIX user’s default directory does not have to be (and usually is not) a UniData account.

What Is a UniData Account? 10-3

Creating a UniData Account

Complete the following four steps to create a UniData account:

1. Make sure the environment variable UDTHOME is set. 2. Use the UNIX mkdir command to create the directory that will house the account. The name of the UniData account directory can be in uppercase, lowercase, or mixed uppercase and lowercase. 3. Change to the directory with the UNIX cd command. 4. Use the UniData newacct command to create the VOC and other UniData- specific files in the directory. Note: You do not need to log on as root to create a UniData account. However, the newacct command prompts you for a user and group for your account. If you are logged on as root, UniData uses this information to set ownership and permissions for the account. If you are not logged on as root, UniData ignores your responses and uses your current logon and group ID.

The following three screens illustrate how to create an account, how to enter UniData in the new account, and how to use the UniData LS command to list the contents of the account:

# mkdir ACCOUNT # cd ACCOUNT # newacct % newacct The UDTHOME for this account is /disk1/ud72/. Do you want to continue(y/n)?

Please enter the account group name: users . . . Notice that, in the example, UDTHOME was already set to the path of /usr/ud72. When you execute newacct, UniData creates the new VOC file using a standard VOC file located in udthome/sys.

Tip: If you want to tailor your standard VOC file before creating new accounts, you may do so. IBM recommends that you save a copy of the standard VOC before making changes.

10-4 Administering UniData on UNIX The next example shows output from newacct:

Initializing the account ...

VOC and D_VOC file are created Creating file D_SAVEDLISTS modulo /1 Added "@ID", the default record for UniData to D_SAVEDLISTS. Creating file D_savedlists modulo /1 Added "@ID", the default record for UniData to D_savedlists. Creating file D__PH_ modulo /1 Added "@ID", the default record for UniData to D__PH_. Creating file D__HOLD_ modulo /1 Added "@ID", the default record for UniData to D__HOLD_. Creating file D_BP modulo /1 Added "@ID", the default record for UniData to D_BP. Creating file D_CTLG modulo /1 Added "@ID", the default record for UniData to D_CTLG. D__REPORT_ file created Create file _REPORT_(&report&), modulo/17 D__SCREEN_ file created Create file _SCREEN_, modulo/17 D_MENUFILE file created Create file MENUFILE, modulo/2 D___V__VIEW file created Create file __V__VIEW, modulo/11 The next example shows output from udt and LS:

% udt UniData Release 7.2 Build: (3089) (c) Copyright IBM Corporation 2005. All rights reserved.

Current UniData home is /disk1/ud72/. Current working directory is /disk1/ud72/demo. : The following table describes the default subdirectories UniData creates with a new account.

Subdirectory Description

BP Used to store UniBasic source and object code.

CTLG Used to store locally cataloged UniBasic programs.

SAVEDLISTS Used to store select lists. Subdirectories in a UniData Account

Creating a UniData Account 10-5

Subdirectory Description

savedlists Used to store temporary information for BY.EXP sorts. UniData automatically creates and deletes the contents of this subdirectory.

_HOLD_ Used to store print files.

_PH_ Used to store output from background processes (created by the UniData ECL PHANTOM command) and captured terminal I/O (created by the UniData ECL COMO command). Subdirectories in a UniData Account (continued) UniData creates the subdirectories empty and populates them as the account is used.

The next table describes the UniData hashed files UniData creates in a new UniData account.

File Description

MENUFILE Stores user-generated menu definitions.

VOC VOC (vocabulary) file, containing references for ECL commands, sentences, paragraphs, and filenames.

_REPORT_ Used to store UReport report definitions.

_SCREEN_ Used to store UEntry screen definitions.

__V__VIEW Used to store UniData SQL view specifications.

D_BP Dictionary for the BP file, which holds UniBasic programs.

D_CTLG Dictionary for CTLG.

D_MENUFILE Dictionary for MENUFILE.

D_SAVEDLISTS Dictionary for SAVEDLISTS.

D_VOC Dictionary for VOC.

D__HOLD_ Dictionary for _HOLD_.

D__PH_ Dictionary for _PH_.

D__REPORT_ Dictionary for _REPORT_. Hashed Files in a UniData Account

10-6 Administering UniData on UNIX File Description

D__SCREEN_ Dictionary for _SCREEN_.

D___V__VIEW Dictionary for __V__VIEWS.

D_savedlists Dictionary for savedlists. Hashed Files in a UniData Account (continued) Note: See Developing UniBasic Applications and Using UniData SQL for information about UniBasic and UniData SQL

Creating a UniData Account 10-7

Saving and Restoring Accounts

UniData includes two commands specifically designed for backing up and restoring UniData accounts. These are described in the following table.

UniData Command Description

ACCT.SAVE Saves a current UNIX directory and its subdirectories to tape; uses UNIX cpio utility, and writes only to the device defined as tape unit 0. (Use the SETTAPE command to define a tape unit, and T.ATT to obtain exclusive use of it within UniData.)

acctrestore [n] Restores a UniData account that was saved with ACCT.SAVE; uses UNIX cpio utility; restricted to a single tape volume. Specify the tape unit number n; the tape unit must be defined with SETTAPE. If you execute acctrestore while UniData is running, you may corrupt your data files. UniData Save and Restore Commands

Note: These commands do not function if you are using the Recoverable File System. An error message displays to the terminal if you attempt to execute them. Note: Use ACCT.SAVE and acctrestore carefully. These commands do not follow either UniData pointers to other directories (set with the SETFILE command) or symbolic links for large dynamic files. They are designed for use with small, self- contained UniData accounts.

Most UniData customer sites use the UNIX tar or cpio utilities, or commercial backup utilities, to back up UniData files and accounts. Consult your host operating system documentation and vendor documentation to determine the procedures to use at your site.

10-8 Administering UniData on UNIX Deleting an Account

There is no UniData command or utility that allows you to delete an entire account. If you need to delete an account, complete the following steps:

1. Analyze the database and identify which files should be deleted. Take care not to delete shared files that other UniData accounts may need. 2. Create and verify a full backup of at least the account you are going to delete. 3. Consider strategy. If the account is self-contained (that is, within one file system), you can use the UNIX rm -r command to delete the account directory. If the account has file pointers to other file systems, or dynamic files with part files on other file systems, you may wish to use the ECL DELETE.FILE command to delete the files before removing the account directory. Use the ECL MAX.USER command to prevent inadvertent access to the UniData account. Warning: Be careful with rm -r. This command removes all files and subdirectories below the directory you specify. If you have nested accounts (a UniData account within a UniData account) and you remove an account directory with rm -r, you could remove more than one account.

Deleting an Account 10-9

Clearing Space in UniData Accounts

The _PH_ and _HOLD_ subdirectories of each UniData account store output from background processes and temporary print files, respectively. These subdirectories can become large, causing disk space problems. The UniData ECL CLEAR.ACCOUNT command removes all files from either or both of these subdirectories.

CLEAR.ACCOUNT

Syntax:

CLEAR.ACCOUNT

You must log on to the UniData account you are clearing. You do not need to log on as root, however, you must have write permission for the _PH_ and _HOLD_ direc- tories. When you issue the command, UniData prompts you for confirmation to clear each directory, as shown in the following example:

:WHERE /disk1/ud72/ACCOUNT :CLEAR.ACCOUNT Clear _PH_ directory(Y/N)? Y Clear _HOLD_ directory(Y/N)? Y Notice that the ECL WHERE command displays the current account.

Warning: CLEAR.ACCOUNT removes ALL files from the subdirectories. Use this command only if you are certain none of the information is needed. Use the UniData DELETE command or the UNIX rm command, if necessary, to remove only selected files.

10-10 Administering UniData on UNIX Chapter Managing UniData Security 11

Logins and Groups ...... 11-3 Adding a UNIX User ...... 11-3 Use Separate Logon IDs ...... 11-4 User Groups ...... 11-4 Home Directories ...... 11-5 Startup Scripts ...... 11-5 Customizing Permissions ...... 11-7 Customizing a VOC File ...... 11-11 Customizing UniData ...... 11-12 Remote Items ...... 11-14 The SETFILE Command ...... 11-16 LOGIN and LOGOUT Paragraphs ...... 11-17 UniData SQL Privileges ...... 11-20 Field-Level Security for UniQuery ...... 11-21 Points to Remember about Field-Level Security ...... 11-21 The QUERY.PRIVILEGE File ...... 11-22 Turning on Field-Level Security ...... 11-24

While UniData uses UNIX permissions as its primary security mechanism, there are a number of procedures you can use for customizing the security of your UniData applications. This chapter describes how to use UNIX logon IDs and groups, how to modify default permissions, how to customize your VOC file, and how to use remote pointers. The chapter also describes the relationship between SQL privileges and other UniData security mechanisms.

11-2 Administering UniData on UNIX Logins and Groups

To access UniData, users need a UNIX logon ID (account).

Note: Utilities for adding and maintaining UNIX accounts are available on many systems. Examples are sam or smit. They require root access, and automate the required steps. For consistency, IBM recommends that you use your UNIX system administration utility to create and maintain UNIX accounts. Refer to your host operating system documentation and vendor documentation for information about procedures available on your particular system.

UNIX typically stores user and group information in two files, as shown in the following table.

UNIX Filename Description

/etc/group Contains a record for each group ID number (gid). Each record includes a group name and defines the user ID numbers (uid) associated with the group.

/etc/passwd Contains a record for each user ID. Each record includes a logon name, uid, gid, password, home directory, default shell identifier, and other optional information. UNIX Security Files

Note: Some UNIX systems use additional files for security. For example, Solaris uses a file called /etc/shadow, and AIX uses one called /etc/security/passwd. Refer to your host operating system documentation for complete information about these files.

Adding a UNIX User

Adding a UNIX user, either manually or through your system administration utility, involves the following steps:

1. Edit the /etc/passwd and /etc/group files (and /etc/shadow if required). 2. Set a password for the user. 3. Create the home directory, and install startup scripts, if necessary.

Logins and Groups 11-3

Use Separate Logon IDs

Although other database systems make use of group logon IDs, which are shared by a number of users, UniData enables you to define a separate logon ID for each user. IBM strongly recommends using separate logon IDs for the following reasons:

„ Most UNIX operating systems impose limits for system resources (such as number of processes) that can be associated with one user. Using separate logon IDs makes your system utilize resources more effectively. „ It is easier to identify processes belonging to an individual user, which facil- itates troubleshooting. „ Using separate logon IDs allows you to define your users’ responsibilities for protecting their passwords and your data.

User Groups

Every UNIX user is assigned to a default group. The system recognizes the user as a member of that default group at logon. UNIX permissions allow you to differentiate access among members of a group and users who are not members of that group. You can use this feature to provide security in UniData by defining applications (or accounts) that should be accessible to certain users, defining groups to which those users belong, and setting the group owners of the files and directories accordingly. For example, consider the directory structure shown in the following example:

. . . drwxrwxr-- 2 root apusers 24 Jun 18 18:18 PAYABLES drwxrwxr-- 2 root arusers 24 Jun 18 18:19 RECEIVABLES . . . The example shows a long style listing for separate UniData accounts for two appli- cations: PAYABLES and RECEIVABLES. Notice the following:

„ Users in group apusers have read, write, and execute access to the contents of PAYABLES, but only read access to RECEIVABLES. They can list the contents of RECEIVABLES, but they cannot add or delete files in RECEIV- ABLES, or cd to that directory, or access it with the ECL LOGTO command.

11-4 Administering UniData on UNIX „ Users in group arusers have read, write, and execute access to RECEIV- ABLES, but only read access to PAYABLES. They can list the contents of PAYABLES, but they cannot add or delete files in PAYABLES, or cd to that directory, or access it with the ECL LOGTO command.

You can assign users to more than one group. Refer to your host operating system documentation for information about your system. The UNIX id command enables root users to display the current uid and group assignment for any logon name. The UNIX newgrp command lets users change between groups. A user can function as a member of only one group at any time.

A user must be defined as a member of a group (in /etc/group) before using newgrp to change to it. In the above example, a user who is a member of only group arusers cannot use newgrp to change to apusers.

Home Directories

UNIX requires you to define a home directory for each logon ID. This directory is the working directory of the user at logon time. You can define the same home directory for a group of users, or create a separate one for each user. The home directory does not need to be a UniData account.

Tip: A user’s home directory can contain startup scripts as well as output from non- UniData applications (such as UNIX text editors). The UniData command stack is also saved here. If you use UniData accounts as home directories, and users generate other kinds of output there, you may encounter space or performance problems. Note: The home directory you define for a user must exist. If you define a directory and you do not create it or set appropriate permissions, the user may be unable to log on to UNIX. Some administrative utilities create the directory and set permissions when you add a user. Check the documentation for your UNIX system administration utility to determine procedures at your site.

Startup Scripts

When a user logs on, the default shell (specified in /etc/passwd) executes a startup script, if there is one, in the user’s home directory. The file name of the startup script depends on the user’s default shell. If the shell is the Bourne or Korn shell, the startup script is called “.profile”. If the shell is the C shell, the startup script is called “.login”.

Logins and Groups 11-5

Tip: Many UNIX systems offer a default shell script that you can copy into a user’s home directory and then customize. Some systems also execute a systemwide script for all logon IDs. Refer to your host operating system documentation for specific information about your system. A sample .login file follows: setenv TERM vt100 stty erase ^H set path=( $path /usr/ud72/bin) umask 002 cd /usr/ud72/PAYABLES udt This example shows how to use a startup script to ensure that a user logs on to a particular UniData account and receives an ECL prompt, rather than a UNIX prompt. The example also defines the user’s terminal type, defines the key sequence for erasing characters, modifies the user’s path to include the UniData bin directory, and specifies the default permissions on files the user creates.

11-6 Administering UniData on UNIX Customizing Permissions

When you install UniData, the system sets default permissions on system files and directories, as shown in the following table.

Permission Permission Settings Settings of of File or Directory Name File/Directory Files/Subdirectories

udtbin drwxr-xr-x See next rows

udtbin/product.info -rw-r--r-- Not applicable

udtbin/us_admin -r-sr-xr-x Not applicable

udtbin/us_update_db -r-sr-xr-x Not applicable

udthome drwxr-xr-x See following rows

udthome/demo drwxrwxrwx -rwxrwxrwx

udthome/lib drwxr-xr-x

udthome/objcall drwxrwxr-x See next rows

udthome/objcall/demo drwxrwxr-x rw-rw-rw-

udthome/objcall/include drwxrwxr-x rw-r--r--

/usr/ud72/ods drwxrwxr-x Varies

udthome/parttbl Not applicable -rw-r--r--

udthome/sybase drwxrwxr-x Varies

udthome/sys drwxr-xr-x Varies

udthome /sys/HELP.FILE -rw-r--r-- Not applicable

udthome/sys/udtpath -rw-rw-rw Not applicable

udthome/work drwxr-xr-x -rw-rw----

/usr/ud72/include drwxr-xr-x -rw-r--r-- Default Permission for UniData Directories and Files

Customizing Permissions 11-7

IBM makes a series of recommendations for customizing these permissions, as shown in the next table.

Directory or File Guidance

udthome/sys/CTLGTB The default permissions for CTLGTB, the global catalog table file, are -rw-rw-rw-. Users responsible for cataloging or deleting cataloged programs need write permission. Other users need only read permission.

udthome sys/DICT.DICT Users need only read permission. Administrators need write permission as well.

udthome/sys/VOC Users need only read permission. Administrators need write permission as well.

udthome/sys/CTLG Users, including programmers, need execute permission to this global catalog directory. In general, do not allow users to add or delete subdirectories to CTLG.

udthome/sys/ CTLG/n and CTLG contains a subdirectory for each letter of the alphabet directories and files within and one for symbols. Users need execute permission to these these subdirectories directories and read access to the files in them. Programmers may need read and write permissions so that they can catalog or delete cataloged programs.

udthome/demo This directory is used for training and experimentation. Use any permissions settings that suit your situation.

udthome/sys/AE_BP All users with access to AE (the line editor) need read and write permissions. Guidelines for Permissions for UniData System Files

11-8 Administering UniData on UNIX When you create a UniData account, IBM suggests the following guidelines for setting permissions for the directory, subdirectories, and files in the account:

Directory or File Guidance

./ Only users who need to create files in the directory should have write permission.

./BP Users need read and execute permissions so they can run UniBasic programs that are not globally cataloged. Programmers need write permission.

./CTLG Users need read and execute permissions so they can run locally cataloged programs. Programmers and administrators need write permission so they can locally catalog and delete locally cataloged programs.

./SAVEDLISTS Users need read and write permissions.

./_HOLD_ Users need read and write permissions.

./_PH_ Users need read and write permissions.

./VOC (This refers to the account VOC file, not the master VOC file in udthome/sys.) Users must have read and write access to enter their accounts unless you have set the VOC_READONLY environment variable. See Using UniData for more information about the VOC file. Suggested Permissions for a UniData Account

Customizing Permissions 11-9

The following screen shows a long listing for a UniData account called PAYABLES, incorporating the suggestions outlined in the preceding tables:

% ls -l /usr/PAYABLES total 386 drwxrwxr-x 2 root unisrc 24 Mar 7 2000 BP drwxrwxr-x 2 root unisrc 24 Mar 7 2000 CTLG -rwxrwxr-x 1 root unisrc 2048 Mar 7 2000 D_BP -rwxrwxr-x 1 root unisrc 2048 Mar 7 2000 D_CTLG -rw-rw-rw- 1 root unisrc 2048 Mar 7 2000 D_MENUFILE -rwxrwx--- 1 root unisrc 2048 Mar 7 2000 D_SAVEDLISTS -rw-rw-rw- 1 root unisrc 4096 Mar 7 2000 D_VOC -rwxrwx--- 1 root unisrc 2048 Mar 7 2000 D__HOLD_ -rwxrwx--- 1 root unisrc 2048 Mar 7 2000 D__PH_ -rw-rw-rw- 1 root unisrc 2048 Mar 7 2000 D__REPORT_ -rw-rw-rw- 1 root unisrc 2048 Mar 7 2000 D__SCREEN_ -rw-rw-rw- 1 root unisrc 2048 Mar 7 2000 D___V__VIEW -rw-rw-rw- 1 root unisrc 2048 Mar 7 2000 D_savedlists -rw-rw-rw- 1 root unisrc 3072 Mar 7 2000 MENUFILE drwxrwx--- 2 root unisrc 24 Mar 7 2000 SAVEDLISTS -rwxrwx--- 1 root unisrc 105472 Mar 7 2000 VOC drwxrwx--- 2 root unisrc 24 Mar 7 2000 _HOLD_ drwxrwx--- 2 root unisrc 24 Mar 7 2000 _PH_ -rw-rw-rw- 1 root unisrc 18432 Mar 7 2000 _REPORT_ -rw-rw-rw- 1 root unisrc 18432 Mar 7 2000 _SCREEN_ -rw-rw-rw- 1 root unisrc 12288 Mar 7 2000 __V__VIEW drwxrwxrwx 2 root unisrc 24 Mar 7 2000 savedlists

11-10 Administering UniData on UNIX Customizing a VOC File

Depending on your application, you may wish to prevent users from executing certain ECL commands. If you remove the entries corresponding to these commands from the VOC file in the account, users logged on to that account will not be able to execute them. When a user enters an ECL command, UniData searches the VOC file in the current account. If there is no corresponding entry, UniData displays an error message instead of executing the command.

The following example shows the results of deleting a VOC entry:

LIST VOC WITH @ID = COPY 19:03:11 May 23 2005 1 VOC......

COPY 1 record listed :DELETE VOC COPY 'COPY' deleted. :COPY FROM DICT INVENTORY TO DICT ORDERS ALL Not a verb COPY The following table lists ECL commands that create or modify UniData files, or allow users to execute UNIX system-level commands. You may want to consider removing some or all of these from the VOC files for your accounts.

Command Description

! Escapes to a UNIX shell prompt.

CLEAR.FILE Clears the data or dictionary of a file.

CNAME Changes a file name.

COPY Copies records.

CREATE.FILE Creates files.

CREATE.INDEX Creates an alternate key index.

DELETE Deletes records from VOC or other files.

DELETE.CATALOG Deletes catalog entries.

DELETE.FILE Deletes a file. VOC Commands Security

Customizing a VOC File 11-11

Command Description

DELETE.INDEX Deletes an alternate key index.

DISABLE.INDEX Disables an alternate key index.

ED Invokes the ED editor.

ENABLE.INDEX Enables an alternate key index.

MODIFY Modifies records in a data or dictionary file.

PTRDISABLE Disables a printer or queue.

PTRENABLE Enables a printer or queue.

RESIZE Resizes a file.

UPDATE.INDEX Updates an alternate key index. VOC Commands Security (continued) Note: You can remove entries from the UniData master VOC file (located in udthome/sys) or from the VOC files in one or more UniData accounts. The master VOC is installed as part of the UniData installation, and is used to build VOC files for your accounts when you execute the newacct command. If you remove commands from the master VOC file, and then create new UniData accounts, users in the new accounts will not be able to execute the commands you removed. Tip: If you choose to modify the master VOC file, make sure you save a copy of it and its dictionary before you begin your modifications. Warning: When you remove a VOC command entry, UniData no longer recognizes that command. UniData displays an error message if a user tries to execute the command, whether at the ECL prompt, or within a UniBasic program.

Customizing UniData

The UDT.OPTIONS command enables you to customize your UniData environment. UDT.OPTIONS 19 allows you to choose whether superusers (root access) can bypass security restrictions created by removing entries from the VOC file. If UDT.OPTIONS 19 is on, UniData prevents even superusers from executing commands after you remove the entries are from the VOC file.

11-12 Administering UniData on UNIX If UDT.OPTIONS 19 is off (the default), UniData allows superusers to execute ECL commands even if the command entries were removed from the VOC file. When a user logged on as root executes a command, UniData first reads the VOC file in the current account, just as it does for any other user. If there is a matching entry, UniData executes the command. If there is no matching VOC entry, and if UDT.OPTIONS 19 is off, root users can still execute the command. The following table shows the behavior of UDT.OPTIONS 19.

UDT.OPTIONS 19 Command Status Behavior

ON VOC entry exists root can execute command Others can execute command

OFF VOC entry exists root can execute command Others can execute command

ON No VOC entry root cannot execute command Others cannot execute command

OFF No VOC entry root can execute command Others cannot execute command UDT.OPTIONS 19

UDT.OPTIONS 19 is turned off by default. Leave it off to allow root users to execute commands that users cannot; turn it on to make root users consistent with other users.

Note: See the UDT.OPTIONS Commands Reference for detailed information about the UDT.OPTIONS command.

Customizing a VOC File 11-13

Remote Items

You can further customize security by replacing some command entries in your VOC file with remote items. A remote item (R-type VOC record) allows you to store a record definition in a location other than the VOC file. You can substitute remote items for sentences, paragraphs, commands, locally cataloged programs, or menus. See Using UniData for more information about R-type items.

R-type items enable you to customize security in two ways:

„ You can use a remote item as a pointer to a location with different UNIX file permissions from the current account, limiting access to the item. „ You can supply a “security routine” for the remote item. R-type items name a cataloged subroutine that is executed when a user invokes the remote item. The subroutine must have one argument, and return a value of 1 (true) or 0 (false). When a user invokes a remote item with a security subroutine, the remote item does not execute unless the subroutine returns 1 (true).

The following screen shows an example of a remote item created for the ECL LIST command:

:LIST VOC F1 F2 F3 F4 WITH @ID = LIST 11:05:23 May 24 2005 1 VOC...... F1...... F2...... F3...... F4......

LIST R OTHER_LIST LIST SECTEST2 1 record listed With this VOC record in place, when a user executes the LIST command, UniData executes a security subroutine called SECTEST2. If that subroutine returns a value of 1, UniData executes the item called LIST in a file called OTHER_VOC.

The next screen shows the security subroutine:

:AE BP SECTEST2 Top of "SECTEST2" in "BP", 4 lines, 66 characters. *--: P 001: SUBROUTINE SECTEST2(OKAY) 002: COMMON /SECUR/ VALID 003: OKAY = VALID 004: RETURN Bottom.

11-14 Administering UniData on UNIX In this example, the subroutine obtains the value of VALID from named COMMON. The value can be set by another subroutine or program. The following example shows what happens if VALID is zero (false) and a user executes the ECL LIST command:

:LIST VOC WITH F1 = PA Not a verb LIST The next screen shows what happens if VALID is 1 (true): :LIST VOC WITH F1 = PA LIST VOC WITH F1 = PA 11:13:27 May 24 2002 1 VOC......

ECLTYPE CP CT SP.OPEN listdict LISTDICT 6 records listed

Remote Items 11-15

The SETFILE Command

You can also customize security by placing data files in a location with different UNIX permissions than your UniData account, and then modifying the corre- sponding VOC entries to point to the location. Use the SETFILE ECL command to establish the file pointers, as shown in the following example:

:SETFILE /usr/SECURE/INVENTORY INVENTORY Establish the file pointer Tree name /usr/SECURE/INVENTORY Voc name INVENTORY Dictionary name /usr/SECURE/D_INVENTORY Ok to establish pointer(Y/N) = Y SETFILE completed. You can set the UNIX permissions at the location of the file to limit access. If a user with insufficient permissions tries to access the file, UniData displays an error message similar to the one shown in the following example:

:LIST INVENTORY ALL Open /usr/SECURE/INVENTORY error. Open file error. : See the UniData Commands Reference for information about the SETFILE command.

11-16 Administering UniData on UNIX LOGIN and LOGOUT Paragraphs

You can define LOGIN and LOGOUT paragraphs in the VOC files of your accounts to provide further control of users’ environments. A typical LOGIN paragraph sets UDT.OPTIONS and invokes an application menu. A typical LOGOUT paragraph executes a routine that cleans up application files and exits the application in an orderly manner. If a user’s .login or .profile sets their working directory to a UniData account and executes the udt command, and the LOGIN paragraph defines the UDT.OPTIONS and displays a menu, the user does not see either the UNIX shell prompt or the ECL prompt.

The behavior of LOGIN and LOGOUT paragraphs are summarized as follows:

„ When a user enters UniData, UniData checks the VOC file in the account the user is entering for a LOGIN paragraph. If there is one, UniData automatically executes it. „ If the user changes accounts with the ECL LOGTO command, UniData does NOT execute the LOGOUT paragraph in the account the user is leaving. UniData looks in the VOC file of the account the user is entering, and executes the LOGIN paragraph there, if there is one. „ When a user exits UniData, UniData checks the VOC file in the user’s current account and executes the LOGOUT paragraph, if one is there. Note: You can also use the ECL ON.ABORT command to prevent users from accessing the ECL or UNIX prompt. ON.ABORT defines a paragraph that executes whenever a UniBasic program aborts. See the UniData Commands Reference for information about ON.ABORT.

LOGIN and LOGOUT Paragraphs 11-17

The following sample session shows simple examples of LOGIN and LOGOUT paragraphs in a UniData account, and a different LOGOUT paragraph in a second account:

:WHERE /users/testacct :CT VOC LOGIN VOC:

LOGIN: PA UDT.OPTIONS 19 ON UDT.OPTIONS 20 ON DISPLAY ENTERING UNIDATA APPLICATION :CT VOC LOGOUT VOC:

LOGOUT: PA DISPLAY EXITING UNIDATA APPLICATION : :LOGTO /usr/ud72/demo :CT VOC LOGOUT VOC:

LOGOUT: PA RUN BP EXIT_PROG DISPLAY LOGGING OUT OF UNIDATA : In the preceding example, the second LOGOUT paragraph executes a program called EXIT_PROG, which simply prints a message. A user-written exit program can perform a variety of tracking and reporting functions.

11-18 Administering UniData on UNIX The next screen shows the response when two of these paragraphs are executed:

% pwd /users/testacct % udt UniData Release 7.2 Build: (3089) (c) Copyright IBM Corporation 2005. All rights reserved.

Current UniData home is /disk1/ud72/. Current working directory is /users/testacct. ENTERING UNIDATA APPLICATION :LOGTO /usr/ud72/demo :WHERE /users/ud72/demo :QUIT EXITING THE INVENTORY APPLICATION LOGGING OUT OF UNIDATA % Notice that the LOGIN paragraph defines UDT.OPTIONS and then prints a message. A LOGIN paragraph can also execute a program or display a menu. If a user’s .login or .profile file sets their working directory to a UniData account and executes the udt command, and the LOGIN paragraph defines the UDT.OPTIONS and displays a menu, the user does not see either the UNIX shell prompt or the ECL prompt.

Notice also that logging out of UniData after the LOGTO command executes the LOGOUT paragraph of the current account only.

Note: If UDT.OPTIONS 20, U_IGNLGN_LGTO, is on, users logged on as root can access an account through the LOGTO command without executing the LOGIN paragraph. If a user logged on as root accesses the account directly, UniData executes the LOGIN paragraph regardless of the setting of UDT.OPTIONS 20.

LOGIN and LOGOUT Paragraphs 11-19

UniData SQL Privileges

When a user creates a UniData SQL table or view, that user has exclusive UniData SQL access to it. Regardless of file permissions set at the UNIX level, no other user can execute UniData SQL operations against the table or view until the owner grants privileges through the UniData SQL GRANT command. The UniData SQL REVOKE command allows the owner (or any other user with ALL privileges) to revoke privileges that were granted. UniData SQL privileges can be granted or revoked for an entire table or for specified commands.

Note: UniData and UniData SQL share data and dictionary files. Therefore, depending on the UNIX file permissions, tables you create in UniData SQL can be accessed by other UniData tools, such as ECL or UniQuery. The GRANT and REVOKE commands affect UniData SQL operations only. See the UniData SQL Commands Reference and Using UniData SQL for additional information about UniData SQL privileges.

11-20 Administering UniData on UNIX Field-Level Security for UniQuery

UniData includes functionality to determine UniQuery access on an attribute-by- attribute basis.

System administrators can set privileges for UniQuery access at the file or attribute level for a single user, or for all users in the UNIX group, by creating a QUERY.PRIVILEGE table in a specified format and adding records to that table.

You can also set a default for your system, defining all files as OPEN or SECURE. In an OPEN system, the ability to access a file or a field with UniQuery is a function of UNIX file permissions, other UniData security implementations, and privileges granted using the QUERY.PRIVILEGE table. In a SECURE system, unless privi- leges are granted in the QUERY.PRIVILEGE table, users cannot access files through UniQuery, regardless of UNIX permissions or other implementations.

Points to Remember about Field-Level Security

Remember the following points about field-level security:

„ Implementing and maintaining field-level security is a completely manual process. You must create and populate the QUERY.PRIVILEGE file manually. „ ECL commands, such as CREATE.FILE, DELETE.FILE, and CNAME, do not update the QUERY.PRIVILEGE table. „ ECL commands are not affected by UniQuery security. „ The UniQuery MODIFY command is not affected by the UniQuery security feature. The security is imposed when a user attempts to SELECT. „ A default of OPEN or SECURE affects all UniData accounts that share the same udthome. You cannot define some accounts as OPEN and some as SECURE. „ Privileges granted on a file are not automatically applied to its dictionary. Therefore, if a user has ALL access to the INVENTORY file and its dictionary, you must consider D_INVENTORY as well. If the system default is OPEN, the user can access D_INVENTORY. Otherwise, if you want the user to access D_INVENTORY, you need a QUERY.PRIVILEGE record for D_INVENTORY as well.

Field-Level Security for UniQuery 11-21

The QUERY.PRIVILEGE File

UniQuery security depends on the existence of the QUERY.PRIVILEGE file, which must be located in udthome/sys. If this file does not exist, UniQuery functions as it has previously, with no field-level security.

Warning: If you create the QUERY.PRIVILEGE file, but do not populate the file with any records, UniData does not allow any user to access any files on the system through UniQuery.

When you install UniData, the UniQuery security is not implemented, and there is no QUERY.PRIVILEGE file. If you wish to turn on this feature, you must create QUERY.PRIVILEGE and D_QUERY.PRIVILEGE manually.

Records in the QUERY.PRIVILEGE file grant the SELECT privilege to users or groups of users, at the file level or the attribute level. Each QUERY.PRIVILEGE record has one attribute. The dictionary of the QUERY.PRIVILEGE file contains four records.

Following is a sample of the dictionary of the QUERY.PRIVILEGE file:

:LIST DICT QUERY.PRIVILEGE LIST DICT QUERY.PRIVILEGE BY TYP BY @ID TYP LOC CONV NAME FORMAT SM ASSOC 15:20:26 Jun 02 2004 1 @ID...... TYP LOC...... CONV NAME...... FORMAT SM ASSOC.....

@ID D 0 QUERY.PRIVILEGE 50L S PRIV D 1 PRIVILEGES 5R M FULLPATH V FIELD(@ID'*' File Path 25T S ,2) USERNAME V FIELD(@ID,'*' User Name 25T S ,1) 4 records listed

11-22 Administering UniData on UNIX The following table describes each QUERY.PRIVILEGE attribute:

Attributes Description

@ID Defines the user or UNIX group and the file for which you are setting privileges. If you are setting up a system default, @ID is DEFAULT. Otherwise, @ID must be of the format username*path or PUBLIC*path.

PRIV Multivalued; lists the field(s) to which you are granting access by location in the data file record. You can grant privileges on all fields in a data file by setting PRIV to ALL. If you are setting up a system default, set PRIV to either OPEN or SECURE.

FULLPATH The absolute UNIX path of the file on which you are setting UniQuery privileges.

USERNAME The UNIX ID of the user (or group) to which you are granting privileges. QUERY.PRIVILEGE Record Attributes

Note: You can customize the length of the dictionary attributes in the QUERY.PRIVILEGE file. The length of @ID should be sufficient to contain the longest UNIX user name and the longest absolute path for a UniData file on your system. FULLPATH and USERNAME should be long enough to handle the longest absolute path and longest UNIX user name, respectively.

The following table shows a very simple example of a QUERY.PRIVILEGE file:

LIST QUERY.PRIVILEGE PRIV 15:28:31 Jun 02 2004 1 QUERY.PRIVILEGE...... PRIVILEGES

user01*/usr/ud72/demo/INVENTORY 1 2 3 4 5 11 /user01*/usr/ud72/demo/CLIENTS ALL DEFAULT OPEN 3 records listed This QUERY.PRIVILEGE file means:

„ Except for INVENTORY and CLIENTS, which are in the demo database, all users have privileges to query all files in all accounts that share the same udthome.

Field-Level Security for UniQuery 11-23

„ user01 can query the fields in positions 1, 2, 3, 4, 5, and 11 only in the INVENTORY file. No other user can query this file. „ user01 can query any field in the CLIENTS file. No other user can query the CLIENTS file.

UniQuery Processing

If you have turned on the security feature by creating and populating the QUERY.PRIVILEGE file, every time a user logs on to UniData, their udt process reads the contents of QUERY.PRIVILEGE and stores the information for reference. Then, when a user attempts a UniQuery access, UniData checks the stored infor- mation, following the following steps:

1. Check for privileges granted to the user’s UNIX group. If the user’s UNIX group has sufficient privileges for the requested access, allow the access. Otherwise, proceed to step 2. 2. Check for privileges granted specifically to the user. If the user has sufficient privileges for the requested access, allow the access. Otherwise, proceed to step 3. 3. Check for privileges granted to PUBLIC. Privileges granted to PUBLIC apply to all system users. If PUBLIC has suf- ficient privileges for the requested access, grant the access. Otherwise, proceed to step 4. 4. Check for a DEFAULT entry. If there is a DEFAULT record in QUERY.PRIVILEGE, and if the default is set to OPEN, allow the requested access. If there is no DEFAULT, or if the DEFAULT is SECURE, disallow the access, displaying the following message: No privilege on filename.

Turning on Field-Level Security

Complete the following steps to implement the UniQuery field-level security feature:

1. Log In

With UniData running, log on as the root user. Users do not need to log off.

11-24 Administering UniData on UNIX 2. Create QUERY.PRIVILEGE

Change your working directory to udthome/sys, and enter udt to start a UniData session. Use the ECL CREATE.FILE command as follows:

:WHERE /usr/ud72/sys :CREATE.FILE QUERY.PRIVILEGE 101 Create file D_QUERY.PRIVILEGE, modulo/1,blocksize/1024 Hash type = 0 Create file QUERY.PRIVILEGE, modulo/101,blocksize/1024 Hash type = 0 Added "@ID", the default record for UniData to DICT QUERY.PRIVILEGE. Make the QUERY.PRIVILEGE file a static hashed file.

3. Set Permissions

The QUERY.PRIVILEGE file and its dictionary should be read-only to all users except root. Check the UNIX permissions and change them, if necessary, as follows:

:!ls -l *QUERY* -rw-rw-rw- 1 root sys 2048 Jun 2 16:03 D_QUERY.PRIVILEGE -rw-rw-rw- 1 root sys 104448 Jun 2 16:03 QUERY.PRIVILEGE : :!chmod 744 *QUERY* :!ls -l *QUERY* -rwxr--r-- 1 root sys 2048 Jun 2 16:03 D_QUERY.PRIVILEGE -rwxr--r-- 1 root sys 104448 Jun 2 16:03 QUERY.PRIVILEGE :

4. Edit the Dictionary

Use UniEntry, AE, or ED to edit D_QUERY.PRIVILEGE. The dictionary must look like the following example:

@ID...... TYP LOC...... CONV NAME...... FORMAT SM ASSOC.....

@ID D 0 QUERY.PRIVILEGE 50L S PRIV D 1 PRIVILEGES 5R M FULLPATH V FIELD(@ID,'*' File Path 25T S ,2) USERNAME V FIELD(@ID,'*' User Name 25T S ,1) Note: You can customize the format for the dictionary items to specify lengths for the attributes that match your system.

Field-Level Security for UniQuery 11-25

5. Add Records to QUERY.PRIVILEGE

For this step, you may prefer to have users logged out of UniData. As you add records to the QUERY.PRIVILEGE file, users logging on to UniData access whatever records are present at the time they log on, which may cause unexpected results.

Use AE, UniEntry, or ED to populate the QUERY.PRIVILEGE file.

11-26 Administering UniData on UNIX Chapter Managing UniData Files 12

UniData Hashed Files ...... 12-4 Static Hashed Files ...... 12-5 Dynamic Hashed Files ...... 12-6 Dynamic Files and Overflow ...... 12-6 Dynamic Files, Part Files, and Part Tables ...... 12-9 When Dynamic Files Are Created ...... 12-12 Tips and Constraints for Creating a Dynamic File ...... 12-14 When Dynamic Files Expand ...... 12-15 Management Tools for Dynamic Files ...... 12-18 Dynamic Files and Disk Space ...... 12-20 Sequentially Hashed Files ...... 12-24 File-Handling Commands ...... 12-28 File Corruption ...... 12-31 What Causes File Corruption? ...... 12-31 Preventing File Corruption...... 12-31 UniData Detection Tools ...... 12-33 guide ...... 12-33 guide_ndx ...... 12-38 UniData Recovery Tools ...... 12-40 dumpgroup ...... 12-40 fixgroup ...... 12-41 fixfile ...... 12-42 Detection and Repair Examples ...... 12-47 How to Use guide ...... 12-48 Error Messages ...... 12-51 File Access Messages ...... 12-51 Block Usage Messages ...... 12-51 Group Header Messages ...... 12-52 Header Key Messages ...... 12-52 Other Header Messages ...... 12-52 Free Block Messages ...... 12-54 Long Record Messages ...... 12-54

12-2 Administering UniData on UNIX In a UniData stores your data database in UniData hashed files of several different types in the database. UniData also supplies other types of files to support your database, including index files, program files, and directory files.

This chapter describes the types of UniData hashed files and explains the commands used to manage them. See Chapter 3, “UniData and the UNIX File System,” for information about other file types.

12-3

UniData Hashed Files

Hashed files are binary files that cannot be viewed at the operating system level or read by text editors external to UniData. Each UniData hashed file consists of a file header and one or more groups of data. Each data group contains the following structure:

„ A fixed-length group header „ A pointer array „ Record IDs „ Data

A record key is assigned to a group in the file according to a hashing algorithm. Then the precise location of the data is stored in the header of that group. The goal of hashing is to make searching for data more efficient by eliminating the need to search an entire file for a record. In a hashed file, UniData searches only the group where the primary key of the record was assigned. UniData supports two proprietary hashing algorithms (hash type 0 and hash type 1). The hash type determines the data group that contains each record.

The most efficient hashing algorithm for a file is the one that provides the best distri- bution of keys across the groups in the file. If the distribution is uneven, heavily loaded groups are accessed more frequently, which results in inefficient disk space use and increased contention for those groups. The default hash type for both static and dynamic files is 0. You can specify hash type 1 when you create a file, and you can switch between hash types with the memresize command.

12-4 Administering UniData on UNIX Static Hashed Files

Static hashed files are created with a specified block size multiplier and a specified modulo (number of data groups). The block size and modulo are stored in the header of the file.

Groups in static hashed files can overflow in two ways, as shown in the following table.

Overflow Type Description

Level 1 overflow The amount of space required for the data in the group is larger than the amount of space available. This happens if the length of a data record is too long for the block size, or if the number of records in the group grows so large that not all of the data fits. UniData appends overflow blocks to the original file, and the data portions of records are stored in overflow. The pointers and keys remain in the primary data file; accessing a record can require two reads, one to determine the address and the second to read the data in overflow.

Level 2 overflow The amount of space required for pointers and keys is larger than the total size of the group. This happens if too many records are hashed to the group. UniData creates overflow blocks which contain both data and keys. Record access can require multiple reads to determine the location and find the data. Level 1 and Level 2 Overflow

Note: When a group in a static file overflows, the overflow blocks are linked to that specific group. If overflow blocks are freed (by deleting records, for example) they remain linked to the original group and are not available to handle overflow from any other group in the data file. The space used by the blocks is not returned to the operating system. Level 1 overflow eventually impacts the performance of a static hashed file. Level 2 overflow impacts performance earlier and more severely, so correct sizing to prevent level 2 overflow is critical.

Static Hashed Files 12-5

Dynamic Hashed Files

Dynamic hashed files automatically increase modulo (number of groups) to minimize level 2 overflow. When viewed at the operating system level, the structure of dynamic files is different from that of static files. A dynamic file is actually a directory containing at least two binary files:

„ One or more data files named dat00x. These are hashed files. dat001 is the primary data file. Each group in a dat file contains a group header, keys, pointers, and data. „ One or more overflow files named over00x. Blocks in these files hold data when a group in a data file is in level 1 overflow.

The following screen shows the structure of the ORDERS file in the UniData demo database:

% ls -l . . . -rw-rw-rw- 1 root sys 2048 May 19 12:04 D_MENUFILE -rw-rw-rw- 1 root sys 3072 May 19 12:04 D_ORDERS -rw-rw-rw- 1 root sys 2048 May 19 12:04 D_PARAGRAPHS . . . -rw-rw-rw- 1 root sys 2048 May 19 12:04 MENUFILE drwxrwxrwx 2 root sys 1024 May 22 14:17 ORDERS -rw-rw-rw- 1 root sys 2048 May 19 12:04 PARAGRAPHS

% ls -l ORDERS total 66 -rw-rw-rw- 1 root sys 24576 May 19 12:04 dat001 -rw-rw-rw- 1 root sys 9216 May 19 12:04 over001

Notice that the dictionary file (D_ORDERS) is not a directory.

Dynamic Files and Overflow

Dynamic files automatically change size (both physical size and modulo) as you add data to them. You create a dynamic file with a specified initial modulo, which is the minimum modulo of the file. As records are added, UniData adds groups to the data file (dat001) to prevent level 2 overflow and adds blocks to the overflow file (over001) to contain level 1 overflow.

12-6 Administering UniData on UNIX Splitting and Merging

When a group in the primary data file reaches level 1 overflow, UniData stores the overflowed data in blocks in the overflow file. Blocks in over001 are linked (internal to UniData) to groups in the primary data file dat001. When the overflow file runs out of blocks, UniData adds blocks to it. To prevent level 2 overflow, UniData splits groups (increasing the modulo of the primary file) whenever the load factor of an existing group reaches a level called the splitting threshold (or SPLIT.LOAD). The splitting process is transparent to the user. When a group splits, the additional group is added to the primary data file, increasing its modulo and physical size.

As records are removed from a dynamic file, groups that had been split can merge back together if all the following conditions are true:

„ The current modulo of the file is greater than the minimum modulo of the file. „ The combined load factor of the two groups is less than a value called merging threshold (or MERGE.LOAD). „ One of the two groups is the last group in the file.

UniData provides two different split/merge types for dynamic files, KEYONLY and KEYDATA. You can set the split/merge type when you create a dynamic file, and you can change an existing split/merge type with the CONFIGURE.FILE command or the memresize command. Use FILE.STAT, ANALYZE.FILE, or GROUP.STAT to display the split/merge type.

KEYONLY Type

The KEYONLY split/merge type is the default for UniData dynamic files. For KEYONLY dynamic files, the load factor of a group is the percentage of the group space that is filled with keys and pointers. By default, the splitting threshold for a group in a KEYONLY dynamic file is 60%, so the group can split into two when the space occupied by keys and pointers reaches 60% of the size of the group. By default, the merging threshold for a KEYONLY dynamic file is 40%, so if the total load in a pair of groups that resulted from a split is under 40% of the size of a single group, the pair are eligible to merge. You can change the splitting threshold for a single KEYONLY file with the CONFIGURE.FILE or memresize commands, and you can change the defaults for all files by changing the SPLIT_LOAD and MERGE_LOAD parameters in the UniData configuration file (/usr/ud72/include/udtconfig).

Dynamic Hashed Files 12-7

KEYDATA Type

The KEYDATA split/merge type is also available for UniData dynamic files. For KEYDATA dynamic files, the load factor of a group is the percentage of the group space that is filled with keys, pointers, and data. By default, the splitting threshold for a group in a KEYDATA dynamic file is 95 percent, so the group can split into two when the space occupied by keys, pointers, and data reaches 95 percent of the size of the group. By default, the merging threshold for a KEYDATA dynamic file is 40 percent, so if the total load in a pair of groups that resulted from a split is under 40 percent of the size of a single group, the pair are eligible to merge.You can change the splitting threshold for a single KEYDATA file with the CONFIGURE.FILE or memresize commands, and you can change the defaults for all files by changing the KEYDATA_SPLIT_LOAD and KEYDATA_MERGE_LOAD parameters in the UniData configuration file (/usr/ud72/include/udtconfig).

Selecting a Split/Merge Type

Use the KEYONLY split/merge type for files whose records differ widely in length (standard deviation from average is large). When record lengths vary widely, the KEYONLY split/merge type makes more efficient use of disk space. Use the guide or FILE.STAT command to determine the record length and standard deviation from average for an existing file.

Use KEYDATA for files whose record length is consistent and in which the length of the data portion of each record is large with respect to the record ID. For files with these characteristics, the KEYDATA split/merge type provides a better distribution of records and less overflow than KEYONLY.

Dynamic Files and Hash Type

For both static and dynamic files, the default hash type is 0. This hash type provides a more even distribution of keys in groups in dynamic files. If key distribution in a file is uneven, you should consider tuning the modulo, block size, and split/merge type of the file. If none of these methods is effective, you should consider switching the hash type to 1.

12-8 Administering UniData on UNIX Dynamic Files, Part Files, and Part Tables

UniData allows a dynamic file to have multiple dat, over, and idx files, so that a dynamic file does not have the 2 gigabyte (GB) limit as does a static file. These “part files” can reside in different file systems. Each part file can approach 2 GB in size. The total number of part files in a dynamic file cannot exceed 255. Part files are numbered sequentially by type (for instance, dat002, over002, and idx002).

The UniData configuration parameter MAX_FLENGTH defines the maximum size, in bytes, for a “part” of a dynamic file. UniData distributes “part files” across file systems using ASCII files called “part tables.” A part table:

„ Lists eligible file systems into which dynamic files are allowed to expand. „ Specifies the amount of reserved space on each file system. Reserved space is not available for dynamic file expansion. Defining reserved space reduces the probability of the disk becoming full.

The default value for MAX_FLENGTH, set when you install UniData, is 1GB (1073741824 bytes). You can increase MAX_FLENGTH, but the maximum value is (2 GB – 16 K).

Location of Part Tables

System Default Part Table

The UniData installation procedure creates a system default part table, udthome/parttbl. The location of the system default part table is identified by the UniData configuration parameter PART_TBL.

Dynamic Hashed Files 12-9

Per-File Part Tables

You can also create individual part tables for dynamic files using vi or any other UNIX text editor. Assign a part table to a file by using the PARTTBL keyword with the CREATE.FILE and memresize commands. Supply the full path of the ASCII file you wish to use as a part table. UniData copies the ASCII file into the dynamic file directory, where it becomes the part table for the dynamic file. Specifying individual part tables allows you to balance the creation of part files across volumes in your system. In the following example, the PARTTBL keyword assigns a part table to a new dynamic file:

:CREATE.FILE TEST.FILE 3,2 DYNAMIC PARTTBL /disk1/unidata/parttbl Create file D_TEST.FILE, modulo/1,blocksize/1024 Hash type = 0 Create dynamic file TEST.FILE, modulo/3,blocksize/2048 Hash type = 0 Split/Merge type = KEYONLY Added "@ID", the default record for UniData to DICT TEST.FILE. :!ls -l TEST.FILE total 22 -rw-rw-rw- 1 terric unisrc 8192 Jul 1 14:38 dat001 -rw-rw-rw- 1 terric unisrc 2048 Jul 1 14:38 over001 -rw-rw-rw- 1 terric unisrc 37 Jul 1 14:38 parttbl Notice that the dynamic file directory contains the parttbl, which was copied from /disk1/unidata.

Components of a Part Table

The name of the system default part table (created at installation time) is udthome/parttbl. The default parttbl looks like:

% more /usr/ud72/parttbl . 1000 %

12-10 Administering UniData on UNIX The following table describes the fields in the part table:

Field Description

Path Name of the file system; the “.” in the default table indicates the file system where a UniData dynamic file is created. The “.” may be used as the first entry in the table; all other entries must be the absolute UNIX path (for instance, /disk6/files).

Reserved Space Amount of free space (in 512-byte blocks) that UniData should leave in the file system after creating part files there. Part Table Fields

Use vi or any other UNIX text editor to create and edit per-file part tables or to modify the default part table for your system. A sample part table follows:

. 10000000 /usr/unidata/partfiles 10000 /disk1/unidata/partfiles 10000

Part Table Tips and Constraints

Consider the following points if you are creating or modifying part tables:

„ Do not use a dynamic file directory as an entry in the parttbl. This causes a number of problems, including difficulty resolving the links to part files and the creation of part files from one dynamic file in the directory for another dynamic file. This, in turn, causes failures when users attempt to execute the CLEAR.FILE or DELETE.FILE command on one of the two dynamic files. „ Do not use multiple entries in the parttbl that resolve to the same device ID. This causes confusion when UniData attempts to check reserved space against available space. „ If you move the system default part table, change the value of the configu- ration parameter PART_TBL. „ When you create an ASCII file to use as a per-file part table, remember that UniData places a copy of that file in a dynamic file directory each time you specify it with the PARTTBL keyword (CREATE.FILE or memresize). If you wish to add a partition or otherwise edit the file, you need to edit the copy in each dynamic file to which it is assigned, as well as at the location where you created the file.

Dynamic Hashed Files 12-11

„ If you execute the memresize command to change the parameters of a dynamic file, and you specify the PARTTBL keyword for a file that already has a per-file part table, you will overwrite the existing per-file part table.

When Dynamic Files Are Created

The following considerations determine where the parts of a newly created dynamic file are located.

Estimating the Size of the File

The estimated space required for a new dynamic file is the smaller of the following:

„ MAX_FLENGTH „ minimum modulo * block size

If (minimum modulo * block size) is larger than MAX_FLENGTH, the new file needs more than one data part file.

Locating the Dynamic File Directory

The dynamic file directory is located in the UniData account where CREATE.FILE was executed.

Locating the Part Files

The part files (or UNIX links to them) are located in the dynamic file directory. UniData reserves space for part files on the UNIX file system where the dynamic file directory is located (the resident file system) by making the following assessments:

„ If the part table (per-file if one exists, or system default otherwise) has an entry for that file system, use the reserved space defined for that entry. For instance, if the dynamic file is /usr/ud72/demo/ORDERS, UniData checks the operating system file system tables for the UNIX file system, or device id, where /usr/ud72/demo resides, and then checks the part table for an entry for that file system. Depending on the configuration of the system, /usr/ud72/demo might reside in a file system called /usr, or /usr/ud72, or /usr/ud72/demo.

12-12 Administering UniData on UNIX „ If the part table does not have an entry for that file system, use the reserved space figure associated with the “.” entry.

UniData checks the resident file system for space as follows:

„ The space available in the file system must be more than the sum of the estimated file size and the reserved space requirement. „ If there is sufficient space, dat001 and over001 are created in the dynamic file directory.

If there is not enough space in the file system, UniData:

„ Checks each file system that has an entry in the part table. „ Creates dat001 and over001 on the first file system that meets the space requirement. „ Creates UNIX links in the original dynamic file directory.

The following screen illustrates creating a dynamic file in the current account directory:

:CREATE.FILE DYN_TEST 3,1 DYNAMIC Create file D_DYN.TEST, modulo/1,blocksize/1024 Hash type = 0 Create dynamic file DYN.TEST, modulo/3,blocksize/1024 Hash type = 0 Split/Merge type = KEYONLY Added "@ID", the default record for UniData to DICT DYN.TEST. : :!ls -l DYN_TEST total 10 -rw-rw-rw- 1 terric unisrc 4096 Apr 4 17:43 dat001 -rw-rw-rw- 1 terric unisrc 1024 Apr 4 17:43 over001 :!printenv MAX_FLENGTH 1073741824 : In the preceding example, the primary data file (dat001) includes a file header and the three data groups for a total of four 1K blocks. The overflow file (over001) includes a 1K file header. Since MAX_FLENGTH is larger than minimum modulo * block size, the primary data file and overflow file each have only one “part.”

Dynamic Hashed Files 12-13

The following example shows what happens when there is insufficient space on the resident partition of the dynamic file. The part table used in the example is one that stipulates a very large reserved space on the current file system, thereby forcing the part files to another file system:

:CREATE.FILE LARGE.TEST 17,8 DYNAMIC PARTTBL /home/terric/parttbl Create file D_LARGE.TEST, modulo/1,blocksize/1024 Hash type = 0 Create dynamic file LARGE.TEST, modulo/17,blocksize/8192 Hash type = 0 Split/Merge type = KEYONLY Added "@ID", the default record for UniData to DICT LARGE.TEST. :!ls -l LARGE.TEST total 6 lrwxrwxrwx 1 terric unisrc 42 Jun 8 15:52 dat001 - > /usr/unidata/partfiles/AALARGE.TEST/dat001 lrwxrwxrwx 1 terric unisrc 43 Jun 8 15:52 over001 -> /usr/unidata/partfiles/AALARGE.TEST/over001 -rw-rw-rw- 1 terric unisrc 69 Jun 8 15:52 parttbl Notice that the dat001 and over001 files were created in a different partition and are referenced by UNIX links.

Tips and Constraints for Creating a Dynamic File

Choosing the Initial Modulo

If you are creating a dynamic hashed file, selecting an appropriate starting (minimum) modulo is critical to the future efficiency of the file. You should select a starting modulo based on the expected future size of the file, because subsequent splitting and merging operations are affected by the initial modulo. Starting with a modulo that is very small (for instance, 3) produces inefficient hashing and splitting as the file grows. Starting with a modulo that is very large produces a file that may initially take up more disk space than needed, but that impact is more desirable than the slow performance and inefficiency that results if the starting modulo is too small.

When you create a dynamic file, estimate the initial modulo by using the same procedure for estimating the modulo for a static file.

12-14 Administering UniData on UNIX Choosing the Block Size

If you are creating a KEYDATA dynamic file, make sure the block size is large with respect to the record length. IBM recommends that you choose a block size that is at least 10 times the average record length. Load factor in a KEYDATA file is based on the percentage of the space in each block that is occupied by both keys and data. If the block size is not large with respect to record size, the file will occupy a large amount of space and much of that space will be unused.

If you are creating a KEYONLY dynamic file, make sure the block size is large with respect to the average key length. IBM recommends that you choose a block size that is at least ten times the average key length. Load factor in a KEYONLY file is based on the percentage of the space in each block that is occupied by keys and pointers. If the block size is not large with respect to the average key length, and the hashing is not even, certain groups will be split over and over, resulting in an inefficient distribution.

When Dynamic Files Expand

When records are added or updated in a dynamic file, a data part file, overflow part file, or index part file may expand. Whenever a write operation requires additional blocks, the following considerations determine whether a new part file is needed, and if so, where it is located.

Determining Whether a New Part File Is Needed

UniData first checks to see if the part file can expand in its current location. The part file can expand if the following two conditions are true:

„ The size of the part file is less than MAX_FLENGTH. „ There is enough space in the file system where the part file resides to complete the current write without expanding into reserved space. For instance, if 10 blocks are needed, and the difference between available and reserved space is more than 10 blocks, the part file can expand.

Dynamic Hashed Files 12-15

Note: To check for space, UniData resolves the directory where the part file resides to its UNIX file system, and then checks the part table for an entry for that file system. For example, if the part file is /usr/ud72/demo/ORDERS/dat001, UniData locates the UNIX file system where /usr/ud72/demo/ORDERS resides, and checks the part table for an entry for that file system. If there is an entry, UniData takes the reserved space defined for that entry. If not, UniData uses the reserved space defined for the “.” entry.

If both conditions are true, UniData adds blocks to the part file and continues processing. If either condition is not true, and the part file is an overflow part file, UniData checks all the existing overflow part files. If one of those part files meets the conditions, that part file is expanded. If no existing part file can expand, UniData must create a new overflow part file. If the part file is a data part file, UniData can expand only the last data part file created. For instance, if the dynamic file directory contains dat001 and dat002, only dat002 can expand.

Locating Space For a New Part File

The following table describes how UniData computes an estimated size for the new part file, depending on its type.

Part File Type Estimated Size

Data part file The smaller of the following: • The larger of (size of dat001) and (current modulo * block size) • MAX_FLENGTH

Overflow part file The smaller of the following: • The larger of (size of over001) and (current modulo * block size) • MAX_FLENGTH

Index part file The smaller of the following: • The larger of (size of dat001) and (current modulo * block size) • MAX_FLENGTH Estimating Size Requirements for Part Files

UniData searches the partitions listed in the part table for the dynamic file for a partition that meets the requirement:

available space > (estimated size + reserved space)

12-16 Administering UniData on UNIX UniData searches the part table in the following order:

1. The resident partition of the dynamic file, because locating the part file in the dynamic file directory saves the overhead required by a symbolic link. 2. All entries, in order from top to bottom. Upon finding a partition that meets the space requirement, UniData: „ Creates an appropriate directory, if needed. „ Creates the new part file. „ Creates a UNIX symbolic link in the dynamic file directory, if needed. If no partition meets the space requirement, UniData: „ Checks to see which partition has the largest amount of free space (available space - reserved space). „ If the amount of free space is 20% or more of the estimated size, creates the new part file at that location. „ If no partition has sufficient free space (20% or more of the estimated size of the part file), UniData prompts the user to reclaim space or change the part table.

How Part Files Are Stored

When dynamic files expand into a different file system, UniData creates directories in the new file system for the part files. To prevent duplicate directory names, UniData assigns a prefix to each. The prefixes are assigned based on an index in the target file system called .fil_prefix_tbl (a hidden ASCII text file maintained by UniData). A sample .fil_prefix_tbl follows:

% more .fil_prefix_tbl AA:/home/terric/SAMPLE AB:/home/terric/NEWACCT AC:/home/terric/NEWACCT/MULTI1 Each entry in .fil_prefix_tbl relates a prefix (AA, AB, and so on) to the path of a parent directory for dynamic files. The parent directory can be a UniData account directory or a UniData multilevel file directory. Using the sample table, this creates the following directories:

„ If a dynamic file originated in the account directory named /home/terric/SAMPLE, the directory created when the file expands is called AAfilename.

Dynamic Hashed Files 12-17

„ If a dynamic file originated in the account directory named /home/terric/NEWACCT, the directory created when that file expands is called ABfilename. „ If a dynamic file is a subfile of the multilevel file /home/terric/NEWACCT/MULTI1, the directory created when that file expands is called ACfilename.

The following screen shows a directory listing that corresponds to the prefix table from the previous example:

% pwd /usr/unidata/partfiles % ls -al|more total 16 drwxrwxrwx 7 root sys 1024 Jun 6 14:16 . drwxrwxrwx 3 root sys 1024 Jun 6 11:40 .. -rw-rw-rw- 1 terric unisrc 78 Jun 6 14:15 .fil_prefix_tbl drwxrwxrwx 2 terric unisrc 1024 Jun 6 11:43 AASAMPLE_FILE3 drwxrwxrwx 2 terric unisrc 1024 Jun 6 11:46 AATESTINV drwxrwxrwx 2 terric unisrc 1024 Jun 6 14:13 ABFAMILY_FILE1 drwxrwxrwx 2 terric unisrc 1024 Jun 6 14:15 ACSUB1 drwxrwxrwx 2 terric unisrc 1024 Jun 6 14:16 ACSUB2 Warning: Do not edit or remove .fil_prefix_tbl. You will encounter unexpected results creating, updating, and deleting dynamic files. If .fil_prefix_tbl is inadvertently removed from a target directory, you can execute the system-level fixtbl command in each of your UniData accounts to rebuild it.

Management Tools for Dynamic Files

UniData supplies three tools for system administrators to manage dynamic files. The tools identify and resolve some of the inconsistencies that result if users manipulate part files at the operating system level or inadvertently modify or delete the prefix table in a target partition.

12-18 Administering UniData on UNIX auditor

The system-level auditor tool reports inconsistencies between the symbolic links and the hidden files which should be resolved to prevent apparent data loss. The tool also reports an error if a part file is not found in the correct destination. Execute auditor from the UNIX prompt or use ! to execute auditor from the ECL command prompt. Your current working directory must be a UniData account. auditor checks all the dynamic files that have pointers in the VOC file of the current account. fixtbl

The system-level fixtbl command detects the following error conditions:

„ .fil_prefix_tbl is missing. If a dynamic file directory contains links to another partition, but there is no .fil_prefix_tbl at that location, fixtbl can create a new one. „ A prefix in .fil_prefix_tbl references a different directory than the symbolic links from a dynamic file in the current account. fixtbl can select a new prefix, then move and relink the part files for consistency. „ There are symbolic links from a dynamic file to another partition, but there is no entry in the .fil_prefix_tbl that matches the links. Assuming the prefix in the links is not used by another directory, fixtbl can create an entry in .fil_prefix_tbl that is consistent with the links from dynamic files in the current account directory. Note: You must stop the UniData daemons (with stopud) before executing fixtbl. mvpart

The system-level mvpart command moves one or more part files of a dynamic file to a different location. mvpart sets or resets symbolic links, if needed, and creates or updates a prefix table (.fil_prefix_tbl) at the destination location, if needed. Using mvpart ensures that the links, file locations, and prefix tables remain synchronized.

Note: You must stop the UniData daemons (with stopud) before executing mvpart.

Dynamic Hashed Files 12-19

Dynamic Files and Disk Space

While it is possible for a dynamic file to shrink with respect to modulo, the disk space “freed” by merging is not necessarily made available for other use. When a group splits, the extra group added to the dynamic file is assigned to the existing group that split. When a merge occurs, the “freed” group remains allocated to the dynamic file, for use if the original group splits again.

When data is removed from blocks in the overflow file, UniData keeps those blocks for the dynamic file. A certain number are reserved for the groups they were part of, and the remainder of the blocks are available for overflow from any group in the file. The UniData configuration parameter GRP_FREE_BLK defines the maximum number of free blocks that should be kept in the free block list for a particular group. If more blocks are freed, they are kept in the free block list at the file level.

Refer to Appendix A, “UniData Configuration Parameters,” for a list of the config- uration parameters.

If you remove all records from a dynamic file with either the ECL CLEAR.FILE command or the ECL DELETE command with the ALL option, the file returns to its minimum modulo, and the disk space is returned to UNIX. However, if you remove all records from a dynamic file using a select list, the file may not return to its minimum modulo. Depending on the order in which records are removed, some groups resulting from earlier splits may not become eligible for merging even though they do not contain any records.

12-20 Administering UniData on UNIX The following screens show splitting and merging in a dynamic file. The first screen shows creating a dynamic file with a minimum modulo of 3:

:CREATE.FILE SIZE.TEST 3,2 DYNAMIC Create file D_SIZE.TEST, modulo/1,blocksize/1024 Hash type = 0 Create dynamic file SIZE.TEST, modulo/3,blocksize/2048 Hash type = 0 Split/Merge type = KEYONLY Added "@ID", the default record for UniData to DICT SIZE.TEST. :FILE.STAT SIZE.TEST File name(Dynamic File) = SIZE.TEST Number of groups in file (modulo) = 3 Dynamic hashing, hash type = 0 Split/Merge type = KEYONLY Block size = 2048 Number of records = 0 Total number of bytes = 0 . . . File has 1 over files, 1 prime files :!ls -l SIZE.TEST total 20 -rw-rw-rw- 1 terric unisrc 8192 Jun 4 17:23 dat001 -rw-rw-rw- 1 terric unisrc 2048 Jun 4 17:23 over001 Notice the following points:

„ Because the file was created with a block size multiplier of two, the size of each block is 2,048 bytes. „ The primary file (dat001) has one block for the file header, and three for the data. „ The overflow file (over001) is initially allocated one block for its header. „ Because the split/merge type was not specified, the file was created as a KEYONLY file.

Dynamic Hashed Files 12-21

The next screen shows what happens when the dynamic file is populated with records:

:FILE.STAT SIZE.TEST File name(Dynamic File) = SIZE.TEST Number of groups in file (modulo) = 12 Dynamic hashing, hash type = 0 Split/Merge type = KEYONLY Block size = 2048 File has 4 groups in level one overflow. Number of records = 537 Total number of bytes = 19858 . . . File has 1 over files, 1 prime files

:!ls -l SIZE.TEST total 92 -rw-rw-rw- 1 terric unisrc 26624 Jun 4 17:25 dat001 -rw-rw-rw- 1 terric unisrc 20480 Jun 4 17:25 over001 Notice the following points:

„ The original three groups have split. „ Now there are 12 groups in the primary data file, and 10 groups (plus the header group) in the overflow file.

12-22 Administering UniData on UNIX The following screen shows the results of deleting all records with a select list:

:SELECT SIZE.TEST

537 records selected to list 0.

>DELETE SIZE.TEST Do you want to delete records in select list?(Y/N)Y . . . :FILE.STAT SIZE.TEST File name(Dynamic File) = SIZE.TEST Number of groups in file (modulo) = 10 Dynamic hashing, hash type = 0 Split/Merge type = KEYONLY Block size = 2048 Number of records = 0 Total number of bytes = 0 . . . File has 1 over files, 1 prime files . . . :!ls -l SIZE.TEST total 92 -rw-rw-rw- 1 terric unisrc 26624 Jun 4 17:28 dat001 -rw-rw-rw- 1 terric unisrc 20480 Jun 4 17:28 over001 Notice the following points:

„ Merging has reduced the modulo to 10. „ The file size as measured by the UNIX ls command has not changed from the previous example. „ Some groups did not merge, and the groups that were added remain allocated to the file.

Dynamic Hashed Files 12-23

The final screen shows the results of CLEAR.FILE:

:CLEAR.FILE SIZE.TEST SIZE.TEST is cleared. :FILE.STAT SIZE.TEST File name(Dynamic File) = SIZE.TEST Number of groups in file (modulo) = 3 Dynamic hashing, hash type = 0 Split/Merge type = KEYONLY Block size = 2048 Number of records = 0 Total number of bytes = 0

Average number of records per group = 0.0 Standard deviation from average = 0.0 Average number of bytes per group = 0.0 Standard deviation from average = 0.0

Average number of bytes in a record = 0.0 Minimum number of bytes in a record = 0 Maximum number of bytes in a record = 0

Minimum number of fields in a record = 0 Maximum number of fields in a record = 0 Average number of fields per record = 0.0 File has 1 over files, 1 prime files

:!ls -l SIZE.TEST total 20 -rw-rw-rw- 1 terric unisrc 8192 Jun 4 17:30 dat001 -rw-rw-rw- 1 terric unisrc 2048 Jun 4 17:30 over001 The ECL CLEAR.FILE command returns the file to its original modulo and size.

Sequentially Hashed Files

A sequentially hashed file has the same structure as a dynamic file, but all records are stored sequentially based on the primary key. The modulo (number of groups) for a sequentially hashed file is fixed, it does not grow and shrink as records are added or deleted.

You create a sequentially hashed files by converting from existing UniData static or dynamic files. You specify a percentage of the file that you want to remain empty to allow for growth. Although the structure for a sequentially hashed file is the same as a dynamic file, the modulo is fixed.

A sequentially hashed file is used for files where the majority of access is based on the primary key.

12-24 Administering UniData on UNIX The dat001 File

The dat001 file is also called the primary data file. As you add records to a sequen- tially hashed file, UniData hashes the keys, based on information in the gmekey file, to groups in dat001. If your data overflows the group (level 1 overflow), UniData writes the overflow data to the over001 file.

The over001 File

As you add records to a sequentially hashed file, whenever the space reserved for data in a group in the primary file gets too full, UniData writes the excess data into blocks in over001. Registers within UniData track how blocks in over001 are linked to groups in dat001. If over001 gets too large, UniData adds additional blocks to it. If the current file system becomes full, or over001 grows larger than a UniData limit, UniData creates an over002 file. If the over002 file resides in a different file system than the over001 file, UniData creates a link to over002 in the original sequentially hashed file.

If the sequentially hashed file has level 2 overflow, the file should be rebuilt using the shfbuild command.

The gmekey File

Each sequentially hashed file contains a static, read-only file called the gmekey file. This file is read into memory when you open a sequentially hashed file. The gmekey file contains information about the type of keys in the file (alpha or numeric), and controls which group a record is hashed to when it is written.

You create a sequentially hashed file by converting an existing dynamic or static file with the shfbuild command:

Syntax:

shfbuild [-a |-k] [-n | -t] [-f] [-e empty percent] [-m modulo] [-b block size multiplier] [-i infile] outfile

Dynamic Hashed Files 12-25

The following table describes each parameter of the syntax.

Parameter Description

-a Only rebuild the last group of the sequentially hashed file. UniData splits the last group into groups according to the records in the group. If you use this option, the outfile should be the name of the sequentially hashed file. Do not specify infile.

-k Build the gmekey file only. If you use this option, outfile should be the name of the sequentially hashed file. Do not specify infile. UniData rebuilds the gmekey file according to the keys in each group of outfile.

-n/-t Force the outfile to be in numeric or alphabetic order. By default, the order of outfile is determined by the infile primary key type. If infile is a sequen- tially hashed file, UniData uses the same order in outfile. If infile is not a sequentially hashed file, the order of outfile is determined by the justifi- cation of the @ID of the infile dictionary record. If it is right justified, it is numeric. Otherwise, it is alphabetic. If you use the -a or the -k option, these options have no effect.

-f Force outfile to truncate before being built.

-m Specifies the new modulo of outfile.

-e Empty percent. This is a number between 0 and 99 which indicates how much space in the rebuilt groups to reserve. UniData calculates the new modulo of the file from empty_percent and the number of records in the rebuilt groups. If you do not specify -e or -m, UniData rebuilds the sequen- tially hashed file according to the default empty percent of 20.

-b Specifies the block size of the sequentially hashed file in kilobytes.

-i infile Load the contents from infile instead of outfile. infile can be any type of UniData file.

outfile The name of the output file. shfbuild Parameters

To convert an existing file, execute the shfbuild command from the system level prompt, as shown in the following example:

% shfbuild -m 59 SEQUENTIAL 175 keys found from SEQUENTIAL. 175 records appended to SEQUENTIAL; current modulo is 59.

12-26 Administering UniData on UNIX After converting a file to a sequentially hashed file, you must manually enter a file pointer in the VOC file in order to access the sequentially hashed file, as shown in the following example:

:AE VOC SEQUENTIAL Top of New "SEQUENTIAL" in "VOC". *--: I 001= F 002= SEQUENTIAL 003= D_SEQUENTIAL *--: FI Filed "SEQUENTIAL" in file "VOC".

Dynamic Hashed Files 12-27

File-Handling Commands

UniData includes a variety of commands for you to create and delete UniData files, as well as to obtain status information, change file parameters, and diagnose and repair damaged hashed files.

Note: Refer to Chapter 3,“UniData and the UNIX File System,” for additional information about index files and index log files.

The following table describes ECL file-handling commands, and indicates the UniData file types they affect.

Command Description

CREATE.FILE Creates a UniData file; works for static and dynamic hashed files, dictionary files, DIR files, multilevel files and multilevel directories.

DELETE.FILE Deletes a UniData file; works for static, dynamic, and sequentially hashed files, dictionary files, DIR files, multilevel files, and multi- level directories.

CLEAR.FILE Removes all records from a UniData file; works for static, dynamic, and sequentially hashed files, dictionary files, DIR files, multilevel subfiles, and multilevel subdirectories.

CNAME Changes the name of a UniData file; works for static, dynamic, and sequentially hashed files and DIR files. Does not work for multilevel subfiles and multilevel subdirectories or dictionary files.

SETFILE Sets a pointer to a UniData file; works for static, dynamic, and sequentially files, DIR files, multilevel files, and multilevel direc- tories. Does not work for dictionary files or for multilevel subfiles or subdirectories.

RECORD Identifies group where a primary key is hashed, and displays a list of keys hashed to that group. Works for static, dynamic, and sequen- tially hashed files and for multilevel subfiles. Does not work for dictionaries, directory files, multilevel directories, or multilevel subdirectories. ECL File Commands

12-28 Administering UniData on UNIX Command Description

FILE.STAT Displays statistics about a hashed file, including modulo, block size, hash type, and record statistics. Works for static, dynamic, and sequentially hashed files, or static or dynamic multilevel subfiles. Does not work for dictionaries, directory or multilevel directory files, or multilevel subdirectories.

GROUP.STAT Displays record distribution in a UniData hashed file. Works for static, dynamic, or sequentially hashed files, or static or dynamic multilevel subfiles. Does not work for dictionaries, directory or multilevel directory files, or multilevel subdirectories.

RESIZE Changes the modulo, block size, or hash type of a UniData static hashed file. Works on static hashed files, static hashed multilevel subfiles, or dynamic files. Does not work on directories, multilevel directories or subdirectories, or dictionaries.

ANALYZE.FILE Displays statistics, including current and minimum modulo, hash type, block size, split/merge type, split load, merge load, and record distribution for a dynamic file. Works on dynamic and sequentially hashed files and dynamic hashed multilevel subfiles only.

CONFIGURE.FILE Changes split/merge type, split load, merge load, part table, or minimum modulo for a dynamic file. Works on dynamic hashed files and dynamic hashed multilevel subfiles only.

REBUILD.FILE Reconstructs a dynamic file using current settings for split load, merge load, and minimum modulo. Used after CONFIGURE.FILE. Works on dynamic hashed files and dynamic hashed multilevel subfiles only.

CHECKOVER Checks UniData hashed files for level 2 overflow. Works on all UniData hashed files and subfiles. Used to check all files in a UniData account directory. ECL File Commands (continued) See the UniData Commands Reference for detailed information about the syntax of file-handling commands.

File-Handling Commands 12-29

The next table describes UniData system-level file-handling commands.

Command Description

auditor Checks all dynamic files in a UniData account to report inconsistencies between symbolic links, part file locations, and file prefix tables.

checkover Checks UniData hashed files for level 2 overflow. Works on all UniData hashed files and subfiles. Checks all files in a UniData account directory. You can execute the system-level version with UniData shut down or with UniData running.

dumpgroup Unloads the contents of a damaged group in a hashed file; you can execute with UniData shut down or with UniData running.

fixfile Repairs damaged groups in a file; you can execute with UniData shut down or with UniData running.

fixgroup Repairs a damaged group; you can execute with UniData shut down or with UniData running.

fixtbl Resolves inconsistencies between symbolic links and file prefix tables for all dynamic files in a UniData account. You must execute with UniData shut down.

guide Identifies damaged hashed files or dictionary files. You cannot execute if UniData is shut down.

guide_ndx Identifies corruption in an index file.

memresize Changes the modulo, block size, or hash type of a UniData hashed file. Works on static or dynamic hashed files and multilevel subfiles. Does not work on sequentially hashed files, directories, multilevel directories or subdirectories, or dictionaries. This command uses shared memory and disk storage, rather than disk storage alone, as working storage. Although you execute this command from a UNIX prompt, you cannot execute it if UniData is shut down. memresize also converts static files to dynamic files, dynamic files to static files, and changes the split/merge type and part table for dynamic files.

mvpart Moves a part file from one location to another, keeping links, location, and file prefix tables consistent. You must execute with UniData shut down.

shfbuild Converts a static or dynamic file to a sequentially hashed file. UniData System-Level File Commands

12-30 Administering UniData on UNIX File Corruption

File corruption is damage to the structure of a file. UniData file management tools diagnose and repair problems that occur if invalid, unreadable, or inconsistent infor- mation is written to file or group headers. Such invalid information can result in UniData being unable to access part or all of the data in the file. UniData provides a series of utilities that allow you to detect and repair damaged files.

Note: UniData file tools do not detect or repair invalid or inconsistent data in files. Detecting data inconsistencies should take place at the application level.

What Causes File Corruption?

File corruption can result from a variety of causes:

„ Hardware failures, including CPU crashes, media or memory failure, controller failures, bad spots on a disk. „ Interrupting a write in progress; for example, killing a UniData process using a UNIX kill -9 command. „ Incomplete writes; for example, a disk runs out of space before a write is complete. This is a rare condition, since UniData prompts the user to reclaim space when this occurs. Note: Overflowed files are more likely to become corrupted, since multiple I/O operations can be required to accomplish a single read or write to an overflowed file. An interrupted write can result in a condition where a primary data block and corresponding overflow blocks are out of synch. The increased chance of corruption is particularly significant for files in level 2 overflow.

Preventing File Corruption

You can reduce the possibility of file corruption by sizing your files to minimize overflow. Level 1 overflow can leave incomplete records in a file, although all the IDs are available. Level 2 overflow can cause more severe data problems because IDs and data can be lost. IBM strongly recommends that you monitor and minimize level 2 overflow. Using dynamic files versus static files minimizes level 2 overflow, which provides some protection. However, using dynamic files greatly increases level 1 overflow, making the risk of file corruption greater than that for static files.

File Corruption 12-31

Certain UniData commands carry a direct risk of file corruption, as shown in the following table.

Command Risk Factor

deleteuser This UniData command first tries to execute a UNIX kill -15. If the kill -15 is unsuccessful after 5 seconds, deleteuser executes a UNIX kill -9 command.

stopud -f This syntax of the stopud command does not allow writes to complete before logging users off. UniData Commands That Can Corrupt a File

There are other operations that can corrupt UniData files. The following list contains some examples:

„ Using UNIX file manipulation commands (for example, rm, mv, cp, and ln) on UniData hashed files while UniData is running can damage files. You should always shut UniData down before performing any UNIX-level manipulations on UniData files. „ Attempting to view/edit a UniData file with a UNIX text, octal, or binary editor can damage the file whether or not UniData is running. In many cases, the file damage is irreversible. „ Backing up and restoring a UniData file with a UNIX command (tar, dd, cpio) while users are accessing the file during backup may damage the restored file. Any UniData file can be damaged in this way, but the risk is particularly great for dynamic files. Note: The file being backed up is not damaged. Danger is only to the file being restored. „ Using system-level maintenance tools (for example, online file checking, which is supported under some UNIX versions) can damage a file, although this is not a common cause of corruption.

12-32 Administering UniData on UNIX UniData Detection Tools

UniData supplies the following tool for detecting damaged files:

„ guide — Use the guide command to detect file damage when UniData is running. guide

Syntax:

guide [file1, file2,...][-options] Note: You may supply guide with the name of a single UniData file or a series of file names separated by commas or spaces. If you do not specify any files, guide processes all files in the current UniData account directory.

Description guide is a system-level utility that provides detailed statistics and suggestions for optimizing file size and ensuring data integrity.

Note: If you do not want the guide utility to report orphan blocks, set the value of the SUPPRESS_ORPHAN_BLOCK_ERROR to a positive integer.

UniData Detection Tools 12-33

Options

The following table lists the options available for the guide command.

Option Description

-a |-A [filename] Controls whether UniData includes management advice in the -na | -NA output. The default filename for the advice information is GUIDE_ADVICE.LIS. You cannot combine the -a and -o (no advice) options, because UniData assumes the -a option when the -o (output) option is present. You may use the -na option in combination with the -o option.

-b | -B [filename] Controls whether UniData produces a file containing a brief -nb | -NB summary of statistical information. The default file name is GUIDE_BRIEF.LIS. (no brief statistics — this is the default)

-d1 | -D1 Includes minimum statistics about the file. Does not work with the -ns option.

-d2 | -D2 Includes statistical information helpful in estimating correct file sizing. This is the default. Does not work with the -ns option.

-d3 | -D3 Includes all information reported with -d2, plus additional infor- mation about distribution of data sizes. Does not work with the -ns option.

-e | -E [filename] Controls whether guide produces a report of structural errors in -ne | -NE the selected files. The default error list file name is GUIDE_ERRORS.LIS. The -e option is assumed when the -o (no errors) (output) option is present, and may not be specified at that time. You may, however, use the -ne option in combination with the - o option.

-f | -F [filename] Specifies the file that should receive a list of damaged groups. The default file name, if none is specified, is GUIDE_FIXUP.DAT. UniData creates this file only if it detects errors.

-ha | -HA Evaluates all hash algorithms (default). Note: the -h option has no effect if specified for a dynamic file.

-h0 | -H0 Evaluates algorithm 0. Note: the -h option has no effect if specified for a dynamic file. guide Command Options

12-34 Administering UniData on UNIX Option Description

-h1 | -H1 Evaluates algorithm 1. Note: the -h option has no effect if specified for a dynamic file.

-i | -I [filename] Directs the guide utility to evaluate all of the files in the file named filename. GUIDE_INPUT.DAT is the default. The file should be composed of one file name per line. UniData treats blank lines and lines that begin with an exclamation point as comments.

-l | -L [count] If you specify the -d3 option, the guide utility displays the keys for the largest records. The key appears in quotes and, if truncated, is followed by an asterisk (*). The -l option controls the number of records that display. The default value is three. Specifying a large number of records results in a significantly slower analysis.

-m | -M Directs the utility to evaluate the effects of a different modulo new_modulo upon the specified file. You must use this option in conjunction with the -h (hash test) option. This option has no effect when specified for a dynamic file.

-o | -O [filename] Controls whether output is combined or directed to separate files. If -o is specified, UniData directs all output to the file specified by filename. If you do not specify a file name, UniData directs the output from guide to “standard out” (usually, your terminal).

-Z num_child_processes Defines the number of concurrent processes to use when analyzing the file. The default is 4. If the file guide is analyzing has less than 100 groups, guide only uses one process.

-p | -P page_length Controls the display of guide output when you specify the -o -np | -NP option and directs output to the terminal. Specify -np to scroll the output past with no pause. Specify -p page-length to pause after (no pagination) displaying each page and prompt with the following message: “Press RETURN to continue...” The following responses are accepted at the prompt: • to display the next page. • “N” to continue with no pauses. • “Q” to quit the application. page_length is the number of lines per page in the screen display. The default value is 24. guide Command Options (continued)

UniData Detection Tools 12-35

Option Description

-r | -R [filename] Specifies whether to produce a reporting file. The filename must be the UNIX file specification of a UniData database file, previ- ously created by the CREATE.FILE command. Use this option to generate file statistics reports using UniQuery. Copy a dictionary for the report file from udthome/sys/D_UDT_GUIDE.

-s | -S count If you specify the -d3 option, the guide utility displays the keys for the smallest records. UniData displays the key in quotes. If the key is truncated, it is followed by an asterisk (*). The “-s count” option controls the number of records to appear in sorted order. The default value is three. Large values result in a signif- icantly slower analysis.

-s | -S [filename] Controls whether UniData includes statistical information about -ns | -NS the file in the output file. If you do not specify a filename, UniData uses GUIDE_STATS.LIS. (UniData assumes the -s (no statistics) (statistics) option when the -o (output) option is present, and may not be specified at that time.) You may use the -ne (no errors) option in combination with the -o option. guide Command Options (continued)

guide Output

The guide utility can create five output files. The following table lists these files. You may change the default names.

File Description

GUIDE_ADVICE.LIS Displays management advice about files that are poorly sized or corrupted.

GUIDE_ERRORS.LIS Lists structural errors detected in the files specified.

GUIDE_STATS.LIS Lists statistical information about the files specified.

GUIDE_BRIEF.LIS Displays summary information about selected files, including record counts, total size, used size and modulo.

GUIDE_FIXUP.DAT Produces a list of damaged groups that can be used as input for fixfile. You can also use this list to input file names/group numbers for dumpgroup/fixgroup. guide Output Files

12-36 Administering UniData on UNIX If you do not specify options, UniData selects the default options: -a, -e, -f, and -s, and places the results in the default files.

The guide utility checks for existing output files. If there are existing output files, guide appends a six-digit timestamp to the end of the existing file before it creates the current output file. The most current output files will not have this time stamp. UniData does not overwrite output files generated in a previous analysis. As a result, you may accumulate a large number of files that will need to be purged periodically. guide Report File

You can use the -r option of guide to create a UniData file containing statistical infor- mation about your database. To use the option, complete the following steps:

1. Create a UniData file in the account where are running guide. 2. Copy the records from udthome/sys/D_UDT_GUIDE to the dictionary of the file you created in step 1. 3. Execute guide -r filename where filename is the UniData file you created in step 1.

The guide utility creates statistical information in filename about the evaluated files. The records contain 62 attributes and are keyed by VOC entry name. You can use UniQuery or ECL commands, or write UniBasic code, to analyze the data and produce reports.

UniData Detection Tools 12-37

guide Example

The following example shows output from guide executed against a directory that contains a damaged file:

# guide -na -ns # pg GUIDE_ERRORS.LIS

TEST.FILE File Integrity: Group 1, block 2, record number 0 = "10053" invalid offset 2047 or length 110 Primary group 1, block 2 has 1 offset(s) less than the offset of the group header Group 1, block 2 bytes left 790 is wrong. Group 1, block 1 is pointed to by multiple links Group 1, block 1 has incorrect group number 0

Files processed: 183 Errors encountered: 5 # pg GUIDE_FIXUP.DAT TEST.FILE 1 Notice that group 1 of TEST.FILE is damaged.

Note: guide works only if UniData is running. Although guide works against recoverable files, guide itself is not recoverable. You must reapply file updates and fixes performed by guide during recovery from a system or media failure.

guide_ndx

Syntax:

guide_ndx{-x | -X} {1 | 2 | 3}, {index_names, ... | ALL} [-t template | -T template] filename

Description

As with other UniData file types, an index file could become corrupt due to hardware failures, the interruption of a write to the index file, or an incomplete write. The guide_ndx utility checks for physical and logical corruption of an index file.

If an index file is corrupt, UniData displays a runtime error when a UniData process tries to access the index. If the index file is associated with a recoverable file, a message is written to the sm.log.

12-38 Administering UniData on UNIX The guide_ndx command creates two files, the GUIDE_XERROR.LIS and the GUIDE_STATS.LIS. GUIDE_ERROR.LIS lists any corruption found in the index file, and GUIDE_STATS.LIS list statistics about the index. If you have a corrupt index, you must rebuild it using the CREATE.INDEX and BUILD.INDEX commands. For more information and creating and building indexes, see Using UniData.

Note: IBM recommends deleting the index with the DELETE.INDEX ALL command. Using the ALL option deletes all alternate key indexes and the index file itself.

UniData Detection Tools 12-39

UniData Recovery Tools

UniData includes the following commands to recover corrupted hashed files:

„ dumpgroup — Use this command to unload complete and valid records from a damaged group, separating valid records from damaged records. „ fixgroup — Use this command to clear records from a damaged group, and to reload and rebuild the group with the valid records unloaded by dumpgroup. „ fixfile — Use fixfile in conjunction with guide. guide provides a FIXUP.DAT file that lists corrupt files and groups. fixfile uses FIXUP.DAT to unload, clear, and rebuild the damaged groups. Note: Although dumpgroup, fixgroup, and fixfile work against UniData recoverable files, their actions are not recoverable. If you are recovering from a system crash or media failure, any file repairs that took place since your last backup are not included in logs or archives. You should check your files after recovery (using guide) and perform any needed repairs before users access them.

dumpgroup

Syntax:

dumpgroup filename group.no [-doutputfile] [-p]

Description

The system-level dumpgroup command unloads readable records from a group you specify in a UniData file. If the file was corrupted, dumpgroup unloads complete, valid records, leaving behind any information it cannot read.

12-40 Administering UniData on UNIX Parameters

The following table describes each parameter of the dumpgroup syntax.

Parameter Description

filename Specifies the name of the file containing groups to be dumped.

group.no Specifies the number of the group to be dumped. The output from either guide identifies groups that are damaged. Use this information to select groups to process.

[-doutputfile] Specifies the name of a file that contains the readable records from the dumped group, in an uneditable form. If you do not specify the -d parameter with an outputfile, the output prints to screen. To load outputfile back into the file, use fixgroup. Note: No space is allowed between -d and outputfile.

[-p] Converts nonprinting field markers to printable characters in editable output file. This option is only valid if -d is used. dumpgroup Parameters

If you execute dumpgroup without specifying an output file, the output simply displays on the screen. You will not be able to use that output to verify records or repair the damaged group. If you do specify an output file, UniData places the records in uneditable form, suitable for reloading, into the output file. UniData also creates a UNIX directory within your current account for each dumped group. The directory is named FILE_GROUP, where FILE and GROUP are the file name and group number you specify on the command line. This directory contains an ASCII file for each record, so that you can check them for consistency before reloading the damaged file.

Warning: When you use the -d option, make sure you name your output file with a name that does not already exist in your account name. If you specify a duplicate name, the preexisting file may be overwritten. fixgroup

Syntax:

fixgroup filename group.no [-iinputfile] [-k]

The system level fixgroup command reloads a hashed file group based on a file created by the dumpgroup command.

UniData Recovery Tools 12-41

The following table describes each parameter of the syntax.

Parameter Description

filename Specifies the name of the file to repair.

group.no Indicates the identifying number of the damaged group.

[-iinputfile] Specifies the name of the file created by dumpgroup. If you do not specify an input file argument, fixgroup simply clears the specified group, without reloading it. Note: No space is allowed between -i and inputfile.

[-k] Allows fixgroup to reload the damaged records from inputfile without zeroing the group first. This option may be useful if the group was updated since dumpgroup was executed. However, for best results, do not allow access to a file while it is being repaired, and clear the damaged groups rather than using the -k option. fixgroup Parameters

Warning: If you execute fixgroup without an input file argument, UniData clears the damaged group. Be sure that you save the readable records with dumpgroup before clearing the group. If you clear the damaged group and you have not saved the readable records, the data in that group is lost. The syntax for clearing a group without reloading it is: fixgroup filename group.no

UniData displays a warning message before clearing the group, as shown in the following example:

%fixgroup INVENTORY 5 Fixgroup INVENTORY 5 will make group 5 empty, do you wish to do it ? [y/n]

fixfile

Syntax:

fixfile {-t | {-dfilename | -k | -p | -f}} [-mfilename] [-wdirectory] [-ifilename | filename group.no]

12-42 Administering UniData on UNIX The system-level fixfile command repairs damaged groups in UniData files by clearing the group, and optionally restoring readable records to the group. Use fixfile in conjunction with the guide utility. Do not let users access files while fixfile is running because you could lose records.

The following table describes each parameter of the fixfile syntax.

Parameter Description

{-t} Directs all output to the terminal only. Each readable record is described in a new line that includes the record key and the record length. All attributes in the record appear on separate lines, each line indented by two spaces. Special and nonprintable characters are translated as follows: Attribute Mark — New line Value Mark — “ } ” Subvalue Mark — “ | ” Text Mark — “ { ” Non-printables — “ . ” The -t and -d parameters are mutually exclusive and cannot be used together.

{-dfilename} A file specification is required. For static files, the readable records are dumped to this file in uneditable format. For dynamic files, this file is created, but the actual records are dumped to a file in /tmp. With the -d option, UniData also writes the records, in readable format, to a directory in your current UniData account. This directory contains an ASCII file for each readable record in the group. The records are in a format suitable for editing. To repair any file, you need both the -d and -f options.

{-k} If you specify the -k option with the -d option, the damaged groups are not cleared. This has the effect of dumping the readable records for examination, but leaving the file corrupt. If you specify the -d and -f option along with the -k option, UniData repairs the file and returns the readable records to the group. Any unreadable records that were not dumped remain in the file as well. fixfile Parameters

UniData Recovery Tools 12-43

Parameter Description

{-p} If you specify the -p option with the -d option, all nonprinting characters and characters with special meaning to UniData are trans- lated. This translation applies to the editable ASCII files created by the -d option. If you do not specify the -p option, only attribute marks are translated.

{-f} If you specify the -f option with the -d option, UniData clears the damaged group and restores the records that were dumped back into the group, creating a fixed file with all readable data restored. You must specify the -d option with the -f option.

[-mfilename] UniData writes all error messages and statistics to the file you specify, instead of the terminal.

[-wdirectory] UniData creates the work files that are generated in the directory you specify.

{-ifilename} UniData uses this file as the source of the names of damaged files and groups to be repaired. If you do not use this option or the {filename group.no} option, UniData uses the GUIDE_FIXUP.DAT file under the current directory. This option is mutually exclusive with {filename group.no}.

{filename group.no} The file name and group number that contains the corruption. If you do not use this option or the {-ifilename} option, UniData uses the GUIDE_FIXUP.DAT file under the current directory. This option is mutually exclusive with the {-ifilename} option. fixfile Parameters (continued)

How fixfile Works with Static Files

When you run fixfile with the -t option on a static file, UniData displays the readable records from the file and group to the terminal. The group is not cleared or repaired. You can supply the names of damaged files and groups from the command line or from an input file. The default input file is GUIDE_FIXUP.DAT.

When you run fixfile with -dfilename on a static file, UniData creates:

12-44 Administering UniData on UNIX „ A UNIX directory or directories for the files and groups being repaired. If only one group in a file is damaged, the directory is named FILE_GROUP where FILE is the damaged file (from GUIDE_FIXUP.DAT) and GROUP is the damaged group. If several groups in a file are damaged, UniData creates a directory named FILE_dir. „ Each FILE_GROUP directory contains an ASCII file for every readable record in the damaged group. Each file name is the key for the corresponding UniData record. These records are in a format suitable for editing. „ Each FILE_dir contains a subdirectory for each damaged group in FILE. The name of each subdirectory is the group number of the damaged group. Each subdirectory contains an ASCII file for every readable record in the damaged group. Each file name is the key for the corresponding UniData record. These records are in a format suitable for editing. „ A file, with the name you specify on the command line, that contains the records fixfile could read, in uneditable format. UniData uses this file to reload the records into the damaged groups after the groups are cleared. Note: If you specify the -p option, fixfile translates nonprinting characters in the records when it creates the “editable” files. Otherwise, only attribute marks are translated to new lines.

When you run fixfile with the -d and -f options on a static file, UniData reloads the records into the damaged groups, taking them from the file you specify on the command line. Unless you specify the -k option, fixfile clears the groups, removing all contents, before reloading the data. If you specify the -k option, UniData adds the records back, but does not clear any data from the group.

Note: It is possible to run fixfile in two steps, one to dump the records for review and the second to repair the file. To dump the records only, run fixfile with the -d option, but without the -f option. Unless you specify the -k option, running fixfile with the -dfilename deletes the readable data from the specified groups when it creates output. To repair the file, run fixfile with both -d and -f options.

How fixfile Works with Dynamic Files

When you execute fixfile with the -d option on a dynamic file, UniData creates the following:

UniData Recovery Tools 12-45

„ A UNIX directory for each file-group combination being repaired. The directories are named FILE_GROUP, where FILE is a damaged file (from GUIDE_FIXUP.DAT) and GROUP is a damaged group. If several groups in a file are damaged, UniData creates a directory for each damaged group. „ Each FILE_GROUP directory contains an ASCII file for every readable record in the damaged group. Each record name is the key for the corre- sponding UniData record. These records are in a format suitable for editing. „ A file containing the records fixfile could read, in uneditable format suitable for reloading into the group after it has been cleared. This file is located in /tmp (or in the directory identified by the TMP environment variable) and is named ud_dp_pid, where pid is the UNIX process id of the process that executed fixfile. Note: If you specify the -p option, fixfile translates nonprinting characters in the records when it creates the editable files. Otherwise, UniData only translates attribute marks to new lines.

When you run fixfile with the -d and -f options on a dynamic file, UniData reads the file you specify with the -d option on the command line, and also reads the uneditable file of dumped records. UniData then reloads the records from that file into the damaged groups. Unless you specify the -k option, fixfile clears the groups, removing all contents, before reloading the data. Otherwise, UniData adds the records back, but does not clear any data from the group.

Note: You can run fixfile in two steps, one to dump the records for review and the second to repair the file. To dump the records for review, run fixfile with the -d option, but without the -f option. (You do not need to use -k for dynamic files. For dynamic files, running fixfile with -dfilename and not -f does not delete the readable data from the groups you specify when it creates output.) To repair the file, run fixfile with both the -d and -f options. If you specify the same filename with -d in both the review and repair steps, UniData will prompt whether or not to clear the damaged groups.

12-46 Administering UniData on UNIX Detection and Repair Examples

The following example shows a file called TEST.FILE being repaired using guide and fixfile:

# guide TEST.FILE -na -ns # more GUIDE_ERRORS.LIS TEST.FILE File Integrity: Group 4, block 5 bytes left 842 is wrong. Group 4, block 5, record number 0 = "10055" record length of 58 is wrong. Group 4, block 5 bytes used 58 and bytes left 842 are inconsistent

Files processed: 1 Errors encountered: 3

# more GUIDE_FIXUP.DAT TEST.FILE 4 # # fixfile -dhold -f 1:grpno error in U_blkread for file '/users/claireg/TEST.FILE', key '', number=3 1:U_blkread error in U_catch_tuple for file '/users/claireg/TEST.FILE', key '10055', number=3 the 0th record may be damaged,key='10055',length=0. **** Record Key='10055', Record length=1 1 record dumped for group 4 of /users/claireg/TEST.FILE. 1 block (including the group header) in group 4 of /users/claireg/TEST.FILE made empty. 1 record written to group 4 of file /users/claireg/TEST.FILE. #

Detection and Repair Examples 12-47

How to Use guide

Complete the following steps to effectively using the guide utility

1. Monitor File Integrity with guide

You should execute guide against your database at regular intervals, as well as when you have had a system crash. You can set up shell scripts to run guide at specified intervals on specified lists of files, or you can simply execute the guide command in each UniData account. You can execute guide against nonrecoverable static hashed files at any time, and schedule guide to run on dynamic files and recoverable files at a time when the system is idle or only lightly loaded.

2. Check Error Output (GUIDE_ERRORS.LIS)

Use the following information to determine what action to take depending on the error output.

No Errors

If there are no errors, proceed to step 5.

Partially Allocated Block Messages

Partially allocated block messages are not false error reports; they indicate an out-of- synch condition in a dynamic file, but they do not mean that the file must be fixed immediately. Users can continue to access the file; this will not cause damage. Complete the repair at a convenient time using the procedure in step 3.

Partially allocated block messages look like:

free block xx partially allocated, xxx bytes remaining Block xx of primary file not a member of any group

12-48 Administering UniData on UNIX Other guide Error Messages guide produces many messages besides the one discussed above. If you see error messages pertaining only to static files, or if you see other error messages pertaining to dynamic files, proceed to step 3.

3. Repair Damaged Files

Complete the following steps:

1. Back up the damaged file(s)—If time and space permit, IBM strongly recommends you back up (or simply make a copy of) the damaged files before proceeding. In the event of a system failure during the repair process, you will be able to restore from the backup copies and repeat the procedures, rather than attempting to recover a partially-completed repair. DO NOT ALLOW USERS TO ACCESS YOUR FILES WHILE YOU ARE BACKING THEM UP! 2. Repair the damaged groups—Once you run guide, you can execute either fixfile or dumpgroup/fixgroup to repair the damaged groups. In either case, the process overview is: dump the readable records from a damaged group, clear the group, and then reload all readable records back into the group. Tip: IBM recommends you not use the -k option with fixfile or with fixgroup. The -k option lets you reload the readable records without clearing the group. However, you may encounter additional errors if you do not clear the group. Use fixfile or fixgroup without -k; this procedure automatically clears the group before reloading the readable records. Warning: Be sure no users are accessing your files before repairing dam- aged groups. The dumpgroup command does not obtain exclusive access, and fixfile/fixgroup only lock the file when the records are being written back to a group. Concurrent access to the file could make corruption worse. 3. Verify the repair—Rerun guide after the repair is complete to verify that the errors are fixed. If they are not, or if additional groups are damaged, repeat the previous step.

4. Back Up the Repaired Files

Back up any files you have just completed repairing.

How to Use guide 12-49

5. Continue Processing

If you shut UniData down to repair files, start it again before allowing users to log in.

12-50 Administering UniData on UNIX Error Messages

This section lists error messages you may see, and provides information about the meaning of them. UniData generates these messages from the guide command.

File Access Messages

File access messages are similar to the following example:

can not obtain an exclusive lock on recoverable file 'filename' error opening 'filename' part files Unable to lock dynamic file 'filename' All of these messages indicate that guide did not process the file because it was unable to obtain an exclusive lock on the file.

Note: These messages display only at the terminal. They are not logged in any file.

Block Usage Messages

Block usage messages are similar to the following example:

Group xx, block xx is pointed at by multiple links The xxth block, grpoff=xx is pointed at by multiple links In file, 'filename', the xxth group, grpoff=xx, have hit by other links. These indicate that a block is found to be referenced by more than one link, which should not occur. These messages indicate damage.

The xxth block of file (filename) has no link. the xxth block has no link A block has been found that is not in the global free chain and is not used by any group. This error can be reported when guide encounters a corrupt block, and is therefore unable to check blocks linked through the corrupted one.

Error Messages 12-51

Group Header Messages

Group header messages are similar to the following example:

Group xx, block xx, invalid flags xx, should be an overflow header block Group xx, block xx, invalid flags xx, should be a normal header block These messages indicate damage.

Header Key Messages

Header key messages are similar to the following example:

Group xx, block xx key area is corrupted, key size xx, number of keys xx Group xx, block xx key area is corrupted, incorrect key size xx Group xx, block xx key area is corrupted, data position xx, key size xx number of keys xx Group xx, block xx key area is corrupted, total key length xx, key size xx, number of keys xx keys xx in group header yy error keysize=xx,keys=xx Bad keyarea: keysize=xx keys=xx datapos =xx These errors indicate that key area size or number of keys have been corrupted in a group header.

Other Header Messages

There are a number of types of other header messages, as shown in the following example:

Group xx, block xx has incorrect group number

The group number recorded in the block header does not match the group being checked.

Group xx, block xx has invalid offset next over (xx) error in group head

12-52 Administering UniData on UNIX The offset (link to next disk block) is not a multiple of the block size, or, for a dynamic file, the offset does not indicate an overflow file offset.

Group xx, block xx data area is corrupted, incorrect data position xx wrong datapos=xx A data position in a group header is damaged.

Group xx, block xx, y number xx = y invalid offset xx or length Group xx, block xx, y number xx = y offset occurs in wrong order Group xx, block xx, y number xx = y offset error Primary group xx, block xx has xx offset(s) less than the offset of the group header the bad length of the xxth key = xx The xxth key='yy', keylen=yy, hashed to xx this group =xx, modulo=xx totalkeylen=xx,keysize=xx,keys=xx key area corrupted the xxth key[] offset(yy) has wrong order The xxth[] off_lens error,offset=yy,len=xx This isn’t an overflow group, but there is xx offset:are xx offsets less than the offset of group header Each group header has an area that stores offset-record length pairs, which are sorted by offset. Each offset equals the offset of the previous record plus its length. If these conditions are not met, corruption results, and some or all of the previous messages display.

Group xx, block xx bytes used xx and bytes left xx are inconsistent Group xx, block xx has wrong number of bytes remaining group header byteleft (xx) wrong *key[xx] (key) record length xx wrong file no in xxth key[] offset() not right byteleft (xx) in blk(yy) wrong byteused (xx) + byteleft (xx) in block (yy) error The counter that records the number of bytes available in a block does not agree with the actual number of bytes in the block.

Group xx, block xx, yy number xx=yy record length error xx &key[xx] (key) record length (xx) wrong The actual record length does not match the offset-length pair of the record.

Error Messages 12-53

Free Block Messages

Free block messages are similar to the following example:

Group xx, free block count in group header (y) error, should be xx The actual count of free blocks for a group does not match the counter in the group header.

Group xx, free block xx partially allocated, xx bytes remaining Group xx, free block xx has invalid flags xx A block is linked to the free block list but not correctly initialized. Blocks linked to the free list should have no bytes used and should be “normal” blocks (not header blocks).

Long Record Messages

“Long” records are records which span more than one block. Messages about problems with long records look similar to the following example:

Group xx, block xx, y number xx =y invalid flags xx Group xx, block xx, y number xx =y longrecord, nextoff(zz) error Group xx, block xx, y number xx =y longrec: file no in nextoff (zz) error Group xx, block xx, y number xx =y invalid next offset xx Group xx, block xx, y number xx =y record length error blockflag for normal block(yy) error (xx) &key[xx] () blockflag (xx) for longrec error longrecord, byteleft(xx) error long record, last block (yy) flag (xx) error. In UniData hashed files, a long record always starts from the beginning of a level one overflow block, which is flagged as “STARTLONG.” Each intermediate block is flagged as “MIDLONG,” and the last block is flagged as “ENDLONG.” If the length of a long record does not match header information, or if any flag in its data blocks is incorrect or the pointer in the chain gets broken, guide reports messages like the preceding ones.

12-54 Administering UniData on UNIX Chapter Managing UniData Locks 13

The Global Lock Manager ...... 13-3 How GLM Works ...... 13-3 Locking in UniBasic ...... 13-5 How Locks Work ...... 13-5 Locking Commands ...... 13-6 Resource Locks ...... 13-8 Listing Locks ...... 13-9 LIST.READU...... 13-9 Parameters...... 13-9 LIST.LOCKS ...... 13-11 LIST.QUEUE ...... 13-12 LIST.QUEUE Display ...... 13-13 Commands for Clearing Locks ...... 13-17 SUPERCLEAR.LOCKS Command ...... 13-17 SUPERRELEASE Command ...... 13-19 Procedure for Clearing Locks ...... 13-20

This chapter outlines file, record, and system resource locking within UniData, describes tools for listing locks and listing the contents of the wait queue, and describes procedures for releasing locks that remain set when a process exits UniData.

13-2 Administering UniData on UNIX The Global Lock Manager

The Global Lock Manager (GLM) is an internal software module that is linked into each udt process to manage logical record locks. Prior toUniData 5.1, logical records locks were managed by the lock control process (lcp) daemon.

How GLM Works

GLM manages local lock tables for each udt process and a shared global lock table in shared memory, which can be accessed by multiple udt processes. The lock tables are hashed tables containing linked lists, which contain lock nodes. When a udt process locks a record, it writes the file name, record ID, and lock mode to both the local lock table and the global lock table. When a udt process requests a lock, UniData first searches the local lock table for that udt to see if that process is holding the lock, then the global lock table to see if another udt process is holding the lock.

GLM udtconfig Parameters

There are four udtconfig parameters which control the size of the shared lock table and the memory UniData allocates for each memory request.

N_GLM_GLOBAL_BUCKET

This parameter defines the number of hash buckets systemwide, used to hold the lock names in shared memory. This setting directly affects performance. Normally, the default value of this parameter should not be changed. However, if you notice signif- icant degradation in performance, or your application intensively accesses specific files, you should increase this parameter. The value should be the closest prime number to NUSERS * 3.

N_GLM_SELF_BUCKET

This parameter determines the number of hash buckets for the local lock table, and is highly application-dependent. If the application requires a large number of locks in one transaction (more than 20), you should increase this setting from the default of 23. The setting should be the closest prime number to the maximum number of locks per transaction.

The Global Lock Manager 13-3

GLM_MEM_ALLOC

This parameter defines the number of lock nodes allocated for each memory request, and is highly application-dependent. If your application requires a large number of locks in one transaction, you should increase this value to the maximum number of locks per transaction * 2.

GLM_MEM_SEGSZ

This parameter specifies the segment size for each shared memory segment required for GLM. The maximum number of segments is 16. Large application environments require a larger size. The default value is 10.

GLM_MEM_SEGSZ must be greater than 4096 and less than SHM_MAX_SIZE. The formula for determining SHM_MAX_SIZE is NUSERS * maximum number of locks per transaction * 512. SHM_MAX_SIZE should be a multiple of 4096.

13-4 Administering UniData on UNIX Locking in UniBasic

A series of UniBasic commands allow you to set read-only and exclusive locks on UniData files and their contents.

How Locks Work

UniBasic locks are advisory rather than physical, so they inform other users of a file or record that the file or record is in use, rather than explicitly preventing access. You can set exclusive locks or shared (read-only) locks.

Exclusive Locks (U Type)

Exclusive (U) locks are respected by all lock-checking commands except those that write and delete. Exclusive locks are set by “U” commands, for instance READU, MATREADU, and RECORDLOCKU.

Shared Locks (L Type)

Shared, or read-only, locks can be shared by more than one user. These locks are set by “L” commands, for instance READL, MATREADL, RECORDLOCKL. A record locked with an L lock can be accessed for reading by another “L” command, but cannot be accessed by “U” commands.

Writing and Deleting

WRITE and DELETE commands execute regardless of lock status. WRITEU, WRITEVU, MATWRITEU, and DELETEU retain locks set by previous commands. To prevent multiple updates to records, you should precede all WRITE and DELETE commands by a lock-setting command like READU.

Locking in UniBasic 13-5

Locking Commands

The following table lists UniBasic commands for setting and releasing locks.

Command Description

FILELOCK Locks the data or dictionary portion of a UniData file against access by commands that check locks.

FILEUNLOCK Unlocks a file previously locked with the FILELOCK command.

MATREADL Assigns the values found in successive attributes of a record to corresponding elements of a matrix, and sets a read-only lock on the record.

MATREADU Assigns the values found in successive attributes of a record to corresponding elements of a matrix, and sets an exclusive lock on the record.

MATWRITE Writes successive elements of a matrix to the corresponding attributes of a record and releases locks previously set on the record.

MATWRITEU Writes successive elements of a matrix to the corresponding attributes of a record without releasing locks previously set on the record.

READBCKL Retrieves one record from a B+ tree based alternate key index and places a read-only lock on the record. Each subsequent READBCKU retrieves the previous record in the index.

READBCKU Retrieves one record from a B+ tree based alternate key index and places an exclusive lock on the record. Each subsequent READBCKU retrieves the previous record in the index.

READFWDL Retrieves one record from a B+ tree based alternate key index and places a read-only lock on the record. Each subsequent READBCKU retrieves the next record in the index.

READFWDU Retrieves one record from a B+ tree based alternate key index and places an exclusive lock on the record. Each subsequent READBCKU retrieves the next record in the index. UniBasic Commands for Locking Files and Records

13-6 Administering UniData on UNIX Command Description

READL Reads a specified record from a file, assigning the record contents to a dynamic array and setting a read-only lock on the record.

READU Reads a specified record from a file, assigning the record contents to a dynamic array and setting an exclusive lock on the record.

READVL Reads a specified attribute of a specified record, assigning the attribute value to a variable and setting a read-only lock on the record.

READVU Reads a specified attribute of a specified record, assigning the attribute value to a variable and setting an exclusive lock on the record.

RECORDLOCKL Sets a read-only lock on a specified record in a specified file.

RECORDLOCKU Sets a read-only lock on a specified record in a specified file.

RELEASE Releases record locks without updating records.

WRITE Writes an expression to a record, releasing locks previously set by READU, READL, READVU, and MATREADU.

WRITEU Writes an expression to a record without releasing any previous locks on the record.

WRITEV Writes an expression to an attribute of a record, releasing previous update locks.

WRITEVU Writes an expression to an attribute of a record without releasing previous locks on the record. UniBasic Commands for Locking Files and Records (continued)

Locking in UniBasic 13-7

Resource Locks

In both UniData and UniBasic, you can reserve a system resource by setting a semaphore lock on it.

Note: Certain device handling commands (T.ATT, T.DET, LINE.ATT, and LINE.DET) set semaphore locks.

The following table lists commands for explicitly reserving system resources from the ECL prompt.

Command Description

UNLOCK Releases system resources reserved by the LOCK command. (These resources are not automatically released when a program terminates.) This command is not needed to release file and record locks.

LOCK Reserves a system resource for exclusive use. (ECL and UniBasic) Locking System Resources Note: Although the LOCK and UNLOCK commands allow you to set and release semaphore locks, UniData does not (necessarily) use UNIX system-level semaphores to control access to system resources. The output from LIST.LOCKS and ipcstat may not appear to be consistent, but UniData is functioning correctly.

13-8 Administering UniData on UNIX Listing Locks

UniData offers three commands for listing record and file locks, semaphore locks on system resources, and processes waiting to get locks.

LIST.READU

The ECL LIST.READU command allows any user with access to the ECL prompt to display a list of file and record locks set on the system.

Syntax:

LIST.READU [user_number | ALL | FILENAME filename | USERNAME user_name] [DETAIL]

Parameters

The following table describes the parameters of the LIST.READU command.

Parameter Description

user_number Displays all locks held by the user number you specify.

ALL Displays all currently active locks.

FILENAME filename Displays all active locks associated with the file name you specify. If the file name does not reside in the current account, nothing is displayed.

USERNAME Displays all active locks associated with the user name you user_name specify.

DETAIL Displays detailed information. LIST.READU Parameters

Listing Locks 13-9

The following example illustrates the output from the LIST.READU command when you do not specify any options:

:LIST.READU UNO UNBR UID UNAME TTY FILENAME INBR DNBR RECORD_ID M TIME DATE 4 6739 0 root ttyp5 INVENTOR 24193 10738 11000 X 16:22:13 Aug 04 5 6758 1172 clair pts/3 INVENTOR 24193 10738 10060 X 16:21:53 Aug 04 : The next example illustrates the output from the LIST.READU command when you specify a user number. The user number can be found in the output from the LIST.QUEUE and LIST.READU commands under the UNBR column.

:LIST.READU 6739 UNO UNBR UID UNAME TTY FILENAME INBR DNBR RECORD_ID M TIME DATE 4 6739 0 root ttyp5 INVENTOR 24193 10738 11000 X 16:25:44 Aug 04 : The next example illustrates output from the LIST.READU command when you specify a user name:

:LIST.READU USERNAME claireg UNO UNBR UID UNAME TTY FILENAME INBR DNBR RECORD_ID M TIME DATE 5 6758 1172 clair pts/3 INVENTOR 24193 10738 11060 X 16:28:14 Aug 04 : The final example illustrates output from the LIST.READU command when you specify a file name:

:LIST.READU FILENAME INVENTORY UNO UNBR UID UNAME TTY FILENAME INBR DNBR RECORD_ID M TIME DATE 4 6739 0 root ttyp5 INVENTOR 24193 10738 11000 X 16:28:24 Aug 04 5 6758 1172 clair pts/3 INVENTOR 24193 10738 11060 X 16:28:14 Aug 04 :

13-10 Administering UniData on UNIX LIST.READU Display

The following table describes the output from the LIST.READU command.

Column Heading Description

UNO The sequential number UniData assigns to the udt process that set the lock.

UNBR The process ID of the user who set the lock.

UID The user ID of the user who set the lock.

UNAME The logon name of the user who set the lock.

TTY The terminal device of the user who set the lock.

FILENAME The file name in which the record is locked.

INBR The i-node of the locked file.

DNBR Used in conjunction with INBR to define the file at the operating system level.

RECORD_ID The record ID of the locked record.

M The type of lock. X indicates an exclusive lock. S indicates a shared lock.

TIME The time the lock was set.

DATE The date the lock was set. LIST.READU Display

LIST.LOCKS

Use the ECL LIST.LOCKS command to display semaphore-type locks that reserve system resources for exclusive use. These locks can be set individually with the LOCK command. They are also set by other ECL commands, including T.ATT.

Syntax:

LIST.LOCKS

The following screen shows the LIST.LOCKS command and its output:

Listing Locks 13-11

:LIST.LOCKS UNO UNBR UID UNAME TTY FILENAME INBR DNBF RECORD_ID M TIME DATE 1 15290 1172ump01 tyv1 semaphor -1 0 65 X 15:14:01 Jun 03 : Note: If you need to clear a semaphore lock that has been left set, you need to note the UNBR and the lock number for the lock. In the example, the lock number is 65 for the lock displayed.

LIST.QUEUE

The ECL LIST.QUEUE command displays a list of all processes waiting to get locks. If a process is waiting for a lock, LIST.QUEUE displays information about the holder of the lock and processes waiting for the lock. Locks are set by each udt process through the general lock manager (GLM) module. LIST.QUEUE displays the internal table managed by GLM.

Syntax:

LIST.QUEUE [USERNAME user_name | FILENAME filename | user_number][DETAIL]

Parameters

The following table describes the parameters of the LIST.QUEUE command.

Parameter Description

USERNAME Lists all locks the user is waiting for. user_name is the operating user_name system logon name.

FILENAME filename Lists all users waiting for locks for the file name you specify.

user_number Lists all locks for which the user_number is waiting. The user number can be found in the UNBR column of the LIST.READU and LIST.QUEUE output.

DETAIL Displays a detailed listing. LIST.QUEUE Parameters

The following example illustrates the output from the LIST.QUEUE command when you do not specify any parameters.

:LIST.QUEUE FILENAME RECORD_ID M OWNER UNBR UNO TTY TIME DATE

13-12 Administering UniData on UNIX INVENTORY 11060 X clair 6031 2 pts/2 11:05:44 Aug 04 ------FILENAME RECORD_ID M WAITING UNBR UNO TTY TIME DATE INVENTORY 11060 X clair 6130 4 ttyp1 11:05:54 Aug 04 INVENTORY 11060 X clair 6188 1 ttyp3 11:06:04 Aug 04 The next example illustrates the LIST.QUEUE output when you specify a user name:

:LIST.QUEUE USERNAME root FILENAME RECORD_ID M OWNER UNBR UNO TTY TIME DATE INVENTORY 11060 X clair 6031 2 pts/2 11:35:46 Aug 04

------FILENAME RECORD_ID M WAITING UNBR UNO TTY TIME DATE INVENTORY 11060 X root 6259 5 ttyp2 11:35:56 Aug 04 : The next example illustrates the LIST.QUEUE command output when you specify a file name:

:LIST.QUEUE FILENAME INVENTORY FILENAME RECORD_ID M OWNER UNBR UNO TTY TIME DATE INVENTORY 11060 X root 6259 5 ttyp2 11:38:16 Aug 04 ------FILENAME RECORD_ID M WAITING UNBR UNO TTY TIME DATE INVENTORY 11060 X clair 6188 1 ttyp3 11:38:36 Aug 04 INVENTORY 11060 X clair 6031 2 pts/2 11:38:46 Aug 04 : The final example shows the output from the LIST.QUEUE command when you specify a user number:

:LIST.QUEUE 6763 FILENAME RECORD_ID M OWNER UNBR UNO TTY TIME DATE INVENTORY 11060 X clair 6758 5 pts/3 14:16:26 Aug 04 ------FILENAME RECORD_ID M WAITING UNBR UNO TTY TIME DATE INVENTORY 11060 X clair 6763 6 ttyp1 14:16:46 Aug 04 :

LIST.QUEUE Display

The LIST.QUEUE display in the previous examples use the default display. Infor- mation about the owner of the lock is listed above the line. Information about processes waiting for the lock is listed below the line, sorted by the date and time the process requested the lock.

Listing Locks 13-13

The following table describes the LIST.QUEUE default display for the owner of the lock.

Column Heading Description

FILENAME The name of the file holding the lock.

RECORD_ID The record ID holding the lock.

M Type of lock held. X is an exclusive lock, S is a shared lock.

OWNER The user name of the owner of the lock.

UNBR The process group ID (pid) of the user who set the lock.

UNO The sequential number UniData assigns to the udt process for the owner of the lock.

TTY The terminal device of the user owning the lock.

TIME The time the lock was set.

DATE The date the lock was set. LIST.QUEUE Owner Display The next table describes the LIST.QUEUE display for the processes waiting for locks.

Column Heading Description

FILENAME The name of the file for which a lock is requested.

RECORD_ID The record ID of the record for which a lock is requested.

M The type of lock requested. X is an exclusive lock, S is a shared lock.

WAITING The user name of the process waiting for a lock.

UNBR The process ID (pid) of the user waiting for a lock.

UNO The sequential number UniData assigns to the udt process waiting for a lock. LIST.QUEUE Waiting Display

13-14 Administering UniData on UNIX Column Heading Description

TTY The terminal device of the user waiting for a lock.

TIME The time the lock was requested.

DATE The date the lock was requested. LIST.QUEUE Waiting Display (continued) The following example illustrates the LIST.QUEUE display when you specify the DETAIL option:

:LIST.QUEUE DETAIL FILENAME RECORD_ID M INBR DNBR OWNER UNBR UNO TTY TIME DATE INVENTORY 10060 X 241938 1073807361 clair 13798 3 pts/0 14:48:47 Nov 19 ------FILENAME RECORD_ID M INBR DNBR WAITING UNBR UNO TTY TIME DATE INVENTORY 10060 X 241938 1073807361 root 13763 1 ttyp2 14:48:57 Nov 19 The following table describes the owner information the LIST.QUEUE command displays when you specify the DETAIL option.

Column Heading Description

FILENAME The name of the file for which a lock is held.

RECORD_ID The record ID of the record for which a lock is held.

M The type of lock held. X is an exclusive lock, S is a shared lock.

INBR The i-node of the file holding the lock.

DNBR Used in conjunction with the INBR to define the file holding the lock at the operating system level.

OWNER The user name of the process holding the lock.

UNBR The process ID (pid) of the user holding a lock.

UNO The sequential number UniData assigns to the udt process holding a lock.

TTY The terminal device of the user holding a lock.

TIME The time the lock was set.

DATE The date the lock was set. LIST.QUEUE Detail Display

Listing Locks 13-15

The next table describes the output for the LIST.QUEUE command with the DETAIL option for processes waiting for locks.

Column Heading Description

FILENAME The name of the file for which a lock is requested.

RECORD_ID The record ID of the record for which a lock is requested.

M The type of lock held. X is an exclusive lock, S is a shared lock.

INBR The i-node of the file for which a lock is requested.

DNBR Used in conjunction with the INBR to define the file for which a lock is requested at the operating system level.

WAITING The user name of the process requesting a lock.

UNBR The process ID (pid) of the user requesting a lock.

UNO The sequential number UniData assigns to the udt process requesting a lock.

TTY The terminal device of the user requesting a lock.

TIME The time the lock was requested.

DATE The date the lock was requested. LIST.QUEUE Detail Display

13-16 Administering UniData on UNIX Commands for Clearing Locks

If you break out of a process that is running, if a process is killed, or if a system resource is not unlocked by a UniBasic program, locks can remain after they should have been released. If a lock remains set, other users experience difficulty accessing a record, file, or resource. As other processes attempt to access the locked item, message queue congestion can result if the process that set the lock is no longer logged on to the system. The typical manifestations of unneeded locks are:

„ Users cannot perform expected operations on a file or record. Over a lengthy period of time, users receive messages indicating that the file or record is locked. „ Performance suffers, either because the item that is locked is heavily used or because a message queue has become clogged due to the lock. „ Batch jobs attempting to access a locked item fail.

Specific symptoms depend on the type of lock and the frequency of usage of the locked item.

UniData includes two commands that allow an administrator with root access to release locks held by other users.

SUPERCLEAR.LOCKS Command

SUPERCLEAR.LOCKS enables you to release semaphore locks on system resources (such as tape drives).

Syntax:

SUPERCLEAR.LOCKS usrnbr [locknum]

The following table describes each parameter of the syntax.

Parameter Description

usrnbr The UNIX process ID (pid) that holds the lock. This number is UNBR in the LIST.LOCKS display.

[locknum] The number of the locked system resource. If you do not specify locknum the command clears all locks set by usrnbr. SUPERCLEAR.LOCKS Parameters

Commands for Clearing Locks 13-17

The following example shows the effects of SUPERCLEAR.LOCKS:

:LIST.LOCKS UNO UNBR UID UNAME TTY FILENAME INBR DNBF RECORD_ID M TIME DATE 1 15454 1172ump01 tyv1 semaphor -1 0 65 X 16:21:01 Jun 03 3 15692 1172ump01 tyv4 semaphor -1 0 66 X 16:27:01 Jun 03 : :SUPERCLEAR.LOCKS 15692 -1 :LIST.LOCKS UNO UNBR UID UNAME TTY FILENAME INBR DNBF RECORD_ID M TIME DATE 1 15454 1172ump01 tyv1 semaphor -1 0 65 X 16:21:01 Jun 03 :

13-18 Administering UniData on UNIX SUPERRELEASE Command

The SUPERRELEASE command enables you to release locks you have set on records.

Syntax:

SUPERRELEASE usrnbr [inbr devnum] [record.id]

The following table describes each parameter of the syntax:

Parameter Description

usrnbr The UNIX process ID (pid) that holds the lock. This number is UNBR in the LIST.LOCKS display.

[locknum] The number of the locked system resource. If you do not specify locknum the command clears all locks set by usrnbr. SUPERRELEASE Parameters

Note: If you execute SUPERRELEASE and specify only usrnbr, you release all record locks held by the process ID corresponding to usrnbr.

The following example shows the effect of SUPERRELEASE:

:LIST.READU UNO UNBR UID UNAME TTY FILENAME INBR DNBR RECORD_ID M TIME DATE 1 4178 1172 clair ttyp2 INVENTOR 24193 10738 10060 X 13:28:40 Sep 24 2 4180 1172 clair ttyp1 INVENTOR 24193 10738 10055 X 13:30:20 Sep 24 :SUPERRELEASE 4180 :LIST.READU UNO UNBR UID UNAME TTY FILENAME INBR DNBR RECORD_ID M TIME DATE 1 4178 1172 clair ttyp2 INVENTOR 24193 10738 10060 X 13:28:40 Sep 24

SUPERRELEASE Command 13-19

Procedure for Clearing Locks

Complete the following steps to identify and clear unneeded locks:

1. Check for Unneeded Locks

Use the UniData LIST.READU and LIST.LOCKS commands to display the locks currently set on the system. Use LIST.QUEUE to identify locks for which processes are waiting. Note locks that meet the following criteria:

„ They are set on files or records that users cannot currently access. „ They have been set for a long period of time (shown by the time and date on the list). „ They were set by users who are not currently on the system (shown by comparing the lists with the UniData LISTUSER command or the UNIX who or ps commands).

2. Note Information for Clearing

For record locks, note UNBR, INBR, DNBR, RECORD NO. For semaphore locks, note UNBR and lock number. To clear record locks, proceed to step 3. To clear semaphore locks, proceed to step 4.

3. Release Record Locks

Log on as root and use the UniData SUPERRELEASE command to clear record locks. If possible, specify only a UNBR to clear all the locks belonging to a process ID. If you have semaphore locks to clear, proceed to step 4. Otherwise, proceed to step 5.

Warning: Some situations that retain locks can also cause file corruption (for example, a udt process is inadvertently killed). Consider checking the file with guide to make certain it has not been corrupted.

13-20 Administering UniData on UNIX 4. Clear Semaphore Locks

Log on as root and clear semaphore locks with the SUPERCLEAR.LOCKS command.

5. Check for Message Queue Congestion

Some conditions that cause locks to remain set also cause message queue congestion when other processes attempt to access the locked item. Refer to Chapter 18, “Managing ipc Facilities,” for step-by-step instructions for identifying and clearing congested message queues.

Procedure for Clearing Locks 13-21 Chapter Managing UniData Users 14

Adding Users ...... 14-3 Every User Needs a Logon ID ...... 14-3 Create Logon IDs at the UNIX Level ...... 14-3 Assign Users to Groups...... 14-4 Monitoring User Processes ...... 14-5 UniData Commands...... 14-5 Removing User Processes ...... 14-7 Using TIMEOUT ...... 14-8 This chapter outlines considerations for adding users to your system and assigning users to groups, and describes how to use UniData and UNIX commands to view user processes for troubleshooting, and to remove user processes when needed.

14-2

Adding Users

Every User Needs a Logon ID

To access UniData on your system, you need a valid UNIX account, including a UNIX logon ID and password. In Pick® systems, where each logon account must be a Pick® account, shared logon IDs and passwords are common. In contrast, UniData allows multiple relationships between user logon IDs and UniData accounts. A user may have access to more than one UniData account, and many users can access a single UniData account using separate UNIX logon IDs. Therefore, IBM strongly recommends you set up a separate UNIX account for each user. IBM recommends separate logon IDs for the following reasons:

„ Since most operating systems impose limits on the number of processes that can be associated with one user, using separate logon IDs allows your system to run more processes at one time. „ It is easier to identify processes and locks belonging to an individual user, which facilitates troubleshooting. „ Using separate logon IDs allows you to define your users’ responsibilities for protecting their passwords and your application data.

Create Logon IDs at the UNIX Level

There are no UniData commands or procedures for setting up UNIX user accounts. Although the basic process for creating a UNIX account is similar across UNIX versions, different system configurations may use different utilities or third-party interfaces for the actual mechanics. Refer to your host operating system and vendor documentation to determine the correct procedure at your site.

Consider the following factors when setting up UNIX logon IDs:

„ Each user should have a defined home directory. UNIX home directories do not need to be unique; many users can share one if this seems desirable on your system. „ Home directories do not need to be UniData accounts, although they may be. „ Saved ECL command stacks are stored in the user’s home directory.

14-3 Administering UniData on UNIX Note: IBM recommends that if some or all of your users require access to the UNIX shell prompt, or if they will be doing development or maintenance programming, their home directory should not be a UniData account that contains production data or programs. Warning: In some configurations, proprietary utilities are in use to automate many of the steps for adding or deleting a user. Make sure that your utilities are clearly documented and understood. If you have a utility that deletes a user’s home directory when that user’s account is removed, for example, you could suffer data loss if you use shared home directories.

Assign Users to Groups

UNIX allows you to define each user as a member of one or more UNIX system groups. File permissions allow differentiation of access between members of a group owner and users who are not members of that group. Refer to Chapter 2, “UniData and UNIX Security,” and Chapter 11, “Managing UniData Security,” for more infor- mation about groups.

Note: The UNIX commands finger and groups display information about users. Refer to your host operating system documentation for information about these commands.

Adding Users 14-4

Monitoring User Processes

For troubleshooting purposes, it is often necessary to identify and monitor processes owned by a particular UniData user.

UniData Commands

UniData includes a group of commands to display a list of current UniData sessions, to display a list of users currently logged on to the system, and to display detailed information about process activity for a specific user, or for all users. The following table summarizes these UniData commands.

UniData Command Description

WHO ECL command; similar to the UNIX who command; displays a list of users currently logged on to the system, including users who are not UniData users.

LISTUSER ECL command; displays a list of current UniData sessions.

listuser UniData system-level command; enter at a UNIX prompt; displays the same information as the ECL LISTUSER command.

udstat UniData system-level command; displays detailed activity statistics about a single UniData process or about all UniData processes.

MYSELF ECL command; similar to the UNIX whoami command. UniData Commands for Monitoring User Processes

Note: You do not need to be logged on as root to execute these commands.

The following screen shows the system response to the WHO, LISTUSER, and listuser commands.

:WHO claireg ttyp1 Sep 24 10:04 claireg ttyp2 Sep 24 11:55 claireg pts/0 Sep 24 14:34

:LISTUSER

Licensed/Effective # of Users Udt Sql Total

32 / 32 2 0 2

14-5 Administering UniData on UNIX UDTNO USRNBR UID USRNAME USRTYPE TTY TIME DATE

1 4192 1172 claireg udt ttyp2 14:34:33 Sep 24 1998 2 4211 1172 claireg udt pts/0 14:35:11 Sep 24 1998

: % $UDTBIN/listuser Licensed/Effective # of Users Udt Sql Total

32 / 32 2 0 2

UDTNO USRNBR UID USRNAME USRTYPE TTY TIME DATE

1 4192 1172 claireg udt ttyp2 14:34:33 Sep 24 1998 2 4211 1172 claireg udt pts/0 14:35:11 Sep 24 1998

: Notice that the output of the WHO command includes the user name but not the process ID. Also, output from the LISTUSER command includes a series of identifi- cations: UDTNO (UniData user number), USRNBR (the UNIX pid), UID (the user’s UNIX uid number), and USRNAME. Displaying further information about a UniData process typically requires the UNIX pid (USRNBR).

Monitoring User Processes 14-6

Removing User Processes

UniData includes commands that enable you to stop a user’s udt process if the process is hung, or if you need to stop UniData while a user is still logged on to the system. The following table summarizes these commands.

UniData Command Description

stopudt Logs a user out of UniData; a current write is allowed to complete, but subsequent operations for that udt do not take place. stopudt uses the UNIX kill -15 command.

deleteuser First attempts to execute the UNIX kill -15 command. If the command is not successful in five seconds, force logs a user out of UniData with the UNIX kill -9 command; may interrupt a write in progress, potentially causing file corruption. UniData Commands for Removing User Processes

The UNIX kill command enables you to stop a process with a variety of options. Use this command with caution, since it can cause data inconsistency and file corruption. Refer to your host operating system documentation for information about the kill command.

You can log yourself out using the stopudt command, but you must be logged on as root to log out other users using stopudt. You must be logged on as root to execute deleteuser.

Warning: Both of these commands can disrupt the consistency of your database, and deleteuser can also corrupt data. These commands should not be used as a substitute for normal user logout. Tip: You can use the UniData system-level command udstat to check the activity of a UniData process. If the process shows activity, stopping or deleting it could damage data. See the UniData Commands Reference for examples of the udstat command. Communicate directly with the owner, if possible, before stopping a udt process, because even if the process seems to be idle, the terminal may be waiting for input and files may be open.

14-7 Administering UniData on UNIX Using TIMEOUT

You can execute the ECL TIMEOUT command at the ECL prompt, in a LOGIN paragraph, or in a UniBasic program. TIMEOUT forces the current udt process to log out after a specified number of seconds. If you include TIMEOUT in the LOGIN paragraphs for your UniData accounts, you can provide some improved security for terminals left idle.

Warning: Be careful with TIMEOUT. Because this command can cause a UniBasic program to terminate at an INPUT statement rather than concluding normally, using TIMEOUT can cause inconsistent data in your database.

Removing User Processes 14-8 Chapter Managing Printers in UniData 15

UniData and UNIX Spoolers ...... 15-3 Configuring the Spooler ...... 15-3 SETOSPRINTER Command ...... 15-6 Spooling from UniData...... 15-7 UniData Printing Commands ...... 15-8 Configuring and Troubleshooting a Printer ...... 15-10 Physical Connection...... 15-10 Spooler Definition ...... 15-10 Definition in UniData ...... 15-11 SETPTR ...... 15-12 Environment Variables ...... 15-15 Disabling Printer Validation ...... 15-15 Defining an Alternate Search Path ...... 15-15 Examples ...... 15-16 Redefining the Default UniData Print Unit...... 15-16 Submitting Concurrent Print Jobs ...... 15-16 Printing to a UNIX Device...... 15-17 Passing Spooler Options to UNIX ...... 15-18 Decoding a UniData Print Statement ...... 15-19 This chapter explains how UniData interacts with UNIX spooler software and describes how to configure and access printers from within UniData.

15-2

UniData and UNIX Spoolers

All printing from within UniData is actually performed by your UNIX system spooler. UniData does not include its own spooler; the UniData printing commands form an interface to UNIX spooler commands.

Different versions of UNIX include different spooling systems. There are two major types: the BSD spooling system, which includes the lpd, lpc, and lpr commands, and the ATT spooling system, which includes the lp command. The spooling system for any UNIX is likely to be derived from one or the other of these types, although each UNIX version may include platform-specific features. Also, some UNIX versions include the user interfaces from both spooling system types, although underlying processing is accomplished by one or the other type. Refer to your host operating system documentation for information about the spooler on your UNIX system.

Users can specify any UNIX spooler command supported on their system by using the ECL command SETOSPRINTER, and defining the spooler configuration file.

Configuring the Spooler

UniData includes a spooler configuration file, UDTSPOOL.CONFIG, located in udthome/sys. This file contains information for the interface between the UniData SETPTR command and the UNIX-level print commands. The UDTSPOOL.CONFIG file contains interface information for the lp command on each system. For some systems, UDTSPOOL.CONFIG also contains information for the lpr command.

15-3 Administering UniData on UNIX The following example illustrates the UDTSPOOL.CONFIG file:

% pg UDTSPOOL.CONFIG * Default $UDTHOME/sys/UDTSPOOL.CONFIG

* Beginning of the configuration file

* Beginning of entry for lp command.

lp DEFAULT * Default print command BANNER: -t DEST: -d COPIES: -n FORM: -f NHEAD: -o nobanner NOMESSAGE: -s DEFAULT: -c

* End of entry for lp command.

* Beginning of entry for lpr command.

lpr BANNER: -J DEST: -P COPIES: -# FORM: -P NHEAD: -h NOMESSAGE: * not available DEFAULT: * not available

* End of entry for lpr command.

* End of the configuration file (EOF): Notice the following points about the UDTSPOOL.CONFIG file:

„ The file contains two entries, one for lp and one for lpr. Each entry lists the spooler command options that correspond to typical SETPTR options. Within each entry, the “DEFAULT” line indicates options that the spooler should always include. „ In the first entry for lp, the lp command is identified as DEFAULT. This identification directs UniData to use lp as the spooler command unless you specify another command.

You can modify the UDTSPOOL.CONFIG file using any UNIX text editor. You can add spooler commands, modify the existing entries, and change the command identified as the DEFAULT.

UniData and UNIX Spoolers 15-4

If a particular option is not available with your spooler command, and you do not want UniData to display an error message each time you invoke your spooler command, you can specify U_IGNORE next to the unsupported option. In the following example, the NHEAD option in the lp command has a value of U_IGNORE. Therefore, UniData ignores the NHEAD option, even if you specify it with the SETPTR or SP.ASSIGN commands:

%pg UDTSPOOL.CONFIG * $UDTHOME/sys/UDTSPOOL.CONFIG for AIX 4.1.1

* Beginning of the configuration file

* Beginning of entry for lp command.

lp DEFAULT * Default print command BANNER: -t DEST: -d COPIES: -n FORM: -d NHEAD: U_IGNORE NOMESSAGE: * not available DEFAULT: -c

* End of entry for lp command.

* Beginning of entry for lpr command.

lpr BANNER: -J DEST: -P COPIES: -# FORM: -P NHEAD: -h NOMESSAGE: * not available DEFAULT: * not available

* End of entry for lpr command.

* End of the configuration file

15-5 Administering UniData on UNIX Enabling and Disabling Printers

The PTRENABLE and PTRDISABLE commands issue platform-dependent UNIX commands to enable and disable a printer, respectively. You may add the appropriate UNIX commands to the UDTSPOOL.CONFIG file for your platform, as shown in the following example:

# pg UDTSPOOL.CONFIG * $UDTHOME/sys/UDTSPOOL.CONFIG for HPUX10

* Beginning of the configuration file

* Beginning of entry for lp command.

lp DEFAULT * Default print command BANNER: -t DEST: -d COPIES: -n FORM: -d NHEAD: -o nb NOMESSAGE: -s DEFAULT: -c PTRENABLE: enable PTRDISABLE: disable -c

* End of entry for lp command.

* End of the configuration file If you do not specify the PTRENABLE and PTRDISABLE commands in the UDTSPOOL.CONFIG file, the defaults of enable and disable are used.

SETOSPRINTER Command

The ECL SETOSPRINTER command enables you to select a UNIX spooler command.

Syntax:

SETOSPRINTER “spooler_command”

The spooler_command must have an entry in your UDTSPOOL.CONFIG file.

The following example illustrates the SETOSPRINTER command:

:SETOSPRINTER "lp" :SETOSPRINTER "lpr"

UniData and UNIX Spoolers 15-6

If you select a printer that does not have any entry in the UDTSPOOL.CONFIG file, SETOSPRINTER displays an error message similar to the following example:

:SETOSPRINTER "my_printer" Can't find my_printer in /disk1/ud72/sys/UDTSPOOL.CONFIG.

Spooling from UniData

Print requests from within UniData are generated by UniBasic commands (PRINT, PRINT ON) and by using the LPTR keyword in UniQuery. When a print request is generated, the following actions happen:

„ UniData uses information from the print request to create a temporary file containing the output to be printed. This temporary file is created by default in the /tmp directory. If you define the UniData configuration parameter TMP, UniData creates the print file in the location specified by TMP. You can override this setting by setting the environment variable TMP at the UNIX prompt before entering UniData. Note: If you execute SETPTR and set the printer mode to 3 or 6, UniData creates the print file in the _HOLD_ directory of your current UniData account. „ UniData uses the name of the temporary file and information from the printer setup (SETPTR for the logical printer to receive the output) to create a UNIX spooler command. „ The UNIX spooler command accepts the temporary file as input. (Notice that the printed output may not reflect changes made to your database after the print request was issued.) „ After the output is printed, UniData deletes the temporary file.

15-7 Administering UniData on UNIX UniData Printing Commands

Note: UniData includes a number of commands that enable you to customize output from UniBasic programs and UniQuery reports. See the UDT.OPTIONS Commands Reference for a complete listing of all available options.

The following table describes ECL commands related to printing.

Command Description

LIMIT Displays the current spooler setting used for printing.

SETOSPRINTER Selects a UNIX spooler command.

SETPTR Defines logical printer units within a UniData session and displays the current spooler setting.

SPOOL Prints the contents of a record to the printer.

PTRDISABLE, Disables a UNIX printer or queue. These commands are equiv- STOPPTR, or alent to your spooling system’s disable command. You must SP.STOPLPTR supply a printer or queue ID that is defined to your spooler. Do not supply a UniData logical print unit number.

PTRENABLE, Enables a UNIX printer or queue. The commands are equivalent STARTPTR, or to your spooling system’s enable command. You must supply a SP.STARTLPTR printer or queue ID that is defined to your spooler. Do not supply a UniData logical print unit number.

SP.CLOSE Closes a print file.

SP.ASSIGN Defines logical printer units within a UniData session (Pick® compatible syntax).

SP.EDIT Manipulates print files in the _HOLD_ directory.

SP.KILL Cancels a job. This command is equivalent to UNIX cancel nnn, where nnn is the job number.

SP.OPEN Opens a continuous print job. This command is equivalent to the UniData SETPTR ,,,,,,OPEN command. UniData Printing Commands

UniData Printing Commands 15-8

Command Description

SP.STATUS Provides printer and queue information. This command is equiv- alent to UNIX lpstat -t.

LISTPTR Prints the names of printers and paths of devices associated with them. This command is equivalent to UNIX lpstat -v.

LISTPEQS or Prints the status of the output request. These commands are SP.LISTQ equivalent to UNIX lpstat -o. UniData Printing Commands (continued) Note: See the UniData Commands Reference for the syntax of these ECL commands.

15-9 Administering UniData on UNIX Configuring and Troubleshooting a Printer

In order for any user to print to a printer from UniData, all the following conditions must be true. Use these conditions as a guideline for setting up a printer and for troubleshooting printer problems.

Physical Connection

The printer must be physically connected to your computer.

Troubleshooting

„ Check cables and connections. „ Check power to the printer and check the printer for error conditions. „ Use the UNIX cat command to write a text file to the printer in serial mode; verify that all contents of the file print successfully. For example, assume you have a file called textfile and a printer attached to /dev/tty01; enter cat textfile > /dev/tty01 at the UNIX prompt to test the connection.

Refer to your host operating system documentation and your printer documentation for information about connecting and troubleshooting a printer.

Spooler Definition

You must define the printer to your UNIX spooler. Depending on your spooling system, a printer can be a discrete destination or a member of a device class.

Troubleshooting

„ Use the UNIX lpstat command to determine if the printer is defined. Check your documentation to learn which option of lpstat to use. „ Attempt to spool a text file to the printer using the default print command (for example, lp -c -dqueuename textfile). Remember that if you added the printer as a member of a device class, spooling to the device class sends the job to the first available member of that class, which may not be the desired printer.

Configuring and Troubleshooting a Printer 15-10

Note: Refer to your host operating system documentation for information about your spooling system. Because different UNIX versions include different spooling systems, procedures for defining a printer to a spooler vary.

Definition in UniData

In order to access a specific printer (or queue) from a UniData session, you need to link the printer, or queue, to an internal print unit in UniData. Use the ECL SETPTR command for this. See “SETPTR” on page 15-12.

Note: If you do not define a specific printer or queue with SETPTR, UniData directs output from UniBasic PRINT statements (following PRINTER ON statements), or from UniQuery statements with the LPTR option, to the default printer or queue on your system.

15-11 Administering UniData on UNIX SETPTR

The SETPTR command enables you to define “logical printer units” within a UniData session. A logical printer unit is a combination of a printer destination, a form type, page dimensions, and additional options. By varying form type and options, you can define more than one logical printer unit for a single physical printer.

With SETPTR, you can define up to 31 logical printer units in a single UniData session. Throughout UniData, you can define up to 255, but you can define only 31 in a single user session.

Syntax:

SETPTR unit [width,length,topmargin,bottommargin] [mode] [“spooleroptions”] [options]

The following table lists the parameters of the SETPTR syntax.

Parameter Description

unit Logical printer unit number; internal to UniData; you can map this to a UNIX printer or queue with the DEST and FORM options. Must range from 0 through 254; default is 0.

[width] Number of characters per line; must be within the range 0-256; default is 132.

[length] Number of lines per page; must be within the range 1-32,767 lines; default is 60.

[topmargin] The number of lines to leave blank at the top of each page; must be within the range 0-25; default is 3.

[bottommargin] The number of lines to leave blank at the bottom of each page; must be within the range 0-25; default is 3.

[mode] Enables additional flexibility to direct output; default is 1; see separate table.

[“spooleroptions”] Enables you to specify desired spooler options as a quoted string, which UniData then passes directly to the UNIX spooler.

[options] Enables you to specify printing options that UniData then inter- prets and passes to the UNIX spooler. See separate table. SETPTR Parameters

SETPTR 15-12

Note: Users familiar with Pick® conventions should be aware that printer unit numbers set with SETPTR are not the same as Pick® printer numbers. SETPTR enables you to define logical printer units, which may be linked to specific printers. UniData printer unit numbers are used with the PRINT ON statement in UniBasic to allow multiple concurrent jobs. Pick® printers (forms) are specified with the DEST option of SP.ASSIGN.

The next table describes modes for SETPTR.

Mode Description

1 Directs output to a line printer only.

2 Must be used with DEVICE option; directs output to the serial device specified by the DEVICE option.

3 Directs output to a _HOLD_ file only.

6 Directs output to both a _HOLD_ file and a line printer.

9 Directs output to a line printer; suppresses display of the _HOLD_ entry name. SETPTR Modes

The next table describes options for the SETPTR command.

Option Description

BANNER [string] Modifies the default banner line (which is the UNIX user ID). Depends on MODE setting; also modifies _HOLD_ entry name.

BANNER UNIQUE Modifies the default banner line, and automatically uses [string] attribute 1 (NEXT.HOLD) in the dictionary for the _HOLD_ file to create unique entry names for jobs sent to _HOLD_.

BRIEF Directs UniData not to prompt for verification upon execution of SETPTR.

COPIES n Prints n copies of the print job.

DEFER [time] Delays printing until the specified time. Refer to your host operating system documentation for the correct syntax for specifying time. You need the documentation for the UNIX at command. SETPTR Options

15-13 Administering UniData on UNIX Option Description

DEST unit (or AT unit) Directs output to a specific printer or queue. The unit must be a valid destination at your site. Refer to your spooler documen- tation and use the UNIX lpstat command for information about valid destinations.

DEVICE filename Used with mode 2 only. Directs output to the UNIX device whose special file is filename.

EJECT Ejects a blank page at the end of each print job.

NOEJECT Suppresses the form feed at the end of each print job.

FORM form Assigns a specified form to each print job. The form must be defined to your spooler before you use this option.

LNUM Prints line numbers in the left margin of each print job.

NFMT or NOFMT Suspends all UniData print formatting.

NHEAD or NOHEAD Suppresses the banner for each print job.

NOMESSAGE Suppresses all messages from your UNIX spooler.

OPEN Opens a print file and directs output to this file until the file is closed by the SP.CLOSE command. SETPTR Options (continued)

SETPTR 15-14

Environment Variables

UniData provides two environment variables that affect printing.

Disabling Printer Validation

You can bypass validation of DEST and FORM in SETPTR against the UNIX spooler’s list of logical printers by setting the NOCHKLPREQ environment variable. (UniData concatenates DEST and FORM prior to validation, since many logical printers can access a single physical printer or queue.) On systems with hundreds of print units defined in the UNIX spooler, the UniData validation can take so much time that it is a processing bottleneck. In such cases, setting NOCHKLPREQ removes the bottleneck. Use the following commands to set NOCHKLPREQ:

Bourne or Korn Shell:

NOCHKLPREQ=1;export NOCHKLPREQ

C shell:

setenv NOCHKLPREQ 1

Consider setting this variable in each user’s .login or .profile.

Warning: If you disable validation, you may encounter unexpected results, including lost print jobs, by specifying invalid DEST/FORM combinations. It is safest to disable checking if you execute your SETPTR commands in a paragraph or a UniBasic program rather than manually, and if you test all options before implementing them in production.

Defining an Alternate Search Path

The LPREQ environment variable enables you to provide an alternate search path for DEST/FORM validation. UniData automatically searches the default directory for your UNIX environment (for instance, on an HP-UX system, the directory /usr/spool/lp/request). If you wish to use a different directory, use LPREQ.

15-15 Administering UniData on UNIX Examples

The following section describes a number of ways you can use SETPTR.

Redefining the Default UniData Print Unit

To keep UniBasic applications general, developers typically use (or assume) printer unit 0, which is the default. The SETPTR command enables you to redefine unit 0 to direct output from different parts of an application to different physical printers or queues, or to change formatting options. The following example redefines the default print unit for different reports:

:CT VOC OUTPUT VOC:

OUTPUT: PA SETPTR 0,80,60,,,1,BRIEF,DEST laser RUN BP REPORT_PRINT SETPTR 0,80,40,,,1,BRIEF,DEST laser,FORM invoice RUN BP INVOICE_PRINT : Notice the following points:

„ Both “laser” and “laserinvoice” must be valid destinations defined to your UNIX spooler. If you have defined NOCHKLPREQ, invalid destinations cause print jobs to fail. If you did not define NOCHKLPREQ, invalid desti- nations cause the SETPTR command in the paragraph to fail. „ If “laser” is a single printer rather than a queue, the two SETPTR commands both access a single physical printer. This type of definition is acceptable.

Submitting Concurrent Print Jobs

With SETPTR, you can define up to 31 logical printer units in a single UniData session. You can use this functionality to submit concurrent print jobs from a UniBasic application. One common implementation follows:

„ Define two logical printer units (for instance, 0 and 1). „ Direct all lines of a report to one printer with the UniBasic PRINT ON command (for instance, PRINT ON 0 PRINT.LINE).

Examples 15-16

„ Direct summary (break) lines to the second printer (PRINT ON 0 PRINT.LINE followed by PRINT ON 1 PRINT.LINE).

In this way, you can print a summary report and a detail report at the same time.

Printing to a UNIX Device

Use mode 2 of the SETPTR syntax to direct output to a UNIX device. The following sample command shows the correct syntax for this option:

SETPTR 0,,,,,2,DEVICE /dev/tty9

When you use mode 2, UniData sets the following options by default:

„ NOEJECT „ NOFMT „ NOHEAD „ NOMESSAGE „ OPEN

When you use mode 2, UniData disables the following options:

„ BANNER „ BANNER UNIQUE „ BRIEF „ COPIES „ DEFER „ EJECT

You must use the DEVICE option with mode 2, and you must specify the output device. The device can be a terminal, a serial printer, a tape drive, or a disk file. You must specify the device special file if writing to an actual device. If you want to spool to a disk file, you must specify the full path of the file.

15-17 Administering UniData on UNIX Passing Spooler Options to UNIX

The SETPTR command enables you to use any option accepted by your default UNIX spooler command, by specifying the options as a quoted string on the SETPTR command line. The following sample command uses the spooler options for banner title (-t) and for copies (-c) directly, rather than the UniData options:

SETPTR 0,,,,,1,"-tACCOUNTS,-c2",DEST laser

Using a quoted string is helpful when you receive unexpected results from UniData SETPTR options, or if your default spooler supports more options than UniData accepts.

Tip: Use the ECL LIMIT command or the SETPTR command to display the default spooler command in your UniData release, and then refer to your host operating system documentation for supported options for that command.

Examples 15-18

Decoding a UniData Print Statement

To research printing problems within UniData, it is sometimes helpful to determine what arguments UniData is passing to your UNIX spooler. If your system has a C compiler, you can code and compile a C program that functions as a “dummy” spooler. This program interprets UniData commands and reports arguments, but does not actually perform any spooling. By executing the dummy program instead of your default spooler command, you can test UniBasic PRINT statements or UniQuery reports to determine how UniData interprets your printing configuration. Complete the following steps to set up the test:

1. Determine Your Default Spooler Command

Use the ECL LIMIT command or the SETPTR command to determine the current spooler command, as shown in the following example:

:LIMIT . . . U_MAXBREAK: Number of BREAK.ON+BREAK.SUP in LIST = 15. U_MAXLIST: Number of attribute names in LIST = 999. U_LINESZ: Page width in printing = 272. U_PARASIZE: Paragraph name and its parameter size = 256. U_LPCMD: System spooler name = lp -c . U_MAXPROMPT: Number of prompts allowed in paragraph = 60. . . . :SETPTR Unit 0 Width 132 Length 60 Top margin 3 Bot margin 3 Mode 1

Options are:

Spooler & options: lp -c : In the output for the LIMIT command, look for “U_LPCMD, System spooler name.” In this example, the current command is lp -c. In the output for the SETPTR command, “Spooler & options” appears at the bottom of the screen.

15-19 Administering UniData on UNIX 2. Create the C Program

The following C program, called prtarg.c, captures the arguments that a UniData or UniBasic printing command sends to your UNIX spooler. Use vi or any UNIX text editor to create the program in a work directory of your choice.

int argc; char *argv[ ]; { int i; for (i = 0; is\n",i,argv[i]); return(0); } Save the file as prtarg.c.

3. Compile the C Program

When you compile the prtarg.c program, you want to produce an executable whose name matches your default spooler command. In the previous example, the spooler command was lp. For some UNIX versions, the command may be different. Use the cc -o command to compile the program, as shown in the next example:

# cc -o /usr/ud72/work/lp prtarg.c # cd /usr/ud72/work # ls -l lp -rwxrwxrwx 1 root sys 16384 Jun 4 10:56 lp #

4. Redefine Your Path

To execute the dummy spooler instead of the UNIX spooler, you need to redefine your execution path so that UNIX locates the dummy version before the real version. The work directory where the dummy version resides must appear at the beginning of your execution path.

The following sample commands show how to redefine the path:

C shell:

set path=(/users/test/work $path)

Bourne or Korn shell:

PATH=/users/test/work:$PATH;export PATH

Decoding a UniData Print Statement 15-20

Once the path is redefined, your dummy spooler executes in place of the default version.

5. Test UniData Printing

With your execution path redefined, log on to a UniData account and test printing commands. The following screen shows sample output from the dummy spooler for a UniQuery report statement that includes the LPTR keyword, and from a UniBasic PRINT statement:

:SETPTR 0 Unit 0 Width 80 Length 60 Top margin 3 Bot margin 3 Mode 1

Options are: Destination laser

Spooler & options: lp -c -dlaser :LIST VOC WITH F1 = PA LPTR Argument number 0 is ->lp Argument number 1 is ->-c Argument number 2 is ->-dlaser Argument number 3 is ->/tmp/PRT2a18917 : :CT BP PRINTON BP:

PRINTON PRINTER ON PRINT “HELLO WORLD” END :RUN BP PRINTON Argument number 0 is ->lp Argument number 1 is ->-c Argument number 2 is ->-dlaser Argument number 3 is ->/tmp/PRT4a18917 :

6. Reset Your Execution Path

When you have completed a testing session, make sure you reset your execution path so that the actual spooler command executes rather than the dummy spooler program.

15-21 Administering UniData on UNIX Chapter Accessing UNIX Devices 16

UniData Tape Handling Commands ...... 16-3 SETTAPE ...... 16-5 Steps for Tape Device Use ...... 16-6 UniData LINE Commands...... 16-9 Communicating with GET and SEND ...... 16-10 Dual-Terminal Debugging in UniBasic ...... 16-12 Setting Up Dual-Terminal Debugging ...... 16-13

This chapter describes UniData commands for identifying and accessing UNIX tape devices. This chapter also describes commands for reading and writing to other UNIX devices, which you can use for transferring data and also for debugging UniBasic applications.

16-2 Administering UniData on UNIX UniData Tape Handling Commands

UniData includes a number of ECL and UniBasic commands for reading data from a tape and writing data to a tape. The following table summarizes these ECL commands.

Command Description

SETTAPE Defines a logical tape unit in UniData; requires root access; must precede all other tape commands.

T.ATT Links a logical tape unit to a UniData process; must precede any reads/writes involving the tape.

T.BAK Moves a tape backward a specified number of files.

T.CHK or T.CHECK Reads a tape created by T.DUMP and check for damage.

T.DET Releases a logical tape unit when a UniData process is finished with it.

T.DUMP Copies the contents of a file or active select list to tape.

T.EOD Moves a tape to end of file.

T.FWD Moves a tape to the beginning of the next file.

T.LOAD Loads records from a tape created with T.DUMP.

T.RDLBL Reads and displays the first 80 characters of the tape label on a tape created with T.DUMP.

T.READ Reads and displays the next record from tape.

T.REW Rewinds a tape.

T.SPACE Moves a tape forward a specified number of files.

T.STATUS Displays current tape device assignments.

T.UNLOAD Rewinds and unloads a tape.

T.WEOF Writes an end-of-file mark on a tape. ECL Tape Handling Commands

Note: See the UniData Commands Reference for information about ECL commands.

UniData Tape Handling Commands 16-3

The next table summarizes UniBasic commands for I/O on tape devices.

Command Description

READT Reads the next available record from tape.

RESIZET Changes the block size used by the WRITET statement.

REWIND Rewinds a tape.

WEOF Writes an end-of-file mark to a tape.

WRITET Writes the value of a specified expression as a record on a tape. UniBasic Tape Handling Commands

Note: See the UniBasic Commands Reference for information about these UniBasic commands.

16-4 Administering UniData on UNIX SETTAPE

The ECL SETTAPE command enables you to define logical tape units in your UniData environment. This command establishes a link between a UniData internal tape unit number and a UNIX file. You can use SETTAPE to relate unit numbers to tape devices or UNIX disk files.

Any user can execute SETTAPE unit.no to display the current settings for a tape unit. However, you must log on as root to define a tape unit or modify settings.

Once you define a tape unit using SETTAPE, users can access it in any UniData account on your system. The tape unit definition remains the same unless it is changed by root.

Syntax:

SETTAPE unit.no [dn.path.nr][dn.path.r][blocksize]

The following table describes the parameters of the SETTAPE syntax.

Parameter Description

unit.no Internal UniData tape unit number. Must be within the range 0-9.

[dn.path.nr] Full path for the no rewind device driver for unit.no.

[dn.path.r] Full path for the rewind device driver for unit.no.

[blocksize] Tape block size in bytes; must be a multiple of 512. The default value is 4096. SETTAPE Parameters

SETTAPE 16-5

Steps for Tape Device Use

Follow these steps to use tape devices from UniData:

1. Define Tape Units

Log on as root and execute the SETTAPE command to define up to 10 tape units for the UniData environment.

Remember that the tape unit number must be within the range 0-9. These are logical tape unit numbers within UniData; the SETTAPE command maps these to UNIX files.

Note: When defining tape units, be sure to define unit 0. Some of the UniData tape handling commands require unit 0 to be defined so that it can be used as a default.

When you define a tape device or modify a definition, you create or update an entry in the ASCII text file udthome/sys/tapeinfo.

2. Attach a Tape Device

You need to attach a logical tape device to your process before accessing it. The T.ATT command attaches a tape device. Any user can execute T.ATT from the ECL prompt or from within a UniBasic program. The following example shows some typical output from T.ATT:

:T.ATT 7 No information for unit 7 yet, ask your system administrator for help. T.ATT failed. :T.ATT 1 tape unit 1 blocksize = 4096. :T.ATT 1 BLKSIZE 16384 TAPELEN 2 tape unit 1 blocksize = 16384 length = 2MB :T.ATT 1 unit 1 is attached by another process It is lock number 65 in LIST.LOCKS. Try again later, T.ATT failed. Notice the following points about T.ATT:

„ You cannot attach a tape unit with T.ATT unless the unit was previously defined with SETTAPE.

16-6 Administering UniData on UNIX „ You can execute T.ATT repeatedly to change the tape block size and tape length. If you do not specify BLKSIZE, T.ATT uses the default block size specified in SETTAPE. „ Only one process can attach a tape unit at any time. You can attach more than one tape unit to a single process, but you cannot attach the same tape unit to more than one process. „ You can use the ECL T.STATUS command to list all defined tape units to find out which ones are attached and which are available. The following example shows sample output from T.STATUS: :T.STATUS

UNIT STATUS UDTNO USER CHANNEL ASSIGNED NUMBER NAME NAME BLOCKSIZE

1 ASSIGNED 1 ump01 /dev/rmt/0mn 4096 2 AVAILABLE 3 ASSIGNED 3 ump01 /dev/rmt/0mn 8192 5 AVAILABLE

3. Read From or Write To the Tape Device

When a tape unit is attached, you can access it from ECL or within a UniBasic program. The following example shows some typical output when a process with tape unit 4 attached executes the ECL T.DUMP command:

:T.DUMP DICT INVENTORY MU 04 16 record(s) dumped to tape :SELECT VOC WITH F1 = PA

9 records selected to list 0.

>T.DUMP VOC 9 record(s) dumped to tape :T.DUMP INVENTORY MU 01 Unit 1 not attached yet.

:T.DUMP INVENTORY MU 09 Unit 9 is not initialized yet. :T.STATUS

UNIT STATUS UDTNO USER CHANNEL ASSIGNED NUMBER NAME NAME BLOCKSIZE

1 AVAILABLE 2 AVAILABLE 3 AVAILABLE 5 AVAILABLE 8 AVAILABLE 4 AVAILABLE 0 AVAILABLE :

Steps for Tape Device Use 16-7

Notice the following points about the example:

„ You cannot write to a tape device unless it is attached with T.ATT. If you have attached more than one device, you need to specify the device to write to or read from. If you have attached only one device, UniData accesses the device you attached. „ With T.DUMP, you can write from an active select list. Note: When you access a tape device, the operation fails if the device is not properly connected or if no tape is mounted. The UniData T.ATT and SETTAPE commands do not detect device configuration problems, so you may be able to define and attach a device, but be unable to complete your access to it. Tip: Due to the differences in Pick® operating systems and manufactured tapes, IBM suggests you use the HDR.SUPP keyword when using the T.DUMP command, and when using the Pick® T-LOAD command, to avoid inconsistencies in tape labels.

4. Release the Tape Device

When you no longer need to access a tape device, use the T.DET command to release it so another process can use it. If you have attached more than one device, you need to release each one separately. If you have attached only one, the T.DET command releases it. You can execute T.DET from ECL or from within a UniBasic program.

16-8 Administering UniData on UNIX UniData LINE Commands

UniData includes a group of commands that enable you to read from and write to UNIX tty-type devices. These commands are used to define and attach line devices for use by the UniBasic GET and SEND commands. GET and SEND allow UniBasic programs to read data from or write data to serial devices such as scales or bar code readers.

Note: You can use GET and SEND and the LINE commands to communicate with a printer or terminal.

The following table describes the UniData commands.

Command Description

SETLINE Defines a UNIX tty device within UniData; requires root access; must precede all other line commands.

LINE.ATT Links a defined device to a UniData process; must precede all reads/writes involving the line.

PROTOCOL Displays or modifys data line transmission characteristics of an attached line.

LINE.DET Releases a device.

LINE.STATUS Displays current line device assignments.

UNSETLINE Removes a UNIX device definition set with SETLINE. Requires root access. UniData LINE Commands

Note: See the UniData Commands Reference for detailed information about the UniData LINE commands.

UniData LINE Commands 16-9

Communicating with GET and SEND

You must follow these steps to use to use the UniBasic GET and SEND commands:

1. Define a tty Device in UniData.

Use the SETLINE command to create a pointer in UniData to any valid UNIX tty device. Use LINE.STATUS to verify pointers and determine which lines may be attached to processes. You must log on as root to create or modify a pointer. The following example shows an example of SETLINE and LINE.STATUS:

:SETLINE 4 /dev/pty/ttyv5 :LINE.STATUS

LINE# STATUS UDT# USER-NAME DEVICE-NAME

0 Available N/A N/A /dev/pty/ttyv4 1 Available N/A N/A /dev/pty/ttyv4 2 Available N/A N/A /dev/pty/ttyv4 3 Available N/A N/A /dev/pty/ttyv4 4 Available N/A N/A /dev/pty/ttyv5

Line number(s) are attached by the current udt process:

None Note: To access a tty device from UniBasic, the device must be assigned a tty number.

When you execute SETLINE to create or modify a pointer, or UNSETLINE to delete a pointer to a device, you update a file in udthome/sys called lineinfo.

16-10 Administering UniData on UNIX 2. Attach the Line To Your Process

Use the LINE.ATT command, either before executing your UniBasic program or within your UniBasic program, to reserve a line for your process. Again, you can use LINE.STATUS to verify the line, as shown below:

:LINE.ATT 3 LINE 3 ATTACHED :LINE.STATUS

LINE# STATUS UDT# USER-NAME DEVICE-NAME

0 Available N/A N/A /dev/pty/ttyv4 1 Available N/A N/A /dev/pty/ttyv4 2 Available N/A N/A /dev/pty/ttyv4 3 In-Use 3 root /dev/pty/ttyv4 4 Available N/A N/A /dev/pty/ttyv5

Line number(s) are attached by the current udt process:

3 Note: Once you attach the line, you can execute the ECL PROTOCOL command to define its transmission characteristics. When you modify these characteristics, be aware that the values you specify remain in effect until modified again by another PROTOCOL command. You may wish to execute PROTOCOL after every LINE.ATT, to ensure that the transmission characteristics are correct for your application.

3. Access the Line

In your UniBasic application, use the GET command to retrieve data from your tty device and the SEND command to direct data to the device. See the UniBasic Commands Reference for detailed information about GET and SEND.

4. Release the Line

Use the ECL LINE.DET command from the ECL prompt or within your UniBasic application to release the tty device.

Communicating with GET and SEND 16-11

Dual-Terminal Debugging in UniBasic

If you are debugging a UniBasic application that performs terminal I/O, it is often more efficient to display debugger messages on a separate screen from the appli- cation. The following table summarizes ECL commands for dual-terminal debugging.

Command Description

SETDEBUGLINE Sets a pointer to the terminal or window where you want debugger messages to display.

DEBUGLINE.ATT Connects to the terminal or window you specify with SETDEBUGLINE.

DEBUGLINE.DET Detaches from the terminal or window to which you are connected.

UNSETDEBUGLINE Removes the pointer you set with SETDEBUGLINE. ECL Commands for Dual-Terminal Debugging You do not need to log on as root to execute these commands.

Note: Refer the UniData Commands Reference for detailed information about the ECL commands for dual-terminal debugging, and see Using the UniBasic Debugger for information about the UniBasic debugger.

16-12 Administering UniData on UNIX Setting Up Dual-Terminal Debugging

Complete the following steps to set up a dual-terminal debugging session.

1. Log On to Two Terminals (or Windows)

„ You need to log on to two terminals or windows. „ You do not need to log on to UniData on the terminal where you will display your debugger messages. „ You need to know the full path of the terminal device special file.

2. Set a Pointer to the Display Terminal

Use the ECL SETDEBUGLINE command to set a pointer from the terminal where your application is running to the terminal where you want messages to display. The following screen shows an example:

:SETDEBUGLINE /dev/pty/ttyv4 : Notice that you must specify the full path for the device special file for your display terminal.

3. Connect to the Display Terminal

Use the DEBUGLINE.ATT command to connect to the terminal you just defined.

4. Conduct the Debugging Session

The following two screens show dual-terminal debugging. On the first screen, a UniBasic program is executed with the -D option:

:DEBUGLINE.ATT :RUN BP EXAMPLE -D

Setting Up Dual-Terminal Debugging 16-13

On the second screen, the messages from the UniBasic debugger appear:

:MYSELF ump01 pty/ttyv0 Jun 4 11:34 :***DEBUGGER called at line 1 of program BP/_EXAMPLE !

5. Detach From the Display Terminal

Use the ECL DEBUGLINE.DET command to return debugger messages to the appli- cation terminal. You can reconnect using DEBUGLINE.ATT, as long as your debug line is still set.

6. Release the Display Terminal

At the end of your debugging session, execute the ECL UNSETDEBUGLINE command to remove the pointer to the display terminal.

16-14 Administering UniData on UNIX Chapter Managing Memory 17

UniData Monitoring/Configuring Tools ...... 17-3 udtconf Main Display ...... 17-3 Calculating udtconfig Parameters ...... 17-4 Checking Configuration Parameters ...... 17-5 Saving Configuration Parameters ...... 17-5 Recalculating the Size of the CTL ...... 17-6 Viewing Current and Suggested Settings ...... 17-6 Exiting udtconf ...... 17-7 Setting Shared Memory Parameters ...... 17-8 shmconf: Main Display...... 17-8 shmconf: Viewing Current and Suggested Settings ...... 17-9 shmconf: Recalculating the Size of CTL ...... 17-10 shmconf: Recalculating Other Parameters ...... 17-10 Shared Memory and the Recoverable File System ...... 17-11 Analyzing UniData Configuration Parameters ...... 17-11 Checking and Changing UniData Configuration Parameters . . . . 17-13 Checking Kernel Parameters ...... 17-15 sms ...... 17-15 Learning about Global Pages ...... 17-18 Learning about Local Control Tables ...... 17-19 UNIX Monitoring Tools ...... 17-21 UniData Configuration Parameters ...... 17-22 UNIX Kernel Parameters ...... 17-22 UniData Error Messages for smm ...... 17-23

This chapter describes UniData commands and utilities for configuring, monitoring, and troubleshooting shared memory. The chapter also lists UniData error messages related to shared memory, and presents suggestions for resolving the errors.

The following commands and utilities enable you to monitor the use of shared memory on your system and provide suggestions for configuration and tuning.

17-2 Administering UniData on UNIX UniData Monitoring/Configuring Tools

The udtconf utility enables you to automatically set all udtconfig parameters, including those for shared memory. Although shared memory requirements are highly application and platform dependent, udtconf can provide suggestions for udtconfig parameters and provide information about the actual state of your system.

Syntax:

udtconf

You do not have to log on as root to run udtconf, but the utility reads information from the udtconfig file, located in /usr/ud72/include, and from the UNIX kernel. If you do not log on as root, you may not have sufficient access to the kernel, and the results will be unreliable.

You should run udtconf with UniData users logged off and UniData shut down. The one exception is to assess the impact of the Recoverable File System (RFS) system buffer. In this case, run udtconf from a UNIX prompt while UniData is running. udtconf Main Display

The following example shows the main screen of the udtconf utility:

UniData Monitoring/Configuring Tools 17-3

To advance to a field displayed on the screen, press TAB. To page down, enter CTRL- D. To page up, enter CTRL-U.

The udtconf utility displays warning messages if some of the kernel parameters are not adequate to support the values udtconf calculates. Make sure that the kernel parameter for semaphore undo structures, usually semmnu, is adequate to support the number of authorized users prior to running udtconf.

Settings for the udtconfig parameters NUSERS, SHM_GNTBLS, N_GLM_GLOBAL_BUCKET, GLM_MEM_SEGSZ, N_TMQ, and N_PGQ are based on the number of authorized users. Although udtconf displays warning messages if kernel parameters are not adequate to support these settings, the udtconfig file is updated with these values. In this case, UniData may not start.

Calculating udtconfig Parameters

If you change a value in the udtconf screen, udtconf can automatically calculate values for udtconfig parameters which are dependent upon the value you change. To calculate parameters, enter CTRL-A.

17-4 Administering UniData on UNIX Checking Configuration Parameters

Press CTRL-K to check the UniData configuration parameters against the kernel parameters. If a UniData configuration parameter cannot be supported by a kernel parameter setting, UniData displays a warning message at the bottom of the screen for each conflicting parameter, as shown in the following example:

When all configuration parameters have been checked, UniData displays the message “Shared memory related configuration values are OK!”

Saving Configuration Parameters

Press CTRL-V to save the configuration parameters to the udtconfig file, located in /usr/ud72/include. If you do not save the parameters, no changes are made to the udtconfig file.

UniData Monitoring/Configuring Tools 17-5

Recalculating the Size of the CTL

Press CTRL-L from any udtconf screen to recalculate the size of your global control table, CTL. This table changes size if you change shared memory parameters such as SHM_GNTBLS, SHM_GNPAGESZ, and so forth. If the table size is greater than the kernel parameter shmmax (and the udtconfig parameter SHM_MAX_SIZE), UniData will not start.

Viewing Current and Suggested Settings

To view current and suggested UNIX kernel settings, press CTRL-P. The following example shows sample output:

udtconf suggests values assuming that UniData is the only software product on your system. If that is true, as long as the current kernel settings for semaphore undo struc- tures, shared memory segments, and so forth, are at least equal to the suggested values, it should not be necessary to rebuild your kernel. If you have additional appli- cations running, you need to consider the combined effect of UniData and all other applications when evaluating your kernel settings.

17-6 Administering UniData on UNIX Exiting udtconf

To exit the udtconf utility, enter CTRL-E. If you have made changes to configuration parameters, make sure to save the changes by using CTRL-V before exiting the program.

UniData Monitoring/Configuring Tools 17-7

Setting Shared Memory Parameters

The shmconf utility enables you to set udtconfig parameters for shared memory automatically. Unlike udtconf, you cannot set udtconfig parameters that do not apply to shared memory through shmconf. Although its usability for this purpose is platform-dependent and application-dependent, the utility provides configuration suggestions and information about the actual state of your system.

Syntax:

shmconf

You do not need to log on as root to execute shmconf, but the utility reads information from udtconfig parameters and from the UNIX kernel. If you do not log on as root, you may not have sufficient access to the kernel, and the results will be unreliable.

In general, you should run shmconf with UniData users logged off and UniData shut down. The one exception is to assess the impact of the RFS system buffer. In this case, run shmconf from the UNIX prompt while UniData is running.

Note: Only one user at a time may run shmconf.

shmconf: Main Display

The following screen shows a sample of the first output screen from shmconf:

17-8 Administering UniData on UNIX Note: Most of the figures displayed are current values read from UniData configuration parameters. The value of SHMMAX, however, is empirically determined by the shmconf program, which tests to determine the largest shared memory segment actually available on the system. Tip: If the value of SHMMAX on this screen is significantly smaller than your kernel configuration (see next example), other applications may be reserving shared memory, or you may have insufficient swap space. shmconf: Viewing Current and Suggested Settings

To view current and suggested UNIX kernel settings, press CTRL-P. The following screen shows sample output:

Setting Shared Memory Parameters 17-9

Note: shmconf suggests values assuming that UniData is the only software product on your system. If that is true, as long as the current kernel settings for semaphore undo structures, shared memory segments, and so forth, are at least equal to the suggested values, it should not be necessary to rebuild your kernel. If you have additional applications running, you need to consider the combined effect of UniData and all other applications when evaluating your kernel settings.

shmconf: Recalculating the Size of CTL

Press CTRL-L from any shmconf screen to recalculate the size of your global control table, CTL. This table changes size if you change shared memory parameters such as SHM_GNTBLS, SHM_GPAGESZ, and so on. If the table size is greater than the kernel parameter shmmax (and the UniData configuration parameter SHM_MAX_SIZE), UniData will not start.

shmconf: Recalculating Other Parameters

shmconf also enables you to recalculate parameters related to shared memory. Press CTRL-A from any screen to do so.

17-10 Administering UniData on UNIX Note: shmconf recalculates parameters, but does not update the udtconfig file unless you specify CTRL-V (saVe).

Shared Memory and the Recoverable File System

If you are using the Recoverable File System (RFS), UniData reserves the amount of shared memory required for the system buffer. UniData reserves this memory during startup, and it is not available to the smm or sbcs daemons. If your system was running close to its limits in terms of memory resources without RFS, allocating the system buffer can have a significant impact. For instance, you may see an increase in error messages indicating smm could not create or attach a shared memory segment.

Analyzing UniData Configuration Parameters

The system-level systest command checks all parameters in the udtconfig file, located in /usr/ud72/include.

Syntax:

systest [-mn] [-sn] [-u] [-f filename] [-v] [c {n|r}]

The following table describes each parameter of the syntax:

Parameter Description

[-mn] Changes memory map display by about n MB. Highly platform dependent. Do not use this unless advised by IBM.

[-sn] Changes memory map display by about n MB. Highly platform dependent. Do not use this unless advised by IBM.

[-u] Creates or updates the UniData the configuration parameters NFILES and/or SHM_ATT_ADD. Use with extreme caution. systest Command Parameters

Setting Shared Memory Parameters 17-11

Parameter Description

[-f filename] Creates a file name you specify containing the UniData configuration parameters systest recommends for the number of authorized users and platform.

[-v] Displays detailed (verbose) output.

[-c {n|r}] Checks current kernel parameter settings against UniData recommendations. Specify the -cr option to compare against recommen- dations for the Recoverable File System. Specify the -cn option if you will not be using recoverable files. systest Command Parameters (continued) Note: The -m and -s options of systest function differently on different platforms and also function differently depending on machine activity. These options help you assess the effects of redefining memory addresses on your system. However, different UNIX versions handle memory allocation so differently that these options may not produce meaningful output. Do not use them unless advised by IBM Technical Support.

You must log on as root to execute systest. Users do not need to log out of UniData.

17-12 Administering UniData on UNIX The following example shows sample output from the systest command, with no options:

# systest Memory Address Layout ------

|------|------|------| MALLOC STACK SHMAT ---> ---> --->

IPC Facilities Test Results ------

Max # of Shmem Segments: 100 # of Shmem Seg. Per Process: 36 Max / Min Shmem Seg. Size: 268435456 (256M) / 1 SHMLBA: 4096 (4K)

Max # of Message Queues: 50 Max # of Bytes On Queue: 32768 (32K) Max Message Size: 32768 (32K)

Max # of Semaphores: 36 Max # of Undo Structures: 150

Shmem Attach Address: 0 Usable Shmem Address Range: unknown # Note: The information displayed in the “IPC Facilities Test Results” section reflects current settings in your UNIX kernel.

Checking and Changing UniData Configuration Parameters

Complete the following steps to update UniData configuration parameters with new values systest suggests. You want to do this if, for instance, you have upgraded your license for more users or you have added physical memory to your system.

1. Use the -f filename option of systest to create an output file in the format of the UniData configuration file (/usr/ud72/include/udtconfig). 2. Execute the UNIX diff command using the file created in step 1 and the udtconfig file to determine the changes suggested by systest.

Setting Shared Memory Parameters 17-13

3. Save a copy of your udtconfig file, then manually update the production udtconfig file to reflect those changes you want to make. Check your work carefully. 4. Make sure users are logged out of UniData. 5. Stop UniData with stopud, and start UniData with startud, to make the changes effective.

The following screen shows typical output from steps 1 and 2:

# systest -f udtconfig.new # diff /usr/ud72/include/udtconfig udtconfig.new 13c13 < NUSERS=40 --- > NUSERS=125 34c34 < SHM_GNTBLS=40 --- > SHM_GNTBLS=125 36c36 < SHM_GPAGESZ=1024 --- > SHM_GPAGESZ=256 96c96 < SB_FLAG=1 --- > SB_FLAG=0 120,121c120,121 < N_PGQ=10 < N_TMQ=10 --- 132c132 < LOG_OVRFLO=/disk1/ud41/log/log_overflow_dir --- > LOG_OVRFLO=

Notice that one of the parameters systest recommends changing is SHM_GNPAGES. If you want to make this change, make sure your UNIX kernel is configured appro- priately. SHM_GNPAGES * SHM_GPAGESZ * 512 must not exceed the kernel parameter shmmax.

Note: If you run systest -u, the recommended changes in the above example are not made. systest -u changes only the udtconfig parameters NFILES and SHM_ATT_ADD, if necessary.

17-14 Administering UniData on UNIX Checking Kernel Parameters

The -c argument for systest checks kernel settings against UniData recommendations. Use the -n option if you are not running RFS. Use the -r option are running RFS, as shown below:

# systest -cr **** WARNING *** There are 2 kernel parameters that are set lower than what Unidata recommends. The kernel parameters are listed in the table below with their recommended values:

Parameter Recommended Value Current Value ------SHMSEG 50 36 MSGMNI 100 50 Note: The recommended values returned by systest are generic UNIX suggestions and may not be appropriate for your operating system. Kernel configuration varies among UNIX versions. Refer to your host operating system documentation for detailed information about your UNIX kernel. sms

The sms command displays information about use of global and local pages by smm.

Syntax:

sms [-h | -g[n] | -G[n] | -L[n] | -l | -Sn | -d]

You do not need to log on as root to run sms. See the UniData Commands Reference for detailed information about the parameters of the sms command syntax.

Setting Shared Memory Parameters 17-15

sms -h displays the current settings of shared memory-related configuration param- eters, as shown in the following example:

# sms -h Shmid of CTL: 30901 ------IDs ------smm_pid smm_trace PtoM_msgqid MtoP_msgqid ct_semid (values) 24075 0 2650 2651 1692 (1,1,1)

------GENERAL INFO ------SHM_GNTBLS = 50 (max 50 global segments / system) SHM_GNPAGES = 32 (32 global pages / global segment) SHM_GPAGESZ = 256 (128K bytes / global page)

NUSERS = 50 (max 50 process groups / system) SHM_LPINENTS = 10 (max 10 processes / group) SHM_LMINENTS = 32 (max 32 global pages / group) SHM_LCINENTS = 100 (max 100 control entries / group) SHM_LPAGESZ = 8 (4K bytes / local page)

SHM_FREEPCT = 25 SHM_NFREES = 1

SHM_FIL_CNT = 2048 JRNL_BUFSZ = 53248

17-16 Administering UniData on UNIX The following example shows a sample output from the sms command with no options:

# sms ------GCTs (50) ------11502 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1

------LCTs (50) ------24230 24244 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 The following table interprets the example.

Field Description

GCTs (50) The number of shared memory segments the system is configured to support. Read from the configuration parameter SHM_GNTBLS.

LCTs (50) The combined number of udt processes the system is configured to support at one time. Read from the configuration parameter NUSERS.

11502 The shared memory segment ID for a segment that has been created. This number also appears in the ipcstat display. Note: The “GCT number” is read from left to right, top to bottom. In the example, only GCT number 1 is in use.

-1 Indicates a resource that is not currently in use.

24230, 24244 UNIX process IDs (pid) for the udt processes currently active. sms Command Output

Setting Shared Memory Parameters 17-17

Tip: Use the sms display along with the output from gstt and lstt to monitor resource availability. Consider increasing SHM_GNTBLS or NUSERS and rebuilding the kernel if needed, when these utilities indicate your system is consistently running near the limits of resources.

Use the -G option of the sms syntax to display information about the segment controlled by a particular GCT. This option enables you to determine which udt process is using each global page in the segment. The following screen shows an example:

# sms -G 11502 GCT-1 is in use ------HEADER ------shmid freed_npages 11502 30

------PAGE MAP (32) ------24230 24230 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 The following table interprets the results of the example.

Field Description

shmid (11502) The shared memory segment ID.

freed_npages (30) The difference between the number of global pages in the segment and the number of global pages in use. UniData reads the number of pages in the segment (32) from the udtconfig parameter SHM_GNPAGES.

24230,24230 The UNIX process ID of the udt process that is using each global page.

-1 Indicates a resource not currently in use. smsCommand: GCT Output

Learning about Global Pages

The gstt command displays information about the use of global pages in shared memory by the smm daemon.

17-18 Administering UniData on UNIX Syntax:

gstt

The following example shows the output from the gstt command:

# gstt ------GCTs Statistics ------

Total GCTs (GSMs allowed): 50 Pages/GSM...... : 32 (4096K bytes) Bytes/Page...... : 128K bytes

GCTs used (GSMs created).: 1 (2% of 50)

Active GSMs....: 1 (32 pages in total, 4096K bytes)

Pages Used...... : 2 (6%, 256K bytes) Pages Freed...... : 30 (94%, 3840K bytes)

Inactive GSMs..: 0

Pages Freed...... : 0 (0K bytes)

Total Pages Used...... : 2 (6%, 256K bytes) Total Pages Freed.....: 30 (94%, 3840K bytes) Total memory allocated: 4096K bytes

------End of GCTs Statistics ------Tip: Use the output from gstt, along with the visual display from sms, to monitor use of shared memory segments. IBM recommends increasing the number of GCTs (SHM_GNTBLS) if the value of GCTs used is consistently higher than 80%.

Learning about Local Control Tables

The lstt command displays information about local control tables in shared memory. Each local control table tracks resource use for a udt (or sql) process.

Syntax:

lstt [-l n | -L pid]

Setting Shared Memory Parameters 17-19

The following example shows the output from lstt with no options:

# lstt ------LCTs Statistics ------Total LCTs (Process Groups allowed): 50 LCTs Used (Active Process Groups): 1 (2% of 50) Total Ps: 2

Total Global Pages Used: 2 (256K bytes) Total Self-created.....: 0 (0K bytes) Total memory used...... : 256K bytes

------End of LCTs Statistics ------Tip: Use the output from lstt, along with the visual display from sms, to monitor use of local control tables. IBM recommends increasing the number of LCTs (NUSERS) if the value of “LCTs used” is consistently higher than 80%. Also, if “Total Self- created” is consistently greater than zero, consider increasing SHM_GPAGESZ or optimizing your application to minimize use of self-created segments.

Use the -l or -L option to display additional information about a specific local control table. The following screen shows an example:

# lstt -l 1 ------LCTs Statistics ------

Total LCTs (Process Groups allowed): 50

LCTs Used (Active Process Groups): 1 (2% of 50) Total Ps: 2

Total Global Pages Used: 2 (256K bytes) Total Self-created.....: 0 (0K bytes) Total memory used...... : 256K bytes

Statistics for LCT-1 (Leader’s pid: 24230)

PI Entries Used (Processes): 2 (20% of 10) MI Entries Used (LSMs).....: 2 (6% of 32) Global Pages...... : 2 (256K bytes) Self-created...... : 0 (0K bytes) Total memory used...... : 256K bytes

CI Entries Used (BLKs).....: 6 (6% of 100) Local Blocks Used...... : 5 Local Blocks Freed.....: 1

------End of LCTs Statistics ------See the UniData Commands Reference for additional information about the parameters of the lstt syntax.

17-20 Administering UniData on UNIX UNIX Monitoring Tools

The UNIX sar, vmstat, swap, and swapinfo commands provide useful information for memory and swap space management. The availability and syntax of these commands is platform-dependent. Refer to your host operating system documen- tation for information about these commands.

UNIX Monitoring Tools 17-21

UniData Configuration Parameters

Chapter 5, “UniData and Memory,” lists the UniData configuration parameters that control how smm creates and assigns memory structures. These parameters are also listed in Appendix A, “UniData Configuration Parameters.”

When designing your configuration for smm, account for sbcs, the system buffer (if you are using RFS), and other applications on your system.

UNIX Kernel Parameters

Chapter 5, “UniData and Memory,” describes UNIX kernel parameters that control creation and allocation of shared memory structures. An exhaustive list of such parameters is beyond the scope of this manual, since the parameters, their names, and the processes for adjusting them vary for different UNIX versions. Refer to your host operating system and vendor documentation for information specific to your system.

Note: Depending on the UNIX version, some kernel parameters can be defined either as fixed values or by internal calculations performed by the system. In some versions, you can tune the kernel while the system is running, while others require you to reboot to make the changes effective. Some UNIX versions (AIX, for example) handle kernel configuration dynamically and do not offer the capability to change the parameters directly.

17-22 Administering UniData on UNIX UniData Error Messages for smm

This section lists error messages that are received when smm is unable to respond to a request for memory resources. These messages are seen by the requesting process, so there is no central location for them. They may appear when you start UniData or at run time.

Consider the following guidelines when troubleshooting error messages:

„ Always note the full text of the error message, any error numbers associated with the text, and when the error occurred. „ Check the error logs (smm.errlog and sbcs.errlog) for additional information. „ When a message includes an error number (errno), check for the corre- sponding UNIX message in /usr/include/sys/errno.h or the UniData message in your UniData account’s ENGLISH.MSG file. The message text is often necessary for distinguishing among possible problems. „ Consider the context in which the message appears. „ If you are configuring UniData for the first time, the error messages provide information you may need to reset configuration parameters or rebuild the kernel. „ If you have installed new application software (including C routines for CALLC or CallBasic), and you begin to see error messages, review the new additions before reconfiguring UniData or rebuilding the UNIX kernel. Programming errors or coding structure errors may masquerade as shared memory errors; these should be corrected by correcting the software rather than reconfiguring your system. „ If your system has been running smoothly with no recent changes, and you begin to see error messages, identify and resolve external causes (such as swap space occupied by temporary files) before reconfiguring UniData or rebuilding the UNIX kernel. „ Consider the resource needs of other applications running on your system. Your system resources must support UniData and all other applications.

UniData Error Messages for smm 17-23

The following table lists error messages, describes their meaning, and offers sugges- tions for resolving them.

Error Message Description

Error on attaching CTL A process cannot attach CTL (Control Table List). This is a fatal (errno=xxx) error. You may need to stop UniData and attempt to restart it. Be sure to save all logs and error logs.

Error on attaching shm A process cannot attach a shared memory segment. The process (xxx, xxx, xxx), has asked for a segment larger than the system maximum or has errno=xxx exceeded the per-process limit for shared memory segments. Increase UNIX kernel parameters (shmmax, shmmni, and shmseg) and/or increase the UniData configuration parameter SHM_GNTBLS.

Error on creating CTL UniData cannot create the CTL. This happens when the size of (errno=xxx) CTL is larger than the maximum size of a shared memory segment. You can increase the maximum size of a shared memory segment (shmmax in the UNIX kernel and the configu- ration parameter SHM_MAX_SIZE), or decrease the size of CTL by decreasing the configuration parameters SHM_GNTBLS and NUSERS.

Error on creating a A shared memory segment of the requested size cannot be shared memory segment created. Typically the requested size is larger than the maximum (size=xxx), errno=xxx size on the system. Adjust the UNIX kernel shmmax and the UniData configuration parameter SHM_MAX_SIZE to increase the maximum size of a shared memory segment.

Error on forming shared Some shared memory segments are created using information in memory key files in the directory /usr/ud61/include. Check to be sure (errno=xxx) /usr/ud72/include exists; check permissions; restore the path from backup if it has been removed.

No more GCTs You are out of GCTs (Global Control Tables), which means you already have as many segments (self-created segments and smm segments) as your system currently supports. Consider increasing shmmni in the UNIX kernel, or increasing the UniData configuration parameter SHM_GNTBLS.

No more LCTs You are out of LCTs (Local Control Tables). Consider increasing the UniData configuration parameter NUSERS. Make sure the kernel parameter semmnu is larger than NUSERS. Error Messages Associated with smm

17-24 Administering UniData on UNIX Error Message Description

No more core You are out of main memory. Check your swap space; check recent software changes for inappropriate memory handling; increase swap space or add more physical memory to your system.

No more entries in CI The CI table in the specified LCT is full. A process has used its table in LCT-xxx limit of local sections. A local section is a local page or several contiguous local pages. Consider increasing the UniData config- uration parameter SHM_LCINENTS.

No more entries in MI The MI table in the specified LCT is full. A process has used its table in LCT-xxx limit of global pages. Consider increasing the size of a global page (SHM_GPAGESZ) or the number of global pages per process (SHM_LMINENTS).

No more entries in PI The PI table in the specified LCT is full. Your application has too table in LCT-xxx many forked processes. Your application may not be structured in the correct manner. Consider increasing the UniData configu- ration parameter SHM_LPINENTS.

No more shared memory You are out of shared memory ids. Adjust the UNIX kernel ids parameter shmmni to increase the limit. smm can’t get the first smm cannot acquire the first shared memory segment to build GSM errno = 22 the necessary control tables because shmmax is not large enough. Increase the kernel parameter shmmax.

Error on malloc a space Memory allocation error. The requested size is too large. Install (size=xxx), errno=xxx more physical memory or increase swap space. Error Messages Associated with smm (continued)

UniData Error Messages for smm 17-25 Chapter Managing ipc Facilities 18

Message Queues, Shared Memory, and Semaphores...... 18-3 UniData Log Files ...... 18-6 Removing ipc Structures ...... 18-7 This chapter describes commands and procedures that monitor the use of message queues and semaphores, and describes how to clear message queues and remove queues when necessary to correct problems. This chapter includes some instructions for monitoring shared memory use; however, shared memory is described more fully in Chapter 17, “Managing Memory.”

18-2

Message Queues, Shared Memory, and Semaphores

The UniData system-level ipcstat command displays a list of message queues, shared memory segments, and UNIX system semaphores currently in use on your system. Output from ipcstat resembles that from the UNIX ipcs command, but the ipcstat display also identifies the facilities in use by UniData.

Syntax:

ipcstat [-s | -m | -q]

The following table describes each parameter of the syntax.

Parameter Description

[-q] Displays information about message queues only.

[-m] Displays information about shared memory segments only.

[-s] Displays information about UNIX system semaphores only. ipcstat Parameters

Entering ipcstat with no options displays information about queues, semaphores, and shared memory segments.

Note: The output from ipcstat provides queue numbers, semaphore numbers, and segment numbers. You need these to research ipc problems. For example, you need the queue numbers to identify and clear congested message queues. Tip: The ipcstat output is also useful for troubleshooting situations where UniData has crashed and restart fails because one or more message queues are left. Use ipcstat to identify these and remove them with the udipcrm command before restarting UniData.

You do not need to log on as root to run ipcstat.

18-3 Administering UniData on UNIX The following example shows sample output from the ipcstat command:

# ipcstat IPC status from /dev/kmem as of Tue Jul 23 16:05:15 2004 T ID KEY MODE OWNER GROUP Message Queues: q 0 0x3c1c0619 -Rrw--w--w- root root -> unknown q 1 0x3e1c0619 --rw-r--r-- root root -> unknown q 2 0xacea0207 -Rrw-rw-rw- root 7721280 -> unknown q 53 0x00000000 -Rrw-rw-rw- root sys -> unknown q 54 0x00000000 --rw-rw-rw- root sys -> unknown q 55 0x00000000 -Rrw-rw-rw- root sys -> unknown q 56 0x00000000 --rw-rw-rw- root sys -> unknown q 57 0x00000000 --rw-rw-rw- root sys -> unknown q 258 0x00000000 -Rrw-rw-rw- root sys -> cm R6.1 q 259 0x00000000 --rw-rw-rw- root sys -> udt R6.1 q 260 0x00000000 --rw-rw-rw- root sys -> tm R6.1 q 261 0x00000000 --rw-rw-rw- root sys -> udt R6.1 q 262 0x00000000 --rw-rw-rw- root sys -> tm R6.1 q 263 0x00000000 --rw-rw-rw- root sys -> udt R6.1 q 264 0x00000000 --rw-rw-rw- root sys -> tm R6.1 q 215 0x00000000 --rw-rw-rw- root sys -> udt R6.1 q 216 0x00000000 --rw-rw-rw- root sys -> tm R6.1 q 217 0x00000000 --rw-rw-rw- root sys -> udt R6.1 q 218 0x00000000 --rw-rw-rw- root sys -> tm R6.1 q 219 0x00000000 --rw-rw-rw- root sys -> udt R6.1 q 220 0x00000000 --rw-rw-rw- root sys -> tm R6.1 q 221 0x00000000 --rw-rw-rw- root sys -> udt R6.1 q 222 0x00000000 --rw-rw-rw- root sys -> tm R6.1 q 223 0x00000000 --rw-rw-rw- root sys -> udt R6.1 q 224 0x00000000 --rw-rw-rw- root sys -> tm R6.1 q 225 0x00000000 --rw-rw-rw- root sys -> udt R6.1 q 226 0x00000000 --rw-rw-rw- root sys -> tm R6.1 q 227 0x00000000 --rw-rw-rw- root sys -> udt R6.1 q 228 0x00000000 --rw-rw-rw- root sys -> tm R6.1 q 229 0x00000000 -Rrw-rw-rw- root sys -> smm R6.1 (request) q 230 0x00000000 --rw-rw-rw- root sys -> smm R6.1 (reply) q 231 0x00000000 -Rrw-rw-rw- root sys -> sbcs R6.1 (tosbcs) q 232 0x00000000 --rw-rw-rw- root sys -> sbcs R6.1 (fromsbcs) q 233 0x00000000 --rw-rw-rw- root sys -> sbcs R6.1 (newversion) q 484 0x00000000 --rw-rw-rw- root sys -> sm R6.1 Shared Memory: m 0 0x2f100002 --rw------root sys -> unknown m 1 0x411c02e0 --rw-rw-rw- root root -> unknown m 2 0x4e0c0002 --rw-rw-rw- root root -> unknown m 3 0x4120082d --rw-rw-rw- root root -> unknown m 4 0xaceca000 --rw-rw-rw- root root -> unknown m 1205 0x451c415e --rw-rw-rw- root sys -> unknown m 206 0x00000000 --rw-rw-rw- root sys -> unknown

Message Queues, Shared Memory, and Semaphores 18-4

m 207 0x00000000 --rw-rw-rw- root sys -> unknown m 208 0x00000000 --rw-r--r-- root sys -> unknown m 5009 0x451c3e09 --rw-rw-rw- root sys -> smm R5.2 (ctl) m 2010 0x00000000 --rw-rw-rw- root sys -> smm R6.1 (glm) m 1011 0x00000000 --rw-rw-rw- root sys -> smm R6.1 (shmbuf) m 1012 0x00000000 --rw-r--r-- root sys -> sbcs R6.1 m 1013 0x651c3e09 --rw-rw-rw- root sys -> sm R5.2 (ctl) m 814 0x00000000 --rw-rw-rw- root sys -> sm R6.1 (sysbuf) Semaphores: s 0 0x2f100002 --ra-ra-ra- root sys -> unknown s 1 0x411c02e0 --ra-ra-ra- root root -> unknown s 2 0x4e0c0002 --ra-ra-ra- root root -> unknown s 3 0x4120082d --ra-ra-ra- root root -> unknown s 4 0x00446f6e --ra-r--r-- root root -> unknown s 5 0x00446f6d --ra-r--r-- root root -> unknown s 6 0x01090522 --ra-r--r-- root root -> unknown s 71 0x00000000 --ra-ra-ra- root sys -> unknown s 72 0x00000000 --ra-ra-ra- root sys -> unknown s 73 0x00000000 --ra-ra-ra- root sys -> unknown s 74 0x00000000 --ra-ra-ra- root sys -> unknown s 75 0x00000000 --ra-ra-ra- root sys -> unknown s 76 0x00000000 --ra-ra-ra- root sys -> unknown s 77 0x00000000 --ra-ra-ra- root sys -> unknown s 78 0x00000000 --ra-ra-ra- root sys -> unknown s 335 0x00000000 --ra-ra-ra- root sys -> smm R6.1 (latch) s 336 0x00000000 --ra-ra-ra- root sys -> smm R6.1 (latch) s 337 0x00000000 --ra-ra-ra- root sys -> smm R6.1 (latch) s 338 0x00000000 --ra-ra-ra- root sys -> smm R6.1 (latch) s 339 0x00000000 --ra-ra-ra- root sys -> smm R6.1 (latch) s 340 0x00000000 --ra-ra-ra- root sys -> smm R6.1 (ctl) s 341 0x00000000 --ra-ra-ra- root sys -> smm R6.1 (journal) s 342 0x00000000 --ra-ra-ra- root sys -> smm R6.1 (smm/sm sync) s 279 0x00000000 --ra-ra-ra- root sys -> smm R6.1 (super-rls) s 24 0x00000000 --ra-ra-ra- root sys -> unknown s 25 0x00000000 --ra-ra-ra- root sys -> unknown # Note: Resources identified as “unknown” do not indicate a problem. These resources are in use by the operating system or by other applications rather than by UniData daemons.

18-5 Administering UniData on UNIX Notice that, because no options were specified, ipcstat displays information about queues, semaphores, and memory segments.

UniData Log Files

When you start UniData, the smm and sbcs daemons record in their logs (smm.log and sbcs.log) information about the ipc facilities they are using. See Chapter 9, “Starting, Stopping, and Pausing UniData,” for examples of these log files.

Note: Occasionally, UniData problems result from another process inadvertently removing one of the UniData message queues. You can compare the log files with ipcstat output to find out if this is the cause of a hang or crash. If a queue is removed, the initial list from the appropriate log includes the queue, but ipcstat does not include the queue.

Message Queues, Shared Memory, and Semaphores 18-6

Removing ipc Structures

Certain types of system failures cause ipc facilities associated with UniData to be left after UniData has been shut down. This can occur if the system crashes or if one of the daemons is inadvertently killed. In such cases, restarting UniData fails because of the remaining ipc structures. You may see symptoms like the following:

„ The startud command fails. „ A message displays in the startud window indicating smm is still running. „ Executing showud indicates smm is not running.

If you encounter these symptoms, complete the following steps:

1. Check for Remaining Facilities

Enter the UniData ipcstat command at the UNIX prompt. If the output shows struc- tures associated with UniData processes, execute showud to see if any UniData daemons are running.

„ If there are daemons running, proceed to step 2. „ If there are ipc facilities, but no daemons, proceed to step 3. „ If there are no UniData daemons running and no ipc facilities, research other causes for the startup failure. Tip: Occasionally icpstat fails to complete. You can obtain the information you need by executing the UNIX ipcs command and comparing the output with smm.log and sbcs.log to identify UniData structures.

2. Stop UniData

If showud indicates that none of the UniData daemons is running, proceed to step 3. Otherwise, execute the stopud command. This command stops the daemons appro- priately. Then proceed to step 3.

3. Decide How to Proceed

Use the UniData udipcrm command (step 4) or the UNIX ipcrm command (step 5).

18-7 Administering UniData on UNIX 4. Remove ipc Facilities with udipcrm

Log on as root, and enter the udipcrm command at a UNIX prompt. This command removes all ipc facilities associated with UniData processes. The following screen shows the output from udipcrm:

# $UDTBIN/udipcrm ipcrm: msqid(1106): not found ipcrm: msqid(1107): not found ipcrm: msqid(1108): not found ipcrm: msqid(1109): not found ... ipcrm: shmid(4308): not found ipcrm: shmid(2709): not found ipcrm: shmid(513): not found ipcrm: shmid(414): not found ipcrm: semid(708): not found # The “not found” messages appear because resources forced out by udipcrm try to send messages to ones that are already gone.

After successfully completing udipcrm, you should be able to restart UniData. Proceed to step 6; you do not need to complete step 5.

5. Remove ipc Facilities with UNIX ipcrm

The UNIX ipcrm command removes specific ipc facilities by specifying their identi- fiers. For this reason, ipcrm is easy to use if you are removing only a few facilities.

Refer to your host operating system documentation for detailed information about the ipcrm command. You must log on as root to execute it. You need the output from ipcstat to identify the resources to remove. Note the type (column 1 of ipcstat output; must be m, q, or s) and the ID (column 2 of ipcstat output).

Removing ipc Structures 18-8

The following screen shows an example of ipcstat - q output:

# $UDTBIN/ipcstat -q IPC status from /dev/kmem as of Wed Jul 24 09:54:34 2002 T ID KEY MODE OWNER GROUP Message Queues: q 0 0x3c1c0619 -Rrw--w--w- root root -> unknown q 1 0x3e1c0619 --rw-r--r-- root root -> unknown q 2 0xacea0207 -Rrw-rw-rw- root 7730112 -> unknown q 53 0x00000000 -Rrw-rw-rw- root sys -> unknown q 54 0x00000000 --rw-rw-rw- root sys -> unknown q 55 0x00000000 -Rrw-rw-rw- root sys -> unknown q 56 0x00000000 --rw-rw-rw- root sys -> unknown q 57 0x00000000 --rw-rw-rw- root sys -> unknown q 258 0x00000000 -Rrw-rw-rw- root sys -> cm R6.1 q 259 0x00000000 --rw-rw-rw- root sys -> udt R6.1 q 260 0x00000000 --rw-rw-rw- root sys -> tm R6.1 q 261 0x00000000 --rw-rw-rw- root sys -> udt R6.1 q 262 0x00000000 --rw-rw-rw- root sys -> tm R6.1 q 263 0x00000000 --rw-rw-rw- root sys -> udt R6.1 q 264 0x00000000 --rw-rw-rw- root sys -> tm R6.1 q 215 0x00000000 --rw-rw-rw- root sys -> udt R6.1 q 216 0x00000000 --rw-rw-rw- root sys -> tm R6.1 q 217 0x00000000 --rw-rw-rw- root sys -> udt R6.1 q 218 0x00000000 --rw-rw-rw- root sys -> tm R6.1 q 219 0x00000000 --rw-rw-rw- root sys -> udt R6.1 q 220 0x00000000 --rw-rw-rw- root sys -> tm R6.1 q 221 0x00000000 --rw-rw-rw- root sys -> udt R6.1 q 222 0x00000000 --rw-rw-rw- root sys -> tm R6.1 q 223 0x00000000 --rw-rw-rw- root sys -> udt R6.1 q 224 0x00000000 --rw-rw-rw- root sys -> tm R6.1 q 225 0x00000000 --rw-rw-rw- root sys -> udt R6.1 q 226 0x00000000 --rw-rw-rw- root sys -> tm R6.1 q 227 0x00000000 --rw-rw-rw- root sys -> udt R6.1 q 228 0x00000000 --rw-rw-rw- root sys -> tm R6.1 q 229 0x00000000 -Rrw-rw-rw- root sys -> smm R6.1 (request) q 230 0x00000000 --rw-rw-rw- root sys -> smm R6.1 (reply) q 231 0x00000000 -Rrw-rw-rw- root sys -> sbcs R6.1 (tosbcs) q 232 0x00000000 --rw-rw-rw- root sys -> sbcs R6.1 (fromsbcs) q 233 0x00000000 --rw-rw-rw- root sys -> sbcs R6.1 (newversion) q 484 0x00000000 --rw-rw-rw- root sys -> sm R6.1 # Warning: Exercise extreme caution when removing ipc resources. Removing the wrong ones will cause problems elsewhere on the system.

6. Restart UniData

Once you remove the ipc facilities that were left over, you should be able to restart UniData with the startud command. UniData should restart normally.

18-9 Administering UniData on UNIX Note: If UniData will not start, repeat steps 1 through 6. If UniData still will not start, the problem is unrelated to ipc facilities. Examine the error logs in udtbin (smm.errlog and sbcs.errlog) and resolve all indicated error conditions.

Removing ipc Structures 18-10 Chapter Managing Cataloged Programs 19

UniBasic Source and Compiled Programs ...... 19-3 UniBasic Compiled Programs...... 19-3 Cataloging UniBasic Programs ...... 19-4 Direct Cataloging ...... 19-4 Local Cataloging ...... 19-4 Global Cataloging ...... 19-5 Managing Global Catalogs...... 19-7 Contents of a Global Catalog ...... 19-7 Verifying a Program Version ...... 19-8 Listing Programs in Use ...... 19-14 Creating an Alternate Global Catalog Space ...... 19-15 Files and Directories Created by newhome ...... 19-15 Procedure for Creating an Alternate Global Catalog Space. . . . . 19-15 This chapter describes the behavior of global, direct, and local cataloging for UniBasic programs. It also describes how to create an alternate global catalog space using the newhome command.

19-2

UniBasic Source and Compiled Programs

UniData stores UniBasic source code in DIR-type files in the UniData account where the source is developed. The default location for storing UniBasic source code files is the BP file, which UniData creates as an empty file when you create a UniData account. Developers store UniBasic source code files as records in the BP file.

Note: In a UniData DIR-type file, like BP, each “record” is a UNIX file.

Each UniData account may contain numerous DIR files for UniBasic source.

UniBasic Compiled Programs

When you issue the BASIC command to compile a UniBasic program, UniData stores the compiled code in the same DIR file where the source code resides. The compiled code is a record whose name is the same as the source record, prefixed with an underscore.

See the UniData Commands Reference and Developing UniBasic Applications for information about the BASIC command.

The following example shows the contents of a program file:

:LIST BP TEST _TEST TEST2 _TEST2 EXAMPLE3 _EXAMPLE3 EXAMPLE5 _EXAMPLE5 8 records listed Records beginning with an underscore are compiled programs. Other records are UniBasic source files.

Tip: Use the ECL RUN command to execute a compiled program. Refer to the UniData Commands Reference and Developing UniBasic Applications for information about the RUN command.

19-3 Administering UniData on UNIX Cataloging UniBasic Programs

Cataloging UniBasic programs simplifies program execution and can improve efficiency of system resource use by allowing multiple users to access a single copy of a compiled program from shared memory. Use the ECL CATALOG command to catalog one or more UniBasic programs.

Note: See the UniData Commands Reference and Developing UniBasic Applications for information about cataloging and the CATALOG command.

Compiled UniBasic programs can be cataloged directly, locally, or globally.

Direct Cataloging

Points to remember about direct cataloging are:

„ Compiled code is located in the program file in the UniData account where the program was compiled and cataloged. „ The VOC file in the account contains a pointer to the compiled code in the program file. Users in the same account can execute the program by entering the program name at the ECL prompt. „ Because users access the compiled code in the program file, developers do not need to recatalog the code if they recompile. „ When a user executes a directly cataloged program, UniData loads a copy of the program into the address space of the user.

Local Cataloging

Points to remember about local cataloging are:

„ Compiled code is located in the CTLG directory in the UniData account where the program was cataloged, as well as in the program file. CTLG is a DIR-type UniData file, and each record is a compiled UniBasic program. „ The VOC file in the account contains a pointer to the compiled program in the CTLG. Users in the same account can execute the program by entering the program name at the ECL prompt. „ Developers must recatalog a program after recompiling to place a new copy of the compiled code into the CTLG.

Cataloging UniBasic Programs 19-4

„ When a user executes a locally cataloged program, UniData loads a copy of the program into the address space of the user.

Global Cataloging

Points to remember about global cataloging are:

„ If you execute the CATALOG command without specifying local or direct cataloging, your program is globally cataloged. „ Compiled code is located in a systemwide global catalog. The default global catalog is udthome/sys/CTLG. „ Developers must recatalog a program after recompiling it to place a new copy of the compiled code into the global catalog. Note: A UniData installation can have more than one global catalog space. The environment variable UDTHOME determines which global catalog space a particular UniData session accesses. See “Creating an Alternate Global Catalog Space” on page 19-15 for more information about creating multiple global catalog spaces. „ A systemwide global catalog is a DIR-type file, with 26 subdirectories named a through z. Compiled code is located in the subdirectory corre- sponding to the first letter of the program name. Compiled programs that begin with nonalphabetic characters are stored in a subdirectory named X. The cataloged program name can be the same as the source and object, or you can specify a different name when you execute CATALOG. Tip: Consider your program naming conventions if you are using global cataloging. Since the compiled code is placed in subdirectories according to name, you may have an unbalanced situation if a large number of your program names begin with the same letter (for instance, a general ledger application where all the files begin with “gl”). „ A globally cataloged program is available to users in all UniData accounts. „ When you execute a globally cataloged program, the shared basic code server (sbcs) checks to see if a copy already exists in the shared memory it controls. „ If so, sbcs notifies the udt process which shared memory segment to attach to access that copy. „ If not, sbcs loads a copy into one of its shared memory segments for the user to execute.

19-5 Administering UniData on UNIX „ The sbcs daemon handles any object file located in the udthome/sys/CTLG file system, regardless of how the program is accessed. „ The sbcs daemon can manage up to 20 shared memory segments for globally cataloged programs. The size of each segment is determined by the SBCS_SHM_SIZE parameter in the UniData configuration file (/usr/ud61/include/udtconfig). The default value for SBCS_SHM_SIZE is 1,048,576 bytes (1 MB), which is set when you install UniData. Users encounter run-time errors if this size is insufficient. You can increase the segment size as long as you do not exceed the configuration parameter SHM_MAX_SIZE. „ For operating systems which impose limits on the number of shared memory segments (such as AIX, which allows only 10), users should increase SBCS_SHM_SIZE. Typical values on AIX systems range from 4 MB to 8 MB. Tip: Refer to Appendix A, “UniData Configuration Parameters,” for additional information about SBCS_SHM_SIZE and SHM_MAX_SIZE.

Cataloging UniBasic Programs 19-6

Managing Global Catalogs

UniData provides a group of files and commands that manage global catalogs. These files and commands accomplish the following:

„ Identify the contents of a global catalog space „ Verify consistency between UniBasic source and a globally cataloged program „ Activate newly cataloged programs and subroutines „ Display use of globally cataloged programs

Contents of a Global Catalog

UniData maintains two files that store contents of a global catalog. The global catalog table, called CTLGTB, is a dynamically maintained file that shows the current contents of the global catalog. You can list the catalog table from a UniData account, as shown in the following example:

:LIST CTLGTB ALL 10:28:53 Feb 08 2005 1 CATALOG NAME...... T ARG ORIGINATOR...... WHO....

SCHEMA_UPDATE_PRIVILEGES S 5 @UDTHOME/SYS_BP SCHEMA root _UPDATE_PRIVILEGES SCHEMA_LIST_USERS S 3 @UDTHOME/SYS_BP SCHEMA root _LIST_USERS SCHEMA_VIEW_CHECK S 7 @UDTHOME/SYS_BP SCHEMA root _VIEW_CHECK US_EXECUTOR S 5 @UDTHOME/SYS_BP US_EXE root CUTOR BUILD.CHARTBL S 0 @UDTHOME/DENAT_BP BUIL root D.CHARTBL 508E S 4 @UDTHOME/SYS_BP 508E root RPROG2_AE S 1 @UDTHOME/AE_BP RPROG2_ root AE JRNL_TEST M 0 @UDTHOME/SYS_BP JRNL_T root EST SCHEMA_DELETE_MAP S 4 @UDTHOME/SYS_BP SCHEMA root _DELETE_MAP NFA.EXECSEL.U S 3 @UDTHOME/SYS_BP NFA.EX root ECSEL.U 70E0 S 4 @UDTHOME/SYS_BP 70E0 root. . .

19-7 Administering UniData on UNIX The _MAP_ file also contains information about the contents of a global catalog. In addition to the information in CTLGTB, _MAP_ includes the size of each compiled program, the date it was cataloged, and the last date it was executed. The _MAP_ file is not dynamically maintained by UniData. The ECL MAP command updates the _MAP_ file to reflect recent activity. The MAP command clears the _MAP_ file, updates the file, and displays its contents, as shown in the following example:

:MAP MAP 10:30:55 Feb 08 2005 1 NAME...... TYPE ARG ORIGINATOR...... WHO.... OBJ... DATE.... LAST REF

508E S 4 @UDTHOME/SYS_BP 508E root 184 01/14/05 01/20/05 COUNT.MSG S 3 @UDTHOME/DENAT_BP CO root 582 01/14/05 01/20/05 UNT.MSG SORT_AE S 1 @UDTHOME/AE_BP SORT_ root 1650 01/14/05 01/20/05 AE 7201 S 4 @UDTHOME/SYS_BP 7201 root 180 01/14/05 01/20/05 NFA.EXECSEL.U S 3 @UDTHOME/SYS_BP NFA. root 154 01/14/05 01/20/05 EXECSEL.U S_VALID_FILE_CHE S 6 @UDTHOME/SYS_BP S_VA root 1712 01/14/05 01/20/05 CK LID_FILE_CHECK ERRMSG.COMMON M 0 @UDTHOME/DENAT_BP ER root 92 01/14/05 01/20/05 RMSG.COMMON RPROG_AE S 1 @UDTHOME/AE_BP RPROG root 4708 01/14/05 01/20/05 _AE SCHEMA_OBJECT_TY S 4 @UDTHOME/SYS_BP SCHE root 1914 01/14/05 01/20/05 PE MA_OBJECT_TYPE S_VALID_NAME_CHE S 3 @UDTHOME/SYS_BP S_VA root 2184 01/14/05 01/20/05 CK LID_NAME_CHECK SCHEMA_REMOVE_SC S 3 @UDTHOME/SYS_BP SCHE root 1364 01/14/05 01/20/05 HEMA MA_REMOVE_SCHEMA . . . By default, the CTLGTB file and the _MAP_ file are located in the same directory as the global catalog: udthome/sys.

Tip: The CTLGTB file and the _MAP_ file are UniData hashed files. You can display records in these files with standard ECL and UniQuery commands to determine if particular programs are in the global catalog.

Verifying a Program Version

The VCATALOG command checks the date/time stamp of a UniBasic source file against the compiled program in the global catalog. If the UniBasic source file was modified after the program was cataloged, the program does not verify.

Syntax:

VCATALOG filename catalog.name program.name

Managing Global Catalogs 19-8

The following table describes each parameter of the syntax.

Parameter Description

filename Name of the file containing the program (BP, for instance).

catalog.name Name given to the program when you executed CATALOG. For example, the command CATALOG BP TRIAL TEST creates a global catalog entry named TRIAL from a program called TEST. So catalog.name is TRIAL.

program.name Name of the program source file. In the example in the previous row of this table, program.name is TEST. VCATALOG Parameters Note: If catalog.name and program.name are the same, you need only supply the name once. The following example shows output from VCATALOG: :VCATALOG BP TEST Program 'TEST' does not verify. :CATALOG BP TEST /usr/ud72/sys/CTLG/t/TEST has been cataloged, do you want to overwrite?(Y/N)Y :VCATALOG BP TEST Program 'TEST' does not verify. :BASIC BP TEST

Compiling Unibasic: BP/TEST in mode 'u'. compilation finished :CATALOG BP TEST /usr/ud61/sys/CTLG/t/TEST has been cataloged, do you want to overwrite?(Y/N) Y :VCATALOG BP TEST Program 'TEST' verifies. : In the example, notice that recataloging the program did not make the program verify. This result indicates that the source code was changed, but was not recompiled or recataloged. After the source code was recompiled and recataloged, the program verified successfully.

Refer to the UniData Commands Reference for additional information about the VCATALOG command. Activating Newly Cataloged Programs and Subroutines

19-9 Administering UniData on UNIX This section describes how to activate newly cataloged programs and subroutines.

Main Programs

When you globally catalog a UniBasic main program, UniData:

„ Copies the new compiled code into the global catalog. „ If there is a version of the program in shared memory, marks that version as obsolete.

The users already executing the main program continue to execute the previous version. Users that execute the program after the new version is cataloged get the new version. Once all users exit the previous version, UniData removes the copy of that version from shared memory.

Note: A user executing a main program continues to execute that version until it completes.

Subroutines

If a subroutine is recataloged while the main program is running, users will not execute the newly-cataloged subroutine until the next time they execute the main program. This prevents inconsistent execution of a subroutine during one execution of the main program. Under certain circumstances, however, a user or system admin- istrator can override the default behavior. Overrides are dangerous in a production environment, but may be useful in a development or test environment.

NEWVERSION Keyword

The NEWVERSION keyword for the CATALOG command enables a user logged on as root to dynamically replace a globally cataloged subroutine. If a subroutine is cataloged with NEWVERSION, any user executing the main program accesses the new version of the subroutine with the next CALL or EXECUTE of the subroutine, rather than waiting until the main program completes. Consider the following sequence of events:

1. A user executes the main program MAIN. 2. MAIN calls a subroutine called SUBR, which completes and returns to MAIN. 3. MAIN continues with other processing.

Managing Global Catalogs 19-10

4. MAIN calls SUBR again. SUBR completes and returns to MAIN. 5. MAIN completes.

If SUBR is recataloged after step 1 without the NEWVERSION keyword, the same version of SUBR is used for both calls (step 2 and step 4). With the next execution of MAIN, the newly cataloged SUBR is used.

If SUBR is recataloged after step 1, with the NEWVERSION keyword, then there are three possible results:

„ CATALOG happens after step 1 but before step 2. In this case, the newly cataloged SUBR gets accessed in both step 2 and step 4. „ CATALOG happens after step 2, but before step 4. In this case, the prior version of SUBR gets accessed in step 2, and the newly cataloged version gets accessed in step 4. „ CATALOG happens after step 4. In this case, the prior version gets accessed in both step 2 and step 4. With the next execution of MAIN, the newly cataloged SUBR is accessed. Warning: Using the NEWVERSION keyword to CATALOG a subroutine can produce inconsistent results for users who are currently executing the main program. For example, the number of arguments could change.

The following sample CATALOG command shows the syntax including the NEWVERSION keyword:

:CATALOG BP SUBR NEWVERSION /usr/ud72/sys/CTLG/s/SUBR has been cataloged, do you want to overwrite?(Y/N) Y :

newversion System-Level Command

The UniData system-level command newversion enables a user logged on as root to dynamically replace a cataloged subroutine (just as the NEWVERSION keyword does), but limits the behavior to a selected user or users.

Syntax:

newversion path userno...

19-11 Administering UniData on UNIX The following table describes each parameter of the syntax.

Parameter Description

path Absolute path of the cataloged subroutine.

userno... UNIX process ID (pid) for a user who should access the new subroutine dynamically. You can specify more than one userno; separate the numbers with spaces. newversion Parameters

The following screen shows an example of the newversion command:

:LISTUSER

Licensed/Effective # of Users Udt Sql Total ~~~~~~~~~~~~~~~~~~~ ~~~ ~~~ ~~~~~ 32 / 32 3 0 3

UDTNO USRNBR UID USRNAME USRTYPE TTY TIME DATE 1 10747 0 root udt pty/ttyv1 11:04:46 Jun 10 2000 2 10763 1283 user01 udt pty/ttyv0 11:06:16 Jun 10 2000 3 10846 1172 user02 udt pty/ttyv3 11:16:56 Jun 10 2000

:CATALOG BP SUBR /usr/ud61/sys/CTLG/s/SUBR has been cataloged, do you want to overwrite?(Y/N) Y : :!$UDTBIN/newversion /usr/ud60/sys/CTLG/s/SUBR 10846 In the example, the newly cataloged subroutine is dynamically available to user02, the owner of pid 10846. If user02 is executing a main program that calls a subroutine, the next call to the subroutine accesses the newly cataloged version. For all users other than user02, the default behavior remains in effect. The newly cataloged subroutine is activated with the next execution of the main program, not the next subroutine call. Notice that, in the example, the subroutine is globally cataloged; this command works with locally or directly cataloged routines as well.

NEWPCODE Command

The ECL NEWPCODE command dynamically activates a cataloged subroutine. This command is useful if a developer uses a UniBasic shell program to modify, recompile, recatalog, and retest a UniBasic program without exiting to ECL.

Syntax:

NEWPCODE path

Managing Global Catalogs 19-12

path is the absolute path of a cataloged subroutine. The following example shows one use of the NEWPCODE command executed in a UniBasic program:

* PROGRAM MAINPROG * NEWPCODE EXAMPLE EXECUTE "MAINPROG2" EXECUTE "BASIC BP MAINPROG2" EXECUTE "CATALOG BP MAINPROG2 DIRECT" EXECUTE "NEWPCODE /usr/ud72/sys/CTLG/m/MAINPROG2" EXECUTE "MAINPROG2" END In the example, a user executing MAINPROG accesses the current version of MAINPROG2 in the first statement. Including the NEWPCODE command before the next execution of MAINPROG2 causes the program to access the newest version. (In the example, MAINPROG2 was recompiled and recataloged after the first step, so the next execution accesses the newly cataloged MAINPROG2.)

Tip: If you are developing programs with the AE editor, the N option of the FI command equates to the NEWPCODE command. For example, FIBCFN compiles a program and catalogs it (locally) with NEWPCODE. You need to use F (force) in conjunction with the N option. Refer to the online help for the AE editor or Developing UniBasic Applications for more information. Note: The NEWPCODE command is effective only in the udt session where it is executed. Although NEWPCODE is an ECL command, a user cannot affect a different user or even a different window with NEWPCODE.

19-13 Administering UniData on UNIX Listing Programs in Use

The UniData system-level sbcsprogs command enables any user with access to a UNIX shell prompt to display a list of globally cataloged programs that are currently in use, with counters for the number of processes currently accessing each one. The following screen shows typical output from sbcsprogs:

# sbcsprogs

Program Name Reference Count

/usr/ud61/sys/CTLG/a/AE 2 /usr/ud61/sys/CTLG/a/AE_ICOM1 1 /usr/ud61/sys/CTLG/a/AE_AE 2 /usr/ud61/sys/CTLG/a/AE_UPS 1 In the example, two users are executing AE, and two are executing AE_AE. The sbcs daemon maintains the counter, incrementing it as users execute a program and decreasing it as users complete execution. When the counter for a routine reaches zero, sbcs removes the copy of the compiled program from shared memory, making the space available for other programs as needed.

Tip: If you run sbcsprogs regularly throughout your processing cycle, you can learn which programs are used most heavily. This information is useful if you are troubleshooting an application performance problem. Note: The reference counter is not decremented when a user terminates abnormally (for example, when a process is killed). Because of this, the count may be inaccu- rately high, causing excess memory to remain held. Stopping and restarting UniData resets the counter and releases memory.

Listing Programs in Use 19-14

Creating an Alternate Global Catalog Space

The system-level newhome command creates a new UniData account containing an alternate global catalog space for use in development and testing.

Files and Directories Created by newhome

UniData creates or overlays the directory indicated by path. This directory contains only the subdirectory sys, which contains the following files and directories:

# ls @README DENAT_BP D_HELP.FILE MULTIBYTE.CONFIG @README-IMPORTANT DICT.DICT D_JAPANESE.MSG SAVEDLISTS @VERSIONS D_AE_BP D_MSG.DEFAULTS SYS_BP AE_BP D_AE_COMS D_SAVEDLISTS UDTSPOOL.CONFIG AE_COMS D_AE_COMS_DEMO D_SYS_BP VOC AE_COMS_DEMO D_AE_DOC D_UDT_GUIDE X_HELP.FILE AE_DOC D_AE_XCOMS D_VOC _MAP_ AE_SECURITY D_BP D__MAP_ _PH_ AE_SYSTOOLS D_COLLATIONS D__PH_ core AE_UPCHARS D_CTLG ENGLISH.MSG install.log AE_XCOMS D_CTLGTB ENGLISH_G2.MSG makefile BP D_DENAT_BP FRENCH.MSG set_sys.sh COLLATIONS D_ENGLISH.MSG HELP.FILE udtpath CTLG D_ENGLISH_G2.MSG JAPANESE.MSG uniapi.msg CTLGTB D_FRENCH.MSG LANGGRP vocupgrade The following directories make up the program catalog spaces:

„ D_CTLGTB „ CTLGTB „ D_CTLG „ CTLG, including subdirectories a through z and X for storing globally cataloged programs.

newhome does not create the entire directory structure that exists in the default UniData home directory, and it does not copy UniBasic executables developed at your site.

Procedure for Creating an Alternate Global Catalog Space

Follow the steps below to create an alternate global catalog space:

19-15 Administering UniData on UNIX 1. Change to the New Account Directory

At the operating system prompt, change to the directory in which you intend to locate the new UniData account, as in the following example:

# cd /home/claireg

2. Execute newhome

Execute the newhome command, indicating the path to the new account. In this example, a new UNIX directory, testenv, will be created under /home/claireg. (If the udtbin directory is in your path, you do not have to precede the command with udtbin.)

Notice that the newhome command is executed from the ECL prompt, and therefore is preceded by the ! ECL command:

:!$UDTBIN/newhome testenv

Creating new udt home /home/claireg/demo/testenv ...

The new UDTHOME is established, now only the sys directory is there and it contains all the UniData system cataloged programs, such as AE editor, NFA sever, etc.. To make it be effective, the environment variable UDTHOME needs to be set to point to the new UDTHOME.

3. Modify VOC Entries for Users

Decide which UniData accounts should access the new global catalog space. For each account, modify the VOC entry for CTLGTB. The entry should point to the new global catalog space, as shown in the following example.

Creating an Alternate Global Catalog Space 19-16

Notice that this example uses a soft pointer to @UDTHOME. This ensures that the VOC always points to the correct catalog space. Refer to Chapter 3, “UniData and the UNIX File System,” for information about setting soft pointers in VOC entries.

:AE VOC CTLGTB Top of "CTLGTB" in "VOC", 3 lines, 45 characters. *--: P 001: F 002: @UDTHOME/sys/CTLGTB 003: @UDTHOME/sys/D_CTLGTB Bottom. *--: You do not need to log on as root to edit the VOC entries; however, you need write permissions on the VOC file in each account.

4. Modify UDTHOME for Users

You need to reset the UDTHOME environment variable for each user who needs to access the alternate global catalog space. The value of UDTHOME defined during a particular UniData session determines the global catalog space a user accesses.

Note: Even if the VOC file of an account is set up to point to the alternate global catalog (CTLGTB), a user whose UDTHOME is set to the default value accesses the default global catalog space.

You can modify the UDTHOME variable in a user’s .login or .profile. Simply use vi or any UNIX text editor to change the variable setting. The user must then log out and log back on to make the change effective.

Users with access to a UNIX shell prompt can reset the UDTHOME environment variable with the setenv command. The value set at the UNIX prompt overrides the .login or .profile:

% printenv UDTHOME /disk1/ud72 % setenv UDTHOME /home/carolw/demo/testenv % udt UniData Release 7.2 Build: (3089) Copyright (C) IBM Corporation 2005 All rights reserved.

Current UniData home is /home/carolw/demo/testenv/. Current working directory is /home/carolw/demo.

19-17 Administering UniData on UNIX 5. Copy Application Programs

After resetting UDTHOME to point to the new space, start UniData from an account that has access to the site-specific programs that you want to access from the new account, and recatalog those programs. Because you have reset the UDTHOME environment variable, UniData places the recataloged programs in the new space.

You can also use the following series of UNIX commands to copy all globally cataloged programs to the new catalog space. Be sure to replace original_udthome and new_udthome with the paths to your old and new catalog spaces:

%cd original_udthome/sys/CTLG find * -type f -print | cpio -pm new_udthome/sys/CTLG

6. Use the Alternate Global Catalog Space

The alternate global catalog space is now ready to use. The following example shows the output of the sbcsprogs command when two global catalog spaces are used:

% sbcsprogs

Program Name Reference Count

... 1 /home/carolw/demo/testenv/sys/CTLG/a/AE /home/carolw/demo/testenv/sys/CTLG/a/AE_UPS 1 /disk1/ud72/sys/CTLG/a/AE 1 /disk1/ud72/sys/CTLG/a/AE_ICOM1 1 /disk1/ud72/sys/CTLG/a/AE_UPS 1 ... % Notice that AE is running twice, but that the two copies are cataloged in different global catalog spaces.

Creating an Alternate Global Catalog Space 19-18 Chapter CallBasic, CALLC, and makeudt 20

Linking C Routines into UniData ...... 20-3 Overview ...... 20-3 Requirements ...... 20-3 Rebuilding for Troubleshooting ...... 20-4 makeudt ...... 20-5 File Examples...... 20-5 Creating cfuncdef_user ...... 20-9 Steps for Linking in C Functions ...... 20-10 Steps for Setting Up a Development Environment ...... 20-15 Accessing UniData from a C Program ...... 20-19 Requirements ...... 20-19 How CallBasic Works ...... 20-19 C Program Example ...... 20-21 UniBasic Subroutine Example ...... 20-24 Steps for CallBasic ...... 20-25 UniData provides the makeudt tool and the UniBasic CALLC command for linking C routines into UniData, and the CallBasic tool for linking UniData into C programs. This chapter describes commands and procedures for using these tools. Note: See the UniBasic Commands Reference and Developing UniBasic Applications for information about the syntax and use of the CALLC command.

20-2

Linking C Routines into UniData

Overview

Many applications use C functions for purposes like environment definition, security, or low-level data transfers (similar to the UniBasic GET and SEND functions). UniData allows you to build a customized version of the UniData “kernel” (the udt executable, located in udtbin) that includes C functions developed at your site. Once you build the customized udt, you can access the C functions from within a UniBasic application using CALLC commands.

You can also create a UniData executable with links to C programs so that they are accessible through InterCall, UniObjects, or UniObjects for Java.

Note: When you use CALLC, your C functions are executed by a udt process. They are not visible to the end user at all.

Requirements

The components required to take advantage of this functionality are:

„ Development environment—Your system should have a full software development kit. (A base compiler is not sufficient). You need network libraries if you are using NFA. „ C functions—You need to code and compile the C functions you are linking into UniData. „ Function definitions and make files—When you install UniData, the files base.mk and cfuncdef are installed into the directory udthome/work. You create a file called cfuncdef_user that contains definitions for your site- specific C functions, and you may need to customize the make file. „ UniData commands—You use the UniData system-level commands gencdef, genfunc, genefs, and makeudt, and makeudapi to build your new UniData executable. If you execute the makeudt or makeudapi commands, these commands are automatically executed. If you use the make utility, you need to execute them manually. Note: Refer to your host operating system documentation and your hardware vendor if you have questions about your C development environment.

20-3 Administering UniData on UNIX Rebuilding for Troubleshooting

You may consider rebuilding the UniData executable, even if you are not linking in custom routines, to troubleshoot UniData problems on your system. The udt executable shipped with your product is “stripped” for maximum efficiency, which means that symbol tables used by system debuggers have been removed. You can run makeudt to generate a nonstripped binary to facilitate debugging.

Tip: Rebuilding udt is often unnecessary for researching problems. Do not undertake this step unless you are advised to do so by IBM.

Rebuilding for Application Testing

Although the default behavior of makeudt overwrites the production version of the udt executable, you can configure your system and use the make files to create executables in a testing area separate from production. In this way, you can conduct tests without disrupting user activity, and replace the production udt only when you are satisfied with the functionality.

Linking C Routines into UniData 20-4

makeudt

Syntax:

makeudt [-n nfa]

makeudt is a UniData system-level command that builds a new UniData executable (udt). The command builds the executable using the following:

„ base.mk—This make file is located in udthome/work. This file is a template that UniData uses to create a second make file called new.mk. Then UniData uses new.mk to create the new udt executable. „ cfuncdef—This function definition file is also located in udthome/work. It contains definitions for any C functions that UniData has incorporated into your released udt. Do not modify this file. „ cfuncdef_user—This file contains definitions for site-specific C functions that you want to link into UniData. You need to create this file. „ UniData Libraries—When you install UniData, you are prompted for the path where you want to locate these.

The following table describes the option of the makeudt syntax.

Option Description

-n nfa Use this option only if you are not using UniData OFS/NFA. This option allows makeudt to use “dummy” libraries rather than network libraries NFA requires. Software development environments may or may not include the network libraries; if yours does not include these, and you do not use the -n nfa option, makeudt fails. makeudt Options

Note: It is best to log on as root to execute makeudt. UniData may be up and running, and users may be logged on. If users are logged on, the makeudt command may or may not allow you to overwrite the production udt, depending on your UNIX version. Some versions display an error message and exit, while others prompt whether you wish to overwrite the production udt.

File Examples

The base.mk file and the cfuncdef file are platform-specific.

20-5 Administering UniData on UNIX base.mk Example

Warning: Do not copy the sample makefile onto your system. If you do, makeudt will likely fail. Always start with the base.mk file released with your UniData release.

makeudt 20-6

The following example shows a base.mk file for UniData on an HP system:

# pg base.mk # pg base.mk # # The Porting Date : Jul. 15, 02 # The System to Be Ported : HPUX11 #

CC = cc CFLAGS = -Ae -q -z +ESsfc -w LDFLAGS = OPTFLAGS = -O +Ovolatile DBFLAGS = libpath = -L/liz1/ud60/lib addlib = -lm -lcurses -lsec addlibpath = odslib = -lodsdummy udlib = -lndbm -lcl -lBSD licnlib = -llicn dclcnlib = nfalib = -lnfaclnt dfslib = ODBC_LIBS = -lodbc -lstd -lstream -lCsup -lm -lcl -ldld -Wl,-B deferred - Wl,+b /.udlibs

OBJS = funchead.o interfunc.o callcf.o efs_init.o cpprt0_stub.o

cpprt0_stub.o: $(CC) $(CFLAGS) $(OPTFLAGS) $(DBFLAGS) -c cpprt0_stub.s

libs_clt = -lshare -ludsql -ludmach -lbasic -lret1 -lperf -lides -lpipe \ -lfunc -lndx $(dfslib) -lrep -lshm -lmglm -lglm -lulc -lcmn - llicn \ -ludus -lunix -lbci -lunirpc -lssl -lcrypto \ $(ODBC_LIBS) $(nfalib) $(odslib)

libs_svr = -lnfasvr -lshare -ludsql -ludmach -lbasic -lret1 -lperf -lides \ -lpipe -lfunc -lndx $(dfslib) -lrep -lshm -lmglm -lglm -lulc - lcmn \ -llicn -ludus -lunix -lbci -lunirpc -lssl -lcrypto \ $(ODBC_LIBS) $(odslib)

libs_srv = -lushare -lusql -lumach -lbasic -lret1 -lperf -lides -lpipe \ -lushare -lumach -lret1 -lperf -lpipe \ -lfunc -lndx -lrep -lshm -lmglm -lglm -lulc -lcmn -llicn \ -ludus -lunix -lbci -lssl -lcrypto \ $(ODBC_LIBS) -lunirpc $(nfalib) $(odslib)

udt: $(OBJS) $(CC) $(LDFLAGS) $(OBJS) $(NEWOBJS) $(NEWLIBS) \ $(libpath) -lapidummy $(libs_clt) \ $(addlibpath) $(addlib) \ -o $@

udtsvr: $(OBJS) $(CC) $(LDFLAGS) $(OBJS) $(NEWOBJS) $(NEWLIBS) \

20-7 Administering UniData on UNIX $(libpath) -lapidummy $(libs_svr) \ $(addlibpath) $(addlib) \ -o $@

uniapisvr: $(OBJS) $(CC) $(LDFLAGS) $(OBJS) $(NEWOBJS) $(NEWLIBS) \ $(libpath) -lapisvr $(libs_clt) -lmsg \ $(udlib) $(addlibpath) $(addlib) \ -o $@

udapi_slave: $(OBJS) $(CC) $(LDFLAGS) $(OBJS) $(NEWOBJS) $(NEWLIBS) \ $(libpath) -lapidummy -licapi $(libs_clt) -lunirpc \ $(addlibpath) $(addlib) \ -o $@

udsrvd: $(OBJS) $(CC) $(LDFLAGS) $(OBJS) $(NEWOBJS) $(NEWLIBS) \ $(libpath) -lapidummy $(libs_srv) \ $(addlibpath) $(addlib) \ -o $@

.c.o: $(CC) $(CFLAGS) $(IDIR) $(OPTFLAGS) $(DBFLAGS) -c $<

#

cfuncdef Example

Warning: Do not copy the sample cfuncdef file onto your system. If you do, makeudt can fail. Always start with the cfuncdef file released with your UniData release.

The following sample is a cfuncdef file, also from an HP system. Notice that, in this instance, there are no C functions specified:

# pg cfuncdef /* this is a test for adding C function to the RUN Machine */ /* comment lines come here. */ /* C function declaration format: function-name:return-type:number-of-argument:arg1,arg2,...,argn */ $$FUN /* begining of C function */ $$OBJ /* *.o come here */ $$LIB /* library comes here */ If C functions were specified, there would be entries under $$FUN, and there might also be entries under $$OBJ and $$LIB.

makeudt 20-8

Creating cfuncdef_user

Although makeudt recognizes the cfuncdef_user file, UniData does not include this file at installation. You can create this file from the cfuncdef file by completing the following steps:

1. Copy the cfuncdef File In the udthome/work directory, execute the command: # cp cfuncdef cfuncdef_user 2. Edit cfuncdef_user Use vi or any UNIX text editor to remove from your new cfuncdef_user file any lines that appear under the lines beginning with $$FUN, $$OBJ, and $$LIB. This step gives you a file that has the format information included for the function definitions, but does not contain references to any UniData routines. 3. Add Local C Functions to cfuncdef_user Use vi or any UNIX text editor to add references to site-specific C routines you are linking into UniData. Follow the format specified in the file.

20-9 Administering UniData on UNIX Steps for Linking in C Functions

Complete the following steps to rebuild your udt executable or to build a udapi_slave executable.

1. Make Backup Copies of Files Log in to your system as root; save and verify a copy of your current udt, your base.mk, and your cfuncdef, as shown below: # cd $UDTBIN # cp udt udt.save # cmp udt udt.save # # cd $UDTHOME/work # cp base.mk base.mk.save # cp cfuncdef cfuncdef.save # diff base.mk base.mk.save # diff cfuncdef cfuncdef.save #

If you are linking in new C functions of your own, make a copy of your lat- est version of cfuncdef_user as well. In the example, the UNIX cmp command verifies the udt executable. The UNIX diff command verifies the text files base.mk and cfuncdef. If you are linking in new C functions of your own, proceed to step 2. If you are simply rebuilding the production udt with no changes, proceed to step 6. 2. Code and Compile Your C Functions If you are linking new C functions into the udt or udapi_slave executables, make sure of the following: „ The C functions should reside in the work directory, along with the makefile and the function definition files. „ The C functions cannot contain the main() function. „ The C functions must compile cleanly (use cc -c to compile them producing .o files).

Steps for Linking in C Functions 20-10

3. Edit cfuncdef_user Add information about your C functions to the cfuncdef_user text file. Use vi or any other UNIX text editor to add the information. Make sure the infor- mation for the function definitions (lines under $$FUN) is added using the format described in the text file. Make sure you define your C functions in cfuncdef_user rather than cfuncdef.The cfuncdef file contains definitions for only UniData-developed C functions that may be part of your UniData release. UniData overwrites the cfuncdef file whenever you install a new UniData version. If you placed your own function definitions there, you will lose them with each new install. UniData does not overwrite the cfuncdef_user file at installation, so your site specific definitions are preserved. The naming convention for UniData functions is U_funcname(). To avoid errors, choose a different naming convention when naming your C functions. 4. Ask Users to Log Out of UniData Although you can create a new udt or udapi_slave executable while users are logged on to UniData, the makeudt command may fail attempting to move this executable from udthome/work into your udtbin directory. You can move the new udt manually at a later time if you wish, but if you are ready to update production with makeudt or makeudapi, users should be logged out of UniData. If your changes are not fully tested, you may prefer not to overwrite the pro- duction udt. See “Steps for Setting Up a Development Environment” in this chapter.

20-11 Administering UniData on UNIX 5. Execute makeudt or makeudapi The following screen shows sample output from the makeudt command: # cd $UDTHOME/work # $UDTBIN/makeudt Are you ready to use "cfuncdef" in "/usr/ud72/work" to make a new udt? (y/n) y

Generating new udt. It takes a while. Please wait ...

make -f new.mk udt: cc -q -z +ESsfc -DNULL_OK -DSQLTP -DUDTENV -DNEW_INTER - g -c funchead.c cc -q -z +ESsfc -DNULL_OK -DSQLTP -DUDTENV -DNEW_INTER - g -c interfunc.c cc -q -z +ESsfc -DNULL_OK -DSQLTP -DUDTENV -DNEW_INTER - g -c callcf.c cc -q -z +ESsfc -DNULL_OK -DSQLTP -DUDTENV -DNEW_INTER - g -c efs_init.c cc -Wl,-a,archive funchead.o interfunc.o callcf.o efs_init.o \ -L${UDTLIB:-/usr/ud72/lib} -lapidummy -lshare -ludsql - ludmach -lbasic -lret1 -lperf -lides -lpipe -lfunc -lndx -lshm -lcmn -llicn -ludus -lnfaclnt -lud \ -lm -Wl,-a,shared -lcurses \ -o udt

The new udt is made successfully. # The new udt has replaced the previous version in udtbin.

Steps for Linking in C Functions 20-12

The next screen shows sample output from the makeudapi command:

# makeudapi Are you ready to use "cfuncdef" in "/disk1/ud72/work" to make a new udapi_slave? (y/n) y

Generating new udapi_slave. It takes a while. Please wait ...

make -f new.mk udapi_slave: cc -q -z +ESsfc -O +Ovolatile -c funchead.c cc -q -z +ESsfc -O +Ovolatile -c interfunc.c cc -q -z +ESsfc -O +Ovolatile -c callcf.c cc -q -z +ESsfc -O +Ovolatile -c efs_init.c cc -Wl,-a,archive funchead.o interfunc.o callcf.o efs_init.o \ -L/disk1/ud61/lib -lapidummy -licapi -lshare -ludsql - ludmach -lbasic -l ret1 -lperf -lides -lpipe -lfunc -lndx -lshm -lmglm -lglm -lulc -lcmn -llicn -ludus -lunix -lnfaclnt -lodsdummy -lunirpc\ -lm -Wl,-a,shared -lcurses -lsec \ -o udapi_slave

The new udapi_slave is made successfully. The new udapi_slave has replaced the previous version in udtbin. 1. Make a Backup Copy of the New udt If you experience a disk failure between the time you execute makeudt or makeudapi and your next regularly-scheduled backup, you need a copy of this new executable to restore your system to its current state. If you do not make a backup copy, and you restore from prior backups, you will replace the old version of udt or udapi_slave.

20-13 Administering UniData on UNIX 2. Produce a Stripped Executable (Optional) The makeudt command produces an executable that contains symbol table information that is helpful if you are using a debugging tool to research a problem. However, including this information increases the size of the exe- cutable, and may result in noticeably slowed response for users. Consider using the UNIX strip command to remove the symbol table information to decrease size and improve performance. The following screen shows an example: # ls -l udt -rwxrwxrwx 1 root unisrc 12671908 May 24 18:11 udt # strip udt # ls -l udt -rwxrwxrwx 1 root sys 4415488 Jun 10 15:20 udt Consult your host operating system documentation for additional informa- tion about stripping executables. 3. Allow Users to Log On to the System All users who log on at this point will access the new version of the udt or udapi_slave executable. If you want to execute makeudt or makeudapi and not overwrite the produc- tion udt, modify base.mk, replacing $@ in the very last line with a new output file name, such as udt.test. Then complete steps 1 through 8. Users can access the new executable by entering that name (for instance, udt.test) instead of udt.

Steps for Linking in C Functions 20-14

Steps for Setting Up a Development Environment

During application development and testing, it may be necessary to rebuild the UniData kernel numerous times to test changes to C functions. You can use the make files provided by UniData to build an executable in a working directory separate from production. Complete the following steps:

1. Establish a New Working Directory Create a directory called work in a partition where there is space for devel- opment. Then, copy the contents of the default udthome/work to the new directory. You can use the make files, and so on, as templates. The following screen shows an example: # pwd /usr/user01 # mkdir work # cd work # cp $UDTHOME/work/* . # ls base.mk callcf.c efs_init.c funchead.c callbas.mk cfuncdef efsdef interfunc.c 2. Develop and Compile C Functions Use the new work directory to develop and compile these functions. 3. Edit the cfuncdef_user File Edit this file in the new work directory. Do not change the default one (in udthome/work) until your changes are ready for production.

20-15 Administering UniData on UNIX 4. Execute gencdef, genfunc, and genefs If you have made any changes to cfuncdef or to cfuncdef_user, you need to execute the UniData system-level commands genefs, gencdef, and genfunc to update working files that the make utility requires. Use the following steps, but be certain that your current working directory is the new work space: # pwd /usr/user01/work # $UDTBIN/genefs # $UDTBIN/gencdef # $UDTBIN/genfunc # ls -tl |pg total 26 -r--r--r-- 1 root sys 2735 Jun 10 16:08 interfunc.c -r--r--r-- 1 root sys 738 Jun 10 16:08 funchead.c -r--r--r-- 1 root sys 760 Jun 10 16:08 callcf.c -rw-rw-rw- 1 root sys 97 Jun 10 16:08 ndef -rw-rw-rw- 1 root sys 422 Jun 10 16:08 cdef -r--r--r-- 1 root sys 1006 Jun 10 16:08 efs_init.c -r--r--r-- 1 root sys 155 Jun 10 16:04 efsdef -rw-rw-rw- 1 root sys 985 Jun 10 16:04 callbas.mk -r--r--r-- 1 root sys 292 Jun 10 16:04 cfuncdef -rw-rw-rw- 1 root sys 1424 Jun 10 16:04 base.mk You must log on as root to execute these commands, and you must execute them in the order shown in the example. If you do not execute these com- mands, and you have changed your function definition files, make can fail. 5. Edit the new.mk File You can change the name of the output executable to provide clarity. In the following lines from a sample make file, the executable will be named new_test: udt: $(OBJS) $(CC) $(LDFLAGS) $(OBJS) $(NEWOBJS) $(NEWLIBS) \ $(libpath) -lapidummy $(libs_clt) \ $(addlibpath) $(addlib) \ -o new_test

Steps for Setting Up a Development Environment 20-16

6. Reset UDTHOME or WORKPATH You need to redefine your environment so that the make utility creates its output in your new work area. You can set the UDTHOME or WORKPATH environment variables. To change UDTHOME, set UDTHOME so that your new work area is iden- tified as udthome/work. The following example illustrates this: # UDTHOME=/usr/user01;export UDTHOME # cd $UDTHOME/work # pwd /usr/user01/work # If you want to change WORKPATH, set the WORKPATH environment variable to your new work area, as shown in the following example: # WORKPATH=/usr/user01/work;export WORKPATH # cd $WORKPATH # pwd /usr/user01/work # 7. Build the Executable Use the UNIX make utility rather than makeudt at this point. The makeudt command overwrites the new.mk file that you updated in step 5. Also, for development, it is safest not to use makeudt until all your changes are ready for production. The following screen shows how to use make: # make -f new.mk cc -q -z +ESsfc -DNULL_OK -DSQLTP -DUDTENV -DNEW_INTER -g -c funchead.c cc -q -z +ESsfc -DNULL_OK -DSQLTP -DUDTENV -DNEW_INTER -g -c interfunc.c cc -q -z +ESsfc -DNULL_OK -DSQLTP -DUDTENV -DNEW_INTER -g -c callcf.c cc -q -z +ESsfc -DNULL_OK -DSQLTP -DUDTENV -DNEW_INTER -g -c efs_init.c cc -Wl,-a,archive funchead.o interfunc.o callcf.o efs_init.o \ -L${UDTLIB:-/usr/ud72/lib} -lapidummy -lshare -ludsql -ludmach - lbasic -lret1 -lperf -lides -lpipe -lfunc -lndx -lshm -lcmn -llicn - ludus -lnfaclnt -lud \ -lm -Wl,-a,shared -lcurses \ -o new_test # ls base.mk callcf.o efs_init.c funchead.c interfunc.o new_test callbas.mk cdef efs_init.o funchead.o ndef callcf.c cfuncdef efsdef interfunc.c new.mk # Notice that the executable, named new_test, is created in the alternate work- ing directory. Refer to your host operating system documentation for information about the make utility.

20-17 Administering UniData on UNIX 8. Test the Executable Users can test the executable simply by specifying the full path when invok- ing it. The user’s udthome and udtbin should not be changed from their ordinary settings. The following screen shows an example: # cd $UDTHOME/demo # /usr/user01/work/new_test UniData Release 7.2 Build: (3089) Copyright (C) IBM Corporation 2005. All rights reserved.

Current UniData home is /disk1/ud72/. Current working directory is /users/ud_72/demo. : :!ps PID TTY TIME COMMAND 12393 ttyv1 0:00 new_test 10660 ttyv1 0:00 csh 10746 ttyv1 0:00 sh 12398 ttyv1 0:00 sh 12399 ttyv1 0:00 ps 10659 ttyv1 0:03 rlogind Notice that the UNIX ps command displays the name of the test executable. You can have several different versions in test at one time as long as they have different names.

Steps for Setting Up a Development Environment 20-18

Accessing UniData from a C Program

The CallBasic application programming interface (API) allows you to access a UniData database by calling UniBasic subroutines from a C program. The CallBasic API is particularly useful for applications like automated processes that gather data and then write the data into a UniData database. The main body of the application may be written in C, and the application uses a UniBasic subroutine, called through CallBasic, to write the data into UniData hashed files in correct format.

Note: When you use CallBasic, your UniBasic routines are called from an external program. UniBasic and UniData are invisible to the users of the external program.

Requirements

The components required to use the CallBasic API are:

„ Development environment—Your system should have a full software development kit. (A base compiler is not sufficient). You need network libraries if you are using NFA. Consult your host operating system documentation and your hardware ven- dor if you have questions about your C development environment. „ C programs—You need to code and compile the C application that will call UniBasic. „ Function definitions and make files—When you install UniData, the file callbas.mk is installed into the directory udthome/work. Use this makefile as a template to build your application, with UniData linked into it.

The examples in this section are developed using udthome/work as a workspace. You can create a separate workspace by creating a new directory and copying all the files from udthome/work into it. Then complete the steps in your own workspace.

How CallBasic Works

The CallBasic API includes three functions:

„ udtcallbasic_init( )—This function initializes UniData and CallBasic. It serves the purpose of opening a “UniData session” for your C application. Your C program must execute this function once and only once.

20-19 Administering UniData on UNIX „ U_callbas( )—This function calls a UniBasic subroutine, passing arguments, and places the results in a pointer to the return value. You may execute this function numerous times in your C application, each time you want to call a UniBasic subroutine. „ udtcallbasic_done( )—This function clears all UniData-related temporary files and other resources, and ends the interface between the C application and UniData. You must execute this function once in your C application. You must also include this function in any error exits in your application that may be taken after udtcallbasic_init().

See Developing UniBasic Applications for a detailed description of the CallBasic functions and their requirements.

Accessing UniData from a C Program 20-20

C Program Example

The following C program, called sample.c, illustrates the components required for CallBasic:

#include #include #include "/usr/ud72/include/share.h"

main() { /* Declare variables */ char *rtn; char arg0[BUFSIZ]; char arg1[BUFSIZ]; char *args[2]; int sts;

/* Initialize the UniData environment */

/* Set up an error handler to shut down gracefully */

int jmpret, sat; U_SET_JMP(jmpret, sat); if (!jmpret) { udtcallbasic_done(1); exit(-1); } udtcallbasic_init(0, 0);

/* Assign arguments for the UniBasic routine */ strcopy(arg0, "Plants"); strcopy(arg1, "Animals"); args[0] = arg0; args[1] = arg1;

/* Call the subroutine */ sts = U_callbas(&rtn, "EXAMPLE", 2, args); if (sts == 0){ free(rtn); } printf("status: %d\n", sts); /* Close everything properly */ udtcallbasic_done(sts); } Notice the following points about sample.c:

20-21 Administering UniData on UNIX Header Files

You must include setjmp.h for the error handler, and you must include the UniData header file /usr/ud72/include/share.h for linking UniData and your C program.

Error Handler

Remember that the CallBasic exit routine, udtcallbas_done( ), must be the last action taken before the C program exits, whether normally or abnormally.

Initializing CallBasic

This statement initializes CallBasic, effectively starting a udt session for your C program:

udtcallbasic_init(0, 0);

Notice that it is executed once and only once in the C program.

If you initialize CallBasic more than one time, you will encounter errors and your program may dump core.

Calling a UniBasic Subroutine

In this program, we call the subroutine and assign the return to a variable.

/* Call the subroutine */ sts = U_callbas(&rtn, "EXAMPLE", 2, args);

Remember that you can call more than one UniBasic subroutine, using the U_callbas( ) function, as long as you do so between initializing and closing CallBasic.

Freeing Resources

Each U_callbas( ) execution allocates memory; remember to free this after conclusion of the subroutine:

if (sts == 0){ free(rtn); Notice that we free the memory if the function completes successfully; UniData frees the memory if the function fails.

C Program Example 20-22

Ending CallBasic

The last step in the C program is:

/* Close everything properly */ udtcallbasic_done(sts); Remember that this function closes the UniData session for the C program, closing all UniData temporary files and releasing all resources held by UniData for this C program.

If you do not exit UniData cleanly, you may lose consistency of data, and you may damage files.

20-23 Administering UniData on UNIX UniBasic Subroutine Example

The following UniBasic subroutine, called EXAMPLE, is a very simplified routine showing the requirements for CallBasic.

SUBROUTINE EXAMPLE(RETNVAL,ARG1,ARG2) PRINT "THE FIRST ARG IS ":ARG1 PRINT "THE SECOND ARG IS ":ARG2 RETNVAL="RETURN" RETURN END Notice the following points about the UniBasic subroutine.

Arguments

The arguments for the UniBasic subroutine match what is sent from the C program. Here is the call to the subroutine:

sts = U_callbas(&rtn, "EXAMPLE", 2, args);

And here is the first line of the subroutine:

SUBROUTINE EXAMPLE(RETNVAL,ARG1,ARG2)

Additional Information

You must create, compile, and catalog the UniBasic subroutine in a UniData account. The routine may be globally, directly, or locally cataloged. However, if you catalog the routine directly or locally, you must execute the C program from the UniData account where the subroutine is cataloged. Regardless how you catalog the UniBasic subroutine, you must execute the C program from a valid UniData account.

UniBasic Subroutine Example 20-24

Steps for CallBasic

Complete the following steps to access UniData from a C program with CallBasic:

1. Code and Compile the C Program C compilers differ between UNIX versions. For instance, some C compilers require ANSI-compliant syntax, and others do not. Consult the documenta- tion supplied with your C compiler for specific coding requirements for your system. The C program should be located in the same directory as the makefile. By default, this is udthome/work. Use the cc -c command to compile the C pro- gram in your workspace. cc -c produces a .o file in the same directory. The following example shows how to compile the sample program sample.c. # ls -l sample* -rw-rw-rw- 1 root sys 745 Jun 11 11:43 sample.c # cc -c sample.c # ls -l sample* -rw-rw-rw- 1 root sys 745 Jun 11 11:43 sample.c -rw-rw-rw- 1 root sys 1624 Jun 11 11:45 sample.o # See “C Program Example” in this chapter for a listing of sample.c. 2. Code, Compile, and Catalog the UniBasic Subroutine Remember that you can catalog the UniBasic subroutine globally, locally, or directly. The following example shows compiling and directly cataloging the sample subroutine EXAMPLE in the UniData demo account: :AE BP EXAMPLE Top of "EXAMPLE" in "BP", 6 lines, 128 characters. *--: P 001: SUBROUTINE EXAMPLE(RETNVAL,ARG1,ARG2) 002: PRINT "THE FIRST ARG IS ":ARG1 003: PRINT "THE SECOND ARG IS ":ARG2 004: RETNVAL="RETURN" 005: RETURN 006: END Bottom. *--: FIBC Filed "EXAMPLE" in file "BP" unchanged.

Compiling Unibasic: BP/EXAMPLE in mode 'u'. compilation finished : Before proceeding further, be sure that both your C program and your Uni- Basic subroutine are thoroughly tested.

20-25 Administering UniData on UNIX 3. Make a Copy of the Makefile Template The template, released by default into udthome/work, is called callbas.mk. When you copy it, name the copy in a way that relates it to your C program, as shown below: # ls callbas* callbas.mk # cp callbas.mk sample.mk # The template for your makefile is platform and release dependent. Always use callbas.mk on your system as a starting point; do not attempt to use the examples in this document.

Steps for CallBasic 20-26

4. Edit Your Makefile Use vi or any UNIX text editor to edit the copy you created in Step 3. The purpose is to produce a makefile that will build your C program, with udt linked into it, forming an executable whose name you choose. There are additional steps required so that the make is successful. Follow the sequence below: a. Correct References for UniData Libraries Locate a line in your make file that looks like this: libpath = -L/disk3/srcman/alpha/ud_72_020715_4088/bin/lib Notice that the path does nt match your production system. It contains a UniData internal naming convention. You need to change this to reflect the actual location of your UniData libraries. The following example uses the default location: libpath = -L/usr/ud72/lib You also need to remove a redundant line. Look for a line that resembles: libpath = -L$$UDTLIB Delete this line from your make file. b. Enter the Name of Your Compiled C Routine Look for a line that resembles: NEWOBJS = callbas.o callbas.o is a default object name released with the template. Change this to match your C routine, as shown below for the sample program: NEWOBJS = sample.o This change allows your C routine to be included in the make. c. Enter the Name for Your New Executable The UNIX make utility will build a new executable linking the UniData ker- nel (udt) with your compiled C routine. You need to modify the makefile to name the resulting executable. Look for a line that resembles: callbas: $(NEWOBJS) $(OBJS) callbas is a default name released with the makefile. Substitute the name you want as shown below: sample: $(NEWOBJS) $(OBJS) Note: As shown in the examples, it is simplest to maintain naming consis- tency between your C program, its .o file, and your new executable.

20-27 Administering UniData on UNIX d. Check Your Work Make certain that you have replaced all occurrences of callbas with the name of your program. 5. Build Your New Executable The executable resulting from this step will be significantly larger than your compiled C program, because this step links in the UniData kernel. You can estimate the total size by recording the size, in bytes, of udtbin/udt. Add the size in bytes of your compiled C program (sample.o in the examples) and then add about 20% overhead, since the executable will not be stripped. Make sure you have enough space available in your work area. Use the UNIX make utility, as shown in the following screen: # make -f sample.mk cc -Wl,-a,archive sample.o funchead.o interfunc.o callcf.o efs_init.o \ -L/usr/ud72/bin/lib -lapidummy -lshare -ludsql -ludmach -lbasic -lperf -lret1 -lides -lpipe -lfunc -lndx -lshm -lcmn -llicn -ludus -lnfa- clnt -lud -lndbm -lcl -lBSD \ -lm -Wl,-a,shared -lcurses -o sample # ls sample* sample sample.c sample.mk sample.o # Notice that UniData creates the executable sample in your working directory. Note: You do not need to log on as root to execute the make utility. You do need to have write permissions at the directory level in your work area. Refer to your host operating system documentation for information about the make utility.

Steps for CallBasic 20-28

6. Produce a Stripped Executable (Optional) The make utility produces an executable that contains symbol table infor- mation that is helpful if you are using a debugging tool to research a problem. However, including this information increases the size of the exe- cutable, and may result in a noticeably slower response for users. Consider using the UNIX strip command to remove the symbol table information to decrease size and improve performance. The following screen shows the results of stripping the sample executable: # ls -l sample -rwxrwxrwx 1 root sys 12657828 Jun 12 12:18 sample # strip sample # ls -l sample -rwxrwxrwx 1 root sys 4423680 Jun 12 12:57 sample # To save time, do not strip your executable until you finish testing and debug- ging it. Consult your host operating system documentation for information about debugging and symbol tables.

20-29 Administering UniData on UNIX Chapter General Troubleshooting 21

Crashes and Restart Problems...... 21-3 UniData Crashes ...... 21-3 UniData Cannot Start ...... 21-4 Response Problems in UniData ...... 21-5 UniData Consistently Slow ...... 21-5 UniData is Hung ...... 21-5 Error Messages ...... 21-6 Common Error Messages ...... 21-6

This chapter outlines several problems you may encounter running UniData, and offers suggestions to research and resolve them. The chapter also describes a number of UniData system messages, along with explanations of their causes.

21-2 Administering UniData on UNIX Crashes and Restart Problems

This section presents suggestions for handling situations where UniData stops running entirely, or where you cannot start UniData.

UniData Crashes

Symptoms: System appears to be “hung”; one or more terminals may display the message Killed or udt dead.

Suggestions:

1. Check to make sure your hardware and operating system is running. Hardware or power problems may cause UniData to crash. If your system is up and running, proceed to step 2. Otherwise, identify and resolve system problems. 2. If you are running UniData on AIX, check swap space. The AIX operating system kills processes when it runs out of swap space. 3. Check to see if UniData is still running. Execute the showud command to check the status of the UniData daemons. If the UniData daemons are still running, proceed to “UniData is Hung.” Otherwise, proceed to step 4. 4. Check the UniData logs and error logs. These are located in udtbin. Consider printing them in case they are needed to research a crash. 5. Identify and resolve problems revealed in the logs. Chapter 12, “Managing UniData Files,” Chapter 17, “Managing Memory,” and Chapter 18, “Managing ipc Facilities” may be useful. 6. When all identified problems have been resolved, log on as root and execute startud. If UniData does not start, proceed to “UniData Cannot Start”. Otherwise, resume normal operations. Note: UNIX system crashes may result in data inconsistencies or file corruption, depending on the activity at the time of the crash. Examine your data files with guide after starting UniData. Additional factors must be considered if you are using the Recoverable File System (RFS). See Administering the Recoverable File System for further information.

Crashes and Restart Problems 21-3

UniData Cannot Start

Symptoms: startud does not complete normally. Error messages may or may not appear in the window where you run startud.

Suggestions:

1. Make sure your UNIX environment is running correctly. Check UNIX system logs for error and warning conditions. Identify and resolve external problems. 2. Check the UniData log files in udtbin. Consider printing them in case they are needed to solve a problem. 3. Check for indications of shared memory problems. (For example, if smm is unable to create the CTL segment, UniData will not start). If there are messages in smm.errlog, review Chapter 17, “Managing Memory,” for suggestions to solve the problem. 4. Check the status of UniData ipc facilities by logging on as root and running ipcstat. If ipc structures were not properly cleaned up after a crash, follow the procedures described in Chapter 18, “Managing ipc Facilities,” to clear the structures. If ipcstat hangs, use the UNIX ipcs command. 5. Log on as root and execute startud to restart UniData. Additional factors must be considered if you are using RFS. See Administering the Recoverable File System for further information.

21-4 Administering UniData on UNIX Response Problems in UniData

UniData Consistently Slow

Symptoms: The system is consistently sluggish whenever UniData is running and users are logged on.

Suggestions: Refer to Chapter 22, “Performance Monitoring and Tuning.”

UniData is Hung

Symptoms: The system has been performing acceptably, but the response slows. One to all terminals may appear hung.

Suggestions:

1. Execute the UNIX ps command. Look for processes with large and rapidly growing cpu time. Explore these processes; kill them if appropriate. 2. Execute showud at a UNIX prompt to make certain all UniData daemons are running. 3. Check the UniData logs on udtbin for clues about the problem. 4. Check for file or semaphore lock problems with the ECL LIST.LOCKS, LIST.QUEUE, and LIST.READU commands. See Chapter 13, “Managing UniData Locks,” for procedures to identify and clear unneeded locks. 5. Identify and resolve message queue problems using the procedures described in Chapter 18, “Managing ipc Facilities.” 6. If the response is still slow, or if steps 1 through 3 did not reveal the problem, check your system to identify and resolve unusual load conditions. The UniData listuser, sbcsprogs, and udtmon programs, and the UNIX uptime, vmstat, and ps commands may provide helpful information.

Response Problems in UniData 21-5

Error Messages

Error messages seen in UniData applications may originate from the following:

„ the application „ UniData „ UniBasic „ one of the layered products „ the operating system „ combined sources

The following table shows some typical formats for error messages.

Format Source

In UniBasic runtime error. The error identifies the program (TEST) /usr/ud61/sys/CTLG/t/T and the line where the error occurred. EST at line 20...

...errno=36 UNIX operating system message. Refer to the file /usr/include/sys/errno.h to translate the error number.

Not enough disk space, UniData error message. Many error messages in UniData can be resize failed. found in the file identified by the VOC pointer ENGLISH.MSG, which is a UniData hashed file. Use ECL ESEARCH or UniQuery to search for messages in this file. See UniData Inter- national for the name of your message file if you have localized UniData to run in your local language.

msgrcv failed. errno=36 UniData error message that includes the UNIX error number. Translate the error number for helpful troubleshooting information. Error Messages and Sources

Common Error Messages

This section describes some common UniData error messages.

21-6 Administering UniData on UNIX Syntax error

„ Occurrence—This error appears when a user is attempting to run an ECL command. „ Causes—This error may result from the following: „ Wrong syntax; refer to online help or to the UniData Commands Reference for the correct command syntax. „ You entered a backspace or other control character; reenter the command. „ ECLTYPE is set to P when it should be U or vice versa; try changing ECLTYPE and reenter the command. „ BASICTYPE is set to P when it should be U or vice versa; try changing BASICTYPE and reenter the command.

Not a verb command

„ Occurrence—This error appears when you are attempting to run an ECL command. „ Cause—command must be either a VOC entry or a globally cataloged program. „ You spelled the command incorrectly; try again. „ You are in the wrong UniData account; command is not an entry in the local VOC file; check the VOC file. „ command is a UniBasic program that is not globally cataloged; determine how it should be cataloged; make necessary corrections; instruct users appropriately. See Chapter 19, “Managing Cataloged Programs,” for further information. cannot open abcdef

„ Occurrence—This is a UNIX-level error which occurs when a process is attempting to open a file called abcdef. „ Causes—This error can be caused by either of the following: „ The file abcdef does not exist; check the file name and path and try again.

Error Messages 21-7

Note: Some UNIX commands return the message “no such file or directory” for a file that does not exist, while others return “cannot open.” „ The user receiving the error does not have sufficient permissions to execute the file. See Chapter 2, “UniData and UNIX Security,” and Chapter 11, “Managing UniData Security,” for additional information. Note: Some UNIX commands return the message cannot open filename: Permission denied and others simply return Permission denied.

[100004]

„ Occurrence—A number of users are logged on to UniData. When an additional user tries logging on, [100004] displays on the terminal. „ Cause—You are out of semaphore undo structures in the UNIX kernel. Use the UniData kp command to display kernel settings; the parameter semmnu should be set to three times the number of users licensed on your system. You need to rebuild your UNIX kernel.

[100000]

„ Occurrence—A number of users are logged on to UniData. When an additional user tries to log in, [100000] displays on their terminal. „ Cause—The UniData configuration parameter NUSERS is too small. This parameter, located in /usr/ud72/include/udtconfig, determines the number of local control tables UniData uses. Each local control table tracks infor- mation for a single UniData user process. This parameter cannot exceed the kernel parameter semmnu. Set NUSERS to the number of authorized UniData users + number of authorized UniData users / 4.

Virtual field too big

„ Occurrence—This message displays after you execute a UniQuery statement.

21-8 Administering UniData on UNIX „ Cause—You have created a formula in a virtual attribute that, when evaluated by UniQuery, creates a stack of C routines that is bigger than the default. The limit that is exceeded is not the number of characters in the formula itself, but an internal limit. Set the environment variable VFIELDSIZE to a number greater than 300 (the default) to resolve this problem. Try setting VFIELDSIZE to 400. The larger VFIELDSIZE is, the more memory your process requires.

Record is too long. Please ask Unidata to extend the U_MAXEXTRA.

„ Occurrence—This message occurs while you are loading records from tape into a UniData hashed file with T.LOAD. When the message occurs, T.LOAD quits, leaving a partial restore. „ Cause—T.LOAD has encountered a record that is too large to process. You can remake the T.DUMP tape with a larger block size, or you can load the tape into a DIR-type file rather than a UniData hashed file. Once you have loaded it into the DIR file, you can use the ECL COPY command to copy the records into a hashed file. If you execute T.LOAD to load a file into a directory, make sure your UNIX file system has enough inodes; you need one inode for every record in the file you are loading.

Numra is maxed out in installshmid

„ Occurrence—This message displays while someone is trying to run a globally cataloged UniBasic program. „ Cause—The sbcs daemon is out of memory to store globally cataloged programs. sbcs can manage up to 20 shared memory segments, and the size of each is determined by the UniData configuration parameter SBCS_SHM_SIZE (1 MB by default). However, some UNIX versions (AIX, for example) limit the number of shared memory segments to 10, thereby limiting the memory available for sbcs. You can resolve this situation by resetting SBCS_SHM_SIZE. Values from 4 MB to 8 MB are appropriate for AIX systems.

Error Messages 21-9 Chapter Performance Monitoring and Tuning 22

UNIX Performance Considerations ...... 22-3 UNIX Performance Monitoring ...... 22-4 Tools ...... 22-4 Tips ...... 22-4 UniData Performance Factors...... 22-6 Database Design Considerations ...... 22-6 Using Alternate Key Indexes ...... 22-6 Sizing Static Hashed Files ...... 22-6 Sizing Dynamic Hashed Files ...... 22-7 UniBasic Coding Tips ...... 22-7 UniBasic Profiling ...... 22-10 1. Compile the Programs with -G ...... 22-10 2. Execute the Programs with -G ...... 22-10 3. Review the Profile Output ...... 22-10 UniData Performance Monitoring: udtmon ...... 22-14 udtmon: UniData User Statistics ...... 22-17 udtmon: File I/O Statistics ...... 22-18 udtmon: Program Control Statistics ...... 22-20 udtmon: Dynamic Array Statistics ...... 22-22 udtmon: Lock Statistics...... 22-24 udtmon: Data Conversion Statistics ...... 22-27 udtmon: Index Statistics ...... 22-28 udtmon: Mglm Performance ...... 22-30 Performance Monitoring and Tuning This chapter outlines factors that can affect UniData performance on your UNIX platform, lists generic UNIX tools for monitoring components of your system, and describes UniData-specific procedures for monitoring performance.

22-2

UNIX Performance Considerations

Optimizing performance for any application on a UNIX platform may involve tuning and balancing among the four components of a UNIX system:

„ Disk I/O Subsystem—Controls physical input/output to disk drives. „ Virtual Memory Subsystem—Manages memory allocation, swapping, and paging. „ Process Scheduling Subsystem—Controls how processes use system resources. „ Network Subsystem—A collection of computers, terminal servers, periph- erals, and workstations that allows users to share data and resources.

The following table outlines the significance of these areas for UniData.

Component Relationship to UniData Performance

Disk I/O Subsystem Significant performance factor for UniData as for many database systems. Although UniData includes many features to increase I/O efficiency, optimizing configuration and imple- menting disk striping (if available) may improve performance.

Virtual Memory Not a performance factor in most cases; UniData processes Subsystem usually fail if memory is insufficient.

Process Scheduling Not a significant performance factor as long as you consider Subsystem elementary scheduling factors.

Network Subsystem May be a significant performance factor due to impacts on I/O performance UniData Performance and UNIX

22-3 Administering UniData on UNIX UNIX Performance Monitoring

Monitor your performance regularly to develop baseline expectations throughout your application’s processing cycle. The performance history of your system provides information you can use to implement new procedures (such as scheduling certain jobs to run off-hours or purchasing more resources) as well as to identify problems quickly to minimize downtime.

Tip: Consider setting up a cron job to gather performance statistics at scheduled intervals. Refer to your host operating system documentation for information about the cron command.

Tools

The following table lists UNIX commands useful for performance monitoring and describes information available from each.

UNIX Command Description

uptime Displays number of users and average system load (number of processes in the run queue).

iostat Monitors CPU activity and disk I/O activity. Useful for identi- fying disk bottlenecks and for balancing disk load.

vmstat Monitors CPU activity and memory usage. Useful for identi- fying memory shortages.

ps Displays information about active processes. UNIX Performance Monitoring Tools

Note: Command names, syntax, and options for performance monitoring tools differ among UNIX versions. Refer to your host operating system documentation for information specific to your system. Tip: Menu-driven performance monitoring toolkits are available from several vendors.

Tips

The following section lists some suggestions for improving performance.

22-4

uptime

If the load average shown by uptime is consistently greater than five, your system is heavily loaded. Check your memory resources; check disk I/O.

ps, vmstat

Poor system performance associated with processes that are paging or swapped may indicate memory shortages. Poor performance associated with processes that are blocked for I/O may indicate disk I/O problems.

iostat

Results that may indicate I/O problems include: CPU in system state more than 50% consistently; CPU has high idle time despite heavy system load; CPU is never idle; disk activity is unbalanced.

22-5 Administering UniData on UNIX UniData Performance Factors

Within UniData applications, the major performance factors are: database design, file sizing, and program coding efficiency.

Database Design Considerations

„ Structure your database so that records do not exceed a size limit of 4K. „ When possible, avoid long, multivalued, variable-length records. „ Locate the most frequently accessed attributes near the beginning of a record. „ As much as possible, make record keys numeric and short in length.

Using Alternate Key Indexes

Using alternate key indexes speeds query access in a hashed file, but can slow file updates. Consider this factor when creating indexes. The more indexes created for a file, the longer an update takes.

Index overflows occur when the maximum key length allocated is too small to contain the attributes being indexed. Index overflows can degrade performance for query as well as update access. The solution is to delete all of the indexes for a file and rebuild them with a longer maximum length.

Tip: Use the UniData LIST.INDEX command to identify index overflow problems. Consider running LIST.INDEX periodically. See Using UniData for information about alternate key indexes.

Sizing Static Hashed Files

Performance suffers if UniData hashed files are allowed to overflow. Level 2 overflow, which occurs when primary keys outgrow the block, has a particularly serious performance impact. Level 1 overflow, which occurs when data overflows the block, eventually affects performance as well.

For information about file sizing commands, refer to Chapter 12, “Managing UniData Files,” and to the UniData Commands Reference and Using UniData.

22-6

Tip: Consider using cron to execute the UniData system-level checkover command in each of your UniData account directories at regular intervals. Use the output to resize files in level 2 overflow.

Sizing Dynamic Hashed Files

Dynamic hashed files differ from static hashed files in that they split and merge with respect to a minimum modulo. Splitting prevents level 2 overflow conditions, because a group splits whenever the percentage occupied by keys exceeds a preset value. Merging supports efficient access to the file, because maintaining the file at a low modulo makes searches faster. For information about dynamic file sizing commands, refer to Chapter 12, “Managing UniData Files,” and to the UniData Commands Reference.

UniBasic Coding Tips

The efficiency of your UniBasic application has a significant impact on UniData performance. Use the following guidelines when designing and coding.

Use Modular Programming

„ Use inserts and include files consistently. „ Open frequently used files to common areas to reduce physical file opens.

Use Efficient Commands

„ Use the EQUATE command when possible. „ Use CASE statements instead of IF, THEN, ELSE structure whenever possible. Avoid nested IF, THEN, ELSE statements. „ Use UniData @variables. „ Minimize new assignment of variables. „ Eliminate GOTO statements. „ Use GOSUB instead of external CALLs when appropriate; use CALL when multiple programs access the same code.

22-7 Administering UniData on UNIX „ When using CALLs: „ Avoid opening files; pass data through common areas or an argument list. „ Minimize the number of local variables in subroutines. „ Use inserts, common areas, and argument lists whenever possible. „ Use A += B instead of A = A + B. „ Use LOOP and REMOVE when extracting data sequentially from a string. „ Avoid unnecessary shell commands; minimize PERFORM, EXECUTE, and MDPERFORM statements. „ Use CALL to run another UniBasic program. „ Avoid repeated ITYPE( ) function calls. „ Minimize use of EXECUTE “SELECT.....” by: „ Using UniBasic SETINDEX, READFWD, READBCK commands. „ Using the UniBasic SELECT command.

Use Dynamic and Dimensioned Arrays Appropriately

„ Use dynamic arrays when you are accessing small strings and variables. „ Use dimensioned arrays when you are handling a large number of attributes and delimited strings.

Use the Correct READ in Each Situation

„ Use READ when you are accessing small records, and the data you want is near the beginning of each record. „ Use READV if your intention is to get only one attribute. „ Use MATREAD when: „ You are handling records with a large number of attributes, and you want to access more than one attribute. „ You are handling large multivalued lists.

Manage Locks Carefully

„ Use the LOCKED clause with a loop.

22-8

„ Remember to release locks promptly.

See Developing UniBasic Applications for tips on using UniBasic locks efficiently.

22-9 Administering UniData on UNIX UniBasic Profiling

UniData enables users to generate execution profiles that track call counts and execution times for UniBasic programs, internal subroutines and program calls. You can use these profiles to identify sections of your UniBasic application that are called most frequently, and then focus optimization efforts in those areas.

Complete the following steps for UniBasic execution profiles.

1. Compile the Programs with -G

Compile UniBasic programs with the -G option to include information about internal subroutines in the profile reports.

2. Execute the Programs with -G

To profile a UniBasic program, run the program with the -G option. See Developing UniBasic Applications for information about compiling and running programs.

3. Review the Profile Output

Profile output is stored in the UNIX directory where the UniBasic program was executed. Output files are called profile.pid and profile.elapse.pid where pid is the process ID of the user’s UniData process.

The following screen shows a sample profile for a UniBasic program:

%time cumsecs seconds calls name

33.3 0.02 0.02 1 BP/_MAIN_PROG:SUBRA 33.3 0.04 0.02 1 BP/_MAIN_PROG:SUBRB 16.7 0.05 0.01 1 BP/_MAIN_PROG 16.7 0.06 0.01 1 BP/_MAIN_PROG:SUBRC

called/total parents index %time self descendents called+self name index called/total children

[1] 100.0 0.01 0.05 1 BP/_MAIN_PROG [1] 0.02 0.00 1/1 BP/_MAIN_PROG:SUBRA [2]

22-10

0.02 0.00 1/1 BP/_MAIN_PROG:SUBRB [3] 0.01 0.00 1/1 BP/_MAIN_PROG:SUBRC [4]

------

0.02 0.00 1/1 BP/_MAIN_PROG [1] [2] 33.3 0.02 0.00 1 BP/_MAIN_PROG:SUBRA [2]

------

0.02 0.00 1/1 BP/_MAIN_PROG [1] [3] 33.3 0.02 0.00 1 BP/_MAIN_PROG:SUBRB [3]

------

0.01 0.00 1/1 BP/_MAIN_PROG [1] [4] 16.7 0.01 0.00 1 BP/_MAIN_PROG:SUBRC [4]

------In this example, the main program is called MAIN_PROG. It has three internal functions, named SUBRA, SUBRB, and SUBRC.

Note: profile.pid reports execution times as CPU execution time, while profile.elapse.pid reports “real” time.

Each profile display includes two sections. The top section presents summary infor- mation, and the bottom section presents detail. The following table describes the fields in the top section of a UniBasic Profile display. There is one line for each function of the program.

Field Description

%time Percentage of the total run time used by the program or subroutine.

cumsecs Running sum of seconds for this program or subroutine and all called programs and subroutines used within a cycle.

seconds Number of seconds used by the program or subroutine in a cycle.

calls Number of times the program or subroutine was invoked in a cycle.

name Name of the program or subroutine. UniBasic Profiling: Summary Information

22-11 Administering UniData on UNIX UniData sorts program functions by execution time, and assigns an index to each function for ease of reporting. For each index, UniData computes information about functions that call, or are called by, the function corresponding to the index. The detail section of a profile contains this information, grouped by index. The next table describes the fields in the detail section.

Field Description

index An identifying number assigned by UniData to the program or subroutine. UniData assigns the indexes in descending order of execution time. The index in column 1 identifies the routine of interest for the group of data (the current index function).

%time Percentage of the total program runtime used by the program or subroutine and its descendents.

self Time spent by the program or subroutine either calling, or being called by, the program or subroutine identified by the current index.

descendents Execution time for descendents of this program or subroutine.

called Line contents differ according to the line of the subsection you are reading: called/total—lines above the index analyze parents; lists number of times this index is called by the parent listed in the name field. called+self—line containing the index; lists number of times the routine is called and the number of times it calls itself. called/total—lines below the index number analyze children and descen- dents; lists number of times this index calls the child listed in the name field.

name Name of the program or subroutine analyzed in this row of the report subsection.

index Index value assigned to the program or subroutine listed in the name field. UniBasic Profiling: Detail Information

The following screen shows one group of data, selected from the sample UniBasic profile:

called/total parents index %time self descendents called+self name index called/total children

0.02 0.00 1/1 BP/_MAIN_PROG [1] [2] 33.3 0.02 0.00 1 BP/_MAIN_PROG:SUBRA [2]

------

22-12

This subset of the report contains data relative to the internal SUBRA, which is identified by index number 2. “Parent” functions, or functions which call SUBRA, are listed above it; “descendents,” or functions called by SUBRA, are listed beneath it.

In the example, the report indicates that 33.3% of the execution time for the entire program is used by SUBRA. The function is called once by the main program, BP/MAIN_PROG.

22-13 Administering UniData on UNIX UniData Performance Monitoring: udtmon

The UniData Monitor Utility, udtmon, enables you to monitor a variety of character- istics at the system level. Many of these characteristics can impact performance. Invoke the UniData Monitor Utility by entering the UniData system-level command udtmon at a UNIX prompt.

Note: You do not need to log in as root to execute udtmon.

The following screen shows the main UniData Monitoring menu.

Select Display to produce a second menu as shown in the following example.

22-14

You can select either a text display or a graphic display. The following two screens show the appearance of a graphic display and a text display, respectively:

Note: In Graphic mode, Display provides a series of lines reflecting the current number of operations covered by the display. You can configure this display using the Configuration menu. By default, 10 operations per second produces a full-width display.

22-15 Administering UniData on UNIX Note: In text mode, Display provides five columns of numerical data reflecting the current, average, minimum, maximum, and total number of operations covered by the display since Monitor/Profile was started. Data is reported in operations per second. The default display interval is 3 seconds; you can modify the interval using the Timer option of the Display menu.

Selecting Display from the Display menu produces a list of characteristics you can monitor. The screen looks like the following example:

22-16

Note: When you highlight an option on the menu, a brief description displays at the bottom of the screen.

udtmon: UniData User Statistics

When you select UniData User from the Display menu, the following screen appears:

22-17 Administering UniData on UNIX udtmon: File I/O Statistics

When you select File I/O from the Display menu, the following screen appears:

22-18

The following table describes the fields in the File I/O display, and indicates consid- erations for performance.

Field Description

File Open Number of file opens at the operating system level, from UniBasic OPEN commands. On UNIX, if the average value in this field is more than 10 (opens per second), performance may suffer. Consider opening files in named common, and using pointers for subsequent access.

File Close Number of file closes at the operating system level, from UniBasic CLOSE commands.

Temp File Close UniData temporarily closes the least recently accessed open file when requests for file opens exceed the limit of open files per process. If the average value in this field is consistently more than zero, consider increasing the kernel parameter that defines the number of open files per process, and increase the UniData configuration parameter NFILES. File I/O Fields

22-19 Administering UniData on UNIX Field Description

Record Read Number of records read by UniData commands (other than UniQuery). For most applications, Record Read is greater than Record Write.

Record Write Number of records written by UniData commands (other than UniQuery).

Record Delete Number of records deleted by UniBasic commands (other than UniQuery).

Level 1 Overflow Number of operations (reads, writes, and deletes) to records in level 1 overflow blocks. Compute the total activity by adding Record Read, Record Write, and Record Delete. If you are using only static files and Level 1 Overflow is more than 10% of the total activity, use guide to analyze your files and resize them appropriately.

Level 2 Overflow Number of operations (reads, writes, and deletes) to records in level 2 overflow. If Level 2 Overflow is more than 2% of total activity, use checkover or guide to identify files in level 2 overflow and resize them appropriately. File I/O Fields (continued) udtmon: Program Control Statistics

Selecting Program Control from the Display menu produces a screen similar to the following example:

22-20

The following table describes the fields in the Program Control display, and indicates performance considerations.

Field Description

Local Call Number of calls to locally cataloged UniBasic programs. Locally cataloged UniBasic programs involve heavy I/O activity and increased memory demand, because each local call loads a copy of the executable in memory. In a development environment, using locally cataloged programs may be normal. In a production environment, if more than 5% of calls are to locally cataloged programs, examine your application and globally catalog programs for more efficient memory use.

Global Call Number of calls to globally cataloged UniBasic programs. In a production environment, this number should be much higher than Local Call. If a program is globally cataloged, users share a single copy of the object code in memory, which reduces both memory requirements and physical I/O load.

CALLC Call Number of calls to an external C function, from UniBasic CALLC statements. Program Control Fields

22-21 Administering UniData on UNIX Field Description

CHAIN Call Number of UniBasic CHAIN statements executed.

GOSUB Call Number of UniBasic GOSUB commands executed.

LOOP and GOTO Number of UniBasic LOOP or GOTO commands executed.

EXECUTE Call Number of external UniData command executions (from UniBasic EXECUTE commands).

PCPERFORM Call Number of PCPERFORM statements, which execute shell or host operating system tasks. If PCPERFORM call is more than 5% of the total activity, analyze your application and consider replacing PCPERFORM logic with C routines.

SLEEP Number of UniBasic SLEEP command executions. A sudden increase in this number may indicate that a number of users are attempting to get a lock on a record. Program Control Fields (continued) udtmon: Dynamic Array Statistics

The Dynamic Array menu item displays a screen similar to the following example:

22-22

The fields are counters for executions of UniBasic commands, described in the following table.

Field Description

COUNT Number of dynamic array counts, from COUNT command.

DELETE Number of dynamic array deletions, from DEL command.

EXTRACT Number of dynamic array data extractions, from EXTRACT command.

FIELD Number of dynamic array string extractions, from FIELD command.

FIND Number of dynamic array finds, from FIND command.

INDEX Number of dynamic array substring indexes, from INDEX command.

INSERT Number of dynamic array inserts, from INS command.

LOCATE Number of dynamic array locations, from LOCATE command. Dynamic Array Fields

22-23 Administering UniData on UNIX Field Description

MATCHFIELD Number of dynamic array substring matches, from MATCH- FIELD command.

MATPARSE Number of dynamic array matrix parses, from MATPARSE command.

REMOVE Number of dynamic array element removals, from REMOVE command.

REPLACE Number of dynamic array replacements, from REPLACE command. Dynamic Array Fields (continued) udtmon: Lock Statistics

The following screen displays when you select Lock Statistics:

The following table describes the fields of the Lock Statistics display, and indicates performance considerations:

Field Display

Record Lock Number of user-level record locks set by commands such as READL and READU.

Record Unlock Number of user-level locks released by commands such as RELEASE.

Semaphore Lock Number of user-level resource locks set by commands such as LOCK and T.ATT.

Semaphore Unlock Number of user-level resource locks released by commands such as T.DET or UNLOCK.

Shared Group Lock UniData-level shared lock on an entire group.

Excl. Group Lock UniData-level exclusive lock on an entire group. For most appli- cations, the number of shared group locks exceeds the number of exclusive group locks. If the number of exclusive group locks is greater than the number of shared group locks, one or more files may be overflowed. Identify these with guide. Lock Statistics Fields

22-24

Field Display

Shared Index Lock UniData-level shared lock on an index.

Excl. Index Lock UniData-level exclusive lock on an index. For most applications, the number of shared index locks exceeds the number of exclusive index locks. If the number of exclusive index locks is larger than the number of shared index locks, one or more index files may be overflowed. Identify these with LIST.INDEX.

End-of-File Lock UniData-level lock that is required when UniData extends a file by adding overflow groups or by splitting a dynamic file. If this number is consistently greater than zero, and the Dynamic File statistics do not show splitting and merging, one or more files has overflowed. Identify these and resize them for improved performance.

Lock Failure Number of times a process attempts to get a user-level lock and fails because the record is already locked. If performance is suffering, analyze your application for improper lock handling.

GLM Lock Request Number of times a udt process checks the global lock manager to get a lock.

GLM Lock Failure Number of times a udt process attempts to get a lock from the global lock manager and fails because the record is already locked. Lock Statistics Fields (continued) Note: In some circumstances, this screen may indicate that more records are being unlocked than are being locked. This is a function of how UniData gathers the statistics and does not indicate a problem.

22-25 Administering UniData on UNIX udtmon: Sequential I/O Statistics

Selecting Sequential I/O displays statistics for I/O operations on sequential files. The following screen shows the display.

The following table describes the fields of the Sequential I/O display.

Field Description

OPENSEQ Number of UniBasic OPENSEQ executions.

CLOSESEQ Number of UniBasic CLOSESEQ executions.

READSEQ Number of UniBasic READSEQ executions.

WRITESEQ Number of UniBasic WRITESEQ executions.

OSOPEN Number of UniBasic OSOPEN executions.

OSCLOSE Number of UniBasic OSCLOSE executions. Sequential I/O Fields

22-26

Field Description

OSREAD Number of UniBasic OSREAD executions.

OSWRITE Number of UniBasic OSWRITE executions.

COMO Write Number of writes to a COMO file. Sequential I/O Fields (continued)

udtmon: Data Conversion Statistics

Selecting Data Conversion Statistics from the Display menu produces a screen similar to the following example.

22-27 Administering UniData on UNIX The information on the screen provides an overview of a UniBasic application in terms of data handling. The following table describes the actions counted in the Data Conversion Statistics display:

Field Description

ASCII Converting strings from EBCDIC to ASCII format.

CHAR Converting characters from numbers to ASCII characters.

EBCDIC Converting strings from ASCII to EBCDIC format.

FMT Formatting strings for output.

ICONV Converting strings from an external format to UniData to internal format.

OCONV Converting strings from UniData internal format to an external format.

SEQ Determining the numeric value of an ASCII character. Data Conversion Statistics Fields

Note: There are no specific performance recommendations for this screen. udtmon: Index Statistics

Selecting Index Statistics from the Display menu produces a screen similar to the following example.

22-28

The following table describes the fields on the Index Statistics display.

Field Description

Node Read Number of index node reads; a node is a component of the B+ tree structure, and a node is analogous to a block in a hashed file.

Node Write Number of index node writes.

Node Merge Number of times two nodes merge; this takes place when entries in one or both nodes decrease.

Node Split Number of times an index node splits; this happens when entries in the original node increase. An unusual amount of split/merge/reuse activity indicates that one or more indexes are not properly sized. Use the ECL LIST.INDEX command to identify these, and then execute DELETE.INDEX...ALL, and CREATE and BUILD the indexes with a longer key length.

Node Reuse Number of times a node previously freed by a node merge is used for a node split. Index Statistics Fields

22-29 Administering UniData on UNIX Field Description

Log Read Number of reads from an index log file; these occur when an index which was disabled is reenabled and updated with the contents of the index log.

Log Write Number of writes to an index log file. These occur while an index is disabled, as UniData tracks changes by recording them in the index log.

Overflow Read Number of times UniData reads from an index overflow node. The system creates overflow nodes when the number of keys in an index exceeds a set limit. Reads to and writes from overflow nodes slow system performance. If overflow activity (reads and writes) exceeds 5% of system activity (node reads and node writes), use the ECL LIST.INDEX command to identify which indexes are overflowed. Then execute DELETE.INDEX...ALL, and CREATE and BUILD the indexes with a longer key length.

Overflow Write Number of times UniData writes an overflow node. Index Statistics Fields (continued) Note: Notice in the sample Index Statistics display the number of Overflow Reads and Overflow Writes indicates that one or more index may be improperly sized. udtmon: Mglm Performance

When you select Mglm Performance, the following screen displays.

22-30

The following table describes the fields on the Mglm Performance display:

Field Description

Toggle Failure The number of failures for test-n-set instruction in the specified time interval.

Latch Wait Reserved for future use.

Latch Total Number of toggles used in specified time interval.

Mglm Normal Number of normal locking operations in the specified time interval. This type of locking is the most frequently used, and reflects I/O performance.

Mglm Upgrade Number of upgrade locking operation in the specified time interval. If no index related operations occur, this number may be 0.

Mglm Downgrade Number of downgrade locking operations in the specified time interval.

Mglm Release Number of release locking operations in the specified time interval. Mglm Performance Fields

22-31 Administering UniData on UNIX Field Description

Mglm Wait Number of times Mglm waits for a lock.

Timeout Reserved for future use.

System Calls Number of system calls used by the lock manager in the specified time interval. This number should be low.

Toggle Rate Toggle Failure / Total Latch. This rate should be low. If the number is consistently greater than 5, increase the TOGGLE_NAP_TIME parameter in udtconfig.

Latch Rate Reserved for future use.

Mglm Rate (Mglm Wait) / (Mglm Normal + Mglm Release + Mglm Upgrade + Mglm Downgrade). This number is used to check I/O performance and should be low. Mglm Performance Fields (continued)

22-32 Appendix UniData Configuration A Parameters

This appendix lists the names and descriptions for all UniData config- uration parameters as of Release 7.2. Refer to Chapter 8, “Configuring Your UniData System,” for additional information about modifying your udtconfig file.

The following tables describe the configuration parameters that are placed in the udtconfig file located in udthome\include at the time of installation. They are system-dependent and should not be changed.

Parameter Description

LOCKFIFO The locking sequence of processes in the system. This parameter should not be changed.

SYS_PV Type of P/V operations used for the Recoverable File System (RFS) only. Determined at installation; platform dependent. Don’t change unless instructed by IBM. udtconfig Parameters That Should Not Be Changed The following parameters may be changed to suit your environment:

Parameter Description

NFILES Number of physical files that can be opened at the operating system level at one time in a UniData session. This limit is for both udt and tm processes; the name of the corresponding kernel parameter varies among UNIX versions.

NUSERS Limit for number of UniData user processes (such as udt and PHANTOM) that can run at the same time.

WRITE_TO_CONSOLE Switch for turning on and off messaging to your console. Must be greater than zero for messages to display at console.

TMP Path of a UNIX directory for storing intermediate work files. Default is \IBM\ud60\temp on UniData for Windows Platforms or /tmp/ on UniData for UNIX. When changing this parameter on UniData for UNIX, do not forget the trailing “/.”

NVLMARK Specifies a character to print to represent the null value. The ASCII character that represents the null value is nonprinting.

FCNTL_ON Used with UniData Physical Lock Manager. If a UNIX platform supports test-n-set instruction, SYS_PV is set to 3 and FCNTL_ON is set to 0. If a UNIX platform does not support test-n-set instruction, SYS_PV is set to 2 and FCNTL_ON is set to 1. Do not change this parameter unless instructed to do so by IBM.

TOGGLE_NAP_TIME If FCNTL_ON is set to 0, the length of time (in milli- seconds) that a process waits to access a shared memory address held by another process. This parameter has no effect if FCNTL_ON is set to 1. Do not change unless instructed to do so by IBM.

NULL_FLAG Toggles null value handling on and off. If 0, null value handling is off. Must be greater than 0 for null value handling to be in effect. udtconfig Parameters That Can Be Changed

A-2

Parameter Description

N_FILESYS Maximum number of UNIX file systems allowed. If you have more than 200 UNIX file systems, increase to your number of file systems.

N_GLM_GLOBAL_BUCKET The number of hash buckets systemwide, used to hold the lock names in shared memory. This setting directly affects performance. Normally, the default value of this parameter should not be changed. However, if you notice significant degradation in performance, or your application intensively accesses specific files, you can increase this parameter. The default value is the closest prime number to NUSERS * 3.

N_GLM_SELF_BUCKET The number of hash buckets for the per-process locking table. This parameter is highly application dependent. If the application requires a large number of locks in one transaction (more than 20), you should increase this setting to the closest prime number to the maximum number of locks per transaction.

GLM_MEM_SEGSZ The segment size for each shared memory segment required for the lock manager. The maximum number of segments is 16. Large application environments require a larger size. Each udt will register the lock names it is locking in its per-process locking table. This table is also organized as a hashed table.

BGINPUTTIMEOUT The number of seconds a background or phantom process waits before timing out. Before the timeout expires, a process may use the UNIX tandem or the UniData for Windows NT TANDEM command to attach to the process.

USE_DF Determines whether UniData reads the mount table or forks a df process when you execute the sms -F command. 0 – UniData loads the shared memory table by reading the mount table. 1 – UniData forks a df process to load the shared memory table. udtconfig Parameters That Can Be Changed (continued)

A-3 Administering UniData on UNIX The following parameter is related to internationalization.

Parameter Description

UDT_LANGGRP The language group ID used to distinguish language groups that use similar special characters. UDT_LANGGRP is composed of the record mark, escape sequence mark, and the null value mark. The default is 255/192/129.

ZERO_CHAR The character you want to use to represent CHAR(0). See OSREAD, OSBREAD, READT in the UniBasic Commands Reference for more information. Internationalization Parameters

The following table describes shared memory related parameters. These parameters may be changed to suit your environment.

Parameter Description

SBCS_SHM_SIZE Size, in bytes, of shared memory segments created by sbcs to store globally cataloged programs. sbcs can attach a maximum of 20 segments systemwide. Runtime errors result if a user attempts to load a new global program that exceeds this limit.

SHM_MAX_SIZE Current kernel setting for maximum size (in bytes) of a shared memory segment. This parameter is set at installation; if you increase the kernel parameter shmmax, you need to increase SHM_MAX_SIZE to the same value as well.

SHM_ATT_ADD Starting address for shared memory attachment. Set at installation; do not change this unless instructed by IBM.

SHM_LBA Alignment size, in bytes, for shared memory attachment. Set at installation; do not change.

SHM_MIN_NATT The minimum number of shared memory segments that should be kept attached to a process.

SHM_GNTBLS Number of GCTs (global control tables) in CTL. Each shared memory segment is associated with a GCT. The GCT registers the use of global pages in its associated shared memory segment. Cannot exceed the kernel parameter shmmni.

SHM_GNPAGES Number of global pages in a shared memory segment. udtconfig Parameters for Shared Memory

A-4

Parameter Description

SHM_GPAGESZ Size of each global page, in 512-byte units.

SHM_LPINENTS Number of entries in the PI table of a LCT, which is the number of processes allowed in a process group. It is set to 10 within the system, regardless of the udtconfig setting.

SHM_LMINENTS Number of entries in the MI table of a LCT, which means the number of global pages or self-created dynamic segments that can be attached by a process. Cannot exceed 255.

SHM_LCINENTS The number of entries in the CI table of each LCT, which is the number of local pages that can be attached by a process.

SHM_LPAGESZ Size, in 512-byte blocks, of each local page in a global page. A global page is divided into local pages, so SHM_GPAGESZ must be a multiple of SHM_LPAGESZ.

SHM_FREEPCT Percentage of freed global pages in an active global shared memory segment that UniData keeps in the global shared memory pool. smm checks the current percentage; if the percentage is less than SHM_FREEPCT, smm creates a new shared segment.

SHM_NFREES The number of inactive shared memory segments that UniData keeps in the system. smm checks the current number of inactive segments; if the number is larger than SHM_NFREES, smm returns some inactive global shared segments to UNIX. udtconfig Parameters for Shared Memory (continued)

A-5 Administering UniData on UNIX The following table describes size limitation parameters.

Parameter Description

AVG_TUPLE_LEN Number of local pages that matches the average length of records in your applications. Specifies the length of a buffer kept by udt for holding a record. Should not exceed the number of local pages in a global page.

EXPBLKSIZE Number of local pages used for expression buffers. udt keeps a buffer of this size for intermediate results. ibm recommends you set this parameter so the buffer is one- quarter to one-half the size of a global page.

MAX_OBJ_SIZE Maximum size, in bytes, of object programs that can be loaded into shared memory. Object programs larger than this size are loaded into the user’s address space instead of shared memory.

MIN_MEMORY_TEMP The minimum number of local pages that should be kept for temporary buffers in a process group. Determined at installation.

STATIC_GROWTH_ The number of table elements in the Static Growth Warn WARN_TABLE_SIZE table. UniData uses this table to track the last time a warning was issued indicating a file was larger than the threshold. When no unused elements are present in the table, UniData uses the oldest element for a new static file. If the file is nonrecoverable, UniData writes the infor- mation to the udt.errlog file. If the file is recoverable, UniData writes the information to the sm.log file

STATIC_GROWTH_ The threshold value for the static file size, expressed in WARN_SIZE bytes.If the file is nonrecoverable, UniData writes the information to the udt.errlog file. If the file is recoverable, UniData writes the information to the sm.log file

STATIC_GROWTH_ The time interval, expressed in seconds, to warn when a WARN_INTERVAL static file is larger than the threshold. If the file is nonrecov- erable, UniData writes the information to the udt.errlog file. If the file is recoverable, UniData writes the information to the sm.log file udtconfig Parameters for Size Limitation

A-6

The following table describes parameters related to dynamic files.

Parameter Description

GRP_FREE_BLK Pertains to dynamic files only; the number of free blocks kept in the free block list at the group level. If more blocks are freed, they are kept at the file level.

SHM_FIL_CNT Maximum number of dynamic files that can be open concurrently, systemwide.

SPLIT_LOAD Default loading factor option (percent) at which a group in a dynamic file using the KEYONLY option splits. Splitting occurs when the percentage of space in a group occupied by keys and pointers reaches the split load. The ECL CONFIGURE.FILE command overrides this for individual files.

MERGE_LOAD Default loading factor (percent) at which a group pair in a dynamic file using the KEYONLY option merges. A group pair is eligible for merging when the sum of the percentages of space occupied by keys and pointers in both groups is less than MERGE_LOAD. The CONFIGURE.FILE command lets users override this for individual files.

KEYDATA_SPLIT_ Default loading factor (percent) at which a group in a dynamic LOAD file using the KEYDATA option splits. Splitting occurs when the percentage of space in a group occupied by keys and pointers reaches the split load. The ECL CONFIGURE.FILE command overrides this for individual files. udtconfig Parameters for Dynamic Files

A-7 Administering UniData on UNIX Parameter Description

KEYDATA_MERGE_L Default loading factor (percent) at which a group pair in a OAD dynamic file using the KEYDATA option merges. A group pair is eligible for merging when the sum of the percentages of space occupied by keys and pointers in both groups is less than KEYDATA_MERGE_LOAD. The CONFIGURE.FILE command overrides this for individual files.

MAX_FLENGTH Upper size limit, in bytes, of each partition file (dat00x) of a dynamic file. When a part file reaches this size, UniData does not add further blocks to it, but creates another part file using the part table. The default value is 1073741824 bytes (1 GB). Must be greater than 32768 bytes (32 KB) and less than 2147467264 bytes (2 GB-16KB).

PART_TBL Path of a UNIX text file that directs UniData where to create dynamic file part files. udtconfig Parameters for Dynamic Files (continued) The following parameter is for NFA server.

Parameter Description

EFS_LCKTIME Used by the NFA Server to specify the maximum time to wait for a lock.

TSTIMEOUT Used by the udtts executable to specify the maximum number of seconds to wait for input from client about device information. If the information is not provided, UniData starts without the device information.

NFA_CONVERT_CHAR If this value is set to 1, UniData converts marks in a stream of data to host-specific marks. udtconfig Parameters for NFA The following table describes parameters related to Journaling:

Parameter Description

JRNL_MAX_PROCS Maximum number of journal processes per journal path.

JRNL_MAX_FILES Maximum number of journal files allowed per journal process. udtconfig Parameters for Journaling

A-8

The following table describes UniBasic file-related parameters.

Parameter Description

MAX_OPEN_FILE Maximum number of hashed files that can be opened by UniBasic OPEN statements, per udt process. Includes recoverable and nonrecoverable, static, dynamic, and sequentially hashed files; each dynamic file counts as one file.

MAX_OPEN_SEQF Maximum number of sequential files that can be opened at one time by UniBasic OPENSEQ statements, per udt process.

MAX_OPEN_OSF Maximum number of UNIX sequential files that can be opened at one time by UniBasic OSOPEN statements, per udt process.

MAX_DSFILES Maximum number of nonrecoverable dynamic part files (dat00x, over00x) a UniData process can open with UniBasic OPEN statements or ECL commands. Each dynamic file has at least two part files. UniBasic File-Related udtconfig Parameters

The following table describes parameters related to UniBasic.

Parameter Description

MAX_CAPT_LEVEL Number of levels allowed for nested UniBasic EXECUTE WITH CAPTURING or MDPERFORM WITH CAPTURING clauses. Individual users can set an environment variable that overrides the configuration parameter. udtconfig Parameters for UniBasic

A-9 Administering UniData on UNIX Parameter Description

MAX_RETN_LEVEL Number of levels allowed for nested UniBasic EXECUTE WITH RETURNING or MDPERFORM WITH RETURNING clauses. Individual users can set an environment variable that overrides the configuration parameter.

COMPACTOR_POLICY Used to guide BASIC memory compactor to do compaction for BASIC strings. 0 — compact when program is finished 1 — compact when EXECUTE (another BASIC pgm) is completed 2 — compact when EXECUTE (another BASIC program) or CALL is completed

VARMEM_PCT The percentage of free memory that should be kept in the first global page for UniBasic variables after compacting. If the actual percentage is less than this value, UniData keeps one free global page. Otherwise, UniData returns all free global pages to UNIX. udtconfig Parameters for UniBasic (continued) The following parameter is used in semaphore operations.

Parameter Description

NSEM_PSET Number of semaphores per semaphore set. udtconfig Parameters for Semaphores

A-10

The following table describes index-related parameters.

Parameter Description

SETINDEX_BUFFER_ KEYS Controls whether READFWD and READBCK state- ments use a buffering mechanism. Default value is 0 (buffering off). Individual environment variable overrides udtconfig setting; BUFFER.KEYS keyword in the SETINDEX statement overrides either.

SETINDEX_VALIDATE_KEY Controls whether READFWD and READBCK state- ments validate a key value just read. Default value is 0 (no validation). Individual environment variable overrides udtconfig setting. VALIDATE.KEY keyword in the SETINDEX statement overrides either. udtconfig Parameters for Indexes

The following parameter is used with the UniData Physical Lock Manager.

Parameter Description

MGLM_BUCKET_SIZE Number of nodes per bucket. If this parameter is inadequate for an application, UniData displays an out of memory message is.

UPL_LOGGING Determines if UPL performs logging. If this parameter is set to 0, UPL does not perform any logging. If this value is set to 1, UPL performs logging. udtconfig Parameters for Physical Lock Manager

The following parameters affect the UniData _HOLD_ file.

Parameter Description

MAX_NEXT_HOLD_DIGITS Enables you to specify the number of digits used for the next _HOLD_ file number, found in the NEXT.HOLD record of D__HOLD.

CHECK_HOLD_EXIST Determines if UniData checks for the existence of a _HOLD_ file prior to unconditionally removing it when you specify the BANNER UNIQUE option with the SETPTR command. udtconfig Parameters for _HOLD_ File

A-11 Administering UniData on UNIX The following parameter is used to turn the Recoverable File System off and on.

Parameter Description

SB_FLAG Toggles system buffer on and off. If zero, system buffer is off. Must be greater than zero for RFS. Recoverable File System On/Off udtconfig Parameters

The following parameters are related to open files with the Recoverable File System.

Parameter Description

BPF_NFILES Per-process logical limit for total number of recoverable files that can be opened with UniBasic OPEN statements at one time. If more recoverable files are opened, UniData closes files and then reopens them, causing heavy overhead. This parameter cannot exceed N_AFT.

N_PARTFILE Systemwide total number of recoverable dynamic part files that can be open at one time. This limit includes files opened by ECL and UniBasic. Each dynamic file has at least two part files, so opening a dynamic file means opening at least two part files. Even if more than one user opens the same dynamic file, each part file is counted once toward the total. udtconfig Parameters for Recoverable Files

The following parameters are related to the active file table (AFT) used with the RFS.

Parameter Description

N_AFT Systemwide limit on the number of recoverable files and indexes that can be open at one time. This is the number of slots in the system buffer AFT. Parameter is like MAX_OPEN_FILES but pertains only to recoverable files. A dynamic file counts as one file. Even if more than one user opens the same file, it is only counted once.

N_AFT_SECTION Number of sections in the AFT. Used for RFS only. udtconfig Parameters for Active File Table

A-12

Parameter Description

N_AFT_BUCKET Number of hash buckets in the AFT. Used for RFS only.

N_AFT_MLF_BUCKET Number of hash buckets in the AFT for tracking multilevel files. Used for RFS only.

N_TMAFT_BUCKET Number of hash buckets in each tm process’s active file table (TMAFT). Used for RFS only. udtconfig Parameters for Active File Table (continued) The following table describes parameters related to the archiving feature of RFS.

Parameter Description

ARCH_FLAG Toggles archiving function on and off for RFS. Must be greater than 0 for archiving.

N_ARCH The number of archive files.

ARCHIVE_TO_TAPE Switch for turning on automatic save of archive files to backup. Changing the value to 1 turns on this feature.

ARCH_WRITE_SZ Size, in bytes, of blocks for the archive process to write from the log files to the archive files. Default is zero, meaning each write is one block. If set to a nonzero value, must be a multiple of the log/archive block size. udtconfig Parameters for Archiving

The following parameters are used for the system buffer in the RFS.

Parameter Description

N_BIG Number of block index groups. This parameter determines the size of an index table for accessing the RFS system buffer. If you enlarge N_PUT, you should enlarge N_BIG as well. Must be a prime number.

N_PUT Number of 1,024-byte pages in the system buffer. The size of the buffer cannot exceed SHMMAX. Sometimes the default value of N_PUT must be decreased in order to complete a UniData installation. udtconfig Parameters for the System Buffer

A-13 Administering UniData on UNIX The following table describes parameters used for message queues with the Recov- erable File System.

Parameter Description

N_PGQ Number of message queues for tm processes to send messages to udt processes. Calculated by installation; one queue for every four licenses.

N_TMQ Number of message queues for udt processes to send messages to tm processes. Calculated by installation; one queue for every four licenses. udtconfig Parameters for Message Queues

The following table describes parameters related to the before image and after image log files used with RFS.

Parameter Description

N_AIMG Number of after image log files in each log set.

N_BIMG Number of before image log files in each log set.

AIMG_BUFSZ The size of the after image buffer, in bytes.

BIMG_BUFSZ The size of the before image buffer, in bytes.

AIMG_MIN_BLKS Minimum number of blocks required in the after image buffer before the system flushes the blocks to the after image logs. Block size is set in the log configuration table.

BIMG_MIN_BLKS Minimum number of blocks required in the before image buffer before the system flushes the blocks to the before image logs. Block size is set in the log configuration table.

AIMG_FLUSH_BLKS Number of blocks that are flushed to the after image logs from the after image buffer at one time. udtconfig Parameters for Before Image/After Image

A-14

Parameter Description

BIMG_FLUSH_BLKS Number of blocks that are flushed to the before image logs from the before image buffer at one time.

RFS_DUMP_DIR Defines where UniData stores the rfs.dump file when you execute the s_stat -s command. The default value is an empty string, with UniData storing the rfs.dump file in the $UDTBIN directory. If the path you specify is invalid when UniData starts, UniData writes the rfs.dump file to the $UDTBIN directory, and prints a message to the sm.log.

RFS_DUMP_HISTORY Specifies how many rfs.dump files to preserve when you execute the s_stat -s command. The default value is 1. With this value, UniData creates the rfs.dump file in the directory you specify with the RFS_DUMP_DIR parameter. If this value is set to a positive integer, for example 4, the rfs.dump files will be named rfs.dump1, rfs.dump2, rfs.dump3, and rfs.dump4. The s_stat -s command uses the first available rfs.dump file. If all rfs.dump files are full, s_stat -s reuses the oldest rfs.dump.file. If this value is set to 0, all rfs.dump files are preserved and named rfs.dump1, rfs.dump2, and so forth. udtconfig Parameters for Before Image/After Image (continued) The following table describes the parameters that determine flushing intervals for RFS.

Parameter Description

CHKPNT_TIME Checkpoint interval: number of seconds between flushes of the system buffer to disk.

GRPCMT_TIME Interval, in seconds, between flushes to the after image log set. udtconfig Parameters for Flushing Interval

A-15 Administering UniData on UNIX The following table describes the parameters related to the sync daemon.

Parameter Description

N_SYNC Determines how many sync daemons UniData should start.

SYNC_TIME Defines the number of seconds the sync daemon should wait before scanning the Block Index Group to flush dirty pages. udtconfig Parameters for the sync daemon The following table describes the parameter related to the Century Pivot date.

Parameter Description

CENTURY_PIVOT Determines the century pivot date. Default is 1930. udtconfig Parameter for Century Pivot Date

The following table describes the parameters related to replication.

Parameter Description

REP_FLAG Turns UniData Data Replication on or off. If this value is 0, UniData Data Replication is off. If this value is a positive integer, it is on.

TCA_SIZE The maximum number of entries in the Transaction Control Area. The default value is 2048.

MAX_LRF_FILESIZE The maximum Log Reserve File size, in bytes. The default value is 134217728 (128 MB). The maximum value is 2147483136.

N_REP_OPEN_FILE The maximum number of open replication log files for a udt or tm process. The default value is 8.

MAX_REP_DISTRIB Reserved for internal use. udtconfig Parameters for Replication

A-16

Parameter Description

MAX_REP_SHMSZ The maximum shared memory buffer segment size. The default value is 67108864 (64 MB).

UDR_CONVERT_CHAR When this value is set to 1, if the publishing system and the subscribing system have a different I18N group, UniData converts marks and SQLNULL marks to those on the local machine on the data passed between the two systems. The default value is 0.

REP_CP_TIMEOUT Specifies the cm daemon timeout interval for repli- cation. The default value is 300 seconds. If this value is set to 0, the cm daemon will not time out. udtconfig Parameters for Replication (continued) The following table describes the parameters releated to Euro processing.

Parameter Description

CONVERT_EURO Turns Euro conversion on or off. If this flag is set to 0, UniData does not perform conversion. If this flag is set to 1, UniData performs conversion.

SYSTEM_EURO The configurable system Euro encoding. On UNIX systems, the default is CHAR(164). On Windows Platforms, the default is CHAR(128).

TERM_EURO Sets the terminal system Euro Code. On UNIX systems, the default is CHAR(164). On Windows Platforms, the default is CHAR(128). udtconfig Parameters for Euro Processing

The following table describes the parameters related to the location of log files.

Parameter Description

LOG_OVRFLO Path to the directory where log overflow files are created.

REP_LOG_PATH Full path to the replication log file. udtconfig Parameters for Location of Log Files

A-17 Administering UniData on UNIX Appendix Environment Variables for UniData B

This appendix lists environment variables that can be set at the UNIX level to customize a UniData environment. Users with access to the UNIX prompt can set them before entering UniData to affect a particular UniData session. System administrators can also set them in a .login or .profile for one or more users to establish defaults for some or all users.

The following table lists environment variables in alphabetical order.

Environment Variable Description

ALLOW_DBPAUSE_ When this environment variable is set to 1, an OSBWRITE OSBWRITE process is successful even if dbpause if active. If the environment variable is set to 0, an OSBWRITE process pauses.

BACKUP_CNTL_FILE Used by the Recoverable File System (RFS); specifies a path where the udt.control.file can be automatically backed up at checkpoint time. If this variable is not defined, udt.control.file is not backed up.

CSTACKSZ Establishes the maximum number of commands in the ECL command stack. Each stack entry can hold a 2720 character command. The default is 49.

INIT_BREAKOFF Enables/disables break key prior to invoking UniData. If [0 | 1] INIT_BREAKOFF is not set, the break key is enabled by default.

LPREQ Identifies an alternate spooler directory. Must be a full path ending in “/”. Environment Variables for UniData

Environment Variable Description

MAX_CAPT_LEVEL Number of levels allowed for nested UniBasic EXECUTE WITH CAPTURING or MDPERFORM WITH CAPTURING clauses. This environment variable overrides the configuration parameter in the udtconfig file.

MAX_RETN_LEVEL Number of levels allowed for nested UniBasic EXECUTE WITH RETURNING or MDPERFORM WITH RETURNING clauses. This environment variable overrides the configuration parameter in the udtconfig file.

MAX_TRANS_ Number of TRANS fields that can be kept concurrently; default FIELD value is 12; must not be greater than 64.

MAX_TRANS_REL Number of TRANS files that can be open concurrently; default value is 32; must not be greater than 32.

NFA_LOG_DIR Used by NFA; name of a UNIX directory where UniData creates NFA client-side. If this variable is not defined, the logs are created in /tmp.

NFA_LOG_LEVEL Used by NFA; debug level for NFA client processes. May be an integer 0-10; level 0 logs only fatal errors, and level 10 logs all traffic and many internal functions. The default is level 0.

NOCHKLPREQ Bypasses UNIX printer verification; useful for large systems with hundreds of printers defined.

SETINDEX_ Controls whether READFWD and READBCK statements use BUFFER_ a buffering mechanism. Default value is 0 (buffering off). Individual environment variable overrides udtconfig setting; KEYS BUFFER.KEYS keyword in the SETINDEX statement overrides both.

SETINDEX_ Controls whether READFWD and READBCK statements VALIDATE_KEY validate a key value just read against the record. Default value is 0 (no validation). Individual environment variable overrides udtconfig setting; VALIDATE.KEY keyword in the SETINDEX statement overrides both.

SGLISTNUM Size of token stack for XREF, in UENTRY. Default is 500. Environment Variables for UniData (continued)

B-2 Administering UniData on UNIX Environment Variable Description

SQL_TMP_MODULO Number of temporary files used by UniData SQL for an equal join operation. Default is 7; if this is set to a number less than 7, UniData SQL uses the default. No upper limit. Must be set before entering udt/SQL, or before starting UniServer.

SUPPRESS_ORPHAN_ Determines if the guide utility reports orphan blocks. Orphan BLOCK_ERROR blocks could exist in both the primary file and the overflow files. guide reports these errors in the GUIDE_ERRORS.LIS file. If you want to suppress these messages, set the value of SUPPRESS_ORPHAN_BLOCK_ERROR to a positive integer.

TABSTOPS Specifies a number of characters to tab in UniBasic PRINT statements. Must be 1-76. The default is 8.

TMP Identifies an alternate directory for /tmp when additional work space is needed by UniData. Must be a directory path ending with /.

UDT_EDIT Identifies the path of the UNIX text editor UniData invokes when users execute the ECL ED command. The default is vi. This variable cannot be set to AE.

UDT_SAVELIST Allows you to specify a default list name for each UniData user. Set in the user’s .login or .profile. Users can also specify a list name when executing the SAVE.LIST command.

UDT_SELECTSIZE Size of a buffer used to keep select list in memory. If a select list is larger then the size of this buffer, UniData writes it to a file. If UDT_SELECTSIZE is not defined, the system uses a buffer size of 10 KB.

UDTBIN Location of UniData executables.

UDTERRLOG_LEVEL Determines the logs to which file open errors are written. If the value of this environment variable is equal to or greater than 2, UniData writes file open errors to the udt.errlog, and on Windows platforms, to the Windows event log. Otherwise, UniData does not log file open errors.

UDTHOME Location of the UniData home directory, which contains direc- tories including sys, demo, lib, work, sybase, and include. Environment Variables for UniData (continued)

B-3

Environment Variable Description

VFIELDSIZE Increases the size for the stack of C routines used to process formulas created in virtual fields. Default is 300. Define a larger number if users see “virtual field too big” errors in UniQuery.

VOC_READONLY If set to a nonzero number, allows UniData to run with read- only access to the VOC file.

VVTERMCAP The UNIX path of the vvtermcap file needed to run the UniData commands UENTRY, shmconf, and confprod. This variable is not necessary if UDTBIN is defined. Environment Variables for UniData (continued)

B-4 Administering UniData on UNIX Appendix Configuring SSL for Telnet C

Server Side Configuration:

To enable SSL for Telnet on UniData, you will need to edit four different files on the database server. The first two are system files named services and inetd.conf. Both of these files reside under the /etc directory on the unix server. Use vi or another suitable text editor to make the changes described below.

Add the following line in /etc/services:

telnets 992/tcp Where “telnets” is the service name. This can be any name of your choosing. 992 is the standard port number used for secure telnet servers and tcp is the protocol name for the service.

Add the following line in /etc/inetd.conf

telnets stream tcp nowait root UDTBIN_PATH/udtelnetd udtelnetd [-dN] [-o dir] Where “telnets” is the telnet service name you specified in /etc/services and UDTBIN_PATH is the path to the UniData bin directory. Udtelnetd is the UniData secured telnet server and it takes the following options.

Option Description

-d N Debug level. Where N is the debugging level to be specified from 0 to 3. A setting of 3 is the highest level of debugging and a setting of 0 means no debugging message will be recorded. The debugging message goes into the TMP_DIR/udtelnet-pid.log where TMP_DIR is a temporary directory and pid is the process id of udtelnetd invoked by inetd.

-o dir Specify the path to the temporary directory. The default setting is /tmp.

There are two new files introduced into the unishared directory on the server that you need to be familiar with, udtelnetd and .udscrfile. They are located on the database server, under /unishared/unitelnet. You can determine the location of the unishared directory by typing cat /.unishared. The two files are listed below.

udtelnetd.conf - This is the UniData telnet server configuration file and has the following format:

?????????????????????????? ???????? Where security_context_record_id and password are the security context record id and password used to get security context in a pre-generated security file (defined in .udscrfile). This security context record id is system-wide, which is managed on a per-machine basis rather than a per-user basis.

.udscrfile - This is the security file containing the path to the security context file. For more information on the Security Context File, refer to the Developing UniBasic Reference Manual.

C-2 Administering UniData on UNIX