Rocket U2 Databases
Incubator Training Samples
General Introduction
Published by the International U2 User Group
Authors: Brian Leach David Jordon Content
Contents Introduction ...... 4 The U2UG Incubator Project ...... 4 Exploring the Incubator Project ...... 5 Who Should Read This Guide ...... 6 Further Information ...... 6 Installing the U2 Bookstore ...... 7 Downloading UniVerse Personal Edition ...... 8 Installing the UniVerse Database Server ...... 9 Installing Dynamic Connect ...... 9 Installing UniDK ...... 10 Installing the U2 Bookstore Database ...... 11 Connecting to UniVerse ...... 12 Logging Into the UniVerse Environment ...... 14 Entering the Account Path...... 15 The UniVerse Command Shell ...... 16 Disconnecting Safely ...... 17 Fundamental Concepts ...... 18 UniVerse Accounts ...... 19 Account Flavors ...... 20 UniVerse Files ...... 21 Listing the Files in your Account ...... 21 Navigating through Listings ...... 22 Listing File Content ...... 22 Records and Fields ...... 23 Files and Dictionaries ...... 23 Listing a File Dictionary ...... 23 Records and Keys ...... 24 Dictionary Definitions ...... 24 Related Fields ...... 25 MultiValued Data...... 25 XML Listings ...... 26 UniVerse Programming ...... 27
Rocket U2 Incubator 2 The U2 Bookstore ...... 28 The U2 Bookstore and U2 Books Company ...... 29 U2 Bookstore Database Layout ...... 30 Book Details ...... 30 Clients and Sales Orders ...... 31 Purchases ...... 32 Exploring the Demo system through the command prompt ...... 33 Exploring the Demo System through Visual Tools ...... 37
Rocket U2 Incubator 3 Introduction
The U2UG Incubator Project
The International U2 User Group (“U2UG”) is a volunteer organization made up of developers, end users and other professionals associated with the UniVerse and UniData business platforms and the associated U2 product lines from Rocket Software, Inc.
You can find details of the U2 User Group on the website, www.u2ug.org .
The user group encourages participation in technical forums, manages a number of technical mailing lists, and runs a number of projects ranging from sourcing and filtering improvement requests for UniVerse and UniData through to various initiatives for creating educational materials for those wishing to learn about different aspects of the U2 technologies.
The Incubator Project is one such initiative.
The Incubator Project has been envisaged as a showcase for the U2 platforms, a central repository of examples, explanatory materials and resources through which developers can explore the potential of the U2 data model and can view working samples of some of the many techniques available for building real-world applications leveraging U2 technologies.
The Incubator Project will, over time, come to hold a wide range of applications and solutions, all of which are founded on a single database and that represent a single fictional company, “U2 Books”. By using a single database, this frees up contributors from having to worry about creating their own demonstration data and should simplify the learning curve for developers wishing to explore and to compare amongst the range of different technologies on offer.
The Incubator Project is a wholly volunteer effort. The U2UG Board would therefore like to express its gratitude to those who have taken the time, and who will take time in future, to contribute to the growth of this project.
Rocket U2 Incubator 4 Exploring the Incubator Project
The U2 technologies are very wide reaching, encompassing different data models, middleware APIs, target languages, data access techniques and application generation tools - and so can be daunting to developers who are encountering these for the first time. Even developers seasoned in other products that share the same data model, can find the sheer weight of the available features overwhelming.
For most, the problem is not that these technologies are difficult in themselves: the difficulty arises in simply knowing which of these approaches to use for a given situation.
The Incubator Project is therefore intended to give a flavour of these various options, by presenting a range of different applications all sharing the same database. By comparing these applications, and by examining how they are built, new and existing developers can assess the various strengths and weaknesses and discover techniques that they can adapt.
The Incubator Project therefore contains:
• A core database representing the fictional “U2 Bookstore”. • Additional databases holding the server-side components for each sample application. • Client side applications showing different data access or application development techniques.
Each of these applications is accompanied by documentation explaining how the application has been put together and pointing out any techniques of special interest.
In addition, this General Guide gives information on the Incubator Project as a whole, and background information on the U2 data model.
Rocket U2 Incubator 5 Who Should Read This Guide
This Guide has been prepared with three different audiences in mind:
• Developers who are experiencing the U2 technologies for the first time, and who are unfamiliar with the underlying MultiValue Database (MVBDMS) data model.
• Developers who are new to the U2 technologies but who have previous exposure to other MVDBMS platforms.
• Developers who are familiar with the U2 technologies but who wish to extend their technical knowledge into new areas.
The General Guide has therefore been divided into the following sections:
• General Introduction This explains the thinking behind the Incubator Project. You have already read this!
• Installing the Bookstore This explains where to find the bookstore application and how to get up and running. Recommended for all readers.
• Fundamental Concepts This explains the fundamental concepts that underlie the U2 Technologies in particular and the MultiValue Database model (MVDBMS) in general. Developers who have worked with MVDBMS platforms can skip this section.
• The U2 Bookstore Database This explains the layout of the demonstration database and the history of the fictional company, “U2 Books”. This is recommended reading for anyone looking at the Incubator samples.
Further Information
Further information about the U2UG and the U2 technologies can be found at the U2 User Group website, www.u2ug.org .
The U2 Product Documentation can be downloaded free from the Rocket website at www.rocketsoftware.com/u2
Rocket U2 Incubator 6 Installing the U2 Bookstore
This chapter provides instructions on downloading and installing UniVerse Personal Edition, the Client Tools required to explore the database and the U2 Bookstore sample database used for the Incubator samples.
Rocket U2 Incubator 7 Downloading UniVerse Personal Edition
The U2 Bookstore database is available for UniVerse and UniData.
Both UniVerse and UniData are supported on a variety of different operating systems, but the U2 Bookstore database and the Incubator Project are designed for a Microsoft Windows platform. If you already have a licensed copy of UniVerse or UniData on Windows, you can use that version to run the U2 Bookstore database. If not, you can download the free Personal Editions from the Rocket U2 website.
The Personal Edition of UniVerse is a fully functional product that is restricted to two concurrent users and that may only be used for non-commercial purposes, including training and demonstrations.
To download a copy of the UniVerse Personal Edition, navigate to the Rocket Software U2 page at: www.rocketsoftware.com/u2
Select the Downloads option from the Resources menu for a list of the tools and editions that can be downloaded:
You should download both the UniVerse Personal Edition for Windows and the accompanying UniVerse Clients for Windows .
NOTE: You will need to register on the website before you can download the products.
Rocket U2 Incubator 8 Installing the UniVerse Database Server
Once you have downloaded the two packages, you can proceed to install the UniVerse server.
You will need to log into your PC using the credentials of a user who has local administrator rights. To install UniVerse onto your PC, open up the UniVerse Personal Edition package and find the file named INSTALL.EXE.
This will display an installation menu similar to the one below:
Select the option UniVerse RDBMS to run the setup and follow the defaults throughout to install the database manager.
Do not select the NLS (National Language Support) or UVNET options.
Installing Dynamic Connect
The UniVerse Clients Package contains a number of tools and forms of middleware for working with UniVerse systems. Most of these are geared towards developers creating client/server applications using various Windows or Java based technologies.
Amongst these tools you will find the Dynamic Connect terminal emulator. UniVerse includes a terminal based interface, which you can use to explore the database and to run administration and other commands.
You can choose to use any terminal emulator – even the built in Windows TELNET client – but Dynamic Connect provides you with some useful additional features and it is available
Rocket U2 Incubator 9 free with UniVerse Personal Edition.
To install Dynamic Connect locate the INSTALL.EXE in the UniVerse Client package. Select the Dynamic Connect option to run the setup, and once again just select the default options throughout.
Installing UniDK
For the final part of the standard UniVerse setup you will need to install UniDK (the UniObjects Developers Kit). This contains the middleware used to build most client applications. Note that some of the incubator samples will require additional middleware, so please keep the downloaded client tools package to hand.
Select the UniDK option from the installation menu to run the setup.
UniDK installs a bunch of related protocols that provide the best means of building client/server applications. Select all of the options to install and just accept the defaults throughout.
Congratulations, you are now ready to run UniVerse!
Rocket U2 Incubator 10 Installing the U2 Bookstore Database
The latest version of the U2 Bookstore database can be found on the Incubator pages of the U2UG website. There you will also find each of the sample applications.
The U2 Bookstore databases for UniVerse and UniData are distributed as standard zipped archives.
The archives contain a number of directories, each of which contains database files that make up the parts of the U2 Bookstore. Both UniVerse and UniData allow you to place their database files anywhere on your hard drive, so you can unzip the archives anywhere: however it is recommended that you unpack this to the root of your system drive (usually C:\).
Once unpacked, you should see the following directories in Windows explorer:
The folders hold different sections of the bookstore database, and are organized as follows:
U2_books Bookstore Application entry point U2_data Demonstration Database data files U2_dw Data warehouse demo (future) U2_res Resources U2_safe Clean copy of data overwrite U2_Data u2_source Development resources only.
The database will be ready for use straight away.
Rocket U2 Incubator 11 Connecting to UniVerse
UniVerse applications, like most other applications, are generally client/server or web based.
But when it comes down to learning how UniVerse works, the command shell is still the best way to navigate around a UniVerse database - even though it may appear limited and primitive compared to the features of a web or Windows based application.
The comm and shell connects you to a UniVerse session, typically using a TELNET connection. So to begin, start Dynamic Connect from your Programs menu.
If this is the first time you have run Dynamic Connect, the New Session Wizard will start automatically. If not , select the Session Wizard option from the File menu:
Click the Next button and on the next page, select Windows Sockets as the transport.
On the next page you will asked for the name of the host system to which you wish to connect. Dynamic Connect c an connect to any of your UniVerse servers, so you need to be careful that you are connecting only to the copy of UniVerse on your PC.
Network transports use the name 'localhost' to refer to the local machine, so enter that at the host name prompt.
Rocket U2 Incubator 12
Click the Next button to run through the rest of the pages in the Session Wizard. You can just accept all of the defaults for these until you reach the final page.
Click the Finish button to start the connection.
Rocket U2 Incubator 13 Logging Into the UniVerse Environment
An y database or application environment needs to have some way to verify user credentials and to restrict access.
UniVerse does not have its own security model for defining operators (although many UniVerse applications may do so). Instead, UniVerse leave s the authentication up to the operating system.
When you connect to UniVerse using Dynamic Connect, you will be asked for a user name and password. Universe will always allow the local administrator to connect, so for now enter the name and password for a user with local administration rights.
Non-administrator users can connect to UniVerse, but need to be explicitly declared before UniVerse will allow them en try.
Rocket U2 Incubator 14 Entering the Account Path
A UniVerse server may typically be running a number of separate applications, or may be storing different sets of information used by different people or departments.
To keep each of these application apart, and to make administration simpler, UniVerse organizes itself into a number of separate working areas known as 'Accounts', each of which is an operating system directory containing UniVerse files.
Whenever you are working inside the UniVerse environment, you are placed in an account. The UniVerse TELNET server will ask which account you wish to enter.
If you unzipped the U2 Bookstore files at the root of your C drive, the account path will be:
C:\incubator\u2_books.
You may see a prompt asking you to update the account to match your current version of UniVerse. If so, enter a Y at the prompt and let the update proceed.
NOTE:
The first time you connect to the database it will run a number of setup procedures, including an rebuild of all the indexes. This may take a few moments.
Rocket U2 Incubator 15 The UniVerse Command Shell
The UniVerse Command Shell provides a text based interface to a UniVerse database. The Command Shell lets you type commands and see the response immediately (similar to the CMD shell under Windows). This makes it the best place to learn how UniVerse works, but it may seem a little alien if you are more accustomed to graphical, point and click environments.
The Command Shell presents a prompt in the shape of a greater-than sign (>), known as the TCL (Terminal Control Language) prompt, ready for you to enter a command. The good news - most UniVerse commands are English-like, and generally make sense!
Type the command WHO and press the enter key. Don't worry about whether it appears in upper or lower case. UniVerse will answer with a response similar to the one below:
>WHO 1292 u2_books From administrator
The WHO command identifies your command session. The first value (1292) is your user number: this is a unique session identifier that keeps track of your connection. An administrator can use this number if they need to disconnect you, send you a message or to release any resources that you have left behind.
The second value (u2_books) tells you in which account you are working. It is important to check this before you run any other commands so that you know that you will be working with the correct information.
The third piece of information (administrator) reports your user name. You can open several connections using the same user name and password, and once again this is mainly for the benefit of administrators.
You may find that your keyboard operates in a different case when connected to UniVerse. This is an idiosyncratic feature left behind by the original UniVerse developers: most UniVerse commands are in upper case and most UNIX commands in lower case, so they chose to invert the case whenever working in UniVerse. This is frankly annoying to the rest of us.
You can turn this off by typing:
PTERM CASE NOINVERT
Rocket U2 Incubator 16 Disconnecting Safely
When you have finished with UniVerse, it is important that you disconnect safely. This will ensure that any resources you have used are released and that UniVerse can decrement the license count for the number of concurrent users. If you do not disconnect safely from the command shell, you may be blocked from connecting again because UniVerse believes you are still connected.
UniVerse provides two commands for disconnecting safely, the OFF and QUIT commands. These will terminate your session and close your connection.
Type the OFF command to finish your session and close the connection in Dynamic Connect.
Rocket U2 Incubator 17 Fundamental Concepts
This chapter will familiarize you with the fundamental architecture of a UniVerse database application, and will introduce the ways in which UniVerse stores information. You will learn how to find files, list their content and describe the information available. You will also learn how to list real and virtual fields.
If you are already familiar with the MultiValue Database Model (MVDBMS) you may skip this section.
Rocket U2 Incubator 18 UniVerse Accounts
UniVerse has a deceptively simple architecture.
At t he top level, a UniVerse system is divided into a number of separate areas known as Accounts. Each Account is a kind of work space, that provides access to a particular set of information and the commands needed to work with that information. An account mi ght contain an application, a set of data, development code or just a subset used by one department. An account may also contain a database schema.
The rules for how a system should be divided into accounts are, like many things where the UniVerse platfor m is concerned, flexible and open to interpretation. Generally an account contains a set of related information and functions that together make up part or all of an application. For example, a typical system might include of three accounts: the first hold ing an order processing system, the second holding a personnel application, and a third holding football scores for the company league:
Physically, each account resides separate directory in the underlying file system. Logically, the arrangement is mor e flexible. Accounts can freely reference information held in other accounts, making the organization more of an administrative feature.
An order processing application , for example, might need to refer to the personnel files to record the name of the per son responsible for taking an order or to record a sale against their targets, but should not be able to see the other personnel information such as workers' salaries or directors' expenses.
Rocket U2 Incubator 19
Account Flavors
UniVerse may be a thoroughly modern business platform, but there are times when its long history catches up with it. One of those concerns the Account Flavor.
UniVerse began its life as an emulator. Its original purpose was to enable people to migrate applications from a variety of proprietary and closed MultiValue database platforms into a single open and UNIX based environment. Just like the various SQL dialects today, different MultiValue database manufacturers had introduced differences into their syntax, which meant UniVerse needed a way to support these variations. Fortunately these generally fell into a number of 'families', and so each account emulates the syntax used by one of these families.
The impact of this today is that each account on a UniVerse system has a particular 'flavor', which sets the various syntax options. So an account determines not only the information that you can see and the applications that you can run, but also defines how some of the commands that you use may be interpreted.
Fortunately most of the standard commands and most of the data retrieval languages operate transparently across all account flavours: but some of the other commands you will meet may differ slightly when you come to use them on your live systems.
You can view the account definition entry for your account by using a special form of the statement stack .L command:
.L RELLEVEL
This displays the release level of UniVerse and the account flavor.
Rocket U2 Incubator 20 UniVerse Files
UniVerse stores data and programs in structures called Files.
Files and Tables
As a developer you are probably already familiar with the concept of database tables. UniVerse files and tables may look similar, but beware - they are not the same thing!
UniVerse implements a SQL data model over the top of its native MultiValue data model: the SQL model adds schema and constraint information to files to create SQL tables. However, the use of SQL is very rare amongst UniVerse developers.
You can think of a UniVerse file as being very similar to a filing cabinet in an office. A filing cabinet typically contains many records - sheets of information often organized into formal structures. Each set of cabinets may be dedicated to holding just one type of record: customer details, invoices, despatch notes or similar.
UniVerse files work in the same way. Each file stores any number of pieces of data organized into records. Just like the paper forms in a filing cabinet, these records generally (but not always) contain similar information.
A single UniVerse file may hold hundreds of thousands or millions of records. Just like any good filing system, the records are organized in such a way that the database can retrieve any particular record instantly.
Listing the Files in your Account
Because UniVerse holds all of its data in files, the first stage when looking for information is to discover the files that are visible in your account.
To see the files in the demonstration account, type the command LISTFILES or LISTF for short:
LISTF
The LISTFILES command displays a listing of all of the files that are directly accessible from your account. The listing will include any files that have been created in that account, and certain files that may be held in other accounts.
Rocket U2 Incubator 21 Navigating through Listings
The LISTF command is displayed one page at a time, using the number of lines UniVerse expects your terminal emulator to support. This is true of many TCL commands when run through the command shell.
At the bottom of each page, you will see the message:
Press any key to continue.
For most commands, this prompt gives you three choices:
• Press the Enter key to display the next page.
• Type Q (short for QUIT) to stop the list at that point, and to return to the command prompt. This prevents you having to page through the rest of the listing.
• Type N (short for NOPAGE) to turn off the paging for this listing. This will then run through the rest of the listing without pausing.
Listing File Content
When you start looking through the list of files, the names that appear may or may not give a clue to the information that the files are holding. So the next step will be to start looking through some of the file content.
There are various means to look at the data held in a file, some of which can be generally more helpful than others. In a perfect world (or a demonstration application), it should be possible for you to get a summary view of the information held in a file by using the command:
LIST filename
where filename is the name of the file to be listed.
For example, type the following command to see the Book Titles:
LIST U2_BOOKS
Rocket U2 Incubator 22 Records and Fields
Each UniVerse record contains a number of pieces of information. To distinguish between each of these pieces of information, each record is divided into a set of entries known as Fields or “Attributes”. A record may have no fields, one field or any number of fields.
The LIST command creates a columnar listing of various selected fields from the records.
Files and Dictionaries
The default listing of a file (using LIST filename) may or may not show you anything useful: you may get lucky and see the information you need included in the display in just the way you want to see it, but in most cases you will need to discover what other information is being held in the file.
This introduces a defining aspect of the UniVerse database: the File Dictionary.
MultiValue databases have a number of pronounced features that separate them from the rest of the database market, one of the most important of which is the use of File Dictionaries. Each UniVerse file is made up of two separate entities:
• a Data section that holds the records. • a Dictionary section that describes the content.
The File Dictionary holds the Metadata that describes the content of the file. This Metadata is the key to understanding the enquiry languages and other structures used by UniVerse.
UniVerse divides records into fields. Each field generally holds one piece of information: the U2_BOOKS file, for example, contains records defining each title in the bookstore catalogue. Each title record contains fields holding the short title of the book, the author, the ISBN number, genre, department, price and stock level amongst others.
Whenever the LIST command encounters a file, it looks at the file dictionary to find the information for each of the fields it needs to display.
Listing a File Dictionary
The file dictionary is always referenced using the name:
DICT filename You can list the content of the dictionary in the same way that you listed the content of the data file: by using the LIST command
LIST DICT filename
Rocket U2 Incubator 23 At the start of each dictionary listing are the field definitions. These are identified by a 'D' (standing for Data) in the Type column of the list. These define the real fields that occupy set positions in each record: the position number is given in the next column.
You can use the familiar LIST command to display individual columns, following the syntax:
LIST filename field
If you are more comfortable using SQL, you can perform the same command using a Universe SQL SELECT statement:
SELECT @ID, SHORT_TITLE, ISBN FROM BOOK_TITLES;
UniVerse supports two enquiry languages: a variant of the MultiValue enquiry language called RetrieVe and a variant of SQL. Both use the same dictionaries and can be used largely interchangeably. Most UniVerse users prefer RetrieVe which is more English-like and has a more friendly syntax.
Records and Keys
When you list the content of a file using the LIST command, you will see an additional column to the left of the fields you request. This is the record key, or @ID field.
Each record in a UniVerse file must have a unique identifier, or record key. UniVerse uses the key when storing and retrieving records, so if two records are supplied with the same key the second will overwrite the first.
Internally, UniVerse applies a hashing algorithm to the key to determine the storage location. In a well maintained system, accessing specific records by key is instantaneous, regardless of the size of the file and number of records held. UniVerse applications can work directly with record keys to give a very fast and light response. For this reason, MVDBMS developers generally choose meaningful keys: a customer account number, a short descriptive word, a date.
Dictionary Definitions
UniVerse dictionaries go further than simply defining the file content. The real power of the dictionary lies in the fact that it can contain other content than pure field definitions. In particular, Dictionaries contain definitions of virtual fields: fields that are based on calculations or expressions. These are used in reports and enquiries in exactly the same way as real fields, so that once a developer has created them there is an easy and consistent means of building derived information.
Look further down a dictionary listing and you will find another series of definitions, this time identified with an I (Interpreted) in the Type column. These are the calculated fields.
Rocket U2 Incubator 24
A calculated field may, for example, report difference between the number of units in stock and the number that have been ordered; define pricing calculations, tax, line totals, order balances and similar financial operations.
Virtual fields are not restricted to numeric calculations. They can also be used perform complex functions, to access remote data, to combine data for selection or to reformat data for display and selection.
Because they are held in the dictionary they are available for any enquiry commands without the need to reproduce the column expressions that make up their calculation separately for each enquiry statement.
Related Fields
One particularly powerful feature of virtual fields, is that the data they present can be sourced from other files – similar to using a lookup in a spreadsheet. A virtual field can present data as if it were part of the file, when in reality it is held somewhere else. This makes database enquiries very much simpler than using join clauses!
MultiValued Data
Most databases use flat representations of data: information is stored in two dimensional tables of columns and rows, like a spreadsheet. If you want to hold something that is three dimensional, like a sales order with a number of separate item lines, you need to break this across two tables: one for the header information and one to hold each item line. This is known as a 'Master-Detail' or 'Parent-Child' relation.
The problem with this format is that in most cases you have relate these all of these rows back together again whenever you need to reassemble that sales order for display. This need to relate information together is the heart of the 'relational database' model. It is also a costly operation in terms of memory and processing power and, whilst academically useful as a model, is practically inefficient. Readers of an RDBMS disposition should note that Codd himself described this as a design, and not an implementation, model.
MultiValued Databases sidestep many of these issues by allowing each field to hold more than one value: an arrangement that more closely matches the way information is handled in real life.
A MultiValued Database does not need to use two separate files to hold master/detail records, such as a sales order. In such a file, each sales order is held in a single record that contains both the header information – the customer details and the time and date of the order – and the individual item lines for the audio titles and their quantities. The LIST command helpfully presents the 'detail' fields side by side if at all possible.
Rocket U2 Incubator 25
As a matter of fact, MultiValued Databases can go a level further. Each field can contain multiple values, each of which can in turn contain multiple sub -values.
Historically some MultiValued databases have been poor at handling sub -values for reporting, and thus the convention has arisen that su b-values are normally reserved for programmers only and are rarely used for enquiry.
XML Listings
Because UniVerse supports repeating values, a listing in UniVerse can behave more like an XML document with nested entities, than a traditional table. So it is not surprising that RetrieVe and UniVerse SQL can both create XML recordsets. These can be captured and exported through various APIs for interchanging data with other systems, and as a convenient means of gathering data for web and graphical client ap plications.
The TOXML keyword converts a LIST or SELECT listing into an XML format.
LIST filename TOXML ELEMENTS
Rocket U2 Incubator 26 UniVerse Programming
Any database platform of substance supports some way to define stored procedures that localize complex data operations to run within the database itself. UniVerse and UniData go further than this – they are fully equipped with a substantial programming language that can be used to build entire applications, and contains features for database operations, device control, printing, exporting, XML navigation and generation, data encryption, socket support, external data access… the list goes on.
Like many modern languages, is modelled after a variant of Basic. Just as Visual Basic .NET has added features appropriate to a graphical, object oriented environment, so UniVerse Basic has added features appropriate to a business application and in particular to working with the file model.
UniVerse BASIC code is written as programs and subroutines (and external functions, but we will ignore these for this guide). Programs are standalone routines used for performing fixed tasks within the database environment: generally running utilities or reports. Subroutines are external blocks of code that can be called from both inside and outside the UniVerse environment, similarly to the stored procedures in some other traditional databases. Subroutines are the building blocks of applications and the way to expose your business logic to the outside world.
Rocket U2 Incubator 27 Introducing the U2 Bookstore
The U2 Bookstore
This chapter provides an overview of the U2 Bookstore database used for all the Incubator sample applications, and provides some assistance with navigating and exploring the database.
Rocket U2 Incubator 28 The U2 Bookstore and U2 Books Company
“U2 Books” is a fictional book store with a physical and on-line presence.
The store has been in existence for three years, with a steady growth in sales driven by a combination of new clients – most of whom enrol through the website – and repeat business from existing clients who appreciate their personal service.
The store maintains a small catalogue of books, including a number of audio books.
Customers can place sales orders over the phone or web. As a start-up company they need to keep costs down, and so they keep their stock levels to a minimum, generating purchase orders for new stock from daily batch routines that examine the outstanding sales requirements and current stock levels. The books are sourced from a select group of suppliers. Even so, occasionally books are rejected on receipt.
The store has experimented in the past with promotional offers, and continues to look to new technology to help them build their business.
Rocket U2 Incubator 29 U2 Bookstore Database Layout
The following diagram displays the main layout of the U2 Bookstore:
U2_ORDERS U2_BOOKS ID U2_PURCHASES CLIENT_ID ID ORDER_STATUS TITLE ID INSTRUCTIONS AUTHOR_ID ORDER_DATE U2_CLIENTS DELIV_ADDR[] ISBN SUPPLIER_ID ORIGIN_CODE DEPT SHIP_COST ID SHIP_ID GENRE DELIV_DATE SURNAME SHIP_COST MEDIA FORENAME DESPATCH_DATE PUBLISHER_ID ADDRESS[] SALES_ID SALE_PRICE ITEMS[] COUNTRY STOCK_LEVEL BOOK_ID JOIN_DATE ITEMS[] MIN_ORDER ISBN ACCOUNT_STATUS ON_ORRDER BOOK_ID SUPPLIER_CODE TAX_DODE PRICE QTY SUPPLIER_ID PRICE TAX_CODE PURCH_PRICE TAX_CODE ORDER_QTY SUPPLIER_CODE DESPATCHED REJECT_QTY LONG_DESCRIPTION PROMO_ID TAX_RATE
U2_PAYMENTS U2_PROMOTIONS U2_SALESTAX U2_SUPPLIERS
ID ID ID ID PAY_DATE SHORT DESCRIPTION FULLNAME CLIENT_ID DISCOUNT_PCT RATE ADDRESS[] AMOUNT START_DATE PHONE END_DATE FAX BOOK_ID[] EMAIL MIN_QTY[] SHIP_COST
Book Details
The Book Store is based on a catalogue of audio and traditional books on stock, held in the U2_BOOKS file. This divides the stock into ADULT and JUNIOR sections, and for audio books also defines the media. Each book has a current stock level and a minimum order quantity set by the supplier: this is used in the automatic purchase order generation.
Regular books exempt from sales tax in the UK, however this is the not the case for audio books. The tax code on the U2_BOOKs entry is linked to the U2_SALESTAX file that holds the rate for each code type.
Authors are identified by their AUTHOR_ID in the U2_BOOKS file. This is a foreign key to the U2_AUTHORS file, where you will find the author name and also a multivalued list of their titles for lookups. The same is true of the publishers, held in the U2_PUBLISHERS file.
Rocket U2 Incubator 30 Clients and Sales Orders
The sales orders are held in the U2_ORDERS file. For ease, these have been given composite keys: each sales order is identified by the date and time of the sale, followed by a suffix to disambiguate sales orders placed at the same moment. This prevents the need to maintain an accumulator or auto-numbering scheme for new sales orders.
Each sales order is associated with a client, with the client details being held in the U2_CLIENTS file. The client details record the client joining date and contact details. An index of the client orders is maintained manually by the system, using the U2_CLIENT_ORDERS file.
The sales orders contain single valued headed information, including the client identifier, status, delivery details and shipping code. The shipping costs and descriptions are read from the associated U2_SHIPPING file. An origin code records whether the order was placed over the web, by phone or via email.
The second part of the sales order is the association that makes up the order lines. Each sales order may have any number of order lines, which are held as a set of multivalued fields. These record the multivalued BOOK_IDS, relating to the U2_BOOKS file, the current sale price when the order was filed, quantity requested and tax code. The order total is not stored, but calculated on the fly taking into account the order line totals, tax lines, shipping costs and any promotions in force.
Promotions are separately defined in the U2_PROMOTIONS file. These offer discounts that come into effect when the client has ordered a given quantity of books from a specified list. For example, readers of Terry Pratchett will receive a 50% discount on his works, where for Stephanie Meyer there is a 100% discount on the second book ordered from the list (i.e. buy one get one free).
Payments are also recorded for clients, though by design these are not directly associated with an order: a client may part-pay an order or a payment may cover multiple orders. Unlike the manual index between the clients and orders, the payments in the U2_PAYMENTS file hold a system index on the CLIENT_ID.
Rocket U2 Incubator 31 Purchases
As books are ordered, so the company must refresh its stock. It does so by raising purchase orders on a daily basis, though for this demonstration the routines must be run manually – in a live environment these would be constantly running background processes.
Each book is identified with a preferred supplier, using the SUPPLIER_ID field. This is a foreign key to the U2_SUPPLIERS table, where the supplier details and standard shipping costs are stored.
The purchase order creation checks all orders received for the day against the current stock and raises purchase orders to the suppliers: one purchase order per supplier per day for the required stock. These are stored on the U2_PURCHASES file.
Occasionally the wrong books may be sent by the supplier, or the books may not be in a suitable condition. In these cases the books can be rejected, and this also feeds into the current stock level calculations.
This describes the main parts of the U2 Bookstore Database.
Rocket U2 Incubator 32 Exploring the Demo system through the command prompt
This section contains additional details about the U2 Bookstore for those who are unfamiliar with the U2 technologies. In this section we will explore some of the parts of the book store using the command prompt, and later using a GUI.
The two folders U2_books and U2_data contain the application business rules and the data respectively. The data and business rules can be stored in the same directory, however for the purpose of the demonstration we have split them to demonstrate some of the differences to the SQL pattern of schema/owner/equity
ACCOUNTS VS SCHEMAS
Here you will note a difference from SQL environments. Instead of a single operating system files made up of databases full of tables, UniVerse has an account directory for each database and the tables are physically stored as separate files within that account. This you can see through Windows explorer.
COMMAND PROMPT
Whilst U2 has a range of visual tools to view databases and programs, we will first get familiar with the command environment to give a base understanding of the application and data server environments. We will Telnet to this environment through Dynamic Connect.
Click on “Start menu>All Programs>U2 Rocket>Dynamic Connect>Dynamic Connect”. When prompted for the user name and password, enter the username and password you used to login into your PC. Enter the pathname to the U2_books directory, i.e. c:\incubatory\u2_books. You will now have a greater than prompt.
Rocket U2 Incubator 33 THE VOC
The first thing to learn about is the VOC file. This is a special file found in each account directory that contains pointers to all the commands and files that are visible to that account.
VOC stands for “vocabulary” and organizes the available words that make up UniVerse commands along the lines of the English language. Commands like WHO, LIST and ED are Verbs. A Sentence is made up of Verbs and nouns, like LIST VOC.
A Paragraph is a series of sentences and is like a script of multiple commands.
To start off type LIST VOC
ENGLISH QUERY LANGUAGE
The UniVerse Database can be accessed through the traditional SQL or a user friendly English like query Language.
Type LIST U2_BOOKS BY TITLE TITLE AUTHOR_NAME
You can see a list of books. Type Q to exit the list.
Now type SELECT TITLE, AUTHOR_NAME FROM U2_BOOKS ORDER BY TITLE;
We will use the English commands going forward to get you familiar with the alternate query language.
BUSINESS RULES (SOURCE CODE )
UniVerse has an integrated application server and data server. The application server has a sophisticated business programming language designed for building complex business rules. We can have a quick looks at these programs through the command line.
Type LIST books.bp
Here you see a list of business rules or otherwise called subroutines. We can edit those through the command prompt to see what they look like.
Type ED books.bp u2_getBookData
Rocket U2 Incubator 34 DATA FILES
We can use the command LISTF (short for LISTFILES) to see the data files, however this will also display system files. To just show the data files we are interested in enter the following command.
LIST VOC WITH @ID LIKE U2... AND F1 LIKE F...
Now you can see a list of files and tables related to this account.
Let us look at how this file/table is referenced in UniVerse. The VOC stores a pointer to each file that the database can see from within an account.
Type ED VOC U2_BOOKS
0001: F Books in Catalogue 0002: ..\u2_data\U2_BOOKS 0003: ..\u2_data\D_U2_BOOKS
The first field or “attribute” starts with F indicating it is a file. The second field identifies the location of the data file The third field points to the dictionary which is the schema layout.
As you will note the data file and dictionary are actually in a different account to the one we logged in. Through remote pointers we can separate data and the application server if we desire.
Type Q
Now let us look at the schema for this file/table.
Type LIST DICT U2_BOOKS
Now you can see a list of fields in this file/table.
TITLE D 1 Title 70L S
The field TITLE is a data field and is in the first column. There is a conversion parameter that can be set for date, monetary formats. We can also provide a description, width and justification.
Type
Note the following field:
SUPPLIER_NAME I TRANS("U2_SUP Supplier_name 20L S PLIERS",SUPPL IER_ID,FULLNA ME,"X")
Rocket U2 Incubator 35 This is a virtual or calculated field. Instead of it being a “D” (Data) type this is an “I” (Interpreted) type. The calculation is built into the dictionary instead of being built in the query making it reusable and also making it easier for developers to see all the possible calculations that have already been defined. The above item gets the supplier name from another file/table without having to do a SQL join.
Type Q
HANDLING MULTI -DIMENSIONAL DATA
Type LIST DICT U2_ORDERS
Note the following BOOK_ID D 10 Book_id 10L M ITEMS QTY D 11 MD0 Qty 8R M ITEMS
These are flagged as “M” for multivalue and provide a third dimension for these attributes. For this attribute, the row can have multiple sub rows. These items are associated by an association name “ITEMS”. Let us look at how this looks in the database.
Type LIST U2_ORDERS ORDER_STATUS CLIENT_ID ITEMS
As you can see for each record/row you can see multiple item details.
Rocket U2 Incubator 36 Exploring the Demo System through Visual Tools
We can use the U2 Basic Developer Toolkit to view programs and tables through a GUI interface. You may need to download this from the Rocket U2 website.
To start the toolkit, click on “Start>all programs>Rocket U2>Basic Developer Toolkit”
SETUP ACCESS TO THE DATABASE Right mouse click on U2 Servers and select New U2 Server. Enter the details for the server you wish to connect to.
Once that is setup, double click the server to connect. Enter the username and password to access the server. (This is the Microsoft windows login).
Expand that server and right mouse click on accounts and select new U2 Account.
Enter a name for the account and the pathname to the server as indicated right.
Now expand accounts and click on the account U2 Books.
Now you can see the database files, catalog programs and other parameter files related to the account.
Rocket U2 Incubator 37 VIEW AND MODIFY DATABASE DICTIONARY
Select a file/table and right mouse click and select open dictionary
You can modify or add a dictionary item. A strength and weakness of UniVerse is that the dictionary schema is not tightly linked to the data base. Data is variable length hence is not impacted by the width set. Data is not typed as in SQL databases hence changing conversion does not impact the data, it only impacts how data is reported or imported through UniVerse tools. It is also easy to add new attributes/columns without impacting the database.
Whilst one can use SQL to update a database, business rules (subroutines) read and write directly to the database. Data verification is done by the program, not by the dictionary.
VIEW AND MODIFY PROGRAMS
Expand books.bp to see all the subroutines. Click on a subroutine to view the code.
Rocket U2 Incubator 38
Understanding the Code
To get you started in exploring the demo system, we will step through some of the code and explain what it is doing. There are two parts to this application, c# client code and U2 Business Rules. There are a number of interfaces to the UniVerse database, ODBC, OLE-DB, .Net, Java, web services, etc. For the purpose of this demonstration we will look at the .Net interface. As in the below diagram, the C# client interfaces through an API called UniObjects which interfaces with the subroutines in the application server and the data in the data server.
Application & Data Server
C# UniObjects UniBasic UniVerse .Net Client Business Rules Database
UNIBASIC BUSINESS RULES Using the U2 Basic Developer Toolkit as described in the previous chapter, let us open the u2_getBookData in the books.source file.
(The $ commands are compiler options and we do not need to worry about these for the moment, comment lines start with an “*”)
SUBROUTINE u2_getBookData(BookId, LockFlag, BookData, ErrText)
This defines this code as a subroutine. U2 applications are typically built from subroutines, each of which encompassing a single operation or a set of related operations. Subroutines can be called from other subroutines, or from the various APIs. You can also build stand alone programs that do not have parameters and functions that return values.
Rocket U2 Incubator 39 The parameters in between the brackets are for passing values between this subroutines and the routine calling the subroutine. When we view the C# code, we will see how data is passed between the .Net client and the UniVerse subroutine. These parameters are combined input and output parameters.
$INCLUDE books.source u2_common.h
This statement includes lines of code from another source. Rather than duplicating common pieces of code in each program, we put them in another record and just include it in each program as required with the $INCLUDE command.
A common use for this is to give the fields in an array a meaningful name, instead of REC<1> we would equate REC<1> to BOOK.AUTHOR to make code more meaningful and easier to modify. The EQUATE (or EQU) statement performs a literal substitution in the compiler and thus can be used to define constants or replace expressions.
ReadU BookRec From F.BOOKS, BookId Locked
This line reads a book record from the books file. There is a routine to open this file that is included in the U2_init source code. You must open a file before reading the file. The read/write feature of UniVerse is different to the cursor processing of SQL. We read records based on a primary key and get the specific record requested.
The U appended to READU is to lock record. UniVerse is very efficient in handling locking compared to SQL and uses a pessimistic locking model. You can guard against contention over updates to a record by locking that record so that no other process is allowed to change it until you have completed your updates. However with a disconnected process such as web services, you would need to cater for optimistic locking as record locks are lost at the time of disconnection.
GoSub BuildData
This statement navigates to a block of code elsewhere in the routine that begins with BuildData: and ends the block with a Return statement. This is an internal subroutine and works similar to called routines in other languages.
BookData
When we read a record from a file, the resultant data is placed into a structure called a dynamic array: an array with unlimited fields, values and subvalues. We use the <..> operator to extract or update a specific field from such an array. BookData and BookRec are record variables. . The parameter U2.BOOKDATA.TITLE is a constant for that attribute and is defined in the $INCLUDE books.inc U2_BOOKDATA.h block of code. Dynamic array elements are numbered from one: Rec<1> get the first field in the record Rec. Rec<2,3> gets the third value of the second field of the record Rec.
Rocket U2 Incubator 40
The above is only a brief explanation of the code. You can find more information about Basic commands in the documentation that comes with the UniVerse install.
Most of these routines are designed to be called from a client UI. The documentation on the U2_Books C# Client provides additional information on how the routines are used.
Rocket U2 Incubator 41