A REAL-TIME, INTERACTIVE, MU LTI PLE COMPUTER SYS EM R. B. McDowell P. F. Bohn N. K. Brown P. M. Kirk A. G. Witte

Introduction

HE COMPUTER, IN THE PAST DECADE, has grown function generation of the aerodynamics along from a device normally employed to manipu­ with the computation of trajectory, while the ana­ lateT numbers to a virtual automaton capable of log computer was to handle the rather high data­ mechanizing a very wide variety of processes. As rate, inherently analog functions involved in the the variety of applications grew, it became neces­ control system. The high precision and easy func­ sary for some applications to tie several computers tion generation of the digital computer could then, together. The connecting together of computers in combination with the high speed of the analog and the use of increasingly versatile terminals with machine, provide a single real-time simulation of these computers seems now to be replacing the all major aspects of a missile system. main trend of the 1950's and early 1960's, which The key problem in combining the two com­ was toward larger computers. Large computers puters was the design of linkage equipment to are still essential, and indeed are often the key to match the input-output characteristics of the two the interconnection of smaller computers and ter­ computers, including the control signals necessary minals; but the ability to interconnect is now a for proper timing of each machine's operations. dominant characteristic in the present general ex­ The first linkages or interfaces were designed pansion of the versatility of computers. in 1955 by EPSCO, Inc. for Ramo-Wooldridge Probably the earliest concept of interconnected Corp. (later Space Technology Laboratories) and computers consisted of an analog computer tied to Convair Astronautics. Key problems soon became a digital computer to form a hybridl computer. apparent: The original rationale for such a system was to 1. Too little control was centralized in the provide a real-time simulation * of a guided mis­ easily programmable digital computer and too sile, including aerodynamics, a long high-precision many unsupervised all-hardware analog opera­ trajectory, and the analog control system. The tions had to stay in precise voltage and timing digital computer was to handle the complicated adjustment for long periods of time. 2. The digital computer was too slow to keep 1 For a comprehensive description of hybrid computers and their applications see the book H ybrid Computation, by Bekey and up with the analog computer in many problems, Karplus, John Wiley & Sons, New York, 1968. and in any case had to be dedicated to handling * Simulation is the analytical or empirical modeling (analogy) of a theoretical or actual physical system. one program at a time.

2 APL Technical D igest Using existing technology, generalizing and extending it, a large real-time, interactive, multiple computer system has been developed. It is believed that this is the first time that real-time, interactive, and multiprogramming features have been successfully combined in a genera/­ purpose computer network. The work done has been oriented to a major extension of the use of computers for simulation. However, much of the interface and software design and implementation are sufficiently generalized to have many applications in systems other than simulation.

3. The use of vacuum tubes, along with too years with the key role of hybrid computers to little error checking in the hardware and software, simulate the Apollo guidance, control, aerody­ resulted in poor reliability and difficult mainte­ namics, etc. These massive simulations were used nance. The linkage, for example, required nine primarily for analysis, design, and testing before racks and contained several thousand vacuum and during hardware development. tubes. A second, and initially somewhat less conspic­ 4. The analog computers required a device uous aspect of computer interconnection also be­ controller and redesigned devices such as auto­ gan to be developed during the 1950's. This was set potentiometers, programmable logic, function the use of terminal devices located remotely, 2 switch control, automatic digital voltmeter addres­ wherever the user might be located, but tied to a sing, etc. in addition to the mode control and data central digital computer. The impetus for such transfer operations that were initially incorpo­ terminals grew from three needs: rated. 1. To extend standard computer peripheral 5. A way was required for the operator/ pro­ functions beyond the central computation center, grammer to communicate on-line with both com­ so that printing and program submission would be puters to modify his program or obtain variou more conveniently available. related computer services. 6. Efficient use of the digital computer re­ 2. To provide on-line conversational features quired a way for non-real-time jobs to run during to the user, minimizing turnaround time and the the increments of central processing unit time not user's dependence on computing center personnel. required by the real-time job. Such terminals as typewriters and graphics devices characterize these terminals. Despite these formidable limitations, both the Ramo-Wooldridge and Convair systems were used 3. To allow direct data acquisition and/ or pro­ (albeit laboriously) to successfully solve several cess control at remote sites to eliminate the need aerospace problems. As a consequence, an appre­ for separate, dedicated computers or the awkward ciation of what might be accomplished by a hy­ brid computer system spread rapidly within the 2 An excellent text on this technology in terms of the hardware is the book Telecommunications and the Computer; by James aerospace industry, culminating in the last severa] Martin, Prentice-Hall, Englewood Cliffs, N.J., 1969.

November - December 1971 3 and time-consuming manual transfer of data by burden be placed on the interface for each device way of magnetic or paper tapc or by punched to be interconnected in order to minimize and cards. hopefully eliminate hardware or software design The terminals used to meet these needs are of changes to previously connected devices. a wide variety, but fall into two general classifica­ 5. The devclopment of terminal support soft­ tions: (a) noncomputer or "nonintelligent" de­ ware sufficient to allow utilization of all hardware vices and (b) computer or "intelligent" devices. features of each interconnected device. On-line The basic difference between the two types is, of operator / programmer communication with exe­ course, whether or not they have a general (usu­ cuting programs is a key feature of much of this ally programmable) computation capability. Mini­ type of software. computers, t which have recently come into prom­ 6. The development, as a part of the basic inence, are finding a major application as the key supervisory software operating system, of such clement in the so-called intelligent terminal. Type­ features as multiprogramming, real-time services, writers are far and away the largest category and multiprocessingt which automatically adjust within the nonintelligent terminal classification. the computer operations to the changing demands Finally, a last and rather recent class of com­ on the overall system. These features are aimed at puter interconnection is that of the digital com­ maximizing the efficiency of the total system, while puter network where several computers, often at the same time providing each user with what quite large ones, are connected together to pro­ appears to be a computer dedicated to his partic­ vide immediate access to a comprehensive data ular job. base. Usually such systems also provide extensive, A seventh key technical problem area for inter­ though as yet rather specialized, computational connected systems in general is reliable, long-dis­ and other data handling services. tance, high-speed data communications. This, All these types of system, however, must find however, was not a major factor for the systems solutions to the same basic hardware and software discussed in this paper. problems associated with the development of the hybrid computer. At this point, it will be useful Development Context to reorganize and restate these technical problems During 1967 and 1968 the Laboratory identi­ as follows: fied four primary areas where simulation require­ 1. The development of comprehensive software ments exceeded existing capabilities: supervisory services to allocate or manage, as well 1. Hybrid computing. as error-check, all hardware and software opera­ 2. Interactive, all-digital simulation coupled tions. IBM terms this the operating system. Others with operating hardware. variously call it the monitor, supervisor, etc. 3. Missile and radar receiver simulation. 2. The development of parallel hardware sub­ 4. Radar system evaluation using realistic com­ systems, particularly for input-output (I/ O) chan­ puter-generated target scenarios. nels to speed up key functions by making them Such needs usually had been handled by sepa­ less dependent on the computer's central process­ rate, stand-alone, often special-purpose computers. ing unit (CPU). In order to gain projected economies of scale in 3. The use of solid-state components in place terms of cost per unit work, and to provide a gen­ of vacuum tubes and electromechanical devices to eralized system capability more adaptable to fu­ increase both speed and reliability. ture needs, a decision was made to develop a 4. The development of sufficiently sophisticated series of computer-to-computer links using the controllers within the terminal devices or associ­ Computing Center's IBM 360/ 91 digital com­ ated data transmission channels to allow simple puter as a general-purpose central processor. Rel­ software-generated commands to exercise all hard­ atively specialized functions were minimized and ware features of the terminal regardless of how confined to smaller computers and/ or related in- complex or specialized. It is important that this :j: Multiprocessing is a system of two or more computers which share adaptively, according to work load, their resources in mem­ t Minicomputers are a class of small digital computer generally ory, peripherals, etc. This was considered to be beyond the im­ priced under $25K, using 16-bit words, accompanied by minimal mediately forseeable technology and was, therefore, not attempted software. in our design. It is mentioned here for completeness.

4 APL T echnical Digest terfaces. § The other computers and specialized The intent, then, of our planning was to meet consoles essentially became terminals to the cen­ the four Laboratory needs listed above by inter­ tral computer. This in turn focused the engineer­ connecting into a single system hybrid computer, ing effort on the implementation of links and the all three types of terminal discussed earlier, and support of these links by software in the central some aspects of a digital computer network. These computer. added systems were to have a variety of inter­ The 360/ 91 already contained a highly devel­ active, real-time, and multiprogramming (time­ oped multiprogramming monitor (OS/ MVT) as sharing) capabilities. In addition, none of the well as time-sharing of typewriter and printer / standard capabilities already available on the 360/ card-reader terminals. A conversational program­ 91 were to be impaired by the addition of the new ming system (CPS) was also in operation, al­ interconnections and supporting software. though it did not prove useful for our particular needs. Two million eight-bit bytes of main core System Configuration storage and a large amount of disk storage were Figure 1 shows the overall APL computing also available. However, there were four items complex. Figure 2 is a hardware block diagram that had to be developed: of the links that are the subject of this paper. The 1. A real-time monitor that would allow the hybrid computer consists of an EAI Model 680 multiprogramming part of the system to continue analog computer connected to the IBM 360/ 91. to operate in background. The simulation console (SIMCON) allows ex­ 2. An I/ O channel that would permit at least ternal hardware to be connected to the 360/ 91 to 200K 32-bit words per second to be transferred operate with all-digital simulations. Missile and simultaneously into and out of the 360/ 91. radar receiver simulations are accomplished on 3. Terminal support programs to allow inter­ the signal processing simulator (SPS), which may active use of the links that were planned. be tied to the 680 alone or to the complete hy­ 4. Appropriate terminal-end interfaces. brid system via the 680. Finally, the radar system Probably the most important work done was in evaluation capability is implemented by tieing a working out in detail the sequence of operations portion of the radar through a Univac Model necessary to solve these four technical require­ ments. Specifically, we had to decide: 1. How the hardware would communicate with 3 REMOTE DIGITAL hardware. COMPUTERS 2. How the software would communicate with APPROXIMATELY 11 1 GRAPHICS TERMINAL software. REMOTE ENTRY USING STANDARD TERMINALS LEASED LINES 3. How the software would communicate with hardware. J=----=-- Here we capitalized on the successes and failures L IBM 3"60/91 of others, as well as the experienced assistance of IBM and Electronic Associates, Inc. (EAI). IBM I DIGITAL COMPUTER l ===""

developed a real-time monitor and a high-speed APPROXIMATELY 70 1 DATA CELL I/ O channel adapted to the overall system plan. TYPEWRITER-TYPE 5 DISK UNITS EAI likewise provided certain basic hybrid sup­ TERMINALS 4 DRUMS 4 MAGNETIC TAPE port software and an interface for the hybrid UNITS computer. Further work was done in-house to de­ 7 PRINTERS AND CARD sign and implement terminal support programs, READERS in particular, most of the interactive features, and HYBRID COMPUTER a sophisticated terminal-end interface to tie three UNIVAC 1230 digital computers into the 360/ 91 and into each UNIVAC 492 other. HONEYWELL 516 USQ20 § We, however, rejected the use of a smaller computer strictly for SIMULATION CONSOLE use as a data and/ or control interface between terminal and cen­ tral computer for the following reasons: (a) restricted I/ O data transfer rate and (b) increased complexity of software and user programming, especially for real-time operation. Fig. l-APL computer complex.

November - December 1971 5 II IBM 360/91 I I I IBM 2909 IBM 2870 ~ MULTIPLEXERl ­ ASYNCHRONOUS :-­ CHANNEL CHANNEl DATA LONG LINES

rr=~~~~-~ ~==~::~::::~ 2703 L 2944 2930 TRANSMISSION 2930 DRIVERS/RECEIVERS DRIVERS/RECEIVERS CONTROL UNIT

IWNG LINE

APLCONTROL APLCOMPUTER EAI 693 AT&T 2944 AND FORMAT -- LINKAGE AND :-- DATA AND CONTROL DATAPHONE UNIT CONTROL UNIT INTERFACE IWNGLINE I 1827 AT&T UNIVAC EAI680 DIGITAL CONTROL DATAPHONE 1230 ANALOG COMPUTER UNIT

SIGNAL 1892 .I II 2741 HONEYWELL UNIVAC I PROCESSING PROCESS OPERATOR'S;I II TYPEWRITER 516 492 CONSOLE II II SIMULATOR (SPS) SIMCON LINK TYPEWRITER DIGITAL COMPUTER LINKS HYBRID COMPUTER LINKS LINK

Fig. 2-lnteractive, real·time computer links.

1230 digital computer to the 360/ 91. The 1230 the larger simulation that already involves the may also be used independently of the radar sys­ 680 analog computer. Therefore, the 680 and the tem, providing a general-purpose computer-to­ SPS use a single link to the 360/ 91. computer mode. Two other digital computers in The SPS is designed primarily for signal fre­ the same facility with the 1230 have also been quencies up to 1 MHz, and possibly as high as 5 tied into the link; these are a Univac 492 and a MHz in the future. This frequency range does not Honeywell DDP-516. allow use (with presently available technology) of such elements as potentiometers that could be Maior System Characteristics automatically controlled. Therefore, there are no The following provides a general description of means provided for other than manual setup for the performance features and other characteristics the SPS. Its only connection with the 680 is of the four simulation systems. Two of the sys­ through data lines to the 680 analog patch board. tems, the SPS and hybrid, use a single link. The The 680 computer, on the other hand, may be SIMeON and 1230 computer use one additional fully controlled for setup and run control from the link each. 360/ 91. The 680, via the 693 interface, allows Signal Processing and Hybrid Computers-The automatic mode control, potentiometer and func­ hybrid and signal processing subsystems are com­ tion relay setup, component readout, data trans­ plementary in that much of the signal processing fer, etc. This control may be exercised from the work requires simulation of a larger physical sys­ digital portion of a simulation program or from tem of which the signal processing is only one independent diagnostic programs. Both types of part. The signal processing simulations, so far, do program may also be run as batch programs or not require hybrid capability, except as a part of may allow control, on-line and interactively, by an

6 A PL T echnical D iRest operator through a typewriter terminal. Of course, its supporting software, to provide virtually any full manual control from the 680 console may also conceivable intervention (except a conversational ,be used and can override the digital control. language) and that a user might want for hands­ Since the analog portion of the hybrid system on, on-line control of his job. Special attention has is remote from the central computer and since the been given to providing all the capabilities tradi­ central digital computer is not dedicated, digital tionally available to the analog computer operator. I/ O equipment and a hybrid system operator's Output consists of printouts, line printer plots, console must both be located remotely in conjunc­ x-Y plots, digital and analog strip chart plots, and tion with the analog computer. An IBM 2780 line dual-trace memory-scope plots. printer and card-reader terminal are used for the The 1827 interface provides analog-to-digital I/ O. An IBM 2741 typewriter terminal is used for (A/ D) and digital-to-analog (D/ A) converters the operator's console, along with special support­ to drive the analog plotters. These converters also ing software. make it possible to tie hardware into digital pro­ Simulation Console (SIMCON)-The SIMeON grams. consists of an IBM 1892 process operator's con­ The 1230 Link-The 1230 link provides two sole and an IBM 1827 interface. The 1892 con­ modes of operation: (a) computer-to-computer trol panel is shown in Fig. 3. It allows interac­ and (b) radar system to 360/ 91. The computer­ tive control of all-digital programs with the fol­ to-computer mode allows the Univac 1230, Uni­ lowing features: vac 492, or Honeywell 516 computers to transfer 1. On-line mode control such as initialize, op­ data back and forth to the 360/ 91. Only one of erate, hold, stop, and terminate. the three terminal computers may use the link at 2. Data manipulation such as enter data (from a time. The terminal computer is selected from a keyboard), display data, save data, and step a the 360/ 91 by the user's job program. This pro­ parameter value. gram continues to operate as the master program, 3. Special pushbutton functions such as print coordinating the operation of the two computers parameters, change integration method and incre­ through the appropriate interrupts, control lines, ment size, and test all console functions. and systems software. The program on the termi­ 4. Operational and diagnostic alphanumeric nal computer is thus a slave operation that may messages to the operator. do computation and other operations solely asso­ The SIMeON has been designed, along with ciated with the terminal computer on an independ-

KEYBOARDIiFUNCTION KEYBOARD .. ~ · J \ DIGITAL-TO-ANALOGDISPLAY ~l~~:~i~~::'i'I""'~ ~l~~ CONTROL SWITCHES ~j~' d':; !~!'f tl' al .-i~~i L-'Y --.) -' -') I. ;a /a !a: R~t:i " ..~ _. < ) a a ~ ----- ~PARAMETER ~~~~~~~~~--~~-----~------ADDRESS BUTTONS

Fig. 3-Simulation console.

November - December 1971 7 ent or synchronized basis; however, it has no which the terminals are to be used, consists of direct control over the link, the central computer, special monitors, translators, emulators, test and or master program. Carrying out job program­ diagnostic routines, and the terminal support pro­ ming with this link requires a higher degree of grams. awareness of the associated access methods and related computer hardware characteristics than for the other two links, where all required digital programming is executed by the central computer. The key factors that must be handled by the pro­ grammer are communication between job program and system software, division of job program LONG LINE functions between computers, and timing or syn­ chronizing the execution of dependent parts of the two job programs. Fig. 4--General purpose computer link organization. The second mode of operation allows a partly simulated, partly hardware, radar system to oper­ ate in real time, with simulated targets being gen­ Basic Control Functions-There are four basic erated by the 360/ 91. The 1230 computer, in types of control function necessary to any linkage this mode, is a part of the radar system and con­ system: request, initiation, checking, and termina­ tains the master program. The 360/ 91 target gen­ tion. Depending on the practical applications for eration program is a slave to the 1230 program which the system is designed, there is a variety of in certain respects but also continues control over subfunctions associated with each of these four the link in synchronism with similar operations functions. managed from the 1230 end of the link. The request function can be carried out by addressing or selecting one device rather than As with the hybrid system and the SIMCON, another. This might be quite passive if the request the radar operation requires an operator's console is acknowledged only when the addressed device to allow the interventions of starting, stopping, is idle. A further step can be taken by allowing terminating, and changing parameters in the 360/ such requests to be queued and possibly some 91 program. Similar intervention on the 1230 may status information sent back to the device initiat­ be carried out at the 1230 console, although such ing the request. Beyond this, deliberate multiplex­ software must be coded by the job programmer ing, either through software or hardware, can be since there are no standard software support pack­ employed. ages or monitor. To implement typewriter opera­ A more imperative form of request referred to tion, an acoustically coupled TST 707 portable as an interrupt is also used. An interrupt system, typewriter terminal is located next to the 1230 while it might provide for queuing, requires that console. the device being addressed stop its present opera­ Link Organization tion and begin a new one. Completion of the first The generalized hardware form for any link is operation is generally suspended temporarily, shown in Fig. 4. While the primary enginee.ring rather than being obliterated, and is resumed effort is centered on the interfaces, consideration when the interrupting operation is completed. must also be given to transmission between inter­ Rather than accepting interrupts on a first­ faces if the distance is more than 100 feet. All come-first served basis, it is desirable to establish three links at APL involve distances of 800 to priorities for the interrupts. It can easily be seen 1000 feet. that if the last interrupt to arrive is always ac­ The software required for the links consists, at cepted, the storage and control for operations a minimum, of access methods to allow applica­ idled by successive interrupts could become un­ tions programs executed by the central computer manageable. One way out is to inhibit and queue to use the hardware interfaces, and through them, any interrupts following the first one until that to use the terminals. Other software, not neces­ operation is complete. This is not necessarily satis­ sarily concerned with driving the links themselves factory; certain following interrupts, such as for but oriented primarily to the applications for a real-time device, might well require more im-

8 APL Technical Digest mediate servicmg. In this case the inhibiting is (Extended Binary Coded Decimal Interchange carried out only for lower priority interrupts. Code). The function of initiating a given operation, Formatting refers to the order in which the once the request phase has been handled, has a basic signal patterns are arranged. Such conven­ number of parts: (a) clear interfaces and/ or re­ tions as the following come under this heading: quested device, (b) initialize, and (c) start opera­ 1. Word length and character length. tion. 2. The order in which particular words or char­ The clear phase is often necessary as a guaran­ acters are transmitted, particularly where control­ tee that any registers and other control circuitry type words are included. that could possibly provide erroneous control of 3. The use of blocks or other groupings of the new operation are not left active. words in batches. The initialization phase is carried out by the The key consideration involved in the APL requesting device, selecting the mode of operation links was how to reliably convert the codes and of the requested device and supplying the neces­ formats of dissimilar devices at minimum cost. sary parameters. This required some tradeoffs between software The start-operation phase completes the neces­ and hardware, as well as providing several modes sary "handshaking" through such events as READ of operation for a link to accommodate a variety READY, READ REQUEST, READ, or what­ of devices. ever the appropriate control sequence is between Signal Transmission-Signal transmission is devices, along with the various acknowledgments ' concerned with the transmission medium, modu­ so that data transfer begins. lation and demodulation, transmitters and re­ Certain checking is normally carried out during ceivers, noise, bandwidth, signal shaping and data transfer by both hardware and some mini­ regeneration, repeaters, level conversion, and mum of error-detecting software. As more check­ deskewing. ing is done by hardware, less time is required in The APL system uses RG-62 coaxial cable for overhead (nonjob program software execution the 1230 and hybrid links, partly suspended above time on the CPU). As more software is employed ground and partly buried in metal conduits in detecting faults, the overhead unavoidably in­ bundled in groups of 24 cables with an outside creases since software in general must operate sheathing of PVC. The data words are trans­ sequentially, while hardware is generally employed mitted in parallel, 30 bits parallel for the 1230 in parallel. For real-time operations this is a link, and 16 bits parallel for the hybrid link. major consideration. Cable lengths are 1000 feet. Finally, data transfer must be properly termi­ IBM 2930 line driver/ receiver and regenera­ nated. This can be accomplished through initiali­ tion equipment is used at the remote ends of the zation parameters, through step-by-step handshak­ 1230 and hybrid links. Voltage levels are 0 to + 3 ing control, or through control exercised asyn­ volts. Rise times after regeneration are maintained chronously by the requesting device. at IOns. Before regeneration, rise times are Data Transfer-The data transfer function, allowed to stretch up to 150 ns. The 1230 and which is the main object of developing computer­ hybrid links connect to the 360/ 91 memory bus to-computer links, consists simply of two func­ via the 2909, which has the same type of drivers, tions: read and write. The purpose of any link is receivers, and regeneration circuitry as the 2930's. to allow I/ O between two devices. The substan­ The SIMCON uses 22 A WG twisted pair cable, tive engineering content of handling data transfer for 16-bit parallel data transfer. The cable length is in terms of coding and formatting the data. is 1000 feet, and two IBM 2944 driver-receiver­ Coding refers to the meaning to be carried by regenerator units, one at either end, are used. the data signal pattern. The meaning is predeter­ Data rates are about 200K 30-bit words per mined, of course, usually according to some rule, second for the 1230 link, 200K 16-bit words per and the hardware and software are designed ac­ second maximum for the hybrid link, and 167K cordingly. In the APL links, the signal pattern is 16-bit words per second maximum for the SIM­ binary. Numbers are, in most cases, represented CON link. These are the maximum rates at which as binary without further coding. Alphabetic and the interfaces can operate. The maximum rate of special characters are represented by EBCDIC the 360 memory bus is 2M bytes per second be-

November - December 1971 9 tween CPU and 2909 and CPU and the 2870 multiplexer. Cable delays and software overhead IBM 2909 reduce these maximum rates to about 167K words ASYNCHRONOUS DATA CHANNEL for the 1230 link and lOOK words for the SIM­ CON. The hybrid link is limited by the A/ D con­ verter and software to about lOOK words per IBM 2860 I(IBM 360/91J: SELECTOR second. CHANNEL Noise is minimized by outside shields around cable bundles and the use of 4/ 0 fine-stranded IBM 2870 welding cable as a ground bus running between MULTIPLEXER buildings, terminating at each end in the equip­ CHANNEL ment complexes. The finely stranded 4/ 0 cable is SELECTOR SUBCHANNELS _~ felt to be a key to minimizing both AC and DC noise, particularly since substantial power ground levels as well as a myriad of AC sources from Fig. 5--APL's IBM 360/91 channel configuration. computers to radars can produce large voltage differences. Noise voltages in the transmission by switching from device to device, giving each system are maintained under 10mV from DC to a portion of the time available. 10 MHz. The selector channel in contrast is, in principle, Central Computer Interfaces-As described dedicated to a single peripheral device or semi­ earlier, computer links generally require interfaces dedicated by being able to address a few devices, at both ends. Those interfaces at the central com­ but only service one at a time until each succes­ puter end are channels into and out of the com­ sive one is finished. Also, rather than using the puter. There are three basic kinds of channel that time left after the needs of the CPU are met, it provide access to IBM 360 computers as shown takes priority in accessing the memory, maintain­ in Fig. 5. ing control until its data transfer is complete. In 1. Multiplexer Channels (Model 2870). many machines, this sort of operation, and varia­ 2. Selector Channels (Model 2860). tions on it, is called by its more or less generic 3. Asynchronous Data Channel (Model 2909). name of direct memory access. The first two types of channel are standard. The The asynchronous data channel is a combina­ last one (2909) is provided subject to customer tion of features from the multiplexer and direct specification. memory access types of channel. It has all the These channels are representative of two basic features of the selector channel, but can also methods of providing access to computers. The multiplex its own multiple subchannels according multiplexer channel allows connection of many to its own priorities. It is, therefore, able to pro­ peripheral devices to the main computer memory vide maximum speed transfers of data to and

TABLE 1 COMPARISON OF THREE TYPES OF IBM INPUT-OUTPUT CHANNEL 2870 2909 Selector 2860 Main Channel Subchannel Maximum Data Rate 2 Million Bytes/ s 110 K Bytes/s 180 K Bytes/s 1.3 Million Bytes/s Type of Data Transfer Multiplex Multiplex/Burst Burst Burst Maximum Distance to 110 Device 1000 200 200 200 (ft)

Data Path Size to 1,2,4, or 8 Bytes 1 Byte 1 Byte 1 Byte 110 Device Channel- Channel- Channel- Channel- Hardware Hierarchy Subchannel- Control Unit- Control Unit- Control Unit- Device Device Device Device

10 APL Technical Digest from the CPU to several external systems up to vices under the real-time monitor and other de­ an aggregate rate that equals the main memory vices under OS/ MVT simultaneously. The ADC input/ output speed and forces all other computer is composed of a main channel and up to 32 sub­ operations to be suspended as necessary. This channels that may contain some of all the func­ channel is, consequently, the most suitable for tions otherwise carried out by a terminal-end or handling real-time I/ O. controller interface. The subchannels can be de­ Table 1 gives a comparison of the character­ signed by IBM for special applications. Three istics of the three channels. The IBM 2870 multi­ types of subchannel are currently installed in the plexer channel has a maximum data rate of 11 OK 2909 at APL (see Fig. 6). 8-bit bytes per second (b/ s) and up to four selec­ tor subchannels may be added, with each sub­ channel having a data rate of 180K b / s. A maxi­ IBM 2909 mum of eight control units and a theoretical ASYNCHRONOUS DATA CHANNEL maximum of 196 I/ O devices may be connected 32-BIT HDGP SIC 1000 to the IBM 2870 main multiplexer channel. In I IBM 2930 WITH EXTENDED FEET addition to this, up to eight control units and 16 DRIVERS AND INTERFACE II CONVERTER I/ O devices may be connected to each selector II RECEIVERS 32-BIT HDGP SIC 1000 subchannel on the 2870. IBM 2930 WITH EXTENDED FEET INTERFACE As the name implies, the multiplexer channel DRIVERS AND CONVERTER uses time division multiplexing to interleave data RECEIVERS II to and from control units. However, a burst mode 16-BIT HDGP SIC WITH EXTENDED II IBM 2930 of data transfer between the channel and control DRIVERS AND 1000 jl unit is also possible using the selector subchannels RECEIVERS FEET INTERFACE which are similar to selector channels in that they PRIORITY INTERRUPT ~ CONVERTER SUBCHANNEL are dedicated and relatively high speed. In this mode a control unit locks out all other control units on the multiplexer until the burst of data is complete. The mM 2860 selector channel operates only in the burst mode and has a maximum data rate Fig. 6--APL's IBM 2909 subchannel configuration. of 1.3 million b/ s. Because of its speed, drums, disks, and sometimes magnetic tape units are gen­ Half Duplex General-Purpose Subchannel erally connected to a selector channel. A theore­ (HDGP). The HDGP subchannels, which allow tical maximum of eight control units and 256 I/ O data transfer in both directions, but not at the devices may be attached to a selector channel. same time, were specified by APL for data links It should be pointed out, though, that while 256 requiring high-speed data transfers, generally from is the maximum number of devices that can be real-time drivers, when each data word consists of uniquely addressed by the selector channel, the 16 or 32 bits. Two key features marked the physical number of I/ O devices is usually much HDGP subchannel design: (a) maximum data less than 256, after being dedicated to one or two rate up to the 360/ 91 memory bus rate of 2.0 similar units such as drums or an extended core million 8-bit bytes and (b) drivers and receivers memory. Also, the 2860 does not truly multiplex built-in which are capable of sending and receiv­ or interleave data transfers between the various ing such signals over 2000 feet of coaxial cable devices. It selects a device and then proceeds to with virtually zero probability that the usual am­ completely service that device to the exclusion of bient noise will cause errors. the others. In addition to these key features, a variety of The mM 2909 asynchronous data channel control lines, some of which are optional, was (ADC), while not associated with standard sys­ provided. This allows more sophisticated error tem 360 I/ O channels, is of particular interest detection and error correction with software or in the simulation environment at APL. It is able, special hardware. It also allows external devices among other things, to handle both real-time de- to be multiplexed, or for the devices to interrupt

November - December 1971 11 on a priority basis, and it allows for off-line Computer Linkage and Control Unit (CLCU). checkout of the 2909 with one subchannel driv­ The CLCU, as shown in Fig. 7, is the primary ing another for a so-called loop test. interface connecting the Univac 1230, Univac IBM 2909 Priority Interrupt Subchannel. The 492, and the Honeywell 516 to the 360/ 91. For IBM 2909 asynchronous data channel at APL the 516 it operates in conjunction with the CPU also contains a 16-level priority interrupt sub­ which makes the 516 conform to Univac char­ channel that allows 16, 32, 48, or 64 levels of acteristics on which the CLCU operation is based. priority interrupts. Each priority interrupt is The CFU also allows a mode of operation where uniquely identified by an address in the I/ O old the 516 is made to conform to IBM characteristics program status word during an I/ O interrupt. allowing it to replace the 360/ 91, where the 516 Priority interrupt levels may be enabled, disabled, and 1230 are to be interconnected with each or reset by issuing commands to the subchannel other. priority mask register from the 360/ 91. Priority The CLCU allows, then, both Univac com­ interrupt levels of lower priority are inhibited puters (one at a time) or the 516 to connect to during the processing of a higher level priority the 360/ 91. It also allows the 516 to connect to interrupt. Software support packages are required the 1230. And it further allows the radar syn­ before any use is made of the subchannel. chronizer associated with the 1230 to be con­ IBM 2909 High-Resolution Timer Subchannel. nected in either a 1230-to-91 mode or a 1230-to- The high-resolution timer subchannel is a timing 516 mode. device that may be used as an interval timer or an elapsed time counter. The subchannel contains a 32-bit register that may be loaded with a count. The count is then decremented at the rate of the RADAR frequency generator in the subchannel. The decre­ SYNCHRONIZER 492 1230 516 (CFU) menting rate may be from 1 to 127 /-ts / bit of the ,-- -J--- 32-bit register. 492 The high-resolution timer subchannel is con­ DRIVER/ trolled by the read direct and write direct instruc­ RECEIVER tion of System 360. An external interrupt is used to signal the 360/ 91 that a time interval has elapsed. Because of this, special software pack­ ages are needed on System 360 in order to utilize the high-resolution timer subchannel. Terminal-End Interfaces-The immediately pre­ ceding material has provided the key character­ SELF TEST istics of the central-computer-end interfaces. This SIMULATOR AND ~ section will describe the terminal-end interfaces. DISPLAYS t The computer linkage and control unit (CLCU) and the control and format unit (CFU) are en­ tirely APL-designed. The CLCU will be described in some detail. The CFU is similar to the CLCU and will, therefore, be described more generally. The EAI 693 data and control interface was *MUL TIPLEXER partly built to APL specification and will be de­ scribed with minimum discussion of the standard characteristics. The IBM 1827 is a completely standard unit, although APL did provide a black­ 360/91 360/91 box modification to allow attachment to the 360/ DRIVER/RECEIVER • L ______~ 91, for which computer the 1827 is not a stand­ ard connection. Nevertheless, the 1827 will be assumed standard for the purpose of this paper Fig. 7-Block diagram of computer linkage and con­ and described only generally. trol unit (CLCU).

12 APL T echnical D igest In its present configuration, the CLCU is dedi­ TABLE 2 cated to one mode at a time, switching to a new CHARACTERISTICS OF TERMINAL COMPUTERS mode only after the current user-computers indi­ USING CLCU cate that they have terminated all communication. Voltage (volts) I/ O Specs. Provision has been made, however, to convert the Word Speed CLCU to mode time-sharing under control of a Logical Logical Length (Max) suitable monitor on any of the terminal-end com­ Computer 0 1 (bits) (words/s) puters. The selection of modes at present may be made manually on the CLCU maintenance panel Univac 1230 -3 0 30 167 K or by instructions sent by anyone of the com­ Univac 492 -3 0 30 64K puters on a first-come-first-served basis. Honeywell DDP-516 0 +3 16 1000K The philosophy of the CLCU control is that (CFU) any computer may act as a "master" at any time and request that another computer be "slaved" to it. Once the communication link is established, levels used internally to the CLCU (see Table 2). the master-slave relationship disappears for the Two multiplexers determine which computers will CLCU, since it will accept further commands access the data buffers and control interface from either computer. Since the other computers blocks which are the heart of the CLCU. in the overall system may not gain control of the Control lines used by the three Univac or CLCU once this communication link has been "1230 side" computers are identical as follows: established, it is essential that a CLEAR CLCU 1. Input Data Request (IDR). Control line instruction be given by one of the active com­ voltage raised by the peripheral device (CLCU) puters when they are terminating their use of the to indicate that data are ready to be input by the CLCU. computer. The following specific functions are performed 2. Output Data Request (ODR). Control line by the CLCU in order to carry out the role de­ voltage raised by the peripheral device (CLCU) scribed above: to indicate that it is prepared to accept data from 1. Provides full-duplex data transfer capability the computer. between Univac-type computers and the half-du­ 3. Input Acknowledge (lA). Control line volt­ plex 32-bit subchannels of tpe IBM 2909. age raised by the computer in response to an IDR to indicate to the peripheral device that the input 2. Satisfies the demand-response interface of data lines have been sampled. all computers so that each computer feels it is in 4. Output Acknowledge (OA). Control line complete control of the data transfer. voltage raised by the computer in response to an 3. Converts logic levels of the transmitting ODR to indicate that data is on its output bus computer's I/ O interface to levels acceptable to ready to be sampled. the receiving computer's I/ O channel. 5. External Function (EF). Control line volt­ 4. Provides switching capability under program age raised by computer to indicate to peripheral control to allow a "master" computer to commun­ device that data is on the output bus. An EF icate with anyone of a number of "slave" com­ differs from an OA in that no response or re­ puters. quest by the device is necessary. 5. Transfers data at the maximum rate allowed 6. Interrupt (INT). Control line voltage gen­ between two computers (limited to the speed of · erated by the peripheral device to cause a jump the slower computer). from the instruction being executed in one pro­ 6. Provides a self-test mode to perform check­ gram to a given instruction in another program. out of the computer link with either the 360/ 91 The control interface handles the control lines or a Univac-type computer. from the IBM and Univac "sides" in order to sat­ This requires that the CLCU be divided into isfy both computers. Figure 8 shows a timing dia­ functional areas as shown in Fig. 7. Each com­ gram for the transfer of two data words from the puter has a level shifting driver/ receiver package Univac side to the mM side. The CLCU responds that transforms the logical voltage levels used by to the mM Read Ready by raising an Output the individual computer I/ O channel to the logic Data Request (ODR). When the Univac-side

November - December 1971 13 computer returns a data word and an Output Acknowledge (OA), the CLCU sends a Demand RADAR SYNCHRONIZER IBM TYPE COMPUTER IN OR OUT STATUS INFORMATION to the IBM side. A WC = 0 11 , EORIt sequence 0= GENERAL PURPOSE o = IBM 360191 ACTIVE satisfies the IBM computer after all data are 1 =RADAR SIMULATION 1 =DDP 516 ACTIVE transferred. This sequence is typical for transfers in the opposite direction using the IDR, lA, and write ready control lines.

COMMUNICATION TYPE UNIVAC TYPE COMPUTER 0= GENERAL PURPOSE STATUS INFORMATION READ DATA TRANSFER 00= IDLE SELECT 1 =SPECIAL RADAR 01 =UNIVAC 1230 SIMULATION DATA ACTIVE READ TRANSFER 10 =UNIVAC 492 READY ACTIVE 11 =DDP 516 ACTIVE ODR

Fig. 9--CLCU status word format. DATA K::::::=!r!J that select the two communicating computers are OA ====:y=..I t.====::::!.l '-======:::11 driven from the status register, changing the reg­ ister contents has the effect of changing the CLCU DEMAND II:::::::==:::::::::::J mode. A computer may request the CLCU to send its status word (by the proper EF or OA) in order to determine if the CLCU is available. The status word is always returned through the requesting computer's driver/ receiver package EOR ~======~ even if the CLCU is being used by two other machines. Fig. 8--Typical timing diagram of control lines for data transfer from the IBM side to the Univac side of the An important diagnostic feature of the CLCU CLCU. Actual times depend on which computers are is its ability to simulate computers. In a simulate being used (see Table 2). mode, the control lines on one side of the control interface may be manipulated from the CLCU The CLCU maintains a status register (see panel (Fig. 7). Single cycling (stepping under Fig. 9) in the control interface that contains among manual control) or automatic operation is pos­ other information: sible. For example, if a problem occurs in the 1. Multiplexer setting on 1230-side (which 360/ 91-to-492 link, the panel may be used to computer is presently supplying data and control simulate the 360/ 91 while the 492 is run nor­ lines) . mally and the operation (including data bits and 2. Multiplexer setting on 360/ 91-side. control lines) is checked. Then the 492 (actually 3. Whether or not the radar synchronizer is a general "Univac" computer) is simulated while controlling timing (SIMFAR or general purpose). the 360/ 91 runs. In this way the source of the The contents of this register are manipulated problem may be isolated. Simulation of two com­ by instructions from the computers. The Univac puters (on both sides) is also permitted as a computers issue specific external functions and the check on the circuitry within the CLCU. This 360/ 91 uses its device address bits to change the allows quite complete off-line checking of the register or read its contents. Since the multiplexers CLCU. Control and Format Unit (CFU). The control II Word Count Equals Zero (WC = D)-This line is raised by the and format unit (CFU) is similar to the CLCU. sub channel when the byte count for the current operation (read or write) has decremented to zero. (Note: Ready also comes high As with the CLCU, its basic function is to match with WC =0.) up the different I/ O control and data between It End of Record (EOR)-This line is raised by the I/O device to two or more computers. Specifically, the CFU signal the channel that no more data will be input or output (depending on the current I/ O operation). matches the I/ O characteristics of the Honeywell

14 APL Technical Digest DDP-516 to those of the 360/ 91 (2909) or the words that follow arc interpreted as commands 1230. This allows the 516 to communicate with rather than data. (If these control lines arc not the 360/ 91 through the CLCU by appearing as a set, data are transferred to the A/ D or from the 1230 or to communicate with any Univac ma­ D/ A converters according to the 693 mode al­ chine through the CLCU by appearing as a 360/ ready set up by previous control signals. These 91. This also allows the 516 to communicate control signals are generated by software on the (with slight modification) with any of the Univac 360/ 91.) The first 16-bit word represents an in­ peripherals directly. struction to the 693 and the second "data" word The specific functions performed by the CFU elaborates the meaning of the instruction. are: Finally, a status word and a fault word arc as­ 1. Transforms the 516 direct memory access sociated with the 693 coupler; these words may be (DMA) high-speed I/ O channel voltage levels, input to the 360/ 91 to enable the digital program pulse shapes, etc. to IBM and Univac require­ to interrogate the condition of the 693-680 units. ments. This is of key importance since the 693 and the 2. Performs packing and unpacking of 16-bit 680 analog computer are complex devices, but words and 32-bit words respectively. lack the inherent ability to diagnose their own 3. Satisfies the channel-response (control lines) faults while in on-line operation. Consequently, function between the DMA and the I/ O channels programs on the 360/ 91 are used to interpret and of the other computers. act on these hardware conditions. 4. Provides data transfer between the 516 and Simulation Console Interface (1827). The the other computers. last interface to be mentioned is the SIMCON 5. Provides for self-test. interface. The main area of concern is the use of 6. Provides switching under program control the IBM 1827-1892 components as a somewhat to allow communication to anyone of the con­ special-purpose console. The configuration pro­ nected computers via the CLCU. vides two interesting capabilities-interactive con­ The CFU is connected to four DMA channels trol of an all-digital simulation running on the of the DDP-516, any of which may be used for 360/ 91 and both analog and digital I/ O inter­ communications with other computers. If a full faces to provide interaction between the digital duplex mode of data transfer is needed, two DMA simulation and actual hardware. These hardware channels are used, with one dedicated for input capabilities are part of the standard IBM con­ and one for output. Since the DDP-516 is a 16- figuration. However, to take advantage of them bit word machine, the CPU must pack and un­ required a major development of software at APL. pack data words sent to the 360/ 91 (32-bit data word) and to the Univac computers (30-bit data word). In the case where the 516 is communicat­ Software ing with the 360/ 91, the reformatting is straight­ Key Software Design Considerations-Provid­ forward. Between the DDP-516 and the Univac ing the many capabilities represented by the soft­ computers, the most significant bit of the DDP- ware is a matter of developing the appropriate 516 word is not used, and two DDP-516 data program and the necessary communication by that words form one 30-bit Univac data word. program with hardware and other software. It is Hybrid Computer Interface (693). The 69 3 the intercommunications features and associated hybrid computer interface is the controlling device error handling and protection that distinguishes in this computer link just as the CLCU was the this work from applications programming. In this controlling device in the all-digital link. The 693 case, both the hardware system and the existing was designed for a minimum number of separate software system must be understood in detail, while control lines, however, requiring that many con­ for applications programming only a relatively few trol functions be sent as commands on the data hard and fast rules imposed by the compiler and/ lines. One of the more obvious problems, then, is or monitor need be followed. The designers of the for the 693 to distinguish between command and System 360 decided that it would be more feasible data words. For example, if one wishes to issue a to manage a multiprogramming computer system command to the 680 analog computer, certain without certain aspects of system operation being control lines must be set; then the first two 16-bit solely the province of the control program or

November - December 1971 15 supervisor itself. Most prominent among these are required by the I/ O supervisor to identify it and program switching and access to I/ O devices. to issue the four privileged I/ O commands to it. Problem programs must not be allowed direct These subprograms in turn build up appropriate access to data on peripheral storage units because tables known as control blocks which contain the of the danger of inadvertent modification of data actual data used by the lOS during an I/ O opera­ belonging to other programs or to the monitor. tion. The lOS, when appropriately called by a The problem runs, therefore, in "problem state" job program through the system supervisor, ex­ and any attempt to execute four privileged I/ O ecutes the new su bprograms to build the control instructions directly results in an abnormal termi­ block tables and then commences I/ O to the de­ nation of the problem by the supervisor (moni­ vice. tor). The question is how the problem program It was decided to incorporate our new devices should make the appropriate I/ O request to the into the present 360 supervisor. The problem pro­ system supervisor. In our application this is done grams would then request access to these devices by an instruction at assembly language level called through the supervisor just as for the more stand­ a supervisory call to the lOS, the argument of ard devices such as line printers, disks, etc. The which contains information on the I/ O operation 360 monitor or operating system is made up of a to be carried out and the identification of the I/ O number of separate though intercommunicating device. Additional information was often also con­ programs. One of these is the I/ O supervisor veyed through control blocks. These control (lOS) and is written in assembly language. The blocks or tables had to be generated or built by lOS is written in such a way as to allow, without the problem programmer at the assembly lang­ modification of existing code, additional subpro­ uage level or by a high-level language compiler grams to describe those features of a new device or by calls to special I/ O routines. Because the

TVPER* I 27411707 2909 ACM RTAM INTERACTIVE ACM T 2909 ACM (PART OF RTM) ~ (2909 ACM _ I 693ACM) SIMCON* (SUPERVISOR! ------OS/MVT ______REAL-TIME ACCESS METHOD) MONITOR

TESTS AND * SPECIALIZED TERMINAL TRANSLATORS DIAGNOSTICS SUPPORT PROGRAM

HYBRID I/O TEST

360-TO-CLCU MASTER I SLAVE

1230 RTM TEST RADAR TEST PROGRAMS

Fig. IO-Link system software additions to OS /MVT.

16 APL T echnical Digest system control program has to be so extremely sational programming system (CPS) that services defensive about problem programming errors, the typewriter terminals, or the graphics process­ providing interactive access methods for the peri­ ing system (GPS) that services the light pen ter­ pherals is but half the battle. The interactive user minals, the various translators, and service rou­ of the simulation-oriented peripherals will invari­ tines. ably arrive at a state of program readiness where Figure lOis a block diagram showing the soft­ he must run the program, linked-to equipment, ware that has been developed to service the com­ and try the various options on-line so as to test puter-to-computer links. In effect, most of this sections of programming that cannot be tested in software becomes an expansion of the operating any other way. Unless the software specialist has system. With this quantity of software and the provided for CPU error recovery, the first pro­ fact that most job programs using the link facili­ gram bug detected by the supervisor will likely ties are on the order of lOOK bytes, core space end the test via abnormal termination of the pro­ becomes an important consideration. Figure 11 gram. Thus, the process of final debugging be­ shows the memory core allocations for all major comes an extremely slow and frustrating one. systems. The software to support the links and associ- "ated terminals is of six types: ~~::=~~=_,:T~OTAL : 4096 K BYTES 1. Access methods. 800K 146K 620 K

2. Monitors. CONVERSATIONAL 3. Terminal support programs. PROGRAMMING BATCH INTERACTIVE SYSTEM (CPS) REMOTE JOB SYSTEM 4. Test and diagnostic programs. WORK PROGRAMS AND TIME ENTRY (RJE) SUPERVISOR SHARING 5. Translators. OPTION (TSO) 6. Emulators. The software was required to operate in a pre­ existing environment with the following character­ Fig. ll-Core allocations. istics: 1. Time shared with a variable number of jobs. Access Methods-The key type of program 2. Batch terminals. necessary for any computer-to-computer link (or 3. Conversational terminals. any peripheral hardware) is the access method 4. Graphics terminals. (ACM). This function may be carried out in any The new software was to add the following number of configurations; it may be a separate basic capabilities: package, part of the monitor, or part of an al­ 1. Real time. ready-existing access method, or it may be left to 2. Control and data transfer over the new com­ the job programmer to include in his program. puter-to-computer links. The access method is the software counterpart 3. External priority interrupts. of the hardware controller or interface. It causes 4. Comprehensive link error routines and di­ I/ O to take place as required by applications pro­ agnostics. grams, handles and generates hardware codes, and 5. Substantial operator interaction with the job contains error-handling features to protect the in­ program from the terminal end of the new links. tegrity of the overall system from programmer The mM 360/ 91 normally operates in a time­ errors, terminal operator errors, and terminal sub­ sharing mode under its operating system with system hardware errors. Access methods must, multiprogramming for a variable number of tasks therefore, handle the following functions: (OS/ MVT). This system allows a variety of jobs Ini tialization/ termination to be under execution at the same time from batch Input/ output card readers, both local and remote, from type­ Read writer terminals, and from graphics light pen ter­ Write minals. Control In adding the computer-to-computer links, no Error handling modification was allowed to be made to MVT or Checks other OS software such as the remote job entry Recovery (RJE) that services all terminals, or the conver- Messages

November - December 1971 17 User exits User initialization User I/ O format conversion User postprocessing The initialization/ termination functions are generally a single routine that interprets and drives the four basic control functions of the hardware interface, as already described. Fig. 12-Nesting of access methods and link hard· The data transfer operation of the interface is ware. driven and controlled in accord with the initializa­ signals through all interface units. In order to min­ tion routine parameters by an I/ O routine. This imize these access requirements, two things have routine handles most of the so-called handshaking been done: (a) the computer linkage and control and I/ O data format operations and also operates unit (CLCU) interface has been made sufficiently in conjunction with an on-line error checking intelligent that it can accept instructions for a routine. variety of options that can then be carried out by The error checking routine interprets any hard­ hardware and (b) the two digital computers are ware flags, while continually checking for logical not treated as hardware terminals with fixed cap­ software problems and data format errors. Some abilities, but are expected to have programs that capability is also programmed to provide limited intercept control and data signals, using these error recovery. The error routine can also provide basic functions in whatever way the programmer for a halt or wait of the job program, with appro­ wishes and the computer hardware allows. priate messages and user intervention routines to With this approach the requirement for a sep­ allow operator-initiated recovery procedures to be arate set of access methods for communicating used. with the CLCU and the digital computers is elim­ It is essential in real-time operations that the inated, the necessary programming being contained ACM be fast since it must operate in series with in the master and slave programs or in actual the real-time hardware and job program. There­ applications (user) programs. This requires the fore, the basic program must be simple, providing 360/ 91 programs to communicate through 91 only a minimum of error checking and recovery. system access methods to the CLCU, but no fur­ In general, there will be one ACM program for ther. Likewise, 1230 programs communicate only each interface and terminal making up a link. The as far as the CLCU. For all practical purposes functional organization of this software and hard­ the CLCU looks to the 91 like a 360 terminal ware can be conveniently viewed as a nested ar­ and to the 1230 like a 1230 peripheral. To make rangement. For instance, the simulation console this programming relatively independent of any system requires the console access method specific knowledge by the user of the interfaces, (POCACM), which communicates through the FORTRAN callable routines are available on the digital control unit access method (DCUACM), 91. On the Univac computers and the 516, how­ which communicates with the DCU itself, which ever, the user must still, at this point, deal in as­ then communicates with the console. Figure 12 sembly or absolute code with the interfaces, al­ shows this diagramatically. The other links follow though similar interface support packages could be the same pattern, and in fact this view can be written. extended to cover the various interfaces and re­ Monitors-Two monitors are required for the lated access methods making up the standard overall system, the operating system with multi­ 360/ 91 system. programming for a variable number of tasks (OS/ This same philosophy is also valid for the links MVT) and the real-time monitor (RTM). having digital computers at either end. However, The basic need for any monitor is two-fold: some modification of the concept, compared to ( a) to provide programs that make available to nonstored program interfaces and terminals, is the job programmer the hardware capabilities of a necessary. computer without requiring him to understand the Where two digital computers are concerned, architecture, hardware codes, and possibly even there is a need for a set of access methods on the circuit design of the hardware and (b) to both computers that can send data and control manage efficiently the communications or inter-

18 APL T echnical D ige st actions of the programs making up the monitor continue execution or services a real-time job and so that the job programmer does not need to associated interrupts. understand the operation of these programs. RTM gives first priority to real-time job execu­ In addition to these basic needs, a considerable tion and handling the associated interrupts. Real­ amount of effort is usually put into hardware and time jobs must identify themselves as such by software error-checking routines and diagnostics. issuing an SVC to R TM. The normal MVT job This serves several purposes: stream and interrupts continue to be processed in 1. To protect the monitor and hardware from background by MVT. Real-time jobs, while op­ improper use by job programmers. erating under RTM initially, may also give SVC 2. To protect job programs from each other, calls to MVT. In this way real-time jobs may which is particularly important in a time-sharing utilize all OS programs, the nonreal-time require­ system. ments of these jobs being processed under MVT 3. To assist job programmers in debugging. in background. 4. To provide the computer operator with R TM as a monitor is composed of such usual knowledge of malfunctions. supervisory programs as task scheduler, super­ OS/ MVT. The operating system with multi­ visory routines, service routines, and access ple, variable number of tasks provides the basic methods. R TM also requires an initialization/ time-sharing or multiprogramming capability as­ termination routine to handle its own initializa­ sociated with the 360/ 91. It is the 360's monitor. tion and termination of control over OS. It also manages all of the basic hardware facilities The supervisory routines consist of such pro- of the 360/ 91 such as storage, I/ O and the as­ grams as the real-time interrupt handler, error sociated access methods, accounting, messages to detection routines, assignment of priority interrupt and from the operator, and oversees the operation levels (PIL's), identification of real-time devices, of such associated software as translators or com­ real-time job initialization, termination procedures, pilers and service programs such as link edit, sort/ etc. The service routines allow the job program­ merge, data set management, etc. Such sophisti­ mer access to certain options and capabilities of cated monitors are major topics in themselves. RTM such as program controlled "waits," and Since APL has not modified the OS/ MVT as a they specify the program to gain control upon the part of the computer program being described, no occurrence of an interrupt, reading and writing to further discussion will be made here. a real-time device, use of a peripheral timer, RTM. The real-time monitor (RTM) adds masking and unmasking selected PIL's, etc. to OS the capability of guaranteeing that a real­ The access methods to real-time devices may be time job is executed and the associated I/ O carried included as a part of R TM. In our case, the 2909 out at given points in time. As necessary, RTM access method has been included. No real-time progressively dedicates the CPU and other facili­ devices, of course, can be accessed via the access ties to maintain real-time operation. In addition, methods that are part of OS/ MVT. high-speed servicing of external interrupts and Terminal Support Programs-Terminal support timer interrupts is accommodated. The RTM is programs are closely related to access methods a set of programs that can be integrated into OS and may have some of the supervisory aspects of as a system task (resident in core as part of OS) monitors. The computer links employ three of or it can be entered as a program in the batch job these terminal support programs: stream. Once in execution, R TM establishes con­ 1. SIMCON for the Simulation Console. trol over OS through the following steps: 2. TYPER for the typewriter terminals. 1. It initializes itself. 3. FORTRAN callable I/ O routines for 360 2. It issues a supervisory control (SVC) call to CLCU. to a program included in the MVT supervisor at These programs work between (or include) the system generation. terminal access method and the job program. 3. It then replaces the MVT program status They are link edited (called in from disk storage words (PSW's) with its own, temporarily sus­ and combined) with the job program and are con­ pending program execution and OS interrupts, if sequently not resident software. any. Basically, the terminal support programs are an 4. It either allows programs under MVT to extension of the terminal access methods. For

November - December 1971 19 special terminals having such features as function Test and Diagnostic Programs-The greatest buttons, displays, and other interactive capabili­ number of new programs that were developed in ties, more servicing is required than just read, support of the three links fall into the category of write, and the four basic control functions of re­ diagnostic software. These programs were keys to quest, initiation, checking, and termination pro­ the successful completion of the overall system vided by the usual access method. The interface and the individual links. For the operation system, with the job program may also be a relatively these programs are used for maintenance, and in high level instead of at the assembly level. the hybrid case some are also used for automatic In the case of SIMCON, the terminal program setup and checkout of the analog computer. is called on by the job program to activate the Most of the diagnostic programs carry out three simulation console and initialize its various special types of operation: ( a) to exercise or drive the functions. It puts out a message on the SIMCON system, (b) to detect errors, and (c) to deter­ nixie lights: "BEGIN." In the course of job pro­ mine the cause of the errors. gram execution and user commands, SIMCON In general, the driver will exercise the system in also displays other English-language-like messages. all normal modes of operation, with a wide va­ SIMCON handles the interpretation and imple­ riety of data and control signals; in some cases, mentation of all function buttons for mode con­ where other software or hardware are supposed trol, parameter changes, selection of integration to cope with failures, errors are deliberately intro­ method, etc. Finally, SIMCON handles the deacti­ duced for testing. This is particularly important vation of the console and termination of the job in checkirig out the monitors and access methods. program as directed by the user from the console. In most cases a diagnostic program will be de­ TYPER handles similar functions for type­ signed primarily to test either hardware or soft­ writer terminals and in conjunction with job pro­ ware since this generally avoids undue complexity grams that mayor may not be associated with a in the programming; it can also minimize the computer link. TYPER allows a user at a type­ number of subsystems that must be tied up at writer terminal to communicate with his program anyone time; and it minimizes CPU time re­ through a series of simple high-level language in­ quired for any particular kind of check. structions. The application program also may be The kinds of function that are often imple­ coded to allow it to output on the typewriter in mented by separate programs are as follows: response to certain calculated conditions or in­ 1. Operator-interactive single-step machine structions from the user. Originally the 2741's code tests. operated through a separate access method, re­ 2. Testing of all hardware features of the sys­ quiring the operator to use several assembly lan­ tem individually and printing out specific data guage-type instructions to call various support pro­ and errors. grams. This was awkward and required some 3. Testing programs for software, with emula­ detailed programming. When TYPER was devel­ tion of hardware functions. oped, it included the 2741 access method. 4. Operational or demonstration programs. The FORTRAN callable routines are a set of Translators-Translators allow a high-level lan­ routines that may be called by a user program guage to be translated to a lower level language. to largely eliminate the need for understanding In general, a translator may be viewed as being the operation of the 2909 and the CLCU inter­ (a) a batch operation, (b) an instruction-by-in­ faces. They also eliminate the need for low-level struction translation with or without immediate I/ O programming. Unlike SIMCON and TYPER, execution, or (c) a conversational or user inter­ these routines are not managed or supervised as active operation. a single package by a program in the package. Developing a translator was not a goal of the The user programmer must, therefore, provide computer-to-computer link system since we ex­ this function in his job program. On the other pected to use those provided by the computer hand, the communication is not interactive and manufacturers. However, the digital simulation the control is relatively straightforward. This language (DSL), which seemed well suited to greatly simplifies the situation compared to hand­ SIMCON applications, did require some adapta­ ling the SIMCON or even the typewriter. tion to the 360/ 91 system. In addition, it was

20 APL T echnical Digest desirable to expand the DSL translator to accom­ blocks. The applications programs communicate modate various functions of the SIMCON. This with the access methods via callable routines or expansion was accomplished by instructions that directly compiled 110 instructions. became "calls" to SIMCON which is then link edited as part of the job program. To meet these secondary objectives, the DSL was modified to provide easy, high-level programming for all jobs to be linked to the SIMCON. A hybrid static analog test (HYSAT) is a second SIMCON translator that was required to implement the var­ POCACM SOFlWARE ious hybrid instructions for such functions as: (a) DCUACM TYPER HYIOS select analog mode, (b) read digital voltmeter, and RTAM ( c) set potentiometer. OS/MVT A great many more such operations are also RTM incorporated into HYSAT. This translator is in­ terpretive, allowing instructions to be handled one at a time under operator control, or a source pro­ gram may be batch compiled. Emulators--Only one emulator was developed for the APL systems, but it was essential for de­ velopment of key software in parallel with the procurement and checkout of the IBM 2909 inter­ face. The emulator simulated on the 360/ 91 the control functions and data transfer of the 2909. This allowed the hybrid 110 system (HYIOS) to be written and checked out in parallel with the 2909. Fig. 13-How the pieces fit together. Wherever hardware has been defined, such em­ ulators can be used to allow software and hard­ Conclusion ware development to proceed in parallel. In addi­ All three of the basic links, SIMCON, hybrid tion to the time advantage of such an approach, computer, and Univac 1230 are in regular use. it also allows a more effective interchange to take The hybrid computer has demonstrated the most place between software and hardware personnel rapid growth in workload and may soon require before final design and implementation. This was more than one 8-hour shift. most important relative to the 2909 since APL, Work that remains to be done to improve the EAI, and IBM were all involved in software and systems falls in a variety of categories. For the hardware interfacing at this point in the system, hybrid computer, fully automated, easy to run, and the 2909 itself was developmental. very comprehensive maintenance diagnostics Basic Software/Hardware-The preceding dis­ would be of major value in minimizing the cussion of software has outlined in some detail maintenance effort. For the multiple computers, the relationship between the hardware and soft­ using the 1230 link, the ability to time-share the ware. Figure 13 shows more directly how the CLCU as well as time-sharing the several CPU's pieces fit together. The hardware terminal systems would greatly increase efficiency of the rather are directly controlled by the lOS portion of the powerful computer complex. All three systems system supervisor (OS/ MVT or RTM). At this would benefit from a better simulation language point the software generates hardware codes with a compiler easily extendable with new in­ (known as channel command words or CCW's in structions, and with an optional conversational IBM terminology). The necessary access methods capability. Also, the development of on-line error support special features and allow the applications recovery and debug facilities for applications pro­ programmer to communicate with the system grams would greatly enhance the usability of the supervisor and the lOS via SVC's, and control systems.

November - December 1971 21