• Multimedia • Application Control

Digital Technical Journal

Digital Equipment Corporation M

......

......

......

......

......

......

...... •......

Volume 5 Number 2

Spring L993 Editorial The Digital Technicaljournal is a refereed journal published quarterly by Digital Jane C. Blake, Managing Editor Equipment Corporation, 30 Porter Road LJ02/D 10, Littleton, Massachusetts 01460. Kathleen M. Stetson, Editor Subscriptions to thejoumal are $40.00 (non-U.S. $60) for four issues and $75.00 (non­ Helen L. Patterson, Editor U.S. $ll5) for eight issues and must be prepaid in U.S. funds. University and college pro­ fessors and Ph.D. students in the electrical engineering and computer science fields Circulation receive complimentary subscriptions upon request. Orders, inquiries, and address Catherine M. Phillips, Administrator changes should be sent to the Digital Technicaljournal at the published-by address. Dorot he a B. Cassady, Secretary Inquiries can also be sent electronically to OTJ®CRL.DEC.COM. Single copies and back Production issues are available for $16.00 each from Digital Press of Digital Equipment Corporation, Terri Auricri, Production Editor 129 Parker Street, Maynard, MA 01754. Recent back issues of the journal are also Anne S. Karzeff, Typographer available on the Internet at gatekeeper.dec.com in the directOry /pub/DEC/DECinfo/DTJ. Peter R. Woodbur)', Illustrator Digital employees may send subscription orders on the ENET to ROYAX :JOLRNAL. Advisory Board Orders should include badge number, site location code, and address. Samuel H. Fuller, Chairman Comments on the content of any paper are welcomed and may be sent to the managing Richard W Beane editor at the published-by or network address. Donald Z. Harbert Richard .f. Holling,worth Copyright © 1993 Digital Equipment Cor poration . Copying without fee is permitted Alan G. Nemeth provided that such copies are made for use in educational institutions by faculty mem­ Jeffrey H. Rudy bers and are not distributed for commercial advantage. Abstracting with credit of Stan Smits Digital Equipment Corporation's authorship is permitted. Allrights reserved. Michael C. Thurk The information in the journalis subject to change without notice and should nor be Gayn B. Winters construed as a commitment by Digital Equipment Corporation. Digital Equipment Corporation assumes no responsibility for any errors that may appear in the journal.

ISSN 0898 -901X

Documentation Number EY-P963E-DP

The following are trademarks of Digital Equipment Corporation: Alpha AXP, AXP, CDA, COD/Repository, COHESION, CX, DDJF, DEC, DEC 3000 AXP, DEC �ilaGlance, DEC OSF/1 CoverDesign AXP, DECaudio, DECchip 21064, DECimage, DECnet, DECNIS, DECpc, DECspin, DECstation, DECvideo, DECwindows, Digital, the Digital logo, GIGASWITCH, HSC50, Dithering and color space conversion are two Megadoc, Open VMS, Open VMS AXP, Q-bus, RA,RV 20, SQL Multimedia, TURBOchannel, of the concepts discussed in "Video Rendering," ULTRIX, UNIBUS, VAX, VAXstation, and VMS. which opens this issue's set of papers on multi­ Apple, Macintosh, and QuickDraw are registered trademarks and QuickTime is a trade­ media technologies. On the cover, the band mark of Apple Computer, Inc. of blue across the bottom of the cover graphic Display PostScript is a registered trademark of Adobe Systems Inc. shO'ws the rectanf!.ularpatterning created DYI and JNDEO are registered trademarks and Intel is a trademark of Intel Corporation. by an ordered dither process using a popular t·ecursive tessellation array The band of bur­ HP is a registered trademark of Hewlett-Packard Company. gundy across the top shows the superior pat­ IBM is a registered trademark of International Business Machines Corporation. terning of the same ot·dered dither process Kodak is a registered trademark of Eastman Kodak Company. with a newly designed void-and-cluster array, Lotus and 1-2-3 are registered trademarks of Lotus Development Corporation. which produces a higher quality image for dis­ Microsoft, M5-DOS, and Excel are registered trademarks and Video for Windows, play by eliminating the rectangular patterns Windows, and Windows NT are trademarks of Microsoft Corporation. and the textures of white noise. The line illus­ MIPS is a trademark of MIPS Computer Systems. tration overlayingthese two arrays presents Motif, OSF, and OSF/ 1 are registered trademarks and Open Software Foundation is a two color spaces, one within the other: RGB trademark of Open Software Foundation, Inc. and YUV(lumincmce-chrominance space used Perceptics is a registered trademark and LascrStar is a trademark of Perccptics by television systems; Y axis not shO"wn). In the Corporation. color conversion process, data transmitted SCO is a trademark of Santa Cruz. Operations, Inc. in YUVspace is converted to RGB space. The cover design shows three faces of the RGB Solaris, Sun, and Sun OS are registered trademarks and SPARCstation is a trademark of Sun Microsystems, Inc. space "lifted off" and infused with the colot·s noted at each comer of the parallelepiped. SPARC is a registered trademark of SPARC International, Inc. System Vis a trademark of American Te lephone and Telegraph Company. The cover concept and illustrations are UNIX is a registered trademark of UNIX System Laboratories, Inc. derived from the paper "Video Rendering" X video is a trademark of Parallax Graphics, Inc. by Bob Ulichney The design was imple­ mented by Linda Fa/vella ofQuantic X Window System is a trademark of the Massachusetts Institute of Technology.

Communications, Inc. Book production was clone by Quantic Communications, Inc. I Contents

7 Foreword JohnA. Morse

Multimedia

9 Video Rendering Robert Ulichney

19 SoftwareMotion Pictures Burkhard K. Neidecker-Lutz and Robert Ulichney

28 Digital Audio Compression Davis Ye n Pan

41 The Megadoc Image Document Management System Jan B. te Kiefte, Robert Hasenaar,]oop W Mevius, and Theo H. van Hunnik

50 The Design of Multimedia Object Support in DEC Rdb Mark F. Riley, James]. Feenan,)r.,John L.Janosik,Jr., and T. K. Rengarajan

65 DECspin: A Networked Desktop VideoconferencingApplication Lawrence B. Palmer and Rid.)' S. Palmer

77 LAN Addressing for Digital Video Data Peter C. Hayclen

Application Control

84 CASE Integration Using ACA Services Paul B. Patrick, Sr.

100 DEC @aGla.nce-Integrationof Desktop Tools and Manufacturing Process Information Systems David Ascher Editor's Introduction

I database can not only store this data but provide fast retrieval Mark Riley, Jay Feenan, John Janosik, and T. K. Rengarajan describe DEC Rdb enhance­ ments that support multimedia objects, i.e., text, still frame images, compound documents, and binary large objects. Managing image documents is the subject of a paper by Jan te Kiefte, Bob Hasenaar, Joop Mevius, and 'I'heo van Hunnik. Megadoc is a hardware and software framework fo r building customized image management applications qu ickly and at low cost.

They describe the UNIX file system interface to Jane C. Blake drives, a storage manager, ami an image Managing Editor \VORJ'vl application framework. Distributing multimedia over a network presents This issue of the Digital Te chnical jo urnal fe atures both engineering challenges and opportu nities fo r papers on multimedia technologies and applica­ applications. DECspin is a real-time, desktop video­ tions, and on uses of the Appl ication Control conferencing application that operates over LANs or Architecture (ACA), Digital's implementation of the \VANs, using TC P/IP or DECnet protocols. Larry and Object Management Group's CORBA specification. Ricky Palmer present an overview of the DECspin The high quality of today's television, film, and graphical interface. They then address network sound recordings have set expectations for com­ issues of real-time conferencing on non-real-time puter-based multimedia; we expect high-quality networks and a solution to network congestion. images, fast response times, good quality audio, The transmission of fu ll-motion video programs availability-including network transmission, and to multiple users requires adaptations in many aU at "reasonable" cost. Bob Ulichney has written parts of a client-server, LAN environment. Peter about video image- rendering methods that are in Hayden's paper focuses on the specific problem of fact fast, simple, and inexpensive to implement. He efficient allocation of network addresses for the reviews a color rendering system and compares transmission of digital video data on a LAN. He techniques that address the problem of insufficient revie,vs alternatives and describes a technique fo r colors fo r displaying video images. Dithering is one the dynamic al location of mu lticast addresses. of these techniques, and he describes a new algo­ The common theme of the two final papers is rithm which provides good quality color and high­ ACA Services, Digital's implementation of the OMG's speed image rendering. Common Object Request Broker Architecture. Paul The dithering algorithm is utilized in Software Patrick has written an instructive paper on CASE

Motion Pictures. SMI' is a method for generating environment development uti I izing ACA. Assuming digital video on desktop systems without the need a multivendor, distributed environment, he dis­ fo r expensive decompression hardware. Burkhard cusses modeling of applications, data, and opera­ Ne idecker-Lutz and Bob Ul ichney discuss issues tions; application interfacing; and environment encountered in designing portable video compres­ management. sion software to display digital video on a range of DEC @aGlance software is an implementation of display types. SMP has been ported to Alpha AXP, ACA that supports the integration of manufacturing Sun, IBM, Hewlett-Packard, and Microsoft platforms. process information systems. David Ascher differ­ Digitized data-video or audio-m ust be com­ entiates between generic integration software and pressed fo r efficient storage and transmission. @aGlance, and describes how ACA is used to inte­ Davis Pan surveys audio compression techniques, grate independently developed applications. beginning with analog-to-digital conversion and The editors thank John Morse, engineering man­ data compression. He then discusses the Motion ager, Corporate Research, and Mary Ann Slavin, Picture Experts audio algorithm and the interesting engineering manager, ACA, fo r their help in prepar­ problem of developing a real-time software imple­ ing this issue. mentation of this algorithm. Even compressed, digitized data takes up tre­ mendous amounts of storage space. A relational

2 Biographies I

David Ascher Dave Ascher joined Digital's Industrial Products Software Engineering Group in 19 77 to work on the DECDataway industrial multidrop network. Since then, he has worked on distributed manufacturing systems as a developer, group leader, and technical consultant, and as an architect of the DEC @aGlance product. As a principal software engineer, Dave leads an effort to develop DEC @aGlance service offerings. He holds a B.S. in psychology from City College of New Yo rk and a Ph.D. in psychology from McMaster University, Hamilton, Ontario.

James J. Feenan, Jr. Principal engineer Jay Feenan has been implementing application code on database systems since 19 78. Present ly a technical leader fo r the design and implementation of stored procedures in DEC Rdb version 6.0, he

has contributed to various Rdb and DBMS projects. Prior to joining Digital in 19 84, he implemented Manufacturing Resource Planning systems and received American Production and Inventory Control Society certification. Jay holds a B.S. from Wo rcester Polytechnic Institute and an M.B.A. from Anna Maria Col lege. He is a member of the US. National Rowing Te am.

Robert Hasenaar Bob Hasenaar is an engineering manager fo r the Megadoc optical file system team, part of Digital's Workgroup Systems Software Engineering Group in Apeldoorn, Holland. He has seven years' software engi­ neering experience in operating systems and image document management sys­ tems. Bob was responsible for the implementation of the first Megadoc optical disk file system in a UNIX context. He has an M.Sc. degree in theoretical physics from the University of Utrecht, Holland.

NT Peter C. Hayden Peter Hayden is an engineer in the Windows Systems Group. He joined Digital in 19 86 as a member of the FDDI team and led several efforts contributing to the development of the l'DDI technology and product set. He then led the Personal Computing Systems Group's multimedia projects

before joining the Windows NT Systems Group in 1992. Before coming to Digital, AT&T B.S. Peter worked on PBX development at Bell Laboratories. He holds a in electrical engineering and an M.S. in computer science from Union College in Schenectady, NY,and has several patent applications pending.

3 Biographies

John L. Janosik,Jr. A principal software engineer, John Janosik was the proj­ ect leader for DEC Rdb version 5.0, the Alpha AX I' port version. John has been a member of the Database Systems Group since joining Digital in I9R 8. Prior to this, he was a senior software engineer for Wang Laboratories Inc. and worked on PAC E, Wa ng's relational database engine and application development envi­ ronment. John received a B.S in computer science from Wo rcester Polytechnic Institute in 19 83 .

Joop W. Mevius A systems architect for the Mega doc system, Joop Mevius has over 25 years' experience in software engineering. He has made contributions in the areas of database management systems, operating systems, and image document management systems. Joop has held both engineering management positions and technical consultancy positions. He has an M.Sc. degree in mathe­ matics from the Te chnical University of Delft, Holland.

Burkhard K. Neidecker-Lutz Burkhard Neidecker-Lutz is a principal engineer in the Distributed Multimedia Group of Digital's Campus-based

Engineering Center in Karlsriihe. He currently works on distributed multimedia services fo r broadband networks. Burkhard contributed to the Xi\lledia layered product. Prior to that work he led the design of the NESTOR distributed learning system. He joined Digital in 19 88 after working fo r PCS computer systems. Burkhard earned an M.S. in computer science from the University of Karlsriihe in 19 87

Lawrence G. Palmer Larry Palmer is a principal engineer with the Networks Engineering Architecture Group. He currently leads the DECspin project for the PC and has been with Digital since 19 84. Larry is one of three software develop­

ers who initiated the PMAX software project for the DECstation 3100 product by porting the ULTR.lXoperatin g system to the MIPS architecture. He has a B.S. (high­ est honors, 19 82) in chemistry from the University of Oklahoma and is a member of Phi Beta Kappa. He is co-inventor fo r five patents pending on enabling soft­ ware technology fo r audio-video teleconferencing.

Ricky S. Palmer Ricky Palmer is a principal engineer with the Computer Systems Group. He joined Digital in 19 84 and currently leads the DECspin proj­ ect. Ricky is one of three software developers who initiated the !'MAX software project for the DECstation 3100 product by porting the ULTRIX operating system to the MIPS architecture. He has a B.S. (h igh honors, 19 80) in physics, a BS. (19 80) in mathematics, and an M.S. (1982) in physics, all from the University of Oklahoma. He is co-inventor for five patents pending on enabling software tech­ nology for audio-video teleconferencing.

4 I

Davis Yen Pan Dav is Pa n jo ined Digital in 1986 after receiv ing a Ph .D. in elec­ trical engineer ing fr om MIT. A pr incipal engineer in the Alpha Perso na l Systems Group, he is responsible fo r the development of audio signal processing algo ­ rithms fo r mu ltimedia products. He wa s project leader fo r th e Alpha/O SF base audio dr iver. He is a pa rticipant in the Intera ctive Multimedia Association Digital Audio Tech nical Wo rk ing Grou p, the AN SI X3L3 .1 Tech nical Wo rk ing Group on MPEG standards activities, and th e ISO/ MPEG standards co mmittee. Dav is is also cha ir of the ISO/ MPEG ad hoc co mmittee of MPEG/a udio software verification.

Paul B. Patrick, Sr. Pa ul Patrick is a pr incipal software engineer in the ACA Serv ices Group. He leads Digital's implementatio n of th e Ob ject Management Group's Co mmon Ob ject Request Broker Ar chitecture. Prev iously, Pau l helped design COHESION , an integrated CA SE env ironment based on the DECset arch itec­ tur e. He also co ntributed to the development of IPSE, an integrated project sup­ port env ironment based on the CO D/Repository software, and designed and implemented the MicroVAX 2000 synchronous co ntroller diagnostic. Prior to joining Digital , Paul held po sitions at GenRacl Inc. and Norand Cor p.

T. K. Rengarajan T. K. Rengarajan, a member of the Da tabase Systems Group since 1987, wo rks on the KO DA software kernel of th e DEC Rdb system. He has co ntrib uted in th e areas of buffer management, high ava ilab il ity, OLTP perfor­ ma nce on Alpha AXP systems, and multimedia da ta bases. Presently, he is wo rk ing on high-performa nce lo gging, recoverab le la tches, asynchronous ba tch writes, and asynchronous prefetch fo r DEC Rclb version 6.0. Ranga hol ds M.S. degrees in co mputer-a ided design and co mputer science fro m the University of Kentucky and the University of Wi sconsin, respectively.

Mark F. Riley Co nsult ing software engineer Ma rk Riley has been a member of th e Databa se Sy stems Grou p since 1989 and wo rks on mu ltimedia da ta type extensio ns in Rdb/V MS. Prior to this, he wor ked fo r five yea rs in the Image Systems Group and develo ped par ts of the DECimage Applica tion Serv ices too lkit. Mark received a B.S. E. E. fro m Wo rcester Polytechnic Institute in 1980 and an M.S. in engineering fro m Da rtmou th Co llege in 1982.

jan B. te Kiefte Ja n te Kiefte is technica l director fo r Digital's Wo rk group Systems Software Engineering Group in Apcldoo rn, Ho lland. He has ov er 20 yea rs' software engineering exper ience in co mpiler development and in th e development of office automation pro ducts. Ja n has held both engineering man­ agement po sitions and technical co nsu ltancy po sitions. He has an M.Sc. degree in ma thematics fro m the Tech nica l University of Eindhov en, Ho lland. Biographies

Robert Ulichney Robert Ul ichney rece ived his Ph.D. (1986) in electr ical engi­ neering and computer sc ience from the Massach usetts Inst itute of Technology and his B.S. (1976) in physics and comp uter sc ience from the Un ivers ity of Dayton, Oh io. He is a consult ing eng ineer in Alpha Personal Systems, where he manages the Codecs and Algor ithms Group. Bob has nine patents pending fo r his contribu tions to a variety of Dig ital products, is the author of Digital Halfto ning, publ ished by The MIT Pr ess , and serves as a referee fo r several tech­ nical soc ieties in cl uding IEEE.

Theo M_ van Hunnik Theo van Hunn ik is an eng ineering project manage r fo r Dig ital 's Wo rkgroup Systems Software En gineer in g Group in Apel doorn, Hol lan d. He has ov er 20 years' software engineer ing exper ienc e in comp iler development and the development of office automat ion prod ucts. Th eo has part icipated in several in ternat ional systems arch itec ture task forces. He manage dthe develop­ ment team fo r Retr ievAl !, the Megadoc image application framework.

6 Foreword

I CD-ROM, adapted fro m audio CD techno logy, wa s th e perfe ct storage med iu m fo r distribu tion of mu lti­ media co ntent; and so fo r th is ma rket segment, CD-ROM and multimedia beca me almost synonymou s. Th ere has emerged a wh ole industry based arou nd the production of multimedia titles on CD-ROM. At th e high end, fo r pu rposes su ch as fu ll-rea lism aircraft simu lation or virt ual reality applicatio ns, th e solu tion wa s to use th e high est performance hardware available, at wha tev er expense. Typically, high -end, th ree-dimensional gra ph ics systems were cou pled either to su perco mpu ters or to massively John Morse A. pa rallel processo r arra ys. The resu lt wa s, and still is, S1: Engineering Manager, impressive. Bu t the co st is still so high that su ch vir­ Corporate Research & tual reality systems are no t yet co mmercia lly viable Architecture except in specialized lo w-vo lu me ma rkets. The vast area in the midd le, into wh ich aJ! of Digita l's business fa lls, bas developed very slowly. In the la te '80s, "multimedia" wa s a ma gic wo rd . The prob lem is tha t our business is based on a It seduced us with glimpses of a brave ne w wo rld mo del of enterprise-wide computa tion. Th e co m­ wh ere au dio and video technology merged with puter systems we design ancl sell no t only include compu ter tech no logy. It promised us eve ryth ing processo rs and displays bu t incorpo rate networks fro m instant high -impact bu siness presentations and servers as well. To introduce mu ltimed ia into to virtua l rea lity. Wo rds like "paradigm sh ift" and such a mo del, one to uches every aspect of the sys­ '·mul tibill io n-dollar industry" were enough to snare tem, from the desktop, th rough th e network, and bo th the te ch no ph iles and th e ea ger entrepreneurs back to th e serve rs. At ev ery turn, we have fo und into believ ing that th e wo rld had su ddenly tha t th e technology that has evo lved over 30 to 40 changed, and we were all go ing to get rich in the years fo r hand I ing nu mbers, text, and (more process. recently) two-dimensional and th ree-d imensional So mewhe re on th e wa y to the ba nk, reality set in, graph ics is no t quite right fo r video and audio. and it wa sn't virtual. Th e reality is that mu ltimedia is Every component of the system, bo th hardware and a lo t harder tha n it looks. Su ccessful mu ltimedia software, needs to cha nge in so me wa y. We need tO requ ires a marriage between analog TV tech nology evolve to a mo de.! of networked client-server multi­ and digital computer tech no lo gy; it requires reco n­ media compu ting. Change of th is magnitude is a cil ia tion between a tech nical/professional marke t­ slow process. place and a co nsumer marketplace. As in any Two cha llenges are so pervasive tha t almost ma rriage, a lot of hard wo rk is requ ired to make it ev ery paper in th is issue addresses them, each fro m su cceed, and mu ch of that wo rk is yet to be do ne. a different perspective. First of all, mu ltimedia Fo r certain segments of the co mpu ter industry, involves th e ha nd l ing of large qua ntities of da ta . mu ltimedia was rela tively eaS}' to implement and so Seco nd, fo r ma ny applications, tha t data must be caught on quickly. Th e first su ccesses hav e been at handled under very tigh t time co nstraints. Th e th e extremes of th e co st spectrum-v ery low-e nd resulting stress and strain on all co mpo nents of the desktop mul timedia on the one hand, and very system translates into a set of technical cha llenges high -e nd virtual rea lity systems on th e other. Th is that has occu pied us fo r th e last fo ur years and has le ft Digita l, with its traditio na l fo cus on the prom ises to keep us busy th ro ugh at least the rest of middle, te mporarily ou t of th e ga me. th is decade. Fo r desktop mu ltimed ia, all that is re qu ired is the Depending on th e pictu re quality chosen, it may ab il ity to ca pture and display video and audio. Since requ ire fro m one million to one hundred million ma ch ines like the Co mmodore Am iga were al ready byres of stOrage to save each second of live video in based more on TV technology than on co mpu ter digital fo rm. Since ma ny applicatio ns of mu lti­ technology (for co st reasons), they cou ld be quickl y media, su ch as arch iv ing telev ision fo otage fo r and ch eaply adapted to handle au dio and motion research or histo ric preservation purpo ses, will video. Thus desktop mu ltimedia wa s bo rn. The need to sav e ma ny hours of video , it is easy to see

7 Foreword that multimedia quickly builds demand fo r many has proved surprisingly hard. People expect the gigabytes (1,000,000,000 bytes) of magnetic or opti­ images they see to be synchronized with the sounds cal disk storage. But storage is only part of the prob­ they hear, and they expect delays to be no worse lem. Once such enormous amounts of data arc than those experienced on a long-distance tele­ stored , the challenge becomes how to retrieve a phone call. Unfortunately, data networks have been particular item of interest. Standard database tech­ designed to maximize throughput and rel iability. niques are oriented toward retrieval of text and They do this at the expense of some delay in trans­ numbers. Retrieval of audio and video information mission-delay that is annoying at best, and unac­ will require new file and database techniques that ceptable at worst, fo r teleconferencing applications. are only beginning to be understood. Successful infusion of multimedia technology An obvious application of multimedia technol­ into enterprise-wide computation is proving to ogy, once the networks are in place, is telecon­ require change on a scale that almost no one antici­ fe rencing. We can envision a day when we can pated. We at Digital are in the midst of this process connect to anyone any place in the world via the of change, and this issue of the Digital Te chnical network and carry on a conversation with them, jo urnal is a snapshot, taken at one point in time, of while each of us sees the other in full-motion video, that process. To gether, the papers describe some of using the audio ami video capabilities of our desk­ the toughest technical challenges that we face and top workstations and PCs. But realizing this vision in many cases give glimpses into possible solutions.

8 Robert Ulichney I

Video Rendering

Video rendering, the process of generating device-dependent pixel data from device-independent sampled image data, is key to image quali�y System compo­ nents include scaling, color adjustment, quantization, and color space conversion. This paper emphasizes methods that yield high image quality, are fast, and yet are simple and inexpensive to implement. Particular attention is placed on the deriva­ tion and analysis of new multilevel dithering schemes. While permitting smaller frame buffers, dithering also provides faster transport of the processed image to the display-akey benefit for the massive pixel ratesassociated with full-motion video.

Perh aps the most in fluent ial ch aracteris tic govern­ on rendering me thods th at ar e fast, simple, an d in g th e perce ived value of a sys tem th at displ ays inexpensive to implemen t. Perf orm an ce at vide o im ag es is the way th e pictures look . Im ag e ap pe ar­ rates can be ac hieved wi th minim al hardw ar e or an ce is largely dependent up on the qu al ity of render­ even softw are- on ly soutil ons. ing, th at is, the pr ocess of tak in g dev ice-independent The Render ing Arch itecture sec tion reviews th e dataan d gener ating dev ice-dependen tda ta tail or ed comp onents of a render ing system an d ex amines for apa rticul ar target displ ay. des ign tr ade- of fs . The paper then presents det ails The topic of th is paper is the pr ocess ing of sam­ of new an d eff icient dither ing im plement ati on s. pled im ag e data an d not synthe tic gr aph ics. For Finally, vide oco lor mapping is discussed. gr aphics render ing, pr imitives such as spec ifica­ ti ons of tr iangles ar e conver ted todi spl ayable pic­ Rendering Architecture ture elements or pixels. The at om ic elements Figure 1 il lustr ates th e major ph ases of avide o ren­ handled by a vide o rendering system ar e dev ice­ dering system: (l) filter an d sc ale, (2) color ad just, in dependen tpixels. Where as apr erendered gr aph­ (3) qu antiz e, an d (4) color sp ace conver t. ic s im age can be comp actly represented as a In th e first st age, the or ig in al im age datamus t be collecti on of tr iangle vert ices, prerendered vide o res ampled toma tch the target wind ow si ze. A sep a­ ac hieves comp act ion by me ans of compressi on rate sc aling sys tem sh ould be used for the horizon­ techn iques. tal an d vert ical direc tions toha ndle the case where Sampl ing br oadc ast vide o requ ir es a datara te of th e pixel as pec t ratio must be ch anged. For ex am­ more th an 9 milli on color pixels per sec ond ; th e ple, such as ymmetr ic sc aling is needed when the need of some relief for stor age an d netw orks is target displ ay expects squ ar e pi...'Cels an d the or ig inal cle ar . Vide o compressi on reduces redund ancy in pixels ar e not squ are . the source im age an d thereby red uces th e am oun t The best filters touse in comb inat ion wi th sc al­ of datato be tr ansmi tted. Dr amati c red uctions in ing have been determ ined fr om a perceptu al poin t datara te can be ac hieved with little degr adati on in of view.:� \Vhen limiting the bandwidth toreduce im ag e qu ality . The Joint Ph otogr aph ic Exper ts th e datara te , aGa ussi an filter with ast an dard dev i­ Gr oup (]PEG) st and ar d for st ill fr ame an d the at ion a = 0.30 X ou tp utperi od is rec ommended. Motion Pic ture Ex perts Gr oup (MPEG) an d Px64 For in te rp olati on , th e fil ter preferred (bec ause th e st and ards for motion vide o ar e current commi ttee filtered results look ed mostli ke th e or igin al) was compressi on te chn iques l Sever al other non­ a casc ade of tw o: first, sh arpen with a Lapl aci an st an dard schemes ex ist, in clud ing a simple com­ filter, an d sec ond , foll ow by conv olu tion wi th a press ion meth od conduc ive to softw are- on ly Gaussi an filter with a = 0.375 X input per iod. implement ati on 2 A ty pical sh arpen ing scheme can be expressed Vide o rendering receives dec ompressed im ag e by the foll owing equ ation: dataas in put. Since every dec ompressed pixelmust be pr ocessed, speed is essent ial. This paper focuses [x,y] = l[x,y] - l3'l'[x,y] •l[x,y], /simp (1)

Digital Technical journal Vol. 5 No.2 Spring 1993 9 Multimedia

DECOMPRESSED the color components red, green, and blue (RGB) IMAGE DATA - -�-- with a simple uniform quantizer causes fa lse con­ I tours to appear in slowly varying regions. This issue FILTER I leads to the third stage in the rendering system, AND SCALE I quantization. + I The three basic classes of techniques fo r cir­ I cumventing the problem of insufficient colors or COLOR I color memory are (l) histogram-based methods, ADJUST I (2) chrominance-subsampled frame buffers, and + I (3) dithering. All histogram-based methods, some­ I times called palette selection, require two passes of QUANTIZE I the entire image data: the first to acquire the his­ I togram statistics to fabricate a three-dimensional I + quantizer to N colors and the second to perform I COLOR the pixel assignments. Perhaps the fastest method SPACE I is the popularity algorithm, where a simple sort CONVERT I finds the N colors with the highest frequency, and -- --__ j t all other colors are mapped to those.s RENDERED PIXELS A more compute-intensive method, but one that in general perfo rms much better, is the often-used, Figure 1 Image Rendering System median-cut algorithm.H In this method, the color space is repeatedly subdivided into smaller rectan­ gular solids at the median planes, with the goal that where J[x,y] is the input image, 'l'[x,y] is a digital each of the selected colors represent an equal num­ 4 Laplacian filter, and "•" is the convolution operator ber of colors in the image. The average of the colors The nonnegative parameter � controls the degree in each of the final regions is the color used in the

of sharpness, with � = 0 indicating no change in quantizer. A later, less compute-intensive variation sharpness. When enlarging, sharpening should is the mean-split algorithm. Nso, several clustering occur before scaling, and when reducing, sharpen­ techniques have been reported that result in less ing should take place after scaling. The fi ltering dis­ quantization error than the above-mentioned meth­ cussed here is assumed to be two-dimensional, ods. One method, for example, minimizes the sum which requires image line buffering. For economy, of the squares of the errors9 In all cases, however, horizontal-only filtering is sometimes used. color problems can occur in other application win­ The simplest means of scaling is known as dows because each frame requires a different color nearest-neighbor scaling, and its simplest imple­ map; the colors in the other windows become mentation is based on the Bresenham scan conver­ scrambled in a different way fo r each color map. sion algorithm fo r drawing straight lines.' This One advantage of representing image data in a algorithm can be applied to image scaling and per­ luminance-chrominance space is that chrominance fo rmed with only three registers and one adder6 requires less spatial resolution than luminance to Further optimizations make this algorithm espe­ achieve excellent image quality. Visual perception cially suited fo r real-time use.7 of differences in chrominance is much Jess than that The second stage of rendering is color adjust, fo r luminance. The television standards have been most easily achieved with a look-up table (LlJT). exploiting this fact fo r decades. The quantization Each color component uses a separate adjust l.UT. approach of using chrominance-subsampled frame In the case of a luminance-chrominance color, an buffers is built on this fact, deferring conversion to adjust LUTfo r the luminance component controls the RGB components until just after the data is read contrast and brightness, and LUTs fo r the chromi­ fo r display.10·11·12 nance components control saturation. Typical implementations of chrominance­ For so-called true-color frame buffe rs with 2 4-bit subsampled frame buffe rs average each of the depths, visual artifactsthat can result from insuffi­ two chrominance values in a given luminance­ cient amplitude resolution do not occur. With chrominance color representation over a region smaller frame buffers, restricting the amplitude of that is either 2 by 2 or 4 by 4 pixels. Assuming 8 bits

10 Vol. No.2 Spring I'J'J.i Digital Te e/mica/ jo urnal Vi deo Rendering

of amplitude resolution per color component, the name describes the dither array design process in 2-by-2-pixel case results in an average of ((2 X 2 X 8 which voids and clusters are located and mitigated. luminance bits) + (8 + 8 chrominance bits))/ For the high-speed case of motion video, an (2 X 2 pixels) or 12 bits per pixel; similarly, the ordered dithering scheme has important advan­ 4-by-4-pixel case results in 9 bits per pixel. This tages over chrominance-subsampled frame buffers approach requires expensive hardware to up-sam­ and histogram-based approaches. Quantization by ple the chrominance components and convert the dithering allows the use of conventional frame color space at video rates. These nonstandard buffers, does not require the time-consuming pro­ frame buffers can also cause severe incompatibility cess of making two passes over each frame (or problems with most applications that expect RGB every Nth frame), does not cause other application� frame buffers. While chrominance subsampled to change color maps with every Nth frame, and frame buffe rs can accommodate most sampled nat­ allows any number of colors to be selected at ren­ ural images, thin-line graphics can be annihilated. der time. Also, experiments have shown that the The third alternative for quantization is to use image quality achieved by dithering is very compet­ a dithering method. Several methods exist that are itive with the other methods, when compared over designed primarily fo r binary output, but all are a range of sample images. Even when 24-bit frame extendable to multilevel color.'' 1H1 1' A "level" is a buffe rs are available, the increased speed of loading shade of gray, from black to white, or a shade of three or fo ur 8-bit color pointers or index values in a color component, from black to the brightest the time required to load a single 24-bit pL'(el makes value. The basic principle of dithering is to use the dithering a viable alternative in the design of desk­ available subset of colors to produce, by judicious top video systems. arrangement, the illusion of any color in between. By way of comparison, Figure 2 illustrates some Although neighborhood operations, most notably of the methods described in this section. A 240-by- error diffusion, produce good-quality dithering, 360-pixel, 8-bit monochrome image was rendered they are computationally complex and require to only two levels and displayed at 100 dots per inch additional storage. For video processing, where (dpi). Figure 2a depicts an image that was dithered speed is essential, we turned our fo cus to those with white noise; in Figure 2b, the same image was dithering methods that are point operations, that is, dithered using an 8-by-8 recursive tessel lation methods that operate on the current pixel only dither array; and Figure 2c shows the image without considering its neighbors. Each color com­ dithered with the new 32-by-.')2 void-and-cluster ponent of every pixel in the image has an associated array To illustrate the effect of sharpening, Figure "noise" or dither amplitude that is added to it before 2d shows the image in Figure 2c presharpened that component is passed to a uniform quantizer. using a digital Laplacian filter as in equation (1),

Historically, the first dithering method used for with a sharpening factor of 13 = 2.0. The goal of this video processing was white noise dithering, where coarse example is tO amplify the different effects. a pseudorandom number was added to each lumi­ The same methods apply to multilevel and color nance value before quantization. This method was output, where the resulting quality is much higher. practiced soon after the dawn of television.1(' However, the low-frequency energy in white noise Fast Multilevel Dithering causes undesirable textures and graininess. This section presents the details of simple, yet pow­ A preferred method is the point process of erful new designs to perform multilevel ordered ordered dithering, where a deterministic noise dithering. The simplicity of these methods allows array tiles the plane in a periodic manner. Dither fo r implementation with minimum hardware or arrays can be designed to minimize low-frequency software only, yet guarantees output that preserves texture. The most popular are the so-called recur­ the mean of the input. The designs are flexible in sive tessellation arrays. 17.JH These arrays yield results that they allow dithering from any number of input superior to those of white noise dithering but suf­ levels tv;, to any number of output levels �· pro­ fe r from structured rectangular patterns. vided N; � IV,,. Note that JV, and N,, are not restricted A new ordered dither array design, called the to be powers of t\VO. "void-and-cluster" method, eliminates both the low­ Each color component of a color image is treated frequency textures of white noise and the rectangu­ as an independent image. The input image Li can lar patterns of recursive tessellation arrays.19 The have values

Digital Te chnical jo ur11al Vo l. 5 No. 2 SjJ ring 1993 ll Multimed ia

(a) Dither with a Wh ite Noise Threshold ( b) Dither with an 8-by-8 Recursioe Tessellation Threshold Array

(c) Dither with a 32-by-32 Vo id-and-cluster (d) Same as (c) with Laplacian Shm1Jening,

Th reshold Array i3 = 2. 0

figure 2 Examples of Rendering to Two Output Levels

L1 E {0,1,2, . . , C« - 1)}, wh ere ;� is the number of template leve ls, wh ich represe nt the levels against wh ich image input and th e output image L, ca n have values values are compared to de termine the ir ma pping L E {0,1,2,. , (1�, - 1)). to the ou tput values. The dither templa te is thus " central to determining the na ture of the re su lting A dete rm inistic dither array of size M X N is used dither pa tterns. that is pe riodic and tiles th e entire input image. To Figure 3 shows a dithering system tha t co mprises simpl ify addressing of th is array, and N should J\11 two me mor ies and an adde r. Th e system tak es an each be power s of two. A dither template defines inpu t level at image location [x,y] and produces the orde r in wh ich dither va lues are ar ranged. Th e L 1 ou tput level L0 at the correspo nding location in the elements of the dith er templa te T have values dithered output ima ge. The dither array is addresse d

TE {0,1,2, ..., (� - 1)), by x'an dy', wh ich represent the low-o rde r bits of

12 Vo l. 5 .Vu. 2 Sp ring I'J'Ji Digital Te chnical jo unlCII Video Rendering

One-mem.ory Dithering Sy stem x· - DITHER ARRAY Using the above expressions, it is possible to sim­ y · - plify th e system by exchanging one degree of fr ee­ d[x',y'J do m fo r another. A bit sh ifter ca n replace the quantizer LUT at the expense of fo rcing the number l s[x,y[ L;[x,yi -�.V1'7'\ • QUANTIZER LUT of input levels N, , to be set by the system. Fo r bard­ war e implementations, th is design affo rds a co nsid­ erable co st reduction. Th e system and method of Figure 3 assume that Figure 3 Dithering .S)stem with Tw o LUTs 1� is given as a fixed para meter, as is usually th e case with mo st imaging systems and file fo rmats. th e image address. The selected dither va lue However, fo r image so urces such as hardware that d[x ; y '] is added to the input level to produce th e generates synthetic graphics, arbitrar ily setting N, sum s. Th is sum is th en quantized by addressing a often has no effect on the amount of co mputatio n quantizer LUT to produce the output level L0. involved. If an adjust LUT is used to mo dify the Th e trick to achieving mea n-preserving dithering image data , including a ga in makes a "modified is to properly generate the LUT val ues. Th e dither adjust LUT." Figure 4 depicts such a system, wh er e array is a nor ma lized version of th e dither template L,. is th e raw input level. Th e unadjusted or ra w specified as fo llows: input image can have the va lues

L,. E {0,1,2, . . 1)}, (2) ,(N, - wh ere N,. is th e number of ra w input levels, typi­ ,·vh er e 8..", the step size between normalized dither cal ly 256. Therefore, th e mo dified adjust LUT must va lues, is defined as impart a ga in of

�� - J . (3) N, .- l To solve fo r N, , recall tha t in the method of Figure and b..Q is the quanti zer step size 3 the quantizer wa s defined to have equal steps of size b..Q as defined in equation (4) . Th e quantizer LUT ca n be replaced by a simple R-bit sh ifter, if th e (4) va riable can be fo rced to be an exact binary b..Q number, No te tha t also defines the range of dither va lues. 8..0 A I..J. = ?II (5) The quanti�er LUT is a uniform quantizer with /1.� Q -. equal steps of size 8..0. N, ca n then be set by the expression s The precise expre sions in eq uations (2), (3), and 11 N1 (N - 1)2 + 1. (6) (4) were arrived at through extensive analysis of the = u average output resulting fro m processing input The integer R is the numbe r of bits the R-bit images of a co nstant va lue, over a wide ra nge of N,, sh ifter sh ifts to the right to achieve quantiza tion. ami N,. fV.,, Specifying R in ter ms of/\�, , equation (6) becomes

x·- DITHER ARRAY y · -

d[x',y'[

MODIFIED 1 s [x.y) 17\ R-BIT ADJUST L; [x,y)-\V--'--'-'-- L0[x,y] f--.-- SHIFTER LUT

Figure 4 One-menwry Dithering Sy stem with an Adjust LUT and Bit Sh ifter

Digital Te chnical jo urnal l'r>f. 5 No. 2 Sp ring 1993 13 Multimedia

1) possible display more effective outputs than = , (N;- (7) to R 1og2 (N . " - 1) inputs. More precisely,

To completely specify this problem requires speci­ t:,.Q CA;, - 1 ) N1 + 1 > l fying the range for fY;. It is reasonable to do this by 1� specifying the number of bits b by wbich the image Effective Levels = (12) input values are to be represented. Specifying b limits 1\� to the range (S) Note that 6.0!1\j in equation (12) is equal to t:..". Parameter b will be a key value in specifying the \Vh en /:::,." < 1, the normalization of the dither array, resulting system. i.e., equation (2), results in integer truncated values Given the two expressions, (7) and (S), and the that are not all unique. At this point, the number of two unknowns, R and fY;, a unique solution exists effective levels saturates to 1\j . because the range of N; is less than a factor of two. and R and 1\� are integers. To solve fo r R, substitute Data Width Analysis equation (6) for 1� in equation (8). The resulting The design of an efficient dithering system, particu­ equation is larly in hardware, depends on knowing the number (9) of bits required for all data paths in the system. This 2'' - 1 < (N - 1)211 + I :S: 2'' 0 section presents an analysis of the one-memory or dithering system shown in Figure 4. The system input b, i.e., the bit depth of the (2 /J- l 1 log , < R ::;; log , --- . (10) image input values, limits the data path fo r lj (2N/J 1) l)x,y]. - N-u 1 - 0 - The analysis shows the derivation of the precise bit depths fo r the other data paths. In summary, the Since 2 ::;; f\, ::;; i\j , the range of the expression in derivation proves that the dither values in the equation (10) must be less than one. Hence. given dither matrix memory require R bits, where R that R is an integer, 111a,· = (b and + d (and thus the R- bit shifter) - I) s = L1 = { (2 '' - require only b bits. R int log2 ,�, _ . (II) 1)1 } Bits Ne eded fo r Ditber J'vla tri:x The amount of N, in equation (6) is now specified. memory needed to store the dither matrix is an As an example, consider the case where N,, important concern; d"'"·'· denotes the maximum 9 equals H7 (levels), b equals (bits), 1\j equals 1,024 value. To determine d11w.\' ' substitu te the maximum (levels, for a 32-by-32 template). and /�. equals 2')6 value of n:x : y '] , which is (1\j - 1 ), into equation (levels) 'fhus, R equals 2, and the R-bit shifter drops (2). The resulting equation is the least-significant 2 bits. N, equals 345 (levels); the dither array is normal ized by equation (2) with t:..d = l/256; and the gain factor to be included in the modified adjust LUT is :') 44/255. This data is loaded (13) into the system represented by Figure 4 ami uni­ formly maps input pixels across the 87 true output levels, giving the ill usion of 2i6 1evels. d'""" ' which depends on 1\j , thus has a value in the The output image that results from either of the range dithering systems illustrated in Figure 3 or Figure 4 (14) appears to contain more effec tive levels than are actually displayed. An effective level is either a per­ For the case of a dither matrix with one value, ceived average level that is (lithered between two namely !\ = 1, d11w.,· equals the lower end of this true output levels or shades or an actual true out­ range. d" w' equals the high end of the range for put level. A small number of template levels 1\j dic­ large dither matrices. where 2R- t ::;; 1\j . An impor­ tates the resulting number of effective levels. When tant observation is that h>r all va lues in the range of N, is large, the number of effe ctive levels is equal expression (14), the number of bits needed is to the number of input levels 1� , because it is not exactly R.

14 Vu l. ) Nn. 2 Sp ring !')'),? Digital Te chnical jo urnal Video Rendering

From equation (11), the value of R increases as N0 Therefore, int{E1 -E2) in equation (22) must be decreases. The smallest possible value of 1� is 2, equal to zero, and the value of R becomes which is fo r bitonal output. In this case, the maxi­ R = b - 1 - K. (23) mum value ofR is v We can express N,; in equation (18) in terms of the R""'x = int{logz(2 - 1)) = (b - 1). (15) same integer K of equation (21) by noting that So, the number of bits needed fo r the dither values (24) is R, which can be as large as (b - 1). where

Bit Width of Adder Recall that s[x, y] = L1[x, y] + 0 < E� :::; 1. (25) d[x,y]. The number of bits needed fo r this sum determines the size of the adder and the size of Observe that E3 is equal to J, where N0 is an exact the R-bit shifter. L1 can be at most (1� - 1) and, as power of 2. Substituting determined in the last section, dmax can be at most (2'/ - 1). So, and equation (23) into equation (18) gives Snwx = (JV; - 1) + (211 - 1). (16) (26) From equation (6), Because of the range of E in equation (25), the 11 3 (N1 - 1) = (.Y0 - 1)2 , (17) range of s,."''x must be b ! ' which gives 2 - - 1 < < 2 ' - 1 5max - , (27)

= 11 1 -\wx 2 (N,- 1) + (21 - 1) = 211N,, - 1. (18) which requires exactly b bits. As a check, the size of the shift register shou ld We can express R in terms of N0 by using equa­ equal the number of bits required for N0 plus R. The tion (11): number of bits needed for 1�, is '' R = int{log/2 - I) - Jog2 (1�, - 1)). (19) int{1 + logiN,, - 1)). (28)

Each of the two terms in the equation (19) can be Using the expression in equation (21), this value expressed in terms of an integer part and a frac­ becomes tional part: int{l + K + E2) = K + 1. (29) '' log/2 - 1) = (b - 1) + E1, (20) So, the size of the shift register must be where (K + 1) + (b - 1 -K) = b bits, O

(21) Color Sp ace Conversion Referring once again to Figure 1, consider the final where K is an integer, and subsystem of a video rendering system-color 0 :::; E2 < 1. space convert. Assuming a frame buffe r that is expecting RGB data, color space conversion is not Now equation (19) can be rewritten as necessary if the source data is already represented R = (b - 1) - K + int{E1 - E2). (22) in RG R, as in the case of graphics generation systems. However, motion video is essential ly E2 is largest when N,,(an integer) is a large power of always transmitted and stored in a luminance­ 2. Because N0 cannot be greater than N,., chrominance space. Such a representation allows ' 2 ' � /\�. subsampling of the chrominance, as mentioned ear­ lier, which reduces bandwidth requirements; all This fact, combined with equations (20) and (21), video standards exploit this method of bandwidth yields the further condition red uction. It is also more intuitive to color adjust in a luminance-chrominance space.

Digital Te chnical journal Vo l. 5 No. 2 Sp ring 1993 15 Multimedia

Prior to proceeding to the quantize subsystem in YUV space. Y represents the achromatic compo­ shown in Figure 1, all color components must be at nent that is loosely cal led the luminance com­ the same final spatial resolution for a dithering ponent. (The term luminance has a specific method to work correctly. Chrominance compo­ photometric definition that is not what is repre­ nents, then, need to be up-sampled to the same rate sented in a video Y component.) U and V are color as luminance components. diffe rence components, where U is proportional to

Although the chromaticities of the RGB primaries (Blue - Y) and Vis proportional to (Red - Y) of the major television standards vary slightly. all Figure 5 is an orthographic projection of YUV television systems transmit and store the color data space. Inside the YUV rectangular solid is the

v R

L \,' I

y w

(LJ

c

(M) M

(RJ_--

8

u (V-axis in) (U-axis out)

Fig ure 5 Feas ible RGB Va lues in the YVVCo lor Sp ace

16 Vo l. 5 Nn. 2 Sp ring 1993 Digital Te chnical jo urnal Vi deo Rendering

parallelepiped of "feasible" RGB space. Feasible RGB ues are simply truncated. This operation effectively points are those that are nonnegative and are not maps colors back to feasible RCfl space along lines greater than the maximum supported value. For ref­ perpendicular to a parallelepiped surface illus­ erence, the corners of the RGB parallelepiped are trated in Figure 5, which can change the color in an labeled black (K), white (W), red (R), green (G) , undesirable way. The use of a color mapping UJT blue (B), cyan (C ), magenta (M), and ye llow (L). RGB avoids these problems. and Y1.N values are related linearly and can be inter­ - converted by means of a 3-by 3 matrix multiply. Summary In the United States video broadcast system, the Video is becoming an increasingly important data chrominancc plane (i.e ., the U-V plane in Figure 5) type fo r desktop systems. This is especial ly true as is rotated 33 degrees by introducing a phase in the distinctions between computing, consumer elec­ quadrature modulation of the chrominance signal. tronics, and communications continue blur. The resulting rotated chrominance signals are to While many factors contribute to the impression renamed I and Q (fo r inphase and quadrature), but one has of the value of a product that displays infor­ the unmodu lated color space is still YLN. mation, the way the images look can make the Figure 6 shows the back end of a rendering sys­ biggest difference. This paper fo cuses on rendering tem that uses dithering as a quantization step prior system designs that are fa st, low cost, produce to color space conversion. A serendipitous conse­ good-quality video, and are conducive to hardware quence of dithering is that color space conversion or software implementation. can be achieved by means of table look-up. The collective address fo rmed by the dithered Y, U, and References V values is small enough to require a reasonably sized color mapping LUT. There are two advantages 1. Sp ecial Issue on Digital Multimedia .\ystems, to this approach. First, a costly dematrixing opera­ Communications of the AOvl, vol. 34, no. 1 RGB tion is not requ ired , and second, infeasible val­ (April 1991) ues can be intell igently mapped back to feasible 2. B. Neidecker-Lutz and R. Ulichney, "Software space off-line during the generation of the color Motion Pictures,'' Digital Te chnical jo urnal, mapping LliT. 1993, This second advantage is an important one, vol. 5, no. 2 (Spring this issue): 19-27. because 77 percent of the valid coordinates Y1.N 3. W Schreiber and D. Troxel, "Tra nsformation RGB are in invalid space, i.e., the space around the between Continuous and Discrete Represen­ RGB parallelepiped in Figure 5. Color adjustments tation of Images: A Perceptual Approach," such as increasing the brightness or saturation can IEEE Tra nsactions on Pa ttern Ana�ysis and push otherwise valid H

y 5. ]. Bresenham, "Algorithm for Computer Con­ DITHER � , SYSTEM trol of a Digital Plotter,'' mM Systems journal vol. 4, no. l (1965): 2'5-30

u COLOR RGB 6. F. Glazer, "Fast Bitonal to Grayscale Image DITHER MAPPING COLOR MA: SYSTEM � LUT � INDEX Scaling," DECfR-50'5 (Maynard, Digital Equipment Corporation, June 1987)

v 7. R. Ulichney, "Bresenham-style Scaling," Pm­ DITHER ceedings of the Annual Conference SYSTEM � IS[.., T (Cambridge, 1'vlA , 199:1) 101-103

8. P Heckbert, '· Color Image Quantization for Figure 6 Sy stem fo r Dithering Th ree-color Frame Bu ffe r Display," Computer Graphics Ak!C Components and Color Mapp ing Stc;·cRA PH '82 Conference Proceedings, the Collective Result vol. 16, no. 3 (1982) 297-307.

Digital Te chuical jounwl \>(>/. 5 Nu. 2 Spring 1993 17 Multimedia

9. S. Wa n, K. Wo ng, and P Prusinkiewicz, "An 15. ). Jarvis, C. Judice, and W Ninke, "A Survey of Algorithm fo r Multidimensional Data Cluster­ Techniques fo r the Display of Continuous­ ing," ACkl Transactions on Mathematical tone Pictures on Bilevel Displ ays,'' Computer Sojtware, vol. 14, no. 2 (1988): 153 -162. Graphics and Image Processing, vo l. 5 (1976) 13-40. 10. C. Sigel, R. Abruzzi, and ]. Munson, "Chro­ matic Subsampling for Display of Color 16. W Goodall, 'Television by Pulse Code Modu­ Images,'' Op tical Society of Arnerica Top ical lation,'' Bell Sy stems Te cbnical jo urnal, vol. Meeting on Applied Vision, 1989 Technical 30 ( 1951): 33-49. Digest Series, vol. 16 (1989): 158-161. 17. B. Bayer, "An Optimum Method fo r Two Level 11. A. Luther, Digital Vi deo in the PC Environ­ Rendition of Continuous-tone Pictures, Pro­ ceedings of the International Confe r­ ment (New Yo rk , iVY: Intertext Publications, IEEE McGraw-Hill, 1989): 193 -194. ence on Communications, Conference Record (1973): (26 -11)-(26 -1'5). 12. L. Glass, "Digital Video Interactive," By te (May 1989): 284. 18. R. Ul ichney, "Frequency Analysis of Ordered Dither," Proceedings of the Society of Ph oto­ 13. P. Roetling, "Binary Approximation of Contin­ op tical lnstrU1nentation Engineers (SPIE), uous-tone Images," Photograpbic Science vol. 1079 (1989): 361-373. and Engineering, vol . 21 (1977): 60-65 19. R. Ulichney, "The Vo id-and-cluster Method for 14. J. Stoffe l and J. Moreland, "A Survey of Dither Array Generation,'' The Society fo r Electronic Techniques fo r Pictorial Reproduc­ Imaging Science and Te chnolog)'/�)nnpo­

tion," IEEE Tra nsactions on Communica­ sium 011 Electronic Imaging Science and tions, vol. 29 (1981): 1898-1925. Te chnolog)' (IS&T/SPIE) (February 1993).

18 Vol. 5 No. 2 Sp ring 1993 Digital Te clmicaljournal Burkhard K. Neidecker-Lutz Robert Ulichney

Software Mo tion Pictures

Software motion pictures is a method of generating digital video on general­ purpose desk top computers without using sp ecial decompression hardware. The compression algorithm is designedfo r rapid decompression in software and gener­ ates deterministic data rates fo r use fron1 C�ROll'l and network connections. Th e decompression part offe rs device independence and integrates well with existing window sy stems and application programming interfaces. Software motion pic­ tures fe atures a portable, low-cost solution to digital video playback.

The necessary initial investment is one of the major SMP functionality inside their own applicatio ns. obstacles in making video a generic data type, like Figure 1 shows the user co ntrols as displayed on a graphics and text, in general-purpose co mputer wo rkstation screen. SMP players are also available systems. The ab ility to display video usually requires on Digital's freeware co mpact disc (CD) fo r use so me comb ination of specialized frame buffer, with Alpha AXP wo rkstatio ns running the DEC deco mpression hardware, and a high-speed network. OSF/1 AXP operating system. In addition, SMP play­ A software-only method of generating a video back is included with several Digital products such display provides an attractive way of so lving the as the video help utility on the SPIN (sound picture prob lems of co st and general access but poses chal­ information networks) application, as well as other lenging questions in terms of effic iency. Al though vendors' products, such as the Mediaimpact multi­ several digital video standards either exist or have media authoring system 2 been proposed, their co mputatio nal co mplexity In the XMedia Too lkit, access to the SMP hmctions exceeds the power of mo st current desktop sys­ is possible through X applicatio ns, co mmand line tems 1 In addition, a co mpressio n algorithm alone utilities, and C language libraries. The applications do es no t address the integration with ex isting win­ ancl utilities support simple editing operations, dow system hardware and so ftware. frame capture, co mpressio n, and other functio ns. So ftware mo tion pictures (SMP) is both a video Mo st of these fe atures are intenclecl fo r use by pro­ compression algorithm and a co mplete software ducers of simple file fo rmats called SMP clips. implementation of that algorithm. SMP was specifi­ The decompression functionality is offe red as an cally designed to address all the issues co ncerning X too lkit widget that readily integrates into the integration with desktop systems. A typical applica­ Open Software Fo undation's (OSF) Mo t if-based tion ofSMP on a low-end wo rkstation is to play back applications. Multiple SMP coclecs (compressors/ co lor digital video at a resolution of 320 by 240 decompresso rs) on a given screen all share the pL'\:els with a co ded data rate of 1.1 megab its per same co lo r resources with one another and with second. On a DECstation 5000 Mo del 240 HX wo rk­ the Display Po stScript X-server extension, which is station, this task uses less than 25 percent of the offered by all major wo rkstation vendo rs. It also overall machine reso urces. plays wel l with the standard colo r allocations used To gether with suitable audio support (audio sup­ in the Macintosh QuickDraw rendering system and po rt is beyo nd the scope of this paper), software Microsoft Windows standard co lor al locations. mo tion pictures provides po rtable, low-cost digital To facilitate flexible but simple access to entire video playback. fil ms of SMP frames, SMP defines SMP clips. Rather than publishing that file fo rmat direc tly, all applica­ The SMP Product tions and widgets are accessed through an encap­ Digital supplies SMP in several fo rms. The most sulating library. This method allows fu ture releases co mplete version of SMP co mes with the XMedia to have application-transparent changes to the To olkit. This too lkit is primarily designed fo r devel­ underlying file structure and co mpletely different opers of multimed ia applications who include the ways to store and obtain SMP frames.

Digital Te clmical journal Vo l. 5 No. 2 SjJ ring 1993 19 Multimedia

described in 1986 by Campbell et af..l CCC is a cod­ SMI' Player 1 I 1 I ing method that is optimized fo r rapid decom­ ile ink Q_ptions !: !:_ !:!_elp pression of color images in software. We built a demonstrator that rapidly displayed ccc-coded images in a loop to create a motion video effect. The demonstrator then served as our study vehicle to create a usable product for digital video playback. Performing digital video entirely in software would stress the systems at all levels (I/0, proces­ sor, and graphics), so we needed to establish upper bounds fo r what we could hope to achieve with our desktop systems and workstations. From the user's perspective, large sizes and high frame rates are desirable. These features need to be balanced with the limitations of real hardware. We modeled the data path through which digital video o� ---1 0 Start Range I S>'IH( !t;Ht

color bw concerning the smallest acceptable image size and ____,I · frame rate, we set our performance goal to play back movies of size 240 by 320 on the slowest DECstation processor with an 8-bit display at 15

Figure I Us er Controls as DisjJ lc�yed on tbe frames per second. Smaller viewing sizes are almost Wo rkstation Screen invisible on a typical high-resolution workstation screen. An example of the latter is the storage of SMP We settled fo r a frame rate of 15 frames per sec­ cJ ips directly in a relational database system in ond. This rate is reasonably smooth: to the human which no files exist, such as SQL Multimed ia. The eye, it appears as motion rather than separate video data is stored directly in database records, images. It can be generated easily from 30 -frame and the client receives the data through the stan­ sou rce material, such as standard video used in dard remote database access protocols. At the North America and Japan, by taking every other receiving client, the SMP clip library is used to gen­ frame. Consequently, on the DECstation 2100 we erate a virtual SMP clip for the application program would have at most by substituting a new read fu nction. The SMP product also contains image converters 12.5 X 10(' clock cycles/second = 10.85 clock that translate to and from the popular PBMPLUS fam­ (320 X 240 x 15) pixels/second cycles ily of image fo rmats, allowing import and export to per pi.:xeJ about 70 different image formats, including the Digital Document Interchange Format (DDIF). This Thus, we must average no more than (approxi­ allows the use of almost any image fo rmat as input mately) ten machine instructions to decode and fo r creation of SMP clips. render each pi.,.,\.el to the screen. In order to set our target for compression Historical Background and efficiency, we looked at the volume of data and pos­ Requirements sible distribution methods. CD-ROM looked promis­ In 1989 Digital's Distributed Multimedia Group ing, and this data rate was also chosen by the experimented briefly with an algorithm called Motion Picture Experts Group (MPEG)-1 standard.� color cell compression (CCC) that had been Hence our coded data rate goal was to maintain

20 Vo l. 5 No. 2 Sp ·ring 1993 Digital Te chllica{ journal Sojfll'(lre Jl!!o tion Pictures

a coded data rate fo r this size and frame rate Comparison with Other that would allow playback from a CD-ROM. To Vi deo Algorithms achieve this goal, we limited the coded data rate To day (early 1993), a number of digital video com­ fo r the video component to 135 to 142 kilobytes pression algorithms are in use. All of them are per second fo r video, leaving 8 to 15 kilobytes per guarded closely as proprietary and therefore second fo r audio. In addition, we had to I imit fluc­ closed, and only one algorithm predates the devel­ tuations of the coded data rate to allow sensible opment of SMP. Although we could not build on use of bandwidth reservation protocols fo r play­ experiences with these fo r our work, we believe back over a network without complex buffering the internal working on most of them is similar to schemes. SMP with some additions. More interesting were the issues tl1at became A popular method fo r video compression is apparent when we attempted to use the prototype frame differencing. Rather than each frame being fo r real applications. The digital video material had encoded separately, only those parts of the images to be usable on a wide range of display types, and that have changed relative to a preceding (or due to its large volume, keeping specialized ver­ future) frame are encoded (together with the info r­ sions for diffe re nt displays was prohibitive. \Ve mation that the other blocks did not change). This would have to adapt the rendition of the coded method works wel l for some input material, fo r material to the device-dependent color capabilities example, in video conferences where the camera of the target display at run time. does not move. The method fa ils, however, on Our design center used 8-bit color-mapped dis­ almost all other video material. plays. These were (ami still are) the most common To enable frame diffe rencing on a wider range of color displays, and the demonstrator was based input scenes, a method known as motion estima­ on them. Integration of the video into applications tion is used by some algorithms. The encoder fo r an in a multitasking environment necessitated that image sequence performs a search fo r blocks that computational as well as color resources were have moved between frames and encodes the available fo r use by other applications. The system motion. This search step is computationally very would have to perform cooperative sharing of expensive and usually defeats real-time encoding, the scarce color resources on displays with limited even fo r special-purpose hardware. color capabilities. One of the earliest algorithms was digital video From the perspective of portability, we needed interactive (DVI) from Intel/IBM. It comes in two to conform to existing Xll interfaces, without any variations, real-time video (RTV) and production hidden back doors into the window system. The level video (PLV) . RTV uses an unknown block X Window System affo rds no direct way of writing encoding scheme and frame diffe rencing. PLV into the frame buffe r. Rather, the MITSHM extension acids motion estimation to this. R1V is comparable is used to write an image into a shared memory seg­ to SMP in compression efficiency, computationally ment, and then the X server must copy it into the more expensive, and much worse in image quality. frame buffer. This method would impact our PLV cannot be done in software and requires already strained CP\J budget fo r the codec opera­ special-purpose supercomputers fo r compression. tion. We would need to decompress video in our Compression efficiency of PLY is about twice as code and have the X server perform a copy opera­ good as SMP, and image quality is somewhat better. tion of the decompressed video to the screen, again The more recent INDEO video boards from Intel using the main CPU. Quick measurements showed use RTV. that the copy alone would use approximately 50 In 1992 Apple introduced QuickTime, which percent of the CPU budget fo r an 8-bit frame buffer, contains several video compression codecs. The and another 5 to 10 percent would be used by react­ initial RoadPizza (RP) video codec uses simple ing the coded data from l/0 devices. frame diffe rencing and a block encoding similar to With approximately five clock cycles per pixel CCC, but without the color quantization step. (This yet to be rendered, it became clear why none of the is a guess based on the visual appearance and per­ standard video algorithms was of any use for such a fo rmance characteristics.) Compression efficiency task. We wenr back to the original CCC algorithm of RP is three times worse than SMP, ami image qual­ and started the development of software motion ity is comparable on 24 -bit displays and much pictures. worse than SMP on 8-bit displays. Performance is

Digital Te chnical jo urnal vbl. 5 No. 2 Sp ring I'J93 21 Multimedia

difficult to compare since SMP does not yet run on Macintosh computers. The newer Compact Video (CV) codec intro­ duced in QuickTime version 1.5 is similar to CCC with frame diffe rencing and has compression effi ciency much closer to SMP. Image quality on 8-bit displays is still lower than SMP, and compres­ sion times are almost unusable (i.e., long). The newest entry into the market for software video codecs is the video 1 codec in Microsoft's Video fo r Windows product. Ve ry little is known about it, but it seems to be close to CCC with frame differencing. Finally, Sun Microsystems has included CCC with frame diffe rencing in their upcoming ver­ sion of the XIL imaging I ibrary. (b) The average of 101 is used as a threshold Three wel l-known standards fo r image and video to segment the block. compression have been established by the Jo int Photographic Experts Group (JPEG) and the Motion Picture Experts Group (JVIPEG) committees of Figure 2 Block Tru ncation Coding of the International Organization for Standardization a 4 by 4 Block (ISO) and by the Comite Consu ltatif lnternationale (CCITT). de Telegraphique et Tetephonique These For each of the two groups, the average is then standards are computationally too expensive to be calculated again, giving a low average, a, ami a high performed in software in all but the most powerful average, b. Mathematically, the first and second sta­ workstations today tistical moments of the block are preserved. Therefore, for a block of m pixels, with q pixels The Algorithm greater than the sample mean X2, and sa mple vari­ ance 0-2,it can be shown that The SMP algorithm is a pixel-based, lossy compres­ sion algorithm, designed fo r minimum CPU loading. C/ = X - 0' q/(111- q) It features acceptable image quality, medium com­ b =.X + a (m -q)lq pression ratios, and a totally predictable coded data rate. No entropy-based or computationally expen­ More intuitively, the bit mask represents the sive transform-based coding techniques are used . shape of things in the block, and the average lumi­ The downside of this approach is a limited image nance and contrast of the block contents are pre­ quality and compression ratio; however, fo r a wide served. With this coding method, for blocks of 4 by range of applications, S1vi P quality is sufficient. 4 p.L'

BTC is a gray-scale image compression technique. However, the direction taken by Campbell et al. The image is first segmented into 4 by 4 blocks. Fo r was computationally faster for decode -' In this each block, the 16 -pixel average is fo und and used approach, a luminance value is computed fo r each as a threshold. Each pixel is then assigned to a high pixel. As in the BTC algorithm, the sample mean of or a low group in relation to this threshold. An the lumi nance in each 4 by 4 block is used to seg­ example of the first stage in the coding process is ment pi.'Xels into low and high groups based on shown in Figure 2a, in which the sample mean luminance values only. The 24-bit color values is 101. Each pixel in the block is thus truncated to assigned to the low and high groups are fo und by 1 bit, based on this threshold (see Figure 2b). independently solving fo r the 8-bit red, green, and

22 Vol. 5 No. J ,)jJrillg /')'.!,) Digital Te e/mica/jo urnal Software Motion Pictures

blue values. This al lows each block to be repre­ digitized material. At the same time, this opened up sented by a 16-bit mask and two 24 -bit color values, use of SMP fo r still image applications. for a coding rate of 4 bits per pi.'

Digital Te chnical jo urnal lkJI. 5 Nn. 2 Sp ring 1993 23 Multimedia

fourfolu, which makes the image look very blocky. tables. Typically, applications and window man­ If too many structured blocks are allocated, regions agers use a fe w of the entries for system colors and of the image that have little detail are encoded with cursors. Quantizing down to a smaller number of unnecessary overhead. Over the wide range of colors (such as 240) could overcome this drawback images we tested, allocating between 30 percent to a certain degree; however, it would make the and ')() percent of structured blocks worked best, SMP-coded material dependent on the device char­ yie.lding a bit rate of 0.9 to 1.0 bits per pixel. For acteristics of a particular window system. color images, the overhead of the color table (768 The second and much more problematic aspect bytes) must be added. is that the SMP frames in a sequence usually have ditfe rent color tables. Consequently, each frame Decompression requires a change of color table that causes a kalei­ The most challenging part of the design of the doscopic effect fo r the windows of other applica­ S,\1 P system, given the performance requ irements, tions on the screen. In fact, flashing cannot be is the decompression step. Efficient rendering eliminated within the SMP window itself. techniques of block-truncation coding are well Neither Xll nor other popular window systems known fo r certain classes of output (levices.' such as Microsoft Windows allow re load of the SMP improves on the implementations described color table and the content of an image at the same in the literature by complementing the raw algo­ time. T'herefore, regard less of whether the color rithm with efficient, device-independent rendering table or image contents is modified first, a flashing 'iH.Io 11 engines -' To maximize code efficiency, a sepa­ color effe ct takes place in the SMP window. lt may rate decompression routine is used fo r each display seem that the update would have to be done in a situation, rather than using conditionals in a more single screen refresh time as opposed to simultane­ generic routine. The cmrent implementation can ously. This is true but irrelevant. Most window render to 1-, 8-, and 24-bit displays. systems do not allow fo r such fine-grain synchro­ Decompression of BTC involves filling 4 by 4 nization; and fo r performance reasons, it was unre­ blocks of pb.:els with two colors under a mask. al istic to expect to be able to update the image in a Because the size ancl alignment of the blocks is single, vertical blanking periocl. known, a very fast, fully unrol led code sequence Alternative suggestions to avoid this problem can be used. Changes of brightness and contrast of have been proposed in the literature. One sugges­ the image can be rapid ly adapted to diffe rent view­ tion is to use a single color table fo r the entire ing conditions by manipulating the entries of the sequence of frames. 111· 1 1 This method is computa­ colormap of the SMP encoding. Most of the work tionally expensive and fa ils fo r long sequences and I ies in adaptation of the color content of the decom­ editing operations. Another proposes quantization pressed data to the device characteristics of the to less than half of the available colors or partial frame buffe r. updates of the color map and use of plane masks. 11 For displays with fu ll-color capabi lities (24-bit This alternative is not particularly portable true color), the process is straightforward. The between diffe rent window systems, and the use of main problem is performing the copy of the decom­ plane masks can have a disastrous impact on perfor­ pressed video to the screen. Since 24-bit data is usu­ mance fo r some frame-buffer implementations ally a I. located in 32-bit wor(IS, the amount of data to such as the ex adapter in the DECstarion procluct copy is fo ur times the 8-bit case. Typically, SMP line. spemls 90 percent of the CPU time in the screen Neither of these methods addresses the issue of copy on 24-bit systems. monochrome displays or the use of multiple simul­ The more common and interesting case is to taneous SMP movies on a single display. (This effect decompress to an 8-bit color representation. Given can be witnessed in Sun Microsystems' recent audi­ that SMP is an 8-bit, color-indexed fi.>rmat, it would tion of CCC couing to their XIL library.) To keep seem straightforward to download the SMP frame device influence out of the compressed material color table to the window system color table ancl and to enable the use of SMP on a wide range of fi ll the image with the pixel indices directly. This devices and window systems, a generic decoupling metbc)(l is impractical fo r two reasons. First, most step was added between the colors in the SMP window systems (including X I 1) do not allow frame and the device colors used for rendition on reservation of all 256 colors in the hardware color the screen.

24 Vol. 5 No. l .\jJring 1993 Digital Te chnical jo urnal Software Motion Pictures

A wel l-known technique for matching color Although the initial scaling is not strictly part of images to devices with a limited color resolution is the SMP algorithm, it is necessary fo r different input dithering. Dithering trades spatial resolution fo r an sources. Fast sealing is offered as part of both the apparent increase in color and luminance resolu­ library and the command-line SMP compressors. tion of the display device. The decrease in spatial Instead of simple subsampling, true averaging is resolution is less of an issue fo r SMP images because used to ensure maximum input image quality. of their inherently limited spatial resolution capa­ The block truncation phase makes two passes bility. Thus the only challenge was the computa­ through each 4 by 4 block of the input. The first tional cost of performing dithering in real time. pass calculates the luminance of each individual Fortunately, we fo und a dithering algorithm that pixel and sums them to find the average luminance allowed both good quality and high speed . 12 It of the entire block. The second pass partitions the reduces quantization and mapping to a few table pLxel pairs into the fo reground and background look-up operations, which have a trivial hardware sets and calculates their respective luminance and implementation (random access memory) and a chrominance averages. reasonable software implementation with a few The flat-block-selection phase uses the desired adds, shifts, and loads. compression ratio to decide how many blocks can The general software implementation of the be kept as structured blocks and how many need to dithering algorithm takes 12 instructions in the be converted to flat blocks. The luminance differ­ lVI IPS instruction set to map a single pixel to its out­ ence of the blocks is calculated, and blocks in the put representation. For SMP decoding, two diffe r­ low-contrast range are marked fo r transition to flat ent colors at most are in each 4 by 4 block. With this blocks. Because the total average was calculated fo r distribution, the cost of dithering is spread over the each block in the preceding phase, no additional 16 pL'I:clsin each block. calculations are needed fo r the conversion of Another optimization used heavily in the 8-bit blocks, and the mask is thrown away. Colors are decoder is to manipulate 4 pixels simultaneous.ly entered into a search structure during this phase. with a single machine instruction. This technique The color quantization phase uses a median cut increases performance fo r decompressing and algorithm, biased to ensure good coverage of the dithering to 3.2 instructions per pixel in the MIPS color contents of the image rather than minimize instruction set, including al l loop overhead, decod­ the overall quantization error. The bias method ing of the encoded data stream, and adjusting con­ ensures that small, colored objects are not lost due trast and brightness of the image (2.7 instructions to large, smoothly shaded areas getting the lion's per pixel fo r gray-scale). This efficiency is achieved share of the color allocations. These small objects by carefu l merging of the decoding, decompres­ often are the important fe atures in motion sion, and dithering phases into a single block of sequences and have a high visibility despite their code and avoiding intermediate results written to small size. memory. The cost of the 1-bit and 24-bit decoders is The final encoding phase builds the color table the same or lower (3.2 and 2.9 instructions per and matches the fo reground/background colors of pixel, respectively). the blocks to the best matches in the chosen color table. Compression The gray-scale compression can be much faster because neither the quantization nor the matching The SMP compressor takes an input image, a desired step need be performed. Also, only one-thiJ·d of the coded image size, and an output buffe r as argu­ uncompressed video data is usually read in, making ments. It operates in five phases: gray-scale compression fast enough to enable real­ time compression on faster workstations and video­ • Input scaling (optional) conferenciog type applications. • Block truncation (luminance) This speed is partly due to the R-bit restriction in the mask of each structured block. This restriction • Flat block selection permits the algorithm to store all intermediate results of the block truncation step in registers on • Color quantization (color SMP only) typical reduced instruction set computer (!USC) • Encoding and output writing machines with 32 registers. The entire gray-scale

Digital Te chnical]ounwl Vo l. 5 Nu. 2 Sp ring 1993 25 Multimedia

compression algorithm can be done on a MIPS Generally, porting the SMP system to another plat­ R3000 with 8 machine instructions per input pixel fo rm supporting the X Window System requires the on average, all overhead (except input scaling) selection of two parameters (host byte order and included. presence of the MITSHM extension) and then a com­ Unfortunately, for color processing, SMP com­ pilation. The same codec source is used on all the pression remains an off-line, non-real-time pro­ above machines; no assembly language or machine­ cess, albeit a reasonably fast one at 220 instructions specific optimizations are used or needed. per pixel. A 25-MHz R3000 processor can process The port to Microsoft Windows shows that more than 40,000 frames in 24 hours (DECstation the same base technology can be used with other 5000 Model 200, 320 by 240 at 15 frames per sec­ window systems, although parts specific to the win­ ond, TX/PlP as frame grabber), equivalent to 45 min­ dow system had to be rewritten. The cocleccode is utes of compressed video material per day. The essentially identical, but the extreme shortage of more recent DEC 3000 AXP Model 500 workstation registers in the 80x86 architecture and the lack of improves this number threefold, so special-purpose reasonable handling of 32-bit pointers in C lan­ hardware for compression is unnecessary even fo r guage under Windows warrant a rewrite in assem­ color SMP. bly language on this platform. We do not expect this to be an issue on Windows version 3.2, clue to Po rtability be released later in 1993. A crucial part of the SMP design fo r portability is the Conclusion placement of the original SMP codec on the client side of the X Window System. This allows porting Software motion pictures offers a cost-effective, and use of SMP on other systems, without being totally portable way of bringing digital video to the at the mercy of a particular system vendor fo r inte­ desktop without requiring special investments fo r gration of the codec into their X server or window add-on hardware. Combined with audio facilities, system. SMP can be used to bring a complete video playback This placement is enabled by the efficiency of the to most desktop systems. The algorithm and imple­ SMP decompression engine, which allows many mentation were designee[ to be used from CD-ROrVI s spare cycles for performing the copy of the decom­ as well as network connections. SMP seamlessly pressed, device-dependent video to the window integrates with the existing windowing system soft­ system. ware. Because of its potentially universal availabil­ Currently, SMP is offered as a product only on the ity, SMP can serve an important function as the DECstation family of workstations, but it has been lowest common denominator fo r digital video ported to a variety of platforms, including across multiple platforms.

• DEC AXP workstations running the DEC OSF/1 Acknowledgments AXP operating system We would like to thank all the people who have • Alpha AXP systems running the OpenVMS oper­ contribu ted to making software motion pictures a ating system reality. Particular thanks go to Paul Ta llett fo r writ­ ing the original demonstrator and insisting on the • DECpc AXP personal computers running the importance of a color version. He also implemented Windows NT AXP operating system the VMS versions. Thanks also to European External Research fo r making the initial research and later • VA X systems running the VMS operating system product transition possible. Last but not least, • Sun SPARCstation thanks to Susan Angebranncltand her engineering team fo r their help and confidence in this work. • IBM RS/6000 system

• HP/PA Precision system References

1. Special Issue on Digital Multimedia Systems, • SCO UNL'{;'Intel Communications of the ACJVJ, vol. 34, no. 4 • Microsoft Windows version 3.1 (April 1991 ).

26 Vo l. 5 No . .! .\fJrin,� /'J')J Digital Te chnical jo urnal Softu.x/re Motion Pictures

2. L. Palmer and R. Palmer, "DECspin: A Net­ actions on Communications, vol. COM-27 worked Desktop Videoconferencing Applica­ (1979): 1335-1342. tion," Digital Te chnicaljournal, vol. 5, no. 2 (Spring 1993, this issue): 65-76. 8. M. Lema and 0. Mitchell, "Absolute Moment Block Truncation Coding and Its Application 3. G. Campbell et al., "Two Bit/Pixel Full Color to Color Images," ltEH Transactions on Com­ Encoding," SIGGRAPJ-1'86 Conference Pro­ munications, vol. COM-32, no. 10 (1984): ceedings, vol. 20, no. 4 (1986): 215-223. 1148-1157 4. D. LeGall, "MPEG: A Video Compression Stan­ 9. P Heckbert, "Color Image Quantization fo r dan.l to r Multimedia Applications," Communi­ Frame Butle r Display," Computer Graphics cations of the vol. 34, no. 4 (April 1991): AD\1, (A! HC SJGGRAPH'82 Conference Proceedings), 47-58. voJ. 16, no. 3 (1982): 297-307 5. 0. Mitchell, E. Delp, and S. Carlton, "Block 10. M. Pins, "Analyse und Auswabl von AJgorith­ Truncation Coding: A New Approach to men zur Datenkompression unter besonderer Image Compression," Conference Record, IELE International Conference Communica­ Berucksichtigung von Bildern und Bildfol­ gen,'' Ph.D. thesis, University of Karlsruhe, tirms, voJ. I Oune 1978): 12B.l.l - 12B.l.4. 1990. 6. T. Kishimoto, E. Mitsuya, and K. Hoshida, "A Method of Still Picture Coding by Using Statis­ 11. B. Lamparter and W Effelsberg, "Digitale tical Properties" (in japanese), Proceedings of Filmiibertragung unci Darstellung im x­ the National Conference of the Institute of Window-System," Lehrstuhl fiir Praktische Electronics and Communications Engineers Informatik rv, University of Mannheim, 1991. of japan, no. 974 (March 1978). 12. R. Ulicbney, ''Video Rendering," Digital Te ch­ 7 E. Delp and 0. Mitchell, "Image Compression nical jo urnal, vol. 5, no. 2 (Spring 1993, this Using Block Truncation Coding," /El;E Tra ns- issue): 9-18.

Digital Te chnical jo urnal V!; l. 5 No. 2 Sp riny, l

Digital Audio Compression

Compared to most digital data types, with the exception of digital uideo, the data rates associated with uncompressed digital audio are substantial. Digital audio compression enables more efficient storage and transmission of audio data. The many forms of audio compression techniques offer a range of encoder and decoder comple:xit)� cornpressed audio quality, and differing amounts of data compression. The J.L-law transformation and ADPCM coder are simple approaches u•itiJ low­ complexi�y, low-compression, and medium audio quality algorithms. The JJPEG! audio standard is a high-complexit)� high-compression, and high audio quality algorithm. These techniques apply to general audio signals and are not specifi cally tuned for speech signals.

Digital audio compression allows the efficient stor­ concludes with a discussion of software-only real­ age and transmission of audio data. The various time implementations. audio compression techniques offe r different levels of complexity, compressed audio quality, and Digital Audio Data amount of data compression. The digital representation of audio data offe rs This paper is a survey of techniques used to com­ many advantages: high noise immunity, stability, press digital audio signals. Its intent is to provide and reproducibili ty. Audio in digital fo rm also useful information fo r readers of all levels of experi­ allows the efficient implementation of many audio ence with digital audio processing. The paper processing fu nctions (e.g., mixing, filtering, aJl(l begins with a summary of the basic audio digitiza­ equalization) through the digital computer. tion process. The next two sections present The conversion from the analog to the digital detailed descriptions of two relatively simple domain begins by sampling the audio input in regu­ approaches to audio compression: p.-law and adap­ lar, discrete intervals of time ancl quantizing the tive differential pulse code modulation. In the fol­ sampled values into a discrete number of evenly lowing section, the paper gives an overview of a spaced levels. 'The digital audio data consists of a third, much more sophisticated, compression sequence of binary values representing the number audio algorithm from the Motion Picture Experts of quantizer levels fo r each audio sample. The Group. The topics covered in this section are quire method of representing each sam ple with an inde­ complex and are intended fo r the reader who is pendent code word is called pulse code modulation fam iliar with digital signal processing. The paper (PCM). Figure I shows the digital audio process.

001 10111000 ... 11001 100100 .1111111111 .. "11111111111'

ANALOG ANALOG AUDIO PCM PCM AUDIO INPUT VALUES VALUES OUTPUT ANALOG-TO-DIGITAL DIGITAL SIGNAL DIGITAL-TO-ANALOG CONVERSION PROCESSING CONVERSION

Figure 1 Digital Audio Process

28 1-•b/. 5 No. 1 Sp ri11g I'J'JJ Digital Te chuical journal Digital Audio Compression

According to the Nyquist theory, a time-sampled signals and are not specifically tuned fo r speech sig­ signal can fa ithfully represent signals up to half the nals. This paper does not cover audio compression sam piing rate. 1 Typ ical sampling rates range from algorithms designed specifically fo r speech signals. 8 kilohertz (kHz) to 48 kHz. The 8-kHz rate covers These algorithms are generally based on a model­ a frequency range up to 4 kHz and so covers most of ing of the vocal tract and do not work well fo r non­ the frequencies produced by the human voice. The speech audio signals.1.-i The fe deral standards 1015 48-kHz rate covers a frequency range up to 24 kHz LPC (linear predictive coding) and 1016 CELP (coded and more than adequately covers the entire audible excited linear pred iction) fall into this category of frequency range, which fo r humans typically audio compression. extends to only 20 kHz. In practice, the frequency range is somewhat less than half the sampling rate j.L-lawAud io Compression because of the practical system limitations. The p.- law transformation is a basic audio compres­ The number of quantizer levels is typically a sion technique specified by the Comite Consultatif power of 2 to make full use of a fixed number of Internationale de Telegraphique et Telephonique bits per audio sample to represent the quantized (CCITI) G.71 values. With uniform quantizer step spacing, each Recommendation P The transfor­ mation is essentially logarithmic in nature and additional bit has the potential of increasing the allows the 8 bits per sample output codes to cover a signal-to-noise ratio, or equivalently the dynamic dynamic range equivalent to 14 bits of linearly quan­ range, of the quantized amplitude by roughly tized values. This transformation offers a compres­ 6 decibels (dB). The typical number of bits per sam­ sion ratio of (nu mber of bits per source sample)/ ple used fo r digital audio ranges from 8 to 16. The 8 to 1. Unlike linear quantization, the logarithmic dynamic range capability of these representations step spacings represent low-amplitude audio sam­ thus ranges from 48 to 96 dB, respectively. To put these ranges into perspective, if 0 dB represents the ples with greater accuracy than higher-amplitude weakest audible sound pressure level, then 25 dB values. Thus the signal-to-noise ratio of the trans­ is the minimum noise level in a typical recording fo rmed output is more uniform over the range of studio, 35 dB is the noise level inside a quiet home, amplitudes of the input signal. The p.-law transfor­ and 120 dB is the loudest level before discomfort mation is begins 2 In terms of audio perception, 1 dB is the 127 minimum audible change in sound pressure level _ 255 - X In (1 + J.LIXI)fo r x � 0 ln(l + J.L) under the best conditions, and doubling the sound y - 127 pressure level amounts to one perceptual step in l 127 - In (1 + J.L ixl)for x 0 X < ln(l + J.L) loudness.

Compared to most digital data types (d igital = where m 255, and xis the value of the input sig­ video excluded), the data rates associated with nal normalized to have a maximum value of 1. The uncompressed digital audio are substantial For . CCITI Recommendation G.71 1 also specifies a simi­ example, the audio data on a compact disc (2 chan­ Jar A-law transformation. The p.-lawtransf ormation nels of audio sampled at 44.1 kHz with 16 bits per is in common use in North America and Japan fo r sample) requires a data rate of about I .4 megabits the Integrated Services Digital Network (ISDN) per second. There is a clear need fo r some fo rm of 8-kHz-sampled, voice-gracle, digital telephony ser­ compression enable the more efficient storage to vice, and the A- law transformation is used else­ and transmission of this data. where fo r the ISDN telephony. The many fo rms of audio compression tech­ niques diffe r in the trade-offs between encoder and decoder complexity, the compressed audio quality, Adaptive Differential Pulse and the amount of data compression. The tech­ Code Modulation niques presented in the fo llowing sections of this Figure 2 shows a simplified block diagram of paper cover the fu ll range from the p.- law, a low­ an adaptive differential pulse code modulation complexity, low-compression, and medium audio (ADPCM) coder6 For the sake of clarity, the figure qua lity algo rithm, to MPEG/a udio, a high-complex­ omits details such as bit-stream fo rmatting, the pos­ ity, high-compression, and high audio quality algo­ sible use of side information, and the adaptation rithm. These techniques apply to general audio blocks. The ADPCM coder takes advantage of the

Digital Te cbnical jom'lu tl Vo l. 5 No. 2 Sp ring 19')1 29 Multimedia

PCM values. This side information can serve (ADAPTIVE) qn] two purposes. First, in some ADPCM schemes OUANTIZER the decoder needs the additional information to determine either the predictor or the quantizer

(ADAPTIVE) (ADAPTIVE) step size, or both. Second , the data can provide PREDICTOR DEQUANTIZER redundant contextual information to the decoder to enable recovery from errors in the bit stream or to allow random access entry into the coded bit stream. The fo llowing section describes the ADPCM (a) ADPCM Encoder algorithm proposed by the Interactive Multimedia Association (IMA). This algorithm offers a compres­ sion factor of (number of bits per source sample)/ 4 to I. Other ADPCM audio compression schemes Cjn) (ADAPTIVE) Dq[n) + Xp[n] DEOUANTIZER include the CCXII Recommendation G.721 (32 kilo­ bits per second compressed data rate) and (ADAPTIVE) Recommendation G.723 (24 kilobits per second PREDICTOR compressed data rate) standards and the compact disc interactive audio compression algorithm.

(b) ADPClvl Decoder TI1 e IMA ADPCM Algorithm The IMAis a consor­ tium of computer hardware and software vendors Figure 2 ADPCM Compression and cooperating to develop a de facto standard fo r com­ Decompression puter multimedia data. The JMA's goal for its audio compression proposal was to select a publ ic­ domain audio compression algorithm able to pro­ fact that neighboring audio samples are generally vide good compressed audio quality with good similar to each other. Instead of representing each data compression performance. In addition. the audio sample independently as in PGvl , an ADPCM algorithm had to be simple enough to enable encoder computes the difference between each software-only, re al-time decompression of stereo, audio sample and its predicted value and outputs 44.1-kHz-sampled , audio signals on a 20-megahertz the PCM value of the diffe re ntiaL Note that (MHZ) 386-class computer. The selected ADPCM the ADPCM encoder (Figure 2a) uses most of the algorithm nor only meets these goals, but is also components of the ADPCM decoder (Figure 2b) to simple enough to enable software-only, real-time compute the predicted values. encoding on the same computer.

The quantizer output is genera lly only a (signed) The si mpl icity of the JMA ADPGv1 proposal lies in representation of the number of quantizer levels. the crudity of its predictor. The predicted value of The requantizer reconstructs the value of the quan­ the audio sample is simply the decoded value of the tized sample by mu ltiplying the number of quan­ immediately previous audio sample. Thus the pre­ tizer levels by the quantizer step size and possibly dictor block in Figure 2 is merely a ti me-delay adding an offset of half a step size. Depending on element whose output is the input delayed by one the quantizer implementation, this offset may be audio sample interval. Since this predictor is not necessary to center the requantized value between adaptive , side information is not necessary fo r the the quantization thresholds. reconstruction of the pred ictor. The ADPCM coder can adapt to the characteristics Figure 3 shows a block diagram of the quantiza­ of the audio signal by changing the step size of tion process used by the IMA algorithm. The quan­ either the quantizer or the predictor, or by chang­ tizer outputs fo ur bits representing the signed ing both. The method of computing the predicted magnitude of the number of quantizer levels for val.ue and the way the predictor and the quantizer each input sample. adapt to the audio signal vary among diffe re nt Adaptation to the audio signal takes place only in ADPCM coding systems. the quanrizer block. The quantizer adapts the step Some ADPCM systems requ ire the encoder to size based on the current step size an'! the quan­ provide side information with the differential tizer output of the immediately previous input.

Vo l. No. 2 30 5 Sj; ring 1993 Digital Te chuicaljounwl Digital Audio Compression

BIT 3 = 1; SAMPLE =-SAMPLE

BIT 3 = 0 ..

BIT 1 = 1; SAMPLE = SAMPLE - STEP SIZE/2

BIT 1 = 0 ..

Figure 3 IMA ADPCM Quantization

This adaptation can be done as a sequence of two lookup, the data used fo r adaptation is completely table lookups. The three bits representing the deducible from the quantizer outputs; side informa­ number of quantizer levels serve as an index into tion is not required for the quantizer adaptation. the first table lookup whose output is an index Figure 4 illustrates a block diagram of the step-size adjustment fo r the second table lookup. This adjust­ adaptation process, and Ta bles 1 and 2 provide the ment is added to a stored index value, and the table lookup contents. range-limited result is used as the index to the sec­ ond table lookup. The summed index value is IMA ADPCM: Error Recovery A fo rtunate side stored fo r use in the next iteration of the step-size effect of the design of this ADPCM scheme is adaptation. The output of the second table lookup that decoder errors caused by isolated code word is the new quantizer step size. Note that given a errors or edits, splices, or random access of the starting value fo r the index into the second table compressed bit stream generally do not have a

LOWER THREE BITS OF QUANTIZER INDEX STEP OUTPUT FIRST ADJUSTME� LIMIT VALUE SECOND SIZE TABLE BETWEEN TABLE + 0-- + 88 I-- LOOKUP ..._ + 0 AND LOOKUP

DELAY FOR NEXT � ITERATION OF STEP-SIZE f- ADAPTATION

Figure 4 IiviA ADPCM Step-size Adaptation

Vn /. 1993 Digital Te chnical jou r11al j No. 2 Sj>ring 31 Multimedia

Ta ble 1 First Ta ble Lookup for the IMA limited and not disastrous for the IMA algorithm. ADPCM Quantizer Adaptation The decoder reconstructs the audio sample, Xp [n],

Three Bits by adding the previously decoded audio sample, Quantized Index XjJ [n -1], to the result of a signed magnitude prod­ Magnitude Adj ustment uct of the code word, C[n], and the quantizer step

000 -1 size plus an offset of one-half step size:

001 -1 ' Xp [n] = XjJ[n-1] + step_size[n] X C [n] 010 -1 where C'in] = one-half plus a suitable numeric 011 -1 conversion of C[n]. 100 2 An analysis of the second step-size table lookup 101 4 reveals that each successive entry is about 1.1 times 110 6 the previous entry. As long as range lim iting of the 111 8 second table index does not take place, the value fo r step_size[n] is approximately the product of the disastrous impact on decoder output. This is usu­ previous value, step_size[n-1], and a function of ally not true for compression schemes that use the code word, F(C[n-1]): prediction. Since prediction relies on the correct step_size[n] = step_size[n -lj X F(C[n -1]) decoding of previous audio samples, errors in the decoder tend to propagate. The next section The above two equations can be manipulated explains why the error propagation is generally to express the decoded audio sample, Xp [n], as a

Ta ble 2 Second Ta ble Lookup for the IMA ADPCM Quantizer Adaptation

Index Step Size Index Step Size Index Step Size Index Step Size

0 7 22 60 44 494 66 4,026

8 23 66 45 544 67 4,428

2 9 24 73 46 598 68 4,871

3 10 25 80 47 658 69 5,358

4 11 26 88 48 724 70 5,894

5 12 27 97 49 796 71 6,484

6 13 28 107 50 876 72 7,1 32

7 14 29 118 51 963 73 7,845

8 16 30 130 52 1,060 74 8,630 9 17 31 143 53 1,166 75 9,493

10 19 32 157 54 1,282 76 10,442 11 21 33 173 55 1,411 77 11,487 12 23 34 190 56 1,552 78 12,635

13 25 35 209 57 1,707 79 13,899 14 28 36 230 58 1,878 80 15,289

15 31 37 253 59 2,066 81 16,818

16 34 38 279 60 2,272 82 18,500

17 37 39 307 61 2,499 83 20,350 18 41 40 337 62 2,74 9 84 22,358 19 45 41 371 63 3,024 85 24,623

20 50 42 408 64 3,327 86 27,086

21 55 43 449 65 3,660 87 29,794 88 32 ,767

32 Vo l. 5 No. 2 Sp ring 1993 Digital Te chnical jo urnal Digital Audio Comp ression

fu nction of the step size and the decoded sample standard addresses the compression of synchro­ value at time, m, and the set of code words nized video and audio at a total bit rate of roughly between time, m, and n 1.5 megabits per second. Likep,- law and ADPCM, the MPEG/audio compres­ Xp [n] = Xp [m] + step_size[m) sion is lossy; however, the MPEG algorithm can n i achieve transparent, perceptually lossless com­ ' X� !IT F(C[j))) X C [i) pression. The MPEG/audio committee conducted i= m+ l j= m+ l extensive subjective listening tests during the development of the standard. The tests showed Note that the terms in the summation are only that even with a 6-to-1 compression ratio (stereo, a function of the code words from time m + 1 16-bit-per-sample audio sampled at 48 kHz com­ onward. An error in the code word, C[q], or a ran­ pressed to 256 kilobits per second) and under opti­ dom access entry into the bit stream at time q can mal istening conditions, expert listeners were resu lt in an error in the decoded output, Xp [q], and I unable to distinguish between coded and original the quantizer step size, step_size[q+ 1). The above audio clips with statistical significance. Further­ equation shows that an error in Xp [m] amounts to more, these clips were specially chosen because a constant offset to future values of Xp [n]. This they are difficult to compress. Grewin and Ryden offset is inaudible unless the decoded output give the details of the setup, procedures, and exceeds its permissible range and is clipped. results of these tests 9 Clipping results in a momentary audible distortion The high performance of this compression algo­ but also serves to correct partially or fu lly the offset rithm is due to the exploitation of auditory mask­ term. Furthermore, digital high-pass filtering of the ing. Th is masking is a perceptual weakness of the decoder output can remove this constant offset ear that occurs whenever the presence of a strong term. The above equation also shows that an error audio signal makes a spectral neighborhood of in step_size[m + 1] amounts to an unwanted gain or weaker audio signals imperceptible. This noise­ attenuation of fu ture values of the decoded output masking phenomenon bas been observed and cor­ Xp [n]. The shape of the output wave fo rm is roborated through a variety of psychoacoustic unchanged unless the index to the second step-size experiments. Jo table lookup is range limited. Range limiting resu lts Empirical results also show that the ear has a lim­ in a partial or full correction to the value of the step ited frequency selectivity that varies in acuity from size. less than 100 Hz for the lowest audible frequencies The nature of the step-size adaptation I imits the to more than 4 kHz fo r the highest. Thus the audible impact of an error in the step size. Note that an spectrum can be partitioned into critical bands that error in step_size[m+ 1) caused by an error in a sin­ reflect the resolving power of the ear as a function gle code word can be at most a change of (1.1)\>, or of frequency. Ta ble 3 gives a listing of critical band­ 7.45 dB in the value of the step size. Note also that widths. any sequence of 88 code words that all have magni­ Because of the ear's limited frequency resolving tude 3 or less (refer to Ta ble 1) completely corrects power, the threshold fo r noise masking at any given the step size to its minimum value. Even at the low­ frequency is solely dependent on the signal activity est audio sampling rate typically used, 8 kHz, 88 within a critical band of that frequency. Figure 5 samples correspond to 11 milliseconds of audio. illustrates this property. For audio compression, Thus random access entry or edit points exist this property can be capitalized by transforming whenever 11 milliseconds of low-level signal occur the audio signal into the frequency domain, then in the audio stream. dividing the resulting spectrum into subbands that approximate critical bands, and finally quantizing MPEG/Audio Compression each subband according to the audibility of quanti­ The Motion Picture Experts Group (MPEG) audio zation noise within that band. For optimal compres­ compression algorithm is an International Organi­ sion, each band should be quantized with no more zation fo r Standardization (ISO) standard fo r high­ levels than necessary to make the quantization fidelity audio compression. It is one part of a noise inaudible. The fo llowing sections present three-part compression standard. With the other a more detailed description of the MPEG;audio two parts, video and systems, the composite algorithm.

Digital Te chllicaljournal Vo l. 5 No. 2 Sp ring 1993 33 Multimedia

Ta ble 3 Approximate Critical Band Finally, the last block takes the representation of Boundaries the quantized audio samp.les and fo rmats the data

Band Frequency Band Frequency into a decodable bit stream. The decoder simply Number (Hz)* Number (Hz)* reverses the fo rmatting, then reconstructs the quantized subband values, and fi nally transforms 0 50 14 1,970 the set of subband values into a time-domain audio 1 95 15 2,340 signal. As specified by the ,vi i'EG re quirements, 2 140 16 2,720 ancillary data not necessarily related to the aud io 3 235 17 3,280 stream can be fi tted within the coded bit stream. 4 330 18 3,840 The MPEC;;a udio standard has three distinct lay­ 5 420 19 4,690 ers for compression. Layer l t< >rms the most basic algorithm, and Layers and 111 are enhancements 6 560 20 5,440 rr that use some elements fo und in Layer I. Each suc­ 7 660 21 6,375 cessive layer improves the compression perfor­ 8 800 22 7, 690 mance but at the cost of greater encoder and 9 940 23 9,375 decoder complexity. 10 1,125 24 11,625

11 1,265 25 15,375 Layer I The Layer l algo rithm uses the basic filter 12 1,500 26 20,250 bank found in all layers. This filter bank divides the 13 1,735 audio signal into 32 constant-width frequency bands. The filters are relatively simple and provide • Frequencies are at the upper end the band. of good time resolution with reasonable frequency resolution relative to the perceptual properties of MPEG/Audio Encoding and Decoding the human ear. The design is a compromise with three notable concessions. First, the 32 constant­ Figure 6 shows block diagrams of the MPEG/ width bands do not accurately reflect the ear's criti­ audio encoder and decoder. 1112 In this high-level cal bands. Figure 7 illustrates this discrepancy. The representation, encoding closely parallels the pro­ bandwidth is too wide fo r the lower frequencies so cess described above. The input audio stream the number of quantizer bits cannot be specifically passes through a fil ter bank that divides the input tuned for the noise sensitivity within each critical into multiple subbands. The input audio stream band. Instead, the incl uded critical band with the simultaneously passes through a psychoacoustic greatest noise sensitivity dictates the number of model that determines the signal-to-mask ratio of quantization bits required fo r the entire fi lter band. each subband . The bit or noise allocation block Second, the filter bank and its inverse are not loss­ uses the signal-to-mask ratios to decide how to less transformations. Even without quantization, apportion the total number of code bits available the inverse transformation wou ld nor perfectly fo r the quantization of the subband signals to mini­ recover the original input signal. Fortunately, the mize the audibility of the quantization noise. error introduced by the fi lter bank is small and inaudible. Finally, adjacent filter bands h�tve a signif­ icant frequency overlap. A signal at a single fre­ quency can affect two adjacent filter bank outputs. The filter bank provides 32 frequency samples, STRONG TONAL SIGNAL / one sample per band, fo r every 32 input audio sam­

ples. The Layer l algorithm groups together 12 sam­ ples from each of the 32 hands. Each group of 12 samples receives a bit allocation and, if the bit allo­ cation is not zero, a scale factor. Coding fo r stereo redundancy compression is sl ightly diffe rent and is discussed later in this paper. 'T'he bit all.ocation FREQUENCY determines the number of bits used to represent each sample. The scale factor is a multiplier that

Fig ure 5 Audio No ise Masking sizes the samples to maximize the resolution of the quantizer. The Layer I encoder fo rmats the

34 V!JI. 5 No. 2 Spring I'J'J3 Digilal Te cbuica/ jo urual Digital Audio Compression

PCM AUDIO ENCODED BIT/NOISE INPUT BIT STREAM TIME-TO-FREQUENCY ALLOCATION, BIT-STREAM MAPPING FILTER � OUANTIZER, AND � FORMATIING BANK CODING -· I I I I ANCILLARY DATA PSYCHOACOUSTIC T MODEL I (OPTIONAL)

(a) MPEG/A udio Encoder

ENCODED DECODED FREQUENCY PCM AUDIO BIT STREAM BIT-STREAM FREQUENCY-TO-TIME SAMPLE UNPACKING r---- f------. MAPPING - RECONSTRUCTION I I I y ANCILLARY DATA (IF ENCODED)

(b) MPEG/A udio Decoder

Figure 6 MPEG/Au dio Compression and Decompression

MPEG/AUDIO FILTER BANK BANDS

I 0 I 1 I 2 I 3 I 4 I 5 I 6 I 7 I 8 I 9 I 10 I11 I 12 I 13I 14 I 15 I 16 I 17I 18 I 19 I 20 I 21 I 22 I 23 I 24 I 25 I 26 I 27 I 28 I 29 I 30 I 31 I,

CRITICAL BAND BOUNDARIES

Figure 7 MPEG/A udio Filter Bandwidths versus Critical Bandwidths

32 groups of 12 samples (i.e., 384 samples) inro a subband, Layer II codes data in 3 groups of 12 sam­ frame. Besides the audio data, each frame contains ples fo r each subband. Again discounting stereo a header, an optional cyclic redundancy code (CRC) redundancy coding, there is one bit allocation and check word, and possibly ancillary data. up to three scale factors fo r each trio of 12 samples. The encoder encodes with a unique scale factor for

Layer II The Layer II algorithm is a simple each group of 12 samples only if necessary to avoid enhancement of Layer I. It improves compression audible distortion. The encoder shares scale factor performance by coding data in larger groups. The values between two or all three groups in two Layer II encoder forms frames of 3 by 12 by 32 = other cases: (1) when the values of the scale factors 1,152 samples per audio channeL Whereas Layer I are sufficiently close and (2) when the encoder codes data in single groups of 12 samples for each anticipates that temporal noise masking by the ear

vbl. 'i . No Digital Te chnical jo urnal .! Spring 1993 35 Multimedia

will hide the consequent distortion. The Layer ll • Al ias reduction - Layer III specifies a method of algorithm also improves performance over Layer I processing the MDCT values to remove some by representing the bit allocation, the scale factor redundancy caused by the overlapping bands of

values, and the quantized samples with a more effi­ the Layer I and Layer II filter bank. cient code. • Nonuniform quantization - The Layer III quan­

tizer raises its input to the 3/4 power before Ill Layer lll The Layer algorithm is a much more quantization to provide a more consistent signal­ 1·1 1 i refined approach. Although based on the same to-noise ratio over the range of quantizer values. II, III filter bank fo und in Layers I and Layer compen­ The requantizer in the MPEG/audio decoder sates fo r some filter bank deficiencies by process­ relinearizes the values by raising its output to ing the filter outputs with a modified discrete the 4/3 power. cosine transform (MDCT) . Figure 8 shows a block diagram of the process. • Entropy coding of clara values - Layer JII uses The MDCTs further subdivide the filter bank out­ Huffman codes to encode the quantized samples puts in frequency to provide better spectral resolu­ for better data compression. 1" tion. Because of the inev itable trade-off between • Use of a bit reservoir - The design of the Layer time and frequency resolution, Layer Ill specifies Ill bit stream better fits the variable length nature of two different MDCT block lengths: a long block of 36 the compressed data . As with Layer II, Layer samples or a short block of 12. The short block length II! processes the audio data in frames of 1,152 sam­ improves the time resolution to cope with tran­ ples. Unlike Layer II, the coded data representing sients. Note that the short block length is one-th ird these samples does not necessarily fit into a that of a long block; when used, three short blocks fixed-length frame in the code bit stream. The replace a single long block. The switch between encoder can donate bits to or borrow bits from long and short blocks is not instantaneous. A long the reservoir when appropriate. block with a special ized long-to-short or short-to­ long data window provides the transition mecha­ • Noise allocation instead of bit allocation - The nism from a long to a short block. Layer Ill has three bit allocation process used by Layers l and II only blocking modes: two modes where the outputs of approximates the amount of noise caused by the 32 filter banks can all pass through MDCTs with quantization to a given number of bits. The Layer the same block length and a mi.,'l:ed block mode III encoder uses a noise allocation iteration where the 2 lower-frequency bands use long blocks loop. In this loop, the quantizers are varied in an and the 30 upper bands use short blocks. orderly way, and the resu lting quantization noise Other major enhancements over the Layer I and is actually calculated and specifically allocated Layer II algorithms include: to each subband.

SUBBAND 0

PCM AUDIO LAYER I SUBBAND 1 INPUT AND -- LAYER II FILTER BANK

SUBBAND 31

LONG, LONG-TO-SHORT SHORT, SHORT-TO-LONG LONG OR SHORT BLOCK WINDOW SELECT CONTROL (FROM PSYCHOACOUSTIC MODEL)

Fig ure 8 MPEG/Aud io Laye r Ill Fi lter Bank Processing, Encoder Side

36 Vol. 5 No . .1 Sp ring 19'J3 Digital Te chnical jo urnal Digital Audio Compression

Th e Psy choacoustic Model and noiselike components of the audio signal The psychoacoustic model is the key component of because the noise-masking characteristics of the the MPEG encoder that enables its high perfor­ two types of signal are diffe rent. mance. 1�> n!u� The job of the psychoacoustic model • Apply spreading function - The model deter­ is to analyze the input audio signal and determine mines the noise-masking thresholds by applying where in the spectrum quantization noise will be an empirical ly determined masking or spreading masked and to what extent. The encoder uses this fu nction to the signal components. information to decide how best to represent the input audio signal with its limited number of code • Find the minimum masking threshold fo r each bits. The MPEG/audio standard prov ides two exam­ subband - The psychoacoustic model calculates ple implementations of the psychoacoustic model. the masking thresholds with a higher-frequency Below is a general outline of the basic steps resolution than provided by the filter banks. involved in the psychoacoustic calculations fo r Wherethe filter band is wide relative to the criti­ either model. cal band (at the lower end of the spectrum), the model selects the minimum of the masking • Time align audio data - The psychoacoustic thresholds covered by the filter band. Where the model must account fo r both the delay of the filter band is narrow relative to the critical band, audio data through the filter bank and a data the model uses the average of the masking offset so that the relevant data is centered within thresholds covered by the filter band. its analysis window. For example, when using • Calculate signal-to-mask ratio - The psycho­ psychoacoustic model two fo r Layer I, the delay acoustic model takes the minimum masking through the filter bank is 256 samples, and the threshold and computes the signal-to-mask offset required to center the 384 samples of a ratio; it then passes this value to the bit (or Layer I frame in the 512-point psychoacoustic noise) allocation section of the encoder. analysis window is (512 - 384)/2 = 64 points. The net offset is 320 points to time align the Stereo Redundancy Coding psychoacoustic model data with the filter bank outputs. The MPEG/audio compression algorithm supports two types of stereo redundancy coding: intensity • Convert audio to spectral domain - The psy­ stereo coding and middle/side (MS) stereo coding. choacoustic model uses a time-to-frequency Both fo rms of redundancy coding exploit another mapping such as a 512- or 1 ,024-point Fourier perceptual weakness of the ear. Psychoacoustic transform. A standard Hann weighting, applied results show that, within the critical bands cover­ to audio data before Fourier transformation, ing frequencies above approx imately 2 kHz, the conditions the data to reduce the edge effects of ear bases its perception of stereo imaging more the transform window. The model uses this sep­ on the temporal envelope of the audio signal than arate and independent mapping instead of the its temporal fine structure. All layers support inten­ filter bank outputs because it needs finer fre­ sity stereo coding. Layer Ill also supports MS stereo quency resolution to calculate the masking coding. thresholds. In intensity stereo mode, the encoder codes some upper-frequency filter bank outputs with a • Partition spectral values into critical bands - To single summed signal rather than send independent simplify the psychoacoustic calculations, the codes for left and right channels fo r each of the 32 model groups the frequency values into percep­ filter bank outputs. The intensity stereo decoder tual quanta. reconstructs the left and right channels based only on independent left- and right-channel scale fac­ • Incorporate threshold in quiet - The model tors. With intensity stereo coding, the spectral includes an empirically determined absolute shape of the left and right channels is the same masking threshold. This threshold is the lower within each intensity-coded filter bank signal, but bound for noise masking and is determined in the magnitude is different. the absence of masking signals. The MS stereo mode encodes the signals fo r left • Separate into tonal and nontonal components - and right channels in certain frequency ranges as The model must identify and separate the tonal middle (sum of left and right) and side (diffe rence

Digital Te chuical jounwl Vol. 5 No. 2 Sp ring 1993 37 Multimedia

of left and right) channels. In this mode, the

encoder uses specially tuned techniques to further SHIFT IN 32 NEW SAMPLES compress side-channel signal. INTO 512-POINT FIFO BUFFER. X;

Real-time SoftwareIm plementations �

The software-only implementations of the J.L-law WINDOW SAMPLES: and ADPCMalg orithms can easily run in real time. A FOR i = 0 TO 51 1, DO Z; = C; X X; single table lookup can do J.L-law compression or decompression. A software-only implementation � of the IMA ADPCM algorithm can process stereo, PARTIAL CALCULATION 44.1-kHz-sampled audio in real time on a 20-MHz 7

FOR i = DO yl = 386 -class computer. The challenge lies in develop­ 0 TO 63, L z l + 64j 1=0 ing a real-time software implementation of the 1\'IPEG/audio algorithm. The MPEG standards docu­ � ment does not offer many clues in this respect. CALCULATE 32 SAMPLES BY There are much more efficient ways to compute 63 MATRIXING S; = L Y; x M;,k the calculations required by the encoding and k=O decoding processes than the procedures outlined by the standard. As an example, the fo llowing sec­ tion details how the number of multiplies and addi­ �

tions used in a certain calculation can be reduced OUTPUT 32 SUBBANO SAMPLES by a factor of 12. Figure 9 shows a flow chart fo r the analysis sub­ band filter used by the MPEG/audio encoder. Most of the computational load is clue to the second­ Figure 9 Flow Diagram of tbe MPEG/A udio from-last block. This block contains the following Encoder Fi lter Bank matrix multiply:

Thus with the almost negligible overhead of com­ � (2 X i+1) X (k -16) X S(i) = .::.., Y(k) cos puting the F(k) values, a twofold reduction in mul­ X [ 64 TI] k=O tiplies and additions comes from halving the range fo r i= 0 ... 31. that k varies. Another reduction in multiplies and additions of more than sixfold comes from using Using the above equation, each of the 31 values of one of many possible fast algorithms fo r the compu­ S(i) requires 63 adds and 64 multiplies. To optimize tation of the inverse DCT.20 2122 There is a similar this calculation, note that the M(i,k) coefficients optimization applicable to the 64 by 32 matrix mul­ are similar to the coefficients used by a 32-point, tiply fo und within the decoder's subband filter un-normalized inverse discrete cosine transform banJ<. (DCT)given by Many other optimizations are possible fo r both MPEG/audio encoder and decoder. Such optimiza­ 31 (2 X i+ 1) X k X tions enable a software-only version of the MPEG/ f( i) = � F(k) cos X [ 64 TI] audio Layer I or Layer 11 decoder (written in the C k=O programming language) to obtain real-time per­ fo ri= 0 ... 31. fo rmance fo r the decoding of high-fidelity mono­ phonic audio data on a DECstation 5000 Model 200. Indeed, 5(1) is identical toj(1) if F(k) is computed This workstation uses a 25-MHz R3000 MIPS CPU as fo llows and has 128 kilobytes of external instruction and data cache. With this optimized software, the F(k) = Y( 16) fo rk = 0; MPEG/audio Layer II algorithm requires an average

= 6 6 6 Y(k + 1 ) + Y( l -k) fo r k= 1 ... 1 ; of 13.7 seconds of CPU time (12.8 seconds of user

= Y(k + 16)-Y(80-k) for k= 17 ... 31 time and 0.9 seconds of system time) to decode 7.47

38 Vo l. 5 No. 2 Sp ring 1993 Digital Te chnical jou rnal Digital Audio Compression

seconds of a stereo audio signal sampled at 48 kHz 7. M. Nishiguchi, K. Akagiri, and T. Suzuki, with 16 bits per sample. "A New Audio Bit Rate Reduction System Although real-time MPEG/auclio decoding of for the CD-I Format," Preprint 2375, 8Jst stereo audio is not possible on the DECstation 5000, Audio Engineering Society Convention, Los such decoding is possible on Digital's workstations Angeles (1986). equipped with the 150-MHZ DECchip 21064 CPU 8. Takahashi, H. Yazawa, K. Ya mamoto, and (Alpha AXP architecture) and 512 kilobytes of exter­ Y. Anazawa, "Study and Evaluation of a New nal instruction and data cache. Indeed, when this T. Method of ADPCM Encoding," Preprint 2813, same code (i.e., without CPU-specific optimization) 86th Audio Engineering Society Convention, is compiled and run on a DEC 3000 AXP Model 500 Hamburg (1989). workstation, the MPEG/audio Layer II algorithm requires an average of 4.2 seconds (3.9 seconds of 9. C. Grewin and T. Ryden, "Su bjective Assess­ user time and 0.3 seconds of system time) to ments on Low Bit-rate Audio Codecs," Pro­ 4 decode the same 7. 7-second audio sequence. ceedings of the Te nth International Audio Engineering Society Conference, London Summary (1991): 91 -102. Te chniques to compress general digital audio sig­ 10. ]. To bias, Fo undations of Modern Auditory nals include p.-law and adaptive differential pulse Theory (New Yo rk and London: Academic code modulation. These simple approaches apply Press, 1970): 159-202. low-complexity, low-compression, and medium audio quality algorithms to audio signals. A third 11. K. Brandenburg and G. Stoll, "The ISO/MPEG­ technique, the MPEG/auclio compression algorithm, Audio Codec: A Generic Standard for Coding is an ISO standard fo r high-fidelity audio compres­ of High Quality Digital Audio," Preprint 3336, sion. The MPEG/audio standard bas three layers of 92nd Audio Engineering Society Conven­ successive complexity fo r improved compression tion, Vienna (1992) performance. 12. K. Brandenburg and ]. Herre, "Digital Audio Compression for Professional Applications," References Preprint 3330, 92nd Audio Engineering Society Convention, Vienna(1992). 1. A. Oppenheim and R. Schafer, Discrete Time Signal Processing (Englewood Cliffs , N) : 13. K. Brandenburg and ]. D. Johnston, "Second Prentice-Hall, 1989): 80-87 Generation Perceptual Audio Coding: The Hybrid Coder," Preprint 2937, 88th Audio 2. K. Pohlman, Principles of Digital Audio Engineering Society Convention, Montreaux (Indianapolis, JN: Howard W Sams and Co., (1990). 1989). 14. K. Brandenburg, ). Herre, ). D. Johnston, 3. ). Flanagan, Sp eech Analysis Sy nthesis and Y. Mahieux, and E. Schroeder, "ASPEC: Adap­ Perception (New Yo rk: Springer-Verlag, 1972). tive Spectral Perceptual Entropy Coding of High Quality Music Signals," Preprint 3011, 4. B. Atal, "Predictive Coding of Speech at Low 90th Audio Engineering Society Convention, Rates," IEEE Transactions on Communica­ Paris (1991). tions, vol. COM-30, no. 4 (April 1982). 15. D. Huffman, "A Method fo r the Construction 5. CCJTF Recommendation G. 711: Pulse Code of Minimum Redundancy Codes," Proceed­ Modulation (PCM) of Vo ice Frequencies ings of the IRE, vol. 40 (1962): 1098-1101. (Geneva: International Telecommunications nion, 1972). 16. ). D. Johnston, "Estimation of Perceptual Entropy Using Noise Masking Criteria," Pro­ 6. L. Rabiner and R. Schafer, Digital Processing ceedings of the 1988 IEEE International Con­ of Sp eech Signals (Englewood Cliffs , NJ : fe rence on Acoustics, Sp eech, and Signal Prentice-Hall, 1978). Processing (1988): 2524 -2527

Digital Te chnical journal Vo l. 5 No. 2 Sp ring 1993 39 Multimedia

17 ). D. Johnston, "Transform Coding of Audio 20. ) . Wa rd and B. Stanier, "Fast Discrete Cosine Signals Using Perceptual Noise Criteria," IEEE Tr ansform Algorithm fo r Systolic Arrays,"Elec­ jo urnal on Selected Areas in Communica­ tronics Letters, vol. 19, no. 2 Oanuary 1983). tions, vol. 6 (February 1988): 314 -323. 21. ). Makhoul, "A Fast Cosine Transform in One 18. K. Brandenburg, "OCF-A New Coding Algo­ and Thro Dimensions," IEEE Transactions on rithm fo r High Quality Sound Signals," Proceed­ Acoustics, Sp eech, and Signal Processing, vol. ings of the 1987 IEEE ICASSP (1987): 141-144. ASSP-28, no. 1 (February 1980).

19. D. Wiese and G. Stol l, "Bitrate Reduction of 22. W-H. Chen, C. H. Smith, and S. Fralick, "A Fast High Quality Audio Signals by Modeling the Computational Algorithm fo r the Discrete Ear's Masking Thresholds," Preprint 2970, Cosine Transform," IEEE Transactions on 89th Audio Engineering Society Convention, Communications, vol. COM-25 no. 9 (Septem­ Los Angeles (1990). ber 1977).

40 Vo l. 5 No. 2 Sp ring 1993 Digital Te chuica/ jo urnal ja n B. te Kiefte Robert Hasenaar joop W. Mevius Theo H. van Hunnik

The Megadoc Image Document Management Sy stem

Megadoc image document management solutions are the result of a systems engineering effort that combined several disciplines, ranging from optical disk hardware to an image application framework. Although each of the component technologies may be fairly mature, combining them into easy-to-customize solu­ tions presented a sigmficant systems engineering challenge. The resulting applica­ tion framework allows the configuration of customized solutions with low systems integration cost and short time to deployment.

Electronic Document Management curve-a computer with a scanner and an optical In most organizations, paper is the main medium disk does not fu lly address the issues of a large­ fo r information sharing. Paper is not only a commu­ scale, paper-based operation. Early adopters of nication medium but in many cases also the carrier electronic imaging experienced a challenge in of an organization's vital information assets. Whereas defining the right electronic document indexing the recording of information in document fo rmat is scheme fo r their applications. Even though the done largely with help of electronic equipment, technology is now mature, the introduction of a sharing and distribution of that information is in document imaging system frequently leads to some many cases still done on paper. Large-scale, paper­ fo rm of business process reengineering to exploit based operations have limited options fo r tracking the new options of electronic document manage­ the progress of work. ment. The Megadoc image document management The computer industry thus has two opportunities: system allows the configuration of customer­ specific solutions through its building-block archi­ I. Capture paper documents in electronic image tecture and its built-in customization options. format (ifusing paper is a requirement) The Megadoc system presented in this paper is based on approximately 10 years of experience 2. Provide better tools fo r sharing and distribution with base technology, customer projects, and among work groups (if the use of paper can be everything in between. In those years, Megadoc avoided) image document management has matured from Organizations that use electronic imaging, as the technology delight of optical recording to an compared to handling paper, can better track work application framework fo r image document man­ in progress. Productivity increases (no time is agement. This framework consists of hardware and wasted in searching) and the quality of service software components arranged in various architec­ improves (response times are shorter and no info r­ tural layers: the base system, the optical file server, mation is lost) when vital information is repre­ the storage manager, and the image application sented and tracked electronically. framework. Imaging is not a new technology (see Ta ble 1). The base system consists of PC-based work­ Moreover, this paper does not document new base stations, running the Microsoft Windows operating technology. Instead, we describe the key compo­ system, connected to servers to r storage manage­ nents of an image document management system in ment and to database services fo r document index­ the context of a systems engineering effort. This ing. Specific peripherals include image scanners, effort resulted in a product set that allows the con­ image printers, optional full-screen displays, and figuration of customized solutions. optional write once, read many (WORM) disks. Those who first adopted the use of image tech­ The optical file server abstracts from the diffe r­ nology have had to go through a long learning ences between optical WORM disks and provides

Digital Te cbuiclll]ournal Vo l. 5 No. 2 Sp ring 1993 41 Multimedia

Ta ble 1 History of Image Document Management

1975 Philips Research combines a 12-inch (30.48-centimeter) videodisk for analog storage of facsimile documents and high-resolution video monitors with a minicomputer for indexing in an experimental image management system. 1979 Philips' image management system switches to digital technology through the availability of WORM disks and random-access memory (RAM) chips (for refreshing a full-page video monitor). 1983 At the Hannover Fair (Hannover, Germany), Philips shows Megadoc, an image document management system with WORM disks containing compressed document images. Dedicated image document management solutions are introduced. 1988 Image document management transitions from dedicated image display technology as part of a proprietary computer architecture to an open systems platform with PC-based image workstations. 1993 The image becomes just another document format that is used next to text-coded electronic documents.

the many hundreds of gigabytes (GB) of storage packages that are running the original application required in large-scale image document manage­ in a window on the Megadoc image PC work­ ment systems. stations allow integration with the unchanged The storage manager provides storage and application. retrieval functions fo r the contents of documents. The fo llowing sections describe the optical file Document contents are stored in "containers," i.e., server, the storage manager, and the image applica­ large, one-dimensional storage areas that can span tion framework. multiple optical disk volumes. The Megadoc image application framework con· MegadocOp tical File Server tains three sublayers: The Megadoc optical file server (OFS) software pro­ UNIX WORM 1. Image-related software libraries for scanning, vides a file system interface for disks. OFS viewing, and printing The automatically loads and unloads these WORMvolumes by jukebox robotics in a completely 2. Application templates transparent way. Thus, from an API perspective, OFS 3. A standard folder management application that implements a UNIX file system with a large on-line provides, with some tailoring by the end-user file system storage capacity. Currently, up to 800 GB organization, an "out-of-the-box" image docu­ can be reached with a single jukebox. ment management solution We implemented the OFS in three layers, as shown Figure 1: The optical file server and the storage manager store images in any type of document fo rmat. 1. The optical clisk filer (ODF) layer, which enables However, to meet customer requirements with storing data on write-once devices and provid­ respect to longevity of the documents, images ing a UNIX file system interface. should be stored in compressed format according 2. The volume manager (VM), which loads and to the Comite Consultatif Internationale de Tele­ unloads volumes to and from drives in the juke­ 4 graphique et Telephonique (CCITT) Group boxes and communicates with the system opera­ standard. tor fo r hand ling off-line volumes. In addition to image document management solutions, Megadoc components are used to "image 3. The device layer, which provides device-level enable'' existing data processing applications. In access to the WORM drives and to the jukebox many cases, a data processing application uses hardware. This layer is not discussed further in some means of identification for an application this paper. object (e.g., an order or an invoice). This identifica­ tion relates to a paper document. Megadoc reuses Op tical Disk Fi ler the application's identification as the key to the When we started to design the ODF, the chief image version of that document. Application pro­ prerequisite was that it should adhere to the UNIX gramming interfaces (APis) for terminal emulation file system interface for applications. The obvious

42 Vo l. 5 No. 2 Sp ring 1993 Digital Te chnical journal The Megadoc Image Document Management Sy stem

magnetic disks. (The average seek time fo r a WOR.NI

STORAGE device is 100 milliseconds; rotational delay is about MANAGER OR 35 milliseconds.) Fetching a disk from a ju kebox OTHER UNIX APPLICATION storage slot, loading it, and waiting for spin-up takes between 8 and 15 seconds, depending on the type of jukebox. Caching solves both issues. We decided that the usual in-memory cache would not be sufficient fo r the huge amounts of WORM data, and therefore, we OPTICAL FILE SERVER use partitions of magnetic disks fo r caching.

ODE WORM Layout To avoid duplicating previ­ ous efforts, we used classical UNIX file systems as a guideline fo r the definition of ODF's WOR.JVI Iayout. However, we had to add some indirect pointer HARDWARE mechanisms to avoid update avalanches. Each file system is mapped onto a single WOR.J\tl partition. These partitions are written sequentially, reducing the free block administration to maintaining a cur­ rent write point. Figure 1 n1 e Three Software Layers of the UNIX Op tical File Server The ODF reuses many notions from file sys­ tems, such as i-nodes, superblock, and the func­ tional contents of directory entries.1 Applying benefit was that the designers would not have to these UNIX notions to the optical fi le system write their own utilities to, for example, copy data, resulted in the fo llowing ODF characteristics: create new files, and make new directories. All UNIX utilities wou ld work as well on WORM devices • The superblock contains all global data fo r a file as on any other file system. system. UNIX Current implementations provide two ker­ • Each i-node contains the block list and all the nel interfaces fo r integrating a new file system type attributes of a file except the fi le's name. into the kernel: the file system switch (FSS), in UNIX versions based on the System V Release 3; and the • An i-node number identifies each i-node. UNIX virtual file system (VFS), in implementations • A directory is a special type of file. like the System V Release 4, Sunos, and OSF/1 oper­ ating systems. We introduced the optical disk filer • Entries in a directory map names to i-node in the FSS and later ported it to the VFS. numbers. The key challenge fo r the design of a file system A new notion in the ODF, as compared to UNIX fo r write-once devices is to allow updates without file systems, is the administration file (admin file). causing an "avalanche" of updates. Note that any One such file exists fo r each file system. The file is update to a sector on a WORM device forces a sequential, and its contents are similar to the first rewrite of the full sector at another location. If disk blocks in classical UNfX file systems: the first pointers to an updated sector exist on the WO RM extent contains the superblock, and all other device, sectors that contain those pointers have to extents fo rm a constantly growing array of i-nodes; be rewritten, also. For example, if a file system the i-node's number is the index of the i-node in the implementation is chosen where the list of data file's i-node array. An important difference between blocks fo r a file, or just the sector location of such a UNIXfi le systems and the ODF is thar the 2-kilobyte list, is part of the file's directory information, any (kB), fixed-size extents of the ODF admin file are update to that file would cause a rewrite of the scattered over the WORM device, instead of being directory sector and the sectors fo r the parent stored as a sequential array of disk blocks, as in directories, all the way up to the root directory. UNIX systems. As a result, any update to an i-node, A second issue to be addressed for removable as a consequence of a file update, causes the invali­ optical disks is perfo rmance. Access time for on-line dation of at most one admin file extent. Since disks is at least eight times slower than fo r current the logical index in the admin file of this i-node, i.e.,

Digital Te cb11ical]ournal Vo l. 5 No. 2 Sp ring 1993 43 Multimedia

the i-node number, does not change, the parent Searching the current write point is a fairly fast directories do not have to be updated. operation implemented through binary search and However, this scheme needs an additional indi­ hardware support, which allow the ODF to distin­ rect pointer mechanism: a list of block numbers guish between used and unused data blocks of IK representing the location of the admin file extents. bytes. The ODF stores this list in the admin file's i-node (aino). The aino is a sequential file that contains ODF Caching Caching in the ODF is file oriented. slightly more than block numbers and is a sequence We suggest a magnetic cache size of approximately of contiguous blocks on the WORM disk that con­ 5 percent of the optical disk space. lf data from a tain the same information. Hence, an update to an file on a WORM disk is read, the ODF creates a cache admin file extent always invalidates the entire aino file and copies a contiguous segment of file data on the WORM device, which makes the aino a more from the WORM disk (64 kB in size, or less in the desirable candidate fo r caching than the admin file case of a small file) to the correct offset in the cache extents. file. The cache file is the basis fo r all i!O operations The fo llowing example, shown in Figure 2, illus­ until removed by the ODF, after having rewritten all trates the steps involved in reading logical block N dirty segments (i.e., updated or changed segments) from the file with i-node number 1: back to the WOR..i\ 1 device. The ODF provides special system calls (through the UNIX fc ntl(2) interface) 1. Read the aino to obtain the block number of /'s to flush asynchronously dirty file segments to the admin file extent. WO R..i\1 device and to remove a fi le's cache file. The 2. Read the admin file extent to get file I, which is flusher daemon monitors high and low watermarks used to translate the logical block number N into fo r dirty cache contents. The daemon flushes dirty the physical block number I(N) . data to the optical disks. The flusher daemon flushes data in a sequence that minimizes the num­ 3. Read physical block I(N) . ber of WORM volume movements in a jukebox. The If the file system is in a consolidated state, i.e , all ODF deletes clean data (i.e., data already present on data on the WOR..i\1 disk is current, the aino and the the optical disk) on a least-recently-used basis. superblock are the last pieces of information writ­ The admin file has its own cache file. The mini­ ten to the WOR..i\1 device, directly before the current mum amount of aclmin file clara to be cached is the write point. Blocks written prior to the aino and superblock. The ODF gradually caches the other the superblock contain mainly user data but also aclmi.J1 file extents, which contain the i-nodes, while an occasional admin file extent, fu lly interleaved. the file system is in use. The ODF writes i-node Figure 3 shows the WORM layout. Since ODF updates to the WOR.ivl device as soon as aU i-nodes in requires the first admin file extent and the com­ the same actmin fileext ent have their dirty file data plete aino to be in the cache, introducing a disk written to the WORM device. The aino has its own with consolidated file systems to another system cache file, also, and is always completely cached. requires searching the current write point, reading If all file clara and i-nodes have been written to the the superblock, determining the aino length from WOR..i\1 device, the file system can be consolidated the superblock, and finally reading the aino itself. by a special utility that writes aino and superblock

AI NO ADMIN FILE EXTENTS EXTENTS OF FILE I r-- SUPERBLOCK EXTENT 0 L....- i-NOOE BLOCK 1 EXTENT 1

I I I I I EXTENT N 1- i-NODE BLOCK K c:=il__ _

Figure 2 Steps Involved in Getting fr om tbe Aino to Exte nt N of File I

44 Vo l. 5 No. 2 Sp ring 199) Digital Te e/m ica/ jo urnal Th e Megadoc Image Document Ma nagement Sy stem

PREVIOUS CONSOLIDATION POINT CURRENT CONSOLIDATION POINT \ WORM PARTITION: I

EMPTY SECTORS

PREVIOUS AI NO CURRENT AINO I AND SUPERBLOCK AND SUPERBLOCK

Figure 3 WORN! Layoutfo r a Consolidated ODF File Sy stem

to the WORM device, hence creating a consolida­ volume is in a jukebox slot and not in a directly tion point. accessible drive. Note that since /dev/WOR.J\1_Ahas For reasons of modularity and ease of implemen­ the same interface as /dev/worm, the OFS could tation, we chose the UNIX standard magnetic disk fu nction without the VM layer in any system that file system implementation to perform the caching. contains only one worm drive and one volume that An alternative would have been to use a magnetic is never removed from that drive. However, since disk cache with an optimized, ODF-specific struc­ this configuration is not a realistic option, the OFS ture. We opted fo r a small amount of overhead, includes the VM layer. which would allow us to add a fa ster file system, The internal architecture of the VM is more com­ should one become available. Our performance plicated than its functionality might indicate. The measurements showed a loss of less than 10 percent VM consists of a relatively small kernel component in performance as compared to that of an ODF­ and several server processes, as illustrated in Figure specific solution. The cache file systems on mag­ 4. The kernel component is a pseudo-device driver netic disk can be accessed only through the ODF layer that receives requests for the volume devices, kernel component. Thus, in an active OFS system, e.g., /dev/WOR.JVI_A, and translates these requests no application can access and, therefore, possibly into physical device driver (!dev/worm) requests corrupt the cached data. using a table that contains the locations of loaded volumes. Ifthe location of a volume can be found in the table, the 1/0 request is directly passed on to the Vo lume Ma nager physical device. Otherwise, a message is prepared In addition to hiding the WORM nature of the under­ for the central VM server process, and the volume lying physical devices, the OFS transparently moves server and the requesting application are put in a volumes between drives and storage slots in juke­ waiting state. boxes that contain many volumes ("platters"). The The volume server uses a file to translate volume VM performs this fu nction. device numbers into volume names and locations. The essential characteristic of the volume man­ It communicates with two other types ofVM server agement layer is its simple functionality, which processes: jukebox servers and drive servers. The is best described as a "volume faulting device." jukebox servers take care of all movements in The interface to the VM consists of volume device their jukebox. Drive servers spin up and spin down entries, each of which gives access to a specific their drive only on request from the volume server. WORM volume in the system. For example, the vol­ ume device entry /dev/WORM_A gives access to the WORM volume WORJVI_A. This volume device entry Storage Manager has exactly the same interface as the usual device The storage manager implements containers, as entry such as /dev/worm, which gives access to mentioned in the Electronic Document Manage­ a specific WOR.J\1 drive in the system, or rather ment section. Large-scale document management to any volume that happens to be on that drive at uses indexing of multiple storage and retrieval that moment. Any access to a volume device, e.g., attributes, typically with the help of a relational /dev/WORM_A, either passes directly to the drive on database. Once the contents of a document are which the volume (WOR.!vi_A) is loaded, or results in identified through a database query on its attri­ a volume fa ult. This last situation occurs when the butes, a single pointer to the contents is sufficient.

Digital Te chnical journ al Vo l. 5 No. 2 Sp ring 1993 45 Multimedia

VOLUME MANAGEMENT

STORAGE MANAGER OR OTHER UNIX APPLICATION I UNIX USER SPACE

I UNIX KERNEL SPACE VOLUME MANAGER I KERNEL I COMPONENT

Figure 4 Global Architecture Showing the VM Component

Also, there is little need for a hierarchically struc­ first generation was based on delivery of source of tured file system. Containers provide large, fl at example applications. However, tracking source structures where the contents of a document are changes appeared to be too big of an issue and ham­ uniquely defined by the container identification pered the introduction of new base fu nctionality. and a unique identification within the container. In cooperation with European sales organi­ TI1e document's contents identification is translated zations, we formulated a Jist of requirements fo r a by the storage manager in a path to a directory where second-generation IAF. The framework must one or more contents files can be written. For multi­ 1. AJlow for standard applications. Standard appli­ page image documents, the .Megadoc system stores cations, i.e., scan, index, store, and retrieve, cover each page as a separate image file in a directory a wide range of customer requirements in folder reserved fo r the document. This scheme guarantees management. Ta iloring standard applications locality of reference, avoiding unnatural delays can be accomplished in one day, without pro­ while browsing a multipage image document. gramming effort. A container consists of a sequence of file sys­ tems, typically spanning multiple volumes. Due to 2. Be usable in system integration projects. The the nature of the OFS, no distinction has to be made IAF must provide AP!s for fo lder management, between WOR.ivl disk file systems and magnetic disk al lowing the field to build applications with file systems. The storage manager fills containers fu nctionality beyond the standard applications sequentially, up to a configurable threshold fo r by reusing parts of the standard applications. each file system, allowing some degree of local 3. Al low image enabling of existing applications. updates (e.g., adding an image page to an existing RetrievAl! should aUow tbe linkage of electronic document). As soon as a container becomes fuJ I, a image documents and fo lders with entities, such new file system can be added. as order number or invoice number, in existing Containers in a system are network-level applications. Existing applications need not resources. A name server holds container locations. be dunged and run on the image workstation Relocation of the volume set of a container to using a terminal emulator running at the image another jukebox, e.g., fo r load balancing, is possible workstation. through system management utility programs and can be achieved without changing any application's 4. Accommodate internationalization. All text pre­ indexing database. sented by the application to the end user should be in the native language of the user. RetrievAl! RetrievAll-The MegadocImage should support more than one language simulta­ Application Framework neously for multilingual countries. Early Megadoc configurations required extensive 5. Al low upgrading. A new fl.mctional release of system integration work. RetrievAl! is the second­ RetrievAl! should have no effect on the customer­ generation image application framework (IAF). The specific part of the application.

46 Vo l. 5 No. 2 5jJ ring 1993 Digital Te ciJuicaljo urual Th e Megadoc Image Document Managernent Sy stem

6. Provide document routing. After scanning the • Container management documents, RetrievAl[ should route references • Security, i.e., user and group permissions to new image documents to the in-trays of users

who need to take action on the new documents. • Logging and auditing

• Installation, customization, tailoring. and local­ Im age Documents in ization Th ei1· Production Cy cle Image documents start as hard-copy pages that arrive in a mail room, where the pages are prepared Ar·ch itecture and Overview fo r scanning. Paper cl ips and staples are removed, As illustrated in Figure 5, the RetrievAl I image appli­ and the pages are sorted, for example, per depart­ cation framework consists of a number of modu les. ment. An image batch contains the sorted stacks of Each module is a separate program that performs a pages. The scanning application identifies batches specific function, e.g., scanning or document index­ by a set of attributes. The scanning process offers ing. Each modu le has an API to control its function­ a wide variety of options, including scanning one ality, and some modules have an end-user interface. page or multiple pages, accepting or rejecting the Modules can act as building bricks under a control scanned image fo r image quality control, batch module. For example, an image document capture importing from a scanning subsystem, browsing application uses through scanned pages, and controlling scanner J. Scan handling, to let an end user scan pages into settings. a batch. The indexing process regroups image pages of an image batch into multipage image documents. Each 2. Scanner settings, to allow the user to set and document is identified with a set of configurable select the settings for a scanner. The user can attributes and optional ly stored in one or more save specific settings fo r later reference. fo lders. Folders also carry a configmable set of 3. Batch handling, to allow the end user to create, attributes. On the basis of the attribute values, the change, and delete batches. document contents are stored in the document's storage location (container). These three modules can operate together under Many users of RetrievAl l appl ications use the the control of the scan control module and in this retrieve functions of the application only to way fo rm a document capture appl ication. The retrieve stored fo lders and documents. Folders and scan control module can, under control of a main documents can be retrieved by specifying some of module, perform the document capture fu nction the attributes. RetrievAl ! allows the configuration in a fo lder management application. of query fo rms that represent different views on the Modules communicate by means of dynamic data indexing database. The re sult of a query is a list of exchange (DOE) interfaces prov ided in the documents or fo lders. For documents, the opera­ Microsoft Wi ndows environment. Each module, tions are view, edit, delete, print, show fo lder, and except the main module, can act as a server, and all put in folder. The Megadoc editor is used to view modules can act as clients in a DOE communication. and to manipulate the pages of the document including adding new pages by scanning or import­ Main Module Any RetrievAl! application has a ing. For fo lders, the operations are I ist documents, main module that controls the activation of major delete, and change attributes. functions of the application. These fu nctions include scanning pages into batches, identifying Document Routing Applications pages from batches into multipage image docu­ A RetrievAll routing application is an extension to a ments and assigning documents to fo lders, ancl fo lder management application. A route defines retrieving documents and folders. The main mod­ how a reference to a folder travels along in-trays of ule presents a menu to se.lecta major fu nction. The users or work groups. main module activates the control modules of the major functions in an asynchronous way. For exam­ Sy stems Ma nagement ple, the main module can activate a second major The fo llowing systems management fu nctions sup­ fu nction, e.g., retrieve, when the first major fu nc­ port the RetrievAl I package: tion, e.g., identification, is still active.

Digital Te chllical jo urnal lkil. 5 No . .! Sp riiiU 199) 47 Multimedia

Figure 5 RetrievAl! Module Overview

Control Modules Each major RetrievAl! fu nction • The existing application is controlled by a termi­ has a control module that can run as a separate nal emulator program running in the Microsoft application. For example, when a PC acts as a scan Windows environment. This terminal emulator workstation, it is not necessary to offe r all the func­ program must have programming facilities with tionality by means of the main module. Control DDE fu nctions. modules can be activated as a server through the • While entering a new order into the system, the DDE API with the main module as client or as a pro­ image document representing the order is on gram item from a Microsoft Windows program the screen. The function to include the image group. can be mapped on a function key of the emula­ tor. Pressing the function key results in a DDE Server Modules All modules, with the exception request to the identification fu nction of the of the main module, act as DOE server modules. RetrievAl! components. This DDE request passes the identification of the document (as known in Configuration files hold environment data fo r the order entry application) to the identification each module. An application configuration file fu nction. describes which modules are in the configuration. The layout of the configuration files is the same as the WlN.INI file used by the Microsoft Windows Summary software, allowing the reuse of standard access This paper has provided an overview of the many functions. components and discipl ines needed to build an effective image document management system. We Making an Application discussed the details of the WORN! file system, the An appl ication can be made by selecting certain storage manager technology, and the image applica­ modules. Figure 5 gives an overview of the modules tion framework. Other aspects such as WORJ\1 used for the standard fo lder management appl ica­ peripheral technology, software compression and tion. The installation program, which is part of the decompression of images, and the integration of standard applications, copies the appropriate mod­ fa csimile and optical character recognition tech­ ules to the target system and creates the configura­ nologies were not covered. tion files. From experience, we know that different cus­ Modules can also be used with applications other tomers have different requirements for image docu­ than the standard ones. Image enabling an existing ment management systems. The same experience, (i.e., legacy) application (see Figure 6), such as an however, taught us to discover certain patterns order entry application where the scanned images of in customer applications; we captured these pat­ the orders should be included, entails the following: terns in the application framework. The resulting

48 Vo l. 5 No. 2 Sp ring 1993 Digital Te chnical jo urnal Th e Megadoc image Document Ma nagement c�ystem

ENCAPSULATED LEGACY APPLICATION

Figure 6 image Enabling a Legacy Application framework allows us to build highly customized Reference applications with low system integration cost and short time to deployment. Future directions are in 1. M. Bach, Th e Design of the Unix Op erating .�ys­ the area of enhanced fo lder management and inte­ tem, ISBN 0-13-201757-1 (Englewood CJ iffs , NJ: grated distributed work flows. Prentice-Hall, 1986).

Digital Te chuicnl jo ur11nl Vo l. 5 No. 2 Sp ring 199.) 49 Mark F. Riley james] Feenan,j1: joh n L. ja nosik,jr. T.K. Rengarajan

Th e Design of Multimedia Object Support in DEC Rdb

Storing multimedia objects in a relational database ojfe rs aduantages Ol'er fi le

sy stem storage. Digital's relational da tabase software product DEC Rdb supports the storing and indexing of multimedia objects-text, still fr ame images, compound documents, audio, video, and any binary large object. After evaluating the existing

DEC Rdb version 3.1fo r its ability to insert, jetcb, and process multimedia data, soft­ ware designers decided to modify many parts of Rdb and to use write-once op tical disks config ured in standalone drive or jukebox configurations. Enhancements were made to the buff er manager and page allocation algorithms, thus reducing

wasted disk sp ace. Performance and capacity fi eld tests indicate that JJLCRdb can sustain a 200-kilobyte-per-second SQL fe tch throughput and a 57. 7- kilobyte-per­ second SQL!Seruices fe tch throughput, insert and fe tch a 2-gigabyte object, and build a 50-gigabyte database.

To accommodate the increasing demand fo r com­ database fe atures, such as transaction processing, puter storage and indexing of multimedia objects, locking, recovery, and concurrent and consistent Digital supports multimedia objects in its DEC Rdb access, are essential to the successhd operation of relational database software product. This paper numerous businesses. Electronic banking, credit discusses the improvements over version 3.1 and card, airl ine reservation, and hospital information presents details of the new features and algorithms systems all rely on these features to query, main­ that were developed fo r version 4.1 and are used in tain, and sustain business records. version 5.1 . This advanced technology makes the However, although a business might have its DEC Rdb commercial database product a precursor numbers and characters organized, control led, and of sophisticated database management systems. managed in a computer database, maintaining the Multimedia objects, such as large amounts of paper and film storage media associated with text, still frame images, compound documents, and database records can be costly, both in dollars and digitized audio and video, are becoming standard in human resources. Some estimates place the data types in computer applications. Devices that worldwide data storage business at $40 billion, and scan paper, i.e., facsimile machines, are inexpensive as much as 95 percent of the information is stored and ubiquitous. Devices that capture and play back on either paper or film. Curre ntly, busines:-;essuch full-motion video and audio are just beginning to as insu rance, banking, engineering, and medicine reach the mass market. Capturing these objects fo r depend on human beings to manage the filing and use within a computer results in many large data retrieval of these extensive paper and fi lm archives. files. For example, one minute of digitized and com­ Human error can resu lt in the Joss of paper and pressed standard TV-quality video requires approxi­ film. Clearly, scanning the paper, storing the infor­ mately 50 megabytes (M13)of storage' mation in a computer, and making this info rmation To date, relational databases have been used available over computer networks is a better way successfully in storing, io<.Jexing, and retrieving to manage paper records. This scheme allows coded numbers and characters. Relational algebra (1) multiple copies to be distributed at once; (2) a is an effective tool fo r reorganizing queries to customer file to be electronically located and reduce the number of records, e.g., from 1 million retrieved in seconds, whereas to materialize a to 70 records, that an application program must paper fo lder can rake days; and (3) properly search to obtain the desired information. Other programmed computers to maintain these types

50 Vol. 5 No. .! .\j!ri11� I'J')i Digital Te cbuica/ ]o urunl The Design of Multimedia Object Suppo1·t in DEC Rdb

of information more efficiently and accurately than locking mechanism to prevent writers and read­ humans can. ers from interfering with one another in a gen­ The idea of eliminating paper-based storage of eral transaction scheme. However, a file system business records in favor of computer storage is does offer locks to prevent readers and writers long-standing. However, only recently have techni­ from simultaneous file access. cal developments made it practical to consider cap­ • The database guarantees, assuming that proper turing, storing, and indexing large quantities of backup and maintenance procedures are fo l­ multimedia objects. Storage robots based on mag­ lowed, that no information is lost as a result of netic tape or optical disk can be configured in media or machine fa ilure. Al l transactions com­ tl1e range of multiple terabytes (TB) at the low cost mitted by the database are guaranteed. A file sys­ of cents per MB. Central processors based on 45 tem can be restored only up to the last backup, reduced instruction sets are getting fast enough to and any files created between the last backup process multimedia objects without having to rely and the system fa ilure are lost. on digital signal. coprocessors. Processor main memory can be configured in gigabytes (GB). In the sections that fo llow, we present (1) the Document management systems, which have results of an evaluation of DEC Rdb version 3.1 fo r thrived over the past few years, deliver computer its ability to insert, fe tch, and process multimedia scanning, indexing, storage, and retrieval across objects; (2) a discussion of the impact of optical local area networks. storage technology on multimedia object storage; Until now, most multimed ia objects have been and (3 ) design considerations for optical disk sup­ stored in files. Document management systems port, transaction recovery, journaling, the physical generally use commercial relational database tech­ database, language, and large object data storage nology to store the documents' index and attribute and transfer. The paper concludes with the results information, where one attribute is the physical of DEC Rdb performance tests. location of the file. This approach has several disad­ Evaluation of DEC Rdbas a considerable custom software must be vantages: Multimedia Object Storage Sy stem written and maintained to make the system appear logically as one database; application programs Given the premise that production systems need to must be written against these proprietary software store multimedia objects, as well as numbers and SQL interfaces; a system based on both files and a rela­ characters, in databases, the Multimedia engi­ DEC tional database is difficult to manage; two backup­ neering team members evaluated the fo llowing and-restore procedures must be learned and Rdb features to determine if the product could sup­ applied; and complications in the recovery process port the storage and retrieval of mu ltimedia can occur, if the database and file system backups objects: are executed independently. • Large object read and write performance Notwithstanding these disadvantages, storing multimedia objects in a relational database offers • Maximum large object size several advantages over file system storage. • Maximum physical capacity available fo r staring large multimedia objects • Coding an application against one standard interface structured query language (SQL) to The DEC Rdb product has always supported a store object attribute data as well as multimedia large object data type called segmented strings, objects is easier than coding against both SQL to also known as binary large objects (BLOBs). The evo­ manage attribute clata and a file system to store lution from support fo r BLOBs to a multimedia the multimedia object. database capability was logical and straightfor­ ward. In fact, the DEC Rdb version 1.0 developers • The database requires only one tool to back up envisioned the use of the segmented string data and monitor data storage rather than two to type fo r storing text and images in the database. maintain the database and the file system. In evaluating DEC Rd b version 3.1 , we came to a • The database guarantees that concurrent users variety of conclusions about the existing support see a consistent view of stored info rmation. In fo r storing and retrieving multimedia objects. contrast to a file system, a database provides a Descriptions of the major findings fo llow.

Digital Te chuical jo urnal Vo l. 5 No. 2 Sp ring 1993 51 Multimedia

The DEC Rdb SQL, which is compliant with the audio translates into 8.7 years of video. However, as standards of the American National Standards mentioned previously, the maximum size and the Institute (ANSI) and the International Organization total number of objects that can be stored are lim­ fo r Standardization (ISO), and SQL/Services, which ited. As part of system testing, we successfu lly is client-server software that enables desktop com­ stored and retrieved a 2-GB object in a DEC Rdb data pu ters to access DEC Rdb databases across the net­ field. work, did not support the segmented string data DEC Rdb uses a database key to reference individ­ type. Note that the most recent SQL92 standard ual segments stored in database pages. A BLOB does not support any standard large object mecha­ belongs to only one column of one row of a rela­ nisms i Object-oriented relational database exten­ tion. The database key value that locates the first sions are expected to be part of the emerging SQL3 segment is stored in the column of a table defined standard 2 to represent the BLOB data type. DEC Rdb imple­ The total physical capacity fo r storing large ments segmented strings as singly I inked lists of objects and fo r mapping tabular data to physical segments. Therefore, version 3.1 must read a seg­ storage devices is insufficient. Al l segmented string ment in order to find the next segment. This pro­ objects have to be stored in only one storage area in cess has two disadvantages: (1) random positioning the database. This specification severely restricts with a BLOB data stream is extremely slow, and (2) the maximum size of a multimedia database and BLOB pages cannot be prefetched asynchronously. thus impacts performance. One cannot store a large Figure 1 illustrates a DEC Rdb version 3.1 singly number of X-rays or one-hour videos on a 2- to 3-GB linked list segmented string implementation. disk or storage area. Contention for the disk would BLOB data transfer performance of DEC Rdb ver­ come from any attempt to access multimedia sion 3.1 was promising. We were able to code a load objects, regard less of the table in which they are test that sustained 65 kilobytes per second (kB/s); a stored. Although multiple discrete disks can be fetch test sustained 125 kB;s. To put these measure­ bound into one OpenVMS volume set, thereby ments in perspective, DEC Rdb is capable of insert­ increasing the maximum capacity, data integrity ing more than one A4-size (210 millimeters [mm] would be uncertain. Losing any disk of the volume by 297 mm, i.e., approximately 8.25 by 11.75 inches) would result in the loss of the entire volume set. scanned piece of paper per second and capable of The maximum size of the database that DEC Rdb fe tching more than two A4-size pieces of paper per can support is 65,535 storage areas, where each area second. The test was conducted by writing and can span 2:12 - 1 pages. That translates to 256 tera­ reading 50-kB memory data buffers to and from 12 pages (i.e., 256 X 10 pages) or 128 petabytes (PB) magnetic storage areas defined by the DEC Rdb soft­ ' (i.e ., 128 X 101 bytes). At a penny per megabyte, a ware. This experiment ignores the overhead of net­ 128-petabyre storage system would cost 1.28 billion work delays and compression. dollars' DEC Rdb version 3.1 can write multiple copies The largest BLOB that DEC Rdh can maintain is 275 of BLOBs, one to the target database storage area 12 TB (i.e. , 275 X 10 bytes). A data storage rate of and one to each of the database journal files. The 1 megabyte per second (MB/s) fo r motion video and journal files provide fo r transaction recovery and

DATABASE KEY LOCATES FIRST PAGE OF BLOB • POINTER BLOB TO BLOB PAGE 1 PAGE 2 POINTER BLOB TO BLOB PAGE 2 PAGE N f-- BLOB 0 PAGE N

Figure 1 Rdb Ve rsion 3. 1 Singzy Linked List Segmented String Implementation

52 Vo l. 5 No. 2 .Spring 1993 Digital Te e/mica/ jo urua/ The Design of Multimedia Object Support in DEC Rdb

system fa ilures, such as disk drive fa ilures. Database high, so 1 TB is equivalent to a stack of paper 80,000 journal files tend to be bottlenecks, because every inches or 6,667 fe et or 1.25 miles (2 kilometers) data transaction is recorded in the journal. high! The total volume of paper is 160 cubic yards Therefore, writing large objects to journal files dra­ (122 cubic meters). A 1-TB optical disk jukebox is matically impacts both the size of the journal file about 3 to 4 cubic yards (2.3 to 3 cubic meters). and the I/0 to the journal file. Assuming TV-quality video, 1 TB can store 308 The volume of storage required fo r most modest hours or approximately 12 days of video. Full­ multimedia appl ications can be measured in tera­ motion video archives suitable fo r use in the broad­ bytes. A magnetic disk storage system 1 TB in size cast industry require petabytes of mass storage. is expensive to purchase and maintain. An alterna­ The gap between affordable and practical config­ tive storage device that provided the capacity at a urations of optical disk jukeboxes and magnetic much lower cost was required. We investigated the disk farms has closed considerably since late 1992. possibility of using Digital's RV20 write-once opti­ Juxtaposing equal amounts (700 GB) of magnetic cal disk drive and the RV64 optical library ("juke­ and optical storage, including storage device inter­ box") system based on the RV20 drives. We quickly connects, installation, and interface software, rejected this solution because the optical disk reveals that magnetic disk storage is about five drives were interfaced to the Q-bus and UNIBUS times more expensive than optical storage. The hardware as tape devices. Since relational databases major disadvantage of optical jukebox storage is use tape devices fo r backup purposes only and not data retrieval latency related to platter exchanges. fo r direct storage of user data, these devices were This latency, which is approximately 15 seconds, not suitable. Note that physically realizing and varies with the jukebox load and how data is maintaining a large data store is a problem for both mapped to different platters. file systems and relational databases. Mass storage technology, including device inter­ DEC Rdb version 3.1 does not support large connects, combines different classes of storage capacity write once, read many (WORJ\1) devices, devices into storage hierarchies. Storage manage­ which are suitable fo r storing large multimedia ment software continues to be a challenging aspect objects. Version 3.1 has no optical jukebox support of large multimedia databases. either. To provide 1 TB of mass storage capacity fo r rela­ tional database multimedia objects at reasonable Storage Te chnologyIm pact cost, we conducted a review of third-party optical disk subsystems, hardware, and device drivers for When we evaluated DEC Rdb version 3.1, a 1-TB mag­ VA X computers running the OpenVMS operating netic disk farm was orders of magnitude more system. A characterization of the available optical expensive than optical storage. Large format 12- or disk subsystems revealed three basic technical alter­ 14-inch (i.e., 30.5- or 35.6-centimeter) WORM opti­ natives. cal disks have a capacity of 6 to 10 GB. The WORM drives support removable media. These drives can 1. Low-level device drivers provided by the drive be configured in a jukebox, where a robot transfers and jukebox manufacturers. platters between storage slots and drives. A fu lly 2. Hardware and software that model the entire loaded optical jukebox, which includes optical disk capacity of an optical disk jukebox as one large drives and a fu ll set of optical disk platters, of virtual address space. approximately 1-TB capacity costs about $400,000, i.e., $0.40 per MB. By comparison, Digital's RA81 3. Write-once optical disk drives interfaced as stan­ magnetic disk drive, fo r example, has a capacity dard updatable magnetic disks. The overwrite of 500 MB and costs $ 20,000. Thus, to store 1 TB of capability is provided at either the driver or the data would require 2,000 RA81 disk drives at a total file-system level, where overwritten blocks are cost of $40 million, i.e., $40.00 per MB! revectored to new blocks on the disk. For exam­ How big is one terabyte? Assume, conservatively, ple, consider a file of 100 blocks created as a sin­ that a standard business letter scanned and com­ gle extent on a WORM device. When requested to pressed results in an object that is 50 kB in size. rewrite blocks 50 and 51 , the WORM file system Therefore, 1 TB can store 20 million business let­ writes the new blocks onto the end of alJ blocks ters, i.e., 40,000 reams of paper at 500 sheets per written. The system also writes a new file header ream. A ream is approximately 2 inches (51 mm) that contains three file extents: blocks 0 to 49

Digital Te chnical journal Vo l. 5 No. 2 Sp ring 1993 53 Multimedia

stored in the original extent; blocks 50 to 51 • Maintain comparable performance, allowing fo r stored in the new extent; and blocks 52 to 100 hardware diffe rences between optical and mag­ stored as the third extent. Obviously, files that netic devices are updated frequently are not candidates fo r DEC WOR!YJ storage. However, immutable objects, Rdb uses the optical disk file system to cre­ such as digitized X-rays, bank checks, and health­ ate, extend, delete, and close database storage files WO RM benefit authorization fo rms, are ideal candidates on devices. Although this approach uses the block revectoring logic in the optical disk file sys­ fo r WORM storage devices. tem, minimal space is wasted. When writing blocks As a result of this investigation, we decided that to WORM devices, DEC Rdb explicitly knows that using write-once optical devices, interfaced as stan­ blocks can be written only once and bypasses the dard disk devices, was the best solution to provide revectoring logic in the optical disk fi le system. optical storage fo r multimedia object storage. This Nonetheless, DEC Rdb software could waste functionality is being met with commercially avail­ space in two major ways. First, when DEC Rd b cre­ able optical disk file and device drivers. ates a storage area on an erasable medium (e.g., In rhe fu ture, WORM devices may be superseded a magnetic or erasable optical disk), the database by erasable optical or magnetic disks. However, pages are initialized to contain a standard page fo r­ experts expect that WORM devices, like microfilm, mat, with page numbers, area IDs, checksums, etc. will continue to be useful fo r legal purposes. Preinitialized database pages help to determine cor­ rupted database pages. However, preinitializing Design Considerations database pages on write-once media makes little sense. The second way in which DEC Rdb could The tamperproof nature of WO RM devices is an waste write-once optical disk pages is to use sror­ asset bur causes special problems in database sys­ age allocation bit maps fo r space management tem design. The evaluation of DEC Rdb version 3.1 (SPAJ\11). SPAM pages are used to keep track of free indicated that several fe atures needed to be added and used pages. As records are a deled to and deleted to the DEC Rdb product to make it a viable multime­ from the database, the SPA.M bit maps are constantly dia repository. This section describes the design of updated. SPAM pages are maintained within each the new multimedia fe atures included in DEC Rdb database fi le. With write-once devices, a page can versions 4.1 through 5.1. be usee! only once. Again, it makes no sense to update SPAM pages fo r write-once media. Mass Storage To el iminate needlessly wasting space on write­ WORM DEC Rdb version 4.1 supports WORM optical disks once media, DEC Rdb does not preinitialize WORM configured in standalone drive or jukebox configu­ pages. As a general rule, areas should not DEC rations. DEC Rdb permits database columns that contain any updatable data structures. Rdb WORM contain multimedia objects to be stored or mapped maintains storage space allocation in the to either erasable (magnetic or optical disk) or database root fi le. The database root file should write-once (optical disk) areas. The write-once always reside on a magnetic disk, because the root characteristic can be set and reset to permit the file is frequently updated and magnetic disks yield migration of the data to erasable devices. No higher performance. The clusterwicte object man­ changes to application programs are required to ager mechanism ensures that the pointer to the end use write-once optical disks, including jukeboxes. of the written area is consistent across a cluster. SPAM The main design goals fo r WO RN! area support pages, although disabled fo r write-once were to areas, are in fact allocated anyway. The reason fo r allocating SPAM pages in a write-once area is to • Reduce wasted optical disk space by taking into provide the ability to migrate the contents of the account the write-once nature of WORi\11 devices storage area to an erasable device. The SPAM pages simply need to be rebuilt to reflect the space uti­ • Not introduce DEC Rdb application program­ ) ization at the point of conversion. ming changes fo r WORi\1 areas This write-once characteristic was the basis fo r • Maintain the atomicity, consistency, isolation, several enhancements to the buffer manager and and durability (ACID) propenies of transactions page allocation algorithms. Given that a free WORM fo r WORJ\11 devices page has never been written to, the buffe r manager

54 Vo l. 5 No. 2 .\jJring 1993 Digital Te chuical jo urual Th e Design of Jl!Iultimedia Object Support in DEC Rdb

simply materializes an initialized buffe r in main cases where a transaction fa ils and the transaction memory fo r write operations without having to has written data to a write-once area, DEC Rdb first read the page from disk. In the case of page employs a logical undo operation. This operation allocation fo r magnetic disks, DEC Rdb must scan de-references the database key that points to the SPAM pages in search of enough free storage space BLOB data written as part of the failed transaction. to satisfy a write operation. The scanning algorithm An example helps to il lustrate how the logical undo is much simpler fo r write-once areas; to store new operation works. records, DEC Rd b allocates one more page at the 1. Consider row R of table T, which contains a col­ end of the written portion of the area to a process. umn defined as data type BLOB. DEC Rdb maintains such allocated pages in a queue cal led the marked WORM page queue on a per­ 2. The BLOB storage map indicates that the large process basis. Whenever a WOIZLVI page is written objects are stored in a write-once area. WORM to disk. that page is taken off the marked 3. A process starts a transaction and updates the page queue. An attempt to store a record checks row storing a BLOB in the write-once area. the queue before allocating new WORM pages to 4 storage. Facil ities exist to allocate many WO IU

Digital Te chnical jo urnal l·bl. 5 No. 2 Sp ring J'J'J.) 55 Multimedia

]o urnaling Design Co nsiderations recorded in the AlJ file . Since they can never be

An effective database management system guar­ erased, WOR,\·1 opti cal disks might be the best antees the recovery of a database to a consistent devices to write an object (or a journal file) to. Even state in the event of a major system fa ilure, such though a jukebox can misfeed and permanently as media failure. Hence, fu ll and incremental back­ da m age the media, disb in a jukebox can be disk ups must be perfo rmed at regular intervals, and shadowed. The trade-off is doubling the l/0 versus the database must record or keep a journal file of risking data integrity. Rather than legislate a policy, transactions that occur between backups. In DEC: DEC Rclb permits applications to disable Al.f Jogging Rdb, the after image journal (Al.f) file records all fo r BLOBs, thus transferring the risk to individual transactions against the database since the last applications. backup. Also, to recover from a system fa ilure. the database must keep track of all outstanding or Database Physical Design Co nsiderations pending transactions. The recovery unit journal The original design of segmented strings speci fied (IUJ.I) file records the state and data associated with a singly linked list, where the segments were al.l pending transactions. written one at a time . as shown in Figure 1. \V hen Journal files are heavily uti! ized in a database writing a new segment, the previous segment management system. Contention h>r the journal. had to be updated with a pointer value that identi­ fi les comes from every process that is updating fied the location of the new segment. For example, the database. To be completely recoverable, the to store a BLOB wi th two segments Rl and R2. database management system must record 13!.0!1 the old algorithm stored R 1, stored R2, and then data, as weli as alphanumeric data, to both the Al.f modified R I to point to R2. Although this algo rithm and the RUJ files. Because multimedia objects are docs not v.:aste srace on a m agnet ic disk, ir does large, eliminating the need to write these objects to waste space on write-once optical disk. Segment the journalfiles is desi.rable. The double-write trans­ Rl must he rewritten to disk with a poi nter to action negatively impacts the perfo rmance of the segment R2. application storing the object and taxes the journa l If we impose the depe n den cy between the two file, one of the most burdened resources in the stores that R2 mu:-;L be stored h ·fore Rl. the store database. dep enden cy fo r BLOBS becomes a reverse order As discussed in the Transaction H.ecovery sec­ of segments. Storing segments in reverse order tion, DEC Rdb uses logical undo operations to undo requires buffning all segments of a multimedia aborted transactions. In addition to the minimal object . \.Vherc:ts buffi: ri ng the entire object in m a in processing required to de-reference a database key memory may be feasible fo r small multimedia pointing to the WORM area pages, Dt:C Rd b automat­ objects, main memory is not large enough to bu ffer ically disables RUJ log writes fo r WO RM a ret records. audio and video data objects. The singly linked This is another advantage of using WORM (levices Jist method that DEC Rcl b used prior to ve rsion •t .l fo r multimedia objects. is not \veil suited for WORi\·1 devices. Therefore, we Record ing multimedia objects in the AIJ fi l.e is redesigned the fi >rmat of BLOU in WO fL\•1 areas to not so straightforward . DEC Rdb uses the AU file el i m i nate the need to buffer l :lrge amounts of dara. fo r media recovery, as we ll as fo r transact ion The new design replaces th• singly I inked I ist recovery. By definition. keeping a media recoven· with III.OB segment pointer arrays and ULOB data journal fo rces twice the number of 1/0 operations, segments. The segment pointer array maintains each to a separate device. DEC Rdb must write a list of database keys that locate each segment, in the multimed ia object to t he storage area desig­ order, fo r a BLOI\, as illustrated in Fig ure 2. Because nated fo r the multimedia object and write a copy of segment pointer arrays are stored as a singly linked the object to the AU file. If the primary storage list, the pointer arrays can become large. device that contains the object fa ils, the database Application data is stored in BLOB data segmcnt s. administrator can apply the last full bacJ,;:up of The new met hod butTe rs and writcs the BLOB seg­ the storage area, fo llowed by any subsequent incre­ ment pointers to disk after ass igning the segmented mental backups, and roll fo rward through the string to a record . AI.J journal fi le to recover the data. If a multi­ llesides cl iminating the waste problem fo r write­ media database is to be com pletely recoverable once devices, the segmen t pointer array has other and consistent, then multimedia objects must be advant:tgcs. DEC Rdh reads the pointer array i n to

\'r1/.. 'i ,\"(). .! .\jJ1"i11g !1)')3 Digitai Teclmical journal The Design of Multimedia Object Support in DEC Rdb

DATABASE KEY LOCATES FIRST PAGE OF BLOB THAT CONTAINS POINTER ARRAY LOCATING THE OTHER BLOB PAGES + DATABASE KEY POINTER TO SEGMENT 1 DATABASE KEY POINTER TO SEGMENT 2 DATABASE KEY BLOB POINTER TO SEGMENT 3 PAGE 2 BLOB � PAGE 3 I DATABASE KEY I I POINTER TO SEGMENT N BLOB ARRAY TERMINATOR I PAGE N I

i'lgure 2 Rdb Ve rsion 4.2 Po inter Array Segmented String implementation

Create storage map BLOB_M AP memory when an application accesses a BLOB. DEC Store Lists Rdb can, therefore, quickly and randomly address in RESUME_A REA any segment in the BLOB. Also, DEC Rdb can begin for (PLACEM ENT_HISTORY, to load segments into main memory before the CANDIDATES.RESUME) PHOTO_AREA application requests them. This feature benefits in for (CANDIDATES .PICTURE) applications that sequentially access an object, in RDB$SYSTEM; such as playing a video game. This code directs the BLOB data from the table Sto rage Map Enhancem entsfo r BLOBs PLACEMENT_HISTORY and the column RESUME of Designers addressed several issues related to stor­ the table CANDIDATES to be stored in the area age mapping. The major problems solved involved RESUME_AREA and the BLOB column PICTU RE of capacity and system management, jukebox perfor­ the table CAl'IDIDATES to be stored in the area mance, and the fa ilover of fu ll volumes. PHOTO_AREA. The remaining BLOB (lata in the database is stored in the default RDB$SYSTEM area. Capaci�y and .'>),stem ,J.Ia nagement DEC Rd b can Restricting the storage of all BLOBs across the map user d:Jta, represented logically as tables, rows, entire database schema to a single fi le or database ami columns, into multiple fi les or storage areas. area was clearly undesirable. The size of the area Besides increasing the amount of data that can would be limited to the largest file that could be be stored in the database, spreading data across created by the OpenVMS operating system and the multiple devices reduces contention fo r disks and mass storage devices available. The limited map­ improves performance. However, as mentioned in ping of one BLOB area mapped to one disk the section Evaluation of DEC Rdb as a Multimedia can be circumvented by using the OpenVMS sys­ Data Storage Sys tem, prior to DEC Rd b ve rsion 4.1, tem's Bound Volume Set mechanism. This mecha· only one storage area could be used fo r storing nism allows n discrete disks to be bound into one BLOB data. Al l BLOB co lumns in the database were logical disk. DEC Rdb can then create a single stor­ implicitly mapped into the single area, which age area on the logical disk that spans the bound severely limited the maximum amount of multi­ set of disks. media data that could be stored in DEC Rdb. However, although the volume set mechanism Prior to new multimedia support fo r BLOBs, DEC solves the problem of limited area mapping, serious Rdb restricted the direct storage of a particular limitations exist in the database system administra­ table column to one DEC Rdb storage area (i .e., file). tion and recovery processes. All database-related This partitioning control is accomplished by means facilities operate at the granularity of a database of the DEC RcJb storage map mechanism, as shown storage area. Thus, if one disk in a 10-disk volume in the fol lowing code example: set is defective, DEC Rdb would have to restore all

Digilal Te chnical jo urunl Vo l. 'i 11. .2 .�j; riug l')')i ')7 Multimedia

10 disks. Not only does restoring data on fu nction­ stored in the three specified areas EMP_PHOT0 _1, ing disks waste processing time, but during the EMP_PHOT0_2, and EMP_PHOT0_3. restore operation, applications are stal led fo r access The ability to define multiple BLOB storage areas at the area level. This situation introduces concur­ and to bind discrete areas into a storage set elimi­ rency problems for on-1 ine system operations. nates the BLOB storage capacity limitation in DEC DEC Rdb version 4.1 and successive versions Rd b. Consider the storage problem of storing 1 MB solve the capacity problem by (1) permitting the of medical X-rays as part of a patient record. Prior to 4.1 , definition of multiple BLOB storage areas, (2) bind­ DEC Rub version the I imited one-BLOB storage ing discrete storage areas into storage area sets, and area could store approximately 2,000 X-rays on a (3) providing the ability to map or to vertically 2-GB disk device. The features included in ve rsion partition individua l BLOB columns to areas or area 4.1 allow the creation of a DEC Rdb storage area set sets. Applications can set aside a disk or a set of that spans multiple disk devices. Also, adding stor­ disks fo r storing employee photographs, X-rays, age areas or disks to a storage area set can expand video, etc. The alphanumeric data and indexes the capacity initially defined fo r the column. can be stored in separate areas as wel l. Figure 3 depicts the employee photograph column being jukebox Pe rformance Problems When a storage mapped to the EMP_PHO T0_ 1, EMP_PHOT0_2, and area set is defined using the SQL storage map state­ EMP_PHO T0_3 storage area set. Al l alphanumeric ment, DEC Rdb implements a random algo rithm data in the table EMPLOYEES is assumed to be to select a discrete area or disk from the set to store mapped to storage area A. the next object. Since multiple processes access Coding this example resu lts in multimedia objects across the entire set, a random Create storage map BLOB_M AP algorithm that evenly distributes data across the Store Lists disks in the area set reduces contention fo r any in (EMP_ PHOT0_1 ,EMP_PHOT0_2 , EMP_P HOT0_3 ) one disk. for (EMPLOYEES . PHOTOGRAPH J Using a random algorithm to select fro m a set in RDB$S YSTEM; of platters in a jukebox is extremely inefficit:nt. A jukebox comprises one to five disk drives with '50 This code directs the BLOB data, i.e., the col umn to 150 shelf slots where optical disk media is stored . PHOTOGRAPH from the table EMPLOYEES, to be A storage robot exchanges optical disk platters between drives and storage s.lots. As described ear­

TABLE: EMPLOYEES lier, a full platter exchange-spin down the platter currently in the drive, eject the platter, insert a new NAME ADDRESS ... PHOTOGRAPH platter, spin up the new platter-takes approxi­ DICK 456 IMAGE OBJECT mately 15 seconds. Each optical disk surface, i.e., FRED 123 IMAGE OBJECT side of a platter, is modeled as a discrete disk to the OpenVMS operating system. Consider, fo r example, ten storage areas defined on optical disks in the jukebox and mapped into a storage area set. Al l MARY 789 IMAGE OBJECT \ I patient X-rays from a single table in the database arc \ ALPHANUMERIC DATA / ' BLOBS MAPPED TO MAPPED to be stored in this area set. Each new X-ray inserted \ ,' \ 1 ' SEPARATE 1 'TO AREA 1 in the database causes DEC Rclb to randomly select a \ STORAGE ' \ SET / AREA , disk surface in the jukebox, which probably results \ ' \ I \ I , \ I \ I \I in a platter exchange. Consequently, each X-ray \ I ' \ I insertion takes 1'5 seconds' \ I ROB STORAGE \ I \ I EMP _PHOTO _1 The solution to the jukebox performance prob­ STORAGE lem was not to eliminate random storage area selec­ ROB STORAGE ROB STORAGE AREA AREA A EMP_PHOT0 _2 tion, which works successfully with fL"ed-spindle SET devices. Rather, the solution was to accommodate ROB STORAGE EMP_pHO T0_3 an alternate algorithm that sequentially fi lled the disks in an area set. Using DEC Rdb, applications can specify random or sequential loading of storage F(�ure 3 DEC Rdb BLOB Storage Area Sets area sets as part of the srorage map statement.

58 lkJI. ) No . .2 .)jJr ing I

Contention fo r a single optical disk in a jukebox is Language Design Considera tions a far more desirable situation, with respect to SQL, the ISO/ANSI standard relational database latency, than causing one platter exchange per structured query language, is well suited to object stored. expressing queries against alphanumeric data When multiple users simultaneously issue yet hard ly begins to address the needs of multi­ requests to read multimedia objects stored in a media objects. Putting aside the fact that sampled jukebox, long delays occur, whether the storage data (i.e., a scanned image) is more difficult to area is loaded sequentially or randomly. Us ing a query than coded data (e.g., text coded in ASCII), transaction monitor to serialize access to the SQL cannot provide data compression and ren­ database helps eJ iminate jukebox thrashing and dition capabilities fo r multimedia objects. improve the aggregate performance of the database Multimedia object processing is better suited to engine. a language like Cor C ++. Ideally, SQL wou ld sup­ port the ability to define objects and ro associate Fa ilooer of Full Vo lumes The introduction of methods with those objects. SQL3 is a new version storage area sets gave rise to another problem: of the SQL standard that the standards organizations What happens when one area in the set becomes are just beginning to work on. SQJ.3 contains the fu ll' Normally, within the DEC Rdb environment, mechanism to define abstract data types and to exe­ disk errors that result from trying to exceed the cute external procedures as part of SQL statements. allocated disk space are signaled to the application However, SQL3 will not become a standard fo r fo ur so that the transaction can be rol led back (dis­ to five years. carded). When related to storage area sets, how­ As discussed previously, DEC Rdb SQL lacks ever, the error is just an indication that a portion of support fo r the segmented string or BLOB data the disk space allocated to the column has been type that was available in the Rdb relational engine. exhausted and that processing should continue. A new DEC Rclb SQL data type, LIST OF BYTE Also, since multimedia objects tend to be exceed­ VA RYlNG, was designed based on the native Rdb ingly large, great amounts of data may have already segmented string data type. The data access mecha­ exhausted cache memory and been written back to nism for the LIST OF BYTE VA RYING data type is the WO RM media, even though the database trans­ a list cursor. which operates I ike a table cursor­ action has not committed. Hand ling such an error open the cursor, fetch segments of a BLOB, and by signaling to the application and expecting the close the cursor. This new data type with asso­ app.lication to roll back ancl retry the transaction ciated access mechanism was also added to would result in the waste of a large number of SQL!Services. SQL/Services software enables remote device blocks that have already been burned. Thus, clients on a network, such as personal com­ DEC Rdb had to implement a new scheme. puters, to attach to remote DEC Rdb databases. DEC Rdb now implements full failover of an area The ability to scroll or to randomly position the within the area ser. Thus, when an area becomes list cursor allows positioning at a particular data fu ll. DEC Rdb traps the error, selects a new area in segment within the multimedia object stream with­ the set, and writes the remaining portion of the out having to physically read through tl1e entire BLOB being written to the new area. This area data stream. fa ilover works whether the storage allocation is Although appl ications can program directly to random or sequential. In addition, the area that list cursors, this interface was cumbersome and did is now fu ll is marked with the attribute of full, and not offer any object typing or processing. The list the clusterwide object manager of DEC Rdb main­ cursor mechanism does not present the straightfor­ tains this attribute consistently throughout the ward byte- stream interface that is common in most cluster. Consequently, writers to the database wil I file systems. Applications want to store objects, consider the area unavailable for future BLOB store such as images and compound documents, not operations. Further, the DEC Rdb database manage­ BLOBs. Data compression was another important ment utilities can remove the attribute ifadditi onal consideration. Multimedia objects should be com­ space is made available to the database area (e .g., if pressed on the client side of the network; then, DEC Rdb moves BLOBs fro m area A to another copy compressed bits are transferred through the net­ of area A that resides on a device with twice the work, servers, and disks. The objects should be capacity). clecompressed when they are to be rendered fo r

Digitctl Te clmicaljournal Vo l. 5 No. 2 Sp ring l':J9) 59 Multimedia

display. Finally, the enormous size of multimedia SQI. Multimedia handles file l/0 operations objects saturates main memory resoHrces on per­ across many diffe rent software environments, sonal computers, so application developers must including the MS-DOS, Windows, Macintosh, use disk storage to buffer as well as persistently ULfRIX, and OpenYMS operating systems. SQL store multimedia objects. Multimedia preserves fi le attributes on insert oper­ YA R'\.1 N

• A file arguments permit the specification of image

60 Vol. 5 No. 2 Sp ring I'.J'J.i Digital Te chnical jo urnal Th e Design of ivlultimedia Object Support in DEC Rctb

process steps and parameters. The DAS toolkit magnetic disks, 6 RA92 magnetic disks, and 2 ES£20 supports Comite Consultatif Internationale de solid-state disks. The total mass storage available Telegraphiquc et Telephonique (CC!TT) compres­ fo r building databases was 10 c;B. We evaluated sion (a ubiqu itous compression standard fo r fac­ the SQL performance of DEC Rdb ve rsion 4.2 Field simile machines) fo r bitonal images and Jo int Test I (FTl) and SQL Multimedia version 1.0 Field Photographic Experts Group (JPEG) compression Test 2 (FT2), and generated the SQL/Services remote (an ISO/ANSI standard) for multispectural images. client data fe tch and insert performance data fo r To improve appl ication performance, SQL DEC Rclb version 4.1 Field Test 4 and SQL .Multimedia

Multimedia can generate multiple rendered ver­ version 1.0 FT2. sions of an image that are stored in a single database This performance data should be used as a gu ide­ field. Therefore, a user can store the original image, line, because the field-test software contained retaining its fidel ity, and also store a miniature implementation errors that affected perfo rmance version of the image fo r fast access or browsing pur­ but were corrected in the released products. As pre­ poses. For example, consider a personnel applica­ sented in Ta ble 1, using the released version of DEC tion where 90 percent of the fe tches fo r employee Rd b, we are able to sustain a 300-kB/s throughput photographs are to be displayed in a passport-size from a magnetic disk DEC Rdb storage area, across fo rmat on an employee information form. If an Ethernet network, to a DECstation 5240 work­ the capture portion of the application stored the station. This test demonstrates fetching a software original employee photograph and directed SQL motion pictures (SMP) video clip out of the data­ Multimedia to generate and store a passport-size base fo r display on an 'LILTRIX-based workstation.1 rendered version in addition to the original, at fe tch Although the video was sampled at 15 frames per time, the 1/0 operations required to transmit the second, we can play back the video clip at 20 image to the employee fo rm wou ld be reduced. frames per second! The performance measured fo r Storing multiple rendered versions would also elim­ an SQL/Services fe tch was 57.7 kB/s, as shown in inate using CPU time to scale the !'e tched image. Ta ble 2. We expect to conduct similar performance tests on a DEC 7000 AXP processor. Sy stem Te sting and Evaluation The performance test inserted and fe tched 50-kB records. Fifty kilobytes is a conservative estimate of After the multimedia engineering of the DEC Rdb a compressed A4-size piece of paper, probably the product was complete, we conducted several test­ ing activities to determine the performance and most prevalent object to be stored in multimedia databases. For both the distributed SQL/Services capacity boundaries. The performance work pre­ sented is not complete but is otkred as an indica­ client and the local SQL interface, 50-kB main mem­ tion of the multimedia object access capabilities of ory buffe rs were the sources and destinations fo r the DEC Rdb software. the inserts and fe tches. In the debit credit domain, the Tra nsaction We bu ilt several 50-MB databases, varying data­ Processing Performance Council ('fPC) tests pro­ base design parameters such as page and buffe r vide a standard procedure to measure the perfor­ sizes, to determine the fastest set of parameters mance of one database as compared to another. fo r the large object performance test. Using the However, no standard multimedia database per­ largest page and buffe r sizes yielded the best perfor­ fo rmance tests exist. The performance of a DEC mance. The database table was organized into three Rdb multimedia database is influenced by many columns: two key columns and a BLOB column. The variables, including the processor, mass storage BLOB column was mapped to a storage area set con­ medium, database design, object sizes, and work­ sisting of mu.ltiple magnetic storage disks. load. The performance data presented in this paper After we establ ished the best database organiza­ should be used only as a guide. tion, we built many 3- to 10-GB databases by

• Va rying the number of processes executing Performance Te sting insert and fe tch operations For performance testing we used a VAX 6 360 pro­ • Varying the number of tables in the database cessor (relatively slow by today's standards) config­ ured with 12H MB of main memory, an HSC50 • Va rying the number of inserts and fe tches per storage interconnect processor with 16 RA70 transaction

Digital Te e/mica(jo urnal Vo l. 5 No. 1 .)j; rinr; 1993 61 Multimedia

Ta ble 1 SOL Performance

SOL Insert Performance

Number of Processes Performing Insert Number of Number of Inserts Throughput Operations Ta bles per Tra nsaction AIJ (kB/s)

No 83.0 10 No 103.4 1 1 Yes 48.0 1 10 Yes 55.9 3 3 32 No 295.3 6 6 32 No 533.7 10 10 32 No 601 .5

SOL Fetch Performance

Number of Processes Performing Fetch Number of Number of Fetches Throughput Operations Ta bles per Tra nsaction AIJ (kB/s)

10 No 194.0 No 184.0 Ye s 181.0 10 Yes 192.5

Ta ble 2 SOL/Services Performance

SOL/Services Insert Performance

Number of Processes Performing Insert Number of Number of Inserts Throughput Operations Ta bles per Tra nsaction AIJ (kB/s)

1 1024 No 44.0 4 4 32 No 91.9

SOL/Services Fetch Performance

Number of Processes Performing Fetch Number of Number of Fetches Throughput Operations Ta bles per Transaction AIJ (kB/s)

1024 No 57.7 4 4 32 No 142.3

• Enabling and disabling Al.f journaling objects from a single table, and a more complicated update test, where multiple writers are simultane­ • Inserting and fetching from an SQL/Services ously updating one table, have yet to be fa bricated client or using SQL fo r local database access and run. When we conducted the performance tests, the To put some of the performance results pre· computer was dedicated to our task; no other activ­ sen ted in Ta ble 1 into perspective: the tested config­ ity was taking place. A simple contention test, uration can sustain approximately 600 kB of insert where multiple readers simultaneously fe tch bandwidth, which translates into twelve 50-kB

Vo l. 5 No. 2 .)jm:ng f'J'Ji Digital Te chnical journal Th e Design of Multimedia Object Support in DEC Rdb

A4-size pieces of paper per second. Even a single media object types (i.e., text, sti II images, com­ process scanning paper at 103.4 kB/s can keep up pound documents, audio, and video). Re lational with some of the fastest paper scanners available. database systems must expand mass storage device Also, scanning both sides of a compressed bank support, database physical database design, lan­ check (scanned at 200 dots per square inch) resu lts guage fu nctionality, and performance to manage in an object size of about 20 kB. Therefore, the par­ the variety of today's information. The development ticular configuration we tested could store 30 of this advanced technology in Digital's DEC Rdb checks per second with multiple processes, and product provides desktop computer-to-optical 6 checks per second with a single process. disk jukebox integration by means of a commercial database. As multimedia technology matures, data­ Capaci�)J Te sting bases must address the need to store and index We conducted two capacity tests. The first stored information beyond numbers and characters. and fe tched a 2-GB object in a DEC Rdb field, and the The work accomplished to support multimedia second built a 50-GB database. A 2-GB known pat­ objects in DEC Rdb is just "the tip of the iceberg." tern was generated in virtual memory. DEC Rdb Current multimedia capabilities are able to success­ wrote this object, with no AIJ, to a field in an empty fully manage the majority of document and still database. The BLOB column was mapped to three frame applications. However, improvement in disks, totaling 2.5 GB of storage. To avoid having to capacity and performance are required before the sustain storage area or file extensions, the storage database can serve multiple channels of video and area set was defined to be 2.3 GB. DEC Rd b was able audio data. As the SQL standard evolves to incorpo­ to successfully insert and fe tch the 2-GI:l object. rate a more object-oriented mechanism, much of To demonstrate the capacity that could be the SQL Multimedia functionality will migrate to achieved with SQL Multimedia, DEC Rub, and opti­ using standard interfaces to define, operate on, and cal storage, we built a 50-GB database. The hard­ query abstract data types. ware configuration consisted of the fo llowing: Acknowledgments • A VA X 4000 Model 500, with 6 Cdi of magnetic A large number of people from various disciplines disk and 12R MB of main memory contributed to the success of this multimedia • A Kodak Automated Disk Library Model 6800, database project, including Becky Jacobs, Michael with 100 GB of storage (with a maximum capac­ Sawyer, John Lacey, Cheri Jones, Bruce Mills, Steve ity of 1.2 TB) Hagan, Ian Smith, Susan Hillson, Peter Spiro, ). M. Smith, Jim Gray, Dave Lomet, Rudy Downs, Ken • DEC Rdb version 4.2 Field Test 0 Cross (Perceptics), Chris Eastland, Mase Merchant, • SQL Multimedia version 1.0 FT2 Scott Matsumoto, Paul Carmen (Eastman Kodak), Jim Lewis (Eastman Kodak), and Marilyn Gulli.ksen. • Perceptics LaserStar optical disk software

Starting with a backup of a 2-GB manufacturing References database that was used by Digital's Mass Storage 1. American Na tional Sta ndard fo r Informa­ Group, DEC Rc.l b added an SQL Multimedia column tion Sy stems-Database Language-SQL, ANSI to a table that contained over 550,000 rows. DEC X3.135-1992 (New Yo rk, NY: American National Rdb then mapped the column to five platters, mod­ Standards Institute, 1992) and eled as ten 9.5 -million-block (5.1-GB) magnetic disks to the OpenVMS operating system, using the Information Te chnology-Database Language­ sequential load algorithm. An update table cursor SQL, ISO/IEC 9075 : 1992 (Geneva: International was devised that returned between 2,000 to 3,000 Organization fo r Standardization, 1992). rows. Using SQL Multimedia, DEC Rdb inserted 2. ). Melton, ed ., Database Language SQL (S QL3), images representing the disk assembly process ISO/ANSI Wo rking Draft, ANSI X3H2-93-091 and until the storage was fu ll. ISO/JEC JTC 1/SC21/WG3/DBL YOK- 003 (February Conclusion 1993). The multimedia fea tures that have been ac.lc.lec.l to 3. B. Neidecker-Lutz and R. Ulichney, ··software Rdb are in direct support of the increasing demand Motion Pictures," Digital Te ch nical jo urnal, fo r computer data storage and indexing of multi- vol. 5, no. 2 (Spring 1993, this issue): 19-27

Digital Te chnical jo urnal Vo l. 5 No. 2 �jJ ring /'}') ) 63 Multimedia

GeneralRe ferences WORM.Dev ices SQL Extensions D. Maier, "Using Write-Once Memory fo r Database Storage," Proceedings of the ACtH S!GMOD/SIGACr K. Meyer-Wegener, V Lum, and C. Wu , "Image Conference on Principles of Database Sys tems Management in a Multimedia Database System," (P ODS) (1982). Proceedings of the IF! P TC 2/WG 2. 6 Wo rk ing Con­ fe rence on Vis ual Database Sy stems, To kyo, Japan S. Cbristodoulakis et al, ''Optical Mass Storage (1989): 497- 523 Systems and Their Performance." IEEE Database Engineering (March 19H8). M. Stonebreaker, "The Design of the POSTGRESS Storage System," Proceedings of the 13th Interna­ S. Christodoulakis anu D. Ford, "Retrieval Perfor­ tional Conference on Ve ry Lmge Databases, mance Ve rsus Disk Space Utilization on WO RM Brighton, UK. (1987): 289-300 Optical Disks," Proceedings of the ACM SIGMOD International Conference on Ma nagement of M. Stonebreaker and L. Rowe, The POSTGRESS Data, OR (l9H9) 306-314 Papers, Memorandum No. lJCB/ERL J\'186/85 (Berke­ Portland. ley, CA: University of California, 1986). Storage Managem.entfo r Large Objects Object Storage Management A. Biliris, "The Performance of Three DatabasL· M. Stonebreaker. "Persistent Objects in a Multi­ Storage Structures fo r Managing Large Objects," Level Store," Proceedings of the ACM SIGMOD Inter­ Proceedings of the ACM S!GJ'v!OD International national Conference on tVlcmagement of Data, Conference on Management of Data, San Diego, Denver, CO (1991): 2-ll. CA (1992): 276-285

64 Vol. 5 No. 2 .\)Jriug I'J'J.i Digital Te ch nical jo urnal Lawrence G. Palmer RickyS. Palmer

DECspin: A Ne tworked Desktop Vi deoconferencing Application

The Sound Picture Information Ne tworks (SPIN) technology that is part of the DECsp in version 1. 0 product takes digitized audio and video fr om desktop comput­ ers and distributes this data over a network to fo rm real-time conferences. SPIN uses standard local and wide area data networks, adjusting to the various latency and bandwidth differences, and does not require a dedicated bandwidth allocation. A high-level SPIN protocol was developed to sy nchronize audio and video data and thus alleviate network congestion. SPIN performance on Digital's hardware and software platforms results in sound and pictures suitable fo r carrying on personal communications over a data network. The Society of Te chnical Communication chose the DECspin version 1. 0 application as a fi rst-place recipient of the Distinguished Te chnical Communication Award in 1992.

In late 1990, we began to design a software product Introduction to Conferencing that would allow people to see and hear one When the SPIN project started, standalone telecon­ another from their desktop computers. The result­ fe rencing products were available but not fo r desk­ ing DECspin version 1.0 application takes digitized top computers. Typically, the products offered audio and video data from two to eight desktops cost as much as $150,000, required scheduled con­ and distributes this data over a network to fo rm ference rooms and operators, and needed leased real-time conferences. The product name rep­ telephone lines. These systems did not operate as resents the fo ur major communication elements part of a corporate computer data network but that unite into one cohesive desktop applica­ instead required dedicated, switched 56-kilobit­ tion, namely, sound, picture, information, and per-second (kb/s), T1 (1.5-megabit-per-second networks. The overall technology is referred to as [Mb/s]), and T3 (45-Mb/s) public telephone compo­ SPIN. This paper first presents an introduction to nents in order to operate. Originally designed conferencing and gives a brief overview of the as two-way conference units, these teleconferenc­ framework on which SPIN was developed. The ing products later included hardware to multiplex paper then details SPIN's graphical user interface. several equally equipped systems. In addition, Although the high-level protocol (which is the the enhanced systems included custom logic to application layer of the International Organization implement a hardware compressor/decompressor fo r Standardization/Open Systems Interconnection (codec) that reduced digital video data rates suffi­ [ISO/OSI] model) that SPIN uses to synchronize ciently to use leased telephone lines. distributed audio and video is proprietary, a gen­ During the last several years, other conferencing eral discussion of how SPIN uses standard data systems have been demonstrated. The Pandora networks fo r conferencing is presented. Perfor­ research project by Olivetti Research resulted in mance data fo r DECspin version 1.0 running on an excellent desk-to-desk conferencing system. a DECstation 5000 Model 200 workstation with Although the Pandora system was expensive per DECvideo and DECaudio hardware fo llows the dis­ user and did not use existing network protocols, it cussion of network considerations. Finally, the did prove the viability of using a digital conferenc­ paper summarizes the fu ture direction of desktop ing system from one's office and demonstrated the conferencing. natural progression from room conferencing to

Digital Te cbllicaljournal Vu l. 5 No. 2 .SjJring 1993 65 Multimedia

office conferencing. This system served as a good area networks' Video help documentation, textual example for our own emerging desktop model, help, and audio help are used on the desktop to DECspin version 1.0. communicate how the application works. So und, Throughout this same period, several compres­ picture, graphics, and network elements are all sion standards suitable fo r video capture and woven together to provide better co mmunication playback have evolved and been implemented. The among conference participants. Joint Photographic Experts Group (]PEG) industry­ Early in 1991, we received our first prototype of standard algorithm results in intraframe compres­ the DECvicleo TURBOchannel frame bu ffer, which sion of frames of high-quality video (on the order of included the necessary hardware to input and cap­ 25 to 1). 1•2 This algorithm is well su ited fo r either ture an analog video signal, to digitize the signal, single-frame capture or motion-frame capture of and to display the pixel information on the screen. video information. This fo rm of compression is The fra me buffer was special in that it displayed most appropriate fo r real-time video capture ami 8-bit pseudocolor, 8-bit gray-scale, and 24-bit true­ playback where low (i.e., frame-by-frame) latency color graphical data simultaneously. This feature is required. al lowed captured video data to be displayed with­ The Motion Picture Experts Group (MPEG) stan­ out data dithering. dard resu lts in interframe compression of motion Dithering is the process of converting each pixel video.5 This algorithm is wel l suited fo r motion­ of video data to a fo rm that matches a limited frame capture of video because only the di.fferences number of available colormap entries. Most work­ between successive frames are stored. Interframe station frame buffers are 8-bit pseudocolor. Hence, compression is appropriate fo r video capture and digitized, 24-bit true-color video data fo r display playback where real-time low latency is not would need pixel-by-pi...'Cel conversion. Algorithms required . exist that could be used to accomplish this conver­ The H.26I standard results in interframe com­ sion. However, a better SPIN conference, in terms pression of motion video that is most responsive to of frame rate and picture quality, was achieved by the demands placed on capturing live video fo r dis­ performing no software dithering, thus relying semination over low-bandwidth public telephone on the ability of the DECvideo hardware to display networks.4 This compression is su itable fo r video 24-bit true-color video or 8-bit gray-scale video.' In capture and playback with reasonable latency but is addition, the DECvideo hardware could scale clown not quite real-time in nature. H.26I is the standard the incoming video image in real time so that fewer used most in the teleconferencing systems on the pixels (i.e., less data) represented the original market today. image. Finally, the last few years have also witnessed Concurrently, SPIN used a DECaudio TURBOchannel the emergence of dramatic new base computer and card that could sample an input analog audio signal network technologies. Reduced instruction set from a microphone and deliver an 8-kilohertz digi­ computer (RJSC)-based workstations supply the tized audio bit stream. The DECaudio hardware needed processing power and 1/0 bandwidth to could also convert a digital audio stream fo r output process large and continuous amounts of data, and to an analog speaker or external amplifier. A fiber distributed data interface (FDDI) technology DECstation 5000 Model 200 with DECaudio and results in 100-megabit-per-second local area net­ DECvideo components provided the core hardware works fo r the desktop. Consequently, the SPIN capability used in SPIN development work. development project got under way to provide a In addition to these new hardware capabilities, novel and innovative software appl ication that the SPfN effort needed new underlying base soft­ could take advantage of the powerful new systems ware capabilities. The DECvideo hardware required and networks. the Xv video extension to the X Window System to allow fo r the display and capture of video data. (The Overview of Un derlying Xv extension was jointly developed by base system Hardware and Software graphics and MIT Project Athena teams.) The We came up with the SPIN project in response to DECaudio component used the AudioFile audio the question: How can we communicate easily server, developed by Digital's Cambridge Research with graphics, video, and audio on the desktop Laboratory, to capture and play hack digital audio as well as over both local and wide geographical data.

66 Vo l. 5 No. 1 Sp ring 1993 Digital 1e chll icaljourllal DECspin: A Ne tworked Desktop Vi deoconjerencing Application

A prototype software base was created to make activates a specific fu nction of the application and fu ndamental measurements of video and audio data is shown in Figure 1. manipulation within the workstation and over a In the main window, the first button from the left network. Te sting the prototype over a 100-Mb/s contains a green circle with a vertical white bar, the FDDI network and a 10-Mb/s Ethernet network international symbol for exit. This button appears demonstrated that a conferencing product running in the same location in each of the pop-up win­ over existing network protocols was possible. dows. It is used to exit the window or, in the main window, to exit the application. Th e SPIN Application The second button from the left is labeled with the communication icon. This button is used to SPIN is a graphical multimedia communications select the call list shown in Figure 2. The call list tool that allows two to eight people to sit at their contains the various buttons and widgets used to desktop computers and communicate both visually place a call to another user, to create and play back and audibly over a standard computer data net­ SPIN files, and to display a I ist of received SPIN mes­ work. The user interface employs a telephone-like sages, if any exist. The list provides a way to play "push" model that allows a user to place an audio­ and manage audio-video answering machine mes­ only, video-only, or audio-video call to another sages. For example, to place a call to another user desktop computer user. Here, the term "push" on the network requires just three steps. means that SPIN conference participants control all aspects of the digitized data they send onto a net­ 1. Enter the computer network name of the work. Thus, users can fe el confident about the secu­ machine and user into SPIN's phone database as rity of their audio and video information. A caller "user@desktop:' A string representing some­ initiates all calls to other users, and a call recipient thing more understandable to a novice is also must agree to accept an incoming SPIN call. Because allowed, e.g. , "[email protected]" becomes all data is in the digital domain, this model makes it "user@desktop l.dec.com Firstname Lastname at almost impossible to use SPIN to eavesdrop on Digital Equipment Corporation." another person. Placing a wiretap on a person's call would involve intercepting network packets, sepa­ 2. Select whether the call is to be sound only, rating data from protocol layers, and then reassem­ picture only, or both. The toggle push buttons under the large note icon control audio select; bling data into meaningful information. If the network data were encrypted, interception would those under the large eye icon control video select. Once the call is established, these but­ be impossible. SPIN also provides other communi­ tons can be set or unset by clicking a mouse or cation services, such as an audio-video answering machine, messaging, audio-video file creation, using a touch-screen monitor and are useful audio help, and audio-video documentation. for muting the audio portion or freezing frames Figure 1 shows a screen capture of a SPIN session in of the video portion. progress, using the DECspin version 1.0 application. 3. To establish a two-way network connection, The product is easy to learn and to use. The press the call push button under the connection graphical user interface is implemented on top of icon (which is labeled with two arrows going in Motif software. Motif provides the framework fo r opposite directions) that appears next to the the SPIN international user interface. A model was desired call recipient. If the person called is chosen in which all actions taken by a user are logged on, a ring dialog box appears on the implemented by push buttons that activate pop-up call recipient's screen and a bell rings. If the call menus. The SPIN application does not use pull­ recipient is not available, a dialog box appears on down menus, because they require language­ the caller's screen asking whether the caller specific text strings to identify the purpose of an wishes to leave a message. The caller can then entry and thus require translation for different choose to leave a message or not. countries. Also, pull-down menus are intended fo r short-term interaction, and SPIN menus usually Depending on the individual settings, users can require more long-term interaction. All push­ see and hear one another in multiple windows button icons are pictorial representations of the on the screen. To connect all conference partici­ intended fu nction. For example, the main window pants in a mesh, press the "join" push button, has a row of five push buttons, each of which which has a triangular icon.

Digital Te ch11icaljournal Vo l. 5 No. 2 SjJ ri11g 1993 67 Multimedia

user@desktop2

30.0 user@desktop3 user

[i] Demonstration v

Figure 1 Sample SPIN Session

68 Vo l. 5 No. 2 .vJring I'J'J.i D(gilal Te chnical jou rual DEC�pin·A Networked Desktop Vi deoconferencing Application

Iilllllillll

DECSpln: Call List JOIN

11-f'JI I�_k- SELECT AUDIO �� SELECT VIDEO

I Liserl®desktopl _j _j D

_j D I user3®desktop3 _j U user4®desktop4 _j I user5®desktop5

.J user6@desktop6

I user7®desktop7 _j c

RECORD --d.J....II A FILE .J I [;u sr/tmp/record.spn PLAY �_1_11 �LOOK AT BACK _j MESSAGES A FILE (_j I /usr/tmp/p lay,spn 11=111

Figure 2 SPIN Call List Pop-up Window

Returningto the main window, the middk push which appear the number of active audio connec­ button is the SPIN control button. As shown in tions and the number of active video connections, Figure 3, the SPIN control pop-up window contains respectively The second slide bar indicates the slide bars that, from top to bottom, allow the caller result of sampling the average outgoing bandwidth to set maximum capture frame rate, hue, color sat­ consumption (measured in Mb/s) of the application uration, brightness, contrast, speaker output vol­ on the network. This measurement is also updated ume, and microphone pickup gain. At the bottom every five seconds. of the control window are buttons fo r selecting Finally, the fifth push button (on the far right) in compression and rendering. the main window is the information button. By To the right of the control button in the main pressing this button and selecting the type of on­ window is the status icon button. Pressing this but­ line information desired, the user can access the ton causes the status pop-up window shown in documentation pop-up windows, as illustrated in Figure 4 to appear. The status window displays, Figure 5. Within each documentation window are below the camera icon, the active size of the cap­ several topics and two columns of toggle push but­ tured video area in pixels. Beneath these dimen­ tons that can be used to obtain either textual docu­ sions is a vertical slide bar that indicates the average mentation or video documentation. The video frames-per-second (frames/s) capture rate sampled documentation comprises short videos that over a five-second intervaL To the right of the contain expert help about the operation of the camera icon is the connection icon, under which application. appears the number of active connections. Below As a final level of help, al l push buttons and wid­ this number are the sound and picture icons, under gets within the application have associated audio

Digital Te chuical jo urnal Vc>l. 5 No. 2 Sp riJJg /'J'J.i 69 Multimedia

D ECspln: CO ntrol

LOW SAT U RATION � llllllll ...... � ...;;:... I _ l ..,;,;o, _ _ _

vi�� A�

R ENDE BLACK- COLOR iiiiliiiiiiilii�AND-WHITE � ftg ure 3 SPIN Control Pop-up Window tracks that tell the user what the buttons and telephone networks often provide the long­ widgets do within their context in the application. distance lines used to make up private wide area To activate the audio tracks, the user must first data networks. select the button or widget and then press the Help The use of data networks allows conferencing key on the keyboard. data to be treated as woul.d any other type of data. SPIN requires no special low-level networking pro­ Ne twork Considerations tocols to transmit its data and uses the transmission SPIN uses standard data networks to transport the control protocol/internet protocol (TCP/IP) or the information that composes a conference. Data net­ DECoct protocol. Also, SPIN requires no changes to works are usually private networks that a user com­ existing operating systems. When performing the munity maintains. Such networks often include a prototype work t() r the SPIN application, we were number of individual networks joined together by not certain whether the real-time nature of confer­ bridges and routers. Unlike public telephone net­ encing could be accomplished on inherently works, which are most frequently used fo r phone non-real-time networks and operating systems. calls, private networks are used for a variety of Consequently, we developed a special. high-layer computer data needs, includ ing file transfers, synchronization conferencing protocol, cal led the remote log.ins, and remote file systems. However, SPIN protocol, that uses existing data networks.

70 Vo l. 5 No. 2 .\jJring 1993 Digital Te chnical jo u-rnal DECspin: A Ne tworked Desktop Vi deoconjerencing Application

EXIT

PICTURE TOTAL NUMBER SIZE IN OF CALLS PIXELS

TOTAL NUMBER OF ACTIVE AUDIO CHANNELS

TOTAL NUMBER OF ACTIVE VIDEO CHANNELS

AVERAGE AVERAGE NETWORK FRAME RATE BANDWIDTH USAGE IN FRAMES IN MEGABITS PER PER SECOND SECOND

Figure 4 SPIN Status Pop -up Window

This protocol is responsible fo r the synchronization Thus, variable video frame rates along with this of audio and video information. The SPIN protocol treatment of audio data allow for the graceful degra­ monitors the flow of data to the network in order dation of a conference as the network becomes to alleviate network congestion when detected. As busy. the network becomes congested, the protocol SPIN has been demonstrated over a variety of makes the decision to withhold further video data, public and private data networks including since video is the largest consumer of network Ethernet (10 Mb/s) , FOOl (100 Mb/s) , Tl (1.5 Mb/s), bandwidth. This withholding of video data is a key T3 (45 Mb/s) , cable television (10 Mb/s, more cor­ feature of the SPIN protocol, because it allows a rectly, Ethernet running over two 6-megahertz conference to vary the video frame rate on a user­ cable television channels), switched multimegabit by-user basis. Thus, video bandwidth can scale to data service (SMDS) (1.5 or 45 Mb/s), asynchronous the lesser of either the bandwidth available or the transfer mode (ATM) (150 Mb/s), and frame relay number of frames/s of video bandwidth that a given (1.5 or 45 Mb/s). Some of these networks are local platform can sustain. or metropolitan area technologies, i.e., local area If the withholding of video corrects the network networks (LANs), whereas others are wide area congestion, video d:ua is once again allowed in the technologies, i.e., wide area networks (WANs), as conference. If not, the SPIN protocol delays audio illustrated in Figure 6. data and stores it in a buffer until the network is Each type of network provides SPIN with diffe r­ able to handle this data. Jf the network outage lasts ent latency and bandwidth characteristics. SPIN approximately 10 seconds, audio data is lost. makes corresponding adjustments to a conference Periods of audio silence are used as a means of to account for these differences and does not recovery from periods of network congestion. require a dedicated bandwidth allocation to carry

Digital Te e/mica/ Jo urnal Vo l. 5 No. 2 Spring 1993 71 Multimedia

SELECT OVERVIEW POP-UP WINDOW

Overview g

Start a Conference SELECT VIDEO v m;] Demonstrati on D DOCUMENTATION v

Introduction Use a Feature v v

v Network v SELECT TEXT Solve a Problem DOCUMENTATION v v Gloss:ary

v Browse

F<� ure 5 SPiN lnformutiun Pop -up Windows

on a conference. If a given net work supports band­ upgraded to rhe DECNIS family of routers, the SPIN width allocation, this fe ature only enhances SPIN's appl ication functioned correctly, even on con­ ability to deliver video and audio inhnmation. gested networks. WA Ns may use a router to interconnect two or To demonstrate daily use of SPIN, we created a more LANs. SPIN has been tested on a number of metropolitan area network (MAN). Figure 7 shows

routers with mixed results, i.e., some routers cor­ the network topology, which spanned the states of rectly hand le SPIN's bic.lirecrional traffic pattern New Hampshire and Massachusetts. The test bed

whereas others do not. Since some n>uters clo not allowed us to demonstrate our FOOl products, correctly handle bid irectional data traffic without including end-station FDDI adapter cards, multi­ packet loss, wide area routers m ust be individually mode FDDI wiring concentrators, and single-mode tested with SPIN to verify proper operariun. Some FUD! wiring concentrators. SPIN was used in :)0 router problems were traced ru tilt" use of old workstations, two of which were attached to large­ firmware or software. Conseque nt]}', SI'IN acted sc reen projection units in con.ferencerooms.

like a diagnostic tool in poimiug out these prob­ Performance lems. For example, running the SPIN application with audio only, across Digital's private II' network, The conference quality achieved when running the yields varied results. Digital's IP network is an exam­ SPIN application depends on many factors. The ple of an open network, with routers from most available network bandwidth, the processor speed, router vendors. We traced must instances of poor the desired frame-rate specification, the compres­ SPIN performance to old or obsolete routers (some sion setting, the picture size, and how the pictures in service fo r the last six years without upgrades). are rendered all affect the quality of the confe rence.

These routers usually dropped packets wht:nro ut­ Ta ble l contains performance data for DECspin ver­ ing between adjacent Ethernet neL wurks that were sion 1.0 at various combinations of settings fo r only 10 percent buS}'· After these ro urers were these factors.

72 Vol. 5 No. .! .\firing 19').) Digital Te cbuical jo untul IJECspin: A Ne tworked Desktop Vi deoconferencing Application

ETHERNET LAN

(a) LAN Usage of SPIN

ETHERNET LAN

------1 DECNIS 600 ETHERNET LAN

( b) WA N Usage of SPIN

Figure 6 LAN and WAN Usage of SPIN

FDDI g (54 MILES, SINGLE-MODE FIBER) ,II II,LITILETON, MA D hhhhhh

Figure 7 Digital'sMAN Te st Bedfo r SPIN

Digital Te chuicaljournal Vo l. 5 No. 2 Sp ring 199.) 73 Multimedia

Ta ble 1 SPIN Performance on a DECstation 5000 Model 200 with DECvideo and DECaudio Hardware

Width X Height Frames/s (Bit (Pixels) Render/Compression Network rate in Mb/s)

256 X 192 Black and White/ No FDDI 10 (4.0) 256 X 192 Black and White/ Yes FDDI 16 (1.5) 256 X 192 Color/ No FDDI 3 (5.0) 256 X 192 Color/ Yes FDDI 10 (4.0) 160 X 120 Black and White/ No FDDI 19 (3.0) 160 X 120 Black and White/ Ye s FDDI 25 (0.9) 160 X 120 Color/ No FDDI 12 (6.0) 160 X 120 Color/ Ye s FDDI 19 (3.0) 256 X 192 Black and White/ No Ethernet 9 (3.8) 256 X 192 Black and White/ Yes Ethernet 16 (1.5) 256 X 192 Color/ No Ethernet 2 (3.0) 256 X 192 Color/ Yes Ethernet 8 (3.0) 160 X 120 Black and White/ No Ethernet 19 (3.0) 160 X 120 Black and White/ Ye s Ethernet 25 (0.9) 160 X 120 Color/ No Ethernet 8 (5.0) 160 X 120 Color/ Yes Ethernet 19 (3.0)

Using a DECNIS Router (Ethernet-to-Router-to-T1-to-Router-to-Ethernet)

256 X 192 Black and White/ No T1 4 (1.4) 256 X 192 Black and White/ Yes T1 15 (1.4) 256 X 192 Color/ No T1 1 (1 .4) 256 X 192 Color/ Yes T1 4 (1 .4) 160 X 120 Black and White/ No T1 10 (1.4) 160 X 120 Black and White/ Yes T1 25 (0.9) 160 X 120 Color/ No T1 3 (1.4) 160 X 120 Color/ Ye s T1 10 (1 .4)

As shown in Ta ble 1, we tested SPIN performance are high-quality video. The best cases in Ta ble I are using two basic picture sizes: 256 by 192 pixels and those that used software compression to del iver a 160 by 120 pixels. The tests were performed over pleasing frame rate with the least amount of net­ both Ethernet and FDDI networks fo r black-and­ work bandwidth consumed together with some white and color cases. Also noted in the table is degradation of individual frame qual ity. The soft­ whether or not software compression was enabled ware compression was tuned to provide nearly the fo r a specific test case. The far right column shows same frame quality as the uncompressed case. the frame rate achieved for the different combina­ Ta ble 1 also shows performance data measured tions and also summarizes the network bandwidth using a DECNIS router. As noted earlier, wide area consumed in each test. The table is presented pri­ usage of SPIN depends on a router with correct algo­ marily to give a sampling of the frame rate and, rithms for handling of bidirectional continuous hence, the level of visual quality achieved for a spe­ stream traffic. The DECNIS family of routers can cific combination of parameters. Frame rates affect supply the fu ll Tl bandwidth when presented with an observer's ability to detect change within a bidirectional SPIN traffic. Other routers on which sequence of frames. With a slow frame rate, the SPIN was tested typically delivered only 25 to 50 resulting video sequence may appear choppy and percent of the Tl bandwidth. Note that this was incomplete, whereas a normal frame rate (24 to 30 only true on the particular routers we tested and frames/s) leads to a smoothly varying video that routers other than DECNIS routers may also be sequence with even continuity from one sequence able to del iver fu ll Tl bandwidth fo r this particular to another. The frame rates in Ta ble 1 below about 6 traffic pattern. to 7 frames/s are considered low quality. Those in Hardware compression technology mentioned in the 8-to-19-frames/s range are considered good the section Overview of Underlying Hardware and quality, and those in the 20-to-30-frames/s range Software reduces the bandwidth requirements for

74 Vv l. 5 No. 2 Sp ring 1993 Digital Te chnical ]Ollr11al DECspin: A Networked Desktop Vi deoconferencing Application

conferencing. Experimentation with motion JPEG very large user community of potential SPIN users compression (using the Xv extension with com­ and dramatically drops the price per user compared pression fu nctions on an Xvideo frame buffe r with the original product. lnreroperabil ity among board) has shown that at a resolution of 320 by 240 platforms and a common user interface give Digital pixels, true-color frames can be used at 15 to 20 a leadership position in this fast-forming market. frames/s at a bit rate of just under 1.0 Mb/s. This bit To day, high-quality conferencing can scale to rate produces a good- to high-quality conference hundreds of seats on a LAN with lower-quality con­ with very low latency. H.261 and MPEG technology ferencing scaling to larger, more geographically dis­ result in similar frame rates and picture size at persed networks. Several fa ctors will lead to the about one-half the bandwidth but higher overall widespread use of this technology: better and less­ latency. Using motion JPEG as the example, high­ expensive hardware, programmable codecs, and quality conferences require about 1 Mb/s per higher-speed and Jess-costly cross-country net­ connection. If all conferences are to be high qual­ works. Less-expensive video hardware allows many ity, this bit rate allows 1 two-party conference users to upgrade their systems to include video, on a Tl connection, 5 two-party conferences on an while programmable compn.'ssion technology Ethernet segment, and 50 two-party conferences allows users to achieve improvements in picture on an FDDI network. Using GIGASWJTCH FDDI quality, compression transcoding, and lower net­ switches, more than 500 two-party conferences work needs. Higher-capacity and less-costly cross­ can take place simultaneously on a network. More country networks allow more users to access users could be supported on Tl, Ethernet, or conferencing services. Ultimately, even homes will GIGASWITCH networks, if lower-quality confer­ have better computer connectivity and bandwidth. ences are acceptable. As these changes occur, and we bel ieve they will, desktop conferencing can become the interactive Conclusion telephone of the twenty-first century. It became clear during the development and deployment of SPIN that high cost per user limits Acknowledgments the widespread use of the application. The cost of The authors wish to acknowledge and thank aU the video fo r DECspin version 1.0 adds about $8,000 to members of the close team that worked on the SPIN the price of a workstation. Audio fo r version 1.0 project and made the DECspin version 1.0 product a adds about $2,000 per workstation. These costs, success. Key individuals in this effort were Diane which are prohibitive to most potential users of LaPointe, Beverly Ol iphant, Jonathan George, the technology, do not include the network cost Garrett Van Siclen, and Jack lbto. We would also impact. like to thank early supporters of the product Digital's Alpha AXP family of computers come efforts, including Jim Miller, Karl Pieper, and Jim with audio input and output hardware as part of the Cocks. In addition, we extend our thanks to Wa lt base workstation. In spring 1993, Digital released to Ronsicki, Videhi Mallela, Nathalie Rounds, and the the Internetcom munity a version of DEC:spin that rest of the team who establ ished the FDDI test bed ; uses this hardware to carry on audio-only confer­ to Dick Bergersen, who handled the quality assur­ ences and shows the user a voice waveform instead ance for DECspin version 1.0 and gave the team of a video image. This version eliminates the add-on excellent feedback on the product ; and to To m hardware cost fo r audioconferencing. A new low­ Levergood and the other members of Digital's cost video option would go far to reduce the add-on Cambridge Research Laboratory who gave us sup­ cost fo r video and facilitate a wider use of the SPIN port and assistance in regard to the AudioFile audio application. server. Finally, we would also like to offe r thanks to The SPIN application and its associated protocol our management, particularly Bil.l Hawe and John have been demonstrated on Digital and non-Digital Morse, who are strong advocates and supporters of computers, operating systems, and networks. In our product efforts. particular, SPIN has been shown on SPARC worksta­ SPIN tions running Solaris software. Additionally, References has been demonstrated on a personal computer using the Microsoft Multimedia Extensions (MME) 1. Digital Co mpression and Coding o. f Continuous­ to Windows software. This platform provides a To ne Still Images, Pa rt I, Requirements and

Digital Te cbTticaljo urnal Vo l. 5 No. 2 Sjn-ing 199.) 75 Multimedia

Guidelines, ISO/IEC JTC 1 Committee Draft 10918-1 de Tetegraphique et Telephonique [CCITT], (Geneva: International Organization fo r Stan­ August 1990). dardization/International Electrochemical Com­ 5. R. Ulichney, "Video Rendering," Digital Te chni­ mission, February 1991). cal}ournal, vol. 5, no. 2 (Spring 1993, this issue): 2. Digital Compression and Coding of Continu­ 9-18. ous-Tone Still /mages, Pa rt 2, Compliance Te st­ GeneralRe ferences ing, ISO/IEC JTCl Committee Draft 10918-2

(Geneva: International Organization fo r Stan­ L. Palmer and R. Palmer, "Desktop Meeting," LAN dardization/International Electrochemical Com­ Magazine, vol. 6, no. 11 (November 1991 ). mission, Summer 1991). TURBOchannel Hardware �p ecification (Palo Alto, 3. Coding of Moving Pictures and Associated CA: Digital Equipment Corporation, TRI/ADD Audio, Committee Draft Standard ISO 11172, Program, 1990). ISO/MPEG 90/ 176 (Geneva: International Organi­ Open Software Foundation, Inc., OSF/Motif, Pro­ zation fo r Standardization, December 1990). grammer's Reference, Release 1. 1 (Englewood Cliffs, N) : Prentice-Hall, Inc., 1991). 4. Vi deo Codec fo r Audiovisual Services at Px 64

Kb/s, CCITT Recommendation H.261 , COM XV-R R. Scheifler, ]. Gettys, and R. Newman, X Window 37-E (Geneva: International Te lecommunica­ Sy stem C Library and Protocol Reference (Bedford,

tions Union, Comite Consultatif Internationale J\1A Digital Press, 1988).

76 vb l. 5 No. 2 Sp ring IY93 Digital Te chn-icaljuurnttl Peter C. Hayden I

LAN Addressing fo r Digital Vi deo Data

Multicast addressing was chosen over the broadcast address and unicast address

mechanismsfo r the transmission of video data over the LAN Dynamicallocation of multicast addresses enables such fe atures as the continuous playback of fu ll motion video over a network with multzple viewers. Design of this video data trans­ mission sy stem permits interested nodes on a LAN to dy namically allocate a single multicast addressfr om a pool of multicast addresses. When the allocated address is no longer needed, it is returned to the pool. Th is mechanism permits nodes to use fewer multicast addresses than are required in a traditional schem e where a unique address is allocated fo r each possible fu nction.

The transmission of digital video data over a local using its computers and data network. The use of area data network (LAN) poses some particular computer equipment installed fo r data transmis­ challenges when multiple stations are viewing the sion would eliminate the need to invest in cable TV material simultaneously. This paper describes the wiring throughout a building. available addressing mechanisms in popular LANs The basic system would consist of two primary and how they alleviate problems associated with components. One computer, or set of computers, multiple viewing. It also describes a general mecha­ would act as a video server by transmitting video

nism by which nodes on a LAN can dynamical ly allo­ program material, in digital form, onto the data net­ cate a single multicast address from a pool of work. Other computers, acting as clients, would multicast addresses, and subsequently use that receive the transmitted video program and present address fo r transmitting a digital video program to a it on the computer's display. Figure I depicts such a set of interested viewers. configuration. The variety of video source material suggests that Project Goals servers may be equipped in several ways. For exam­ The objective of this project was to design a mecha­ ple, accessory hardware can receive broadcast nism suitable for providing the equivalent of broad­ video programs; hardware and software can con­ cast television using computers and a local area vert analog video into digital fo rmat; and hardware data network in place of broadcast stations, air­ and software can compress the digital video fo r effi­ waves, and televisions. The resulting system had to cient use on a personal computer and data net­ provide access to broadcast, closed circuit, and on­ work. 1·2·� Figure 2 shows a server equipped to demand video programs throughout an enterprise handle different types of video program sources.

LOCAL AREA NETWORK

• • TRANSMITTED DATA STREAM 1 I SERVER I I CLIENT I I CLIENT I [CLIENT I TRANSMITTINGI RECEIVINLG NOT RECEl IVING RECEIVINGL DIGITAL VIDEO DIGITAL VIDEO DATA DIGITAL VIDEO

Figure I Client-server Sy stem fo r Vi deo Data Tra nsmission

DiRital Te cbnical]oun�al Vo l. 5 No. 2 Sp rinf:!. 1993 77 Multimedia

LOCAL AREA NETWORK • I NETWORK INTERFACE

I t I MAIN MAGNETIC PROCESSOR DISK STORAGE 0 STORED DIGITAL VIDEO ANALOG t • LASER COMPRESSION VIDEO DIGITIZER DISC 0 HARDWARE SOURCES �I 0 -I I TV OR PERSONAL COMPUTER CABLE OB TUNER

Figure2 Ty pes of Video Prog ram Sources

Video program material is categorized as live, because all stations receive the data whether they e.g., the current program broadcasting on a televi­ are interested or not. Compressed digital video rep­ sion network, or stored and played on demand, resents from I to 2 megabits per second of data; e.g., a recorded training session . In both cases, it is therefore nodes not expecting to receive the video desirable for more than one cl ient to be able to data are impacted by its unsolicited arrival.1•1 As a monitor or view the transmitted video program. fu rther complication, when two or more video pro­ To implement the client-server system described grams are playing simultaneously, stations receive I above, many technical hurdles had to be overcome. to 2 megabits per second or more of data fo r each This paper, however, fo cuses on one narrow but video program. This renders many systems inoper­ critical aspect, the addressing method used on the ative. Furthermore, LAN bridges pass broadcast LAN fo r delivery of the digital video data. The char­ messages between LAN segments and cannot con­ acteristics of digital video and the need fo r multiple fi ne digital video data to a LAN segment H As a result stations to receive programs from a wide range of of these drawbacks, use of the broadcast address is possible sources combined to create some interest­ unsuitable fo r transmission of digital video data. ing challenges in devising a suitable addressing Unicast addressing sends data to one unique method. node. The use of unicast addressing eliminates the problems encountered with broadcast addressing Choosing an Addressing Method by confining receipt of the digital video data to a

1() transmit digital video over a data network, an single node. This approach works quite wel l as long effective addressing mechanism had to be chosen as only one node wishes to view the video program. that would satisfy the project's goa ls. Most LANs ff multiple clients wish to view the same program, support three basic data addressing mecha nisms: then the server has to retransmit the data for each broadcast, unicast, and mu lticast. Lor,; Each method participating client. As the number of viewing of transmitting digital video over a LAN has charac­ clients increases, this approach quickly exhausts teristics that are both attractive and undesirable. the server's capacity and congests the LAN . Because Broadcast addressing uses a special reserved des­ unicast addressing cannot practically support one tination address. By convention, data sent to this server in conjunction with multiple clients, it too is address is received by all nodes on the LAN. unsuitable for transmission of digital video data. Transmitting digital video data to the broadcast Multicast addressing uses addresses designated address serves the purpose of permitting multiple to simultaneously address a group of nodes on a clients to receive the same transmitted video pro­ LAN . Nodes wishing to be part of the addressed gram while permitting the server to transmit the group enable receipt of data addressed to the mu lti­ data once to a single address. Viewed another way, cast address. This characteristic makes multicast this convention is a significant disadvantage addressing the ideal match fo r the simultaneous

7H Vo l. 5 No. :! Sp rin,� 1993 Digital Te clmicaljounwl LAN Addressingfo r Digital Vi deo Data

transmission of digital video data to multiple client A technique, therefore, is needed by which a block nodes without sending it to uninterested nodes. of multicast addresses is permanently allocated fo r Furthermore, many network adapters provide the purpose of transmitting video programs on a hardware-based filtering of multicast addresses, computer network, and individual addresses are which permits high-performance rejection/ dynamically allocated from that block fo r the dura­ selection of data based on the destination multicast tion of a particular video program. address.9 Because of these advantages, multicast addressing was selected as the mechanism fo r trans­ DynamicAlloca tion Me thod mission of digital video data. A dynamic allocation method should have several characteristics to transmit video programs on a Multicast Addressing Considerations LAN. These desired characteristics To ge ther with its advantages, multicast addressing 1. Must be consistent with current allocation pro­ brought significant problems to be overcome. The cedures used by standards bodies like the problems were in the assignment of multicast IEEE addresses to groups of nodes, all of which are inter­ 2. Should be fu lly distributed and not require a ested in the same video program. If a single multi­ central database (improves reliability) cast address were assigned fo r all stations interested in receiving any video program, then 3. Must support multiple clients and multiple only interested stations would receive data. All par­ servers ticipating stations, however, would receive all pro­ 4. Must operate correctly in the face of LAN per­ grams playing at any given time. If multiple turbations like segmentation, merging, server programs were playing, each station would receive fa ilure, and client failure data fo r all programs even though it is interested in the data fo r only one of the programs. The obvious It is clearly desirable to use a dynamic allocation solution is to allocate a unique multicast address fo r mechanism that does not require changes to the each possible program . The fo llowing sections way addresses are allocated by standards commit­ examine various allocation methods. tees. Changes to protocols only create another level of administrative complexity. Instead, a single set of Traditional Address Allocation addresses should be allocated on a permanent basis Traditionally, a standards committee allocates mul­ fo r use in the desired application. Drawn from a ticast addresses, each of which serves a specific pool of addresses, these allocated addresses could purpose or fu nction. For example, a specific multi­ be dynamically assigned to video programs as they cast address is allocated fo r Ethernet end-station are requested fo r playback. When playback was hello messages, and another is allocated fo r fiber complete, the address would be returned to the distributed data interface (FOOl) status reporting pool. frames. to. tt.L2 Each address serves one explicit func­ Regard less of which allocation mechanism is tion. This static allocation breaks down when a adopted, it needs to support multiple servers and large number of uses fo r multicast addresses fa ll multiple clients. This implies that some fo rm of into one category. cooperation exists between the servers to prevent It clearly is not possible to allocate a unique multiple servers from allocating the same address multicast address for all possible video programs fo r two different video programs. One node could fo r several reasons. At any given time, hundreds act as a central clearinghouse fo r the allocation of of broadcast programs are playing throughout addresses from the pool, but the overall operation the world, and thousands of video programs of the system would then be susceptible to fa ilure and clips are stored in video libraries. Countless of that node. The preferred approach is a fu lly dis­ more are being created every minute. Assigning a tributed mechanism that does not require a central­ unique address to each possible video program ized database or clearinghouse. would exhaust the number of available addresses LANs tend to be constantly changing their config­ and be impossible to administer. Furthermore, urations, and nodes can enter and leave a network it would waste multicast addresses since only at any time. As a result, an allocation mechanism those programs currently playing on a given must be able to withstand common and uncommon LAN (or extended LAN) need an assigned address. perturbations in the LAN. It must accommodate

Digital Te chnical jo urnal Vo l. 5 No. 2 Sp ring 1993 79 Mul timedia

events such as the segmentation of a LAN into two fo r allocation of multicast add resses through the LANs when a bridge becomes inactive or discon­ IEEE. All cJ ients and servers participating in this nected, joining of two LANsin to one when a bridge protocol use the same set of addresses. For the sake is installed or becomes reactivated, and fa ilure or of this discussion, these addresses are denoted as disconnection from the LAN at any time by both AI, A2, ..An. Address Al is always used by the par­ server and client nodes. ticipating stations fo r exchange of information nec­ essary to control the allocation of the remaining Other Multicast Allocation Methods addresses for use by the participating stations. The A variety of different group resource allocation remaining addresses A2 through An fo rm the pool mechanisms exist, and the one most nearly appl ica­ of available multicast addresses. ble to transmitting digital video over a LAN is used in the internet protocol (IP) suite. Deering dis­ Server Announcements cusses extensions to the internet protocols to sup­ AI I servers capable of transmitting digital video port multicast delivery of internet data grams. 1' data continuously announce their presence and In his proposal, multicast address selection is algo­ capabilities by transmitting a message at a predeter­ rithmically derived from the multicast IP address mined interval; for example, a message is addressed and yields a many-to-one mapping of multicast to A 1 every second. In these announcements, the IP addresses to LAN multicast address. As a conse­ servers include information identifying their gen­ quence, there is no assurance that any given multi­ eral capabilities, data streams they are currently cast address will be allocated solely fo r the use of transmitting, and data streams they are capable of a single digital video transmission. This undermines transmitting. the goal of using mu lticast addressing to direct the A server's general capabilities include its name heavy flow of data to only those stations wishing to and network adclress(es). Other usef-ul information receive the data. Deering discusses the need fo r can also be announced, but it is not relevant to allocation of transient group address and a.lludes to this discussion. To identify the data streams cur­ the concepts presented in this paper. rently being transmitted, the server describes the data and the multicast address to which each Mo delfo r Dynamically Allocating data stream is being transmitted. In this way, it Multicast Addresses announces those multicast addresses that the sta­ Given the overall goals of the project and the tion is currently using, along with a description of desired characteristics of the application, the fol­ the associated video program. The data streams the lowing model was developed. It transmits digital server is capable of transmitting are identified by video on a data network using dynamically allo­ some fo rm of a description of the data stream. cated multicast addresses. First, simple operational cases on the LAN are described. Then complicated Jde ntzfy ing Servers and scenarios dealing with network misoperations arc Available Programs addressed. With each server continuously announcing the pro­ It should be noted that the protocols described gram material available fo r playback, clients wish­ add ress the location of video program material as ing to receive a particular data stream can monitor we ll as the allocation of multicast addresses fo r the server announcements being sent to add ress delivery of that material. Because of the one-to-one AI. By receiving these announcements, a client can correspondence between video material and ascertain the address of each server active on the address allocation, it is convenient to combine LAN, the data streams currently being transmitted these two functions into a single protocol; how­ by each server and the multicast address to which ever, the focus of this paper re mains on the address each is being transmitted, and the data streams allocation aspects of the protocol. available fo r transmission. With a large repository of program material, Multicast Address Po ol it cou ld easily become impractical to annou nce This model assumes a set of n multicast addresses all ava ilable material. In this case, the announce­ permanently al located and devoted to it. The ments could be used only to locate available addresses are obtained through the normal process servers, and an inquiry protoco.l or database search

HO Vo l. 5 No. 2 Sp ring 1993 Digital Te chnical jo urnal LAN Addressing fo r Digital Vi deo Data

mechanism could be used to locate available mate­ Clashing allocations of multicast addresses can be rial more efficiently. held to a minimum if servers allocate an address at Once a client identifies a server that is offering random from the remaining pool of addresses rather the desired data stream, it can request that the than all servers allocating in the same fixed order. server begin transmission. The client sends a mes­ sage identifying the desired playback program Ide ntifying and Stopp ing Playback material. In response, the server allocates a unique After a client requests playback of new material, it multicast address, includes the new material and can then examine the server's announcements, and multicast address in its announcement messages, when the desired data stream appears as being and begins transmitting the program material. transmitted by the server, the client can begin receiving data from the advertised multicast Address Allocation and Tr acking address. At this point, any other client stations on Each server maintains a table containing the usage the LANcan also receive the same video program by of each of the A2 to An addresses. Each ad dress is enabling receipt of the same address. tagged as either currently used or available fo r use. When no more cl ients wish to view a partic­ When a server receives a client's request fo r trans­ ular program, a mechanism is needed to inform mission of a new data stream, the server selects a a server to stop transmission and return the asso­ currently unused multicast address and includes ciated address to the free pool. Two alternative the address and data stream description in its approaches were considered to stop playback; one announcements of data streams currently being was chosen fo r several reasons. transmitted. After sending two announcements, In the first approach, each server tracks the num­ the server begins transmitting the data to the cho­ ber of clients that have requested a particular pro­ sen multicast address. Sending two announcements gram by simply counting the number of requests before beginning transmission provides client fo r that program. In addition, clients are required to nodes with ample time to ascertain the address to notify the server when they are finished viewing. which the data will be sent and to enable reception The server then continues to transmit the material of the video program. until all interested clients have indicated they are In addition to sending announcement messages, no longer interested in viewing. This approach has the servers also listen to the announcements from two problems. If a viewing client node is reset or

other servers to keep track of all multicast disconnected, or iJ its message to end viewing is addresses currently in use on the LAN. Each time a lost, the server could lose track of the number of server receives an announcement message from viewing clients and never stop playing a particular another server, it notes the addresses being used program. The second problem, which is more of a and marks them all as used in its table. This pre­ nuisance, is that clients have to request playback of vents a server from allocating an address already a program even if it is already playing to enable the used by another server and eliminates the need fo r servers to track the number of viewers. a central database or clearinghouse. In the preferred approach, interested clients If a server observes that it is using the same periodically remind the server that they wish to address as another server, then the server moves continue viewing the program. Servers then simply its data transmission to another address if and only keep playing the material until no client expresses if its node address is numerically lower than the interest for some period of time. For example, other server's node address. The new address is clients could reiterate their interest in a program allocated exactly as it would be if the server were every second, and a server could continue transmit­ beginning to transmit the data stream for the first ting a requested program until it did not receive a time. This algorithm resolves conflicts where two reminder fo r 3 seconds. This time lapse would or more servers choose the same available multi­ accommodate lost reminder messages from clients, cast address at the same time. In addition, it and client failure would result in transmission ter­ resolves a similar confl ict that occurs when two mination within 3 seconds. In addition, when all separate LA.I'I segments become joined and two clients had finished viewing the material, the servers suddenly find they are using the same multi­ server, multicast address, and consumed network cast address. bandwidth would be released within 3 seconds,

Digital Te chuical]ournal Vo l. 5 No. 2 Sp ring 1993 81 Multimedia

making them available fo r other uses. Selection of Ex te nsion to Interconnected LANs the actual timer value depends on the desired bal­ The described protocols and allocation methods ance between ongoing consumption of network fu nction correctly across multiple LANs intercon­ resources (bandwidth and multicast addresses) nected by bridges since bridges nominally fo rward after all receiving parties have stopped viewing the multicast traffic. Many bridge implementations per­ data, and network, end system, and server resource mit management control over the fo rwarding of consumption caused by more frequent reminder multicast data. This can unintentionally interfere messages. with the desired operation of this proto col, but it can also as serve as a useful tool to confine data Changing Multicast Add1'esses traffic to particular LAN segments. Another prac­ tical consideration in the particular application Aside from receiving and processing the data fo r a described here is the ability of a bridge to forward video program, client stations must also continue the large amounts of data traffic involved in digital to examine the server announcement messages and video without detrimentally impacting the time­ remain alert to possible changes in the multicast dependent nature of the data. address to which the received program is be ing Extending the protocols to a wide area network transmitted. As noted above, address al location can is a more difficult procedure. Routers do not fo r­ change at any time due to merging of LAN segments ward multicast traffic, but they could if used as or duplicate allocation by two servers. Anytime a proxy nodes between LANs. Ro uter fo rwarding client notes a change in address, it must stop receiv­ performance teJ1(1S to be even lower than bridge ing data on the previous address and resume receiv­ fo rwarding rates, which discourages the operation ing with the new address. A momentary disruption of this system over a router. in playback is likely to occur, but such disturbances are infrequent because only merging LANs cause duplicate allocations of addresses in the midd le of Conclusions playback. Dynamic allocation of multicast addresses is criti­ Under the circumstances described earl ier, a cal to enable features such as the continuous play of client can find itself receiving two data streams on fu ll motion video over a network with multiple the same multicast address fo r some fi nite time viewers. It is not fe asible (or at least is very difficult) period until the servers resolve the al location of fo r a server to transmit a data stream individually that address. Clients can gain immunity to this si tu­ to all clients wishing to receive it. If, on the other ation by noting the source address of the server that hand, the desired data stream is transmitted to the originally provided the data stream, and discarding broadcast address, all stations on the LAN have to all data received on the multicast address that is not receive an enormous volume of data whether they from the source address. With this improvement, are interested or not. It is highly desirable not clients can easily distinguish the data strea m of to inundate uninterested clients with video data interest from another which might momentarily streams, but to sencl them to clients that want to appear addressed to the same multicast address. receive specific video data streams in which they The allocation and resolution of multicast are interested. address use can be improved if servers send their Multicast addresses are well suited (in fact announcements at an increased rate fo r some time designed) fo r transmission to some arbitrary group period after a new data stream begins transmitting of stations. To prevent a client that is receiving one or when a data stream changes address. Such accel­ video strea m from being inundated by other video erated announcements permit client stations to streams, a unique multicast address is required more quickly identify the address of a requested fo r each unique data stream. Since there are infi­ data stream, and more quickly identify when a data nite individual data streams to choose fr om, it is stream has moved from one address to another. impossible to allocare a unique multicast address They also permit servers to more qu ickly identify fo r every data stream. A mechanism to allocate instances of clashing multicast addresses and a unique multicast address from a finite set of resolve them. For example, the announcement addresses fo r the duration of the data stream is the interval could be increased from 1 second to one­ ideal choice. The described mechanism also has the quarter second fo r a 2-secon

82 l!rJ/. 5 No. .2 SjJrinx /')'}.) Digital Te cbnical jour11al LAN Addressing fo r Digital Video Data

multicast addresses; therefore it is more reliable as 6. To ken Ring Access Method and Physical servers join and leave the LAI'I . Lc�yer Sp eCifications (New Yo rk: The Institute Although transmission of digital video data has of Electrical and Electronics Engineers, Inc., prompted this system design, the basic mechanism 1986). fo r dynamically allocating mult icast addresses can To ken-Passing Bus Access Method and Physi­ be applied to any application with similar needs. 7 cal Layer Sp ecifications (New Yo rk: The Insti­ Acknowledgments tute of Electrical and Electronics Engineers, 6 I would like to acknowledge the assistance of Inc., 198 ). john Forecast of Digital's Networks and Commu­ 8. Local Area Ne twork MA C (Media Access nications Group in enumerating the necessary Control) Bridges, IEEE Standard 802.l(d) pathological conditions in this work and for acting (New Yo rk: The Institute of Electrical and as a sounding board fo r proposed solutions. Electronics Engineers, Inc., 1990). References 9. JV/C6883 8 Media Access Controller Us er's 1. K. Harney, M. Keith, G. Lavelle, L. Ryan, and Manual (Phoenix, Arizona: Motorola, Inc., D. Stark, "The i7'50 Video Processor: A To tal 1992). Multimedia Solution," Communications of 10. Logical Link Control, ANSI/IEEE Standard the AOVI, vol. 34, no. 4 (April 1991). 802.2-1985, ISO/DIS 8802/2 (New Yo rk: 2. G. Wa llace, "The ]PEG Still Picture Compres­ The Institute of Electrical and Electronics sion Standard," Communications of the ACM, Engineers, Inc., 1985). vol . 34, no. 4 (April 1991). 11. A Primer to fDDf: Fiber Distributed Data 3. D. Le Gall, "MPEG: A Video Compression Stan­ Interfa ce (Maynard, MA: Digital Equipment dard fo r Multimedia Applications," Communi­ Corporation, Order No. EC-H0750-42 LKG, cations of the ACiJ.I, vol. 34, no. 4 (April 1991). 1991). 4. Carrier Sense !V!ultiple Access with Collision Detection (CSMA/CD) Access Method and 12. FDD! Station Management-Draft Proposed Physical Layer Sp ecifi cation (New Yo rk: American Na tional Standard (New Yo rk: The Institute of Electrical and Electronics American National Standards Institute, Engineers, Inc., 1986). June 25, 1992).

5. Fiber Distributed Data Interface-To ken 13. S. Deering, "Host Extensions fo r IP Multicast­ Ring Media Access Control (New Yo rk: ing," Internet Engineering Ta sk Fo rce, RFC American National Standards Institu te, 1987). 1112 (August 1989).

Digital Te chnical Jo urnal Vo l. 5 No. 2 Sp ring I'J'J3 83 Paul B. Patrick, Sr. I

CA SE Integration Us ing ACA Services

Digital uses the object-oriented software Application Control Architecture (A CA) Services to address the problems associated with data access, interapplication com­ munication, and work flow in a distributed, multivendor CA SE environment. Th e modeling of applications, data, and op erations in ACA Services provides the fo un­ dation on which to build a CASEenvironment . ACA Services enables the seamless integration of CA SE applications ranging fro m compilers to analysis and design tools. ACA Services is Digital's implementation of the Object Management Group 's (Oil{ G) Common Object Req uest Broker Architecture (CORBA) sp ecification.

Based on work accomplished in many computer­ Data integration, i.e., information sharing, is vita.l aided software engineering (CASE) projects, this to any CASE environment because it reduces the paper describes how Digital's object-oriented amount of information users must enter. However, Appl ication Control Architecture (ACA) Services data integration must be accompanied by a mecha· can be used to construct a CASE environment. The nism that allows control to pass from one app.l ica­ paper begins with an overview of the types of CASE rion to another. This mechanism, commonly called environments currently available. It describes the control integration, provides a means by which object-oriented technique of modeling applica­ the appropriate application can be started and tions, data, and operations and then proceeds to requested to perform an operation on a piece of discuss design and implementation problems that information. Control integration is also used to might be encountered during the integration pro­ exchange information between cooperating appli­ cess. The paper concludes with a discussion of cations, regardless of their geographic locations. environment management. These two integration mechanisms used in tandem can solve many of the problems presented by a dis­ CA SE EnvironmentDescri ption tributed, multivendor CASE environment. To day's CASE environments are required to operate ACA Services is Digital's implementation of the in network environments that consist of geographi­ Object Management Group's (OMG) Common Object cally distributed hardware manufactured by multi­ Req uest Broker Architecture (CORBA)specific ation. ple vendors. In such environments, access to data, ACA Services is designed to solve problems asso­ metadata, and the fu nctions that operate on this ciated with application interaction and remote data data must be as seamless as possible. This can be access in distributed, multivendor environments accomplished only when well-architected proto­ such as the CASE environments just described. This cols exist fo r the exchange of information and con­ support includes the remote invocation of applica­ trol. These protocols need not be defi ned at the tions and components without the need fo r multi­ level of network packets, but rather as operations ple logins or the use of terminal emulators. The that have well-defined, platform-independent inter­ encapsulation fe atures of ACA Services allow the faces to pred ictable behaviors. use of applications not designed for distributed In addition to utilizing the various applications, environments. ACA Services can also be configured, environments deal with how applications are orga­ in a way transparent to the appl ication, fo r use on a nized or grouped within a project and how work local host. flows between applications and within the environ­ The central fo cus of a CASE environment is on ment as a whole. These concepts are discussed how easily functions such as compiling. building, later in the paper as are the diffe rent styles of inte­ ami diagra mming can be performed . The functions gration that an application can employ. av ailable fo rm the foundation on which the

84 Vo l. 5 No. 2 Sp ring 1993 Digital Te clmica/jo urnal CA SE integra tion Us ing ACA Services

environment is constructed. Therefore, the first ed it operation is sent to the object representing the step in the design of a CASE environment is to deter­ file. However, a debugger may also use the same mine what functions to offer. The applications cur­ editor to display source code. The operation to rently available to support these functions may be position the cursor on a particular line is sent integrated using one of two paradigms: application­ directly to the text editor application, rather than oriented or data-oriented. to a data object such as the line. An environment with such a split fo cus avoids the expense and com­ Application-oriented Pa radigm plexity of presenting a complete object-oriented interface to the environment and results in the CASE environments that fo llow the appl ication­ existence of both appiication- and data-oriented oriented paradigm fo cus on standalone applica­ paradigms. tions used to develop software such as editors, Regardless of which paradigms and applications compilers, and version managers. Application­ a CASE environment uses, the primary fo cus of the oriented environments normally comprise a col­ environment is on the objects and on the opera­ lection of applications that support the necessary tions that are defined on those objects. Therefore, fu nctions. In application-oriented environments, after determining what functions to offer, the sec­ integration tends to be fo cused on direct communi­ ond step in designing a CASE environment is to cation between two different applications. In this understand how applications, data, and operations paradigm, the requesting application knows which are modeled using an object-oriented approach, in class of application can be used to satisfy a par­ particular the one provided by ACA Services. ticular request. Environments that present an application-oriented paradigm to the user require the user to have knowledge of the applications that CASEIntegr ation in can be used to perform specific tasks. Object-oriented Ter ms As the level of task complexity increases, it Describing environments using object-oriented becomes increasingly important to build environ­ techniques can simplify the design of an environ­ ments that utilize a paradigm fo cused on the data ment. Techniques such as abstraction and poly­ associated with the task being done and not on the morphism can be used to describe the objects applications used to perform the task. The realiza­ that comprise the environment, the operations that tion of this problem has brought about the exis­ can be performed on those objects, and any rela­ tence of data-centered environments. tionships that exist between objects. Further­ more, using these techniques makes it possible to Data-oriented Pa radigm describe an environment as a set of classes and ser­ ACA CASE environments that use a data-oriented para­ vices for each class. Services performs the role of the method dispatcher, matching an object and digm arc centered around the data associated with the task the user is performing. To accomplish an operation with the fu nction in an application that can implement that operation. To realize the a task in such environments, operations are per­ benefits of this approach requires constructing fo rmed on a data object. Using the object being models fo r the applications, data, and operations addressed, the operation, and preferences supplied that will be present in the environment. by the user, the environment determines which application will be used to perform the requested operation. Thus, the requesting appl ication requires Mo deling Applications and no knowledge about which application implements Application Relationships an operation. This paradigm is extremely useful in Applications that are integrated into an environ­ CASE environments because of the diversity of ment can provide various fu nctions or services to objects and range of applications available to per­ other members of the environment. The number of fo rm certain operations. services an application provides depends not only The application and the data paradigms can on the capabilities of the application but also on coexist in a single CASE environment, and in fa ct, the way it is modeled. These services are stand­ tightly integrated CASE environments exploit the alone pieces that can be plugged into a system to strengths of each paradigm. A text editor can be perform specific functions. An application can used to illustrate this point. Typically, when the define a single operation whose sole fu nction is to contents of a source file need to be modified, an start the application; an application can export the

Digital Te chnical jo urnal Vu /. 5 No. 2 Sp ring l'J

entry points of its callable interface; or an applica­ of each component of an application depends on tion can define sets of operations fo r each type of whether a component contains a superset or a sub­ object it manipulates. In support of application set of the functions contained in the components modeling, ACA Services provides the concepts of of other applications in the environment. application classes, methods, and method servers. Once the application's components have been Figure 1 illustrates the relationships among the var­ classified, the integrator must determine how the ious pieces of information used to model an appli­ application will make its capabilities available to cation in ACA Services.1 the environment: as an operating system script, as a ln ACA Services, the definition of an application is callable interface, or as an executable image. The divided into two pieces: interface and implementa­ implementation definition represents the actual tion. The interface definition is concerned with the implementation of the application. An application publicly visible aspects of the application. These may comprise a number of executable files and include class definitions for the objects that the shared libraries. Typically, only the executable file application manipulates, a class definition fo r the used to start the application is modeled as a method application itself, and definitions of operations that server. If the functions of the application are pro­ the application supports. The operations, which vided through a shared library or image, only the represent the functions provided by the applica­ shared library is modeled as a method server. tion, are modeled as messages on the application The implementation of the fu nctions or services class definition. These messages define a consistent exported to the environment are modeled as meth­ interface to various implementations of the opera­ ods. Methods describe the callable interfaces or tions. Placement of the application class definition operating system scripts that implement a particu­ affects the behaviors this definition inherits. This is lar operation and are associated with only one sometimes called classification. The classification method server. 2 During the method selection pro­ cess, the messages defined fo r the application ami the objects it manipulates are mapped onto one or more methods.

Modeling Data and Data Relationships Data modeling is another significant aspect of creat­ ing CASE environments, especially environments that utilize a data-oriented paradigm. Identifying the data objects that the application uses is a key 1,N METHOD 1.N SERVER element in the process of integrating that applica­ tion. The list of data objects should include those APPLICATION DATA CLASS CLASS objects fo r which the application provides a ser­ vice, as well as those objects on which the applica­ tion makes requests. The variety and quantity of O,N O,N data objects can vary from application to applica­ tion and depend on an application's capabilities and the paradigm utilized. To support the modeling METHOD of data objects, ACA Services uses the concept of data classes. Note that, rather than provide instance management fo r data objects, ACA Services pro­ vides a means to represent the data classes used by an application as metadata. Because environments that utilize a data­ oriented paradigm may contain many data classes, O,N O,N ACA Services organizes the data classes into an inheli­ MESSAGE tance hierarchy. This hierarchy allows responsi­ bilities, such as operations and attributes, to be inherited by other data classes. Data classes fo und Figure 1 ACA Services Jl!Ietadata Model in an ACA Services inheritance hierarchy are related

86 Vo l. 5 No. 2 Sp ring 1993 Digital Te chuical jou rnal CA SE In teg ra tio n Us ing ACA Services

to one another through an "is-kind-of" relationship. of the client application is blocked only for the A class that has an "is-kind-of" relationship with amount of time required to deliver the request. one or more superclasses must support all opera­ Client execution resumes once the request has tions defined on the superclasses from which it been delivered. Upon completing the processing of inherits .\ A subclass is not limited to those opera­ the request, the server application notifies the tions and attributes defined by a superclass but may client application of the completion and returns have other operations, as wel l as refinements to any output data. inherited operations and attributes. The request/reply in teraction model, shown in Figure 2c, is most appropriate for requests whose Mo deling Op erations implementations cannot perform the operations As mentioned previously, operations are modeled requ ired to obtain the necessary output data. as messages in the CASE environment. The name of Gateway and message-forwarding applications are the message describes the type of operation. Some examples of applications fo r which this type of messages are data oriented, i.e., Edit, Reserve, and interaction model is best su ited. In this model, the Copy, whereas other messages are appl ication ori­ message that represents the request cannot have ented, i.e., ExecuteCommand and TerminateServer. any output arguments and must pass an application Messages provide a consistent abstraction of the handle to itself. Tbe server application uses the fu nctions provided by applications. This abstrac­ application handle to return any output informa­ tion allows the details of how a fu nction is tion to the requester by sending a message that rep­ implemented to be hidden from the requesting resents the reply. In a request/reply model, a single application. Since AC:A Services supports more than reply message should be defined fo r returning infor­ one implementation for a single message, it also mation, thus reducing the number of messages an provides a means to hide various implementations. application must support. The developer should anticipate different imple­ mentations of a message within the environment Message Argwnents A message argument fo r and be aware that a message may apply to a variety passing the object being manipulated need not be of classes. The developer must consider how the defined. ACA Services automatically passes the operation on an object might be used by various object to which the message was sent to the applications and in future environments. ' In this method. Each method routine can access the object way, adding new types of objects to an environment through a structure containing context informa­ requires only minor changes, if any, to applications tion fo r the current invocation. that are already integrated. The arguments of a message should not be designed around a specific instance of an applica­ Op eration Interactions The semantics of a mes­ tion, nor should they imply how an object is physi­ sage dictates which particular interaction model is cally stored. To help meet these design criteria, all to be used. AC:A Services can be used to construct references to an object should be passed as instance a number of different interaction models: syn­ handles. In this way, the application that receives chronous request, asynchronous request, and the instance reference can use it directly fo r sub­ request/reply, as shown in Figure 2. The syn­ sequent operations on that object. In addition, chronous request interaction model, shown in when defining the message arguments, developers Figure 2a, is useful when serial operations originate should consider other appl ications that could be from a single source. This model blocks the execu­ instances of a particular class and possibly used as tion of the client application during a request. replacements. Control is returned to the client application only However, all instances of an application do not after the server application receives and executes have the same set of capabilities. To support the var­ the request and outputs data, ifany . ious capabilities, the developer may have to define The asynchronous request interaction model, additional arguments to represent bit masks and shown in Figure 2b, is usefu l in situations where flags. An argument list or an item list can be used the cl ient can process other work until the server to pass information about diffe rent data types or application completes the request. This model is quantities. The message design should not require especial ly beneficial when the requested operation implementation-specific information for proper takes a considerable amount of time to complete or application operation; this design implies that rea­ if the server is busy with other requests. Execution sonable defaults accommodate any unspecified

Digital Te chnical jo urnal Vo l 5 No. 2 Sj;ring II.J'J.! 87 Application Control

CLIENT APPLICATION SERVER APPLICATION

Reserve( )-foo.c ACAS _lnvokeMethod( ); CMS_ReserveMthd( ) (

if (status '= SUCCESS) ·------return( SUCCESS ); )

(a) Sy nchronous Request

CLIENT APPLICATION SERVER APPLICATION

Browse( ) - foo.c ACAS_InvokeMethod( ); LSE_BrowseMthd( ) ( : return( ); : )

CompletionCallback( ) �------return( SUCCESS ); { )

}

(b) Asynchronous Request

CLIENT APPLICATION SERVER APPLICATION

Connect( ) - Gateway ACAS_InvokeMethod( ); MVS_ConnectMthd( )

{ : return( ); return( SUCCESS ); l l Reply( ) - Client ReplyMthd( ) LU62_ConnectAck( ) { { return( ); return( ); } l

(c) Request/Rep ly

F(f? ur·e 2 Op eration Interaction Jl!Jo dels

information. In cases where proper operation of an around the modeling of objects in the environment. application requires implementation-specific infor­ As discussed in the previous section, abstraction is mation, the most suitable design is to use the con­ used to hide much of the actual implementation of text object as a place to store the default values. the operations on objects from the requesting With such a design, the application no longer needs application. However, additional context may be to use hard-coded default values and can be cus­ requ ired fo r further operations. If the application is tomized fo r the environment. using an application-oriented paradigm, most oper­ ations are directed to an application class that pro­ Integration Frameworks vides the service. In cases where a data-oriented A number of issues must be resolved in the con­ paradigm is used, the application typically directs struction of a CASE environment before the first line operations to the data class of which the object is of code can be written. Many of these issues center an instance.

88 Vol. 5 No . .! Sp ring 1993 Digital Te chuical]ounutl CASE Integration Us ing ACA Services

Besides the application and data objects fo und in ing basic information about various aspects of the the environment, the designer must also take into environment. consideration the other components of the CASE environment itself Figure 3 shows the major com­ AddingNe·w Im plementations ponents of a CASE environment: activities, applica­ Updates to the environment may include adding tions, appl ication and data interfaces, work flow new application classes, data classes that the new management, and handle management. Each com­ application supports, method definitions fo r the ponent represents a particular aspect of the overall application, and possibly a method server defini­ environment. The components are introduced in tion. As described earlier in the paper, ACA Services this section and described in detail elsewhere in uses data and application classes to represent the the paper, as indicated. different classifications of data and application Activities provide the basic work structure for a objects fo und in an environment. Storage classes particular task within an environment. Each activ­ represent the classifications of storage and how ity comprises one or more applications and a num­ objects are referenced in the environment. Each ber of data objects, fo rming a single composite class, i.e., data, application, and storage, contains a object. Applications within an activity operate list of messages that represent the operations that through the application interfaces. The section can be performed on the class. Application Integration describes the principles of Digital's CASE environment, COHESION, was an activity and includes a discussion of the sharing designed to present a data-oriented perspective to of applications within and among other activities. the user. An initial level of integration was achieved Application interfaces, illustrated in Figure 3 as by utilizing this same data-oriented approach to arrows connecting the various applications, fo rm application integration. Implementation of a data­ the primitives by which integration is accom­ oriented approach required that method maps for plished. Some of the more general concepts fo r messages on data classes contain an indirect refer­ application interfaces were discussed in the sec­ ence to an abstract application class.' Figure 4 illus­ tion Modeling Operations; these concepts are trates this concept by showing two different described in detail in the section Styles of messages: the Edit message, which uses an indirect Appl ication Interfacing. method reference, and the Browse message, which Finally, the section Environment Management uses a direct method reference. An indirect method addresses how to manage the flow of work within reference has two parts separated by the character the environment. This section describes the '@': first, the name of the message to be sent; and management of instance and application handles, second, the name of the class on which to send the the use of storage classes as a means to provide message. Although not commonly done, an indirect data transformations, and the management of method reference allows the original message to be events within the environment. To better under­ mapped to another message on a different class, stand each of these topics requires the fo llow- given that both messages have arguments of the

CASE ENVIRONMENT

ACTIVITY

Figure 3 Components of a CA SE Environment

Digital Te e/mica/ journal Vu /. 5 Nu. 2 Sp ring I'J'J.i 89 Application Control

OBJECT

I DATA_OBJECT I

FILE Editor Browser I I � �

- ..,...,.-- - ...... - .J ... ----- " , ' , METHOD MAP ETHOD MA Text_File " � , � P � _ � _ _, �.... Vi Browse ., _ ,-"" ...... _- ____ .... _ ...._v� -C e r:..,. ..,. - - - � � - � ...... ------.... -- --.... " ' ' � ---, METHOD MAP ' --_,,� ETHOD MAP � ',��d� �-E-d�� ' ... Vi Browse ; ," .... ------,

r---�----�----�lContext Object Table User_Preferences Edit @ Editor = View @ Vi End Table

Figure 4 Direct and Indirect Method References

same type, direction, and order. Both messages CASE application to be added . Consequently, the must also return the same type of object. class hierarchy contains both abstract and instance On encountering an indirect method reference, classes. The application class is required to contain ACA Services first looks at tables in the context all the messages defined on irs superclass, plus any object fo r an attribute that matches the reference. If additional messages that the application supports. such an attribute is fo und, ACA Services uses the The method map of each message on an appl ication attribute value to determine the class and message class should contain a direct reference to the that should be checked next. Thus, users can pro­ method that implements the operation. Although vide a mapping to their preferred application fo r the better than the other alternatives, the COHESION operation. If no matching attribute is fo und, ACA approach has no default implementation unless one Services uses the message and class specified in the is explicitly specified in a context object. To over­ indirect method reference as the next place to come this problem, an entry for each message check. defined on the abstract appl ication class must be The approach used in COHESION has many advan­ created in one of the context objects. The values tages over specifying either a direct reference to a fo r these entries point to the corresponding mes­ method or an indirect reference to a specific appli­ sage on the class of application used as the default

cation class. This approach does nor I imit the user's implementation. ability to specify application preferences associ­ ated with using direct references to methods, nor Co mmon Classes

does it burden the installation of the application Common classes for a CASE environment provide with determining all the data classes that will need CASE application developers with a description to be updated (as required with indirect references about how an application fits into the environ­ to a specific appl ication class). In addition, the ment, the behaviors the application must support, approach allows the application developer to do the and the messages that resu lt in those behaviors. least amount of work and still provide the maximum The notion of pl ug-ancl-play in the environment level of support for user preferences in applications. is achievecl through the use of common classes. Using ACA Services, the applicatio n developer An implementation that ad heres to the descrip­

must create an application class definition fo r each tion of a particular class of applications can be

No 90 v'r! l. 5 . .2 .�)! ring I'J9.! Digital Te chnical jo urnal CASE Integration U� ing ACA Services

easily switched with another implementation that integration of data throughout the CASE environ­ adheres to the same application class semantics. ment. The set of classes, although not exhaustive, Programs like COHESION are wo rking toward serves as a basis on which a CASE environment can a set of common classes fo r CASE environments. be built. Extensions of the hierarchy will occur as The set currently defined contains classes fo r many new classes of applications and their associated types of data and applications fo und in CASE envi­ data objects are integrated into the environment by ronments fo cused on the coding and testing phases independent software vendors, cuswmers, and of the software development process. A grap hical other CASE vendors. view of the data portion of the hierarchy is shown Most data classes are subclasses of the data class in Figure 5. The hierarchy is partial ly based on the SOURCE_FILE, because the initial data class imple­ hierarchy fo und in t\TIS, a standard fo r tool integra­ mentation was targeted at a CASE environment tion, and uti I izes the strength of the ATIS data consisting of editors, compilers, builders, and ana­ model.r' (Shaded boxes indicate the classes that are lyzers. Additional data classes fo r both file and specific to ATIS.) Encompassing the ATIS model, the nonfile objects will be added when appl ications hierarchy presents a uniform data model fo r the that provide and manipulate these objects are

DATA OBJECT

ELEMENT

NAMED ELEMENT

FILE CONTAINER

DIRECTORY

FILE DIRECTORY

EXECUTABLE FILE

Note: Shaded boxes indicate ATtS-specific classes.

Fig ure .5 Hierarchy of CA SE Co mmon Data Classes

Digital Te chnical jourual Vo l. 5 No. 2 .vJring IY'J:i 91 Application Control

integrated intO the environment. A number of data environments. Until now, most integration was classes represent composite objects such as tests accompl ished by linking one application with

how the object is physically stored in the environ­ tions. However, such applications tend to be unable ment. Classes that represent composite objects to operate independently, without the other mem­ have attributes with values that are :�ctualtyother ber. Al so, each coupled member tends to have objects. For example, the test data class typically its own application programming interface (APT). has attributes that represent the result of a rest run, Integration perh>rmed in this manner resu lts in an

an operating system script or program used to per­ applic:�tion that must maintain code to surport fo rm the test, and a benchmark against which a test multiple APis, if the application is to work in a num­ run is compared . Each of these attributes may have ber of environments. Such support can increase the as a value a reference to the file object that contains maintenance cost and the time and effort required the actual data. to integrate witiJ other implementations of appl ica­

The portion of the hierarchy that is used to spec­ rions ami environments. Other by-products of this ify application classes contains only abstract appli­ approach are an increased image size and a need to cation classes, as shown in Figure 6. These classes rerelease software when a dependent application provide structure, but more important, they define changes. The degree to which rerelease occurs the operations that are inherited by any appl ica tion varies with the platform and operating system. that is an instance of a class. Abstract classes are ACA Services can be used to minimize the num­ provided fo r a number of the applications fo und in ber of interfaces that an application must maintain

CASE environments that deal with the coding and withou t removing fu nctionality; a common API resting fu nctions. The hierarchy does not contain provides the interface to all potential hmctionality. any classes that represent particular instances of an The ACA Services API, along with a set of com­ appl icarion. Such application classes exist only mon classes, allows the same level of interaction when applications are installed in the environment. between applications that can be accomplished through a private API, without the negative siue Consistent Integration In terfa ce effects previously described. Through the use of Many CASE vendors are building products fo r a common classes, an appl.ication can integrate with number of different environments, including elec­ multiple implementations of another application tronic publ ishing, office automation, computer­ without requiring a separate effort for each. On aided design, and computer-aided manufa�turing, platforms where uynamic loading of libraries or in addition to CASE. Therefore , vendors must decide shareable images are supported, applications can how to integrate these applications into the various use ACA Services to locate the appropriate library,

CONFIGURATION MANAGER

PERFORMANCE ANALYZER OBJECT FILE CONVERTER

MACRo MESSAGE u I L .______.l l I._I __T_ P_u_ ___. ll.______,

figure G Hiemrcb)' of CASE Co mmon Applicotion Classes

92 V" l. 5 No . .! .'.)H ing 1')9.) Digital Te chnicaljourua/ CA SE In tegration Us i11g AC4 Seru ices

find the proper entry point, and transfer control to because the structure of the code requires no major the appropriate routine. ACA Services also provides changes. To register the current executing instance a transparent mechanism fo r encapsu lating applica­ of the application with ACA Services, a call to the tions that have no callable interfaces. Use of this ACAS_RegisterServer routine must be added to the mechanism extends the number of applications application's initialization routine. During the pro­ that can be integrated and rem oves the need to cess of run-time registration, ACA Services registers develop operating system-specific code to start various information about the applicatio n, includ­ applications. ing the identifier of the process in which the appl i­ cation is executing, the owner of the process, and Styles of Application Interfa cing the class- and instance-unique iuentifiers fo r the application. As part of the registration, an applica­ Creating an interface to an application that is to be tion can specify an abstract name by which it can integrateu is diffe rent from integrating an applica­ be located and the routines to be called when an tion into an environment. Application interfacing ACA Services event arrives, e.g., when the server deals with the public interface or interfaces that is instructed to shut down or when a session ends. the application provides to another application. In Once registered with ACA Services, the appl ica­ turn, these interfaces provide the primitives that tion must enter its event dispatching loop. Because can be used in the integration of applications. many applications have existing event dispatcbing Application interfaces can be created in various mechanisms, ACA Services been designed ways, with differing levels of effort. Software devel­ has t(H· easy integration with most mechanisms. ACA opers can design new applications to utilize all the Services provides this support by allowing the capabiJities of ACA Services. Existing applications appl ication to define a routine called the event can also take advantage of the fu ll capability of notifier, which is called at signal level each time an ACA Services, if the source code to the application ACA Services event occurs. The event notifier rou­ is available and if the application can be easily tine places an event on the appl ications work adapted to use an event-driven model. However, queue fo r the ACA Services event. Upon encounter­ even if the source code to an application is not ing the event, the application's event dispatcher available. applications can still be integrated into routine calls the ACAS_Dispatch routine allow the environment using ACA Services. If the applica­ LO ACA Services to dispatch the appropriate method or tion has a cal lable interface, a server can be written management routine for the event. A description of that receives messages and calls the appropriate API how ACA Services dispatches operation requests routines. If the application does not have a cal.lable fo llows. interface, the aprlication can be integrated by encapsulation through the use of an operating sys­ tem script. The remainder of this section describes App lication Servers how to use each of these techniques to create an When the application to be integrated does not interface through which the application can be have a user interface but provides a cal.lable inter­ integrated into a CASE environment. face, integration is best accomplished by creating an application server. Considered a fo rm of encap­ Application Modifications sulation, an application server provides a consis­ An existing application can easily be adapted to use tent programming interface to the appl ication. An ACA Services, if the source code to the appl ication application server provides jacket routines that use is available. With minimal changes, an application the application's callable interface, hiding the that utilizes an event-driven design, like that used actual details of this interface. This technique is also by most window-based appl ications, can operate as used to create applications that have a clean separa­ an application server. 'T'he actual modifications tion of presentation and functions. required to provide ACA Services support diffe r Applications that implement persistent data across appl ications, but fo r most window-based stores, such as databases, code managers, and applications the changes are similar. As an illustra­ repositories, are prime candidates for this style of tion of this style of integration, consider an editor. integration. By using an application server to Most editors are implemented as event-driven access persistent data stores, a requesting appli­ applications, which allows easy integration cation need not know how the data store is

Digital Te cbnical jo unwl Vol. 'i No. 2 .\jJ ring /')')� 93 Application Control

implemented and which implementation is to be the proper fo rmat and sends this information to the used. This technique promotes the reuse of existing transport, which actually sends the information fu nctions contained in the environment regard less back to the cJ ient application. of the actual implementation of the fu nction. Using the DEC/CMS application server as an exam­ Digital's Code Management System (DEC/CMS) and ple, the software developer must create a main rou­ COD/Repository software are examples of applica­ tine to (I) pert<>rm any setup required to use the tions that have been integrated using the appli­ callable interface and (2) register the existence of cation server technique. Figure 7 illustrates the the server with ACA Services. Registration includes typical structure of the various components specifying the method dispatcher routine, which is invol.ved in this style of integration. generated by ACA Services, so that the appropriate As shown in Figure 7, the integration process method routine will be dispatched fo r the message involves the fo llowing steps. (1) An invoke from the received. client application of the message "H.escrve" on the A method routine exists fo r each operation that object "foo.c" goes through the resolution code and the server is capable of performing. The set of

(2) out the transport to the server application. This method routines is analogous to the operating sys­ may result in starting the server application, if no tem script fo r compilation used to explain applica­ server was available to service the request. ()) The tion encapsulation later in this section. Because the server application's main routine cal.ls the event DEC/CMS application server is not an operating sys­ dispatcher and waits fo r work to arrive, when the tem script, message arguments are passed into the server is started. (4) When the "Reserve" message method routine directly. As mentioned earlier in arrives on the transport, the transport notifies the the section CASE Integration in Object-oriented server application , (5) causing the event dispatcher Te rms, the object on which the current operation is to dispatch the "Reserve" message by calling the to be performed is available to the method routine method dispatcher routine. (6) The methou uis­ through the use of the invocation context struc­ patcher routine calls the appropriate method inter­ ture. Information about the object, such as its class, face routine. (7) The method interface routine does name, and generation, can be obtained by calling any work required to cal l the appropriate callable the ACAS_ParselnstanceHamlle routine. The class interface routine. (8) When the cal lable interface of the object can tben be used to determine if the routine returns control to the method in terface object is an element under version control, a collec­ routine, the routine can perfo rm any work neces­ tion, or a group. sary before (9) returning control to the method The name of the object and its generation are dispatcher routine. (10) The method dispatcher contained in the reference data field of the instance rou tine then puts any arguments to be returned in handle that represents the object. Because each

APPLICATION PROCESS SERVER PROCESS

APPLICATION SERVER

MAIN ROUTING INITIALIZATION CODE CLIENT APPLICATION J(3) .. EVENT DISPATCHER DATA TYPE/LIST CALLS ACAS_InvokeMethod(Reserve, foo c) + (4) (5) - TRANSPORT � .. (1) ... I t_(10) RESOLUTION CODE METHOD DISPATCHER I DATA TYPE/LIST CODE 1(6) t (9) t (2) I TRANSPORT 1-+'-- � METHOD INTERFACE + l (7) I (Bl I t CALLABLE INTERFACE

Figure 7 Block Diagram of a Code Management ,\j>stenz Application Server

94 Vol. 5 No . .2 .\j!ri11g /'}';).) Digital Te chnical journal CA SE In tegra tion Using ACA Services

diffe rent code management system has its own COMPILE FOO.C - COMPILE( )- FOO.C ACAS SCRIPT /DEBUG - representation of generation, it was necessary to SERVER /NOOPT create a canonical fo rmat to represent all imple­ mentations. Therefore, the method must convert $ @SYS$LIBRARY COMPILE. COM %INSTANCE( ) the canonical generation representation to a format that is native to the implementation, Le., DEC/CMS APPUCONT GET ARGUMENT DEBUGNALUE = DBG DBG = "TRUE" IF specific In addition, any method that returns a ref­ THEN erence to a versioned object must convert the DBG_OUAL = "/DEBUG" ENDIF native generation representation to its canonical format. Ta ble 1 shows how an object reference can CC 'P1 'DBG_QUAL be mapped between its canonical and DEC/CMS­ specific to rmats.

Once the necessary information about the object Figure 8 Example of em Encapsulated Cornpiler has been retrieved and converted to a fo rmat native to the implementation, the method can call to the The purpose of an operating system script fo r appropriate callable interface routine, possibly compilation is to convert the generic compilation based on the object's data class. Once the call com­ qualifiers, which are passed as message arguments, pletes, the method must convert any objects to be into the compiler-specific options. The /DEBUG returned into a canonical fo rmat, at which point and /NOOPT qualifiers shown in Figure 8 are exam­ the method can return the status of the operation ples of generic compilation qualifiers. Many operat­ and output arguments. ing system scripting languages limit the number of parameters that can be passed on the command line. The compilation scripts avoid these limita­ Application Encapsulation tions by passing the name of the file to be com­ Encapsulation. the simplest integration technique, piled as the only command line parameter, as is appropriate fo r appl ications that do not have a shown in the command @SYS$UBRARY :COMPILE.COM callable interface or in cases where no source code %INSTANCE( ) in Figure 8. ACA convenience com­ is available. Compilers are an ideal candidate fo r mands, such as APPL/CONT GET ARGUMENT, are used this style of integration, because they perform syn­ to retrieve and set the values of the message argu­ chronous operations. Encapsulation of compilers ments in the operating system script. When all the provides a consistent programming interface to any switch values are gathered, the operating system compiler that is integrated into the environment, script converts the generic values into speciJic regardless of the qualifiers used to specify particu­ qualifiers. Finally, the actual command line is con­ lar compilation options. This technique can also be structed and executed. This same technique can used to prov ide a generic compile command that is also be used to encapsulate linkers and any other platform independent. Encapsulation of a compiler types of applications where no source code or is best accomplished through the use of an operat­ callable interface is available. When applications ing system script. Figure 8 illustrates an example of provide a callable interface, even tighter integration an encapsulated compiler. can be achieved by creating an application server.

Application Integration Ta ble 1 Converting Generation Integration of applications goes beyond the inter­ Representations faces that applications present to the environment; Native Representation it concerns how appl ications interact with one Canonical Object Object another. Integration also takes into account the Format Name Generation policies used in an environment to allow a collec­ UTIL.C(10:BL7:3) UTIL.C 1083 tion of applications to be grouped into a single

DISPAT CH.C(1 ) DISPATCH.C composite object. This section discusses concepts

DUMP.B32(1 :A:8) DUMP.B32 1A8 such as an activity, locating an application within

GRAPH.BAS GRAPH.BAS 1+ an activity, context sharing, and the sharing of appl ications across multiple activities.

Digital Te ch nical jo urnal Vol. 5 No. 2 Sp rint; 1993 95 Application Control

Activity Pa rticipation ACTMTY NM

Since more than one activity may be active at server is started in the COHESION environment. The any given time, an activity must be able to locate method server definition specifies as the value of the other applications participating in the activ­ the start-up environment attribute, the names of ity. Data-oriented environments provide a means the context tables and attributes that are to be to loosely couple the various data and applica­ defined as environment variables upon start-up. tion objects into a single composite object. The Another way of providing a common context COHESION integrated environment refers to this across an activity is to propagate context object composite object as an activity. The implementa­ tables and attributes as implicit arguments to tion of an activity diffe rs depending upon the envi­ method servers. Specifying this information as ACA ronment: ATIS uses a persistent process; file implicit arguments instructs Services to propa­ system-based environments generally use a direc­ gate these attributes to the context object of the tory hierarchy; and environments built on a private method server serv icing the request. data store can use a data fi le. In the COHESION envi­ The context object can also be used directly to ronment, an activity is represented as an ACA create a common context across an activity, i.e., by Services context object that contains attributes that holding information that needs to be shared. This reference a directory hierarchy. The context object information can include references to directories, is used to set up the execution environmen t in preferences of applications, and default values. wh ich a set of applications will operate and to locate other applications that are execu ting within Sharing between Activities the activity. Reusing app.l ications that are active within an activ­ ity reduces the overall system resources required to Locating Activity Applications perform the activity. However, a problem occurs The ability to locate an application that is executing when two or more activities are active at the same in an activity allows fo r reuse of the application by time and require the same application. With the other applications executing in that same activity. addition of windowed interfaces and the need to Such locating provides fo r better util ization of utilize other services, application sizes have greatly applications and reduces the amount of context increased. Consequently, it is often impractical to that must be propagated from one application to expect a separate instance of an application to be another. To locate an application within an activity, associated with each activity that is active. an application must have registered its presence in In order fo r an appl ication to be shared between ACA the activity. When registering with Services, multiple activities, the application needs a means the application must specify the activity name as by which to determine if a request is part of an ACAS_SERVER_REGISTRY. the value of the attribute ongoing dialog with another appl ication or is the The appl ication must also register itsel f with the beginning of a new dialog. These dialogs, called event manager to allow centralized management of "sessions," represent a conversation between a pair the activity and to participate in the flow of work of appl ications. Each time a cl ient appl ication within the activity. makes a request to a new application server, a ses­ CASE applications determine if they are execut­ sion is established and an identifier is associated ing within an activity by checking t(> r the existence with the session. ACA Services passes the session ACTI VITY_NAN IE. of the environment variable If this identifier to the server appl ication. environment variable exists, its value is the activity The management of sessions can be accom­ identifier. To allow an activity to extend beyond a plished by using the session 1D as a lookup key into single host and to support diffe rent activities with a Jist of structures that represent the active ses­ the same name, the activity is identified by a unique sions. When the server application locates the identifier. structme associated with the session identifier, the appl ication can establ ish the appropriate context Sharing within Activities for that session. In the example of DEC/CMS applica­ Applications executing within an activity operate tion server, the strucru re would contain the hand le ACA in a common context. Services provides a set to the library associated with the session. of mechanisms that can be used to provide ACA Services also notifies an application server this common context. The environment variable when a session is to be terminated between a client

Vol. 5 No. 2 ,\j>riiiJ< 1993 Digital Te ch11 ical ]o unwl CA SE In tegra tion Us ing ACA Services

ami a server appl ication. When notified, the appli­ allows fo r the unique identification of an object in cation server determines the appropriate course of the cache across multiple hosts. In addition to the action. Using the CMS example, the server releases location and reference data, the structure contains any cached information it has kept about the ses­ a pointer to the instance handle returned from the sion, closes the specific CMS library, and then frees call to the ACAS_CreatelnstanceHandle routine. the library data block. Reuse of the instance hand le saves the time required to create the hand le, including any over­ En vironment Management head associated with using storage classes. Reuse also reduces the total amount of memory required. After defining application imerfaces and integrat­ However, instance hand les are not the only hand les ing appl ications into an activity, CASE environmem that require management; application handles need developers must fo cus on the management of the to be managed as wel l. environment as a whole. This includes the manage­ ment of references to applications and data, the transformation of object references into platform­ App lication Ha ndles specific fo rmats, and the flow of work within the Application handles are references to appli­ env ironment. cation objects. Each application handle can represent one or more method servers. A method Ha ndle Ma nagement server can generate a handle by calling the ACAS_CreateApplicationHandle routine, or the In the CASE environment, objects arc the targets of ACAS_InvokeMethod routine can return an applica­ all operations. Sending a message to an object tion handle as an output argument. As with requires understanding how to create and manage instance hand les, application handles can be references to the object. Since ACA Services does passed as arguments to a message. Management of not manage instances of objects, it uses references application handles is similar to the management to instances of objects. These references take the of instance handles. Each entry in the cache of fo rm of instance and application handles, which application handles contains the location of the reference data and application objects, respec­ application and the name of the class of appli­ tively. Proper management of these handles leads to cation. The entry also contains a pointer to the more efficient use of application objects, thus application handle and a count of the number of reducing the amount of network resources and outstanding references the hand le. Freeing an memory consumed by the application. Appropriate to application handle results in the termination of all hand Je management can also enhance performance sessions between the client and any method and guarantee predictable behavior. servers referenced by the handle; it also releases all memory associated with the handle. Instance Ha ndles Each instance handle should be associated with a The creation of an object reference is performed by corresponding appl ication hand le. This association cal l ing the ACAS_CreatelnstanceHandle routine. allows the application handle to be reused when ACA Services (l) creates an instance handle from sending additional requests to the appl ication con­ the information passed as arguments to the routine, cerning the data object. An application handle asso­ (2) allocates memory to the handle and manages ciated with a cache entry can be used to make the this memory, and (3) sends a message to a storage request. Failure to find the application in the cache class, if one was specified. could indicate that the appropriate invocation flag To avoid creating numerous copies ofan instance should be used to obtain an application when call­ handle, each with its own memory, a cache ing the ACAS_Invoke.Method routine. of objects should be used . This is especially As described, proper handle management can true in CASE environments that use the data­ result in better performance, better resource uti­ oriented paradigm. Each object structure con­ lization, and predictable behavior within the envi­ tains poimers to both the previous and the next ronment. However, hand le management does not object strucmre in the queue. The structure also deal with how to create an object reference that, contains values for the location and reference when presented to an application on a remote host, data fiel.cls that were passed as arguments to the is in a format native to that platform. For this capa­ ACAS_CreatelnstanceHandle routine and, thus, bility, we must turn to storage classes.

Digital Te chuical jou rnal Vo l. 5 No. 2 Sp ring t

Data Tra nsformations Us ing the method associated with the Setlnstancc mes­ Storag e Classes sage must determine the data class of the object ret� Distributed CASE environments, whether homoge­ erence, as wel l as transform the reference data into neous or heterogeneous, must concern themselves its network fo rmat. The determination can be made with the representation of object references that in a number of ways, the most common of which are shared among different applications. File speci­ is to base the class on the ex tension of the file. fications exemplify this problem. Given mu ltiple AJtbough not the most accurate method of deter­ hosts, it is unlikely that two hosts have the same mining the class, this appro

lo solve the problem, the environment can uti­ event manager, which acts as a central registry of lize the fu nctionality provided by ACA Services stor­ active applications and their associated activities, age classes. Storage classes provide a mechanism can provide a simpk fo rm of work flow manage­ for translating an object's reference data from one ment within the environment. However, the event file system representation to another. A solution manager is used only in a limited capacity in the to the scenario described involves implementing a COHESION integrated environment. In COl lESION, set of methods that would be executed when the the event manager is notified each time an applica­ object reference uses a storage class. tion is started or stopped in an activity. The applica­ The SC_COHESION storage class is a CASE-specific tion provides an application handle to itself, which storage class, which is a refinement of the SC_FILE is used by the event manager to notify the applica­ storage class provided by ACA Services. As a refi ne­ tion of events of interest. The usc of the event man­ ment, SC_COHESION in11erits Jll the messages defined ager removes the need fo r an application to fo rward on its parent storage class, including the messages certain messages, as a result of an event in the envi­ Setlnstance and Getlnstance. The methods fo r these ro nment, to all applications with which it has been two messages provide an implementation for map­ communicating. Removing the need to fo rward ping file system specifications from platform­ messages reduces both the chances of loops fo rm­ specific formats to platform-independent fo rmats ing in a set of applications ancl any communication and back again. The storage class methods do this by dead locks between applications. utilizing device and directory information, called directory mappings, fo und in the context object. Events and 11'iggers The directory mappings stored in the context On registration, an application can express interest object provide a means to associate a physically in being notified about particular events. Events shared directory path with a network path name. are categorized into two classes: system events The network path name is a platform-independent and appl ication events. System events affect the name that, when presented to a remote platform, overall operation of the environment. These events can be mapped into a fo rmat native to the platform include shutdown and changes in activities. All receiving the request. A network path name Jnd its applications in the C:OHESJO:\i environment are mapping are stored as an attribute-value pair in the notified of the system events for activity shutdown, PATH NAME_REGISTRY table of a context object. iconification, and deiconification. Application The directory mapping function:�lity allows ref events occur when the state of an object in the envi­ erences to file objects to be passed between appli­ ronment changes. file modification or completion cations on different hosts in a way independent of of a build step are typical examples of application the platform. This same scheme can also be used to events. Other applications in an activity can use convert object references in object identifiers, such these events fo r synchronization or as notifications as ATIS element IDs fo r use with the CDD/Repository that cause a change in behavior. Such notifications software. In the implementation fo r the file system, have traditionally been called triggers.

9H Vol. 5 No. 2 .\/Jri11g I'J'.J.i Dip,ita/ Te clmicaljo urnal CA SE Integration Using ACA Services

For ex ample, in a simple build system such as the Services provides the infrastructure necessary to make utility, events can create a work flow that integrate the large number of existing applications would automatically compile and link an appl ica­ into distributed, heterogeneous environments. tion when one module changes. If the build process completes successfully, the work flow automati­ Acknowledgments cally starts the debugger to debug the newly built The author wishes to thank Jackie Kasputy, Chip executable file. If the build fa ils, the work flow Nylander, and Gayn Winters fo r their invaluable loads the faulty module into a program editor and insights and contributions on distributed, multi­ positions the cursor to the line where the error vendor CASE environments. occurred . References Summary E. Yo urdon, Modern Structured Analysis (Engle­ ACA Services can be used to resolve many problems 1. wood Cliffs, NJ : Yo urclon Press, 1989). encountered in a distributed, multivendor environ­ ment. The object-oriented approach provided by 2. DEC ACA Services Sy stern Integrator and ACA Services can aid in the construction of a CASE Programmer's Guide (Maynard, MA: Digital environment that promotes the plug-and-play con­ Equipment Corporation, Order No. AA-PQKMA­ cept across a number of diffe rent platforms and TE, 1992). network transports. ACA Services provides a means 3. G. Booch, Object Oriented Design with Applica­ of developing client-server applications and of tions (Redwood City, CA: Benjamin/Cummings abstracting the network dependencies away from Publ ishing Company, 1991) the developer. This fe ature, together with the use of storage classes and data marshaling, can help to 4. R. Wirfs-Brock, B. Wilkerson, and L. Wiener, exchange information in a heterogeneous environ­ Designing Object-Oriented Software (Engle­ ment. At the same time, ACA Services can provide a wood Cliffs, N.J: Prentice-Hall, Inc., 1990). consistent programming interface to al l compo­ Services Reference klanual (Maynard, nents in the system. The dynamic nature of ACA 5. DEC ACA MA: Digital Equipment Corporation, Order No. Services allows new components to be added to the AA-PQKLA-TE, 1992). environment without the need to rebuild the entire environment. The flexibility of ACA Services al lows 6. ). Liu, "Future Direction fo r Evolution of IRDS its use to construct a CASE environment regardless Services Interface," X3H4/92-161 , Proposed spec­ of the integration paradigm used and while sup­ ification submitted to ANSI X3H4 and ISO IRDS, porting a number of interaction models. ACA 1992.

Digital Te chuical jo urual Vo l. 5 No. 2 .Sjn·ing !')')) 99 David Ascher I

DEC @aGlance-Integration of Desktop To ols and Manufacturing Process Iriformation Sy stems

Th e DEC @a Gla nce architecture supports the integration of manufa cturing process information sy stems with the analysis, scheduling, design, and management tools that are used to impmve and manage production. DEC @aG!ance software com­ prises a set of run-time libraries, an application development tool kit, and exten­ sions to popular sp readsheet applications, all implemented with Digital's object-oriented Application Control Architecture (A CA ) Services. The tool kit helps developers produce DEC @aGlance client and server applications that will interop­ erate with other independent�y developed DEC @aGlance applications. Sp readsheet extensions (a dd- ins) to Lotus 1-2-3 fo r Windows and to Microsoft Excel fo r Windows allow users to access real-time and historical data from DEC @aGlance servers. With DEC @aGlance software, control engineers and other manufacturing process profe s­ sionals can use fa miliar desktop tools on a variety of platforms and have simple, interactive, and tra nsparent access to current and past process data in their plants.

At a chemical plant that has been producing nylon data to be analyzed, visualized, manipulated, and using the same process fo r over 35 years, the lead explored in ways that support creative problem contro l engineer told an interviewer that what he solving. Getting timely information about the pro­ likes about his job is that "it is totally different every cess into the appropriate problem-solving tools is, day." 1 To an outside observer, the operation of a however, difficult. This paper begins with some process plant, such as a refinery or paper plant, background about manufacturing process infor­ appears to be an unchanging flow of materials mation systems and the need for access to system into a tightly controlled and repetitive process data. The paper then describes the development of that produces a continuous flow of unvarying DEC @aGlance software and the choice and use of product-24 hours a day, 365 days a year. In reality, Application Control Architecture (ACA) Services to the operation of these plants is far more complex solve the problem of integrating independently and challenging, involving constant adjustment to developed applications in the manufactu ring changing conditions, aging equipment, and varia­ space.2 tions in raw materials, as well as constant monitor­ ing fo r equipment malfunctions. Background The operation of a large process plant involves In large manufacturing facilities, the production the functioning of numerous valves, switches, process is controlled through the use of advanced pumps, other actuators, and sensors measuring and automation systems. These systems may track thou­ controlling the levels, pressures, temperatures, and sands of temperatures, flows, pressures, and levels flows of various materials through a complex series and can drive hundreds of pumps, valves, and other of pipes, tubes, tanks, ancl vessels. Tn addition to actuators. To implement control strategies, such detecting and managing failures in these compo­ systems may compute large numbers of complex, nents, a large proportion of the personnel in the dynamic control algorithms. Usually, additional sys­ plant is involved in process and product improve­ tems measure various physical properties of the ment. The personal computer or workstation product, such as color, weight, viscosity. thickness, and an array of sophisticated desktop tools allow and moisture content. Supervisory control systems

100 14>1. 5 No. 2 Spring 1')')3 Digital Te chnical ]o unwl DEC @aGlance-Integration of Desktop To ols and Manufacturing Process Information Sys tems

often coordinate parts of a complex process, as control systems may exist to implement control well as implement higher-level control and produc­ algorithms that reflect changes in the markets fo r tion strategies and keep historical records of key products, market opportunities, and fluctuations in process variables. raw material availability and composition, along The control of a large plant is usually imple­ with the information about the process that is sup­ mented through strategies that allow the control plied by the lower- level systems (high-level con­ problem to be divided into smaller parts, as illus­ trol). Scattered among these levels may be various trated in Figure 1. Each piece of the system is additional systems that schedule preventive main­ responsible fo r the control of a subsystem (e.g., tenance, identify equipment fa ilures, and advise on steam generation and distribution, or cooling flu­ process improvements-all based on information ids), a part of the process (e.g., premixing, material about process from the other systems in the plant. storage, or reaction), or an area of the plant (e.g., Distributed control systems include an operator packaging line, product stream, or finished goods console that consists of multicolor displays, push management). Within each subsystem, there is typi­ buttons, warning lights and buzzers, a touch screen cally a hierarchy of control. The lowest-level com­ or trackball, and industrialized keyboards with as ponents control activities that require responses many as a 36 special fu nction keys. The displays within less than a second to as much as one minute allow an operator to oversee all parts of the process (direct control). The next level of systems control fo r which the operator is responsible. Ty pical dis­ activities that require responses within Jess than plays show recent trends of key variables and mimic a few minutes (d istributed control). Above this diagrams showing the current state of the manufac­ level of response are systems that control activities turing equipment (e.g., valve positions and tank lev­ that may not change for long periods or that imple­ els) and of the material flowing through the ment control algorithms that involve measurements process. The keyboard and other input devices from more than one lower-level system (super­ allow the operator to select displays, request visory control). At the plant level, additional reports, and modify control settings. Response to

HIGH·LEVEL ARTIFICIAL ADVANCED CONTROL INTELLIGENCE CONTROL SYSTEMS PROCESS HISTORIAN

SUPERVISORY � CONTROL HISTORICAL PROCESS DATABASE

REAL·TIME PROCESS DATABASE DISTRIBUTED DISTRIBUTED DISTRIBUTED CONTROL CONTROL CONTROL SYSTEM SYSTEM

DIRECT CONTROL LOOP LOOP LOOP CONTROLLER CONTROLLER CONTROLLER

SENSORS AND ACTUATORS

PUMP GAUGE VALVE

Figure 1 Ty piectl Levels of Control in a Process Plant

Digital Te chnical journal vb l. 5 No. 2 Sp ring 1993 101 Application Control

problem or alarm conclitions and modification of • Resource optimization. Usually, process plants the process to change the product are effected are capable of producing different grades of through the console. product, as well as mixtures of end products. Process operators are responsible for maintain­ An oil refinery, fo r example, produces various ing the routine operation of a plant. Operators use grades of fuel oil and also home heating and the control system to change process parameters in lubricating oils, aJJ from a single process. While order to produce different mixes or variants of the the operators adjust the equipment to control product, or to respond to an equipment fa ilure by the product mix, a process planner or produc­ rerouting material around nonoperational process tion manager determines the best production equipment. schedule based on customer orders and the effi­ To perform their functions, manufacturing plant cient use of the process equipment. production and engineering support personnel Process information is available to operators (e.g., control engineers, process engineers, produc­ and engineers who are trained to work with the tion supervisors, production planners, mainte­ various control and management systems in the nance supervisors, and manufacturing engineers) plant. Using proprietary tools fo r each system also need access to information in the control and allows reports to be generated and specific types supervisory systems. These professionals regu larly of analyses to be performed on the data contained access inforn�ationcontaine d in multiple manufac­ within each of these systems. However, extracting turing systems and have an occasional interest in the data from these systems to an engineer's desk­ particular measurements or parameters within top fo r analysis by generic tools, such as spread­ other parts of the process. The fu nctions of these sheets and statistical analysis packages, is difficult manufacturing plant personnel include or even impossible. Lack of console- ami tool­ • Complex problem analysis and solution. specific training is another obstacle to accessing Locating sources of product or process variation process information. involves analyzing information from different parts of the process that may be under the con­ Manufacturing Process Information trol of different automation systems. Comparing Sy stems and Desktop Sys tems: the flow that exits one parr of the process Goals and Barriers with the flow that then enters the subsequent Production and engineering support personnel part, fo r example, could disclose a fa ulty flow want to be able to use the desktop tools of their meter, a previously unknown temperature con­ choice to explore and analyze data from manufac­ trol problem, or a leak. turing systems. Spreadsheets, simulation tools, report generators, visualization tools, statistical • Product improvement. Improving product qual­ analysis tools, planning tools, charting tools, and ity and consistency involves investigating how graphic-generation tools have aJJ become accepted the product is affected by existing variations parts of the array of computer-aided techniques and in the production process. For example, investi­ tools available to the contemporary knowledge gation may involve the study of a process vari­ worker. The interactive, easy-to-use graphical user able that cannot be measured directly but can be interface, which can run on relatively inexpensive calculated from the values of other process vari­ platforms under the complete control of the end ables. Examining sets of variables over time and user, has not only encouraged the wide use of these exploring possible relationships may result in desktop tools but also enhanced their effectiveness. discovering combinations of process variables These tools stimulate professionals to creatively that yield unexpected effects on product explore the character of large amounts of data and attributes. thus support the discovery of previously unex­ • Process improvement. Improvements in process pected patterns and relationships. yield and process rel iability and reduction of The fu rther an end user's primary function is waste and hazardous by-products may involve from production, the more likely it is that such a the study of historical data va lues from the pro­ user will want access to multiple systems. System cess. Studying measurements obtained from interfaces, which may differ widely and arc gener­ multiple control systems may also result in pro­ aLly oriented toward production use, discourage cess improvements. users from making ad hoc inquiries into the system.

102 Vo l ) No. l Sp ring /993 Digital Te chuical jo urual IJ/;'C @aGlance-Jnteg ration uf Desktop To ols ancl i'vlcm ufacturing Process Information Sy stems

Conse()uently, manufacturing system data may not simplifies transcription bur still requires that a be easily accessible to users of the many desktop specialist extract the data using proprietary inter­ tools available fo r such purposes as decision sup­ fa ces. In addition, the data may need to be con­ port, research, analysis, and simulation. verted from string to numeric fo rmat to be usable To day, the use of data from the manufacturing with in a particular spreadsheet. process in planning, reporting, and managing the The International Organization fo r Standard­ operation of a plant is hampered by the difficulty in ization standard Manufacturing Messaging Sp eci­ accessing the data from plant control and process fication (IS9506 or MMS) addresses the problem information systems. It is typical fo r a production of data exchange between applications and dedi­ supervisor who needs data from a control system cated manufacturing systems (referred to in the to reCJuest the data from a process operator. Once standard as manufacturing devices) .H Although in hand, the data is then manually entered into a some manufacturers of programmable controllers spreadsheet or other desktop tool fo r analysis. The (that is, dedicated control systems that are pri­ results of the analysis often reCJuire entering new marily used in discrete manufacturing industries) parameter values into the control system. Th is task offe r MMS capabilities, the process industry manu­ is typically performed by another person, trained fa cturers and their control system suppliers have to use the control system, who transcribes the val­ not widely accepted MMS. Use of the standard ues from a hard copy of the tool's output. The pro­ has been perceived as expensive, inefficient, ami cess is time-consuming, costly, and error prone. oriented primarily toward the needs of discrete Problem-solving activities are limited to those that manufacturing. A committee of the Instrument can just ify the trouble and expense involved in sim­ Society of America (!SA) is developing a companion ply accessing the data. standard (ISA 72.02) to use with MMS in com muni­ cating with distributed control systems in process manufacturing 9 An important aspect of this pro­ Ex isting In tegra tion Effo rts posed standard is a data model that describes the The desire to use data from the control systems organization and types of data in a distributed con­ to analyze and improve the understanding and con­ trol system. trol of the manufacturing process has spawned a variety of effo rts since the late 1980s. This work bas attempted to ease the transfer of information Requirements fo r Integration between computing systems and control systems. Digital designed the DEC @a Glance architecture not However, the resu lting products and standards are to be a generic appl ication integration mechanism not oriented toward supporting ad hoc in()uiries but rather to support the integration of popular and, therefore, are not widely used. desktop tools with manufacturing process informa­ Many currently available manufacturing systems tion systems. An application that complies with the may be connected to the plant network, but witb­ architecture can be installed on any system within out standard higher-level interfaces, access to these a network, run, and immediately exchange data systems re mains limited.'-- Through such network witb other com pi iant applications. Some key char­ connections, some manufacturing systems pro­ acteristics of the environment that helped to drive vide limited access to OpenVMS and/or DOS system the architecture are users. However, the access is typically restricted to • Multiple vendors. Al though , MS-DOS personal the use of unique, proprietary programming inter­ computers are the most popular desktop envi­ faces or to proprietary tools targeted at performing ronment, VA Xstation, Macintosh, ancl UNIX work­ a manufacturing-related function, sucb as statistical stations have a clear presence in particular CJU

D(�ital Te chnical journal Vn l. 5 No . .! Sp ri11g 19')3 103 Application Control

• A large variety of desktop applications and user and the state of materials within the process. These interfaces. Each class of deskto[) application users have little or no interest in such aspects has a different way of interacting with users. as network topologies and protocols, operating Spreadsheets, fo r example, have very different systems, and byte ordering on different hardware user interfaces from statistical packages and data platforms. visualization packages. Some applications have Some major concerns of the end user that the elaborate macro languages, whereas others are architecture should address are almost entirely graphically driven. • The identity of the manufacturing control sys­ • Multiple types of large networks. In the typical tem. Generally, a large plant is controlled process manufacturing facility, large networks through the use of several control systems, each are already in place. While many plants use of which might control a part of the process, DECnet fo r their network, an increasing number such as refining or packaging, or an aspect of the of plants are choosing to use the transmission plant operation, such as steam distribution or control protocol/internet protocol (TCP/IP), waste reprocessing. A particular data point and some plan to migrate to Open Systems resides in a single manu facturing control system. Interconnection (OS!) networks (including The user should be able to specify precisely Digital's DECnet Phase V) from multiple vendors. which manufacturing system is to supply the PC LAN s are also becoming popular. data values. The architecture should be capable of establishing a relationship with the specific • Conservative computing strategies. Large application that owns the data of interest to the manufacturing facilities cannot affo rd to halt user. The end user should not have to specify operation to make major changes in their either the network node, the operating system, production-related computing systems and net­ or the hardware platform on which the applica­ works. Such facilities look to standards -based tion is running. Neither should the end user have products as a way of achieving stabil.ity and of to specify the network communication proto­ ensuring confidence in the longevity of a partic­ cols required. ular technology.

• The length of the relationship between the desk­ Architectural Issues top tool and the manufacturing control applica­ tion. The relationship should be able to remain Simply stated, the problem that the DEC @aGlance active for multiple transactions to al low end architecture attempts to address is, how can a users work interactively with desktop tools set of existing applications running on heteroge­ to to explore possibilities. For example, end users neous platforms, distributed across a variety of may want to examine different data points or the networks, and developed by different vendors same data point over various time intervals. (with only peripheral interest in integration) be Thus, usage of a desktop tool could involve mul­ easily integrated' A good understanding of both tiple requests for data from a manufacturing the nature of the applications involved and how end control application. Establishing a relationship users would use them if they were integrated is between applications over a network is time­ important fo r evaluating potential answers to the consuming, and therefore establishing long­ question. lived relationships wo uld be advantageous. The The applications that we considered integrating ability continuously monitor a set of points can be divided into two grou ps: those that "own" to and have their values reported on a time or manufacturing data, i.e., the manufacturing control change basis is another desirable fe ature that systems, and those that are consumers of that data, wou ld require the establishment of long-l ived i.e., the desktop tools. From the viewpoint of an relationships. end user, some aspects of the relationship between a desktop tool and a manufacturing control applica­ • Multiple access to the applications. Applica­ tion must be considered in order to accomplish tion relationships should not be exclusive. work goals. End users in this environment are Each application should be able to !lave concur­ primarily concerned about the manufacturing re nt relationships with several partner applica­ process, the equipment controlling the process, tions. Each desktop tool may requi.re data from

104 HJI. 5 No. 1 SfJrinf, I'J')i Digital Te e/mica/ jo urua/ DEC @aGlance-Jntegra tion of Desktop To ols and Manufacturing Process InforrnationSy stems

several manufacturing systems, and conversely, Usage Model several users of desktop tools may need to To help us understand how a user might go about access the same control system simul taneously. employing the capabilities that we were consider­ The relationships between desktop tools and ing, we developed a simple usage model. We based manufacturing control systems is illustrated in the model on the scenario that an end user makes a Figure 2. series of ad hoc inquiries into the state of a process. We assumed that the user was fa miliar with the • The data model. Applications should agree about manufacturing process but not necessarily expert how to reference data and about data types. in all the details of the process. The user would Within the context of this environment, a rela­ know, fo r example, what the major areas of the tively simple data model exists in the draft stan­ plant were called and what functions they per­ dard iSA 72.02. Data should always be converted fo rmed but might not know the internal reference to types appropriate to the local system and to identifier of every flow meter in each control sys­ the application. A spreadsheet user should not tem. We fo cused on how the user of a spreadsheet have to manually convert strings into numeric tool might reasonably expect to proceed to get data values. into a spreadsheet and how services that we might • The user interface. Application integration provide could aid in exploring the data. should not require the use of any particular The information within a manufacturing system desktop user interface, such as the X Window consists of the many parameters and measurements System or OECwindows software, or even the that the system uses to monitor and control the pro­ existence of a windowing system. Also, the user cess. Generally, this data is organized into blocks, interface of the manufacturing data application each one related to a particular part of the process, should be of no concern to the desktop user. such as flow, level, temperature, or pressure. As the typical data block in Figure 3 illustrates, every block has a unique name or tag that can be used fo r reference purposes. In control systems, tag names are assigned as part of the configuration. Large plants use a naming con­ vention to ensure the assignment of unique tag names to the thousands of blocks spread through­ out the plant and over several control systems. In

Single Client-serverConnection addition to the tag, the block contains attributes such as the parameters of the control algorithm, measured input values, unit conversion algorithm identifiers. The data model proposed by the !SA 72.02 committee describes seven types of blocks, each with a standard set of attributes with associ­ ated names and data types.

Multiserver Connection

BLOCK TYPE TAG NAME DESCRIPTOR ANALOG PROCESS VALUE ANALOG CURRENT VALUE HI ALARM LIMIT LO ALARM LIMIT PROPORTIONAL INTEGRAL DERIVATIVE ENGINEERING UNIT Multiclient Connection

Figure 2 Relationships between Desktop To ols Figure 3 A Ty pical Data Block in a and Manufacturing Control -�)!stems Manufacturing Control Sys tem

Digital Te chnical jo urnal Vo l. 5 No. 2 5/Jring l.'J 'J.) lOS Application Control

This usage model allows a user to easily deter­ Configuration Information mine the tag names recognized by a particular One service is defined fo r requesting the tag names manufacturing system. To examine the data values that the server finds in the control system's associated with a specific tag, the user needs to database. An additional service returns a list of know the valid attributes. (Al l blocks do not have attribute names tbat are defined fo r a specified tag the same attributes, e.g., an analog loop control name or a list of tag names. block has more attributes than a simple digital mon­ itoring block.) Once the tag names and their val id Data Va lue Exchange attributes are known, the user can inquire about Services are defined fo r reading and fo r wntmg current values as well as historical values. current and historical data point values. For current The use of operating prototypes, including simu­ values, services support reading or writing either lated servers and a simple spreadsheet, advanced a Jist or a table of data point val ues. A read or write the development of the usage model. The proto­ Jist request specifies pairs of tag names and types were shared with potential end users and attributes. A read or write request fo r a table of (lata application developers at customer visits and indus­ point values specifies a I ist of names and a I ist try trade shows. Feedback obtained from demon­ of attributes. The table of data points consists of strations and discussions of the usage model helped all tag names paired with their corresponding expand and refine the services. attributes. Both the list and the table requests can be used to read or write a single data point, collaps­ Architecture ing to either a list or a table of one data point.

The DEC @aGlance architecture defines two kinds By using the DEC @aGJance services to get lists of of applications, a set of services for accessing data tag names, attribute names, and data point values, in the control systems, a data specification model, and the name of a server, an end user can generate and some basic types of data. The application a wide range of ad hoc queries without knowing classes are (1) manufacturing data servers and much about the control system in advance. A com­ (2) cl ients. Typical manufacturing data servers are mon data point attribute is the descriptor, which the manufacturing control system applications. characterizes the function of the data point, e.g., Typical clients include desktop tools such as south tank level. Thus, it is a fa irly straightforward spreadsheets and statistical analysis tools, as well as task to use DEC c a(; lance services to build a list of production planning, production scheduling, and tag names and descriptors that provide a basis fo r other production management applications. An further inquiries. application may be a client in relation to one appli­ The services fo r historical data values are defined cation and a server in relation to another. to deal with tables of historical values for a list of data points. Historical data service requests specify A data point is specified to DEC @aGiance appli­ I cations by the name of a server, a tag name, and an a ist of tag name and attribute pairs and a time attribute name. A data point has a current value and specification that is applied to all the data points. may also have historical values (if the manufactur­ The time specification consists of a start time, a ing system has a historian capability). A current time interval, and the number of intervals fo r which value is the most recent available value of a parame­ values are to be returned. ter or measurement within the system. A historical value is a value that the data point bad at some time Mo nitoring in the past. A historical value is specified by the Monitoring is useful fo r reading the values of a name of a server, a tag name, an attribute name, and set of data points at intervals in time or when a sig­ the time associated with the value. nificant change in value occurs for any of the data The services defined by the DEC @aGlance archi­ points. A graphical display program can run on tecture fall into one of four fu nctional categories: a desktop system and make minimal use of the net­ configuration information, data value exchange, work and computing resources while maintaining monitoring, or management. Each service defines an accurate representation of what is occurring in an operation that may be requested by one applica­ the manufacturing process. Monitoring could also tion of a partner appl ication. The services defined be used to update a spreadsheet at regu lar time are not necessarily the same fu nctions that an end intervals or whenever a particular process variable user requests. changes.

106 Vu l 5 No. ] .�jJri11g 1')93 Digital Te chnical jou rual DEC @aG/ance-Jntegration of Desktop To ols and Manufacturing Process lnfonnation Sy stems

No standard definitions exist fo r what consti­ support future enbancements to the DEC @aGlance tutes a significant change in value. Definitions sup­ architecture. ported fo r various systems include (a) detection of The DEC @a Glance architecture al lows an existing change outside of a specified range or "dead hand," desktop tool to be integrated with existing manu­ (b) change by more than some percentage of the facturing control systems, as shown in Figure 4. The previously reported value, and (c) change by more architecture effectively combines the fu nctional than some percentage of a fL>:ed value. Therefore, capabilities of the desktop tool fo r analysis, visual­ the service is defined to support moniroring and ization, computation, etc., with the capabilities of reporting of changes on a time basis or on some the manufacturing control system fo r monitoring other basis that is specific to the data server appl i­ and controlling a manufacturing process. The indi­ cation. Whenever the requested monitor condition vidual applications were , of course, originally is fu lfilled, the data server application uses a moni­ designed and written withou t any knowledge of tor update service to send the new data point val­ each other's existence. Therefore, to fa cilitate inte­ ues to the original cl ient application. Since the gration efforts, implementation of DEC @aGiance server initiates monitor update requests, the usual software shou ld localize and minimize required relationship between the client and the server is changes to the applications. temporarily reversed. A network protocol such as DECnet, the transmis­ sion control protocol/internetpr otocol (TCP/IP), or Ma nagement one of the local area network (LAN) protocols could Connection management services are provided to have provided the network services required establish a connection, to terminate a connection, fo r DEC @aG!ance's interappl ication communica­ and to test a connection. tions. However, this approach lacks a mechanism fo r locating servers on the network, requires Implementation Considerations DEC @aGlance to support the multiple network Using ex isting networking and application integra­ protocols that exist in the manufacturing environ­ tion technologies to implement the DEC @aGlance ment, requires DEC @aGiance to include data type architecture was important both in terms of conversion between application platforms, and reducing development efforts and improving com­ necessitates the development of monitoring and patibility with existing environments. Technol­ management tools unique to DEC @aGlance. A bet­

ogy used in the implementation had to provide ter approach is to use an existing product that is as many as possible of the capabilities described available on an appropriate set of platfo rms, sup­ in the architecture while imposing minimal restric­ ports an appropriate set of networks, and already tions on the end-user operating and network solves these problems. environments and on the developers of the appli­ A remote procedure call (RPC) mechanism cations. In addition, it was desirable that the under­ appears to have many of the capabilities that lying technologies offer capabilities that could the DEC @aGlance architecture requires. R.PC

STATISTICAL OTH ER SPREADSHEETS ANALYSIS GRAPHICS (AI. AVS, ..)

VMS MACINTOSH VMS MS-DOS 0��RIX/OSF ULTRIX /OSF1 u � ._ ._ - I � ULTRIX /OSF VMS ULTRIX /OSF MS-DOS ULTRIX/OSF VMS ULTRIX /OSF VMS SUN OS I I I l I I PROCESS PROCESS REAL-TIME SUPERVISORY CONTROL DATA EXPERT SYSTEM SYSTEM HISTORIAN SYSTEM

PROCESS DATA SERVERS

Figure 4 Integrating Desktop To ols and Ma nufacturing s: vsterns

Digital Te clmical]ourual Vol. 5 No 2 .\)iring !')').) 107 Application Control

mechanisms provide fo r location of a partner or server can be defined tO offer the capabilities of server appl ication, and they provide data type con­ a DEC @aGiance data server as well as additional version and reliable network services. The RPC capabilities. The older DEC @aGiance servers would model of application integration, however, is actu­ actually provide the DEC @aG iance services while, ally more appropriate for the distribution of a single transparent to the client applications, the new application across multiple systems in a network. server would make the new capabilities available. This use implies a simple, static relationship ACA Services has been selected as a major com­ between the parts of an application: one part is ponent of the Object Management Group's (OMG) always a client that requests the execution of a pro­ Object Request Broker, which in turn has been cedure, and the other part is always an RPC server selected as a part of the Open Software Founda­ that executes the procedure and returns the results. tion's (OSF) Distributed Computing Environment In such a relationship, each request generates a sin­ (DCE). ACA Services is designed to be independent gle response. This model would be poorly suited of the type of network that provides the interappli­ fo r supporting the DEC @aGlance monitoring ser­ cation com munications services and currently vice. Wh en DEC @aGiance was being developed, no works over both DECnet and TCP/IP networks, the commercially available RPC implementation ran on networks most commonly fo und in manufacturing the key platforms, the OpenYMS and Microsoft environments. Therefore, applications using ACA Windows environments. Furthermore, no one had Services need not be concerned about network announced their intention to produce a portable communications. implementation that would be available on the ACA Services is supported on the OpenVMS, wide range of platforms that we considered impor­ Microsoft Windows, Macintosh, and SunOS operat­ tant fo r future versions of DEC @aGiance software. ing systems, the most often used platforms in this Digital's ACA Services was chosen as the basis application space. In fact, ACA Services is the only for implementing DEC @aGlance software because application integration mechanism currently avail­ it implements an application integration model able on all these platforms. Moreover, ACA Services that closely matches the requirements of the supports the kind of asynchronous services DEC @aGi ance environment. ACA Services supplies required by DEC @ia(;lance. many capabilities required of the integration mech­ Although it provides many important compo­ anism including nents of the required integration service, ACA Services does not completely solve the integration • Abstraction of functions from implementations problem. ACA Services is a tool intended to be used • The ability to encapsulate existing applications to integrate applications; it does not define the data model nor does it define the set of services that • Location of partner applications on a variety of applications are provide. Application integrators networks to are expected to define (1) the classes of applica­ • Establishment and management of re i iable, long­ tions that provide sets of services, (2) the services, lived communication links (3) and the meaning and type of data to be • The ability to easily add new applications to the exchanged by applications using the services. system

• The ability to easily install new versions of exist­ DEC @aGlanceSo ftware: ing applications in the system The To ol Kit and Add-ins

• The correct handling of data q'pe conversions As shown in the DEC @aGlance component diagram ACA between heterogeneous systems in Figure 5, DEC @aGlance software uses Services as a basic application integration facility. • Commercial availability of portable interfaces Above ACA Services, DEC @aGlance adds definitions on OpenYMS, Microsoft Windows, Macin tosh, of a class of manufacturing data server applications and a wide variety of UNIX platforms from multi­ (servers), a set of definitions of the services pro­ ple vendors vided by the servers, and definitions of the data ref The class hierarchy capabilities of A CA. Services erence model. allow the creation of new combinations of appli­ ACA Services provides a general capability to cations integrated to provide new capabilities integrate sets of applications. DEC @aGlance soft­ without additional coding. Thus, a new class of ware provides a set of routines that are specifically

108 HJ/. No. J .\jJring I'J'J. i Digital Te chnical jo uruaf DEC @aGlance-lntegration of Desktop To ols and Manufacturing Process Information Sy stems

DESKTOP APPLICATION

CALLABLE INTERFACE

(SLOTS FOR DATA ACCESS ROUTINES) IMPLEMENTATION

l DEC @AGLANCE CLIENT CLIENT LINK TEMPLATE DEVELOPER'S KIT J ACA SERVICES

NAS NETWORK TRANSPORT (DECNET, TCP/IP, ETC.)

ACA SERVICES

SERVER LINK TEMPLATE DEVELOPER'S KIT

DEC @AGLANCE SERVER

1 (SLOTS FOR DATA ACCESS ROUTINES) IMPLEMENTATION

CALLABLE INTERFACE

PROCESS DATA SOURCE

Figure 5 DEC @a Glance Components designed to simp! ify the implementation of the set DEC @aGlance clients. Proper functioning includes of services that DEC @aGiance supports. For server verifying the installation and configuration of the applications, DEC @aGiance software supplies a set network and of the ACA Services and DEC @aGlance of callback points, as well as callable routines fo r run-time components of the systems on which the declaring callbacks, fi ltering strings, and support­ cl ient and server applications reside. ing monitoring activities. For client applications, Software add-ins, i.e., extensions, fo r two pop­ DEC @aGlance software supplies a set of caLlable ular spreadsheet applications, Lotus 1-2-3 for routines fo r requesting each of the defined ser­ Windows and Microsoft Excel fo r Windows, are vices, as well as cal I back points in support of moni­ also DEC @aGlance products. These add-ins allow tor updates. users of the spreadsheets to request data from man­

The DEC @aCilance server I ibrary also supports ufacturing data servers by means of the spread­ a test connectivity capability used to verify that an sheets' macro fa cilities. The add-ins provide a interapplication re lationship can be established to dialog box to gu ide untrained users through the the server application. This capability simplifies process of constructing a DEC @aGiance macro. the diagnosis of problems encounrerecl during both Once built, a macro can be executed one or more server development ancl client-server installation. times, modified if necessary, and saved in a work­ To reduce dependence upon properly written sheet for reuse at some other time. server code, the test connectivity capability oper­ ates entirely within the library. Thus, once a server To ol Kit calls the DEC c aGiance initialization routine, and The tool kit was developed to encourage the rapid if the server is still running, this service should and successful development of DEC @a Glance appli­ function properly in response to requests from cations by third parties. Successfu I applications are

1993 Digital Te chnical jo urnal Vo l. 5 No. 1 Spring 109 Application Control

those that interoperate with other DEC @aGlance • Cancel a monitor request applications upon delivery to a customer site with • Initiate a session no additional coding, no application recompila­ tion, and no application re building. • Te rminate a session The key components of the tool kit are • Execute a server-specific request • A DEC @aGlance client or server library • Te rminate the server • Example code The control system-specific section consists of • ACA Services definition files for the DEC c aGiance code modules that execute calls to the control sys­ class and methods tem application programming interface (AJ>l). These modules have to convert parameters to and from • Simple test fa cilities the DEC @aGlance fo rmat and the control system­ • Tbe DEC @aGlance Programmer's Guide�<> specific to rmat. The entry point of each module is declared as a callback point during initialization. The ACt\ Services definition fi les contain the In addition, callable routines are provided fo r information required to define the manufacturing sending monitor updates and fo r session manage­ data server class and the services that members of ment. The DEC @aGiance section of the server is the class support. Supplying the definitions in this contained entirely within a library of callable form ensures strict consistency among all server server routines. This section handles inrentc­ and client developers with regard to these defi­ alI tions with ACA Services, including server registra­ nitions. The routines in the DEC @aGiance client tion and session management. It also handles the and server libraries use these definitions. The dispatch of incoming requests to the callback rou­ DEC c aGiance libraries contain all the code required tines and a number of housekeeping tasks t()r to establish and maintain an ACA Services session. which each server developer would otherwise have to develop and implement solutions. The Server Applications DEC @a

• Get a I isr of tag names a user interface fo r performing some class of generic fu nction such as decision support, statisti­ • Get a list of attribute names cal analysis, quality control. or production schedul­ • Get a list of c!ata point values ing. Other types of applications that could make use of process data, such as report generators, • Get a table of data point values batch schedulers, and maintenance tracking sys­ • Put a list of data point values tems, can also provide the basis of DEC @aGlance client appl ications. Adding DEC @aGlance support • Put a table of data point values to an existing tool allows the user to treat data fro m • Get a table of historical values DEC c aGlance manufacturing data servers like data entered manually or from other data sources. • Put a list of historical values A DEC @aGlance client application incorporates • Register a monitor request the DEC @a Gl ance client routine library, which

110 Vi>!. 5 No. 2 Sp ring !1)').) Digital Te e/mica/ jo urnal DEC @aGfance-Jntegration of Desktop To ols and Manufacturing Process Jnfonnation Sy stems

provides callable routines for initial ization and for mechanism, the application simply fo rmats the each of the fo llowing DEC l!!:aGlance services: request and calls the appropriate DEC @aGlance service request routine. Upon completion of the • Get a I ist of tag names routine, status (and if requested, data) is returned from the server application. If data is returned that • Get a list of attribute names is to be further processed by the client application, • Get a list of data point values the application moves the data to its workspace in preparation for additional processing. • Get a table of data point values

• Put a list of data point values DEC @aGlance Lotus 1-2-3 fo r • Put a table of data point values Wi ndows and Microsoft Excel Add- ins

Whereas most manufacturing control systems pro­ • Get a table of historical values vide a cal lable library that allows the development • Put a list of historical values of applications that access the data in the system, some desktop tool applications have mechanisms • Initiate a monitor request that allow for extension of their capabilities in the

• Cancel a monitor request field . Spreadsheet appli cations such as Lotus 1-2-3 and Microsoft Excel support the use of add-in mod­ • Initiate a session ules to add external functions and external macro

• Te rminate a session capabilities. Add-ins fo r these two spreadsheets are available as DEC @aGiance software products. • Execute a server-specific request With the add-ins, spreadsheet users can access DEC • Te rminate the server most @aGiance services and thus can

• Te rminate the client • Fill a range of cells with a list of tag names from a server In addition, support routines help mon itor updates.

To support the DEC @aGlance monitoring capa­ • Fill a range of cells with a list of attribute names bility, a client appl ication must have some server associated with a range of tag names in a server characteristics. Once a monitoring request has • Fill a range of cells with a list of data point values been initiated, the server issues monitor update

requests when the monitoring condition is satis­ • Fill a range of cells with a table of data point fied . The moniwr update requests are received by values, as shown in Figure 6 the cl ient application using the same callback • Write a list of data point values to a server me chanism that the server uses when servicing

client requests. • Write a table of data point values to a server A typical client calls the DEC @aGlance initializa­ • Fill a range of cells with a table of historical tion routine ami then continues to perform its nor­ values fo r a specific time interval mal functions. When a DEC @aGiance service is

requested through the user interface or other • Write a list of historical values

A B c D E

1 UNIT4 1 APV ASP ALMST DESC

TIC001 134.7 140.0 - FEED TEMP 2

3 LIC001 65.3 50.0 HIGH FEED LEVEL

4 FI001 185.8 - RATE + FEED RATE

5 FRC005 65.6 50.0 HIGH HIGH REFLUX RATE

6 TRC085 145.4 145.0 NONE REFLUX TEMP

Fig ure 6 A Ta ble of Data Po int Va lues in a Sp readsheet

Digital Te chnical jo urnal Vo l. 5 No. 2 SfJring 1993 111 Application Control

The interface fo r the add-ins was designed to sup­ a marketable product; ancl Mike Renzullo and Alan port ad hoc inquiries. A dialog box guides the end Ewald of the ACA Services Development Group, for user through the process of supplying the approp­ their support. riate parameters fo r a selected function. Where appropriate, defaults are suggested based upon the References previous inquiry. 1. This quotation was taken from the transcript Summary of an interview conducted by C. Kukla et al., who have pubI ished the results of their study DEC @aGiance software has been specifically in "Usability Turning Technology into To ols," designed to make it easy fo r users of desktop tools Designing Effective Sy stems: A To ol Approac!J, to access, explore, and analyze clata from dis­ P Adler and T. Winograd, eds. (New Yo rk, NY: tributed control systems, supervisory control sys­ Oxford University Press, 1992). tems, and other common systems used to run manufacturing processes. An analysis of the infor­ 2. DI:.'C ACA Services Sy stem Integra tor and mation environment and the ways in which end Programmer's Guide (Maynard, MA: Digital users want to access the data led to the refinement Equipment Corporation, Order No. AA-PFYUA­ 1992). of the architectural requirements. The analysis also TE, led to the decision to use ACA Services as the appro­ 3. CM 11-320 Ovi50N Us er Manual, Order No. priate mechanism for integrating desktop and man­ (Phoenix , AZ: Honeywell Industrial Controls ufacturing control appl ications. The creation of a and Automation, 1991). usage model and rapid deployment of prototypes were instrumental in the analysis. To promote 4. Computer/Highway Interfa ce Pa ckage widespread availability of plug-compatible appl i­ (CHIP) User Guide, Part No. D001093XOI2 (Marshalltown, Fisher Controls Interna­ cations that use DEC @aGiance, a developer's tool lA: kit was created. The tool kit contains libraries of tional, Inc, 1987) . DEC @a(�lance routines that both simplify and 5. AIM Co nnectivizy Software Us er's Manual encourage proper ancl consistent usage of ACA (Houston, TX: W R. Biles and Associates, Inc., Services to integrate DEC @aGJance applications. 1992). DEC @aGiance add-ins fo r the popular spread­ 6. SCADA sheet programs Lotus 1-2-3 for Windows and 512 Sy stem Description, Document 502.0001 Microsoft Excel fo r Windows were developed also. No. (Dallas, TX: Te xas Instruments, With the add-in, users can interactively explore Industrial Systems Division, 1988). data in plant manufacturing control systems from 7. PI Sy stem Plant Information Sy stem Te chni­ within a fa miliar spreadsheet, as well as write cal Overview (San teandro, CA: Oil Systems, reusable worksheet macros fo r performing Inc., 1990). repeated tasks like report generation. 8. Manufacturing Messaging Sp ecification, 9506 Acknowledgments ISO/IEC (Geneva: InternationalOrganiza­ tion fo r Standard ization/International Elec­ The author gratefully acknowledges the contribu­ trochemical Commission, 1990). tions of the members of the DEC @aGiance develop­ ment team: Judie Dow, Bob Harrison, Nick Miller, 9 Ma nufacturing Messaging Sp ecification: Ramesh Swaminathan, Patrick Ta ber, and the lead Compa nion Standard fo r Process Om trol, 72.02 NC: developer, Charl ie Trageser. l would also like to !SA (Research Triangle Park, Instru­ thank Steve Dawson, fo r introducing our group to ment Society of America, 1993). the problem and general ly educating me about the 10. DEC @aGlance Programmer's Guide (May­ process manufacturing environment; Chuck Kukla, nard, Digital Equipment Corporation, fo r introducing me to his research on how people MA: Order No. AA-PQB8A-TK, 1992). work in manufacturing and fo r his work with cus­ tomers and control vendors that helped lead to the design of the product; Jim Thompson, fo r pushing and pulling all the strings that it took at every stage of the effort to bring the concept to

112 vbl. 5 1\o. 2 Sp riug J')'J.i Digital Te clmical jo ur11al · mamaamar� · = ·: :: \:-�·� ::·� ;·�-: · �:. . . . .: . . . .r .: ... � . • • • � ;=i�:.. �.. • • • • . . .• • . • I • ._ '\ •; ,/' : ._ ._• . . . .. : .. : . . :·:. . : · ::: : :-�·:. .-.; - . ·: ·. :. ·. ·::: = ·. �; � · :��-·� : =;:·.:. � :;. : : ..: :"::_:· ·�=��:; � ...... :\. .. -...... ;-.. .�1. .�:��.· . .�.. � .� . . • • • . .• • · •• � .: •• -.-: •• : :• -: .'! ;·. . : ...: ;: :. . . ·:- .. .· ..: :·. ::...... :...�. � .� ...... : ...... - .. .. ·� ...... ·...... :...... • • . • • • • • • • • · • • · • I • ...... - ...... · .· .· ... ·.· . ·.·.. . : , .. : · : ·: �/ ,· .... : .· . . ·.·. . ·.·.·.·.·� �·.·. .-. �·.·...... ·...... ·. ·.· ·. ·.. · . .�:

ISSN 0898-90lX

Pnntcd in U.S.A. EY-P9(,3E-DP/93 08 02 17.0 Copyright () Digital Equipment Corporation. All Rights Reserved.