Middleware for Distributed Industrial Real-Time Systems on ATM Networks

MITSUBISHI ELECTRIC RESEARCH LABORATORIES http://www.merl.com

Ichiro Mizunuma, Chia Shen, Morikazu Takegaki

TR96-10a September 1996

Abstract In this paper we address the problem of middleware design for constructing ATM LAN based distributed industrial plant monitoring and control systems. In particular, we present a real-time client-server programming model based on a uniform ATM network. This model is being realized in our middleware called MidART. The middleware provides a set of industrial application speciﬁc but network transparent programming abstractions and application programming interface (API) that support individual application QoS requirements. In order to achieve on-demand transmission of plant data, we have developed a concept called selective real-time channels to be supported by MidART. We present the design and protocols of selective real-time channels and describe how QoS requirements of applications are guaranteed.

Proceedings of the 17th IEEE Real-Time Systems Symposium, Washington, DC 1996

This work may not be copied or reproduced in whole or in part for any commercial purpose. Permission to copy in whole or in part without payment of fee is granted for nonproﬁt educational and research purposes provided that all such whole or partial copies include the following: a notice that such copying is by permission of Mitsubishi Electric Research Laboratories, Inc.; an acknowledgment of the authors and individual contributions to the work; and all applicable portions of the copyright notice. Copying, reproduction, or republishing for any other purpose shall require a license with payment of fee to Mitsubishi Electric Research Laboratories, Inc. All rights reserved.

Copyright c Mitsubishi Electric Research Laboratories, Inc., 1996 201 Broadway, Cambridge, Massachusetts 02139 MERLCoverPageSide2

Middleware for Distributed Industrial Real-Time Systems on ATM Networks

y z y

Ichiro Mizunuma , Chia Shen , and Morikazu Takegaki

y Industrial Electronics and Systems Lab.,

Mitsubishi Electric Corp.

8-1-1, Tsukaguchi-honmachi, Amagasaki, Hyogo, 661, Japan.

email: mizunuma,[email protected] dl.m elco .co. jp

z A Mitsubishi Electric Reseach Lab.

201 Broadway, Cambridge, Massachusetts 02139 USA

email: [email protected]

Abstract In this paper we address the problem of ble, exp ensive, dicult to mo dify and not scalable.

midd leware design for constructing ATM LAN based Figure 1 is a typical example of a current industrial

distributed industrial plant monitoring and control control system, similar to those systems from Mit-

systems. In particular, we present a real-time client- subishi Electric Corp. and those discussed in [Meinert

server programming mo del based on a uniform ATM 95]. The network architecture contains twolayers, a

network. This model is being realized in our mid- control layer and an information layer. The network

d leware cal led MidART. The midd leware provides a bus in the control layer is compliant with FDDI con-

set of industrial application speci c but network trans- necting together proprietary subsystems such as pro-

parent programming abstractions and application programmable logic controllers PLCs, and op erator sta-

gramming interface API that support individual ap- tions OPSes. A proprietary proto col, called cyclic

plication QoS requirements. In order to achieve on- transmission, is used to achieve guaranteed end-to-

demand transmission of plant data, we have developed end delays in the control layer among the subsystems.

aconcept cal led selective real-time channels to be sup- Each no de subsystem in the network contains a dual

portedbyMidART.Wepresent the design and proto- p ort memory for cyclic transmission. The contents of

cols of selective real-time channels and describe how each cyclic area are transmitted within some constant

QoS requirements of applications are guaranteed. p erio d to all the no des. The information layer employs

standard general purp ose comp onents for low cost.

1 Intro duction

This layer contains an Ethernet LAN and a number

of workstations EWSes. The EWSes handle plant

We address the problem of middleware design for

information management, plant maintenance, and re-

constructing ATM LAN based industrial plant mon-

p ort generation.

itoring and control systems. These systems are real-

This twolayer system architecture has some obvi-

time in nature, requiring the guarantee of hard tim-

ous problems:

ing constraints in resp onse to b oth human op erations

and sensor noti cation of plant states. Traditionally,

Lack of scalability: The contents of the cyclic

such distributed plant control systems have b een built

memory areas are p erio dically transmitted even

mostly with proprietary network comp onents and host

if there have b een no changes in the data in

systems, and real-time constraints are met with phys-

those areas. This causes network resources to b e

ical isolation of sub-networks. In order to guarantee

wasted, thus limiting the amount of communica-

the timing constraints of plant control, the worksta-

tion the network can supp ort. In addition, the

tions supp orting the human interface for plant infor-

scale of the distributed system is limited by the

mation management often do not reside on the same

size of the cyclic memory area.

physical network as the network in the controlled plant

sites. Similarly, another separate physical network is Fixed QoS Quality of Service: The QoS is xed

used for video transmission from industrial TV cam- by the cyclic transmission p erio d and cannot b e

eras in the plant monitoring sites. Consequently, these altered according to dynamic application require-

current distributed plant control systems are in exi- ments.

with an ATM network. The advantages of such a uni-

AAAAAAAAAAAAAAAAAAAlayer Information

Information

work architeture include: AAAAAAAAAAAAAAAAAAAMonitoring management Maintenance Report-generation form net

AAAAAAAAAAAAAAAAAAAEWS EWS EWS EWS

Flexible QoS can be supp orted on per applica- General purpose LAN

AAAAAAAAAAAAAAAAAAA(ex. Ethernet)

Trac that needs deterministic guar- AAAAAAAAAAAAAAAAAAAOperating Post-analysis tion basis.

Gateway

tee and trac that can tolerate larger trans- OPS Data an

AAAAAAAAAAAAAAAAAAAlogger

delay variation can p otentially co-exist

AAAAAAAAAAAAAAAAAAA mission Control layer Control

Control bus (FDDI based)

the same network. For distributed real-time

AAAAAAAAAAAAAAAAAAA in e can readily utilize the concept of AAAAAAAAAAAAAAAAAAA applications, w

PLC PLC PLC

hannels [Ferrari 90]. AAAAAAAAAAAAAAAAAAAFieldbus real-time c

AAAAAAAAAAAAAAAAAAA

The con guration of the system can be easily

AAAAAAAAAAAAAAAAAAASensors / Actuators

mo di ed by adding or removing subsystems on

a plug-and-play basis, since we can employ stan-

Figure 1: Con guration of current systems.

dard platforms now.

With a uniform network technology for the entire

system, we have eliminated b ottlenecks such as

Hierarchically layered network: Gateways are em-

gateways which p ose p erformance and reliability

ployed to connect the control bus and Ethernet.

problems .

The gateways can b e a system b ottleneck for p er-

formance and reliability.

Not all the real-time data need to b e p erio dically

transmitted | on-demand transmission of plant

data or video can b e accommo dated.

By integrating video and audio, new services can

Human (e.g. operators)

provided. For example, alarm video can be

AAAAAAAAAAAAAAAAAAA be

implemented in which abnormal plant b ehavior

AAAAAAAAAAAAAAAAAAACl

ent

now be shown using video to op erators on AAAAAAAAAAAAAAAAAAAData logger R eport- can

O PS M anagem ent s orkstations. AAAAAAAAAAAAAAAAAAAgeneration their w

AAAAAAAAAAAAAAAAAAA

In order to obtain these advantages in a plant

Integrated ATM Network

and control system, a new style of pro-

AAAAAAAAAAAAAAAAAAA monitoring

construct distributed real-time systems

AAAAAAAAAAAAAAAAAAA gramming to

Ser

needed. In particular, each application must be C ontroller Multi-media is AAAAAAAAAAAAAAAAAAAserver ver

ITV s

to request and acquire its own individual QoS.

AAAAAAAAAAAAAAAAAAAFieldbus able

ts a real-time client-server program- AAAAAAAAAAAAAAAAAAA This pap er presen

Sensors / Actuators

del for distributed real-time systems based on

AAAAAAAAAAAAAAAAAAA ming mo

uniform ATM network. This mo del is b eing real-

Plant a

ized in our middleware called MidART Mid dleware

and network A rchitec ture for R eal-T ime industrial

applications. This middleware provides a set of in-

Figure 2: New system architecture with ATM net-

dustrial application sp eci c but network transparent

work.

programming abstractions and application program-

ming interface API, as well as a set of QoS mapping

schemes and proto cols that translate application level

Weenvision that the four key technical advantages

real-time requirements into ATM QoS parameters and

of ATM, i.e., high bandwidth, end-to-end individual

trac descriptors, including schemes to ensure that

QoS, virtual connections and exible topology , will en-

this mapping maintains the timing guarantees at b oth

able us to build future distributed plant control sys-

the sender and the receiver. In addition, to achieve

tems on op en standard network technology with in-

on-demand transmission of plant data, we have also

tegrated communication services including data, im-

develop ed a concept of selective real-time channels.

ages, video and audio. This should lead to cost re-

Current commercially available ATM switches already sup-

duction, more exibility and more dynamics in appli-

p ort hot-swappable duplex switch b oards in the case of faults

cations. Figure 2 represents a new, exible system

in primary b oard, the second b oard stands for immediately for

reliability and continuous op eration. architecture of a plant monitoring and control system

The rest of the pap er is organized as follows. Sec- cation pro cess guarantee. To provide application-to-

tion 2 gives a brief overview of the MidART architec- application QoS guarantee in a distributed real-time

ture. In Section 3, we present our real-time client- system, we must ensure that a sucient resources

server programing mo del and show programming ab- e.g., CPU, bu er space, network bandwidth, etc.

stractions on the mo del. In Section 4, we present are reserved for the services that a user task needs in

the concept of selective real-time channels which pro- b oth the client and the server, and b that each ser-

vides plant monitoring and control applications the vice do es not use more of the resources than what has

necessary abstraction to obtain data on demand, and b een reserved for it. Thus we need admission control

enables the ecient usage of the network. Section for the former and run time scheduling for the latter

5 shows how to guarantee application-to-application in b oth the host systems and the network. Systems

QoS. We compare our work presented in this pap er implemented with MidART achieve this application-

with other related work in Section 6, and the pap er to-application QoS guarantee with the combination of

concludes with Section 7. 1 sp ecifying application QoS with RT-API, 2 ap-

plication to network QoS mapping byRT-ATM, 3 lo-

2 A Brief Overview of MidART

cal admission control byLAC, 4 network admission

Figure 3 depicts the architecture of MidART with

control by GAC, and 5 real-time op erating system

a two-no de system, a client no de and a server no de.

task scheduling using commercially available real-time

Each host in the network contains one copy of Mi-

op erating systems.

dART . MidART includes three comp onents, a RT-

MidART prototyp e is b eing built on an ATM LAN

API, a RT-ATM, and a LAC. RT-ATM contains a

based platform. The prototyp e system set up includes

set of QoS mapping schemes and proto cols, and is re-

an ASX200BX ATM switch from FORE Systems, Inc.

sp onsible for translating application's QoS parameters

[FORE 95], and a numb er of HP 734rt computers with

into ATM service trac descriptors and QoS parame-

the HP-RT real-time op erating system connected to

ters, multiplexing application trac and establishing

the ATM switch using ATM adaptors from Interphase

ATM virtual connections. [Shen 96] gives examples

Inc . The HP 734rt computers implement a Multi-

of how this translation and multiplexing can be ac-

media Server, an Op erator Station and a set of sen-

complished. RT-API provides a set of programming

sors/actuators.

abstractions and API as library routines to supp ort

3 Programming Mo del and Program-

application development. It also contains application

task management facilities sp eci c to industrial plant

ming Abstractions

control application domain. LAC is resp onsible for

3.1 Programming mo del

checking the schedulability of tasks with resp ect to

Figure 4 represents our real-time client-server pro-

the resources in the end host systems.

gramming mo del for industrial applications. In a dis-

tributed plant monitoring and control application, the

most common physical comp onents include sensors, AAAAAAAAAAApplication AAAAAAAAAAApplication

(client) (server)

I/O or programmable logic controllers, AAAAAAAAAA AAAAAAAAAA actuators, RT-API Task RT-API management Network op erator stations OPSes, I/O devices, and database Selective real- independent

RT-CRM time channels

storage systems. Our real-time client-server program- Accept / Communication Application QoS

Reject QoS parameters

antages parameters Notify ming mo del is designed to b est exploit the adv Accept /

Host RT-ATM Reject RT-ATM

ATM high sp eed networks for these applications Application QoS of RT-CRM parameters LAC Network QoS Mapper Local Admission Network dependent and to facilitate ease of programming in this applica- controller (LAC) independent

Accept / Reject tion domain.

h subsystem in the mo del is classi ed as a client AAAAAAAAAAAOS / Device driverAAAAAAAAAAAAAAAAAAAOS / Device driver Eac

AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA

er. A client is a subsystem which is used by Network interface (NI) Network interface (NI) or a serv

AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAGAC / standarAAAAAAAAAAAd CAC

umans directly e.g. OPS, information management,

AAAAAAAAAAAAAAAAAAAAAAAANetwork AAAAAAAA h

or indirectly e.g. database via

AAAAAAAAAAAAAAAAAAAAAAAA rep ort-generation

other subsystems. A server is a subsystem which mon-

Figure 3: MidART architecture.

itors the plant with sensors or industrial TVs ITVs,

and controls the plant directly by actuators. For ex-

ample, PLCs programmable logic controllers and a

Application-to-application or human-to-plant in

We are also investigating the p ossibility of using PCs with

our particular application domain QoS guarantee in-

the LynxOS from Lynx Real-Time Systems, Inc as host sys-

cludes end-to-end i.e., network interface to network

tems. This, however, must dep end on the availability of LynxOS

drivers for ATM adaptors. interface guarantee and network interface to appli-

multimedia server with ITVs can be servers. Each or message deadlines can b e sp eci ed and guaranteed.

client can request a server to send plant data or video

RDA is a real-time unidirectional connection the

of the plant and can request a server to issue com-

direction from the server to the client for an input

mands to the plant sp ecifying the QoS needed. The

device and the opp osite direction for an output device

b ehavior of the servers is required to b e as simple as

between a task in a client and a server task controlling

p ossible such that it is predictable. More complicated

a device. The task sp eci es the QoS parameters of

op erations and functions are usually done in clients.

the connection, an inter-arrival time of messages and

The servers are required to b e reliable and must run

their deadline. Once an RDA is established, the client

continuously for a long time.

task can receive messages from the input device or can

send messages to the output device with the library

functions. The rate control in the middleware on the

Input Output

tees the inter-arrival time. RDA can devices devices Client Storage sender side guaran

Client (Database)

t alarms and control functions. Client b e used to implemen

(OPS)

RT-CRM is a software-based unidirectional re ec-

tive memory. It provides data re ection with guaran-

Request (Device ID/data ID, QoS)

timliness, while allowing applications to sp ecify - Connection to device teed

- Attachment to data area

how and when data are re ected. This is in contrast

Name service

with the current commercially available hardware sup-

re ective memory in which each write is re-

Service R equests p orted

ected to all the no des in the network immediately. Server

Server Server Data re ection can b e done p er p erio d or up on a write.

T-CRM employs a writer-push data re ection mo del.

Data DataData R

The writer of a re ective memory area in RT-CRM is After

Sensors Actuators Sensors Actuators ITV Actuators resp onsible for the creation of the memory area.

the creation, any remote reader application can attach

itself to this re ective memory area for reading, and

Figure 4: Real-time client-server programming mo del.

can sp ecify its QoS parameters in terms of data re ec-

tion p erio d, or delay b ounds of data re ection up on

writes.

A server can b e thought of as a certain kind of I/O

4 Selective Real-Time Channels

controller with limited pro cessing power e.g., lo op

4.1 Concept

control only and storage memory capacity. A server

is usually connected to various input devices e.g. sen-

At any instance, an OPS usually only need the

sors and ITVs and output devices e.g. actuators via

plant data and video stream sampled by a small subset

Fieldbus or other kinds of low-level bus. The services

of the sensors or ITVs. The traditional cyclic trans-

provided by a server may include sampling data from

mission, however, transmits al l of the sampled data

the input device, storing it as a byte stream, and send-

at regular intervals limiting the amount of real-time

ing control data to the actuator. The server has data

transmission that can b e supp orted in a network.

areas in its address space. Data sampled from the in-

Assume that client C and m servers S ;S ;:::;S

1 2 m

put devices and set-up values for lo op control can b e

are connected to each other via an ATM network with

stored in the data areas, for example.

a switch. If we implement a cyclic transmission on

3.2 Programming abstractions

the network, we need a bandwidth of m B at the

link b etween a client and the switch gure 5. Here,

In the distributed industrial plant control applica-

supp ose that the transmission rate from a server is

tion domain, application pro cesses often need to ac-

constant and let it b e B . It is obvious that a large m

cess remote devices in a timely fashion, or to share

can easily saturate the bandwidth of the link.

data with certain real-time up date semantics and tim-

ing constraints. With the real-time client-server pro- In the case where a client actually needs data from

gramming mo del describ ed in section 3.1, we brie y only one server, for example, we need a bandwidth

intro duce two new programming abstractions, remote of only B at the link between the client and the

device access RDA and real-time channel-based re- switch. To achieve this, one obvious solution is to

ective memory RT-CRM. These programming ab- establish channels dynamically when a client requires

stractions di er from similar general purp ose ones in data transmission from a sp eci c server or ITV, and to

that they provide sp eci c real-time supp ort. In par- employ the p oint-to-p oint real-time channels [Kand-

ticular, QoS requirements such as data up date rates lur 94]. With this approach, however, it is dicult C Client C

C Client Channel switching reserved Switch resource in use AAAA m B AAAA S S S S Server S S AAAAS S m

Switch n = 1, k = 1

Concept of selective real-time channels. B Figure 6:

Server S S S S Applications Applications

m receive switch switch Port read Transmission

D' Bandwidth used with cyclic transmission. Figure 5: Switchable Transmission Switchable AAAAAAA A Daemons

AAAAAAA A D1 D2 Dm

hannel establishment and it is to b ound the time of c Drivers

Data areas

teed that the channel with its QoS require-

not guaran Devices

ments can be always established successfully. In the

a) RDA b) RT-CRM

case of OPS, resp onse times to an op erator's request

to change the sensor or ITV for monitoring must be

Figure 7: Programming abstractions with selective

b ounded, e.g., less than or equal to a few tens of mil-

real-time channels.

liseconds.

To solve the ab ove problems and to utilize network

resources eciently, our MidART middleware man-

ages network resources as follows:

Figure 7 a shows an example of the selective real-

Scheme BA bandwidth allo cation

time channels for RDA-based communications, called

If a client requires data transmitted from at most k

Switchable RDA. An application task in a client can

communicate messages to application tasks in servers

same time, assign bandwidth k B at the link b e-

and vice versa via p orts in the client and the servers.

tween the client and the switch SW that the client

Figure 7 b conceptually represents the API of se-

is physically connected to, and assign bandwidth B

lective real-time channels for RT-CRM. Supp ose that

at the link b etween each server and the switch SW

a data area D in a client is shared with data ar-

that the server is connected to. For all the inter-

eas D ;D ;:::;D in the same server or in di erent

1 2 m

mediate network links between SW and SW , we

1 e

servers. Data area D and one of D ;D ;:::;D can

1 2 m

assign a bandwidth of minfk B; s B g to eachof

p erform the one-way transmission or copy of their con-

them, where s is the numb er of servers that actually

tents at any instance. A task in the client can read the

go through the link.

contentofD . If the transmission is in the direction of

the client to the servers, a task in the client writes the

We call a set of real-time channels b etween a client

content. A task in the client can request the change or

and m servers selective real-time channels if at most

switch of the partner of data area D . MidART guar-

k k < m of them are used at the same time and

antees that the time from the request to the nish of

the combination of k channels can b e changed within

switching is b ounded. In other words, a task in the

a constant time changed in real-time. Figure 6

client can read/write the contents of D ;D ;:::;D

1 2 m

represents the concept of selective real-time channels

in a b ounded time including the switching times.

n =1;k = 1.

4.2 Channel Switching Algorithm

We showhow selective real-time channels are real-

A set of selective real-time channels is established ized with programming abstractions describ ed in sec-

by the following algorithm supp ose that all of the tion 3.2.

routes from clients to servers have b een decided. in middleware are executed at higher priorities than

those of applications. The transmission b etween the

1. Establish a multicast real-time channel from each

two hosts in this example is p erio dic.

client to all servers This channel is used by the

client to send control messages to the servers.

2. Reserve bandwidth on each link of the routes from

Table 1: Primitive delays

The time from an interrupt of the input device

servers to the client according to Scheme BA.

to the activation of the application task in the

w r ite

sender host.

ac- Channel switching from server S to server S

i i

The time that the application task needs to

w r ite

write the sampled data into the data area PD.

cording to a request of an application in client C is

The p erio d of the deamon task in the middle-

tr ans

p erformed by the following algorithm.

ware in the sender host.

The execution time of the deamon task.

mid send

1. Client C sends all servers `STOP' requests with

The time that the OS needs to transmits the

send os

packets to the NIC.

the name of the server to b e stopp ed S . Then,

The time from the completion of the transmis-

the client waits for an `ACK' message from the

sion of NIC until all packets have b een trans-

send nic

server S .

mited into the network.

The time that a cell needs to travel from the

netw or k

NIC in the sender to the one in the receiver.

2. On receiving a STOP request, each server checks

The time from the arrival of the last cell to the

NIC in the receiver to when all packets have

nic r ecv

whether the server name in the request message

b een reassembled.

matches its name. If it is, the server stops the

The time that the OS needs to load all packets

D whichhave b een reassembled by the NIC into

transmission to client C and sends it an ACK

os r ecv

application memory.

message. Otherwise, the server ignores the mes-

The time from when the deamon task in the

sage.

middleware is activated by the OS to when the C

mid r ecv

task nishes all required op erations.

The p erio d of the application task in the re-

3. On receiving an ACK request, client C sends all

r ecv

ceiver host.

servers `START' requests with the name of the

The time from the start of the p erio d of the

task to that when the task is actually acti-

server to start transmission S .

r ead

vated. This time o ccurs b ecause of the pre-

emption of tasks with higher priorities.

4. On receiving a START request, each server checks

C The execution time of the application task.

r ead

whether the server name in the request message

matches its name. If it is, the server starts the

transmission to client C . Otherwise, the server

If we can obtain the maximum values of the

ignores the message.

all primitive delays ab ove, we can estimate the

application-to-application delay D as the sum of

atoa

The algorithm ab ove is one for the case of k =1. It

the primitive delays in table 1. Here, supp ose that

will be easy to extend the algorithm for the case of

the time of interrupts and the time of context switch

k>1by rep eating the algorithm k times.

can b e neglected. We can estimate the maximum val-

To reduce the bandwidth reserved for these mes-

ues of all the delays as follows.

sages at each link on the route, we use a multicast

D and D are waiting times for resources

w r ite r ead

channel for control messages from clients to servers

such as CPUs and semaphores. Their maximum val-

instead of using p oint-to-p oint real-time channels.

ues are calculated using rate monotonic algorithm

5 QoS Guarantee Implementation

RMA [Liu 73] [Leho czky 89] with its extensions for

In this section, we describ e the mechanism which

resource contention [Ra jkumar 91] by LACs in the

guarantees QoS, as a delivery delay and a transmission

hosts. C , C , and C are estimated as lin-

w r ite os send r ead

rate, in a system based on a real-time client-server

ear functions of the data size S . C , C ,

mid send mid r ecv

mo del.

T , T , and D are constant. D and

tr ans r ecv os r ecv nic send

5.1 Application-to-application delay

D dep end on the cell scheduling algorithm of

nic r ecv

This subsection shows that what kinds of primi- the NICs and trac characters such as minimal inter-

tive delays an application-to-application delivery de- arrival times of other channels in the NICs. Here, we

lay consists of, then shows how to obtain their b ounds. assume that the scheduling algorithm is FIFO or xed

Let us treat two applications which communicate with priority based and the minimal inter-arrival times of

each other via RT-CRM. It is obvious that the de- cells in all channels are known. With this assumption,

lays using RDA is simpler. Figure 8 and table 1 and D based on a one can calculate D

send nic r ecv nic

shows the primitive delays included in an application- usual RMA. How to calculate D is describ ed in

netw or k

to-application delay. Supp ose that the deamon tasks subsection 5.2.

tion C; T ; D onto VBR parameters with calculating

Interruption by a device

D by the metho d prop osed in [Zheng 94a] or Dwrite netw or k

Cwrite uma 95], for example. Sender that in [Mizun Cmid_send M iddleware

Ttrans

Related Work D 6 OS nic_send Cos_send

have b een a couple of research e orts in NIC network There

ting with intro ducing early ATM network

Network nic_recv exp erimen

into industrial systems. The Mercuri ATM NIC os_recv systems

OS C

avan 95] is a pro cess monitoring

mid_send testb ed [Guha 95] [P

M iddleware C TM platforms. read system with rst generation standard A Receiver T

recv D

ercuri also provides a client-server mo del

read Although M

applications, the ob jective of the Mercuri testb ed

NI-to-NI delivery delay to

from ours in that their fo cus is mostly to ex-

Application-to-application delivery delay di ers

p eriment with multimedia data video and audio on

ATM networks. Thus message throughput and round

Figure 8: Application-to-application delivery delay.

trip delay,aswell as video frame jitter are their main

concerns for p erformance measurements, rather than

application-to-application deterministic QoS guaran-

tee. On the other hand, we are fo cusing on providing

5.2 Network delay and bandwidth

programming mo dels and mechanisms for QoS sp eci-

As delays in the NICs, the delay in the network,

cation and guarantee.

D , dep ends on the scheduling algorithm of the

netw or k

[Pokam 95] also prop osed using ATM networks to

switches and the trac characters of the other chan-

incorp orate multimedia in pro cess control systems.

nels going though the switches.

Their system, however, is not scalable due to their

Signalling and CAC algorithms of current com-

ring top ology. In addition, their approach needs sp e-

mercial ATM pro ducts mainly supp ort p oint-to-p oint

cial hardware for communication interface STRAM

or p oint-to-multip oint connections, not multip oint-to-

communication b oards rather than using standard

p oint connections which are needed for selective real-

platforms.

time channels. For data transmissions in selective real-

Task scheduling and QoS management in middle-

time channels, we use PVC connections and FORE

ware or op erating systems for multimedia end systems

ASX-200 BX's functions of bandwidth and bu er over-

have b een the fo cus of numerous research e orts re-

b o oking for VBRs [FORE 95]. This means that cur-

cently [Mercer 94], [Nishio 94], [Kawachiya 95], [Je ay

rent selective real-time channels are able to b e estab-

95]. Although most of them are not dealing with in-

lished only statically, not dynamically.

dustrial plant control systems with deterministic guar-

We show how to realize selective real-time chan-

antee requirements, we can still nd similarities in our

nels with PVC and BX's overb o oking. Supp ose that

resp ective pro jects.

we have a client and m servers and the bandwidth

The approaches we are taking for our host system

from server S i = 1;:::;m to the client is B . In

i i

admission control in LAC and the application to op er-

a case where at most k k << m servers send data

ating system QoS mapping supp orted in RT-ATM are

at the same time, we can give an integer value of

similar to the reservation concept in [Mercer 94] and

[Je ay 95]. One di erence lies in that we use com-

b 100c as the parameter vbrob at the in-

k max B

i i

mercially available real-time op erating systems i.e.,

put link of the client, for example. Here, the switch

HP-RT or LynxOS which are monolithic, rather than

assign a VBR channel bandwidth of 100 100=vbrob

micro-kernel based technology.

of that required to assign. For transmissions of control

A middleware for continuous media applications

messages from the client to the servers, we can use a

is presented in [Nishio 94] and [Kawachiya 95]. A

CBR multicast channel.

conductor/p erformer architecture is used to guaran-

If we can obtain an ATM platform which sup-

tee QoS of continuous media and to provide dynamic

p orts multip oint-to-p oint connections in future, GAC

QoS control. The middleware is a set of system servers

will map application requirement for communica-

on the RT-Mach micro kernel. The fo cus of this work

We can also use CBR unicast channels for the transmission.

is very di erent from ours | dynamic QoS control

This approach lacks scalability in the numb er of servers, but

among multiple sessions in one host system for con-

bandwidth assigned for control messages is very narrow and

tinuous stream of video data.

we b elieve that wehave no problem with at most one hundred

servers in an actual application. There have b een a lot of studies lately on real-time

[Kandlur 94] D. D. Kandlur, K. G. Shin, and D. Ferrari. Real- communication with end-to-end network interface-to-

time Communication in Multihop Networks. IEEE Trans-

network interface QoS guarantees in a packet switch-

actions on parallel and distributed systems, Vol. 5, No. 10,

ing network or an ATM network [Kandlur 94], [Raha

pp. 1044{1056, Oct. 1994.

96], [Zheng 94a], [Zheng 94b] and [Mizunuma 95].

[Kawachiya 95] K. Kawachiya, M. Ogata, N. Nishio, and

H. Tokuda. Evaluation of QOS-Control Servers on Real-

Most of these work are based on the cencept of real-

Time Mach. The Fifth International Workshop on Net-

time channels [Ferrari 90] and derive delay b ounds

work and Op erating System Supp ort for Digital Audio and

given packet scheduling algorithms and trac source

Video. pp 123-126. April 1995.

sp eci cations. These are imp ortant results that we

[Leho czky 89] J. P. Leho czky, L. Sha, and Y. Ding. The Rate

Monotonic Scheduling Algorithm: Exact Characterization

exp ect the next generation ATM networks can make

and Average Case Behavior. IEEE RTSS '89, pp. 166{171,

use of.

Decemb er 1989.

[Liu 73] C. L. Liu and J. W. Layland. Scheduling Algorithm

7 Conclusion

For Multiprogramming in a Hard-Real-Time Environment.

In this pap er, wehave given an overview of our mid-

JACM, Vol. 20, No. 1, pp. 46{61, January 1973.

dleware MidART Mid dleware and network A rchitec

[Meinert 95] G. Meinert. Op enness for Automation Networks.

InTech, pp 29-32. Oct. 1995.

ture for R eal-T ime industrial applications, and pre-

[Mercer 94] G. Mercer, S. Savage and H. Tokuda. Pro cessor

sented the details of our new real-time client-server

Capacity Reserves: Op erating System Supp ort for Mul-

programming mo del for distributed industrial plant

timedia Applications. IEEE International Conference on

Multimedia Computing and Systems. pp 90-99. May 1994.

control applications. For ease of use and predictability,

our programming mo del and API has stressed simplic-

[Mizunuma 95] I. Mizunuma, S. Horiike, and M. Takegaki.

Real-Time Communication in Plant Monitoring / Control-

ity and uniformity. MidART is one of the rst e orts

ling Systems with ATM Networks. RTCSA'95, pp. 243{

in intro ducing the emerging ATM technology into dis-

250, Oct. 1995.

tributed industrial plant control applications. We are

[Nishio 94] N. Nishio and H. Tokuda. Conductor-Performer: A

Middle Ware Architecture for Continuous Media Applica-

currently implementing and evaluating our middle-

tions. 1st International Workshop on Real-Time Comput-

ware on an ATM LAN platform.

ing Systems and Applications, pp.122-131, 1994.

MidART is an on-going e ort. Besides the evalu-

[Pavan 95] A. Pavan, J. Liu, A. Guha, J. Pugaczewski, and

ation of its real-time p erformance, some of the op en

M. Midani. Exp erimental Evaluation of Real-Time Sup-

p ort on the Mercuri Wide Area ATM Testb ed. LCN '95,

issues that we are continuing to address include sig-

pp. 82{91, 1995.

nalling algorithms at the ATM layer to facilitate the

[Pokam 95] M. R. Pokam, J. Guillaud, and G. Michel. ATM

ecient implememtation of selective real-time chan-

Networks Allow Multimedia in Pro cess Control Systems.

IEEE ICCCN '95, pp. 324{331, 1995.

nels , as well as e ective QoS mapping algorithms for

the RT-ATM comp onentof MidART.

[Raha 96] A. Raha, S. Kamat, and W. Zhao. Admission Con-

trol for Hard Real-Time Connections in ATM LANs . IEEE

INFOCOM '96, March 1996.

Acknowledgements

[Ra jkumar 91] R. Ra jkumar Synchronization in Real-Time

The authors would like to thank Dr. Satoshi Hori-

Systems: A Priority Inheritance Approach. Kluwer Aca-

ike of IES Lab., Mitsubishi Electric Corp. and Dr. Qin

demic Publishers, 101 Philip Drive, Assinippi Park, Nor-

Zheng of MERL for their valuable discussions during

well, Massachusetts 02061 USA. 1991.

the course of the research rep orted in this pap er.

[Shen 96] C. Shen. On ATM Supp ort for Distributed Real-Time

Applications. IEEE RTAS '96, pp. 70{78, June 1996.

References

[Zheng 94a] Q. Zheng, K. Shin, and C. Shen. Real-Time Com-

[ATM Forum TM 4.0] ATM Forum Technical Committee.

munication in ATM Networks. LCN '94, pp. 156{164, Oct.

Trac Management Sp eci cation 4.0 April 1996.

1994.

[ATM UNI 3.1] ATM Forum Technical Committee. ATM User-

[Zheng 94b] Q. Zheng and K. Shin. On the Ability of Establis-

Network Interface UNI Sp eci ccation Version 3.1 Sept.

ing Real-Time Channels in Point-to-p ointPacket-switched

1994.

Networks. IEEE Transactions of Communication, Vol. 42,

No. 4, pp. 1095{1105, 1994.

[Ferrari 90] D. Ferrari and D. Verma. A Scheme for Real-Time

Channel Establishment in Wide-Area Networks. IEEE

Journal on Selected Areas in Communications, 83:368{

379, April 1990.

[FORE 95] FORE Systems, Inc. F or eRunner ASX-200BX

ATM Switch User's Manual, MANU0026 - Rev. D. Septem-

b er 1995.

[Guha 95] A. Guha, A. Pavan, J. Liu, A. Rastogi, and

T. Steeves. Supp orting Real-Time and Multimedia Appli-

cations on the Mercuri ATM Testb ed. The IEEE Journal

on Selected Areas in Communications, Vol.13, No.4, pp.

749{763, 1995.

[Je ay 95] K. Je ay and D. Bennett. A Rate-Based Execution

Abstraction for Multimedia Computing. The Fifth Interna-

tional Workshop on Network and Op erating System Sup- p ort for Digital Audio and Video. pp 67-78. April 1995.