- 198 -

DESIGN CONCEPTS WO EXPERIENCE IN THE APPLICATION OF DISTRIBUTED COMPUTING TD THE CONTROL OF LARGE CF,GB POWER PLANT

By 3 N Wallace, CECB, 5outh Western Region, Bristol, England.

1- INTRODUCTION

With the ever increasing price of fossil fuels it become obvious durinq the 197O's that Dembroke (4 x 500MW oil fired) and Power Station (4 x 50DMW fired) were going to operate flexibly with many units two-shifting frequently. The region was also expecting to refurbish nuclear plant in the 1980'a. Baaed on previous experience with mini-computers, the region initiated a research/development proqramme aimed at refitting Pembroke and Didcot using distributed computer techniques that were also broadly applicable to nuclear pJant* Major schemes haws now been implemented at Pembroke and Didcot for plant condition iRonitoring, control and display. At the tine of writing all computers on two units at each station are now functional with a third unit currently being set to work.

This paper aims to outline the generic technical aspects of these schemes, describe the implementation strateqy adopted and develop some thoughts on • . plant applications.

2. THE DISTRIBUTED COMPUTER CONTRDt/HOWlTnaiNC/DlSPLAY SYSTEMS AT PEMBROKE ASP

2.1 S1 stem Keguirements

In examining the various solutions available for a new control/monitciring/dlapley systems at Pembroke and Didcot) a number of ba3ic requirements were identified. These included:

Performance The need For a system capable of hiqh control performance to ensure minimum plant damage end environmental problems whilst meeting the new demanding operating regimes.

Flexibility The need for a system that could cope with a variety of problems, encompass new developments as they arose and allowed further system expansion in the future.

Integrity and Reliability Important requirements were that there should be no sinqle coint failures, that the system should be designed around a fail safe philosophy and that a3 hardware failed the system oerformance should oeqrade gently.

Hardware ft general requirement was that the hardware purchased should be commercially availaDle. The amount of special purpose hardware wa3 therefore to be minimised.

Ease of Commissioning The system had to be caoable of 'jeing developed and commissioned on a piecemeal basis since the 500MW units were fully operational. Vo specific planned outage time was available since thi3 would have considerably increased total scheme noat.

Ease of Hainfrenance/Traininq The system was required to be fully maintainable by station staff with limited training or specialist skills. Spares holding and availability were also crucial considerations.

Project Timescales/Cost Project timescale and coat are inextricably linked. A svstem capable of rapid installation, development, commissioning and refinement vas therefore needed, Timescale was also important because of the rapidly approaching need to 2 shift the units without incurring plant damage.

Parallel Experimental Development In view of the project tfmescales and complexity of the performance problems there was a requirement thet The system should permit small scale experimental -work to be incorporated directly into the main scheme.

South Western Region viewed the above from a background of experience on Power Stations with stand alone mini- computer systems. It was clear that a distributed computer based control/display svstem supporting a high level, engineer orientated software system was the only feasible solution. The system described in subsequent sections is a direct conseauence of this conclusion.

2.2 pygtem Design

The eventual design for each boiler/turbine unit at Pembroke called for five interconnected computers' arranged in derarchical fashion {Fig.l(a)) whilst at Didcot (Fig.Kb)) a ring structure was adopted. In the liqht of the system requirements identified above the following key technical discussions were made:

Hardware Modularity All computers were tD identical in fora in orc!er to minimise maintenance, trainina end spares problems. Function or" each computer was to be determined by software olu3 the plant input/outDut connections.

functional Distribution It «83 known *.hat T>e fewer loops per computer the smaller the effect of failure. However, economics prevailed to the extent .hat each plant area was allocated one computer. In addition, at Pembroke a fourth computer was allocated the dual functions of primcry control displays and unit supervisory control. Since all plant area computers displayed th-:.ir data via the unit supervisor it was felt that a bock uc display computer should also be incorporated. Hbvinq no control function to perfoiTt it was thus also aoie to act as system host and plant condition monitor. - 199 -

Incremental Actuator "rives An incremental system usino stepper rotors drivinq Z/° or F/H convectors mas selected because of the natural freeze on fail characteristics. To avoid plant area computer processor overloat individual steoper motor drive carets were incoroorated. These >erc 1 bit viicroorocessor based and provided with niuitol I/O posts and 2 dedicated ADC's. Individually cowered 'ran the secure 51V r>C battery supplv they were desiqned to give moderate performance, sinale looo Dock- 'jp control in the event of plant area comouter failure. This ensured that 2 shiftinq without extreme nlnnt transients was nuaranteed.

iTommunicot ions In any distributed svotem an inter orocessor communications cvstem is a necessity. Serial data transfer at up to °.6K baud rhut in practice 4.8K baud) wes selected because of its fail safe charocteristics. insensitivity to cable run lenaths and capability for ruture expansion. Rosic data communication was hBsed at Pembroke on a ooint to ooint arrangement with the unit supervisor at the data centre collector/distributor. However a serial bus linking all computers to the plant monitor ho3t provided a back uo data transfer path for display purposes. Thia serial bu3 was also desiqned to carry all hosted software loading and oroqramme amendment activities. The flidcot system was constructed on a rina basis because or" the hioher monitorinq/ display requirements.

nisolav qnd Command A ma.ior innovation in the scheme was 'he intention to orovide operators with a much improved dai.3 disDiay via the twin coluur VDU's a**J a wide range of command facilities via multiplexed pushbuttons mountea in modular 72mm DIN qrid plaques. These pushbuttons were designed to assert ooto isolated diqital inputs and their function was to be determined by application software within each computer. As a matter of policy all desk modules were purely passive, includinq the computer/standby manual actuator drive plaque3. Potentially hiqh reliability and flexibility were key factors in this decision.

Software Previous experience with a variety of software systems/approaches led h.o the conclusion that the use of a simple, efficient, high lovef, multi task software system containing a range of 'uncrashable' general purpose statements was an optimum method of develooina integrated control, monitoring and display systems. An enhanced nultiprocessor version of existing software was therefore develooed and used by a variety of engineers to test the equipment and subsequently set it to wort.-.

The svstem design outlined above resulted in the functional hierarchy oortrayed in Fig.2(a).

.3 T-"ical °lant Area Computer 5v3tem

Inalonue plant data is obtained throuoh a 64 channel scanner «ith a 2HH channel/second caoabilitv. [n practice, a lower scon rate is employed, with the ADC integrating over a full nains evele ?o live good supply freauencv noise rejection. Solid state incut selectors are used throughout the control c&mouters whilst reeos are used for low level 3iqnals with variable common mode signal level. 50V dc statu3 sionais are sensed ana controlled by the comouter using opto isolated status inputs and reed relay isolated output cards.

°lant area comouter required to orovide displays have a solid state display interface and 32K ,'or 64K! solid date backing store for storina disolav aeneretion programmes and data.

Control outouts to plant actuators go through a high integrity outDut interface -hich is described in more detail in 5ection 2.4.

2.A Typical Intelligent Actuator ^rive Card

The only oiece of hardware that was not commercially available was the actuator drive card shown in Fia.'Ka). Each is individually powered from the secure 50V dc supply and is desiqned to drive a two pnase stepper motor using series resistors and 5tlV de supply. Opto isolation is used throuqhout to ensure non-interaction with other cards and signals. Two phase stepper motors were incorporated into the E/M 3nd F/p converters in order that a totally passive standby manual driive could be implemented. Whenever the card watchdog drops out 'bringing a rla3hing supply to the 5M plague 'computer fail' button) or the 'computer fail' button is depressed the changeover relav isolates the steoper motor from the card and incorporates a ohasino caoacitor across the stepper motor. The standby manual piaoues raise/lower buttons then merely apply 2&AC to one stepner motoo ohasi and the motor rotates in the appropriate direction.

The card was originally conceived as a device ""or minimising ISI-ll CP" utilisation whencirivir.g multiple 'up to 14) stepper motors. However, merely by adding the two floating ADC's and opto isolated lioitai I/O an intelligent backup controller was obtained with minimal increase in cost/complexitv. The cards operate fixed programmes blasted into PROM and operate in one of two modes:

1- Normal: Oata handshake with LSI-11 defines the number of pulses requested and I/n status, "enuests for too nany pulses dre totally ignored for security reasons. Watchdog reset onlv when the LSI-H calls the card ,md hence *ero pulse requests must be made to keeo the watchdoq set. 'Tyoical rtropout time is 2 seconds). Havinq received a pulse request the microprocessor nenerates the anoronriate steopsr waveforms at a rixed freauency of 200 stena/cecond. during the narnal lata handshake the steoper card sends .analog and digital input values back to the LSI-11 where they are checked against values scanned by the LSI-11. This Gives an important increase in system security by allowing earlv failure detection/correction rather than waiting until back up auto is selected.

2. °.ack uo: If a backup orogramme exists then failure to communicate with the L?I-ll t- :.ults in the fixed back uo control algorithm operating using the two ADC's for .measurements. This programme then has the responsibility for keepino'the watchdog set. For safety, transfer hack to L"51-11 control is not sossihle until standbv manual has "irst been selected. In some cases two cards/actuatcrs aff»ct a common process variable. Intercard communications usino the opto isolated digital I/n tracts resolve this conflict. - 200 -

A problem arose in creatinq e fail safe drive for the shunt wound 240V dc speeder gear nator controlling governor valve lift. This could not be changed and so a special interface was developed. This upratea the normal stepper rnotor sauare wave drive and drives a 50Hz AC transformer. The resulting square wave is rectified end applied to the motor armature, giving o floating dc drive which disappears when the stepper card stops pulsing.

2.5" Svstem Software

If engineers are to successfully apply computers to their many varied problems ot a minimal cost it is essential that they should be provided with system software which enables them to write their own application software. After surveying the available software in the early 1970's it was apparent that to obtain suitable system software it would have to be produced in house. This approach has proved very successful with a large number of installations covering a range of real-time applications in power stations, transmission sub3totion3 and the laboratory all using common (configurable) system software with application software written by enqineers rather than programmers.

To achieve adequate performance- and enqineer acceptability the following design reauirements here adopted and met;

1. Real Time flulti-task Executive

The system had to incorporate a real time executive to allow the overall work for each machine to be broken down into a number of discrete tasks of differing priority. The main function of the executive is to stimulate tasks when time'or external events demand they should run, and to queue conflicting requests from several tasks for a single resource (e.g. the c.p.u. or a hardware device).

2. Simple High Level General Purpose Language

The system had to incorporate a high level language tD enable engineers to write programmes with minimal difficulty. High level languages are for more acceptable to non-specialists and their great benefit is that they greatly reduce the number of coding errors and also constrain the damage which can be produced by such errors.

The language had to be simple for two reasons:

(a) to make it acceptable to people with no formal computer training.

(b) so that it could be implemented in small process control machines with only a limited proaramminq resource.

A general purpose language we3 required as the range of work to be covered precluded the use of special languages with the resources available.

3. Time Efficient

Experience with manufacturers languages (particularly BASIC) revealed that great cere was required to get sufficient throughput. The languaqe had to incorporate inteqer end logical variable type3 for efficient orocessing and real variables for applications demanding greater precision. The programme source is translated into a threaded polish object code which is exacuted with reasonable efficiency yet does not demand a sophisticated compiler.

A. Space and Co3t Efficient

The software was designed to run on the low cost end of the POP11 ranqe which meant it had to run in not more that 28k memory and could not be dependent on expensive backing store. This constraint is of course less important row than it was eight years ago when the work started.

5. Table Oriver Compiler

To allow Language extensions to meet the demands of different applications and to give the necessary configurability for each application, a table driver compiler with a free syntax was adopted.

6. Multiprocessor Support

The system had to support multiprocessor configurations as these were vital for reasons outlined elsewhere in this paper.

7. Line-by-line Translation

The language comDilation is handled one line at a time as eoch line is entered, with only a small mrnber of whole programme checks at the end. This enables the machine to report syntax errors etc. back to the engineer 03 early as possible which is highly desirable when inexperienced people are learning to programme the system.

B. Wide Ranqe of Utilities

A wide range of overlaynble utilities '/ere provided which allow the user not only to perform essential functions 3i.,ch as proqramme compilation, saving and restoring programme on backing store etc., but also givpg other useful facilities such as the ability to determine the status of running jobs and examine the contents of programme varinble in running jobs. These facilities are heavily utilised during debugging and commissioning. - 201 -

2.6 Communicntions

As described in Section 2.2 all data tranafer hetween processors is achieved U3inq serial links running at 4.8K hour). Thi.T baud rate is also selected for the link to the analogue scanner since in both cases the data transfer failure rate is much reduced (to neglible proportions). The actual amount of data transferred from point to point is not large, the following figures being typical:

. Control data ID words every 3 seconds

. Primary display data 100 wordD every 2 seconds

. Fast display data 5 words every 0.5 seconds.

The use of integers, bit orientated status and alarm data and selection of qlobal data for displayserve to minimise the serial link baud rote reguirement3.

Data transfer is controlled by application level software calling system software routines. The standard CEC3 protocol GENNET was found to give unacceptable performance with even modest levels of electrical noisa and a modified protocol was thus developed. A handshaking procedure is employed to ensure accurate data transfer with the transmitter making up to five attmepts to transfer each byte before flaqging a communications error. This is particularly important in the Pembroke configuration where the unit supervisor is extremely busy processing five asynchronous serial lines in addition to sophisticated control calculation and data display- Each epplication job dedicated to transmitting/receiving data has additional handshaking/watchdog facilities inbuilt to ensure that data being displayed is not frozen due to loss of the data link. In addition, data for control purposes is further checked with amplitude and rate limits relevant to the particular epplication. High integrity is thus assured.

2.7 Applications Software The applications software is written in a hiqh level lanauaae by enaineers and occupies some 8-12K of memory. The total computer function is subdivided into a number of small jota each of which performs a functionally distinct task. Each job consists of a series of simple (crash proof) statements operating on local and global variables, the latter being heavily used for inter-job co

Typically each plant area computer contains 10-15 control loops implemented using some 25-30 separate jobs. Inteoer variables are employed in most, cases since this gives maximum speed, minimum space utilisation and data transfer overheads and allows bit as well as word manipulation. Particularly in the alarm and interlock field bit -nanioulation has proved highly space and time efficient and significantly reduces interprocessor data rates without resort to compression techniques. Real variables are employed in specialised applications such as square roots, Kalman filter covariance matrices etc.

Each engineer tailors his application software to the particular application needs. However, typically any complete system would contain the job illustrated in Table 1. Experience has shown that subdivision of job by function rather than 'loop' is more efficient, safer, easier to comprehend,commission, and manage- and makes best use of the job priority structure.

Within the system there are naturally a nunber of cascaded looos and,at the higher levels of cotimisation and unit supervisory control,closely coupled multivarioble control algorithms. However, as a matter of policy each process variable or control input ha3 a single loop identity and all loops conform to the same status format. This format consists of eight separate conditions:

CA - Computer auto: Plant area or unit supervisor computer evaluates the control action required. Loop set poinc derived either from another loop or by computer manual raise/lower commands fran the operator.

CM - Computer manual, auto not available: same as CM but software interlocks prevent selection of CA.

TR - Tracking: Loop output is adjusted to track a variables measured value when that variable is not under computer auto control and bumpless transfer is required.

SA - Standby auto: An individual actuator drive card requl_te3 a single actuator usinn two plant measurements. Modest performance provided on critical control loops when the plant area computer has failed or communications have been lost. Transfer back to ncrmal computer manual/auto is achieved by prior selection of standby manual to ensure that the operator is aware of the status chanqe.

D?] - Desk manual: When a loop is in standby auto its set point can be adjusted by simultaneous operation of the normal desk 'loop' pushbutton and a raise/lower pushbutton. This deliberately contrasts with the normal seguential operation required under computer manual operation.

5M - Standby manual: Direct raise/lower drive of an actuator using the standby manual plaaue.

SH - Standby manual with computer not available: Same 83 normal stBndby manual but software interlocks prevent selection of computer drive of the actuator. - 202 -

2.3 Digital Concroi Techniques

A vide variety of digital control techniques are enployed in the system. For fasc sub loops digital controller based on analog 3 term controllers are typically employed. However, full use is made of the general purpose language to optimise each of these controllers for the particular application in question. For example, different kinds of rate and amplitude limits are employed and digital compensators (for time delay ecc.) are also required in certain conditions. Specific techniques are also employed Co cope with steady operation close to or at limits and. initial transients as loops are brought into play.

Optimisation and supervisory loops employ techniques which are essentially digital in nature. Such loops aim to manipulate set points of sub loops with a view to minimising a quadratic performance index of the general form.

(a) J, - q, x~(k)

(b) J = Z_, q x^(k) + q, XjCk) * q3 xhk) * ... q x~

- k

Where a simple process model can be obtained (e.g. load/pressure/superheater temperature,! che cype (b) performance index is employed since che feedback control gains can be calculated independently from a matrix riccati equation. Plant state estimates are obtained from a real time, parameter Adaptive Kaloan filter which also provides significant smoothing of noisy (pressure) measurements and a dynamic estiizane of plant disturbances. Research work is continuing on the application oc such techniques both at Pembroke and Didcot since initial plant trials have been encouraging.

2.9. Plant Data acquisition and Concroi Actuation

The acquisition of reliable, plant data is crucially important to the performance and integrity of any computer based control system. Thus solid state (racher than dry reed relay) input selectors are utilised for all concroi computers. All control signals are supplied from new, dedicated transducers/ current loop transmitters and each signal is scanned only by one computer. Signal sharing is accomplished by digital data transfer using the serial links in order to maximise system integrity. Since each stored signal, is directly controlled in one plant area computer all other computers utilising the data for scheduling etc. are designed to fail safe in the event of (rare) data link failure. Degraded performance is accepted.

Validity checking of raw scanned data is regarded as vital. Experience has shown that real plant measurements are capable of the most obscure and unexpected faults. Thus as a minimum, cate and amplitude checks are applied. Experiments are currently being condt. :ced using model prediction errors (Kaloan filtering) with a view to further improve signal failure detection capability. The particular checks applied and action taken when data is detected as being invalid (bad) are application dependent. However, table 3 gives an indication of the kind of checks currently involved.

Checking of control outputs to actuators is also employed (Cable 3). For example, if an actuator drive card fails to respond correctly then the drive card scatus is indicated as failed and the loop tripped off computer- Actuator measured position is also compared with the computers prediction based on .iccuraulated pulses. If a discrepancy of more than 5-8Z arises che loop is again tripped off automatically. Since che measured position almost invariably comes from a potentiometer, cLacfcing of response to individual pulse increments has proved fruitless. - 203 -

2.10 Marrnr. and Interlocks

* wide variety of alarms and software interlocks ooerate within each conauter. These ^re application soeciFic nut tvpically include:

'. Calculation of both advisory and mandatory operatinq limits.

. Auaihlp warning to ooerators of serious alarm candtions such 93 a looo trip to computer manual.

Control of IOOD status in the face of lonicol constraint", such es had innut -1ata, actuator failure or unavailability of other loops, actuators etc.

. Control of Looo set points, limits and actuator positions under specific plant ooeratinq conditions 3uch as trios noa start uos<.

*s a matter of policy all Looos are tripped to computer manual when the qonrooriate prime control variable measurement or actuator is labelled as havinn failed ?bed). However, with auxiliarv data auch as that used for control looo oarnmeter ^chedulinq/-he interlock 3ystem often sets a safe default value in the aonrooriate frozen neasurenent, thus avoiding a looo trip hut incurrinq degraded control oerfornance.

Visa ds a matter of oolicy, auto cannot be reenqaaed until the 'bad' status of Drime control measurements or actuator haa been specificially reset by the operator from the aesk- In some cases the measurement must lie within a specified ranne before the 'reset* instruction is accented but in others this is not only not appropriate but inadvisable. Tor safety reasons no loop ever self selects auto, even when *had data1 ''laps are cleared. Similarly, in some cases the selection of auto on the outer loop of a cascaded pair automatically selects auto on the inner loop whilst interlocks prevent this haopenina in other cases.

"reet care is taken with toe alarm and interlock system in order to maximise both system inteqrity and 'auto' looo status availability.

2.11 ^isolav md Command System

'he unit aerator is provided with a modular iesk to enable him to closely xonitor and suoervise oiant behaviour in 3 fashion not previously attainaDle. At "Udcot the desk Tiodificationa *ere incorporated directly into the normal "esk whilst 3t Pembroke a comoact extension *vis initially ?een addec. ria.3b illustrates this dssk extension 'based on a ~2mm DVi grid) *hich in a lenath of only AfV enccmoosses *"ive najor areas 3f control conoustion. temoerature, "eea. burners and ijnit suoervision/load) olus 'lisolav selection ror boch 1'0iI'3.

Ul joeratpr oo.Tmunication with the computer is oiaital, each aushhutton asserting dioital status inouts in the apprcoriate oiant area computer wnilst acceptance and status is indicated to :he operator by digital reed reiavs liqnting pushbutton lamps. The oushhuttona ire laid out in nlant areas radially r'rom the central •sunervisorv control and oisolav racilities. The individual actuator "tendbv nanual olaaues ure arranaen 3t the ^ack since ^.hey are intended for use only in the event of comourer -"ailure.

*11 normal xierations reouire a seouence of cush buttons 'tynieallv two or three'1 to be pressed to select ann implement 3 "unction. This nultiolexinq of buttons ensures a hioh standard of safetv and yet nakes for a very comoact, °asy to use. ^vstem. "or example, there is normally onlv one VM - raise/lower plaaue, each additional looo reouirinq the addition of only one extra pushbutton. Additional non-standard or unexoected ^unctions can readily be iricoroorated at any itaoe of the development since the function is determined solely bv applications software and the label •attach to the oushbuttpn.

The central supervisory area provides additipnal operator facilities. cor -xamole a combined raise/lower njmber oad :G orovided •so that 1ata can be entered numerically, or using incninq or continuous raise/lower technioues. [noortant system oarameters can be cnanaeo from the desk but only on insertion of a

. Too section - Permanent selected alarm summerv.

- :'entrc section - °riniary data dnd alarm rormats

. lottom -jection - Hiqh -5Deed interactive disolav for individual looos.

7he interactive ^rea updates every '1.5 -jeconds and self updates whenever sn operator selects a looo anvwhere in the system. This provides ^"ine raise/iower resolution and raoid interrogation caoabilitv independently of the ~iain disnlav format. °rimarv disolav formats update typically every 2.H seconds and are selected bv cnosina 01 u r =eoui;ntiai lv olant area, type 'data, alarm; and numher 'I- . owt*verf anain or raoid interroaation durino transients/incidents-»kev cpntrol formats are available bv sinolt? pushriutton renuest. The alarm summary classifies ill alarm conditions into one of four :ateqories &nn "lisplavs these .is in table 1.

Extensive use is nade of simple, cleerlv understood lisplavs usina dounli: size characters, colour ;ortinot revprsR video ind flashina ?ffect,s. ; common standarq has ^een estahlisned ann adopted For all lisolav *brnats •.ithin the system. The -^ain features or' this are •summarised in taole 2,

The second olant Tonitor VOil-provides -1 comnrehensive net of trends, ^istooram etc- laain, careful ^toice -w* colour afvi -liaplav nattern readily communicates information to onerators without recourse to detailed Tornat jnaivsis. \t ^idcot, hack >JP control lisnlavs are -ivailahle on the second vnn

3. IMPLEMENTATION EXPERIENCE

3. I. Projec t Scracegy

Previous experience had shown Chat in dealing with complex problems of Che kind presented by Pembroke and Didcot that CECB engineers were best qualified to write the applications software. A previous scheme had therefore been attempted on this basis with the commercial manufacturer supplying fully cesteu hardware and system software. However, both proved co have technical problems and much manpower was wasted in persuading the manufacturer to correct faults. Severe problems were encountered with the cocc:i»rcial confidentiality aspects of che system software. With this background. South West Region therefore adopted the following project strategy :

1. Control system strategy design and development - CECB

2. System software development - CEGB

3. Application software - CEGB

4. Engineering details - CEGB

5. Hardware commissioning - CEGB

6. Hardware design and supply - CONTRACTOR

7. Demonstrations of hardware performance - CONTRACTOR

This has proved to be a successful method of implementing a major refurbishnenx requiring significant technical developments and innovation for their success. The interface with che contractor is well defined, the aost difficult area being diagnosis of hardware/system software faults. There is scope for contractors blaming hardware faults on Che system software which is strictly not his responsibility. However, in practice, given ready access to the system software, the problems encountered appear to be fewer than those occurring under turnkey arrangements. Furthermore, this approach allows CEGB staff to accrue detailed knowledge of hardware characteristics thac would otherwise be difficult to obtain. This is extremely useful in assessing equipment capability, integrity and longer tera maintenance/training requirements.

The systems for Pembroke and Didcot were therefore ordered in the second half of !977 on an equipment only basis. Separate contracts were awarded for the computer equipment and the unit desk codifications in order to make best use of specific contractors expertise. >bst or the hardware was scill at che design/prococype stage and almost no reLevant application software existed. However, in parallel with che rain contractors, an experimental system employing prototype equipment was installed on one unit. The inherent flexibility of a computer based system running a high level language allowed rapid development and proving of applications software and provided useful sice based experience with key pieces of commercial hardware.

3.2. System Testing and Installation

Before commissioning of the equipment commenced boch computer hardware and system software were subjected co rigorous testing procedures. These consisted of :

Factory

>bdule development/testing using contractors own diagnostic software.

System integration testing using contractors own diagnostic software.

Module testing using CEGB system software and special purpose application software.

System integration testing using CEGB system software and special purpose application software.

Partial system testing using CEG3 software under harsh enviromcental conditions.

Site

System integration testing using contractors own diagnostic software.

Partial system testing (of experimental equipment) using CEGB system software and real application software.

System integration testing using CEG3 system software and special purpose application, software. - 205 -

The most important point3 to note are firstly that much tine was saved by factory testing prior to shipment to site ond secondly that complete hardware system testing with the final software system revealed a ho3t of hardware flnw3 and module interactions previously undetected by the contractors own diaqnostic3. Indeed it wag 3Dme oix months after the contractor first declared the system ready for shipment that it passed the system tester written by CEXS staff. The whole testinq procedure was repeated at site where further problems were detected and corrected.

The actual installation of the computer and desk equipment was simplified by the use of flying leads throughout. Stntion stoff were then able to install cabling, recks, marshalling cubicles, auxiliary power supplies, instrumentation and modified actuator head3 whilst awaiting delivery of desk modules and computer equi, '~nt. Final installation was then merely a case of terminating flying leads in accordance with a predefined schedule. Some detailed problems arose but generally the approach was successful and can be recommended.

3.3 Commissioning Procedures

Commissioning of the new system represented a major task since:

1. It had to be carried out with the minimum interference to high merit flexible plant.

2. Trc csferring an actuator to computer control irrevocably put the existing controls out of service.

3. Testing had to cover both equipment and computer software.

Cxtensive documentation was produced to cover both individual hardware item3 and complete system functional checks. The latter covered all alarms and interlock conditions as well as modulating control performance. Acceptance of each individual test carried out was dependent not only on correct behaviour of the appropriate plant area computer but also the associated communications and display system software in the display computers.

In order to avoid prejudicing plant output or safety, loops were commissioned in a planned sequence with two or three week gaps allowed for evaluation purposes. Applications software wa3 generally tested using plant simulation but critical looos software was also verified on the experimental system (which could be switched out of service if required) prior to final commissioning.

Development of the system continues after initial commissioning in ordar to incorporate operational experience and optimise performance. Where minor changes occur, these are directly incorporated and documented to the appronriate standard. However, where significant changes occur, the complete set of system functional checks is repeated. Only in this way can 3ysttm integrity be maintained. The procedure is tedious but experience has shown it to be worthwhile.

3. A System Maintenance and Management

The development of new software and control strategies is the responsibility of specialist regional staff. However, at all times, station staff are responsible for hardware maintenance and maintenance/nnnagenant of commissioned software.

Hardware maintenance is covered by shift staff up to a specific point and thereafter it is classified 83 an area call for more skilled day 3taff. Qn interesting result of the comorehensive I/O data checking, alarms and interlocks incorporated into the application software is that the system provides a hiqh degree of self diagnostic fault finding. Interrogation of the appropriate plant area alarm formats usually leads maintenance staff to the cause of the problem.

Cards within the plant area computer are tested using special purDOse diaqnostic software whilst the intelligent actuator drive cards are tested using a special test box. This allows rapid testing of every facility on the enrd and it is a concept that could usefully be extended. 4 selection of spare cards is maintained running in a hot spares rack. This is mobile and can be located next to a plant area computer to assist fault finding.

Software maintenance procedures are an important part of system maintenance. Maintenance is required to incorporate changes brought about by operational experience, the development of a new control strategy, reque3ts for additional desk or display facilities or changes in system software. The software system is deliberately designed to allow such changes to be readily incorporated. However, unless the associated documentation is submitted to the Station staff responsible for software manaqement/maintenance the chanqes are not formally recorded or accepted. Only by rigorous adherence to this rule can software integrity be maintained.

3.5 Traininn

Whilst Regional staff had accumulated several years experience with small real time computer systems, the majority of Station staff involved in the Pembroke and Oidcot schemes had no previous direct experience of this kind of work. In addition, unit operators were faced with a new unit desk, new layout and operational procedures. This posed a major challenge to those regional/national staff involved in the schemes inceDtion to demonstrate that non specialist staff could be successfully trained to operate and maintain the new systems. - 206 -

The atntegy adopted con be suimiarised as follows:

1. The installation of a experimental prototype sy3tem embracing all key features of the system. This familiarised mainter^nce and ooerations staff with the new concepts but allowed then to instantly re- select the original control system if problems arose.

2. The Station maintenance staff were involved in equipment testing at the contractors factory and were totally responsible for testing and maintenance of equipment as it arrived on 3ite. Naturally, expert guidance and advice was given thus providing 'on the job' training.

3. Key operations staff were involved in system development, testing and commissioning. They were then responsible for training specific shift operator foremen who in turn trained all other unit operators. Simulation techniques were employed to assist with the training which was almost exclusively carried out on the installed eouipment.

The strategy has been successful and Station staff are now fully competent in the maintenance and operation of the computer based systems.

3.6 Financial

The cost of the schemes at Pembroke at Oidcot can be assessed on the basis of expenditure of equipment and contractors labour, together with CEG8 manpower effort. The following is an approximate breakdown of the expenditure and manpower consumed.

Cost (S) Manpower (%)

Actuator modifications 5.8 Commissioning 8.0

New transmitters/instrumentation 6.6 Installation 13.3

Installation 13.2 System software (shared between projects) 10.6

Desk 9.1 Control system development 3D.8

Computer equipment 65.3 Engineering and design 37.3 "100.0 mo. n

It is often difficult to assign financial benefits to improved control/display systems. However indications nre that costs will be recovered in three to five years due to:

. Reduced plant damage

. Improved efficiency

However, there are many other less tangible but important benefits accruing from the approach outlined in this paper including:

. Improved environmental performance

. Improved plant flexibility

. Improved equipment reliability end easier maintenance.

• Adaptability to future unforseen events

. Imoroved system integrity

. Increased operator awareness of plant conditions, and control system status/behaviour. b. DISTRIBUTED COMPUTER CONTROL/DISPLAY SYSTEMS FOR NUCLEAR PLANT

4.1 Nuclear Plnnt Characteristics and Requirements

Many of the system requirements, design concents and dinitBl techniques described in this paper are relevant in the context of Nuclear Plant Control. However, there are substantial differences which must be considered in the application of distributed computer control to the latter. These include:

. Increased numbers of plant input signals.

. Increased numbers of important control Ioops/actuator3.

. Increased requirement for high reliability systems with predictable, fail safe characteristics. - 207 -

.. Increased requirement for failure diaqnostic capability.

. Increased requirement for plant condition monitoring, intelligent data analysis and effective corrmunication with operations otaff.

. Increased need for methods of system teatinq ond evaluation.

. Increased need for digitul control techniques capable of rapidly adaptinq to unexpected situations arid optimising plant performance in response to particular sets of operational constraints.

Whilst the basic approach outlined in this paper is believed to be correct, the author believes that further developments are needed to cope with the additional requirements and constraints outlined above. Principally, these involve:

1. Higher levels of functional distribution

2. Development of distributed hardware structures

3. Consolidation of the integrated control, display and commend system concepts-

4. Development of improved digital techniques for failure detection, control and system testing.

5- Software. a.2 Future Developments

Based on che expecience of cte Pembroke and Didcot Projects, consideration in South West Reqion is currently beinq given to the future application of distributed computing concepts to Nuclear plant. The following appear to be important areas of development:

Functional Distribution

In the recent past, the plant area computer concept wa3 the only economic solution available. However, on nuclear plant, the loss of 10-20 reactor control loops due to computer failure is not likely to be acceptable. One can, of course, introduce redundancy but this inevitnbley introduces complexity. The alternative is to increase the level of functional distribution to the point where loss of one or more computers does notreprssent a safety hazard or en operational constraint. Despite its many specific limitations, the intelligent actuator drive card developed for Pembroke and Didcot offers simplicity, minimal loop interaction under failure conditions and easy maintenance/test procedures. It is thus felt that for nuclear plant the present plant area computer with its multiplexed scanner and actuator drive system should be replaced by a series of smaller systems containing simplified non-multiplexed E/0. Thus a sinqle system conteinina perhaps 30 cards would be replace by ten systems of 3 cards or even 30 sinqle cards. F.ach of these small systems would still be capable of running a hiqh level ianguaqe such as the CEGB standard CUTLASS in order to achieve the high performance standards required. With steadily reducing semi-conductor memory costs such a hic/i level of distribution is likely to be economic, particularly if the distributed hard-ware structure described below is also adopted.

Distributed Hardware Structure

In implementing a hiohly distributed computer system two critical hardware areas are plant input/output interfacing and inter-processor communications. The author believes that multiplexed eneloQue scanners for plant data input could be replaced by digital instrumentation and digital transmitters. These could be interfaced to the computer via standard opto isolated digital input/outout parts which are readily available on almost every manufacturers system. Figure 4(a) illustrates schematically auch a digital transmitter suitable for direct connection to thermocouple signals. The data is supplied serially in response to computer requesta thus simplifying hardware design and makinq maximum use of the computer. Since many such modules can be driven in parallel the baud rate need be very low (e.g. 300 baud) thus eliminating the need for complicated line driving, ^t auch low data rates the c.p.u overhead in the computer is not likely to be a problem. High common mode and noise rejection capability is inherent in the design. Since there are many thermocouple signals on a nuclear station the transmitter module could be produced in volume to a defined specification. In such a case, cost per channel is believed to be comparable with multiplexed scanners but with the significant advantage that there is minimal interaction between one channel and another.

The use of low speed serial digital transnisDion using oDto isolators could be extended to intelligent actuator drive systems. Such devices are under development elsewhere in the CHIfl with the result that a distributed hardware structure with an all digital computer/plant interface could thus be proposed. The syst*jn would thus take the General form suggested in Fig.4b. Plant interfacing (tD a defined standard) is almost exclusively achieved through standard isoleted digital parts the only exceptions beinq the serial ports for hosting and intemrocessor communications.

For interprocesaar communications, it is suggested that a high speed (e.g. H.25M baud) serial bus based on fibre optic technology could provide a highly reliable, secure solution. Cheap LED based transmitter/receiver pairs are now available as are multiple pc *" star couplers. Thus serial bus configurations based on both star and ring structures are now possible. V is most appropriate is a matter for development.

Resides offering technical odvantoges, it is beli interface could be helpfull in construe*1— ~c — specified at the usual early stage but „ the non-trivial problem of equipment bee

Display and Command Systems

The provision of radically improved disolny and command facilifcies on the PenbroL-e and Didcot unit desks has been beneficial. The unit operators have a much improved appreciation of plant conditions and consequences of their actions. Such fnctora are important in the context of nuclear plant operation and is expected that further development work will continue. Mew technologies such 09 touch sensitive \DU screens will undoubtedly suggest alternative strategies for unit desk layout and interaction with the computer system. Experimental work both on simulated and real power plant will probably be necessary to evaluate the various eroonomic and safety factors involved. - 208 -

Oiqitnl Techniques

There is considerable potential for exploiting the powerful fncilitirn now offerer) by low rr-j-r-t r.n~.put .;>r nno S^i-h level, real tir** software systems. Evidence ta date indicotrs tlmt the npplicttion rcndrjm c-t innt: o> n:d control theory using small scale plant models can moke a significant contribution to:

. Methods of measurement iind actuator failure detection.

. Oato .smoothing and nnalysis.

. Reduction in actuator wear and maintenance requirements.

. Robust, adaptive control algorithms requiring little or no specification or tuning.

. Supervisory techniques for real time optimisation of total plant performance.

The ubove, in conjunction with the capability to rapidly devnlop nppJicntion r.oftv.nre tnUort-1 to IXJJVL- •:?••<:±fie problems,should Dignificantly improve control system integrity ond parfornnnco on nude or pJont.

Software

Software performance and reliability will always be n limiting foe Kir on any ennuter tvrsed cnntrol or rtirpLtiy system. In a nuclear environment, demonstration and tea ting of software rnl inhi lity ri'twiim nn :ntiortnnr ;M-,;J active area of research. Thu3 proceduren for software modification need oir*;ful attonticn. ^xri'fi'i(?nrp f-n'* •;>hown that interactive, high level software provides the engineer with a rapid r^chud of ^ndifyim mvjlicaticn software. Indeed minor changes can often beneficially be made on-line whr.-n oiierotinq on cunvcntinnal plant.

In a nuclear environment it is difficult to see how such a net hod of working could t>e accept RR1 . fhpre *i 1 ] thi;;i be n nef?d for methods cf approved automatic verification of software changes. Some safety features can, of course, be inbuilt to the system software but many checks are highly application snecif ic. H.p veri^icri* : cm procedures will need to be reasonably time efficient if soine of the benefits of engineers tiling hitf-* lrveJ software systems are not to be lost.

CflNCLUSIONS

This paper hns described the cteaiqn concepts behind the Pembroke end Didcot distributed computer control and .Jisplriy systems. fmolementation experience has also been reported as havincj sr^e prop03a La on further devel cprp-nt:-, for nuclear plant application. Judged by the initial renuircments described in Section 2.1 th*» pmjcctn lire a 5UCC*-:KI. High sttmdords of inteqrity and performance hove been achieved on canvcntionnl plant- It is therefore expectrH Ik-jt the distributed computer approach will be increasingly apnlied to nuclear plant.

The design and imple;menl'ation of the systems described in this paper has been the rn3u 11 of join" station, r*jtnonn ana* national stoff co-operotion in which the author hna been priveleged to takt» part.

This paner has been produced by permission 01 zhc Director ConcraL of SuuLh West Ru^ior. ->i cne Centr.ii EK'cinci ;v C£r.er3cine Board.

AG0105JNW. - 209 -

TABLE 1 TYPICAL CONTROL JOB STRUCTURE

Analogue scan/validity check tone per scan race) Digital scan Analogue data scaling Analogue da'.:a smooching Control action calculation (I per loop typically) Control loop parameter adoption and . _hedi»ling Actuator drive Desk command handling and Lamp driving Interprocessor communications for control Interprocessor coanunicacions for general display Intcrprocessor communications for high speed display Software uatchdog Hardware uatchdog Alarm and limit calculations Software interlocks System error logging/printing General interactive programmes for debugging

TABLE 2 VDtf COLOUR CODING

TEXT FOREGROUND BACKGROUND BLINK MEANING COLOUR COLOUR

Alarm summary - plane limit \ BLACK BLACK SO nsasurement ^ RED RED YES New alarra - format not selected actuator i computer }

Portrait background CREEN BLACK SO Other colours used for mimics Foreground data YELLOW BLACK SO Normal valid data BLACK. YELLOW so Plant limit alarm BLACK YELLOW YES Urgenc plant limit alarm RED BLACK NO 'Bad' data BLACK RED NO Data not being updated

Loop status CA BLUE BLACK NO Cairouter auto CM YELLOW BLACK NO Computer manual CM BLACK YELLOW NO Computer manual - auto unavailable TR CREEN BLACK SO Tracking SA RED BLACK NO Standby atico DM RED BLACK NO Desk manual SM GREEN BLACK NO Standby nanual SM BLACK GREEN NO Standby manual - computer not available

TABLE 3 - MEASUREMENT AND ACTUATOR FAILURE CRITERIA

TEST RESULT

Rate or change exceeded 'Bad' measurement Rate ox change exceeded Ignore — use model estimate Race of change exceeded 'N' times in 'J' samples 'Bad' ueasurement Above max.operating limit 'Bad' oeasu: Below min,operating limit 'Bad' measu: •ertent Model prediction error variance above max,limit Alarm condi >bdel prediction error variance below min.limit Alarm cor.di ion (signal frozen?) ! SingLe scanner error Ignore - u nodel estimate or last scan I Consequent scanner errors 'Bad' measui ecsenc f Scanner input overload Fre e ze ac I st valid measurement or signal' i

Actuator drive card responds incorrectly 'Bad' actuator Actuator drive card responds incorrectly Ignore Actuator drive card responds incorrectly 'Bad' actuator 'M' ciaes in 'J1 calls ' Actuator 'ADC' measurements discrepancy j Alarm condition Measured and computed actuator position discrepancy j 'Bad' actuator - 210 -

Figure is Pembroke Distributed System

DISPLAY OS DISPLAV ON UNIT DESK UNIT DESK

BACKINC PLANT MONITOR STORE UNIT SUPERVISOR t SYSTEM HOST 4 LOAD CONTROL COMPUTER FLOPPY COMPUTER BACKING DISC STORE T I .-J COMMUNICATION LINKS

I.I COMBUSTION BURNER FEED TEMPERATURE CONTROL MANAGEMENT CONTROL CONTROL UJD OPTIMISATION COMPUTER COMPUTER COMPUTER COMPUTER

SACK IT COMMUNICATION LISX

Figure Jb Didcot Distributed System

DISPLAY DISPLAY ON ON UNIT DESK UNIT DESK

m •• ••••« a^ IM

FEED I COMBUSTION I CONTROL I CONTROL OWuTER :======! COMPUTEP. I

PLANT ms'ITOR & SYSTEM HOST COMMUNICATION L1HKS VUSl MONITOR II TERMINAL COMPUTER

BACKING BACKING STORE STORE

STEAM TEMPERA- UNIT SUPERVISOR TURE CONTROL 1 !.OAD COSTP-OL COMPUTER COMPUTER J - 211 -

UU1LJ1

AUXILIARY DATA » FLANT MO:: [TDK rLA.".7 co.'.'DZT ro.. :J)*::TO:: ANd DAIA ANALYEIS/RHCORDa.

,

SUPERVISORY CONTROL DATA - UNIT SLPLKVISOK SLTEXVISOliV AUTO/MANUAL l

PLANT AREA PRIMARY CONTROL DATA • COMPUTER COMPUT™. AITO.':HMAL

INTELLIGENT 5INO.E LOOP PLAXT DATA ACTL'ATOR DRIV1 SIAXD3V ALTC CAM

TO STEPP-K MTOR FigurE 2b Typical Pliant firea

"LAST SIGNALS •

LSI-! I 6- CHANNEL .MICRO- ANALOGUE PROCESSOR H SERIAL I m 28k X 16 BIT DATA MEMORY WITH INTERCHANGE BATTERY BACK-C MODULE (DIM)

32k K 16 BIT OPTO-ISOLATED __ 50V D.C BACKING STATUS INPUT _- STATUS SICNAiS MEMORY | CARD 32 BITS — e.f FROM DESK

WATCHDOO RELAY ISOLATED _^i.g 50V D.C TO DEIECT STATUS O'JTPLT _» STATUS SI" ALS COMPL1ER FALLIJTIE MODULI 32 BITS — TO DESK LAMP?

I SOLID STATE I DAIA I COLO'JR DISPLAY [ INTERCHANGE j INTERFACE ! MODULI (DIM- j (YTV-3OH | PARALLEL DATA LINE TO COLOUR >iONITOR 0:: DESK DATA CONTROL OUTPUT INTERCHWCE INTERFACE — TO PLANT «ODO.E '.DIM' (STEPPER HOTOr.

DATA Z3C7EXC1LANGE MDOU.E IDIS:1 - 212 -

tyua uiBGiatiLjdii i i_ii sue uuiu

MULTIPLE FLOATING DC/DC CONVERTOR ABC ~L fLAST DATA BATTER* 30V DC FLOATING ADC

BATIE10. 50V DC OPTO-ISOLATED C/O TO ACTUATOR STEPPER DRIVE RELAY

I \K avTT" STAND 5HSCq- ACTCATOR POSITION I oVE PROM DESK PLAQU1UE L PROCESS VARIABLE

:ARD ADR. OPTO ISOLATED nDECODE 3 BIT DICIS I^tTER-CAED COMHIWICATIOSS OPTO ISOLATED 8 BIT DIGOUT

OPTO ISOLATED I/O

DIM DATA BUS

Figure 3b PesnbrD^e integrated Corfrdi/Oispiay - 213 -

DC /DC OS AC DC

REMOTE DIGITAL TRANSMITTER

STANDARD OPTC-ISOLATED DIGITAL INPOT POETS SHIFT DIGITAL BUFFER 12 BIT REGISTER AMP ADC LOGIC CURRENT LOOP STAKDARH OPTO- ISOLATED DIGITAL OUTPUT POP.TS

Figure Qb Highly Distributed System modules

ANALOG PLANT DATA

Di:::iL DIGITAL rj a DESK P/B LED LAMPS PLANT STATUS INFORMATION

11 OPTO D1GIN/OUI PAIRS OPTO DICOLT OPTO ISOLATED D1GINS

nLL MICROPROCESSOR CPU/RAM/ROM/WATCHDOG DIGITAL ICTEPJACE I I I [ OPTO DICIS/DIGOUT PUES FIBRE OPTIC CUP.REXT LOO- I OPTO DIGOIT SERIAL HOST SERIAL PORT

!

AfTUATOP. ACTUATO.". HICil Si'iED HICil SPtiD ALAK.MS WITH WITH S;nlAL EL'S SERIAL BUS INTERLOCKS DIGITAL DIGITAL INTERFACE INTERFACE COXTmi DATA DISPLAV DATA