168 - Black DERoverlayAPPARATEWELT

EIGENWELT

168

Page IMAGE PROCESSIN G Experimental Television Center, 1986

Sherry Miller Hocking & Richard Brewster

Thefollowing are excerptsfrom the Experimental Television Center (ETC) in Owego, N.Y., "Image Processing Manual."The manual was written in the early 1980's by SherryMiller Hocking, in collaboration with Richard Brewster, and is an outgrowth ofa prior "how-to" manual in 1976, cataloguing the video equipment atETC. The manual contains a review offundamental video principles contained in waveforms, timing and synchronization. It acts as an educational and training vehiclefor reference by visitors to the Center, but also includes descriptions ofunique video art tools and how to use them These depictions are accompanied with an underlying ideology : thatinformed exposure to the language and expressions oftechnological art tools will point toward their creative use in the video arts. -J.S.

SIGNALS : As a kinetic as well as an electronic aesthetic definitions ." There are several general form, video concerns itself with the time/space categories of signals specified by the processing equation. Video image movement occurs within a system which include video, audio, control, and predetermined space, and the process ofchange, by synchronizing or timing signals; as shall be seen, definition, is a temporal event occupying a specific signals may perform functions within several cate- length of time. Changes in the time frame or time gories. One signal, for example, can influence an base of the signals which define the image result in image and also produce a sound. The term "signal," changes in the duration of images and in the derived from the Latin signummeaning sign, refers locations of sections of images within the two- in a general sense to the use of conventional sym- dimensional space of the image's display. On the bols which refer to a verbal description ofa concept level of electronics, the very construction of the or event. A signal then is a translation of the video image, its generation as well as its display, is description of an event from one set of symbols to time dependent . The composition of the signal, another set of codes. It is the representation ofthe then, defines the visual nature of the image as it event. The signal conveys information concerning existsin time; it dictates boththe appearance ofthe the state of the event in any given instant through single "still" image, which exists within a specific time. Video images are codes of information con- length of time, and its behavior through time. veyed by signals . The specific video picture On a primary level, the signal can be viewed as information conveyed by a signal is in the form of the art-making material; the creation of an elec- changes in voltage; changes in voltage dictate tronic image is an architectural process and con- changes in the information being carried. Voltage structed in time. The signal refers to changes in changes can be categorized in terms of changes of energy levels and reveals a physical nature by strength, increased or decreased voltage, and forming andinfluencing images. Specific devices in changes of direction, alternating or direct current an electronic image processing system perform signals. specific functions or operations on signals, generat- Electricity is usually defined as the orderly ing and altering the signals, or codes, and therefore movement ofelectrons through a conductive mate- theresulting images.Inthis way thehardware ofthe rial. Whenavoltage is appliedto a conductor, a force system can be viewed, in part, as a "carrier of field is established which causes electron move- 169

Page 169 - Black overlay EIGENWELT DER APPARATEWELT

ment and therefore electrical energy. The rate at the foundation ofmany scan processing devices . which electrons move past a given point is a meas- Electrical signals have a waveform which con- ure of current strength expressed in ampheres or veys the time limits ofthe event, the strength ofthe amps . When a current of one amp flows through a event and the direction of change of the event conductor, 6x 10 raisedto the 18thpowerelectrons relative to abase line or reference point. The electri- are passing a given point each second. Electrons cal signal can be graphically displayedin a number move only when an unbalanced electrical force or of ways. potential differenceis present; voltage is a measure On a fundamental level the waveform of an of the force causing electronic motion and is often electrical signal is displayed as anXY plot ofvoltage described as electrical force or pressure. Ground is changing throughtime. By convention, the horizon- a referencepointwhichhas zero potential energy or tal orX axis represents the time dimension and the zero volts. Because of the properties and dimen- vertical or Y axis represents the voltage or signal sions of the conductive material, there is a resis- strength. An oscilloscope is a test instrumentwhich tance to the flow of electrons . Resistance is often visuallydisplays any electricalsignal as a change in likenedtofriction andis measured in ohms. It refers voltage through time. A waveform monitor is a to the impedance ofa current flowandresults in the specialized oscilloscope which graphically portrays dissipation of power in the form of heat. Although the composite video signal. the degree ofresistance is dependent on thenature Indiscussing a black andwhitevideo signal, the ofthe material, the resistance ofany given material range of the video or picture portion of the entire is constant. signal provides an indication ofthe relative bright- Ohm's Law expresses the relationship between ness or darkness of the image represented by the current, resistance, and voltage; it states that volt- signal. Ahighervoltage level measuredonthe Y axis age equals current, measured in amps (I), multi- indicates awhiter portion ofthe imagewhile a lower pliedbyresistance, measuredin ohms (R). Because level indicates a blacker portion of the image. the resistance of material does not change, voltage The concept of graphic representation of wave- is proportional to current. Increases ordecreases in forms is crucial to the understanding of an image voltage simultaneously produce proportional in- processing system. As we will see, the time dimen- creases or decreases in current. A watt is a unit of sion or time frame of the signal may be extremely electrical power produced when one volt causes a brief as in the representation of a single line of the current of one amp to flow through a circuit. video image which occurs in 1 / 15,750th of a sec- Two of the effects of electrical current are heat ond; the timeframe mayalso be relativelylongas in and magnetism. The resistance of the conductive the representation ofa frame ofvideo, a collection of material to the flow ofelectrons produces heat; this 525 lines which occurs in 1 /30th of a second. The is easily demonstrated by the warmth ofan incan- basicXYformat can also be extended to incorporate descentelectriclightbulb. Anelectrical current also a third parameter represented along the Z axis induces a magnetic field; this can be seen in the which can be conceived ofas a vector extending out deflection of a compass needle placed near a wire into space. This notionis important to understand- through which a direct and steady current is flow- ing the technique of colorization. Woody Vasulka ing. The force ofthe magnetic field is at right angles developed a technique using this type of vector to the direction of current flow. Michael Faraday in diagramto locate parameters ofthe time frame ofa 1822 demonstrated the reverse ofthis lawbyshow- video image, employing the Rutt/Etra Scan Proces- ing that an electrical current can be induced by a sor. This graphic representation defines the line magnetic field. A flow ofelectrons can thus produce rate, field rate and intensity information. a magneticfield and is also produced by a magnetic A waveform can be described in terms of its field; a magnetic field can therefore be employed as shape, the number oftimes it repeats per time unit, a means of controlling the movement of a flow of its strength, placement and direction. electrons, a process basic to the functioning of the A waveform may begin at any point but when it scanmotions inavideo camera or monitorand also returns to the point past which it started, the

170

Page 170 - Black overlay HOCKING & HO C KIN G

waveform has completed one cycle. Cycle refers to is abbreviated Ppv. the completion of one rise, fall and return of the The term gain defines the total peak to peak signal. It is important to notethat thewaveformmay voltage excursion ofa given signal and indicates the pass through a number of times the particular relative strength of the signal. An increase in the voltage at which it began before one cycle is com- gain of the signal causes an increase in the signal pleted. For example, the sine wave begins at the level and conversely, a decrease in the gain results point exactly half way through one cycle before in a decrease in the signal level; gain thus equates ending atthis value at the second cycle after begin- with the amount of amplification of the signal. It ning. The time it takes for one waveform to be expresses the ratio of the amplitude of the input completed is called the period ofthe waveform. The signal to the amplitude of the output signal. term periodic refers to a waveform wherein a regu- The term attenuate means to reduce inforce or lar, repeating pattern is observable as the voltage intensity; with respect to an electrical signal; at- changes through time; sine, square, and triangle tenuation refers to the lowering ofthe amplitude of are all periodic waveforms with specific shapes. the signal with respect to ground. Instantaneous Sine, square, andtriangleare thebasic waveshapes amplitude refers to the distance between a specific which can be combined with each other to produce point in the waveform and the base line or ground complexwaveforms. As we will see, the sine wave is and is expressed in volts. actually the fundamental form from which square The signal can be further defined by its positive and triangle are derived. andnegative voltage dimension. An AC or alternat- Noise refers to a signal which is not periodic but ing current refers to a signal which has both a random in nature, with unpredictably varying sig- positive and negative voltage dimension . An AC nal strengths; it is often defined as extraneous voltage rises to a maximum point and then falls information present in the signal which is deter- through zero to a negative voltage level which is mined to be undesirable eitherthrough the process equal in amplitude to the maximum. A DC or direct of comparing the signal to a reference signal or by current voltage does not change direction; the sig- personal decision. Noise can be manifested either nal does not vary and is always either positive or aurally orvisually and can also be used as a control. negative. Polarity refers to the existence of two Snowis an exampleofvideo noise; snowis arandom opposite changes, one positive and the other nega- organization ofmonochromatic blotches and ispart tive. When a signal is inverted, the polarity of the of the vocabulary ofimage processing because it is signal is reversed. Positive signals become negative used as an image element in composition in much andnegative become positive. In the case ofa black the same way that audio noise is used in electronic andwhitepicturesignal, allthe blackbecomewhite. music composition. The term bias indicates the repositioning of the The number oftimes a waveform is repeated per signal relative to ground; the absolute amplitude unit oftimeis called the frequencyofthe waveform; and frequency of the signal are unchanged. The frequency then implies the speed of the signal. The termphase refers to the relative timingofone signal numberofcycles thesignal completes inone second in relation to another signal. If one signal is "in ismeasuredin cycles persecond expressedinHertz phase" with another, they both possess identical or Hz. timing and have begun at the same instant. The amplitude of the signal refers to the maxi- A waveform may also be frequency and ampli- mum strength attained bythe signal. It is measured tude modulated. In amplitude modulation, the by the height of the waveform expressed in volts. amplitude ofthe signal, called the carrierwaveform The signal may have both a positive and negative is determined by the amplitude of a second control voltage dimension . Thereference line ofzero volts is signalcalledthemodulatingsignalwhich isinputto called ground. The total voltage excursion of the a function generator. In this case, the frequency of signal, obtained by the addition of the maximum the output remains the same as the normal output. positive and maximum negative points reached by The amplitude of the modulated or output signal the signal, isreferred to as peakto peakvoltage and changes in proportionto the amplitude ofthemodu-

Page 171 - Black overlay EIGENWELT DER APPARATEWELT

lating or control signal. In frequency modulation, allowable range, anyvalue can be input or output. theamplitude oftheoutputsignalremainsthesame Conventional video cameras are analog systems; as the normal outputsignalbut thefrequency ofthe the video signal continuouslyvaries andrepresents output signal is determined by the frequency of a a pattern of lights and darks at which the camera second signal, the modulating signal, which is fed points. Avideo monitoris also an analog device, but into the function generator. The change in fre- the representation flow is reversed in direction. The quency of the modulated signal is proportional to pattern oflights and darks, the image onthe screen, the amplitude of the modulating or control signal. represents a continually varying voltage, the video Modulation refers to the process of changing some signal. A sine wave is also an example of an analog characteristic ofa signal so that the changes are in signal. The sinewave oscillator is an analog system step or synchronized with the values of a second which is specialized; it always produces a specific signal as they both change through time. waveshape, the sine wave. In the process of filtering, certain predeter- Digital signals are frequently explained as sig- minedinformation is masked off, allowing a specific nals which describe information consisting of dis- portionofdatato pass throughunchanged whilethe crete levels or parts. Digital signals are concerned remaining is eliminated. Mostcommonly, filters act with stepped information ; the change from one on frequency ranges although they can also act on value to another in a waveform does not vary amplitude ranges. For example, a low pass filter continuously but, with some qualification, occurs cuts offhigh frequencies whilepassing lowfrequen- instantly. Digital devices are constructed from cies, while a high pass filter rejects low frequencies switcheswhich have only two states; theyare either and passes high frequencies . The cut offfrequency on or off, open or closed. All of the various voltage value can usually be controlled either by manual levels in a digital waveform must be expressible by adjustment orwith the technique ofvoltage control. two numbers, onerepresenting the off, closed orlow A variable pass filteris actually a low and high pass state; and the other representing the on, open or filter working in series. The frequency range which high state. One point of an event or voltage can be passes is located between the cut off levels of the represented as a series of open or closed switches; high pass and the low pass filter. The reverse the numberofopen and closed switches is counted, process operates in a notch or band-reject filter. and this information is translated to one value. A When the cut-offfrequencies of both high and low number of these values can thus be constructed pass filters connectin parallel overlap, thefrequen- which will eventually plot a complete waveform. A cies located between the two cut-offfrequencies are digital waveform then has a stepped, square-edged rejected. appearance; the square waveis a simple example of Signals can be further specified as analog or this type of signal. digital structures; the terms refer to ways ofrepre- Several number places may be required to ex- senting or computing changes which occur during press a complex digital signal. Ifwe have a number an event. On a basic level, an analog signal is with one place and each place can only be a zero or frequently explained as describing an event, a volt- a one, then we can use this number to express one age for example, which continuously varies within oftwo states; 0 and 1 . Ifwehave two places andeach its allowable range, ie. a thermometer . The meas- place can be either zero orone, then we can express urement of the temperature is limited only by the four different states: 00, 0 11. The number resolution ofthe scale and how accurately the scale places are calledbits, a contraction ofbinary digits. reading can be estimated. The position ofthe mer- The large number of combinations mathematically cury relative to the scale markings must be esti- possible using onlytwo numbers and a given num- mated. Analog indicates that the signal as meas- ber of places allows for the expression of many ured on a scale represents or is analogous to the signals. Many electronic image processing systems information related by the signal. In a sense the have both analog and digital components and are scalerepresents the event.Analog devices useinfor- often described as hybrid systems. It is important to mation which is constantly varying; within the note that with analog signals the waveform or one

172

Page 172 - Black overlay HOCKING & HO C KIN G

characteristic ofthe waveform is manipulated. With voltage, the control voltage. If the control voltage digital signals, the information about the waveform frequencyis within the range ofhuman hearing the is altered and then used to reconstruct the wave- signal canfunction both as a control voltage and as form. an audible sound; this dual role for signals and the A signal then conveys certain information about resulting relationship between image and sound is an event. It contains anumber ofvariables, such as a technique used frequently in electronic imaging. frequency, amplitude or placement which can be By use of control voltages the problem of continu- changed and controlled. Control over these vari- ously varying changes is overcome; one can move ables is anissuecentral to electronic image process- between discrete values without having to proceed ing. Whether achieved by manual or automatic through intervening values. means, controlis exerted on a signalwhich defines Control voltages can be periodic or non-periodic an image and not the image itself. The achievement waveforms. Because they are signals, control volt- of control over the signals which define images is ages themselves can be processed by techniques important to theuse of electronic imagingas anart- such as mixing or filtering or can be amplitude or making medium. frequency modulated before they are used as con- A potentiometer or pot offers manual control trol signals. Control voltage signals can exert influ- over voltage through the adjustment of a knob. A ences on audio signals, video signals or other con- familiar example ofa pot is the volume control on a trol signals. They can be generated by voltage con- television receiver. Turning the knob results in an trol modules, audio synthesis equipment, or com- increase or decrease of the amplitude or the audio puters. signal and thus an increase or decrease in signal SYNC: In video, the image is actually an elec- strength or loudness. A pot allows only a continu- tronic signal. This video signalhas two basic parts: ous type ofchange over a signal. It is not possible to The section containing picture information, and the move from one discrete setting of a pot to another section containing sync information . Synchroniza- withoutproceeding through all the interveningvolt- tion is derived from the Greek syn and chronos-to agelevels. On a basiclevel, a pot provides a method be together in time; the term implies that several ofmanual control over the signal; the rate ofchange processes are made to occur together in time at the can be altered but is limited by the speed at which same rate so that they are concurrent. For a coher- the knob can be turned by hand and the process of ent picture to beformed which is easily readable to change is always continuous. A pot has three con- the eye and brain, the scanning motions ofboth the nection pointsor terminals.Two ofthe terminalsare image or signal generating device, for example a connected to a material which resists signal flow. camera, and the image or signal display device, the The position ofthe third terminal, called a wiper, is monitor, mustproceedin an orderly and repeatable adjustable along this resistive material. By chang- manner.The scanningprocessesinboth camera and ingthe position ofthewiper by manual adjustment monitor must begin and end at precisely the same ofthe knob, the amountofresistanceto current flow time. The camera and monitor must be synchro- is changed andtherefore the signal. Frequentlypots nized. As the camerabegins scanning the objects in are calibrated, often by a series of relative number front ofit, the monitor begins to scan the line which settings; because the change is continuous, the the camera is scanning. As the camera ends the resolution of the scale to some extent determines scanline, themonitormustalso end thatline. When the accurate repeatability of the manual setting. the camera reaches the bottom of the field, the Control over signal parameters can also be monitor must be exactly in step. Without this syn- achieved by exerting an automatic rather than chronization, the camera image and the monitor manual control over the pots. The technique of image will have no relationship to each other. voltage control in effect allows the pots to be ad- Horizontal sync maintains the horizontal lines in justed by another voltage rather than by hand. The step ; withouthorizontal sync the picture will break principle of voltage control is the control of one up into diagonal lines. Horizontal sync tells the voltage, often called the signal voltage, by another camera and monitor when each horizontal line

173

Page 173 - Black overlay EIGENWELT DER APPARATEWELT

begins and ends. Vertical sync also keeps the pic- the next scan; it is during this interval that the ture stable;withoutthis, theimagewill roll. Vertical horizontal sync pulses are inserted. They insure sync tells the camera and monitor when each field that the line just scanned by the camera can be begins and ends. Both together are essential to a accurately reproduced by the monitor and tell the stable rectangular shape. Sync then can be con- monitor when each line is to be scanned. One ceived of as an electronic grid which provides complete horizontal line is scanned in 63. 5 micro- horizontal and vertical orientation to the image. seconds, or .0000635 seconds. The visible picture Eachvisible lineforming theraster is drawnfrom portion ofthis line takes approximately 52 .7 micro- left to right across the CRT. Before beginning the seconds. The remaining time, 10.8 microseconds, is next line, the beam must return to the left, and this the horizontal blanking period. Theblanking period return must be invisible. During this horizontal consists ofthe frontporch sectionwhich is approxi- retrace period, the beam is blanked out; this proc- mately 1.27microseconds, thehorizontal sync pulse ess and the time interval necessary to perform this and the back porch section each of which are function arecalledhorizontalblanking. Horizontal approximately 4.76 microseconds. The back porch blanking is a part ofsynchronization. At the end of is approximately 3.5 times as long as the front each field the beam mustreturn from the bottom to porch. the top ofthe CRTbefore beginning to scanthe next The vertical sync pulses occur within the blank- field. Again, this vertical retrace is not seen. This ing interval at the beginning of each field. The first process and the interval are referred to as vertical 21 lines ofeach field consist only oftiming informa- blanking. Vertical blanking is also a part of syn- tion. They do not contain any picture information . chronization. They are collectively known as vertical blanking. Each ofthese blanking intervals includes infor- The following 241 .5 lines of the CRT are scanned, mation necessary to maintain proper timing rela- andthen the beamhas traced allofthe picture lines. tionships between camera and monitor so each The period oftimeit takes for the beam to return to begins scanning line and field atthe same moment. the top after each field is scanned is called vertical The information which is contained in the blanking blanking, approximately 1330 microseconds long, intervals is not picture information . The blanking much longer than horizontal blanking. intervals containthetiming signalswhichare called The first series of pulses to occur during the sync pulses. These sync pulses keep the images blankinginterval are six equalization pulses. These stable and accurate in terms of color. are followed by the vertical sync pulse serrations, Sync thus indicates a synchronization process. which arefollowed by another series ofsixequaliza- A number of sync pulses are required by an elec- tion pulses. The duration of each set of pulses or tronic image processing system. Normally these three horizontal lines, is abbreviated 3H. The fre- sync pulses are provided by a sync generator, a quency ofthe equalization pulses is twice the hori- separate device external to the system which pro- zontal frequency. These equalization pulses help to vides the sametiming signals to each ofthe discrete maintain the interlace between fields and also help devices within the system which need sync to oper- to keep the oscillators which control the horizontal ate. The single external sync generator provides scanning in step during the time in which no lines identical sync signals to all of the cameras within are being scanned. The equalization pulses insure the system. that the vertical deflection occurs at the same time One complete horizontal line includes both the as vertical sync. They also keep the horizontal visible picture information and also horizontal deflection in step. blanking. Within the period ofhorizontal blanking Thevertical sync controls the field-by-field scan- the horizontal sync pulse occurs. The horizontal ning process performed by the electron beam and sync pulses occur oneachlineduringthehorizontal also maintains the horizontal oscillator in step. The blankingintervalandbeforethepictureinformation function of the vertical sync pulses is to indicate to ofthat lineis displayed. After the beamhas scanned the monitor when each field has ended so that the one line, the beamis blanked out in preparation for camera and monitor begin and end each field in

174

Page 174 - Black overlay HOCKING & HO C KIN G

direct relationship to each other. The vertical serra- titles for hearing impaired individuals. The cap- tion pulses help maintain proper horizontal fre- tions, provided in 1980 by several of the networks quencyduringthe vertical interval. Thefrequency of and PBS, appear on the screen as text when used the serration pulses is twice the horizontal fre- with user-purchased decoders. Other systems can quency.The horizontal sync pulses which conclude provide data such as weather, sports and news the vertical blanking interval also help to keep the reports. horizontal oscillator in step during retrace. Sync and drive pulses are the timing pulses In order to achieve interlaced scanning, each whichkeep one orseveral cameras instep witheach field contains a half line ofpicture information. The other andwith thevideotape recorderor monitor. In line precedingthevertical interval ofthe oddfield is a single camera system, sync can be obtained from one complete picture line. This line is the last line the internal sync generator built into the camera. scanned in the even field. The vertical interval, The video and sync information together are then occupying 21 H lines, then follows. The first picture sent to thedeckorthemonitor. Inamultiple camera line of the odd field which follows the vertical system, the internal sync generator in each of the interval is one full picture line. 241 .5 picture lines cameras cannot beused to send timing information follow. The 21 lines of the vertical interval and the to the rest ofthe system. All cameras must receive 241 .5 lines of the picture information total the the same sync signals from a common source at the 262.5linesneededfor onefield. The last 1/2 picture same time, from a sync generator external to all of line of the odd field then immediately precedes the the cameras. Video or picture signals from all the vertical intervalfor the even field. cameras are then mixed in the processing system The odd field, as noted, is preceded by one and combined with sync information. This single complete picture line, the last in the even field. The composite signal, containing both picture and sync vertical interval for the odd field begins with six information, is sent to the deck to be recorded. equalization pulses occupying 3H. Six serration A black and white sync generator usually sup- pulses follow, also occupying 3H lines. After the plies horizontal andverticaldrive pulses, composite next six equalization pulses, the horizontal sync blanking which includes both horizontal and verti- pulses occur. The first ofthese occupies 1/2 line. It cal blanking, and composite sync which also in- is here, in part that the off-set relationship occurs cludes horizontal and vertical components. The which provides for interlaced scanning. Eight to function of the sync pulses is to indicate to the twelvehorizontal sync pulses without picture infor- camera or monitor when one line, in the case of mation conclude the vertical interval. Following the horizontal sync, or one field in the case of vertical 21stlineoftheverticalintervalisthefirstpictureline sync, will end and the next begin. The blanking of the oddfield, a complete horizontal line. The odd pulses make sure that the retraces, both horizontal field scans 241 complete horizontal picture lines andvertical, arenotvisible. Drivepulses controlthe and ends with 1/2 picture lines. timing of the beam's scan. The vertical blanking period under broadcast The color sync generatorsupplies horizontal and conditions contains two additional signals which vertical drive, composite sync andcomposite blank- are used for reference and testing. The first, called ingand two additional signals variouslycalled burst the Vertical Interval Reference or V1R signal, is or burst flag and subcarrier or 3.58 MHz. Color added toline 19 ofboth fieldstomaintainthe quality signals must carry all color information, including of the color transmitted . Certain color receivers are the hue, brightness and saturation ofthe colors, by now made which use this signal to automatically theuse ofthreeprimarycolors: red, green, andblue. adjust hue or color and saturation. The second In addition, their structure must be such that they signal, called the Vertical Interval Test, or VIT are compatible with black and white systems. A signal, is used as a test signal to evaluate the color signal must play on a black and white televi- performance of equipment and appears on lines 17 sionwithno interference . Color signals mustthere- and 18. Other information can be coded into the fore contain both luminance and chrominance vertical blanking interval, including program sub- information . Luminance conveys the variations of

175

Page 175 - Black overlay EIGENWELT DER APPARATEWELT

light intensity and is the part of the signal used by vertical sync occur. Horizontal blanking and hori- the black and white monitor. Chrominance conveys zontal drive control the direction and speed ofeach variations ofhue, saturation, and brightness. of the beam's horizontal traces and retraces. Verti- The subcarrier signal, with a frequency of ap- cal blanking and vertical drive control the change proximately 3.58 MHz, carries information about from one field to the next. colorvalue. This frequency is produced by an oscil- The sync signalstellthe camera ormonitorwhen lator in the sync generator, and is modulated or the scan is to change. Horizontal sync controls the changed by the color information coming from the beginning and end of each horizontal line. Vertical colorcamerato the colorizerinthe imageprocessing sync controls the beginning and end ofeach field. It system.Theways inwhichthe subcarrieris changed assists in keepingthemonitorinstep orinsyncwith convey information about the color, its saturation the camera. In a multiple camera system it also and hue. For example, changes in the phase of the keeps all cameras synchronized with each other. chrominance signal indicate changes in hue. Some cameras use sync ratherthan drive signals to In order thatchanges inphase, forexample, and produce the deflection signals which control the the resulting changes in hue can be identified, a beam's scanning processes. reference signal is required. The burst signal sup- Line frequency, or 15,750 Hz, isproducedinthe plies 8 to 10 cycles of the 3.58 MHz subcarrier camera and monitor by crystals which oscillate or frequencywithout anycolorinformation .This serves vibrate naturallyat the speedof31,500 Hz. Thisrate as a reference pointto establish the phase relation- is then dividedin half electronically to produce the ship of the subcarrier signal before it is modulated required 15,750 Hz frequency . and starts to carry color information. The burst The field frequency of 60 Hz can be derived by signal is located on the back porch ofeach horizon- dividing the line frequency by the number of lines tal blanking pulse. Itis notpresent afterthe equali- (525). These pulses can thenbe sent to the horizon- zation or vertical pulses ofthevertical interval. The tal and vertical deflection circuits of camera and average voltage ofthe color burst signal is equal to monitor to insure proper scanning. the voltage of the sync signal. Burst then helps to Line and field time base stability refer to the synchronize color. precision at which the line and field frequencies The horizontal drive signal occurs at the rate of operate. Exact operation is essential to the opera- 15,750 Hz. Its durationis 1 /10th ofthetime it takes tion of the system as a whole and compatibility from the beginning of one horizontal line to the between output signals from different systems. The beginning ofthe next, or about 63.6 microseconds. stability of the time base found in small-format Verticaldrive occurs atthe rate of60Hz and lasts for recording canbecorrected through theuse ofatime about 666 microseconds. Both pulses are sent to base corrector . the cameras to control horizontal and vertical de- Ifthe signal sentfrom the camera to the monitor flection circuitry, that which dictates the scanning includes pictureinformation, horizontal sync, hori- processes. zontal blanking, vertical sync, and vertical blank- Horizontal andverticalblankingpulses arepulses ing, then the signal is called a composite black and which make invisible the retrace lines which occur white video signal. If the sync information is not as a line or field is ended and the beam returns to included in the signal sent from the camera, this begin the next trace. Vertical blanking lasts about signal is called a non-composite video signal. 1330 microseconds and horizontal blanking about In a single camera system, the sync generator 1 microseconds. Composite blankingwiththe addi- inside the camera may be used to generate the tionofthevideo signalis sentto the monitortoblank necessary sync information for recording and dis- outthe verticaland horizontalretraces . The camera play. Inthisinstance, the camera is usually referred usually is not supplied with vertical and horizontal to as being on internal sync, and the composite blanking because the horizontal and vertical drive video signal is sent to the recorder or monitor. A pulses can accomplish the same function. It is single camera system may also be used with a sync during the blanking intervals that horizontal and generator which is external to the camera. In this

176

Page 176 - Black overlay HOCKING & HO C KIN G

case, the external sync generatorgenerates the sync era and monitor occur at the same intervals with information which is then sent to the camera. The respect to video. The sync generator also provides camera in this case is on external sync. Some blanking signals,bothhorizontal andvertical, which cameras can operate on either internal or external are added to the video waveforms. If the sync sync. There is a switch on the camera for selecting generator also supplies timing signals to drive the the sync option. Many cameras only operate on deflection systems of camera and monitor which internal sync. These cameras cannot be used in then maintains the phase relationships between multiple camera systems. Whether internally or horizontal andvertical scanning, the 2:1 interlaced externally locked, signals are produced which drive scanning is achieved. If 2:1 interlaced scanning is the deflection systems ofthe camera andinsert the not present, the sync is termed "industrial" or waveform onto the video out signal which causes random. the monitor to be in sync with the camera. If the horizontal and vertical signals are not In a multiple camera system, such as an image locked together in phase but are derived independ- processing system, the sync information for all ently, then random interlace scanning results . In cameras must come from one common source. In a this type of scanning, the horizontal lines in each multiple camera system, each of the cameras is field are notin anyfixed position andare not evenly sending picture information which will eventually spaced. Occasionally because of this lack of even be combined and treated by a variety of image horizontalpositioning, horizontallines maybetraced processing devices. Techniques such as mixing, on top of each other. This results in the loss of switching, or keying can be employed. None of the picture information contained in the superimposed cameras in the system is generating its own sync lines and a degradation ofthe picture. This is called information . A common sync source is sending line pairing. identical syncinformation to each ofthe cameras in During the vertical sync period, it is necessary the system. The camera then sends back to the thathorizontal information be supplied or the hori- system a composite video signal containing both zontal oscillator in the monitor may drift. The picture and sync. If each of the cameras in the frequency corrections to that oscillator which are system were to generate its own sync, there would then needed following the vertical sync period may beno consistenttiming information throughout the cause flagging at the top of the image. Flagging system. It would then be impossible to achieve a appears as a bend toward the left orrightin the first stable image. In an image processing system the several lines of the image. sync generator which sends sync to all of the VIDEO SYNTHESIZERS: A video synthesizer or cameras is usually external to each of the other "image processor" is a general term referring to an components in the system. One common source assemblage of individual video signal sources and sends thesameinformation to each ofthe cameras. processors, all of which are integrated into a single The monitor or deck receives a composite video system. There are three general categories of de- signal from the system which includes: picture vices: 1) signal sources-devices which output a information, horizontal andvertical blanking, hori- signal used in the system to generate an image, a zontal and vertical drive, andhorizontalandvertical control signal or a sync signal. 2) processors- sync as well as the signals needed for color. The devices which perform some operation upon the blanking, drive and sync information are used to signals, such as gain or phase changes, and are control the deflection ofthe scanning beam, so that often used to mix inputs and put out combined or the image displayed is a stable and faithful repre- processed signals. 3) controllers-devices which sentation of the camera images. generate signals which are themselves inputs to The sync generatorthen serves as a master clock processing devices to controlanaspect ofthe image. which establishes the time frames for the signals These devices can be analog or digital in nature. which, when decoded, produce images. The sync A video source is any device which internally generator insures that the scan and retrace proc- generates a signal that can be displayed, and in- esses for both horizontal and vertical in both cam- cludes cameras, decks, character generator or os-

177

Page 177 - Black overlay EIGENWELT DER APPARATEWELT

cillator.Aprocessoris a devicewhicheither changes inputto the image processing system a video signal the parameter of the incoming signal (e.g. gain, from a pre-recorded videotape. Since the sync from polarity, waveshape) or combines two or more sig- the sourceis controlled from the point of origin (the nals and presents them to the output (e.g. mixing, VTR) itis necessary to "lock"the system sync to the switching,wiping). Video processors include keyers, sync from the source. A genlock is required for this VCAs, mixers, colorizers, sequencers, SEGs and operation. Genlock is also used for cameras which frame buffers. These terms are not absolute but are not externally syncable. This includes most have meaning relative to one another. A signal can consumer cameras. The output of the camera goes be routed through processors in a linear order. to the genlock input ofthe SEG and the system will Signals have direction: that is, they are origi- lock to the internal sync of that camera. A VTR nated, are passed through devices and eventually cannotbe used as a direct source once the genlock wind up at a device which transduces, or changes, is occupied by the camera. the signal into a form ofinformation that is directly Three devices are used to check the signal com- meaningful to our senses. In the case of video, the ing from the output amp: The waveform monitor, electrical signal is changed into information by the the vectorscope, andthecolor or black andwhite video monitor which our eyes understand as light monitors. These are also used to compare the and ultimately pictures. The video image itselfdoes signal at different points in the path. not travel through the machine, rather it is an Thewaveform monitor does no processing ofthe electronic signal which represents the image that video signal. It allows us to examine the quality of travels. This signal originates in a video camera or the video signal by giving us a graphic representa- some other type of signal generator such as an tion of thevoltage ofthe video signalwith respect to oscillator in an analog synthesizer. time. The waveform monitor is really a special Thethreemaintypesofimageprocessing signals purpose oscilloscope . Verticaldistanceonthewave- are: 1) video signals-thosewhich contain the com- form display represents voltage, while horizontal plete information necessary for a monitor to con- represents time. There is a choice of"strobe" times struct an image 2) sync signals those containing so that one field or line of the video signal can be structural, rather than picture, information, which observed. when combinedwithpicture information allowsitto Normally, the waveformmonitor is setto display be stable and rectangular . 3) controlsignals-those two horizontal periods, ortwo lines ofvideo.What is which contain information for the control of pro- seen is actually an overlay of many different lines. cesses. Within this, you can see the luminance and black An image processing system is then a collection level ofthe signal as wellas the stabilityofthe sync. of devices the structure ofwhichincludes: 1) sync, Other settings show the vertical scan period and an 2) routing, 3) an output amplifier, 4) a method of enlarged view of color burst. monitoring, 5) a method of control. A vectorscope shows the color portion of the Termination : A commonly used video connec- video signal. It uses the convention of a color wheel tion scheme is the looped-through input, some- to represent the signal. Chroma, or saturation, is times called a bridged input. This set-up facilitates indicated by how far the signal extends from the ease in formulating multiple connections while center. It shouldnotexceed the outer circumference maintaining the ability to "terminate" the video of the circle of noise which may appear in the signal.Termination isrequired at thefarthestinput. recorded signal a few generations later. The hues This is usually done by connecting a terminator to are marked at specific points initialed M (magenta), the remaining bridged connector. Sometimes a R (red), G (green), Y (yellow), B (blue), and C (cyan) . switch is provided on a monitor input for termina- The vectorscope has a phase adjustment which tion, labelled "75 ohms" in one position and "high" places burst at 180 degrees . At this setting when in the other. The 75 ohm positionis the terminating color bars are patched to its input, the signal's six position. points will correspond to the marks. Genlock: Sometimes it is desirable to take as an There are several types of color monitors. There

178

Page 178 - Black overlay HOCKING & HO C KIN G

is also abank offourblack andwhitemonitors, used ary ofthat shape. Into thathole you canthen insert as preview monitors. Their inputs are routed from the portions of a second image which spatially four separate points on the matrix and are used to correspond as ifthe two images were to besuperim- look at primary image sources, for example, cam- posed. The development of a keyer as a three-input eras. They can also be used to look at the output of device, withvoltage controllable parameters as well modules in various stages of a patch. They do not as its use in an image processing system necessi- indicate the signal coming from the output amp. tates a broader understanding ofthe functions ofa Everymodule thatyoupassacolorsignal through keyer. will change the phase of that signal. Since the There are three channels in the luminance, or output amp is always the final destination, you can black and white, keyer: Two main channels, A and readjust the phaseto compensatefor hue changes. B, and a clip input. Each of the main channels is a When mixing a live camera or color genlock tape VCAorvoltage controlledamplifier, whichsends the with the colorized image, consider the last module incoming signal to the same electronic switch. At in the system where the two signals will be com- anygivenmoment, this switchchooses either signal bined. Looking at the module's output through the A or B at its output. The rate of switching is fast, output amp, adjust the phase control so that the taking place several times within each horizontal "real"color is correct. Then adjust the hues of the lineofeachframe. The video signalthat is goinginto colorized signal with the controls on the colorizer. the clip input controls this switch. A clip level Some modules will generate noise at their out- determines a certain threshold point, and the clip puts when a color signal is routed through their input signal is compared to the threshold. It is the inputs. These include the clip inputs to keyers and voltage levels of the signals that are being com- the inputs to the frame buffer. The noise with the pared. When displayed on the raster, these voltage keyers may generate useful effects. Do not use a levels become the gray levels of the image. The colorsignal as input to the buffer.A colorkill is used comparison is being made at each point on each witha separateinput and output on thematrix. This horizontal line. When the voltage of the clip input strips the chroma and the original color signal still signal exceeds this threshold point, and the signal available from its point on the matrix. A monochro- is therefore brighter than a predetermined gray matic version to the signal is also available at the level, channel A is presented at the output of the output ofthe matrix for routing into the buffer and switch.Whenthevoltage ofthesignalfallsbelowthe key clips if desired. threshold, and is therefore darker than a certain A sequencer is a multi-input device which gray level, channel B is seen. Moving the clip level switches from one input to another automatically . control knob clockwise increases the threshold The number of inputs and rates of switching vary point. This allows more ofB to be seen than A. Thus with the device. channelsA and Bwill always be on opposite sides of Keying is a process of graphically combining the clip edge. A key reversal simply exchanges the video signals. It originally was developed in the positions ofchannelA and B relative to this thresh- television industryfor the purpose ofelectronically old point. imitating a filmic technique known as matting. In The conventional use ofa keyeras a matte device this context, the most conventionaluse has been to is a specific case in which one of the two signals take two camera images and juxtapose them in a going into the VCAs is also being used as an input way which creates the illusion ofa single, continu- to the clip channel. This technique is often referred ous three-dimensional space. Thus keyers areoften to as internal keying. Some keyers are hardwired referredtointermsofplacingone image "behind" an in a way which allows internal keying only. When a object in a second image or of "inserting" an image third signal, separate from either of the VCA input "Into" an area ofanother image. Using a keyer, you signals is patched to the clip input, this is called can create a shape in a first image, by defining the external keying. gray values that comprise that shape, and then Split-screens are a specific application ofexter- remove all portions ofthe image within the bound- nal keying. An externally-synced oscillator is used

179

Page 179 - Black overlay EIGENWELT DER APPARATEWELT

as the clip input signal to switch between the two 1974-75 main channels. Acontinuous changein the thresh- Workshops and performances based on image old point, or clip level, from low voltage to high processing were conducted atThe Kitchen, Authol- voltage, or vice versa, is often called a wipe. ogy Film Archives and the Contemporary Art Mu- A colorizer takes as its input a black and white seum in Montreal. NYSCA supported a series of video signal, then adds color in a fashion according travelling performances by Walter Wright on the to the type of colorizer. Usually a colorizer unit video synthesizer. Over ten organizations through- contains other video processing as well, such as out New York State and Canada took part. The negative video, keying, mixing. workshop program at the Center continued . NYSCA provided funding for the development of the Jones A chronology ofthe activities at the Experimental Colorizer, a four channel voltage controllable col- Television Center, Binghamton, New York, 1971- orizerwithgraylevel keyers.The oscillatorbankwas 1978,foundedbyRalphHockingandcurrently under completed and installed. The SAID (Spatial and the direction ofRalph &Sherry Miller Hocking. Intensity Digitizer) was developed by Dr. Don McArthur (April 1975) as an outgrowth of research 1972 on b/w time base correction. Work was begun by NYSCAfunding to the Center for construction of David Jones, Don McArthur and Walter Wright on Paik/Abe Video Synthesizer . One system was de- a project to explore computer-based imaging, and signed and built at the Center by Shuya Abe and the interface of a computerwith a video processing NamJunePaikforeventual placement at theTVLab system. Artists-in-ResidenceincludedNeilZusman at WNET-TV in 1972. This system was used while and Gary Hill. still at the Center to produce aportion ofPaik's "The Selling of New York,"included in the PBS series 1975-76 "Carousel," broadcast 1972 by WNET. A second The Residency Program included artists Nam systemwas builtfortheArtistinResidencyprogram June Paik, Phil Jones ofIthaca Video Projects, Ken at the Center and used in `72 by artists such as Marsh and Ken Jacobs. The NEA in 1975 provided Ernie Gehr, Hollis Frampton, Jackson MacLow and support for initial research into the computer-video Nick Ray, and also included in an exhibition at the processing project, which was expanded by Jones, Everson Museum. A raster scan manipulation de- McArthur, Wright, and Brewster to incorporate vice was also constructed, the principles of which parallel research efforts by Woody and Steina were defined by Paik's early TV experiments, such Vasulka, and Jeffrey Schier. The LSI-11 computer as Dancing Patterns. was chosen as the standard. Jones developed hard and soft-edged keyers and a sequential switcher, 1972-73 which along with the Jones Colorizer was incorpo- David Jones became technician at the Center. rated into the processing system. A commercially Artists participating in the Residency program in- available SEG was modified to incorporate these cluded Taka Iimura, Doris Chase, and Michael keyers . A 64-point push button switching matrix Butler. Workshops inimagingwere held regularly at was designed andbuilt. Webegan towriteamanual, the Center, and also at Global Village and at York developed initially to be used as a operator's guide University inToronto . Oscillators were designed for to 1/2" reel-to-reel equipment, portapaks, editing use as signal inputs to the synthesizer. Initial equipment, and the like. The concept was later researchinto theJones gray levelkeyer and produc- broadened to included step-by-step construction tion of a black and white keyer. Modification of an information onaPaik Raster Control Unit. By 1985, existing SEGfor directsync interfacewith the Paik/ the information was expanded to include systems Abe with provision for external wipe signal input. structure and theoryofelectronic signals and proc- essing techniques. These manuals have been dis- tributed to manyindividuals and organizations over the years. Cloud Music by David Watts, David Behrman, and Bob Diamondwas presented at ETC. 180

Page 180 - Black overlay HOCKING & HO C KIN G

1976-77 1978 TO PRESENT Artists such as Barbara Buckner, Aldo Tam- Due to the historical nature of the exhibition, this bellini, Nam June Paik, and the ADA continued to listingonly covers the early years. However, since participate in the Residency Program. The com- 1978, the Center has continued their active pro- puterprojectcontinued.Theexhibitionseries, Video gramming based upon their missionasdescribed. by Videomakers, was begun, introducing to this R&D Program Concepts: region video works by Beryl Korot and Barbara 1. Modification ofexisting equipment: to expand its Buckner. The computer was installedas part ofthe capabilities in order to bring out all possible system and made available to artists; software controls to the artist. research began. For the secondyear, we conducted 2. Design and construction of image-processing a series ofworkshops in school districts throughout equipment: to expand the Center's system; to theregion, in collaboration with Binghamton'smajor make equipment and/or information available arts center, the Robertson Center. to individual artists. 3. Development of print information and educa- 1977-78 tional strategies to teach artists and others the NYSCAfunding helpedsupportthedevelopment principles of image processing; to encourage by Jones and Richard Brewster ofthe Analog Con- artists to approach video as a directly mediated trolBoxallowingtheproduction ofelectronic sounds art practice; to encourage artists to use tools and also signalswhich controlledparameters ofthe themselves in art-making ; to encourage artists video signal. The computer project proceeded, as- to build or purchase equipment for their per- sisted by Paul Davis, then director of the student sonal studios. computer lab at the School for Advanced Technol- 4. Design considerations : Flexibility; low cost; ease ogy at SUNY-Binghamton. Artists-in-Residence in- ofuse; greatest number ofpossibilities forimage cluded Shalom Gorewitz, Sara Hornbacher, Hank and sound generation, manipulation and con- C. Linhart, and Hank Rudolph. We conduct work- trol. shops for the City of Binghamton, Headstart, Tri Cities, 4H Program, and Center for Media Studies .

Page 181 - Black overlay