Of Covered Windows

A Complete and

Efficient Implementation of Covered Windows

Brad A. Myers University of Toronto

Sapphire, the Screen Allocation use of the screen, and (3) the metaphor is The Sapphire window Package Providing Helpful Icons more familiar to users. The covered win- manager supports and Rectangular Environments, is dow paradigm is used by a large number of a window managerl for the Accent existing window managers. 1,31 many important operating system2 running on the PERQ The disadvantages are that it is more features, including personal workstation. It has been used at complex and expensive for the software, it Carnegie-Mellon University and byPERQ is more difficult to provide programs that flexible window Systems Corp. and its customers. Like the automatically manage windows, and users window managers for many other person- may waste time rearranging windows. 12 refreshing, full- al workstations and intelligent terminals, Window managers that do not support functionality Sapphire supports the covered window covered windows only allow the windows paradigm, in which windows are rectangu- to be side by side, often stacked in columns subwindows, and lar and can overlap arbitarily like pieces of (these are called tiled window managers), paper on a desk (sometimes called the and are growing in popularity. 13tl4 optimized raster-op, desktop metaphor). One ofthe major goals of Sapphire is to that are not In this paradigm, a window may be on provide to users and application programs top of another window, just as one piece a wide range of functions so it would not supported in most ofpaper may be on top ofanother piece of unduly limit how application programs comparable systems. paper. The window that is behind is cov- could use the screen or interact with users. ered by the window on top and the parts The window size can be changed easily, under the top window do not show through and both characters and full bitmap (see Figure 1). graphic operations are supported. Sap- There is a total ordering imposed on all phire supports full covered windows, and, the windows where windows higher in the unlike some older window managers, lets order cover windows that are lower. This windows be updated while they are cov- is often called 2½/2 dimensions, since the ered. Windows in Sapphire can also be rectangles are thought to be layered in partially or fully off the screen in any another dimension (z) pointing out of the direction. screen. Windows that do not interact are It can also divide a window into subwin- still ordered. The top window is not dows. An application might provide dif- covered by any windows. The window that ferent areas for various types of displays is most covered is said to be on the bottom. while keeping them together in the same The advantages of the covered window parent window so they can be manipulated paradigm are that (1) there can be several as a unit. Subwindows also help the system large windows that would not all fit on the and user organize the screen using screen if they were required to be side by context. 15 Subwindows can overlap and side, (2) the user can make more efficient operate with full functionality. In fact,

September 1986 S t b 9-9162/86/0900-0057$901.00 © 1986 IEEE 57 overlap arbitrarily or may even coincide. Since the screen bitmap is stored in main memory on the PERQ, the hardware raster-op can be used with both the screen memory and off-screen memory, and can perform transfers between the two. The hardware raster-op on the PERQ operates at full memory speed no matter what the bit orientations of the source and destination rectangles are, so no special con- siderations need be made for the memory alignment used to hold bitmaps (as is done in Blit5). Window raster-op can be used to implement many other graphics operations. For example, to display text, a window raster- op is performed for each letter to move it from a font table (containing a picture for each character) to the appropriate place on the screen. In addition, window raster-op is used to implement changing a window's size, position, and covering. Handling window refresh When a window is partially covered and then becomes uncovered, the parts that Figure 1. A portion of an actual Sapphire screen. Window W3 is covering window W2, were previously hidden must be displayed. and both windows W3 and W2 are covering W 1. W 1 is on the bottom and W3 is on the This can be the responsibility of the win- top. W2 has a gray border to show that it is accepting user input (typing). The window dow manager (and thus hide the need to in the lower part of the screen contains a number of icons showing process and win- refresh from the application) or it can be dow state information. 1 the responsibility of the application (and thus free the window manager from having to remember the picture). The method for handling refresh is the main distin- since top-level windows are actually sub- pixel can be either white or black). For ex- guishing difference among the implemen- windows of the full screen window, all ample, to set a spot on the screen to black, tations of covered window managers. If it windows are subwindows. Other systems the corresponding bit in memory is set is the responsibility of the window man- such as SunWindows71imit the func- to 1. ager, the window manager can save either tionality of subwindows. The user inter- The primary method for doing graphics the picture contained in the windows or face to Sapphire, which includes a novel on the PERQ's bitmap screen is the raster- the picture that the window covers (that use of icons, windows with title lines, pro- op (sometimes called bitblt). 17 To avoid are under the window). Interlisp-D1I gress indicators, 16 and a simple but confusion this article uses "hardware saves the picture under every window, and powerful user interface using the puck and raster-op" for the low-level function that Smalltalk8 and others save the picture keyboard, is described elsewhere. 1 actually moves bits, since it is implemented underneath for pop-up menus, but vir- Like most other window managers, with special hardware on the PERQ. (The tually all other systems-including Sap- Sapphire is implemented entirely in soft- functionality of the hardware raster-op phire-save the contents of windows in- ware. A hardware implementation of the may actually be implemented by some stead. Saving contents and areas under- covered window paradigm exists,4 but is combination of hardware, microcode, neath in the same window manager is very probably too expensive to be practical in and software on other computers.) "Win- difficult. No matter how the window man- the near future. dow raster-op" refers to the high-level ager implements the saving internally, the function that works with covered win- procedure is hidden from programs using dows. the windows. Background Hardware raster-op moves an arbitrary Ifthe window manager saves the picture bit rectangle from one part of memory to under a window, the full bitmap must be Sapphire runs on a raster (also called another. It also combines the source pic- saved. However, if the contents of a win- bitmap) display, which means that an area ture with the destination picture using a dow are saved, there are several implemen- of memory holds the picture shown on the number of different Boolean functions, tation possibilities, including saving screen. Each point on the screen (pixel) such as AND, OR, XOR, and NOT. The * the entire picture for each window in corresponds to one bit in memory (so each source and destination rectangles may a separate off-screen buffer,

58 COMPUTER * only the covered portions offscreen, Wi. * a general-purpose display list describ- ing what was displayed, and W V * combinations, such as bitmaps for pictures and characters for text. W3__ The first approach keeps a full shadow ____---- bitmap for every window in a separate off-screen buffer. Graphics operations are done to this buffer, and any visible portions are copied to the screen. This approach is fairly simple to implement and very popular. It is used by SunWindows.7 The main problem is that it takes a lot of extra memory to hold all the bitmaps, some of which is redundant, and it takes W2 extra time for displaying because graphics in visible parts must be drawn twice (once to shadow memory and once to the Figure 2. The rectangles that would be produced for window Wl in Figure 1 based on screen). the windows W2 and W3. The rectangles marked V are visible, and those marked C The second approach tries to alleviate are covered. these problems by saving only in off- screen buffers the covered portions. The width). Also, this might be of limited ap- example, to hold a series of pictures being graphics operators must be changed so a plication since many text-only applica- transferred to the screen one by one for an single operation will work partially on the tions, such as sophisticated text editors, animation). The program simply creates a screen and partially in an off-screen buf- use graphics as part oftheir user interfaces window that has automatic refresh (so it fer. This clearly makes the graphics opera- (such as MacWrite for the Macintosh). 10 has backup memory), positions the win- tions more complex. There may be some If an application handles window re- dow so it is entirely off screen, and then operations provided by the hardware, fresh, it must remember the contents ofits stores the picture in the window. Since the such as filled polygons or circles, that can- window and regenerate the picture on de- window is off screen, it will not be visible not work separately on different parts of mand. This is the approach in the Apple to users-but all the graphics operations the picture. Saving the covered portions Macintosh1I and IRIS Mex6 window will work normally, and the application would be inappropriate in this case. Blit managers. Many programs, such as text can copy the contents ofthe window to the and Sapphire can use this approach since and graphics editors, must save the con- screen whenever desired. The window their graphics are limited mostly to raster- tents of the window anyway, so it is easy manager will also allocate and manage the ops. for them to regenerate the screen picture temporary buffers automatically. The third approach calls for a general when necessary. On the other hand, some display list mechanism, such as that used applications may find it difficult to on calligraphic (vector) screens, but this is manage window refresh. The application Raster-op for covered inappropriate for a bitmap screen because program must be prepared to handle asyn- windows saving a description of the picture may chronous refresh requests that may occur easily take more memory than the picture at any time, including while the program is Most computers provide little hardware itself, especially for complex images. modifying the picture. This may adversely support for the covered window para- Refreshing from a display list also takes affect the program's structure and the ease digm. Therefore, to display only the por- more time than refreshing from a stored of porting programs written for nonwin- tions of a window visible on the screen picture, and it may be difficult to handle dow systems. (and hide covered parts), window manag- raster-selective erasures with display lists. To provide maximum flexibility-and ers implementing the covered window par- The fourth approach calls for a mixed still conserve memory whenever possi- adigm in software must calculate which style such as a limited form of display list ble-Sapphire implements two tnethods, portions of windows are covered. This is when appropriate. Because many windows allowing the programmer the choice of especially true if the window manager contain only text, an obvious optimization automatic refresh by saving only covered allows graphics to appear in a window is for those windows to save only the portions off screen or of application- while the window is covered. (The Inter- characters displayed in them rather than handled refresh. Providing application- lisp-D II and Smalltalk 8 window manag- the bitmaps for the characters. This will handled refresh was partially based on the ers only allow graphics to be output to un- typically save more than an order of observation that although Blit uses the covered windows. Interlisp-D automati- magnitude of storage even on a one-bit- memory-saving techniques ofthe covered- cally brings a window to the top before do- deep screen (nine by 13 characters take 117 portions approach, some of its pro- ing output graphics, which causes a great bits versus seven bits for the ASCII code). grams-such as the editor and debug- deal of flashing when multiple windows In systems where this has been imple- ger-use a different technique for covered are being used. Most modern window mented, however, such as the ICL PNX subwindows to conserve memory. managers support updates to any window window manager,4 many restrictions Providing automatic refresh lets normal at any time.) typically apply (for example, only one font windows be used as the temporary buffers The window intersection information is per window, which must have a fixed often needed in interactive graphics (for usually precalculated and stored as a list of

September 1986 59 rectangles for each window, marked as visible (which means they appear on the screen) or covered (which means they are covered by other windows and not visible on the screen). Figure 2 shows the set of rectangles that would be generated for window WI in Figure 1. When using window raster-op to transfer a picture to a new position that over- Figure 3. When scrolling the left window up one line to get the configuration on the laps the old one, the order in which the bits right, the first line must be moved first, overwriting the second line. If the third line is are moved is important. It is important to moved before in will the second, then the information the second be lost. note that the ordering pitmblems discussed in this section are less relevant for window managers that use full window shadow bitmaps. For example, when scrolling up a window, the top must be moved before the bottom or else the bottom part will cover k portions not yet transferred (see Figure 3). 1 2 k Window raster-ops of this form are used when scrolling text up or down in an editor or when panning a picture around in a k+1 2k graphics program...... Figure 4. Typical bitmap This problem is analogous to shifting the elements in a conventional array, n . i i+1 i+2 . organization. The first pixel is in the upper left and there are k where the shift must be done from the cor- __ pixels across and n down. The rect end to avoid losing information. The . i+k i+k+1 i+k+2 dotted rectangle in the center hardware raster-op, like the conventional ...... isto...... be shifted either starting array shift, needs only two directions to with pixel i or pixel i+ k+ 2. work correctly: top to bottom and bottom When moving the rectangle to top (see Figure 4). When moving a rect- nk for ex- towards the upper left, angle up or to the left, the hardware raster- ample, the pixel order will be op will move i, i+ 1,i+2,i+k,i+k+1, first the upper-leftmost bit, i+ k+ 2. then the bit to its right, and so on as shown in Figure 5. When moving down or to the right, raster-op must start with the last bit. The covered-window raster-op applies ULP the hardware raster-op to each rectangle I I (as in Figure 6) as a unit. It therefore must ULP ULP ::)CD QC) deal with the processing order of the rectangles. In this case, four different orders ULP~~~~sBRP I Hi1 1+2 are needed. Imagine a picture covering the three rectangles in Figure 6. When moving the BRP BRP ...... : picture to the upper right, the order for the B P rectangles must be B, C, A-or else some necessary portion will be overwritten. (a) (b) When going to the lower left, the order is Figure 5. (a) To move the rectangle in Figure 4 in any direction requires only two A, C, B, which is the reverse of the order orders for the pixels, starting from the upper left pixel (ULP) or from the bottom when going to the upper right. When go- right pixel (BRP). (b) Starting from the upper left pixel (I) is correct when shifting a ing to the upper left, however, the order is rectangle to the upper right, while starting from the bottom right pixel (i+ k+ 2) A, B, C-which does not have any natural would be incorrect since pixels i+ 1 and i+ 2 would be overwritten before they are moved.

ABC or BCA . up-left up-right Figure 6. (a) If the rectangles need to be moved to a ABC BCA new position overlapping the old position, the moving order is important. (b) The correct order for the eight ...... AC A \CBCA. or directions. The orders for up, down, left, and right can ACB / CBA be either of the orders on the neighboring diagonals. \\ Rectangle B has no other rectangles in its upper right ACB CBA quadrant, so it can be moved to the upper right (a)D down-left down-right without interfering with other rectangles. (a) _ (b) ACB or CBA

60 COMPUTER correspondence to the upper rightAower left order. The lower right order is the W...... E Figure 7. Even if all the rectangles are squares of the same size, four orders are reverse of the upper left order: C, B, A. needed if the new positions can overlap The order is always important, no matter the old positions. B must be moved whether the rectangles are stored in con- before A can move into the dotted posi- tiguous memory (all on the screen) or if S-E~~~~~~~~~ tion, and A must be moved before B can they are each stored in separate buffers. move there. The reason that two orders do not suf- fice for covered windows is that the ob- jects to be moved (rectangles in this case) are larger than the distance they can be Procedure WindowRasterOp(RasterFunction, SourceCoords, Destination- moved, so the new positions partially Coords) overlap parts of the old positions. Simply Calculate RasterOp direction from source and destination coordinates. making all the rectangles be squares ofthe FOR dr: = EACH destination rectangle is in order DO same size would not solve the problem, as IF dr is not covered OR Figure 7 shows. IF (dr is covered AND has backup memory) THEN In a more formal analysis, when going dtmp: = Clip destination coords to dr to the upper right, the rectangle whose up- IF anything left inside dtmp THEN per right quadrant does not overlap with FOR sr: = EACH source rectangle in order DO any other rectangles must first be found stmpl: = compute corresponding place in source for dtmp (see Figure 6). This rectangle can bemoved stmp2: = clip stmpl to bounds of sr in any direction in the quadrant without IF anything is left inside stmp2 THEN affecting any other rectangles. Since there IF sr is covered THEN {no picture in sourcefor this area} are a finite number of nonoverlapping save stmp2 on a list for update by the application rectangles, there will always be such a rect- ELSE angle. After this first rectangle is moved, it do hardware RasterOp on stmp2 is eliminated from consideration and the get next source rectangle in the current order. next maximal rectangle is found. A similar get next destination rectangle in the current order. operation is performed to move in the END Procedure WindowRasterOp other four directions. Some window managers (such as Blit) sort the rectangles when the window Figure 8. Outline of the code for covered window raster-op in pseudo-Pascal. raster-op is performed. To avoid this overhead, Sapphire uses a technique that generates the rectangles in the correct order. Because graphic operations are done much more frequently than recon- figuring the rectangles (which is done only IW ...... I when windows are created, deleted, or modified), it is more efficient to generate ____ ~~~~w4_ _ _...._ the rectangles in sorted order, as Sapphire does. The rectangles in Sapphire are stored in a quadruply linked list, one thread the rectangles for each ofthe four B---- 1--- through - _------_-- - - - orders. Thewindow raster-op then follows the correct thread based on the direction of the window raster-op transfer. Each rectangle of the source is compared with Figure 9. The dashed lines show the rectangles created for window W when covered each rectangle of the destination in the by windows W2, W3, and W4. When transferring area A to area B, there are O(n 2) rect- where n is the number of Area C is covered rect- correct order, and any overlapping parts angles to move, rectangles. by O(n) angles, one for each window. are transferred. It does not matter whether the source and destination windows are the same or different. Figure 8 gives an the number of rectangles in any one source rectangles are all horizontal and the outline of the code for covered window window. destination's are all vertical, as shown by raster-op. The 0(n2) complexity for window areas A and B in Figure 9. It is therefore The window raster-op clearly takes raster-op cannot be improved in the worst not possible to write an algorithm that has 0(n2) time, where n is the number ofrect- case, since the number of overlapping rec- less than quadratic complexity in the worst angles. The maximum number of rectan- tangles can be 0(n2) because every rec- case. gles in a window is a constant multiple of tangle in the source area may intersect with The general case can be made faster, the number of windows, so n can be con- every rectangle in the destination area. however, by trying to limit the number of sidered to be the number of windows or This will happen, for example, when the rectangles compared on average. For ex-

September 1986 61 ample, the entire destination and source When performing a covered window areas for the window raster-op can be raster-op, there are two cases where there compared to their respective rectangle lists may not be a picture available in the to see which rectangles could be affected source. First, the source of the window by the particular window raster-op (this is raster-op might be a window that is akin to the bounding box test often used to covered and does not have backup mem- optimize other graphics operations). The ory (and so uses application-handled comparison takes 0 (n) time and general- refresh). An example of this is scrolling a Figure 10. When window Wi is scrolled ly will probably save a lot of time, since text window where some text comes out up, the picture appearing from behind many window raster-ops affect only one from behind another window (see Figure window W2 needs to be displayed. If source and one destination rectangle. 10). The other case occurs when the there is no backup memory for Wl, the Many other optimizations are possible. specified source is outside a window. application must regenerate the picture For example, window raster-ops in un- Some systems may flag this latter window for the dotted rectangle. covered windows (which have only one raster-op as an error, but Sapphire allows rectangle) can be performed directly with it for consistency because the destination ext. top only a clip to the windows' borders. for graphics can extend outside the win- ...... The general hardware raster-op is often dow. Sapphire surrounds every window ext. left window ext. right used to erase or invert a single rectangle. In with four special rectangles (see Figure 11)

...... this case, the source and destination rect- for the exterior ofthe window that extends ext. bottom angles are identical. For example, a rect- to ± co (implemented as ± max-integer) angle can be set to white (0) by XORing it away from the window. This makes it pos- with itself. When the rectangles are the sible to avoid a special case test and extra Figure 11. Special exterior rectangles same, a much more efficient algorithm is boundary clipping in the window raster- surrounding a window. They extend to used in Sapphire that goes through the list op code. ± Xo (± max-integer). of rectangles only once and therefore has Whenever parts of the source are not 0(n) complexity. available, the rectangles in the destination

X covering left middle right all

b a c b a b a a top 3 4 3 2 c d c b

a ~~~~~aa a 2 3 2:3:4 2:3: 2___

...... ~~~~~~~~~~~ -...... ~~~~~~~~~~~ ...... ::

m.e . . . ..oto..not C:ll dc11 b 1 bb b1

allib a cba b a a

Figure 12. The 17 two rectangles can interact. The numbers show the up left ordering, and the letters show the up right ordering. The down right ordering is the reverse of the up left, and the down left is the reverse of the up right. The solid rectangle is to be divided based on the dotted window. If the dotted window covers the solid one, the rectangle inside both the dotted and solid rectangles is covered. If the dotted window is the parent of the solid one, the area of the solid rectangle outside of the dotted area is covered.

62 COMPUTER for those parts are saved on a list and later enclosed in the area ofB, they all still must The procedure that implements the sub- passed to the appropriate application pro- follow A and precede Cwhen going to the dividing algorithm traverses each list from gram for regeneration using an operating upper left. Therefore, ifthe new rectangles back to front and adds each new rectangle system exception mechanism. The appli- that replace B are in the correct order with after the current rectangle, so newly added cation is told exactly what portions need to respect to each other, they will be in the rectangles will not be investigated during be regenerated so it does not waste time correct place with respect to A and C, and the pass of the algorithm that creates drawing intact parts (although it can if it therefore with respect to the entire list. them. Figure 13 gives an outline ofthe pro- wants to). Procedure WindowIntersect(W,RL, b: (wantOutside, wantInside)) Rectangle intersection {intersect window W with the current rectangle list RL. b is wantOutside when Wcovers RL, b is wantInside when W isparent ofRL} algorithm {Rectangles in RL are screen coordinates} {RL is changed to have the new rectangle list} The coordinate system for each window convert W's coordinates to screen coordinates {so it can be compared to in at upper cor- Sapphire has 0,0 the left RL} ner; x grows to the right and y grows FOR current = EACH rectangle in RL starting with firstDownRight DO down. The bounds of the window are IF current is covered THEN {ignore since don't need to subdivide therefore given by the lower right corner covered rectangles} of the window and are inclusive. For ELSE graphics, however, coordinates can be Calculate intersection between current and W in X and Y directions used that are of the window outside (as in Figure 12) (negative to the upper left or greater than IF the bounds for the lower right) and the either X OR Y not intersected THEN Current picture is simply clipped to the inside ofthe Change rectangle coveredness to be NOT wantOutside window. {doesn't intersect} Sapphire's rectangle intersection algo- ELSE CASE x interaction OF rithm is fairly simple. It is based on the straightforward enumeration of all the xleftCov: CASE y interaction OF ytopCov: {first create rectangles and add to the upLeft, ways two rectangles can intersect. In each downRight lists} x and y direction, the rectangles can be t2 = AddUpLeft(current, 2a, b = wantInside) covered in one of five ways. For x, they {t2 becomes upLeft ofcurrent. Last argument to are left covered, middle covered, right covered, all covered, and not covered. The AddUpLeft is coveredness ofthe new rectangle} t3 = = y values are similar. It turns out that if AddUpLeft(t2, 3c, b wantInside) the rectangles do not intersect in either {next, change size andposition ofti to be Ib; last is the x or the y direction, the rectangles do arg coveredness} tl = = not overlap. There are thus 17 different Change(current, lb, b wantOutside) possibilities (4 x 4 + 1). These are shown in {now add new rectangles to the upRight, Figure 12. downLeft lists} The rectangles are optimized for max- AddDownLeft(tl, t2) {t2 becomes downLeft oftl} imal horizontal extent. This direction was AddUpRight(tl, t3) {t3 becomes upRight oftl} chosen to minimize the number of rectan- ymiddleCov: gles crossed in typical text operations t2 = AddUpLeft(current, 2c, b = wantOutside) = (which are usually horizontal). The only t3 = AddUpLeft(t2, 3b, b wantlnside) difficulty in implementing the subdivision t4 = AddUpLeft(t3, 4d, b = wantlnside) is to avoid fence-post errors. (Fence-post tl = Change(current, la, b - wantlnside) {t3 } errors are ± 1 errors. The name comes AddDownLeft(tl, t3) becomes downLeft ofti from the problem of determining how AddDownLeft(t3, t2) many posts are needed to fence a yard that AddDownLeft(t2,t4) is 10 feet long if one is placed every foot.) ybottomCov:... Each case splits the rectangle into one to yallCov:... five new rectangles, some covered and END CASE on y some visible. These rectangles are added to xrightCov: CASE y interaction OF the four different rectangle lists in the correct order. END CASE on x = To demonstrate that this preserves the current next downRight from current ordering, it must be proved that dividing END Procedure WindowIntersect one rectangle into a set of rectangles that Figure 13. Outline of the code to do rectangle subdivision. There are four cases for x replace the old rectangle in the list pre- in the outer case statement. Each branch of the case has four cases for y, making 16 serves the overall order. Imagine that rect- cases. The 17th case is the special test for not intersecting performed before the angle B in Figure 6 is to be divided. Since cases. In each branch, the numbers (like 2a, 3c) correspond to the rectangle label in all the new rectangles will be entirely Figure 11. All 16 case branches are similar to the ones shown.

September 1986 63 This is also inherent in the problem. Since each covered portion of the screen will be represented by a rectangle for each Figure 14. W2 is a subwindow of Wi, so it is clipped to the boundary of Wi. The window on it (the area marked Cin Figure part of W2 that is outside of W1 is marked 8 is represented by n rectangles, one for covered (and are not visible). The part of each window covering that area), the total Wl that is underneath W2 is covered. A number of rectangles for all windows is 0(n3). There are some heuristics available, however, to improve the general case. These include trying to limit the Procedure OuterLooplntersectWindows(W) number of windows processed (for exam- [Calculates the rectangle listfor window W. ple, by ignoring windows that are totally The list is stored in the local variable RL.3 off screen) and trying to run the algorithm = to screen RL: rectangle for W converted coordinates [start rectangle list as few times as possible. with one rectanglefor the entire inside of W] Sapphire implements subwindows. Sub- IF W is offscreen THEN windows can overlap within their parent Set RL to be covered [optimization] and may extend outside the parent win- ELSE dow, but any parts outside are clipped and [* first, clip to the inside ofparent and itsparent, etc. *3 not visible, as Figure 14 shows. Because t: = W's parent window the screen is represented as the parent ofall WHILE t < > NIL DO other windows, it is trivial to let windows Call WindowIntersect(t, RL, wantInside) [get the inside of t] extend partially or totally off screen. This t: = t's parent window [go up the window hierarchy] also makes it easy to change the screen size [* next, remove areas covered by my immediate children *3 (the PERQ can be configured with various FOR t: = EACH immediate child window of W DO screen sizes), and some of the memory Call WindowIntersect(t, RL, wantOutside) [get the outside of t) normally used for the screen can be t: = next child of W allocated to other applications by reducing [* finally, check all other windows that might cover W *3 the screen window size. The extension to t:= W the subdivision algorithm to handle sub- WHILE t's parent < > NIL DO windows is very simple (see Figure 13). FOR t2: = EACH sibling window of t that is higher priority (more When clipping a subwindow to its parent, towards the top) than t DO the identical algorithm is used-except Call WindowIntersect(t2, RL, wantOutside) [outside] areas inside the parent are visible and areas t2 : = next sibling outside are covered, which is the reverse of t: = t's parent window [go up the window hierarchy] the case for overlapping windows pre- clean up I sented above. Subwindows can recursively convert RL to be in W's coordinate system contain their own subwindows to an ar- add the exterior rectangles bitrary depth. set list to RL W's rectangle be Because subwindows cover the parent END Procedure Outer LooplntersectWindows window, the immediate subwindows of a window affect that window's rectangles (see Figure 14). Subwindows of sibling Figure 15. Outline of the code for the main loop for calculating window's covered- windows do not have to be investigated, ness. The procedure Windowintersect is outlined in Figure 13. however, because they are entirely enclosed within the sibling window. An outline for the outer loop for window sub- Ws new division is shown in Figure 15. All the place between calculations are done in screen coordinates these windows to make it easier to compare different windows. As the final step of the algorithm, the Aft cled rectangles are converted into the window's rl L - ~~windcced Figure 16. Bringing window W more for- I i coordinate system. As part ofthis step, the ward (towards the top) can only affect Win is old plac rectangles are associated with the memory that will hold their picture: some rectangles correspond to windows on the screen and others correspond to off-screen memory holding covered portions of the cedure. The algorithm has quadratic com- dow intersecting with n other windows is picture. If the window does not have off- plexity, but this is inherent in the problem n 2, as shown in Figure 9. Since this screen memory (so the application must since, in the worst case, the number of algorithm will be run on all n windows, the handle refresh), the rectangles for covered rectangles that can be created in one win- total complexity is 0(n3). parts are marked as having no memory to

64 COMPUTER hold the associated picture. Output to case, the windows more covered than W's these portions will simply disappear (as old position and less covered than W's new wi shown in Figure 8). Also in this step, the position are checked to see if any portions W4 four special rectangles for the exterior of must be regenerated. The actual regenera- the window (Figure 11) are added to the tion is done in the same manner as for top, Wo.. I,W31 four rectangle lists in the correct order. except that W itself is handled before the other affected windows, since any por- ...... tions that become covered in W must be

memory screen ...... Manipulation saved to backup before the :w now: operations is overwritten by the newly uncovered ...... windows. Deleting a window works the same as Top, bottom, create, and delete. Mak- Figure 17. Modifying a window is a three it to bottom. ing a window less covered (towards the sending the step process. First, windows affected by top) requires two steps. First, any portions W's new place (Wl, W3, and W4) have of the other windows that become covered Move and grow. Move and grow are their newly covered portions stored to must be stored in their backup memory. much more complicated than top and bot- backup memory. Second, W's picture is Then portions of the window brought for- tom since as much of the original picture moved to the new place. Third, windows ward that become uncovered must be as possible should be retained without that need to be refreshed because por- W regenerated. Most systems only allow win- using extra temnporary buffers, and sub- tions are no longer covered by (Wl, W2, and W3) are regenerated. Note that dows to be to the top (so they are windows shouild move with the parent brought Wl and W3 are affected twice. not covered by any windows) or sent to the window (so the subwindows remain in the bottom. With window raster-op, Sapphire same relative places within the parent win- lets windows be moved to any position in dow). Another requirement is that win- out to be surprisingly difficult. W may be the covering order. In the following dows be movable even while being covered covered in different ways at its source and description, the window being changed is by other windows in the source or destina- destination, and its new position may called W. tion. Sapphire therefore transfers as much overlap its own old position (see Figure First, Sapphire checks those windows of the old picture as possible to the new 17). A further problem is that the screen closer to the front Qess covered) than W's location and only requires the application picture for the entire window must be old place and less covered than W's new to refresh what is absolutely necessary. moved as a unit-including the pictures in place (see Figure 16). Only these windows Because the screen contains the only copy any subwindows. Neither the window nor can be affected by the change. For each of of the picture for the visible parts of the its subwindows can be moved indepen- these windows, Sapphire checks to see if window, no screen picture should be dently since they may overlap at the source its entire area intersects with the entire destroyed before it is moved. and destination (see Figure 18). Therefore, area for W. If so, all its subwindows are Having subwindows also complicates to move the window itself, the following checked, since some may also be affected. the modify operation. Clearly, if the win- three steps are performed: For each affected window, the old rectan- dow is made bigger, the application will First, any portions of the window and gle list is saved and the new list is generated need to adjust the picture (and possibly the all of its subwindows that will be covered for the window with the new covering. subwindows) to fill in the new areas, so it in the destination position are saved into To do the actual update, Sapphire sim- must notified. In other cases, however, the backup memory with a special version of ply calls window raster-op to transfer from application can remain uninvolved. Move the covered window raster-op that only the old rectangle list to the new one. The and grow are implemented by one pro- transfers the covered parts of the destina- standard window raster-op call (which cedure, called modify. tion and ignores the uncovered parts. takes two windows as parameters) is used Again we call the window being Second, the screen picture from the old by having a dummy window to which the modified "W." The modify procedure is a location is transferred to the new location. old rectangle list is assigned. Ifthe affected three-step process (see Figure 17). First, all To do this, Sapphire must calculate the window had backup memory, the window areas covered by Ws new position must be covered rectangle sets for the old and new raster-op will automatically move any copied into their backup memory (if any). locations, ignoring all of W's subwin- newly covered portions to the backup The algorithm for implementing this is dows. The new rectangle lists are needed memory. similar to that used when Wis brought to because W may be covered by other win- After each affected window is updated, the top. Second, the picture for Wmust be dows (not subwindows of W) at both the a similar operation is performed for W moved to its new position. Third, any win- source and destination, as shown in Figure itself. Here, however, the window raster- dows that become exposed must be regen- 18. Any backup memory that W might op will copy from backup memory any erated. The algorithm for the last step is have is ignored here. portions of W that become uncovered. If similar to that used to send Wto the bot- Third, any parts of the picture for Wor Wdid not have backup memory, the win- tom. The calculations for the first step its subwindows that become exposed are dow raster-op will inform the appropriate must take into account W's old position as transferred to the screen from backup application program to regenerate the cor- well as its new position, so portions of W's memory or redrawn by the application rect portions of W. Creating a new win- screen picture are not overwritten. There- program (if there is no backup memory). dow is similar to moving it from the bot- fore, some windows may be updated twice When changing a window's size, it is tom (the old rectangle list is empty). if they are affected by both W's new and often inappropriate merely to adjust the Making a window more covered uses old positions. subwindows' size proportionally to the the same technique-but in reverse. In this Step 2, where W itselfis modified, turns parent's, since some subwindows may

September 1986 65 substantial. This is partially due to the large number ofrectangles for typical windows. The average number of rectangles for covered windows has been measured as WI moving eight (which includes the four exterior this way rectangles) while some windows, such as slghtly the full-screen window, often have more than 60 rectangles. Figure 18. When Wi is moved, its subwindow W2 moves with it so that it remains in The Sapphire optimizations have in- the same relative place inside W1. The screen picture for W1 and W2 cannot be moved creased the window raster-op efficiency by separately to the new position. Imagine that Wl is to be moved slightly to the right. If a factor of two on average and a factor of the subwindow is moved first inside Wi, the part of Wi marked a will be erased. If the 10 in certain special cases. Window raster- uncovered part of W1 (gray area) is moved first, the portion of W2 marked b will be ops in uncovered windows bypass the ex- erased. Therefore, the entire picture (Wi including W2) must be moved as a unit, but pensive algorithm altogether and there- window W3 must still not be affected. fore perform almost at the hardware raster-op's speed. Of course, the window raster-op overhead could be vastly re- have special size constraints. For example, but their algorithms are usually produced by coding it in microcode. If the a header subwindow may be exactly one prietary, so efficiency comparisons are covered window raster-op is made faster, text line high, irrespective of the window difficult. it will not only help applications but also size. Therefore, Sapphire leaves all sub- Sapphire was influenced by many ofthese Sapphire itself, since window raster-op windows the same size and in the same systems; but the algorithms described here performs all window manipulations. relative position with respect to the origin are original. Another area of efficiency is memory of the parent window, and notifies the ap- For most operations, Sapphire and Blit usage. In the Accent operating system sup- propriate application program of the win- have the same complexity. For example, porting Sapphire, it is very expensive to dow's size change so it can reconfigure the window raster-op in both is 0(n2) (where allocate memory for pictures. Therefore, subwindows (if necessary). n is the number of windows). In the rec- whenever a window iscreated with backup Although window modification is fairly tangle intersection algorithm, Sapphire's memory, enough memory is allocated for complex, it does maintain the most infor- complexity is the same as for Blit (O(n3) the entire window, even though only parts mation possible and provides a great deal for all rectangles for all windows). Sap- of the window may be covered. This will of flexibility. When a window has no sub- phire may create fewer rectangles because clearly waste a lot of memory. During up- windows, many of the above steps are covered rectangles are not subdivided as dates, the appropriate parts of the mem- omitted. Because windows are moved they are in Blit, but this saving may be ory buffer are addressed for the covered rarely and only at a user's command, the overridden in practice by the four exterior portions of the window. (The rest of the modify procedure has proved acceptable. rectangles. The WHIM window manager 3 buffer is left unused.) essentially uses the Blit algorithm, but The window intersection algorithm Efficiency comparison does not subdivide covered rectangles, so allows memory allocation and dealloca- The covered window paradigm has been it will have fewer rectangles. tion that can be added easily if the around for a fairly long time. It was When doing a window raster-op, Blit operating system makes this feasible. developed at the Xerox Palo Alto (Calif.) must sort the rectangles. in Sapphire, the When a window's covering changes, how- Research Center in the Smalltalk 8 and In- rectangles are already sorted. The disad- ever, two pieces of memory would be al- terlisp "I environments. These and other vantage ofSapphire's technique, however, located during the update (as in WHIM 3). early implementations of covered win- is that the full intersection algorithm must The old memory would be released after dows typically only allowed applications be run whenever windows are manipu- the update was completed. On Blit, only as to update a window if it was not covered. lated, while Blit must only transfer some much memory as needed is allocated, and The Blit window manager,5 which does rectangles from one window to another. an XOR swap transfers pictures from one provide output to covered windows, was (Of course, this is done only when win- buffer to another. Sapphire could use this probably the first covered window system dows are moved, created, deleted, grown, technique if the temporary extra memory to have its implementation openly pub- or reduced.) Sapphire has been optimized, usage was a problem, but this takes three lished. Sapphire implements a superset of however, for the case where a window hardware raster-ops instead of the two Blit's features. simply becomes more covered (sent to the now used. For example, the Blit implementation bottom). Here, the existing rectangle set is The theoretical efficiency of a window only provides automatic refresh with simply intersected with the new window. manager is not nearly as interesting as the saved bitmaps, while Sapphire allows This is much faster than recreating the measured performance of actual proce- application-handled or automatic refresh. rectangle list from scratch and is about as dures. Unfortunately, many ofthe optimi- Also, Blit does not support subwindows or efficient as the Blit technique, while zations applied to Sapphire have proved changing a window's size. (The change- preserving the rectangle order. ineffective because a much larger propor- size operation on Blit is implemented by The major time-critical part ofSapphire tion of the time is actually spent in deleting the window and recreating it, and is the window raster-op itself. Some mea- operating system interprocess communi- therefore losing the contents of the win- surements show that, in many operations, cation. TheAccent2 operating systemuses dow.) There are many other window manag- the covered window raster-op overhead message passing with separate address ers supporting covered windows today,1'341 over the hardware raster-op is quite spaces for separate processes, and Sap-

66 COMPUTER phire is implemented as a separate process 13. User's Manual for Release I of the from all applications for protection and Acknowledgments Information Technology Center Proto- structured design. type Workstation, Information Technol- Help with the algorithms came from Unfortunately, the time to send a mes- ogy Center, Carnegie-Mellon Univ., Stoney Ballard, John Strait, and Dave Pittsburgh, 1984. sage to Sapphire, along with the required Golub of PERQ Systems Corp. Amy process swapping, swamps the time to per- 14. W. Teitelman, "A TourThroughCedar," Butler and Dave Golub have been largely Software, Vol. 1, No. 2, Apr. 1984. form the actual graphics. Therefore, the responsible for maintaining Sapphire. most 15. L. Bannon et al., "Evaluation and Analy- worthwhile optimization is allowing Thanks also to Alain Foumier of the Uni- applications to do graphics directly to un- sis of Users' Activity Organization," versity of Toronto for help in evaluating Proc. SIGCHI 83: Human Factors in covered windows and thereby eliminate the algorithms' complexity and to Rob Comp. Syst., Boston, Dec. 1983, pp. the message to Sapphire. To provide the Pike of AT&T Bell Laboratory for check- 54-57. necessary protection and synchronization, ing the Blit information. For help and 16. B. A. Myers, "The Importance of Per- however, this optimization has required support with this article, I thank my wife, cent-Done Progress Indications for that more of the window manager be im- Bernita Myers, and William Buxton, Computer-Human Interfaces," Proc. plemented in the operating system kernel. Brian Rosen, Joyce Swaney, Ron Baeck- SIGCHI 85: Human Factors in Comp. Unfortunately, the current design does not er, Eugene Fiume, and many others at the Systs., San Francisco, Apr. 1985. allow applications to do raster-ops to University of Toronto and PERQ Systems 17. W. M. Newmanand R. F. Sproull, Princi- covered windows without a message to Corp. ples of Interactive Computer Graphics, Sapphire, since Sapphire stores the rect- Second Edition, McGraw-Hill, New angle lists in its private address space. York, 1979, pp. 261-265.

The algorithms performing covered References window operations in Sapphire have 1. B. A. Myers, "The User Interface for many advantages over other algo- Sapphire," Computer Graphics and rithms. These include application-handled Applications, Vol. 4, No. 12, Dec. 1984, or automatic refresh, moves, and size pp. 13-23. changes of windows, support for subwin- 2. R. Rashid and G. Robertson, "Accent: A dows, and generally more efficient win- Communication Oriented Network Oper- dow raster-ops because the rectangle lists ating System Kernel," Proc. Eighth are always kept sorted. Some of this flex- Symp. Operating Syst. Princ., Asilomar, ibility does not appear to be required in Calif., Dec. 1981, pp. 64-75. practice. For example, windows virtually 3. M. J. Goodfellow, "WHIM, The never change coveredness except to the top Window Handler and Input Manager," or bottom, and it is rare to change a win- Proc. First Int'l Conf. Computer Work- dow's size or position while it is covered by stations, San Jose, Calif., Nov. 1985, pp. other windows. Also, applications rarely 12-21. create subwindows that overlap. 4. ICL PERQ Guide to PNX, International More investigation needs to be done on Computers Ltd., Reading, England RG3 what techniques and facilities are impor- INR, UK. tant in practice and on efficient ways of 5. R. Pike, "Graphics in Overlapping Bit- implementing them. Also, as graphics map Layers," ACM Trans. Graphics, Vol. 2, No. 2, April 1983, pp. 135-160. hardware supports more sophisticated Brad A. Myers is a PhD candidate in computer such as color 3D 6. R. Rhodes, P. Haeberli, and K. Hickman, science at the University of Toronto and an oc- operations, and transfor- "Mex - A Window Manager for the mations, as in the IRIS,6 methods forwin- casional consultant. From 1980 until 1983, he IRIS," Usenix Summer Conf. Proc., worked at PERQ Systems Corp., where he dowing these must be investigated. It will Portland, Ore., June 1985, pp. 381-392. to on designed and implemented the Sapphire win- also be useful gather statistics typical 7. Sun Windows Programmers' Guide, Sun dow managerand many PERQ demonstrations window manager use and efficiencies to Microsystems, Inc., Jan. 1984. for the SIGGraph equipment exhibition. His evaluate the trade-offs between simple 8. L. Tesler, "The Smalltalk Environment," research interests include User Interface shadow bitmaps, complex clipping algo- Byte, Aug. 1981, pp. 90-147. Management Systems, or UIMSs, user inter- rithms as described here, and application- 9. A. J. Wilkes et al., "The Rainbow faces, interaction techniques, window manage- handled refresh. Workstation," The Computer Journal, ment, programming environments, debugging, The Sapphire implementation demon- Vol. 27, No. 2, 1984. and graphics. strates that full-functionality covered win- 10. G. Williams, "The Apple Macintosh Myers received the MS and BS degrees from dows can be provided while saving only Computer," Byte, Feb. 1984, pp. 30-54. the Massachusetts Institute of Technology, the covered portions off screen. Whether during which time he was a research intern at 11. Interlisp Reference Manual, Xerox Xerox PARC. these algorithms will be appropriate de- Corp., Pasadena, Calif., Oct. 1983. pends on the particular circumstances, but 12. S. A. Bly and J. K. Rosenberg, "A the general principles and complexity Comparison of Tiled and Overlapping Myer's address is Dynamic Graphics Project, results will continue to be important. rl Windows," Proc. SIGCHI 86: Human Computer Systems Research Institute, Univer- Factors in Comp. Systs., Boston, Apr. sity of Toronto, Toronto, Ont., M5S IA4, 1986, pp. 101-106. Canada.

September 1986 67