TheThe IconIcon AnalystAnalyst
In-Depth Coverage of the Icon Programming Language
April 1999 Number 53 ploration of weaving, which focuses on patterns. Instead, we use pattern-forms [3], which include In this issue … the Painter weaving language repertoire [2,4]. Weaving Drafts ...... 1 Pattern-Form Drafts Graphics Corner ...... 4 It’s easy enough to represent the five parts of A Small Programming Problem ...... 10 a draft by strings. The threading and treadling Built-In Generators ...... 16 sequences (T-sequences) can be composed from Answers to Structure Quiz ...... 19 characters that label the shafts and treadles, re- spectively. The warp and weft color sequences (C- Quiz — Expression Evaluation ...... 19 sequences) can be composed from characters that What’s Coming Up ...... 20 label colors. Weaving Drafts The tie-up is a matrix that can be represented by, say, concatenated rows composed of zeros and We now know that handwork is a heritage which no ones. It’s also necessary to add dimension informa- machine can ever take from us; we are adjusting our tion, since the matrix need not be square. needs to this knowledge. There is one missing ingredient: the colors — Marguerite P. Davison [1] themselves. To be general, we’d need the actual color values. For our purposes, however, Icon’s The term draft is used in weaving for any built-in color palettes do nicely. There are two description of a weave that can be used to produce reasons for this: (1) the number of different colors it on a loom. For a treadle loom, a draft has five in actual weaves is small, and (2) color fidelity is parts: not necessary for exploring patterns in weaves; in threading sequence fact, it is not even achievable in actual weaving. treadling sequence There is one potential problem with using warp color sequence palettes. The color palettes span color space in weft color sequence various ways. No color palette can come even close tie-up to representing, say, 50 shades of blue (although the grayscale palettes can represent up to 256 shades There are many other aspects of a weave that of gray). We’ve not seen this kind of problem in we won’t consider here, such as thread thickness. practice, but we’re working on programmer-de- fined palettes to supplement the built-in ones. Draft Formats Such defined palettes could handle situations the Drafts traditionally have been hand-drawn built-in ones cannot. Although programmer-de- on grid paper [2]. Most computer weaving pro- fined palettes do not exist yet, they at least give us grams display an on-screen grid that is familiar to an out for now. weavers, but weaving drafts are stored in propri- The name of the palette from which the color etary formats. There is also a program-indepen- labels in C-sequences are composed adds a line to dent format for exchanging drafts between pro- pattern-form drafts. We added a line for the draft grams. We’ll describe it later in this article. name, making a pattern-form draft seven lines in None of these formats is suitable for our ex- all. Figure 1 on the next page shows an example of a pattern-form draft.
The Icon Analyst 53 / 1 Grammar Drafts To keep a draft together in one file, we de- signed a grammar-draft format that is composed We have used grammars to find and charac- of three sub-grammars — one for the T-sequences, terize patterns in various kinds of strings [3, 5-8]. one for the C-sequences, and a token grammar in To use this approach for pattern-form drafts, we which the name, tie-up, and palette are hidden need a corresponding grammar draft. The prob- away. The sequence pairs are connected in defini- lem is that a weaving consists of several relatively tions that use variables not contained in the se- unrelated parts. quences. The T-sequences naturally go together, since A grammar draft is shown in Figure 2 with they often have patterns in common — in fact, in long sequences truncated. It is the grammar from many cases the two T-sequences are the same: which the pattern-form draft in Figure 1 was de- “tromp as writ” in weaving parlance. rived. The C-sequences have similar relationships and use the same labeling characters. But the char- WIFs acters in T- and C-sequences mean different things: shafts and treadles versus colors. To combine them WIFs (Weaving Information Files) provide a all in one grammar invites chaos. program-independent way of specifying weaves. WIFs are widely used to exchange drafts between There are also patterns in tie-ups, but they weaving programs <1>, and there are several stand- tend to be two-dimensional in nature and search- alone programs for converting between propri- ing for such patterns in a one-dimensional repre- etary forms and WIFs <2-5>. sentation does not work well.
aquadesign name [[1>8]∗5][[7<1]8∗4]765432[[1>8]∗5][[7<1]8∗4][7<1] threading [[1>8]∗10] treadling [G∗20][H∗39][G∗20][G∗19][H∗39][G∗20] warp colors [0–>80] weft colors c1 palette 8;8;1001001111000001111000000111000000111001100101000100101000100101 tie-up Figure 1. A Pattern-Form Draft
name:T–aquadesign.wgr comment: axiom:@ genr:1 @–>X;Y X–>1234567812345678123456781234567812345678765432187654321876543218765432... Y–>1234567812345678123456781234567812345678123456781234567812345678123456... end: name:C–aquadesign.wgr comment: axiom:@ genr:1 @–>U;V U–>GGGGGGGGGGGGGGGGGGGGHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH... V–>0 end: name:t–aquadesign.wgr n–>aquadesign p–>c1 t–>8;8;1001001111000001111000000111000000111001100101000100101000100101 end: Figure 2. A Grammar Draft
2 / The Icon Analyst 53 The WIF format is text-based, very general, [WIF] and extensible. Its syntax is based on the Windows Version=1.1 INI format [9], which uses named sections that Date=April 20, 1998 contain lists of keywords and associated values. [email protected] Source Program=QD WIF Figure 3 shows portions of a typical WIF. Source Version=0.9.2 WIFs have properties that make them inap- [CONTENTS] propriate for direct use in our investigations: Color Palette=yes Weaving=yes … • The format is very verbose; a typical WIF [COLOR PALETTE] contains a large amount of redundant infor- Entries=16 mation. Form=RGB Range=0,255 • Parsing a WIF file is a messy process and not [WEAVING] something to do repeatedly. Shafts=16 Treadles=25 • The format provides no mechanism for de- [THREADING] scribing patterns — it’s just raw data. 1=1 2=2 On the other hand, WIFs provide a ready 3=1 source of material for studying patterns in weaves. 4=2 5=3 Some WIFs are available on the Web <6-8> and 6=4 there is a moderately priced CD-ROM that con- … tains thousands of WIFs [10]. [TIEUP] 1=4,5,6,7,8,9,10,11,12,13,14,16 To use WIFs, we need to be able to convert 2=1,2,6,11,12,13,14,15 them to pattern-form drafts. This is the kind of task 3=1,2,3,4,8,12,16 for which Icon is well suited. As is typical with 4=1,2,3,4,5,6,10,15 5=1,2,6,11,14 format-conversion programs, the process is messy 6=1,2,3,4,5,6,7,8,12,16 and infested with special cases. We won’t list the … program here: It would take up a lot of space [TREADLING] without providing much in the way of insight. We 1=17 2=18 will, however, include it with the Web material for 3=17 this issue of the Analyst, along with a consider- 4=18 ably simpler program to convert pattern-form drafts 5=17 6=18 to WIFs. … [COLOR TABLE] Further Articles 1=192,192,192 2=152,152,152 We have several articles on weaving in the 3=0,0,0 4=176,144,144 works, including 5=116,100,152 • a weaving case study 6=152,100,132 … • finding patterns in draft sequences [WARP COLORS] • creating images from drafts 1=6 • creating drafts from images 2=5 3=3 • name and algebraic drafting 4=3 5=3 • network drafting 6=8 • name and algebraic drafting … • sequence drafting [WEFT COLORS] 1=14 • dobby devices and liftplans 2=12 3=8 • weavability 4=14 • a weaving program 5=3 6=14 We’re also investigating Jacquard loom tech- … nology and exotic topics like three-dimensional Figure 3. A WIF weaving.
The Icon Analyst 53 / 3 http://www.gac.edu/~max/weaving/ptn2wif/ The situation with respect to weaving is remi- niscent of versum sequences — the more we ex- 4. QD WIF (Macintosh) for Macintosh ProWeave , plore, the more we find to explore. Incidentally, WeaveMaker One, WeaveNet, and WeavePoint: we’re not finished with versum sequences — we’re http://www.softweave.com/html/QD_WIF.html just taking a breather. And don’t be surprised to see a versum weave. 5. WIFCNVT (Windows) for WeaveIt: Acknowledgment http://www.weaveit.com/support.htm We owe thanks to Jane Eisenstein, an experi- 6. Weaving File Exchange: enced handweaver. She uses several weaving pro- http://www.wyellowstone.com/users/ww/ grams for the Macintosh and is the author of QD weaving.htm#File Exchange WIF for that platform <5>. She has patiently an- 7. Association of Northwestern Weavers Guilds: swered our naive questions as we’ve been learning about weaving. She’s also given us pointers to ftp://anwg.org/pub/weaves/ other useful resources related to weaving. 8. Maple Hill Software: References http://www.mhsoft.com/wif/files/ 1. A Handweaver’s Pattern Book, Marguerite P. Davison, Marguerite P. Davison, Publisher, 1994. 2. “A Weaving Language”,Icon Analyst 51, pp. 5-11. Graphics Corner — Changing Image Colors 3. “Character Patterns”,Icon Analyst 49, pp. 1-6. There is no room for frivolity where color is concerned. 4. “A Weaving Language”,Icon Analyst 52, pp. 1-3. – Color Marketing Group [1] In the last issue of the Analyst [2], we showed 5. “Character Patterns”,Icon Analyst 48, pp. 1-7. how mutable colors can be used in to create anima- 6. “Versum Deltas”,Icon Analyst 49, pp. 6-11. tions. This article describes a quite different use of mutable colors — changing colors in an image 7. “Analyzing Character Patterns”,Icon Analyst interactively. 50, pp. 1-7. The Application 8. “Tricky Business — Image Grammars”,Icon Analyst 50, pp. 14-18. The idea is simple — create a copy of an image in which each color is mutable and provide an 9. Inside Windows: 3.11 Edition, Jim Boyle, et al., interface through which a user can select a color New Riders, 1994. and change it. Figures 1 and 2 show the interface 10. Over 7000 Draw Downs in Color, Eleanor Best, and an example image. multi-platform CD-ROM, [email protected],1998.
Links 1. WIF Specification: http://www.mhsoft.com/wif/wif.html 2. WeaveConvert (Windows) for Patternland 5 and Fiberworks 3: http://www.blarg.net/~ender/weaveconvert/
3. ptn2wif (Windows and source code) for Figure 1. The Application Interface WinWeave:
4 / The Icon Analyst 53 Figure 2. An Example Image The palette on the interface contains a cell for Figure 4. Modified Color Dialog each color in the image. When the user clicks on cell, a color dialog is presented, as shown in Figure In Figure 4, the selected color has been changed 3. to black, the background color for the image, caus- ing the ornamentation in the center of the image to disappear. The result is shown in Figure 5.
Figure 3. Color Dialog
Note that the dialog provides information about Figure 5. The Modified Image the use of the selected color in the image. When the user changes the color in the dialog, If the user dismisses the color dialog with Cancel Okay as shown by the example in Figure 4, that color rather than , the previous color is changes in the image. restored. The application also provides for saving a changed image and reverting to the original colors. Many other features could be added. See the Back Issues Extensions section at the end of this article.
Back issues of The Icon Analyst are avail- The Program able for $5 each. This price includes ship- ping in the United States, Canada, and The complete program is shown in Figure 6 Mexico. Add $2 per order for airmail post- on following pages. The comments about the pro- age to other countries. gram are keyed to the numbered circles in the listing.
The Icon Analyst 53 / 5 ᕡ link graphics Theposx and posy attributes link interact are added to the attributes VIB pro- link numbers vides for the application window. link tables The intent is to position the applica- global cellsize # size of palette cell tion window at the upper-left cor- global colors # mutable color list ner of the screen. This generally is global count # table of pixel counts not possible, since the window man- global image_window # window for user image age provides a frame around the global mutant # image with mutable colors window that must fit on-screen. global orig_colors # list of original colors Most window managers do the best global palette # color selection palette they can to position the window as global panel # palette window global pixels # number of pixels in image window requested. However, the actual co- global mutant_posx # location of mutant window ordinates may not be the same as global mutant_posy the requested ones. $define ColorRows 8 # palette rows ᕢ mutant_posx and mutant_posy $define ColorCols 16 # palette columns are set to position the window for $define FrameWidth 24 # allowance for window–manager frame (ad hoc) the mutable image to the right of procedure main() the application window. In general, local atts, vidgets it’s not possible to compute the win- atts := ui_atts() dow manager’s frame width. An ad put(atts, "posx=0", "posy=0") ᕡ hoc constant is used to approximate (WOpen ! atts) | stop("∗∗∗ cannot open application window") it. mutant_posx := WAttrib("posx") + WAttrib("width") + FrameWidth ᕣ The cell size is computed from mutant_posy := WAttrib("posy") ᕢ the “usable” size of the region, vidgets := ui() which is inside its visual frame. We palette := vidgets["palette"] assume here that the region size has cellsize := palette.uw / ColorCols ᕣ been set up in VIB so that this works out properly. panel := Clone("bg=black", "fg=black", "dx=" || palette.ux, "dy=" || palette.uy) Clip(panel, 0, 0, palette.uw, palette.uh) ᕤ ᕤ A window for the palette grid is clear_palette() cloned from the application win- GetEvents(vidgets["root"], , shortcuts) ᕥ dow and the clipping is set so that drawing on the grid cannot over- end write areas outside it. # Set up empty palette grid. ᕥ GetEvents() is used because this procedure clear_palette() local x, y application is entirely event driven. That is, computation is done only as Fg(panel, "black") a direct result of user requests. EraseArea(panel) WAttrib(panel, "fillstyle=textured") ᕦ GetEvent() never returns; user ac- Pattern(panel, "checkers") tion causes program termination. Bg(panel, "very dark gray") ᕦ The cells are filled initially with every x := 1 + (0 to ColorCols – 1) ∗ cellsize do every y := 1 + (0 to ColorRows – 1) ∗ cellsize do a dark checkered background. This FillRectangle(panel, x, y, cellsize – 1, cellsize – 1) ᕧ makes it easy to distinguish unused cells from cells filled with colors, WAttrib(panel, "fillstyle=solid") Bg(panel, "black") which are solid. return ᕧ Notice that the filled rectangles end are one pixel less on a side than the # Handle File menu. cell size. This provides the black Figure 6. The Program borders around the cells.
6 / The Icon Analyst 53 ᕨ Before an image is opened, the procedure file_cb(vidget, value) previous image, if any, is closed. case value[1] of { The nonnull test fails if there is not "open @O" : image_open() a previous image. "quit @Q" : quit() "revert @R" : image_revert() ᕩ The image is opened with its "save @S" : image_save() canvas hidden. A window for the } mutable image will be created, and return there is no need to produce a dis- end traction by having two windows # Open new image. visible on screen. If the image can- not be opened, WOpen() fails and procedure image_open() local i, x, y the user is notified. WClose(\image_window) ᕨ µ After the mutant window is cre- repeat { ated, Raise() is used to bring the if OpenDialog("Open image:") == "Cancel" then fail application window to the front and image_window := WOpen("canvas=hidden", "image=" || dialog_value) | { make it the “focus“ from which Notice("Cannot open image.") ᕩ events are then accepted. (The fo- next cus may not be set by all window } managers.) break ¸ } orig_colors is a table created by mutate(). Its keys are the mutable mutate(image_window) | fail # create the mutant window color numbers and the correspond- Raise() µ ing values are the original image clear_palette() # clear old color cells colors. The palette cells are filled colors := vallist(orig_colors) ¸ using the mutable colors, which at i := 0 this point have the same color val- ues as those in the image. every y := 1 + (0 to ColorRows – 1) ∗ cellsize do ¹ every x := 1 + (0 to ColorCols – 1) ∗ cellsize do { If there is no mutant window, an Fg(panel, colors[i +:= 1]) | break break attempt to save an image results in FillRectangle(panel, x, y, cellsize – 1, cellsize – 1) a notification to the user. This can } only happen if the user tries to save return an image before opening one, but end an application needs to handle ab- errant user actions (not to mention # Save current image, which is in the mutant window. crazed beta-testers) in a proper way. procedure image_save() The use of fail instead of return here snapshot(\mutant) | { ¹ and in other similar situations is Notice("No image to save.") primarily for documentation pur- fail poses and for tracing. It has no ac- } tual affect on program behavior. return Ƹ The original color values are in end orig_colors. To revert to them, the # Restore original image colors. corresponding mutable colors in the procedure image_revert() color panel are reset. Changing the local old, color color value associated with a mu- every old := key(orig_colors) do { Ƹ table color changes the color value color := orig_colors[old] in all windows of an application. It Color(panel, color, old) therefore is not necessary to change } the color values in the mutant win- Figure 6 (continued). The Program dow explicitly.
The Icon Analyst 53 / 7 ƹ NewColor() fails if no more return mutable colors are available. end This means that the image has # Get mutable colors and window from image. too many colors for this appli- procedure mutate() cation. Only 256 colors are local c, width, height, n, x, y available altogether. Two are needed for every color in the WClose(\mutant) image — one for the color itself orig_colors := table() and another for its mutable count := table(0) counterpart. Consequently, width := WAttrib(image_window, "width") images that can be processed height := WAttrib(image_window, "height") cannot have more than 128 pixels := width ∗ height characters. The actual number mutant := WOpen("width=" || width, "height=" || height, is less in most cases, since black "posx=" || mutant_posx, "posy=" || mutant_posy) | { and white are reserved and the Notice("Cannot open mutant window.") application interface uses two fail other colors. Note that if there } are too many colors, the par- every y := 0 to height – 1 do { tially completed mutable win- x := 0 dow is closed. Opening an- every c := Pixel(image_window, 0, y, width, 1) do { other image causes this also, if not(n := \orig_colors[c]) then { but it’s better not to leave the orig_colors[c] := n := NewColor(c) | { Notice("Too many colors in image.") ƹ screen cluttered, even tempo- WClose(mutant) rarily. fail ƺ For every pixel in the im- } } age, a corresponding pixel in count[n] +:= 1 the mutant window is drawn. Fg(mutant, n) Drawing an image of more DrawPoint(mutant, x, y) ƺ than trivial size, point by point, x +:= 1 is time-consuming. The prac- } tical size of images for which } this application can be used is return limited. end ƻ The user presses a mouse # Handle callbacks on the palette. button with the mouse pointer procedure palette_cb(vidget, e, x, y) positioned on a cell on the pal- local color, new ette grid to indicate that the if e === (&lpress | &mpress | &rpress) then { ƻ corresponding color is to be color := Pixel(x, y, 1, 1) # get pixel color adjusted. Note that if the color if not integer(color) then fail # not mutable of the pixel on which the user new := Color(panel, color) # get color clicks is not an integer, it is not if ColorDialog( a mutable color. This can hap- "Adjust color (" || count[color] || " pixels, " || pen if the user clicks on an frn((100.0 ∗ count[color]) / pixels, , 2) || "%):", empty cell or, more likely, ac- Color(panel, color), track, # procedure to track color changes Ƽ cidentally clicks on a border color # color being mutated pixel between cells. If this hap- ) == "Okay" then new := dialog_value pens, the event is ignored. Color(panel, color, new) Ƽ Color(mutant, color, new) The third argument in a call } of ColorDialog() is a procedure to call for every change the Figure 6 (continued). The Program user makes to the color. This
8 / The Icon Analyst 53 ColorDialog() is called (in this return case, the mutable color being end changed). The second argu- # Quit the application. ment to the tracking function procedure quit() is the current color from the snapshot(\mutant) dialog. Both the pixels in the palette grid and the image it- ƽ exit() self are changed. end # Handle keyboard shortcuts. Extensions procedure shortcuts(e) The functionality of the if &meta then case(map(e)) of { program described here could "o" : image_open() be extended is several ways. "q" : quit() Some suggestions are: "r" : image_revert() "s" : image_save() • selection of a color by click- } ing on a pixel in the mutant return image end • palette color sets that can be # Track the color in the color dialog. saved and loaded procedure track(color, s) ƾ • an undo facility Color(panel, color, s) • freeing colors when the mu- Color(mutant, color, s) tant image is closed so that return they can be reused for another image end • multiple mutant windows #===<
Acknowledgment allows user changes to be tracked in the mutant image so the user can see the effect of changes as The images used in the examples in this article and they are made and before they become perma- on the last page were adapted from an “air horse” nent. created by Laurens Lapré, using his very sophisti- cated Lparser application <1,2>. ƽ The application should keep track of whether or not the image has been changed since it was last Next Time saved and offer to save it only if it has changed. In the next Graphics Corner, we’ll return to pat- ƾ When the tracking procedure is called, its first terns and describe an application for finding inter- argument is the fourth one provided when esting tiles in even the most mundane image.
The Icon Analyst 53 / 9 References We received a number of solutions, some 1. This remark is from a 1995 press release by The though icon-group, others directly, and we added Color Marketing Group <3> in which they an- a few of our own. Two solutions were erroneous: nounced the new color names for 1995, including They failed to handle zero correctly. Coralando and Cricket. Other new colors this or- We originally planned to present the results ganization has “created” in recent years include exactly as received, preserving the author’s style, Vinaigrette, Stetson, Canyon Cloud, Elephant’s variable names, and so on. This approach, how- Breath, Highway, Red Haute, and Treasure. The ever, made it difficult to compare the solutions, so new colors they announced for 1999 include we’ve modified the solutions to use similar styles Cosmetique Peach, Beignet, Blue Moon, Par Four and names. We’ve also eliminated comments in Green, Pink for Sure, Hip Hop Yellow, and Mineola the few cases they were provided in favor of out- Orange. of-line remarks. In some cases we’ve combined very similar solutions. 2. “Animation — Mutable Colors”,Icon Analyst We’ve not associated authors’ names with 52, pp. 11-16. particular solutions (no glory; no embarrassment). Links The authors are, however, listed at the end of this article. 1. Lparser: We’ve classified the solutions by the kinds of http://www.xs4all.nl/~ljlapre/lparser.htm data structures and operations that were used.
2. Lparser for the Macintosh: Basic String and Cset Operations: http://www.geocities.com/CapeCanaveral/9376/ Several of the solutions used only basic string meshula.html#LParser and cset operations. 3. Color Marketing Group: Solution 1: http://www.colormarketing.org/ procedure digsort(i) local c, s s := "" A Small Programming Problem — every c := !"–0123456789" do Sorting Digits every find(c, i) do s ||:= c The chief cause of problems is solutions. — unknown return integer(s) end The Problem This is about as straightforward as you can Late last year we posted the following prob- get. It could be made slightly faster, on average, by lem to members of icon-group: handling 0 differently: Write a procedure digsort(i) that returns the every c := !"–123456789" do integer that results from sorting the digits of every find(c, i) do i, preserving sign. For example, digsort(201) s ||:= c should return 12 and digsort(–1042) should return integer(s) | 0 return –124. You may assume i is an inte- ger. This saves looking for zeros in the integer, leaving s as the empty string only if i is 0. This is handled We later had to clarify this with the additional by the alternative in the return expression, since provision that duplicate digits should be preserved, integer() fails and 0 is returned. Note that the so that, for example, digsort(10881) should return alternative is evaluated only if i is 0; there is no 1188. (Without this provision, the problem is es- overhead otherwise. sentially trivial — integer(cset(i))). Incidentally, some solutions similar to this
10 / The Icon Analyst 53 one used upto(c, i) instead of find(c, i). This requires generation rather than via an intermediate cset. converting the string c produced by element gen- But what about this? eration to a cset for every call of upto() and hence is slower. !(i –– '0') Solution 2: This removes any zeros and produces a cset of the remaining digits. The result is not the same as that procedure digsort(i) for the previous expression, however, if i is 0. local c, s Therefore a procedure that uses it might end as s := "" shown in the remarks following Solution 1: i := string(i) return integer(s) | 0 every c := !cset(i) do every s ||:= (find(c, i) & c) String Scanning return integer(s) Solutions that used string scanning were simi- end lar to those that used basic string operations. They generally were longer, if (perhaps) somewhat easier This solution, as simple as it appears, has to formulate. several clever aspects. The first is the conversion of i to a string, so that it doesn’t have to be done in Solution 4: every call of find(). This solution also uses cset(i) to procedure digsort(i) remove digits that do not occur in i. For some data, local s, c this may be faster, but it requires the creation of a cset and conversion back to a string for the element s := "" generation operator. Note that there is an implicit i ? { recognition that "–" occurs before the digits in the every c := !cset(i) do { collating sequence (for both ASCII and EBCDIC). while s ||:= (tab(find(c)) & move(1)) This fact was used in many solutions. tab(1) } Solution 3: } procedure digsort(i) return integer(s) local c, s end s := "" i := string(i) Compare this solution to Solution 2. every c := !cset(integer(cset(i))) do Solution 5: every s ||:= (find(c, i) & c) procedure digsort(i) return integer(s) local s, s1, s2 end s := string(i) This slight variation on Solution 2 contains an s2 := "" obscure trick (that is, it’s really clever). The expres- every s1 := !cset(s) do s ? { sion while tab(find(s1)) do s2 ||:= move(1) !cset(integer(cset(i))) } removes any zeros in i by converting all the charac- return integer(s2) ters to an integer. It would be (slightly) faster to use end !integer(cset(i)) This solution differs from the previous one in since in this case the integer, which contains at that string scanning is inside the character-genera- most one instance of every digit in i, is converted tion loop rather than outside it. directly from an integer to a string for element string05.icn
The Icon Analyst 53 / 11 Solution 6: local L, j, k procedure digsort(i) L := [] local s j := abs(i) while j ~= 0 do { s := "" put(L, 0 ~= j % 10) every s ||:= (i ? (tab(find(!cset(&subject))) & j /:= 10 move(1))) } return integer(s) every k := !sort(L) do end j := j ∗ 10 + k Here string scanning is inside the concatena- if i < 0 then j := –j tion that is building the result, while the element- return j generation operation is inside string scanning. This end formulation seems to us to be a bit strange, but it is different. Surprise: Actual arithmetic! Lists Solution 10: The solutions that used lists did so to accumu- late characters or character counts and then sort() procedure digsort(i) to get them in the right order. local L, s Solution 7: L := [] procedure digsort(i) i ? { s := tab(any('–+')) | "" local L, s while put(L, move(1)) L := list() } every put(L, !i) L := sort(L) s := "" while s ||:= get(L); every s ||:= !sort(L) return integer(s); return integer(s) end end This solution initializes the string of sorted This solution illustrates the basic method — digits in string scanning. Note that the plus sign is straightforward and clear. spurious; an integer (assumed), when converted to a string, never has a leading plus sign. Solution 8: Solution 11: procedure digsort(i) local L, s procedure digsort(i) local L, j, d, s every put(L := [ ], !i) L := list(9, "") every (s := "") ||:= !sort(L) every d := !abs(i) do return integer(s) L[d] ||:= d end s := "" This solution is essentially the same as the last every s ||:= !L one, but it is made slightly shorter, in some sense, return if i < 0 then –s else (integer(s) | 0) by nesting expressions. end Solution 9: This solution differs from the preceding ones procedure digsort(i) by using failure on an out-of-bounds subscript to
12 / The Icon Analyst 53 dispense with zeros. Since a minus sign would every T[!i] +:= 1 cause an error, it works on the absolute value. Notice that unary minus operation converts nu- L := sort(T, 3) meral strings representing integers to integers. s := "" Solution 12: while d := get(T) do procedure digsort(i) s ||:= repl(d, get(L)) local L, j, s return integer(s) L := list(9, 0) end every L[!abs(i)] +:= 1 Like Solution 12, this one keeps counts of s := "" characters and uses repl() rather than repeatedly concatenating digits. every j := 1 to 9 do s ||:= repl(j, L[j]) Solution 15: return if i < 0 then –s else (integer(s) | 0) procedure digsort(i) end local L, s if i = 0 then return 0 This solution is a variant of the preceding one, counting digits instead of concatenating them one L := table(0) repl() at a time. Then is used to build the string of every L["0" ~== !i] +:= 1 digits. This solution should be faster for large integers. L := sort(L, 3) s := "" Tables while d := get(L) do Tables were used for basically the same rea- s ||:= repl(d, get(L)) sons as lists. return integer(s) | 0 Solution 13: end procedure digsort(i) local T, s This solution filters out zeros while counting other characters. Is it worth the time and trouble? T := table("") Solution 16: every s := !i do T[s] ||:= s procedure digsort(i) local T, L, d, s s := "" T := table(0) every s ||:= T[!cset(i)] every T[!i] +:= 1 return integer(s) delete(T, "0") end L := sort(T, 3) Notice the similarity between this solution and Solution 11. Using tables, however, allows a s := "" minus sign to be handled like any other character. while d := get(L) do s ||:= repl(d, get(L)) Solution 14: return integer(s) | 0 procedure digsort(i) local T, L, d, s end if i = 0 then return 0 This solution counts zeros but removes the T := table(0) table entry before constructing the final string.
The Icon Analyst 53 / 13 your own judgement of these. Two more or less The Different objective measurements can be made: size and speed. Here are two solutions that are quite different in character from the others. Size Solution 17: The procedures all are small, so some mea- sures that would be meaningful for large programs procedure digsort(i) are not so meaningful here. For what it’s worth, return if i < 0 then –digsort1(–i) else digsort1(i) here are the sizes measured in terms of non-blank lines, bytes of source code, and the number of end syntactic tokens. The latter probably is the most procedure digsort1(i) significant measure. The smallest values are un- local pt1, pt2 derlined. if i[pt1 := 2 to ∗i] < i[pt2 := 1 to pt1 – 1] then soln. lines bytes tokens ∗ return digsort1(i – (i[pt2] – i[pt1]) 1 8 162 28 ∗ ∗ (10 ^ (pt1 – pt2) – 1) 10 ^ ( i – pt1)) 2 8 176 36 return i 3 8 188 40 4 11 220 41 end 5 10 208 43 The author has this to say about the solution: 6 6 147 30 “This won’t win any brevity or obscurity awards, 7 8 152 32 but it is another approach“. We’re not so sure about 8 6 137 29 the obscurity part, at least in the context of Icon. 9 13 257 58 10 11 207 45 Solution 18: 11 9 207 50 procedure digsort(i) 12 9 219 54 local j, sw 13 9 175 39 14 11 235 57 i := string(i) 15 11 243 59 sw := 1 16 11 232 57 17 10 301 71 while \sw do { 18 15 297 60 sw := &null every j := 1 to ∗i – 1 do { Speed if i[j] >> i[j + 1] then { We tested the solutions with 5,000 instances i[j] :=: i[j + 1] of seven kinds of integers: positive and negative 5- sw := 1 digit, 10-digit, and 15-digit ones and one of all } zeroes — 35,000 calls in all. Except for the zeros, the } integers were produced at random. The times in } milliseconds from runs on a 233MHz DEC Alpha return integer(i) are shown in Figure 1 on the next page. The times for the fastest solutions are underlined. end The apparent anomaly for Solution 15 with all We suppose there had to be one actual char- zeroes is easily explained — it’s the only solution acter sort. In this case it is the classic bubble sort, that tests for zero before doing anything else. It’s which is one of the least efficient sorting methods, not surprising that the last two solutions did not on average. Note that a minus sign is handled fare well. The magnitude of the difference, espe- properly, since string comparison is used. cially for Solution 17, was surprising to us — it runs Evaluation nearly 154 times slower than Solution 2. But what about large integers? There are none Clarity and style are subjective. You can make for the timings above (the Dec Alpha has 64-bit
14 / The Icon Analyst 53 soln 0 5-digit + 5-digit – 10.digit. + 10-digit – 15-digit + 15digit – total 1 650 934 1000 1233 1283 1517 1584 8201 2 250 567 617 850 916 1100 1184 5484 3 300 617 650 850 916 1100 1184 5617 4 400 1167 1333 1933 2116 2650 2850 12449 5 367 1100 1283 1883 2050 2650 2800 12133 6 350 900 1017 1517 1650 2100 2234 9768 7 300 600 667 1017 1066 1400 1467 6517 8 300 600 633 983 1016 1367 1434 6333 9 267 1217 1217 2167 2116 3034 3034 13052 10 450 934 967 1550 1566 2150 2200 9817 11 484 884 833 1200 1166 1517 1517 7601 12 1234 1584 1517 1817 1783 2084 2084 12103 13 434 1084 1217 1750 1883 2367 2500 11235 14 34 1350 1533 2083 2283 2700 2917 12900 15 17 1317 1517 2000 2233 2617 2817 12518 16 450 1317 1533 2033 2266 2600 2817 13016 17 100 4417 4483 35250 35833 131717 133034 344834 18 184 2684 3050 10967 11716 25467 26767 80835
Figure 1. Timings words). We tried timing the programs for some 9 416 782234 large integers, using 5,000 terms in the versum 10 266 487050 sequence for 196. These integers get large quickly. 11 233 433734 Their average length (number of digits) is more 12 216 399484 than 1,048 and the last one has 2,088 digits. Most of 13 433 771584 these numbers are large enough for the quadratic 14 250 450917 complexity of conversion between integers and 15 233 439267 strings to have a significant impact [1]. 16 233 435567 17 ? – Here are the results: 18 343933 – soln. last all We didn’t attempt to perform the tests for the 1 1883 3188534 entries that have a dash. We did try Solution 17 for 2 233 402050 the last integer. We ran it in the background ex- 3 216 380184 pecting it to complete in a large but reasonable 4 433 759684 amount of time. It had been running over 25 days 5 266 495000 (more than 2×10 9 milliseconds) when its process 6 250 506350 was accidentally terminated. You might think it 7 250 468850 (or Icon) was in a loop. But we tried a much smaller 8 250 452734
Supplementary Material
Supplementary material for this issue of the Analyst, including color images and Web links, is available on the Web. The URL is http://www.cs.arizona.edu/icon/analyst/iasub/ia53/
The Icon Analyst 53 / 15 large integer for which Solution 17 ran a very long know, misunderstand them. time but completed successfully. There is a lesson Articles about generators and sequences have here. appeared in past issues the Analyst [1-4], and they’ve been used in central ways in articles about Contributors applications such as numerical carpets [5-6]. And The following persons contributed solutions generators have appeared in some context in al- to the digit-sorting problem: Wade Bowmer, Michael most every issue of the Analyst. Nevertheless, Glass, Ralph Griswold, Nevin Liber, Todd generators and sequences have not been given the Proebsting, Gregg Townsend, Ken Walker, Steve coverage they deserve. This article is the first in a Wampler, and Cheyenne Wills. series in which we plan to explore generators and sequences in depth. We’re starting with this ar- Although we’ve not identified the authors of ticle, which reviews Icon’s built-in generator rep- the particular solutions, it seems only fair to give ertoire. Most of the material that follows is el- Steve Wampler kudos for submitting the fastest ementary and designed to provide a foundation solution. for future articles.
Reference Element Generation 1. The Art of Computer Programming, Vol. 2, Element generation, !x, is one of the most Seminumerical Algorithms, Donald E. Knuth, versatile operations in Icon’s repertoire. Addison-Wesley, 1969. !x applies to any value that has “elements”. This covers much territory — strings, lists, records, Built-in Generators sets, tables, and files. In addition, the element generation operation can be applied to any value If a language doesn’t affect the way you think about that can be converted to a string, such as a cset. programming, it’s not worth knowing. Strings, lists, and files are sequences of val- — Alan Perlis ues and generation from them is from the first element to the last. Note that element generation One of the defining characteristics of Icon is its treats files as sequences of lines in the fashion of extensive use of sequences — values in order. read(). Records can be viewed as a sequence of Strings are sequences of characters. Lists are values in the order their fields are given in the sequences of values that may be of any type. Files declaration. are sequences of characters or lines, depending on The elements of sets and tables have no in- how they are interpreted. herent order, but their values can be generated, Data objects that are sequences are nothing which imposes an order of sorts on them. As noted new, although in most programming languages the in the quiz answers on page 19, the order is not concept is not stressed. Generators that produce sequences of values in time are central to computa- tion in Icon. Since most programming languages don’t have generators, programmers who come upon Icon with experience in other languages often overlook the possibilities generators offer — or, in an attempt to understand them in terms of what they already
Downloading Icon Material
Implementations of Icon are available for downloading via FTP: ftp.cs.arizona.edu (cd /icon)
16 / The Icon Analyst 53 random, but it is unpredictable and depends on the There is one generator in Icon’s graphics rep- inner workings of the implementation [7,8]. ertoire: Pixel(). It generates the color values of the The generation of values from a table is fur- pixels in a specified rectangular area. The values ther complicated by the fact that a table element are either strings — comma-separated RGB triples consists of a pair of values, a key and it’s corre- — or for mutable colors, small negative integers. sponding value (for which we have no better name). Control Structures !T generates the values corresponding to the keys, not the keys themselves. The function key(T) There are two control structures for compos- generates the keys. There probably is a better way ing generators from expressions: alternation and to design element generation for tables, but it’s far repeated alternation. These control structures are too late now to change anything in the main com- not generators in and of themselves. Instead they putational repertoire of Icon. evaluate expressions in ways that result in genera- tion. Integer Generators Alternation, e1 | e2, is one of Icon’s most There are two ways to generate integers by useful control structures. It produces the sequence fixed increments in increasing or decreasing nu- of e1 followed by the sequence for e2. Note that if merical order. The more familiar is e1 and e2 are not generators but produce only one value each, their alternation is a generator (that i to k by j produces two values). The function seq(i, j) does the same thing, but Repeated alternation, |e, is rarely used and its with no limit. potential is easily overlooked. |e repeatedly pro- duces the sequence produced by e. It can be thought String-Analysis Generators of as Three string-analysis functions generate the e | e | e | … positions of substrings within strings: bal(), find(), However, repeated alternation treats one case and upto(). specially. If e produces no result (fails), |e fails. These functions produce integers in increas- This handling of failure is designed to prevent ing numerical order. The number of values these repeated alternation from going on endlessly with- functions can produce is, of course, limited. out ever producing a result — going over a sort of evaluation event horizon. Keywords It’s important to realize that an expression in There are five keywords that are generators: repeated alternation may produce values for a &features while and then fail, terminating the repeated alter- nation. For example, |read(f) generates lines from f ®ions but terminates when the end of the file is reached &storage and read(f) fails. &allocated Perhaps the most important use of repeated &collections alternation is to create a generator from a non- The values generated by &features depend on generator, as in the example above. Another ex- the features that are available in a particular imple- ample is |?0, which generates an unending se- mentation, such as co-expressions and graphics. quence of real numbers in the range 0.0 to 1.0. ®ions &storage and generate three values The Rationale for Generators related to storage utilization, while &allocated and &collections generate four values related to stor- Why are some operations generators and oth- age management. ers not? The general design philosophy for Icon was Miscellaneous to make an operation a generator only if there was function() generates the names of the built-in a good reason. This was partly a conservative functions. The values depend on the platform and attitude toward language design, and it was not the features supported. motivated by other considerations, such as pos-
The Icon Analyst 53 / 17 sible implementation problems. guage designed with reliance on good program- The best reason to cast a computation as a ming practice would not get very far. generator is when the computation naturally pro- The other side of the coin is that it is easy duces a sequence of values and it’s necessary to enough to make any non-generator into a genera- maintain an internal state to get from one value to tor using repeated alternation. the next. This is the case for element generation, Some of Icon’s operations were cast as gen- integer generation, and the string-analysis func- erators for reasons other than those stated above. tions mentioned earlier. Note that where the num- In the case of the keywords and function(), it was ber of possible values is limited, a list containing all easier to implement them as generators than to, of them is an alternative formulation. This, of say, produce lists of values. This probably was a course, does not work for generators with a poten- mistake from a design viewpoint, but at least these tially unlimited number of results, such as seq() generators are needed infrequently. and |e. Another advantage of generators is that Pixel(x, y, w, h) is a generator for entirely values are only produced as needed — a form of different reasons. The operation could be formu- lazy evaluation. And, in practice, it’s often the case lated (and originally was) to return the color value that not all of the possible values are needed. of a single pixel. In this formulation, Pixel() would There are two ways to cast a computation that have to be called many times for a large rectangular does not meet these criteria as a generator. One is area. But that was not the problem. In a client- to generalize it so that it does. The other is to just server environment, each call of Pixel() produces a have the computation done repeatedly. request to the server. With a slow communication An example of the first way is the string- link, performance could be intolerably bad. As it is analysis function match(s1, s2), which succeeds if cast, one call of Pixel() requires only one server s1 is an initial substring of s2 and returns the request, which produces all the color values in the position i in s2 after s1. In the initial design of Icon, rectangle, which then are generated. we considered generalizing match() so that if re- sumed, it would generate i – 1, i – 2, … 1. We tried References this with disastrous results — straightforward string analysis produced unexpected results and it 1. “Generators”,Icon Analyst 3, pp. 8-10. was necessary to use limitation to prevent them. 2. "Result Sequences",Icon Analyst 7, pp. 5-8. The lesson here is that we did not have a good reason for this generalization — nothing clearly 3. “Programming Tips (element generation)”,Icon useful to offset the complications. Analyst 8, p. 12. Casting a computation as a generator simply by having it do the same operation repeatedly can 4. “Programming Tips (recursive generators)”, have unexpected consequences. Consider this ex- Icon Analyst 13, pp. 10-12. ample, which takes different actions depending on whether or not a value read is "end". 5. “Anatomy of a Program — Numerical Carpets”, Icon Analyst 45, pp. 1-10. if read() == "end" then … else … If read() were a generator but the value read not 6. “Exploring Carpet Space”,Icon Analyst 47, "end", read() would be resumed to read another pp. 5-10. line. If no line was "end", the entire file would be 7. “The Design and Implementation of Dynamic consumed, which probably would not be the in- Hashing for Sets and Tables in Icon”, William G. tended effect. Griswold and Gregg M. Townsend, Software — The limitation control structure could be used Practice & Experience, Vol. 23, No. 4, April 1993, pp. to avoid this, but if there were many such genera- 351-367. tors, programs would have to be littered with limitations to assure there was no unexpected gen- 8. Supplementary Information for the Implementation eration. of Version 8 of Icon, Ralph E. Griswold, Icon Project You might argue that programmers shouldn’t Document 112b, The University of Arizona, April write the kind of code shown above. But a lan- 7, 1996.
18 / The Icon Analyst 53 Answers to Quiz 18. True. on Structures 19. False in general. See Icon Analyst 52 for the quiz questions. 20. 1. False.L[0] is an attempt procedure keyset(T) to reference an element be- S := set() yond the end of L, which fails. every insert(S, key(T)) 2. True. return S 3. They both produce the same results, but the first end leaves L unchanged, while the second removes all its elements, leaving it empty. 21. Values can be obtained from keys, as inT[key(T)] . The converse is not true. 4. It adds a null-valued element to the right end of L. 22. True, although this is a property of the imple- mentation and not specified in the language. 5. They all leaveL unchanged, although some dif- fer in intermediate steps. 23. False. Although there is no language or imple- mentation limit on the number of elements in a 6. L := sort(set(L)). table, it must fit in memory, which is limited. 7. True. 8. False. 9. False. Quiz — Expression 10. False. IfR has a field named center, they pro- Evaluation duce the same results, but if R doesn’t have a field named center, R.center causes a run-time error, 1. True or false: Arepeat while R["center"] fails. loop only can be termi- 11. True. nated by evaluating a break expression within it. 12. True. 2. True or false: In 13. False. if e1 then e2 else e3 14. False; the order is unpredictable but not ran- if e1 is a generator and dom. produces a result but e2 fails, e1 is resumed. 15. ∗sort(S). 3. True or false: Acase expression can fail. 16. procedure elim_str(S) 4. True or false: Acase expression can generate a sequence of values. every x := !S do if type(x) == "string" then delete(S, x) 5. True or false: Abreak expression can generate a sequence of values. return S end 6. What does the following expression do? 17. set(R), set(S), set(T), and list(S) cause run-time i := 1 to 5 errors. table(S) creates a table whose default value 7. True or false: The number of results thate1 | e2 is S. can generate is the sum of the number of results
The Icon Analyst 53 / 19 that e1 and e2 can generate separately. 11. What do the following expressions do? 8. True or false:!x can produce an unlimited num- every write((1 to 5) | (5 to 1 by – 1)) ber of results. every write((1 to 5) + (5 to 1 by – 1)) 9. True or false:|?x always produces an unlimited every write((1 to 5) > (5 to 1 by – 1)) number of results. 12. What do the following expressions do? 10. What do the following expressions do? every write(seq() \ 9) write(every (1 to 5) & 7) every write((1 to 20) \ 9) every write((1 to 5) & 7) every write(seq() \ (9 | 3)) every write(seq() \ (1 to 5)) every write((seq() \ 9) \ 7) every write((seq() \ ((1 to 3) | (3 to 1 by –1)))) 13. What are the consequences of evaluating the The Icon Analyst following expressions?
Ralph E. Griswold, Madge T. Griswold, |(2 = 3) and Gregg M. Townsend |2 = 3 Editors 2 = |3 |2 = |3 The Icon Analyst is published six times a year. A one-year subscription is $25 in the United States, Canada, and Mexico and $35 elsewhere. To subscribe, contact Icon Project Department of Computer Science The University of Arizona P.O. Box 210077 Tucson, Arizona 85721-0077 U.S.A.
voice: (520) 621-6613 fax: (520) 621-4246
Electronic mail may be sent to: [email protected]
What’s Coming Up
Machinery is aggressive. The weaver becomes a web, the and machinist a machine. If you do not use the tools, they use Bright Forest Publishers you. — Ralph Waldo Emerson Tucson Arizona In the next issue of the Analyst, we plan to have a weaving case study, an application for discovering interesting tiles in images, an article on animation by image replacement, and another article on generators and sequences. © 1999 by Ralph E. Griswold, Madge T. Griswold, and Gregg M. Townsend We’ll also have answers to the quiz on expres- sion evaluation and another quiz, probably on All rights reserved. Icon’s preprocessor facility.
20 / The Icon Analyst 53