ARRAY SET ADDRESSING: ENABLING EFFICIENT HEXAGONALLY SAMPLED IMAGE PROCESSING

By

NICHOLAS I. RUMMELT

A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY

UNIVERSITY OF FLORIDA

2010

1 °c 2010 Nicholas I. Rummelt

2 To my beautiful wife and our three wonderful children

3 ACKNOWLEDGMENTS Thanks go out to my family for their support, understanding, and encouragement. I especially want to thank my advisor and committee chair, Joseph N. Wilson, for his keen insight, encouragement, and excellent guidance. I would also like to thank the other members of my committee: Paul Gader, Arunava Banerjee, Jeffery Ho, and Warren Dixon. I would like to thank the Air Force Research Laboratory (AFRL) for generously providing the opportunity, time, and funding. There are many people at AFRL that played some role in my success in this endeavor to whom I owe a debt of gratitude. I would like to specifically thank T.J. Klausutis, Ric Wehling, James Moore, Buddy Goldsmith, Clark

Furlong, David Gray, Paul McCarley, Jimmy Touma, Tony Thompson, Marta Fackler, Mike Miller, Rob Murphy, John Pletcher, and Bob Sierakowski.

4 TABLE OF CONTENTS page ACKNOWLEDGMENTS ...... 4

LIST OF TABLES ...... 7

LIST OF FIGURES ...... 8

ABSTRACT ...... 10

CHAPTER 1 INTRODUCTION AND BACKGROUND ...... 11 1.1 Introduction ...... 11 1.2 Background ...... 11 1.3 Recent Related Research ...... 15 1.4 Recent Related Academic Research ...... 16 1.5 Hexagonal Image Formation and Display Considerations ...... 17 1.5.1 Converting from Rectangularly Sampled Images ...... 17 1.5.2 Hexagonal Imagers ...... 19 1.5.3 Displaying Hexagonal Images ...... 21

2 ARRAY SET ADDRESSING (ASA) ...... 24 2.1 Forming an Array Set from a Hexagonal Grid ...... 24 2.2 Neighborhoods ...... 26 2.3 Forms of Array Sets ...... 26 2.4 Efficient and Compact Storage in Digital Memory ...... 27 2.5 Converting Between Cartesian Coordinates and ASA ...... 28 2.6 Distance Measures Using ASA ...... 34 2.6.1 Euclidean Distance ...... 34 2.6.2 City-Block Distance ...... 34 2.7 Vector Arithmetic ...... 36 2.7.1 Proof of the Associativity of Addition ...... 37 2.7.2 Proof of Distributivity of Scalar Multiplication (w.r.t. Scalar Addition) 39 2.7.3 Proof of Compatibility of Scalar Multiplication (with Multiplication of Scalars) ...... 41 2.8 Comparison of ASA to Spiral Addressing ...... 42 3 BASIC IMAGE PROCESSING OPERATIONS ...... 44

3.1 Convolution ...... 44 3.2 Gradient Estimation ...... 46 3.3 Edge Detection ...... 51 3.3.1 Gradient-Based Edge Detector ...... 51 3.3.2 Laplacian of Gaussian (LoG) Edge Detector ...... 52

5 3.3.3 Canny Edge Detector ...... 57 3.4 Downsampling ...... 66

4 TRANSFORMS ...... 68

4.1 Fourier Transforms ...... 68 4.1.1 The Hexagonal Discrete Fourier Transform (HDFT) ...... 68 4.1.2 The Hexagonal Fast Fourier Transform (HFFT) ...... 69 4.1.3 The Hexagonal Discrete-Space Fourier Transform (HDSFT) .... 79 4.2 The Z Transform ...... 80 4.3 Wavelet Transforms ...... 81 5 QUANTITATIVE COMPARISON USING REAL IMAGES ...... 89

6 CONCLUSION ...... 92 6.1 Summary of Accomplishments ...... 92 6.2 Significance of Accomplishments ...... 94 6.3 Final Thought - Rectangular Component Conjecture ...... 94

APPENDIX

A PROOFS OF ASA Z-MODULE PROPERTIES ...... 96

A.1 Preliminaries ...... 96 A.1.1 Domains ...... 96 A.1.2 Identities ...... 96 A.2 Commutativity of Addition ...... 96 A.3 Identity Element of Addition ...... 97 A.4 Inverse Elements of Addition ...... 97 A.5 Distributivity of Scalar Multiplication (w.r.t. Vector Addition) ...... 97 A.6 Identity Element of Scalar Multiplication ...... 99 B PYTHON SOURCE CODE FOR ASA TO HIP RUNTIME COMPARISON .... 100 REFERENCES ...... 108 BIOGRAPHICAL SKETCH ...... 113

6 LIST OF TABLES Table page 2-1 Forms of Array Sets ...... 27

2-2 Average Number of Operations Executed (Yang vs. Hybrid) ...... 29

2-3 Properties of Array Set Addressing (ASA) Vectors ...... 37

2-4 Run Time Comparison Between ASA and HIP (µs, mean (std)) ...... 43

2-5 Complexity Comparison Between ASA and HIP ...... 43 4-1 Computations Required for Hexagonal Fast Fourier Transform Arrays ..... 74

7 LIST OF FIGURES Figure page 1-1 Retesselating a Source Image ...... 18

1-2 Example of Converting a Source Image ...... 19

1-3 Trapezoidal Image Sensor Design ...... 20

1-4 Depiction of “Brick Wall” Display Technique ...... 21

1-5 Four Common Displays and Two Potential Hexagonal Red-Green-Blue (RGB) Displays ...... 23 2-1 Hexagonal Grid Spacing ...... 25 2-2 Example of Separating the Hexagonal Grid into an Array Set ...... 25

2-3 Determining the Neighbors of a Given Pixel ...... 26

2-4 Storage of Array Set Addressing (ASA) Addresses ...... 28

2-5 Representation of xs -ys Space ...... 30 2-6 Resolving the Boundary Case ...... 31 3-1 ASA Convolution Example ...... 45 3-2 Comparison of Nearest Neighbor Neighborhood Sizes ...... 45 3-3 Example of Fitting a Plane to a Hexagonal Neighborhood ...... 47 3-4 Hexagonal Neighborhood Indexing ...... 47

3-5 Convolution Masks for Local Gradient Estimation ...... 50

3-6 Example 1 of Gradient-Based Edge Detection ...... 53 3-7 Example 2 of Gradient-Based Edge Detection ...... 54 3-8 Example 3 of Gradient-Based Edge Detection ...... 55

3-9 Example 1 of Laplacian of Gaussian (LoG)-Based Edge Detection ...... 58 3-10 Example 2 of LoG-Based Edge Detection ...... 59

3-11 Example 3 of LoG-Based Edge Detection ...... 60 3-12 Quantization Directions of Canny Edge Detector ...... 62

3-13 Example 1 of Canny Edge Detection ...... 63

3-14 Example 2 of Canny Edge Detection ...... 64

8 3-15 Example 3 of Canny Edge Detection ...... 65 3-16 Downsampling with ASA ...... 67

4-1 Standard Fast Fourier Transform (FFT) Butterfly Diagram - Full Computation . 75

4-2 Standard FFT Butterfly Diagram - Reduced Computation ...... 75 4-3 Non-Standard Transform 1 (NST1) Butterfly Diagram - Full Computation .... 76

4-4 NST1 Butterfly Diagram - Reduced Computation ...... 76 4-5 Non-Standard Transform 2 (NST2) Butterfly Diagram - Full Computation .... 77 4-6 NST2 Butterfly Diagram - Reduced Computation ...... 77

4-7 Non-Standard Transform 3 (NST3) Butterfly Diagram - Full Computation .... 78 4-8 NST3 Butterfly Diagram - Reduced Computation ...... 78

4-9 Idealized Wavelet Transform Regions of Support (Hexagonal vs. Rectangular) 83

4-10 Wavelet Transform Analysis Filter Bank ...... 84 4-11 Wavelet Transform Synthesis Filter Bank ...... 85

4-12 Lilies Image (Hexagonally Sampled) ...... 86 4-13 Hexagonal Discrete Wavelet Transform of Lilies Image ...... 87 5-1 Trapezoidal Imager ...... 89 5-2 Example of a Radial Sine Wave (Left) and its Fourier Transform (Right) .... 90 5-3 Experimental Results ...... 91

9 Abstract of Dissertation Presented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy ARRAY SET ADDRESSING: ENABLING EFFICIENT HEXAGONALLY SAMPLED IMAGE PROCESSING By Nicholas I. Rummelt December 2010 Chair: Joseph N. Wilson Major: Computer Engineering

It has long been known that there are numerous advantages to sampling images hexagonally rather than rectangularly. However, due to various shortcomings of the addressing schemes, hexagonal sampling for digital images has not been embraced by the mainstream digital imaging community. The idea of using hexagonal sampling for digital imaging applications has been around since the early 1960s, yet no efficient addressing method for hexagonal grids has been developed in that time. This disser- tation introduces a new hexagonal addressing technique called array set addressing (ASA), that solves the problems exhibited by other methods. This new approach uses three coordinates to represent the hexagonal grid as a pair of rectangular arrays. This representation supports efficient linear algebra and image processing manipulation. ASA-based implementations of several basic image processing operations are developed and shown to be efficient. A hexagonal fast Fourier transform (HFFT), based on the fact that the Fourier kernel becomes separable when using ASA coordinates, is also developed. An ASA implementation of the hexagonal discrete wavelet transform (DWT), which enjoys several advantages over the rectangular DWT due to the superior angular resolution and symmetry of the hexagonal grid, is demonstrated and shown to be efficient. The results of an experiment that quantifies and demonstrates the superior bandwidth of hexagonal sampling using real sensor hardware are presented.

10 CHAPTER 1 INTRODUCTION AND BACKGROUND

1.1 Introduction

The intent of this effort is to introduce a novel addressing approach for the hexagonal grid, called array set addressing (ASA), and show how it can be used to efficiently realize the benefits of hexagonal sampling for digital imaging. It has been known for decades that there are several significant advantages to sampling images hexagonally, but despite various attempts, no one has managed to devise an efficient addressing scheme for the hexagonal grid. It has also long been known that the hexagonal grid can be separated into two array components, but surprisingly, this fact has never been exploited for addressing. This dissertation introduces the ASA addressing technique along with the mathematics necessary to perform coordinate operations and develops basic image processing methods using ASA, showing them to be efficient. The hexagonal discrete Fourier transform (HDFT) is converted to ASA coordinates and shown to have an efficient algorithm for its computation, based on the fact that the Fourier kernel becomes separable in ASA coordinates. The discrete wavelet transform (DWT) for hexagonally sampled images is also implemented in ASA coordinates and shown to be efficient. Lastly, an experiment is conducted that demonstrates the bandwidth advantage of hexagonal sampling using a unique camera that has the ability to sample both hexagonally and rectangularly.

1.2 Background

There are only three regular shapes that can be individually used to tile a two- dimensional plane without gaps or overlapping, forming a regular tessellation of the plane. Those three shapes are triangles, squares, and hexagons. Each shape leads to a different sampling grid for sampling two-dimensional signals, such as images. It is well known that the compound eyes of arthropods consist of a roughly hexagonal arrangement of ommatidia and the arrangement of cones in the human foveal region of

11 the retina is similarly hexagonal [1], [2]. Curcio [2] claims that the packing geometry of the cones contributes to the functional grain of the primate retina.

Research into the sampling of two-dimensional signals began appearing in the literature during the late 1950s. In 1962, Petersen and Middleton [3] developed a multi-dimensional sampling theory and showed that a regular hexagonal grid is the optimal sampling lattice for two-dimensional isotropically band-limited signals (an isotropically band-limited signal is one whose Fourier transform is zero outside of a circular bandregion). Around that same time, the ILLIAC III computer was being built at the University of Illinois to scan and process images. The design allowed the programmer to choose between rectangular and hexagonal sampling [4]. In the late 1960s, Golay [5] showed the “logical difficulty” that arises from the connectivity ambiguity in square sampled images when performing morphological operations and showed the “greater simplicity” of performing these operations on hexagonally sampled images. The connectivity ambiguity arises from the fact that four of the nearest neighbor pixels share a side and the other four share a corner, resulting in the need to specify 4-way or 8-way connectedness. The hexagonal grid has no such ambiguity since all nearest neighbor pixels share a side. Golay also pointed out the greater angular resolution the hexagonal grid has over rectangular ones. Shortly after that,

Deutsch [6] experimentally compared the performance of thinning algorithms on rectangular, hexagonal, and triangular arrays. He determined that the hexagonal array offered the best overall performance, due largely to its requiring substantially less processing time. Despite all those results and the biological examples, virtually all two- dimensional signal processing systems of the time were using rectangular sampling. The reasons for this are varied: the processing algorithms could be generalized from the one-dimensional case straightforwardly; those algorithms were easily understood and implemented; and it was easier to build hardware that sampled rectangularly [7]. In 1979, Mersereau authored a seminal paper providing an introductory theory to

12 the processing of hexagonally sampled images [7]. He confirmed that the regular hexagonal grid is the optimal sampling lattice for two-dimensional isotropically band- limited signals. He also developed the hexagonal discrete Fourier transform (HDFT) but found the Fourier kernel to be non-separable in his representation. Lastly, he showed that recursive systems for hexagonal arrays can be developed and that FIR filters can be easily designed and implemented. He concluded with a claim that hexagonal sampling provides a savings ranging from 13 to 50 percent in both computation and storage over rectangular sampling for many two-dimensional signal processing applications.

One of the nice things about rectangular sampling is that it produces a simple array that is easy to store and process. This fact almost certainly contributed to rectangular sampling being used over hexagonal sampling despite the theoretical work that showed the superiority of hexagonal sampling. The hexagonal grid cannot be indexed as a simple array and requires a special addressing method. In 1980, Burt developed sept-tree and pyramidal data structures for processing binary hexagonally sampled images [8]. According to van Roessel [9], Lucas and Gibson introduced the General Balanced Ternary (GBT) addressing approach in an unpublished document in 1970. However, that date is in question. The same document [10] is cited by Gibson herself in a final report for an Air Force Office of Scientific Research (AFOSR) contracted effort [11] and is dated 1980 in that citation. Regardless, GBT is a hierarchical addressing method that can be used for n-dimensional tessellations of Euclidean spaces. In two dimensions, it can be used to address the hexagonal grid. The two-dimensional form of GBT is the only one being considered here. GBT is based on a hierarchy of cells, where at each level the cells are aggregates of seven cells from the previous level, with the lowest level being the hexagons tiling the plane [11]. Each hexagon is addressed with a string of base-7 digits where each digit indicates the hexagon’s position within that level of the hierarchy. There are also slightly modified versions of GBT in the literature such as HIP, described in Middleton’s Hexagonal

13 Image Processing book [12] and spiral architecture, described in Sheridan’s Ph.D. dissertation [13]. Crettez introduced the same addressing scheme for a hexagonal

grid in 1980 [1] and called the arrangement of “heptuplets” a “heptarchy”. Note that all

of these GBT-based approaches will be collectively referred to as spiral addressing. Alternatively, non-orthogonal axes can be used to address a hexagonal grid. There are several variations of this approach due to the different possible orientations of the oblique axes. A description of the variations and citations of the researchers that have used each can be found in Middleton’s book [12]. Another addressing approach

was introduced in the early 1990s by Her [14]. Her’s approach is an interesting three coordinate system that places the image on an oblique plane in three dimensions, thereby preserving the natural symmetry of the hexagonal grid.

A coordinate system to represent the points of an image should have a compact and efficient machine representation, be able to use the techniques of linear algebra, and be able to take advantage of the tools that have been developed for processing rectangularly sampled images. None of the approaches described in the literature to date, namely spiral addressing approaches, variants of oblique addressing, and Her’s

approach, possess all of these properties. Snyder et al. [15] describe and analyze a typical oblique addressing approach, which is referred to as the h2 system. Such a system of oblique coordinates usually introduces negative indices in order to have a non-oblique (rectangular) image. The h2 system avoids negative indices by keeping the image an appropriate distance away from the coordinate origin, however that introduces the need to keep track of the offset. In an attempt to demonstrate efficient memory allocation, the h2 system introduces an additional set of row and column coordinates to address the pixels through an array of pointers to each row. However, the row and column coordinates do not directly map to the h2 coordinates, making the overall system cumbersome for image processing. The approach introduced by

Her [14] places the image on an oblique plane in three dimensions. The constraint that

14 the coordinates must always sum to zero makes avoiding negative indices impossible. Although implementation details are not provided in Her’s paper, a direct implementation would seem to require a data structure that includes three image-sized arrays to hold the coordinate values, as well as one to hold the image intensity values. It is apparent that both of these approaches require cumbersome machine representations that lead to inefficient image processing operations. Spiral addressing approaches have some interesting mathematical properties, but fail to be convenient or efficient for image processing due to various shortfalls such as non-rectangular images, rotations caused by downsampling, and special addition and multiplication requirements.

1.3 Recent Related Research

In the past few years, several papers related to processing hexagonally sampled images have been published, demonstrating that research in this area continues despite the lack of an efficient addressing scheme. He’s group published several papers [16]–[19] on topics such as image translation and rotation, uniform partitioning, sampling grid conversions, and image analysis. These were all based on using hexagonally sampled images addressed with spiral addressing. Condat et al. published a series of papers [20]–[22] on spline functions for hexagonally sampled data. The spline functions represent the discrete data in a continuous function that can be resampled to perform interpolation, grid conversion, etc. They published another series of papers [23]–[25] on reversible operations that can be performed via 1-D filtering to apply a sequence of shearing operations. They demonstrate reversible grid conversion and reversible image rotation. This author also published a paper [26] that introduces a family of wavelets on the hexagonal grid. Jiang published several papers [27]–[29] about the design of filter banks for wavelets on hexagonal grids. Faille and Petrou published a paper [30] about using splines to reconstruct images from irregularly placed samples on hexagonal grids. Gardiner et al. published a couple of papers [31], [32] on designing gradient operators for hexagonally sampled images. Puschel¨ and Rotteler¨ applied

15 their algebraic signal processing theory to hexagonal images demonstrating spectral transforms and fast transform algorithms [33], [34]. And lastly, Vince and Zheng showed

how to compute the discrete Fourier transform (DFT) using group theory and GBT [35].

1.4 Recent Related Academic Research

Yang used a hexagonal grid while examining spatial smoothing methodology

variations for his dissertation [36]. The dissertation does not discuss the addressing scheme, but rather uses a hexagonal grid to develop a fast binning algorithm and

subsequent smoothing operations. He generally uses the three principal lattice lines and performs smoothing along those lattice lines. The binning algorithm converts arbitrary Cartesian coordinates to the nearest hexagonal grid point. This is also the goal of a Cartesian to ASA conversion algorithm. Therefore, the Yang binnning algorithm was implemented and compared to the original algorithm developed for ASA with inconclusive results: one ran faster in Matlab while the other ran faster with Python. Before a formal analysis was completed, it was realized that a hybrid version could be faster than both of the original algorithms. Indeed, that turns out to be the case and the hybrid version has been adopted as the Cartesian to ASA conversion algorithm.

Refer to Section 2.5 for additional details. Barbour uses a hexagonal grid in part of his thesis [37] on path finding. He addresses the hexagonal grid using standard row and column addressing where one of the coordinate axes follows a sawtooth path rather than a straight line. This is not a useful addressing scheme for digital image processing. Boyd

also uses the same scheme in his thesis [38] but refers to it as a tex grid. There is some ambiguity with that name because Barbour also used a structure that he referred to as a tex grid, but it was described as being square tiles arranged in a hexagonal pattern, similar to the brick wall structure described in Section 1.5.3 and shown in Figure 1-4.

Guenette uses a hierarchical hexagonal grid structure for global mapping techniques in his thesis [39]. This structure is indexed similarly to the spiral addressing techniques,

16 but is being used to tile a sphere rather than a plane. Zhou [40] and Zheng [41] both use hexagonal grids and oblique addressing in their dissertations.

1.5 Hexagonal Image Formation and Display Considerations

There are three main technology hurdles to the use of hexagonal sampling for digital imaging applications:

1. The lack of an efficient addressing scheme 2. The lack of imagers that sample hexagonally 3. The lack of displays for hexagonally sampled images

The main portion of this document introduces an approach that solves item 1). The following sections will discuss solutions to items 2) and 3). It is important to note that these are not the only hurdles to the use of hexagonal sampling. This technology now faces a culture of rectangular sampling that has existed for decades. Even if it can be shown to be superior, it may be difficult to gain a foothold in the marketplace and breakout of the existing culture. As an example, consider the battle between the VHS and Betamax video tape formats. Betamax is considered by many to be a superior format, but for various reasons the VHS format won out in the marketplace after a roughly ten year battle. For hexagonal sampling, the battle may be even more difficult since rectangular sampling is already so firmly established in digital imaging. 1.5.1 Converting from Rectangularly Sampled Images

As an alternative to imagers that sample hexagonally, it is possible to convert high resolution rectangularly sampled images into ones that approximate hexagonally sampled images. Additionally, one can convert the same high resolution images to rectangularly sampled ones that are roughly equivalent to the hexagonally sampled version, allowing direct comparisons of the two sampling techniques. The requirement for high resolution source images comes from the fact that the conversion process significantly lowers the image resolution. The lowering of the image resolution causes

17 this method to be unacceptable for commercial digital imaging applications, but it is deemed acceptable for research use.

It is assumed that the source image was sampled with unit area square sampling.

The source image is then resampled to convert unit square pixels into appropriate rectangles. The image is retesselated by conglomerating 36 pixels into the appropriate shape (hexagon or square) as shown in Figure 1-1. An example of the results of this process is shown in Figure 1-2.

Figure 1-1. Retesselating a source image to produce roughly equivalent hexagonally sampled and rectangularly sampled images.

18 Figure 1-2. Example of converting a source image into equivalent hexagonally and rectangularly sampled images.

1.5.2 Hexagonal Imagers

There have been a few attempts to build imagers that sample hexagonally such as Carver Mead’s Silicon Retina [42] and the prototype described in Hauschild’s paper [43], but there are no commercially available hexagonally sampling imagers. There is no physical reason why it can’t be done, rather a lack of motivation to do so, arguably due to the absence of an efficient addressing scheme.

One of the main goals of this research effort is to concretely show that the advantages of hexagonal sampling can be realized with little or no more processing complexity than is needed for processing equivalent rectangularly sampled images. To that end, a trapezoidal imager has been developed that allows simultaneous hexagonal and rectangular sampling of a scene. By using appropriate trapezoidal unit cells, the units can be combined to form either hexagonal or rectangular pixels. This produces two images for each frame, where the only difference between the two is the sampling method.

Figure 1-3 shows the pixel layout of a portion of the trapezoidal image sensor. The bright green areas are the photosensitive areas, approximating trapezoids. The size of

19 the horizontal side of the trapezoid is 18µ. Notice how in this example the trapezoidal units can be combined to create seven hexagonal pixels (center surrounded by six neighbors) or six rectangular pixels in 2 × 3 format (disregarding the top and bottom rows of trapezoid unit cells). The full imager will have 512 × 304 trapezoidal unit cells, producing a 255 × 152 hexagonally sampled image and a 256 × 151 rectangularly sampled image, when appropriately combined.

Figure 1-3. Part of the trapezoidal image sensor design.

20 1.5.3 Displaying Hexagonal Images

There are no known commercial displays that operate in the hexagonal domain. For machine vision applications, this is a minor or even non-existent problem. Applications requiring the highest possible display resolution may find this hurdle currently limits the utility of sampling hexagonally. Regardless, for most digital imaging applications, the

display hurdle can be overcome. One option is the use of the “brick wall” technique [12], [44], which uses four square pixels per hexagonal pixel to display the hexagonally

sampled image. This results in a factor of four reduction in the resolution and displays the hexagons as rectangles, but produces qualitatively good results. The brick wall

technique works by using a 2 × 2 block of square pixels to represent each hexagon with

every other row shifted by one pixel as depicted in Figure 1-4.

Figure 1-4. Depiction of “brick wall” display technique.

The left side of Figure 1-5 shows an image [45] of some common color (RGB) display geometries. The two upper images are of cathode ray tubes (CRT), a display technology that is seemingly becoming obsolete. It is interesting, however, to note the hexagonal arrangement of the pixels in these images. The two lower images are of liquid crystal displays (LCDs). The XO-1 LCD could employ the brick wall technique on

the sub-pixels creating a pseudo-hexagonal RGB display, but the color distribution per

pixel would be mismatched, and would need to be corrected. The standard LCD design

21 is shown in the lower right quadrant of the image. One can imagine using horizontal line sub-pixels or reshaping the sub-pixels of the standard LCD as in the right side of

Figure 1-5 to create a hexagonal RGB display. Note that the scope of this effort is limited to grayscale images, though it would be straightforward to extend ASA addressing to color images.

22 23

Figure 1-5. Four common display geometries (left) [45] and two potential hexagonal RGB display geometries (right). CHAPTER 2 ARRAY SET ADDRESSING (ASA)

2.1 Forming an Array Set from a Hexagonal Grid

An image exists as a 2-D representation of a 3-D scene. As a 2-D object (a plane), any point in the image can be identified using a set of two Cartesian coordinates. For digital images, the image is usually sampled using a square grid with the pixels formed from the Voronoi cell of the grid. Each pixel is represented by its grid point (center point of the Voronoi cell) whose Cartesian coordinates are normalized to produce an integer indexing. This results in the familiar coordinates of rows and columns indexing a rectangular array. If, however, the image plane is sampled using a regular hexagonal grid, the Voronoi cells (and hence the pixels) are hexagons. In this case, the Cartesian coordinates of the grid points can no longer be normalized to produce integers. This is √ due to the requirement that the vertical and horizontal spacings have a ratio of 3/2 as shown in Figure 2-1. An integer indexing could be achieved if the requirement that both axes use the same unit of measure were removed, however, the resulting array would resemble a checkerboard since some indices would not correspond to image locations.

The ASA approach is based on representing the hexagonal grid as two rectangular arrays which can be individually indexed by integer-valued row and column indices. The hexagonal grid is simply separated into rectangular arrays by taking every other row as one array and the remaining rows as the other array, as the example in Figure 2-2 demonstrates. The arrays are distinguished using a single binary coordinate so that a full address for any point on the hexagonal grid is uniquely represented by three coordinates

(a, r, c) ∈ {0, 1} × Z × Z (2–1) where the coordinates represent the array, row, and column respectively. Note that this domain is valid for a hexagonal tiling of the entire infinite plane. Hexagonal images, however, will typically be restricted to having non-negative row and column indices.

24 Figure 2-1. The spacing is important to maintaining the natural symmetry of the hexagonal grid.

Figure 2-2. Example of how the hexagonal grid is separated into two arrays using ASA.

25 2.2 Neighborhoods

Consider the task of finding all of the immediate neighbors of a given pixel. For rectangularly sampled images, this task highlights the connectedness ambiguity discussed in Section 1.2. It must be specified whether the immediate neighbors are the four that share a side with the given pixel or all eight that are adjacent. Regardless, their addresses can easily be calculated by a simple ±1 offset from the given pixel’s coordinates. For any of the spiral addressing approaches, this task seems to require a simple addition for each of the six neighbors, however, the special mathematics defined for these approaches requires a look-up table operation for each pair of digits being added with the possibility of double-digit carries, leading to a computational complexity of O((log N)2), where N is the number of pixels in the image. Using ASA, there is no ambiguity and no special mathematics are required. The neighbors of a given pixel can be calculated by simple offsets as shown in Figure 2-3.

Figure 2-3. The addresses of the nearest neighbors of pixel (a, r, c) are easily determined using these expressions.

2.3 Forms of Array Sets

Array sets can take on several forms so it helps to establish some conventions.

Table 2-1 lists the nine possible forms of an array set given that the origin (in Cartesian coordinates) is always at ASA address (0, 0, 0). Refer to Section 2.5 for an explanation of the B1 designation in Table 2-1. Working with a single form for images simplifies

26 implementation, so all images will be assumed to be form 1. Any other form can be converted to form 1 by removing at most 1 row and 1/2 column. However, convolution masks (filters) must necessarily have arbitrary form. Table 2-1. Forms of Array Sets Cartesian Cartesian Coordinates (B1) Coordinates (B1) Form 0-Array Size of (0,0,0) 1-Array Size of (1,0,0) ³ √ ´ 1 3 1 RxC (0,0) RxC 2 , 2 ³ √ ´ 1 3 2 RxC (0,0) R-1xC 2 , 2 ³ √ ´ 1 3 3 RxC (0,0) R-1xC-1 2 , 2 ³ √ ´ 1 3 4 RxC (0,0) RxC-1 2 , 2 ³ √ ´ 1 3 5 RxC (0,0) RxC+1 − 2 , 2 ³ √ ´ 1 3 6 RxC (0,0) R-1xC+1 − 2 , 2 ³ √ ´ 1 3 7 RxC (0,0) R+1xC 2 , − 2 ³ √ ´ 1 3 8 RxC (0,0) R+1xC-1 2 , − 2 ³ √ ´ 1 3 9 RxC (0,0) R+1xC+1 − 2 , − 2

2.4 Efficient and Compact Storage in Digital Memory

The ASA data structure can be stored in digital memory in a contiguous, linear fashion by constraining the number of columns to be a power of 2. The number of rows remains unconstrained allowing for any image aspect ratio. For an N × 2j image, the § ¡ ¢¨ N number of bits required for the column index is j and for the row index is m = log2 2 with the array index requiring 1 bit. Assuming a 32 bit address space, the base address is then 32 − (j + m + 1) bits. Arranging the bits in base+row+array+column order yields a row-major order storage scheme as illustrated in Figure 2-4. The image aspect ratio in √ j 3 Cartesian space is 2 :( 2 )N. For example, a 443 × 512 format image would have an approximately 4 : 3 aspect ratio (in Cartesian space) and the memory addresses would be of the form XXXXXXXXXXXXXXRRRRRRRRACCCCCCCCC, with each memory

location holding the corresponding pixel’s value. Reading the data in a sequential

manner provides a progressive scan sweep of the data.

27 Figure 2-4. Illustration of row-major order storage.

2.5 Converting Between Cartesian Coordinates and ASA

A regular hexagonal lattice in Cartesian space can be characterized by two linearly

independent basis vectors, r1 and r2, with the following three constraints:

1. kr1k = kr2k, ◦ ◦ ◦ ◦ ◦ ◦ 2. r1 and r2 each form a 0 , 60 , 120 , 180 , 240 , or 330 angle with the x-axis, ◦ ◦ 3. the angle between r1 and r2 is not 0 or 180 .

Let d = kr1k = kr2k and A = {{0, 1} × Z × Z} and C = {R × R}. Starting with the conversion from ASA to Cartesian, the function fd : A 7→ C is a simple matrix multiplication. Let    a      w =  r  ∈ A   c

and   d 0 d  2  Bd =  ³ √ ´ √  3 (d) 2 (d)( 3) 0

28 where d scales the hexagonal lattice and is the Euclidean distance between any two adjacent grid points on the hexagonal lattice. Then     ¡ ¢ a  (d) 2 + c   x  fd (w) = Bd w =  ¡√ ¢ ¡ ¢  =   ∈ C. (2–2) a (d) 3 2 + r y

Converting from Cartesian to ASA is a little more involved but still very efficient. The

function gd : C 7→ A is defined by a process. This process is partially inspired from Yang’s fast binning algorithm [36]. It was determined that a hybrid algorithm using Yang’s fast binning algorithm and the original Cartesian to ASA conversion algorithm could be faster than either original algorithm. Although all have O(N) complexity, an analysis of the average number of operations performed by the algorithms was completed on Yang’s binning algorithm and the hybrid version. The portions of the algorithms that are equivalent (scaling the coordinates, round/floor operations, and converting coordinates to ASA) were not included in the analysis. Note that the floor() and round() functions are assumed to be equivalent in terms of run time. The results are shown in Table 2-2. The mean run times (10,000 runs) for Yang’s fast binning algorithm, the ASA original Cartesian to ASA algorithm, and the hybrid Cartesian to ASA algorithm using the code

provided in Appendix B were 7.6 µs, 5.6 µs, and 4.5 µs, respectively.

Table 2-2. Average Number of Operations Executed (Yang vs. Hybrid) Algorithm Comparisons Add/subtract Multiply Yang 2 6 1 Hybrid 2.5 2.5 0.5

The hybrid algorithm proceeds as follows. First, the x and y dimensions of the Cartesian space are scaled as 2 x = x s d 2 ys = √ y d 3

29 to create a space where the hexagonal grid points fall on integer coordinates. The

coordinates (xs , ys ) ∈ C are then rounded to the nearest integer, producing (xr , yr ).

Figure 2-5 shows a representation of this space where (xs , ys ) coordinates that fall within the green boxes have been rounded to the hexagon centers and the ones that fall within the red boxes have been rounded to the boundary between two hexagons. Notice

Figure 2-5. Representation of xs -ys space.

that it is easy to distinguish between these two cases by checking if xr and yr have the

same evenness or oddness. If they are the same, then (xr , yr ) is a hexagon center and

(xs , ys ) is in that hexagon, otherwise it is a boundary. An additional step is necessary for the boundary case to determine which of the four surrounding hexagons contains

the original point. Referring to Figure 2-6, it is easy to determine which quadrant (I, II,

III, or IV) (xs , ys ) falls in by comparing xs to xr and ys to yr . Once the quadrant is known, it is a matter of determining whether (xs , ys ) lies above or below the line dividing that quadrant. Due to the geometry of the space, the slope of the dividing line is always

1 known and is ± 3 , depending on the quadrant. A point on the line is also always known 1 and is (xr , yr ± 3 ), again depending on the quadrant. Using the slope and the point to calculate the equation of the line, it is easy to determine which hexagon contains the original point and offset (xr , yr ) appropriately. Lastly, convert (xr , yr ) to ASA coordinates.

30 Figure 2-6. Illustration of how the boundary case is resolved.

This method for converting a set of Cartesian coordinates, (x, y), to ASA coordinates,

(a, r, c), given d, is provided in pseudo-code below. xs = 2*x/d ys = 2*y/(d*sqrt(3)) xr = round(xs) yr = round(ys) m = 1/3 if (xr MOD 2) XOR (yr MOD 2) \\Boundary Case if xs >= xr if ys > yr \\Quadrant I if (ys - (yr + 1/3) - m*(xs - xr) <= 0) xr = xr + 1 \\Below Line else yr = yr + 1 \\Above Line else \\Quadrant II if (ys - (yr - 1/3) + m*(xs - xr) >= 0) xr = xr + 1 \\Above Line else

31 yr = yr - 1 \\Below Line else if ys > yr \\Quadrant IV if (ys - (yr + 1/3) + m*(xs - xr) <= 0) xr = xr - 1 \\Below Line else yr = yr + 1 \\Above Line else \\Quadrant III if (ys - (yr - 1/3) - m*(xs - xr) >= 0) xr = xr - 1 \\Above Line else yr = yr - 1 \\Below Line

// Convert to ASA Coordinates a = yr MOD 2 r = (yr-a)/2 c = (xr-a)/2

This method has also been implemented in MIPS32 assembly language to demonstrate the efficiency of working with ASA in the digital domain. The following assumes the necessary data is in memory starting at address 0x10010000. The

results, (a, r, c) will be written to memory addresses 0x10010014, 0x10010018, and 0x1001001c, respectively. Although, there are 65 lines of code, the algorithm runs less than 30 lines of code on average.

addi $1, $0, 0x10010000 lwc1 $f2, 0($1) # Load constant value 1/3 lwc1 $f4, 4($1) # Load constant value 2/d lwc1 $f6, 8($1) # Load constant value 2/(d*sqrt(3)) lwc1 $f8, 12($1) # Load x lwc1 $f10, 16($1) # Load y mul.s $f8, $f8, $f4 mul.s $f10, $f10, $f6 round.w.s $f12, $f8 round.w.s $f14, $f10 mfc1 $2, $f12 mfc1 $3, $f14 andi $4, $2, 1 andi $5, $3, 1 xor $6, $4, $5 beq $6, $0, Convert

32 cvt.s.w $f12, $f12 cvt.s.w $f14, $f14 c.lt.s $f8, $f12 bc1f Q1or2 c.le.s $f10, $f14 bc1f Q4 sub.s $f16, $f8, $f12 # Quadrant 3 mul.s $f16, $f16, $f2 sub.s $f18, $f14, $f2 sub.s $f18, $f10, $f18 sub.s $f16, $f18, $f16 c.lt.s $f16, $f0 bc1f xrm1 yrm1: addi $3, $3, -1 b Convert Q4: sub.s $f16, $f8, $f12 # Quadrant 4 mul.s $f16, $f16, $f2 add.s $f18, $f14, $f2 sub.s $f18, $f10, $f18 add.s $f16, $f18, $f16 c.le.s $f16, $f0 bc1f yrp1 xrm1: addi $2, $2, -1 b Convert Q1or2: c.le.s $f10, $f14 # Quadrant 1 or 2 bc1f Q1 Q2: sub.s $f16, $f8, $f12 # Quadrant 2 mul.s $f16, $f16, $f2 sub.s $f18, $f14, $f2 sub.s $f18, $f10, $f18 add.s $f16, $f18, $f16 c.lt.s $f16, $f0 bc1t yrm1 xrp1: addi $2, $2, 1 b Convert Q1: sub.s $f16, $f8, $f12 # Quadrant 1 mul.s $f16, $f16, $f2 add.s $f18, $f14, $f2 sub.s $f18, $f10, $f18 sub.s $f16, $f18, $f16 c.le.s $f16, $f0 bc1t xrp1 yrp1: addi $3, $3, 1 Convert: andi $16, $3, 1 # Convert to ASA sra $17, $3, 1

33 sra $18, $2, 1 sw $16, 20($1) sw $17, 24($1) sw $18, 28($1)

2.6 Distance Measures Using ASA

2.6.1 Euclidean Distance

Using Cartesian coordinates, the Euclidean distance between two points on a

plane, p1 = (x1, y1) and p2 = (x2, y2), is determined using the familiar formula p 2 2 de(p1, p2) = (x1 − x2) + (y1 − y2) .

This formula is easily converted for use with ASA coordinates by substituting the

expressions for x and y from Equation 2–2. The resulting formula for the Euclidean

distance between two points represented in ASA coordinates, p1 = (a1, r1, c1) and

p2 = (a2, r2, c2), is

q ¡ ¢ ¡ ¢ 1 2 1 2 de(p1, p2) = (d) ( 2 (a1 − a2) + (c1 − c2)) + (3)( 2 (a1 − a2) + (r1 − r2)) .

This method of substituting the expressions for x and y from Equation 2–2 into existing formulas is an easy way to convert existing useful methods into ASA coordinate representations.

2.6.2 City-Block Distance

The distance measure where the path is constrained to be along straight lines between adjacent grid points is known by many names such as city-block or Manhattan

distance. Luczak and Rosenfeld [46] developed the equations for the city-block, or d6, distance on a hexagonal grid using oblique addressing. They showed that    |i − h| + |j − k| if (i − h) and (j − k) have the same sign d ((i, j), (h, k)) = 6   max (|i − h| , |j − k|) otherwise (2–3)

34 where (i, j) and (h, k) are the oblique coordinates of two points on the grid. It is straight- forward to convert the oblique coordinates to Cartesian coordinates using the oblique basis vectors       1   q 1 p +  x   2   p   2    =  √    =  √  . (2–4)  3   q 3  y 0 q 2 2 Solving Equation 2–4 for p and q in terms of x and y yields

y p = x − √ 3 2y q = √ . 3 Now, substituting the ASA expressions for x and y (with d = 1) from Equation 2–2 and simplifying results in p = c − r

q = a + 2r.

Next, let

i = c1 − r1 h = c2 − r2

j = a1 + 2r1 k = a2 + 2r2 and

U = i − h = (c1 − c2) − (r1 − r2)

V = j − k = (a1 − a2) + (2)(r1 + r2) Lastly, substituting these expressions back into Equation 2–3 provides the city-block

distance between two points represented in ASA coordinates, p1 = (a1, r1, c1) and

p2 = (a2, r2, c2),    |U| + |V | if U and V have the same sign d (p , p ) = . (2–5) 6 1 2   max (|U| , |V |) otherwise

This is an example of how one can easily convert equations developed using oblique

addressing into ASA coordinate representations.

35 2.7 Vector Arithmetic

Let the set of all possible ASA addresses be ASA = {0, 1} × Z × Z. An addition operator, (ASA, +), can be defined as    a1 ⊕ a2      p1 + p2 ≡  r1 + r2 + (a1 ∧ a2)    c1 + c2 + (a1 ∧ a2)

where ⊕ is the symbol for the exclusive-or operation, and    ak      pk =  rk  ∈ ASA.   ck

A negation operator, (ASA, −), can be defined as    a      −p ≡  −r − a    −c − a

which, for convenience can be combined with the addition operator to define a subtraction operator, a binary −, as

p1 − p2 ≡ p1 + (−p2).

Scalar multiplication can be defined for non-negative integer scalar multipliers, k ∈ N, as    (ak) mod 2   ¥ ¦  kp ≡  k   kr + (a) 2   ¥ ¦  k kc + (a) 2

and

−kp ≡ k(−p).

36 Table 2-3. Properties of Array Set Addressing (ASA) Vectors Property Significance Commutativity of addition p1 + p2 = p2 + p1 Associativity of addition p1 + (p2 + p3) = (p1 + p2) + p3 Identity element of addition ∃0 ∈ S : p + 0 = p, ∀p ∈ S Inverse elements of addition ∃q ∈ S : p + q = 0, ∀p ∈ S Distributivity of scalar multiplication k(p + q) = kp + kq (w.r.t. vector addition) Distributivity of scalar multiplication (k + j)p = kp + jp (w.r.t. scalar addition) Compatibility of scalar multiplication k (jp) = (kj)p (with multiplication of scalars) Identity element of scalar multiplication 1p = p

Given the definitions above, ASA vectors can be shown to satisfy the properties listed in Table 2-3, making ASA a module over the ring of integers, or a Z-module. Note that in an earlier paper [47] it was erroneously stated that ASA formed a vector space. The

difference between a module and a vector space is that the scalars of a vector space are required to lie in a field rather than an arbitrary ring. For the following sections, refer to Section A.1 of Appendix A for the domain definitions and identities used. Proofs of the remaining properties can be found in Appendix A. 2.7.1 Proof of the Associativity of Addition

Lemma 1. If x, y, z ∈ {0, 1}, then only one of x ∧ y and ((x ⊕ y) ∧ z) can be true, that is, ¬((x ∧ y) ∧ ((x ⊕ y) ∧ z)).

Proof. ¬((x ∧ y) ∧ ((x ⊕ y) ∧ z)) =

¬((x ∧ y) ∧ (x ⊕ y) ∧ z) (associativity of conjunction)

=¬(x ∧ y) ∨ ¬(x ⊕ y) ∨ ¬z (DeMorgan’s Law)

=¬(x ∧ y) ∨ ¬((x ∨ y) ∧ ¬(x ∧ y)) ∨ ¬z

=¬(x ∧ y) ∨ ¬(x ∨ y) ∨ (x ∧ y) ∨ ¬z (DeMorgan’s Law)

=(x ∧ y) ∨ ¬(x ∧ y) ∨ ¬(x ∨ y) ∨ ¬z

37 =T ∨ ¬(x ∨ y) ∨ ¬z

=T .

Lemma 2. x ∧ y + ((x ⊕ y) ∧ z) = (x ∨ y) ∧ (x ∨ z) ∧ (y ∨ z)

Proof. Since only one of x ∧ y and ((x ⊕ y) ∧ z) can be true (Lemma 1),

x ∧ y + ((x ⊕ y) ∧ z) = (x ∧ y) ∨ ((x ⊕ y) ∧ z)

= ((x ∧ y) ∨ (x ⊕ y)) ∧ (((x ∧ y) ∨ z)

= ((x ∧ y) ∨ ((x ∨ y) ∧ ¬(x ∧ y))) ∧ ((x ∧ y) ∨ z)

= ((x ∧ y) ∨ (x ∨ y) ∧ ¬(x ∧ y)) ∧ ((x ∧ y) ∨ z)

= (x ∨ y) ∧ ((x ∧ y) ∨ z)

= (x ∨ y) ∧ (x ∨ z) ∧ (y ∨ z).

Theorem 2.1. If p1, p2, p3 ∈ ASA, then p1 + (p2 + p3) = (p1 + p2) + p3.

Proof.      a1 ⊕ a2   a3          (p1 + p2) + p3 =  r1 + r2 + (a1 ∧ a2)  +  r3      c1 + c2 + (a1 ∧ a2) c3    (a1 ⊕ a2) ⊕ a3      =  r1 + r2 + r3 + (a1 ∧ a2) + ((a1 ⊕ a2) ∧ a3)  (2–6)   c1 + c2 + c3 + (a1 ∧ a2) + ((a1 ⊕ a2) ∧ a3)

38 and      a1   a2 ⊕ a3          p1 + (p2 + p3) =  r1  +  r2 + r3 + (a2 ∧ a3)      c1 c2 + c3 + (a2 ∧ a3)    a1 ⊕ (a2 ⊕ a3)      =  r1 + r2 + r3 + (a2 ∧ a3) + (a1 ∧ (a2 ⊕ a3))  . (2–7)   c1 + c2 + c3 + (a2 ∧ a3) + (a1 ∧ (a2 ⊕ a3))

The equality of Equations 2–6 and 2–7 is proven by considering the coordinates separately. First, because exclusive-or is associative,

(a1 ⊕ a2) ⊕ a3 = a1 ⊕ a2 ⊕ a3 = a1 ⊕ (a2 ⊕ a3).

Second, by Lemma 2,

(a1 ∧ a2) + ((a1 ⊕ a2) ∧ a3) = (a1 ∧ a2) ∨ (a1 ∧ a3) ∨ (a2 ∧ a3)

= (a2 ∧ a3) ∨ (a2 ∧ a1) ∨ (a3 ∧ a1)

= (a2 ∧ a3) + ((a2 ⊕ a3) ∧ a1)

= (a2 ∧ a3) + (a1 ∧ (a2 ⊕ a3)).

Thus

r1 + r2 + r3 + (a1 ∧ a2) + ((a1 ⊕ a2) ∧ a3) = r1 + r2 + r3 + (a2 ∧ a3) + (a1 ∧ (a2 ⊕ a3)).

Finally, c1, c2, c3 can be substituted for r1, r2, r3 yielding

c1 + c2 + c3 + (a1 ∧ a2) + ((a1 ⊕ a2) ∧ a3) = c1 + c2 + c3 + (a2 ∧ a3) + (a1 ∧ (a2 ⊕ a3)).

2.7.2 Proof of Distributivity of Scalar Multiplication (w.r.t. Scalar Addition)

Theorem 2.2. If k, j ∈ N and p ∈ ASA, then (k + j)p = kp + jp.

39 Proof.    (a(k + j)) mod 2   ¥ ¦   k+j  (k + j)p =  (k + j)r + (a)  (2–8)  2  ¥ k+j ¦ (k + j)c + (a) 2 and      (ak) mod 2   (aj) mod 2   ¥ ¦   ¥ ¦  kp + jp =  k  +  j   kr + (a) 2   jr + (a) 2   ¥ ¦   ¥ ¦  k j kc + (a) 2 jc + (a) 2    ((ak) mod 2 + (aj) mod 2) mod 2   ¥ ¦ ¥ ¦  =  k j  . (2–9)  (k + j)r + (a)( 2 + 2 ) + (((ak) mod 2) ∧ ((aj) mod 2))   ¥ ¦ ¥ ¦  k j (k + j)c + (a)( 2 + 2 ) + (((ak) mod 2) ∧ ((aj) mod 2))

The equality of Equations 2–8 and 2–9 is proven by considering the coordinates separately. First,

(a(k + j)) mod 2 = (ak + aj) mod 2

= ((ak) mod 2 + (aj) mod 2) mod 2. (Identity 3)

Second,

¹ º µ ¶ k + j k + j − (k + j) mod 2 (a) = (a) (Identity 1) 2 2 µ ¶ k + j − (d ⊕ d ) = (a) j k (Identity 4) 2 µ ¶ k + j − (d + d ) + (2)(d ∧ d ) = (a) k j j k (Identity 6) 2 µµ ¶ µ ¶¶ k − d j − d = (a) k + j + (a)(d ∧ d )) 2 2 j k µ¹ º ¹ º¶ k j = (a) + + (a)(d ∧ d )) (Identity 1) 2 2 j k µ¹ º ¹ º¶ k j = (a) + + (a ∧ (d ∧ d )) (Identity 8) 2 2 j k

40 µ¹ º ¹ º¶ k j = (a) + + ((a ∧ k mod 2) ∧ (a ∧ j mod 2)) 2 2 ¹ º ¹ º k j (a) + (a) + (((ak) mod 2) ∧ ((aj) mod 2)). (Identity 2) 2 2

Thus ¹ º ¹ º ¹ º k + j k j (k + j)r + (a) = (k + j)r + (a) + (a) + (((ak) mod 2) ∧ ((aj) mod 2)). 2 2 2

Finally, by substituting c for r, ¹ º ¹ º ¹ º k + j k j (k + j)c + (a) = (k + j)c + (a) + (a) + (((ak) mod 2) ∧ ((aj) mod 2)). 2 2 2

2.7.3 Proof of Compatibility of Scalar Multiplication (with Multiplication of Scalars)

Theorem 2.3. If k, j ∈ N and p ∈ ASA, then k(jp) = (kj)p.

Proof.      (aj) mod 2   ((k)((aj) mod 2)) mod 2   ¥ ¦   ¥ ¦ ¥ ¦  k(jp) = (k)  j  =  j k  (2–10)  jr + (a) 2   kjr + (ka) 2 + ((aj) mod 2) 2   ¥ ¦   ¥ ¦ ¥ ¦  j j k jc + (a) 2 kjc + (ka) 2 + ((aj) mod 2) 2 and    (akj) mod 2   ¥ ¦   kj  (kj)p =  kjr + (a)  . (2–11)  2  ¥ kj ¦ kjc + (a) 2 The equality of Equations 2–10 and 2–11 is proven by considering the coordinates separately. First,

((k)((aj) mod 2)) mod 2 = a ∧ (j mod 2) ∧ (k mod 2) (Identity 2)

= a ∧ (dk ∧ dj )

= a ∧ (kj) mod 2 (Identity 5)

41 = (akj) mod 2. (Identity 2)

Second, ¹ º ¹ º µ ¶ µ ¶ j k j − d k − d (ka) + ((aj) mod 2) = (ka) j + ((aj) mod 2) k (Identity 1) 2 2 2 2 µ ¶ µ ¶ j − d k − d = (ka) j + (a ∧ d ) k (Identity 2) 2 j 2 µ ¶ µ ¶ j − d k − d = (ka) j + (a)(d ) k (Identity 8) 2 j 2 µ ¶ kj − kd + kd − d d = (a) j j j k 2 µ ¶ kj − (d ∧ d ) = (a) j k (Identity 8) 2 µ ¶ kj − (kj) mod 2 = (a) (Identity 5) 2 ¹ º kj = (a) . (Identity 1) 2

Thus ¹ º ¹ º ¹ º j k kj kjr + (ka) + ((aj) mod 2) = kjr + (a) . 2 2 2 Finally, by substituting c for r, ¹ º ¹ º ¹ º j k kj kjc + (ka) + ((aj) mod 2) = kjc + (a) . 2 2 2

2.8 Comparison of ASA to Spiral Addressing

The authors of the book Hexagonal Image Processing [12] provide Python source code for some basic coordinate operations, allowing a direct comparison of ASA to HIP for these operations. Recall from Section 1.2 that HIP is one of the spiral addressing techniques. The results in Table 2-4 are the mean run times of 10,000 operations performed on randomly selected addresses. The source code of the comparison can be found in Appendix B. Table 2-5 compares the complexity of each operation. The

42 Table 2-4. Run Time Comparison Between ASA and HIP (µs, mean (std)) Operation HIP ASA Run Time Ratio Address (vector) Addition 23.85 (3.15) 2.11 (0.97) 11.28 Address (vector) Subtraction 33.98 (3.56) 2.56 (0.47) 13.28 Scalar Multiplication 6652.08 (4076.89) 3.73 (0.73) 1782.20 Get Nearest Neighbor Addresses 118.94 (10.49) 3.31 (0.75) 35.89 Calculate Euclidean Distance 15.83 (2.43) 2.73 (0.56) 5.79 Conversion from Cartesian 9189.68 (3784.79) 4.48 (1.13) 2052.31 Conversion to Cartesian 8.30 (1.85) 2.12 (0.93) 3.92

Table 2-5. Complexity Comparison Between ASA and HIP Operation HIP ASA Address (vector) Addition O((log N)2) O(1) Address (vector) Subtraction O((log N)2) O(1) Scalar Multiplication O(N(log N)2) O(1) Get Nearest Neighbor Addresses O((log N)2) O(1) Calculate Euclidean Distance O(log N) O(1) Conversion from Cartesian O(N(log N)2) O(1) Conversion to Cartesian O(log N) O(1) complexity analysis for the HIP operations was based on the source code provided in the book. An assumption that the magnitude of the scalar is uniformly distributed and bounded by N was used in the HIP scalar multiplication analysis. The complexity analysis for the ASA operations was based on the source code in Appendix B.

43 CHAPTER 3 BASIC IMAGE PROCESSING OPERATIONS

3.1 Convolution

A convolution calculates a new value for each pixel from a linear combination of its neighbors and itself. Mathematically, in two dimensions,

X X M(x, y) ∗ I (x, y) = I (u, v)M(x − u, y − v) u v where ∗ is the symbol for matrix convolution, M is the convolution mask (or filter), and I is the intensity image. Convolutions in ASA (denoted by ⊗) are performed by representing the convolution mask in ASA coordinates, convolving each mask array with each image array (using standard matrix convolution), and summing the results to produce the ASA representation of the convolved/filtered image. Let M be the ASA representation of the convolution mask, I be the input image, and J be the image resulting from the convolution. Then J = M ⊗ I is defined as

J0 = M0 ∗ I0 + M1 ∗ I1 and

J1 = M0 ∗ I1 + M1 ∗ I0, where the subscript indicates the array coordinate. Figure 3-1 shows this operation pictorially. ASA convolution is no more complex than performing a convolution on a rectan- gularly sampled image and the convolutions on the individual arrays could be done in parallel. For example, assume an N × N rectangularly sampled image and an N × N hexagonally sampled image. Also, assume a zero valued border surrounds the images so that all pixels use the full convolution mask. A convolution operation on the rectangularly sampled image using a m × m convolution mask requires m2N2 multiplications and (m2 − 1)N2 additions. Performing a convolution on the hexagonally

44 Figure 3-1. An example of how convolutions are performed using ASA.

Figure 3-2. A graph comparing the sizes of nearest neighbor neighborhoods (hexagonal vs. rectangular).

45 sampled image using an m2 point convolution mask requires µ» ¼ µ ¶ ¹ º µ ¶¶ m2 N2 m2 N2 (2) + = m2N2 multiplications 2 2 2 2

and µµ» ¼ ¶ µ ¶ µ¹ º ¶ µ ¶ ¶ m2 N2 m2 N2 N2 (2) − 1 + − 1 + = (m2 − 1)N2 additions, 2 2 2 2 2

exactly the same number of operations as the rectangularly sampled image. However,

since a hexagonal neighborhood is 7 pixels (nearest neighbors) and a rectangular neighborhood is 9 pixels (nearest neighbors, 8-way connected), one would use a 7-

point mask in place of the 9-point mask, making the ASA convolution slightly faster for neighborhood-based filters. Additionally, as the neighborhood size increases (including 2nd nearest neighbors, then 3rd nearest, etc.), the gap between the sizes of the masks increases as well, as shown in Figure 3-2. So ASA convolution increasingly improves in performance compared to rectangular convolution as the size of the neighborhood increases. From the above analysis, it is easy to see that the complexity of ASA convolution is O(n), where n = N2 is the number of pixels in the input image, assuming n >> m2.

3.2 Gradient Estimation

The following is a derivation of the convolution masks for local gradient estimation on the hexagonal grid. It demonstrates working with ASA coordinates directly by

calculating a least-squares fit of a plane to a hexagonal neighborhood in Euclidean space. The goal is to determine the gradients in the x and y directions of a fitted plane. The diagram in Figure 3-3 shows an example, where the black lines indicate the intensity values of the neighboring pixels and the goal is to determine the values of the x and y coefficients, e and f respectively, of the blue fitted plane. The derivation parallels

that of Snyder [15], who derived convolution masks for local gradient estimation using an

46 Figure 3-3. Example of fitting a plane to a hexagonal neighborhood.

Figure 3-4. Depiction of hexagonal neighborhood indexing.

oblique addressing scheme. The convolution masks resulting from the ASA derivation are equivalent to Snyder’s results. Let the subscript k index the pixels of a hexagonal neighborhood with k = 0 being the center pixel, k = 1 being its neighbor immediately to the right, and k increasing

T counter-clockwise around the center as depicted in Figure 3-4. Then pk = (ak , rk , ck ) ∈

ASA, k = 1...6, are the neighboring pixels of p0. Let   1  2 0 1  B1 =  √ √  3 2 3 0

47 and    e  A =   . f

And let

Zk = B1pk    Xk  =   Yk  

ax  2 + ck  =  √  ak 3( 2 + rk )

where Zk is the vector of Cartesian coordinates for the ASA address pk , and A is the vector of coefficients for the x and y variables of the fitted plane. The offset constant of the fitted plane has no effect on the gradients of the plane and is therefore omitted for simplicity. Let Ik be the intensity value of the pixel at location pk (the black lines in Figure 3-3) and define the error function as

X6 2 ¡ T ¢2 E = Ik − A Zk . (3–1) k=1 Differentiating Equation 3–1 with respect to A results in ¡ ¢ ∂ E 2 X6 ¡ ¢ = −2I Z + 2Z ZT A . (3–2) ∂A k k k k k=1 Settting Equation 3–2 equal to zero results in

X6 0 = SA − Ik Zk (3–3) k=1 where

X6 T S = Zk Zk k=1

48   6 X  Xk Xk Xk Yk  =   . (3–4) k=1 Xk Yk Yk Yk

Equation 3–3 can be rearranged to yield

X6 SA = Ik Zk . (3–5) k=1 Therefore,

X6 −1 A = S Ik Zk k=1  P6  k=1 Ik Xk  = S−1   . (3–6) P6 k=1 Ik Yk

T Assuming p0 = (0, 0, 0) , the pk coordinate values can be substituted into

a X = k + c k 2 k and √ ³a ´ Y = 3 k + r k 2 k which can be used to solve Equation 3–4 resulting in    3 0  S =   . 0 3

The familiar formula for the inverse of a 2 × 2 matrix yields   1  3 0  S−1 =   . 1 0 3

−1 Substituting S , Xk , and Yk into Equation 3–6 yields     1 1 1 1 1 1  e   3 I1 + 6 I2 − 6 I3 − 3 I4 − 6 I5 + 6 I6    =  √ √ √ √  . (3–7) 3 3 3 3 f − 2 I2 − 2 I3 + 2 I5 + 2 I6

49 Equation 3–7 gives expressions for e and f which are obviously implementable as convolutions. The resultant (scaled) convolution masks for estimating the local gradients in the x and y directions are    −1 1      Fx =  −2 0 2    −1 1 and    −1 −1      Fy =  0 0 0  ,   1 1 shown pictorially in Figure 3-5. Also, because of the natural symmetry of hexagonal grids, these masks can be rotated to provide gradient estimates along six different axes.

Figure 3-5. Convolution masks that estimate local gradients and the six axes along which gradients can be estimated.

50 3.3 Edge Detection

Another important task that uses convolution is that of edge detection. This processing task helps simplify complex images by retaining only the location of the edges in an image. Three different edge detectors have been implemented using ASA: gradient-based, Laplacian of Gaussian (LoG), and Canny. Note that B1 is assumed to be the conversion matrix for this section. 3.3.1 Gradient-Based Edge Detector

Several edge detectors that have been developed for rectangularly sampled images (e.g. Prewitt and Sobel edge detectors) are based on gradient estimation and thresholding. However, the convolution masks for those operators do not directly translate to a hexagonal grid. Since the convolution masks for local gradient estimation on a hexagonal image were derived directly in the previous section, those masks are used here for the gradient-based edge detector. Edge detection results on the hexagonal image are compared to the result of running the Prewitt [48] edge detector on an equivalent rectangularly sampled image. Gradient-based edge detection proceeds by estimating the local gradients in the x and y directions, calculating the magnitudes of the resultant gradient vectors, and thresholding the magnitude to determine the edges. Let I be the ASA representation of the hexagonally sampled input image. Using the gradient estimation filters, Fx and Fy , from the previous section, let

Gx = Fx ⊗ I and

Gy = Fy ⊗ I be the gradient images in the x and y directions, respectively. The magnitude of the gradient is then determined by

p 2 2 M(a, r, c) = Gx (a, r, c) + Gy (a, r, c) .

51 The magnitude image, M, is then thresholded using Otsu’s Algorithm [49] to select edge locations. Figures 3-6, 3-7, and 3-8 show the results of running gradient-based edge detection on three different images. The images were obtained from high-resolution square-sampled images using the technique described in Section 1.5.1. Each figure shows the input image at the top, the resultant edge map in the middle, and the gradient magnitude image at the bottom. The gradient magnitude image includes a color bar with the threshold selected by Otsu’s algorithm given and indicated by an arrow. The hexagonally sampled images are on the left and the rectangularly sampled images are on the right. There does not appear to be any significant differences between the resultant edge maps of these three examples.

3.3.2 Laplacian of Gaussian (LoG) Edge Detector

The LoG edge detector was introduced by Marr and Hildreth [50] as part of a model of mammalian early vision processing. Their first step is to select a smoothing filter to reduce the range of scales of intensity changes. They select a Gaussian smoothing filter to optimize the trade-off between smooth, band-limited frequency response and good spatial localization. Next, they determine that the best way to detect intensity changes (edges) is to find zero-crossings in the second (directional) derivative of the intensity image, in the appropriate direction. Of course the appropriate directions are unknown a priori, so rather than compute multiple directional derivatives, they reduce the computational burden by selecting the Laplacian (an orientation-independent second derivative operator) instead. The problem (for rectangularly-sampled images) is thus reduced to finding the zero-crossings of

L(x, y) = 4(G(x, y) ∗ I (x, y)) = (4G(x, y)) ∗ I (x, y)

∂2 ∂2 where 4 = + is the Laplace operator, G is the Gaussian smoothing filter, and I ∂2x ∂2y is the intensity image. The Laplace operator can be applied to the Gaussian filter alone

52

¢ £

¢ ¢

¡ ¡

Figure 3-6. Hexagonally sampled (left) and rectangularly sampled (right) images used in gradient-based edge detector example.

53

¢ ¢

¡ ¡

Figure 3-7. Hexagonally sampled (left) and rectangularly sampled (right) images used in gradient-based edge detector example.

54

¡ ¢ ¡ ¢

Figure 3-8. Hexagonally sampled (left) and rectangularly sampled (right) images used in gradient-based edge detector example.

55 due to the fact that ZZ ∂ ∂ (G(x, y) ∗ I (x, y)) = I (u, v)G(x − u, y − v)dudv ∂x ∂x ZZ ∂ = I (u, v) G(x − u, y − v)dudv ∂x µ ¶ ∂ = G(x, y) ∗ I (x, y). ∂x

Hence, the LoG filter can be pre-computed as

LoG(x, y) = 4G(x, y) ∂2 ∂2 = G(x, y) + G(x, y) ∂2x ∂2y µ ¶ µ ¶ µ ¶ 1 x 2 + y 2 − 2σ2 x 2 + y 2 = √ exp − (3–8) 2πσ σ4 2σ2 and the LoG image is obtained through a regular convolution operation,

L(x, y) = LoG(x, y) ∗ I (x, y).

Similarly, for hexagonally sampled images, the LoG filter can be pre-computed in ASA √ ¡ ¢ a a space by substituting ( 2 + c) for x and 3 2 + r for y in Equation 3–8,

LoG(a, r, c) = 4G(a, r, c) µ ¶ á ¢2 ¡ ¢2 ! à ¡ ¢2 ¡ ¢2 ! 1 a + c + 3 a + r − 2σ2 a + c + 3 a + r = √ 2 2 exp − 2 2 . 2πσ σ4 2σ2

The ASA representation of the LoG image is obtained through an ASA convolution operation as described in Section 3.1,

L(a, r, c) = LoG(a, r, c) ⊗ I (a, r, c), and the edges are found by locating the zero-crossings. The zero-crossings occur where a negative valued pixel has one or more positive valued neighbors (or vice versa).

Figures 3-9, 3-10, and 3-11 show the results of running LoG edge detection on the same three images used in the gradient-based edge detection. Each figure shows the input

56 image at the top, the resultant edge map in the middle, and the LoG image at the bottom with a color bar. The hexagonally sampled images are on the left and the rectangularly sampled images are on the right. Again, there does not appear to be any significant differences between the resultant edge maps of these three examples. 3.3.3 Canny Edge Detector

The Canny edge detector [51] is based on the goal of optimizing performance with regard to the following criteria:

1. Low error rate edge detection 2. Good edge localization 3. Singular response to a single edge

Canny develops the equations and derives the optimal step edge operator in one dimension. It is then noted that the optimal operator is very similar to the first derivative of a Gaussian. A quantitative comparison shows that the first derivative of a Gaussian operator is about 20% worse than the optimal operator for criteria 1 and 2, and about 10% worse for criterion 3. Since this performance degradation is considered minor for real images and because the derivative of a Gaussian (DoG) operator can be computed in two dimensions much more easily than the optimal operator, the approximation is selected over the optimal. The Canny edge detection method consists of several steps:

1. Filter the image with a Gaussian smoothing filter to reduce noise 2. Apply the DoG operators to get local directional gradient estimates 3. Calculate the magnitude and direction of the local gradient estimates 4. Perform non-maxima suppression using gradient directions 5. Apply hysteresis thresholds using gradient magnitudes

The first step is performed via convolution with a Gaussian smoothing filter

Is (a, r, c) = G(a, r, c) ⊗ I (a, r, c) where µ ¶ Ã ¡ ¢2 ¡ ¢2 ! 1 a + c + 3 a + r G(a, r, c) = √ exp − 2 2 . 2πσ 2σ2

57 Figure 3-9. Hexagonally sampled (left) and rectangularly sampled (right) images used in LoG edge detector example.

58 Figure 3-10. Hexagonally sampled (left) and rectangularly sampled (right) images used in LoG edge detector example.

59 Figure 3-11. Hexagonally sampled (left) and rectangularly sampled (right) images used in LoG edge detector example.

60 The size of the filter is a function of σ. For these examples, the filter extends out radially while G(a, r, c) > 0.0001. Next, the smoothed image is convolved with the DoG operators to get local gradient estimates in the x and y directions,

Ix (a, r, c) = Gx (a, r, c) ⊗ Is (a, r, c)

and

Iy (a, r, c) = Gy (a, r, c) ⊗ Is (a, r, c),

where à ¡ ¢! à ¡ ¢ ¡ ¢ ! µ ¶ a a 2 a 2 1 − 2 + c 2 + c + 3 2 + r Gx (a, r, c) = √ exp − 2πσ σ2 2σ2

and à √ ¡ ¢! à ¡ ¢ ¡ ¢ ! µ ¶ a a 2 a 2 1 − 3 2 + r 2 + c + 3 2 + r Gy (a, r, c) = √ exp − . 2πσ σ2 2σ2 Then the magnitude and direction of each local gradient are calculated by

p 2 2 M(a, r, c) = Gx (a, r, c) + Gy (a, r, c)

and µ ¶ G (a, r, c) D(a, r, c) = arctan y , Gx (a, r, c) respectively. The gradient directions are then quantized into six directions as shown

in Figure 3-12. Note that with rectangularly sampled images, the gradient directions are quantized into four directions only. This improved angular resolution is another benefit of hexagonal sampling. Non-maxima suppression is used to thin the candidate edges by keeping only the pixels that are greater than their neighbors on both sides along the (quantized) gradient direction. And lastly, thresholds are applied to the gradient magnitudes using hysteresis. Any candidate edge pixel that exceeds the high threshold is automatically added to the edge map and labeled a strong edge. Any pixel that exceeds the low threshold and is connected to a strong edge pixel is considered a weak edge pixel and is also added to the edge map. In this context, connected means

61 Figure 3-12. Local gradient directions are quantized into six directions. that there is a path from the weak edge pixel to a strong edge pixel consisting of weak edge pixels. Recall that there is no ambiguity about connectedness with hexagonally sampled images, thus simplifying this step compared to rectangularly sampled images.

This hysteresis approach reduces the streaking phenomenon that is common with edge detection methods that use thresholding. Canny uses a fixed percentage of the histogram to select the high threshold and claims it should be 2 or 3 times the low threshold. Otsu’s algorithm [49] can also be used to select the high threshold with the low threshold set to 40% of the high. Figures 3-13, 3-14, and 3-15 show the results of running Canny edge detection on the same three images used in the previous edge detection methods. Each figure shows the input image at the top, the resultant edge map in the middle, and the magnitude image following non-maxima suppression at the bottom with a color bar. The hexagonally sampled images are on the left and the rectangularly sampled images are on the right. For the Canny method, there are differences between the hexagonally and rectangularly sampled edge maps. Namely, the hexagonally sampled version detects noticeably more edges. Compare the locations circled in red on the edge maps in Figures 3-13, 3-14, and 3-15 to see some examples.

62

¡¢ ¡¢ ¡ £¤ ¡£¤

Figure 3-13. Hexagonally sampled (left) and rectangularly sampled (right) images used in Canny edge detector example.

63

¢

¤ ¥

¢ ¤

¡

¡

¡ ¡ £

Figure 3-14. Hexagonally sampled (left) and rectangularly sampled (right) images used in Canny edge detector example.

64

¢ ¤ ££

£¥

¡ ¡ ¡ ¡

Figure 3-15. Hexagonally sampled (left) and rectangularly sampled (right) images used in Canny edge detector example.

65 3.4 Downsampling

Downsampling in image processing is the process of systematically reducing the resolution of an image. It allows one to perform multi-scale processing operations such as wavelet decomposition or optical flowfield calculations. One of the advantages of using ASA over spiral addressing is the ability to downsample without unwanted

image rotations. In the HIP framework [12], downsampling produces a 7× reduction in resolution and introduces a −40.9◦ rotation of the image per iteration. Downsampling in ASA produces a 4× reduction in resolution and introduces no rotation. It is an efficient operation consisting of a convolution with an anti-aliasing filter followed by subsampling to form the downsampled image as shown in Figure 3-16. Note that for wavelet decompositions, the anti-aliasing filter is generally replaced by an FIR filter bank. Let I be the N × N intensity image to be downsampled and let   1 1  8 8     1 1 1  F =   .  8 4 8  1 1 8 8

be the anti-aliasing filter. Then, H = F ⊗ I is the filtered image. Let D be the downsampled image formed from the selected pixels of H. The following code snipet shows the method for subsampling the smoothed image.

a = 1; for(r=0, r+=2, r

}

66 Let n = N2 be the number of pixels in the input image. Then, the complexity of downsampling in ASA is

O(n) (ASA convolution) ³ √ ´ ³ √ ´ n n + 2 2 (subsampling) = O(n).

Figure 3-16. Downsampling with ASA. After convolving the original image with the anti-aliasing filter, the light gray pixels form the 0-array and the dark gray pixels form the 1-array.

67 CHAPTER 4 TRANSFORMS

4.1 Fourier Transforms

4.1.1 The Hexagonal Discrete Fourier Transform (HDFT)

In his seminal paper on processing hexagonally sampled images [7], Mersereau defines the HDFT and its inverse for a hexagonally sampled two-dimensional (2-D)

signal x(n1, n2) as X X · µ ¶¸ (2n1 − n2)(2k1 − k2) n2k2 X (k1, k2) = x(n1, n2) exp −jπ + (4–1) 2N1 + N2 N2 n1 n2

and

X X · µ ¶¸ 1 (2n1 − n2)(2k1 − k2) n2k2 x(n1, n2) = X (n1, n2) exp jπ + , (4–2) K 2N1 + N2 N2 k1 k2

where R(N1, N2) is the region of support for x(n1, n2), (n1, n2), (k1, k2) ∈ R(N1, N2),

K = N2(2N1 + N2), and (n1, n2) and (k1, k2) are coordinates of oblique addressing schemes. N1 and N2 are the hexagonal periods of the hexagonally periodic extended version of x(n1, n2). Using the method of Section 2.6.2 to convert from oblique to ASA coordinates, it is easy to show that

n1 = a + r + c

n2 = a + 2r

where (a, r, c)T ∈ ASA. Similarly, let

k1 = b + s + d

k2 = b + 2s

T where (b, s, d) ∈ ASA. Let x(a, r, c) = x(n1, n2) and X (b, s, d) = X (k1, k2) and assume that x(a, r, c) and X (b, s, d) are form 1 array sets with arrays sized n × m. Then, it can

68 be shown that n N = m − 1 2 and

N2 = n.

Therefore, equations 4–1 and 4–2 can be rewritten as · µ ¶¸ X X X (a + 2c)(b + 2d) (a + 2r)(b + 2s) X (b, s, d) = x(a, r, c) exp −jπ + 2m n a r c (4–3)

and · µ ¶¸ 1 X X X (a + 2c)(b + 2d) (a + 2r)(b + 2s) x(a, r, c) = X (b, s, d) exp jπ + , 2mn 2m n b s d (4–4) where R(A, R, C) is the region of support for x(a, r, c) and (a, r, c), (b, s, d) ∈

R(A, R, C). Thus, equations 4–3 and 4–4 are the HDFT and its inverse for hexagonally sampled images addressed with ASA.

4.1.2 The Hexagonal Fast Fourier Transform (HFFT)

A fast Fourier transform (FFT) is any algorithm that can compute the DFT in

O(N log N) time where N is the number of samples in the signal. For 2-D signals, this is normally achieved by exploiting the fact that the Fourier kernel is separable in Cartesian coordinates, which allows one to calculate two one-dimensional (1-D) FFTs. By his choice of oblique coordinates, Mersereau encountered an “insurmountable difficulty” [7] when considering the same approach since the Fourier kernel is no longer separable

as in equations 4–1 and 4–2, due to the product term containing both coordinates in the exponent. However, notice in equations 4–3 and 4–4 that the row coordinates, r and s, are separate from the column coordinates, c and d, meaning that the coordinates do

not appear together in a product term in the exponent. Therefore, the Fourier kernel is

separable in ASA coordinates! Some researchers have devised other methods to make

69 the HDFT kernel separable as well. Morikawa et al. [52] enforced constraints on the “sampling point generating” and “periodic extension” vector systems to derive coordinate systems which lead to a separable kernel in the HDFT. However, the constraints lead also to coordinate systems that have oblique basis vectors which exacerbate the problems of oblique addressing schemes as described in Section 1.2. Alim and

Moeller¨ [53] achieve separability by assuming rectangular periodicity in the spatial domain which corresponds to rectangular sampling in the frequency domain. Hence, this approach disallows taking advantage of the benefits provided by the hexagonal grid in the frequency domain. ASA, on the other hand, provides an elegant coordinate system that uses the hexagonal grid in both spatial and frequency domains and leads to an efficient FFT based on the separability of the Fourier kernel.

Let µ ¶ X (c)(b + 2d) gK (b, r, d) = x(a, r, c) exp −j2π a K c and

2m ga(b, r, d) = ga (b, r, d).

Then, · µ ¶¸ X X X (a + 2c)(b + 2d) (a + 2r)(b + 2s) X (b, s, d) = x(a, r, c) exp −jπ + 2m n a r c · µ ¶¸ X X (2c)(b + 2d) (2r)(b + 2s) = x(0, r, c) exp −jπ + 2m n r c · µ ¶¸ X X (1 + 2c)(b + 2d) (1 + 2r)(b + 2s) + x(1, r, c) exp −jπ + 2m n r c µ ¶ X (2r)(b + 2s) = g (b, r, d) exp −jπ 0 n r " µ ¶# µ ¶ X X (1 + 2c)(b + 2d) (1 + 2r)(b + 2s) + x(1, r, c) exp −jπ exp −jπ 2m n r c µ ¶ X (2r)(b + 2s) = g (b, r, d) exp −jπ 0 n r

70 µ ¶ µ ¶ (b + 2d) X (1 + 2r)(b + 2s) + exp −jπ g (b, r, d) exp −jπ 2m 1 n µ r ¶ X (2r)(b + 2s) = g (b, r, d) exp −jπ 0 n r · µ ¶¸ µ ¶ b + 2d b + 2s X (2r)(b + 2s) + exp −jπ + g (b, r, d) exp −jπ . 2m n 1 n r

Let µ ¶ X (r)(b + 2s) f K (b, s, d) = g (b, r, d) exp −j2π a a K r and

n fa(b, s, d) = fa (b, s, d).

Then, · µ ¶¸ b + 2d b + 2s X (b, s, d) = f (b, s, d) + exp −jπ + f (b, s, d). (4–5) 0 2m n 1

Each of the functions ga and fa can be calculated in O(log N) time, making equation 4–5 an HFFT.

Computing the Functions g and f. Starting with the 0 array of ga, µ ¶ mX−1 (c)(2d) g (0, r, d) = x(a, r, c) exp −j2π a 2m c=0 µ ¶ mX−1 (c)(d) = x(a, r, c) exp −j2π . (4–6) m c=0

Equation 4–6 shows that ga(0, r, d) is the standard 1-D FFT of the rows of x. Therefore, any standard FFT algorithm may be used to calculate the 0 arrays of g0 and g1 from the rows of x. Let µ ¶ −j2π W x = exp x , K K and assume n, m ∈ {2i : i is a positive integer}. Applying the standard decimation-in- space (DIS) radix-2 approach (usually called a decimation-in-time approach, but since these are images, DIS is used here instead) results in the standard DIS radix-2 FFT

71 definition,

2m d m m ga (0, r, d) = W2mga, odd(0, r, d) + ga, even(0, r, d) (4–7)

m 2m m where ga, odd is comprised of the odd columns of ga and, similarly, ga, even is comprised

2m of the even columns of ga . Notice that equation 4–7 is a recursive definition of

ga(0, r, d) that can be computed in O(log N) time.

Next, consider the 1 array of ga, µ ¶ mX−1 (c)(2d + 1) g (1, r, d) = x(a, r, c) exp −j2π a 2m c=0 Ã ¡ ¢! mX−1 (c) d + 1 = x(a, r, c) exp −j2π 2 . (4–8) m c=0 Equation 4–8 is not the standard Fourier transform due to the constant offset applied to the d variable. Equation 4–8 will be referred to as the non-standard transform 1 (NST1). The DIS radix-2 approach can still be applied as follows,

m−1 X 1 (c)(d+ 2 ) ga(1, r, d) = x(a, r, c)Wm c=0 X 1 X 1 (c)(d+ 2 ) (c)(d+ 2 ) = x(a, r, c)Wm + x(a, r, c)Wm c odd c even m −1 m −1 X2 1 X2 1 (2k+1)(d+ 2 ) (2k)(d+ 2 ) = x(a, r, 2k + 1)Wm + x(a, r, 2k)Wm k=0 k=0 m −1 m −1 1 X2 1 X2 1 (d+ 2 ) (k)(d+ 2 ) (k)(d+ 2 ) =Wm x(a, r, 2k + 1)W m + x(a, r, 2k)W m 2 2 k=0 k=0 d+ 1 m m ( 2 ) 2 2 =Wm ga, odd(1, r, d) + ga, even(1, r, d). (4–9) µ ¶ Ã ! 2kn −j2π −j2π kn Note that WK = exp 2kn = exp K kn = W K . Equation 4–9 is again a K 2 2 recursive definition of ga(1, r, d) that can be computed in O(log N) time. Similarly, µ ¶ Xn−1 (r)(2s) f (0, s, d) = g (0, r, d) exp −j2π (4–10) a a n r=0

72 and µ ¶ Xn−1 (r)(2s + 1) f (1, s, d) = g (1, r, d) exp −j2π . (4–11) a a n r=0 Equations 4–10 and 4–11 are not standard Fourier transforms and will be referred to as non-standard transforms 2 and 3 (NST2 and NST3), respectively. Using the same

approach as the derivation of Equation 4–9, fa(0, s, d) and fa(1, s, d) can be defined as

n n 2s 2 2 fa(0, s, d) = Wn fa, odd(0, s, d) + fa, even(0, s, d)

and

n n (2s+1) 2 2 fa(1, s, d) = Wn fa, odd(1, s, d) + fa, even(1, s, d).

Again, these are recursive definitions of fa(b, r, d) that can be computed in O(log N) time.

Performance Analysis and Comparison of the HFFT. A basic analysis of the standard (rectangularly sampled) radix-2 DIS FFT typically employs a butterfly diagram [54] to visualize the required computations, which can also simplify the counting of the complex additions and complex multiplications required to calculate the FFT.

Figure 4-1 shows a standard DIS butterfly diagram for an 8-point 1-D signal with all computations. Notice the input is considered to be presented in bit-reversed order. It is obvious from the diagram that the full computation version requires N log N complex multiplications and N log N complex additions for an N-point 1-D signal. However, it is

k (k+N) k (k+N/2) common to exploit the periodicity (WN = WN ) and symmetry (WN = −WN ) of the complex exponential in order to reduce these computations. Although additional optimizations are possible, this analysis will be limited to the ones mentioned above. Figure 4-2 shows the result of applying these optimizations to an 8-point 1-D signal. The

1 number of complex multiplications are reduced by 2 and half of the complex additions have become subtractions. Consider a 2n × m rectangularly sampled image where

n, m ∈ {2i : i ∈ N}. The standard radix-2 DIS FFT using the above mentioned

73 Table 4-1. Computations Required for Hexagonal Fast Fourier Transform Arrays Complex Complex Array Input Transform Additions Multiplications¡ ¢ g Array 0 x Array 0 FFT nm log m n m log m 0 ¡ 2 ¢ g Array 1 x Array 0 NST1 nm log m n m log m 0 ¡ 2 ¢ g Array 0 x Array 1 FFT nm log m n m log m 1 ¡ 2 ¢ g Array 1 x Array 1 NST1 nm log m n m log m 1 ¡ ¢ ¡2 ¢ f Array 0 g Array 0 NST2 m n log n m n log n 0 0 ¡ 2 ¢ ¡ 2 ¢ f Array 1 g Array 1 NST3 m n log n m n log n 0 0 ¡ 2 ¢ ¡ 2 ¢ f Array 0 g Array 0 NST2 m n log n m n log n 1 1 ¡ 2 ¢ ¡ 2 ¢ m m f1 Array 1 g1 Array 1 NST3 2 n log n 2 n log n X Array 0 f0 Array 0 and f1 Array 0 N/A nm nm X Array 1 f0 Array 1 and f1 Array 1 N/A nm nm optimizations requires µ ¶ ³m ´ 2n (2n) log m + (m) log 2n = nm log 2nm complex multiplications, and 2 2

(2n)(m log m) + (m)(2n log 2n) = 2nm log 2nm complex additions.

The computations required for the various arrays used during the calculation of the HFFT are slightly different than the standard FFT for most of the arrays. Table 4-1 gives the number of operations required to calculate each array needed in the computation of the HFFT. The sum of these values results in the number of operations required to compute the HFFT as described above. The HFFT requires

2nm log 2nm complex multiplications, and

2nm log 2nm2 complex additions.

This is less than twice the number of total operations required for the standard FFT. The HFFT can similarly be described using signal flow diagrams just like or similar to the butterfly diagrams. Figures 4-3, 4-5, and 4-7 show the full computation diagrams for NS1, NS2, and NS3, respectively. Figures 4-4, 4-6, and 4-8 show the reduced computation diagrams for NS1, NS2, and NS3, respectively.

74 Figure 4-1. A standard FFT butterfly diagram with all computations. The arrowheads indicate complex multiplications and the dots indicate complex additions.

Figure 4-2. A standard FFT butterfly diagram with optimizations. The arrowheads indicate complex multiplications and the dots indicate complex additions.

75 Figure 4-3. NST1 butterfly diagram with all computations. The arrowheads indicate complex multiplications and the dots indicate complex additions.

Figure 4-4. NST1 butterfly diagram with optimizations. The arrowheads indicate complex multiplications and the dots indicate complex additions.

76 Figure 4-5. NST2 butterfly diagram with all computations. The arrowheads indicate complex multiplications and the dots indicate complex additions.

Figure 4-6. NST2 butterfly diagram with optimizations. The arrowheads indicate complex multiplications and the dots indicate complex additions.

77 Figure 4-7. NST3 butterfly diagram with all computations. The arrowheads indicate complex multiplications and the dots indicate complex additions.

Figure 4-8. NST3 butterfly diagram with optimizations. The arrowheads indicate complex multiplications and the dots indicate complex additions.

78 According to Vince and Zheng [35], images addressed with the GBT approach are highly amenable to a fast Fourier transform since the number of pixels is always a power of 7. Fast algorithms for computing the HDFT using spiral addressing have

been developed [12], [35], [41] that require O(N log N) operations. However, the spiral addressing techniques employ radix-7 decimation rather than radix-2 decimation, resulting in a slight speedup. As shown above, the ASA HFFT requires N log2 N complex multiplications; Middleton [12] shows that the HIP HFFT requires N log7 N complex multiplications. Therefore the ASA HFFT requires about log2 7 ≈ 2.8 times more operations than the HIP HFFT. 4.1.3 The Hexagonal Discrete-Space Fourier Transform (HDSFT)

The other form of the Fourier transform that uses 2D discrete input is the discrete- space Fourier transform (DSFT). The DSFT transforms a complex valued function of discrete variables into a complex valued function of real variables. Assume f is the

hexagonally sampled version of a complex valued function of real variables, fr , Z Z ∞ ∞ h ³ a´ ³√ ´ ³ a´i f (a, r, c) = fr (x, y)δ x − c + , y − 3 r + dxdy, (4–12) −∞ −∞ 2 2

where δ(i, j) is the two-dimensional Dirac delta function (2D Dirac comb). Equation 4–12 can be rewritten as

f (a, r, c) =f (0, r, c) + f (1, r, c) Z Z ∞ ∞ √ = fr (x, y)δ(x − c, y − 3r)dxdy −∞Z −∞Z · µ ¶ µ ¶¸ ∞ ∞ 1 √ 1 + fr (x, y)δ x − c + , y − 3 r + dxdy. −∞ −∞ 2 2

Taking the DSFT yields

F (ω1, ω2) =F0(ω1, ω2) + F1(ω1, ω2) X∞ X∞ h ³ √ ´i = f (0, r, c) exp −j ω1c + ω2 3r r=−∞ c=−∞

79 · µ µ ¶ µ ¶¶¸ X∞ X∞ 1 √ 1 + f (1, r, c) exp −j ω c + + ω 3 r + 1 2 2 2 r=−∞ c=−∞ X∞ X∞ h ³ √ ´i = f (0, r, c) exp −j ω1c + ω2 3r r=−∞ c=−∞ " à √ !# ω ω 3 X∞ X∞ h ³ √ ´i + exp −j 1 + 2 f (1, r, c) exp −j ω c + ω 3r , 2 2 1 2 r=−∞ c=−∞ (4–13)

the hexagonal discrete-space Fourier transform (HDSFT).

4.2 The Z Transform

The Z transform is a more general version of the discrete-time Fourier transform (DTFT). Like the DTFT, the Z transform transforms a complex valued function of a discrete variable into a complex valued function of a real variable. The difference is that the DTFT exists only on the unit circle (or unit sphere in 2D) in the frequency domain, whereas the Z transform is defined on the entire frequency space. The Z transform is the Laplace transform of a continuous-time signal with z = exp(sT ), where s is the frequency variable and T is the sampling frequency (in 1D). This fact can be used to derive the Z transform for a hexagonally sampled 2D signal. Assume fq is the

continuous-time representation of the hexagonally sampled function of real variables, fr ,

X1 X∞ X∞ h ³ a´ ³√ ´ ³ a´i f (x, y) = f (x, y)δ x − c + , y − 3 r + , q r 2 2 a=0 r=−∞ c=−∞

where δ(i, j) is a 2D Dirac comb. Taking the Laplace transform of fq(x, y) results in

Z ∞ Z ∞ Fq(s1, s2) = fq(x, y) exp [− (s1x + s2y)] dxdy −∞ −∞ Z Z ∞ ∞ X1 X∞ X∞ h ³ a´ ³√ ´ ³ a´i = f (x, y)δ x − c + , y − 3 r + × r 2 2 −∞ −∞ a=0 r=−∞ c=−∞

× exp [− (s1x + s2y)] dxdy Z Z X1 X∞ X∞ ∞ ∞ h ³ a´ ³√ ´ ³ a´i = f (a, r, c) δ x − c + , y − 3 r + × 2 2 a=0 r=−∞ c=−∞ −∞ −∞

80 × exp [− (s1x + s2y)] dxdy X1 X∞ X∞ h ³ ³ a´ √ ³ a´´i = f (a, r, c) exp − s c + + s 3 r + , 1 2 2 2 a=0 r=−∞ c=−∞

where f (a, r, c) is the set of discrete samples of fr (x, y). Let z1 = exp (s1) and z2 = ¡ √ ¢ exp s2 3 , then

1 ∞ ∞ X X X a a −(c+ 2 ) −(r+ 2 ) Fz (z1, z2) = Fq(s1, s2) = f (a, r, c)z1 z2 . (4–14) a=0 r=−∞ c=−∞

Equation 4–14 is the hexagonal z-transform. Since the z-transform is a generalization

of the DSFT, the compatibility of this result with that of Section 4.1.3 can be verified

by evaluating Equation 4–14 on the unit sphere, i.e. with z1 = exp (jω1) and z2 = ¡ √ ¢ exp jω2 3 , and comparing the result to Equation 4–13,

³ ³ √ ´´ F (ω1, ω2) =Fz exp (jω1) , exp jω2 3

X1 X∞ X∞ h ³ a´i h √ ³ a´i = f (a, r, c) exp −jω c + exp −jω 3 r + 1 2 2 2 a=0 r=−∞ c=−∞ X∞ X∞ h ³ √ ´i = f (0, r, c) exp −j ω1c + ω2 3r r=−∞ c=−∞ · µ µ ¶ µ ¶¶¸ X∞ X∞ 1 √ 1 + f (1, r, c) exp −j ω c + + ω 3 r + 1 2 2 2 r=−∞ c=−∞ X∞ X∞ h ³ √ ´i = f (0, r, c) exp −j ω1c + ω2 3r r=−∞ c=−∞ " à √ !# ω ω 3 X∞ X∞ h ³ √ ´i + exp −j 1 + 2 f (1, r, c) exp −j ω c + ω 3r . 2 2 1 2 r=−∞ c=−∞

4.3 Wavelet Transforms

The wavelet transform is a mathematical representation of a signal by a set of scaled and translated wavelets, oscillating functions with finite extent. It is similar to the

Fourier transform, but because of the finite extent of the wavelet, frequency components

of the signal can be localized in time/space (depending on the dimensionality of the

81 signal). The Fourier transform, in contrast, uses infinite extent basis functions (sines and cosines) which disallow the localization of the signal’s frequency components, unless windowing techniques are employed. The discrete wavelet transform (DWT) is the

practical implementation of the wavelet transform, which is also known as multiresolution or multi-scale processing. The wavelet transform is a relatively new tool that has been

successfully used to improve a wide range of signal processing applications [26]. The DWTs used most commonly for rectangularly sampled 2D signals use separable filter banks [26], which simplify implementation. Separable filter banks consist of a low-pass filter, two high-pass filters that are equivalent after a 90 degree rotation, and a third high-pass filter that covers the remaining disjoint regions of the

frequency space [55], as shown on the left side of Figure 4-9. The sub-band resulting

from this third high-pass filter contains mixed orientations [56] and therefore does not have a straightforward directional interpretation [26]. The sub-band is considered to

be “troublesome” [55] and a deficiency of separable implementations of the DWT [26]. In contrast, filter banks designed for computing the DWT of hexagonally sampled images are generally not separable and can take advantage of the higher symmetry of the hexagonal grid. Hexagonal DWT filter banks consist of a low-pass filter and three high-pass filters that are equivalent after rotations of 120 degrees and 240 degrees, as shown on the right side of Figure 4-9. Since all of the high-pass sub-bands have straightforward directional interpretations, it seems hexagonal DWT filter banks can

be designed to have better properties than any possible filter bank designed for the rectangular grid [55].

As pointed out by Middleton [12] and Jiang [27], research in the area of hexagonal wavelets has been slow. However, there have been some researchers that have addressed the design of DWT filter banks for hexagonally sampled images. Early

efforts [57]–[59] focused on the notion of the image pyramid with fairly simple filtering, which led to perfect reconstruction transforms that were overcomplete and lacked

82 Figure 4-9. Idealized Wavelet Transform Regions of Support (Hexagonal vs. Rectangular).

orientation information [60]. Quadrature mirror filters (QMFs) provide oriented partitions and perfect reconstruction while producing codes that are the same size as the original

image [60], and several researchers [56], [60]–[62] have used QMFs to design filter banks for image pyramids on the hexagonal lattice. Another approach by Allen [55], [63] was based on a novel block structure that led to filter banks with certain symmetry [27]. More recently, Jiang has published several papers [27]–[29] on the design of hexagonal filter banks with various orthogonality, symmetry, and refinement properties. Due to the availability of designs and design methods for hexagonal filter banks, filter bank design using ASA was not pursued.

A typical DWT (regardless of sampling geometry) proceeds by applying each filter of a filter bank to the signal then downsampling (refining) the filtered images as

shown in Figure 4-10. This is called the analysis step, which is recursively applied to the low-pass sub-band and leads to the coefficients of the wavelet transform. The inverse of the analysis step is called the synthesis step, which proceeds by upsampling each

of the lowest level sub-bands, filtering the upsampled sub-bands with the synthesis

filter bank filters, and summing the results to produce the next higher level low-pass

83 sub-band as shown in Figure 4-11. Note that the number of filters in the filter bank and the refinement have the following relationship: if there are n filters in the filter bank, a √ √ n refinement is produced by downsampling. The hexagonal grid allows 3, dyadic, √ and 7 refinements [29]. Of the three, a dyadic refinement is the most appropriate for hexagonal grids that are represented using ASA since the downsampled image can also be addressed directly with ASA. The other two refinements require rotations of the coordinate system at each refinement level. Dyadic refinement also parallels what is typically done with rectangular DWTs.

Figure 4-10. Wavelet Transform Analysis Filter Bank.

A highly desirable characteristic of DWTs is known as perfect reconstruction (PR), where the synthesis step perfectly reconstructs the input to the analysis step from the DWT coefficients. There are two types of filter banks that lead to PR wavelets, these are known as orthogonal and bi-orthogonal. The orthogonal filter banks use the same filters (except for a possible rotation) for both the analysis and synthesis steps, whereas the

bi-orthogonal filter banks use different filters for the two steps. A much more complete,

detailed, and mathematical explanation can be found in Strang’s book [64].

84 Figure 4-11. Wavelet Transform Synthesis Filter Bank.

The implementation of hexagonal DWTs using ASA is straightforward and efficient. The filters are all applied using convolution which was described and shown to be efficient in Section 3.1. Downsampling in ASA is described in Section 3.4, however the anti-aliasing filter has been replaced by the filter bank. Upsampling for the synthesis step is just the exact opposite of downsampling. As an example, the orthogonal PR filterbank published by Allen [55] was represented in ASA and used to perform a

DWT on the ASA addressed, hexagonally sampled image shown in Figure 4-12. The decomposition (wavelet coefficients image) after three iterations is shown in Figure 4-13. The coefficients where then fed into the synthesis step using the same filter bank (rotated by 180◦) and the original image was recovered. Perfect reconstruction was verified by taking the root mean square error of the reconstructed image, which was

−16 Erms = 2.8x10 for this example. The total runtime for both the analysis and synthesis steps of a Matlab implementation of this example averaged 0.5017 seconds with a standard deviation of 0.0077 seconds over 100 runs. In comparison, the total runtime for both steps of a Matlab implementation of the Cohen-Daubechies-Feauveau (CDF)

85 9/7 wavelet on the same image averaged 0.5484 seconds with a standard deviation of 0.0080 seconds over 100 runs. The CDF 9/7 wavelet filter banks are similar in size to the ASA implementation of the Allen filter banks.

Figure 4-12. Lilies Image (Hexagonally Sampled).

In order to compare an ASA DWT implementation to a HIP DWT implementation, the two component operations, convolution and downsampling/upsampling, are compared. Assume an N-point input image and filter banks consisting of m-point filters. An ASA convolution can be computed as

mX−1 J(x) = I (x ⊕ ni )M(C ⊕ ni ) i=0 for each pixel address, x ∈ ASA, where I is the input image, M is the filter, J is the filtered image, ⊕ depicts ASA addition, C ∈ ASA is the address of the center pixel of the

filter, and ni ∈ ASA are the offsets to the neighboring pixels covered by the filter. So an ASA convolution with a single filter requires mN multiplications, (m − 1)N additions, and

86 Figure 4-13. Hexagonal Discrete Wavelet Transform of Lilies Image.

2mN ASA additions. Since all of these are O(1) operations, ASA convolution requires 4mN − N operations per filter. As discussed above, dyadic refinements are the natural choice for ASA addressed images so there are a total of 4 filters in an ASA DWT filter

N bank and the downsampling step requires the selection of 4 pixels per filter. A complete decomposition requires blog4 Nc stages with the number of image pixels being reduced by 4 at each stage. Therefore, ASA requires

blog Nc−1 µ ¶ blog Nc−1 µ ¶ µ ¶ X4 4mN − N N X4 N 3 (4) + = 4m − (4–15) 4i (4)(4i ) 4i−1 4 i=0 i=0 operations for a complete decomposition. Similarly, a HIP convolution can be computed as mX−1 J(x) = I (x ⊕ ni )M(0 ⊕ ni ) i=0 for each pixel address, x ∈ HIP, where I is the input image, M is the filter, J is the

filtered image, ⊕ depicts HIP addition, 0 ∈ HIP is the address of the center pixel of the

87 filter, and ni ∈ HIP are the offsets to the neighboring pixels covered by the filter. So a HIP convolution with a single filter requires mN multiplications, (m − 1)N additions,

and 2mN HIP additions. As shown in Section 2.8, HIP addition is a O((log N)2)

2 operation. Therefore, HIP convolution requires 2mN − N + 2mN(log7 N) operations √ per filter. Because of the hierarchical nature of HIP addressing, a 7 refinement is the natural choice for a HIP DWT, so there is a total of 7 filters in a HIP DWT filter bank

N and the downsampling step requires the selection of 7 pixels per filter. A complete

decomposition requires blog7 Nc stages with the number of image pixels being reduced by 7 at each stage. Therefore, HIP requires

blog Nc−1 µ ¶ Ã µ ¶ ! blog Nc−1 ¡ ¡ ¢ ¢ X7 N N 2 X7 N 2m 1 + (log N − i)2 − 1 (7) 2m − 1 + 2m log = 7 7i 7 7i 7i−1 i=0 i=0 (4–16)

operations for a complete decomposition. Comparing the expressions in Equations 4–15 and 4–16, one can see that an ASA-based DWT is an O(N) operation and a HIP-based DWT is an O(N(log N)2) operation, making the ASA-based DWT significantly faster. Note that the synthesis step of the DWT should have the same complexity as the analysis step for both coordinate systems.

88 CHAPTER 5 QUANTITATIVE COMPARISON USING REAL IMAGES

The trapezoidal imager described in Section 1.5.2 and shown in Figure 5-1 was used to conduct an experiment that was designed to concretely demonstrate the advantages of hexagonal sampling using a real sensor. Recall that the trapezoidal imager has the unique ability to simultaneously sample the exact same scene both rectangularly and hexagonally. The experiment was conducted by presenting a series of radial sine waves to the imager, with the frequency of the sine wave increasing as the experiment progressed. The reason for choosing a radial sine wave was that the magnitude of its Fourier transform is a circle in Fourier space with the radius of the circle being proportional to the frequency of the sine wave. Figure 5-2 shows an example image of a radial sine wave and the magnitude of its Fourier transform. This characteristic allows one to visually determine the frequency at which aliasing begins to occur (when the circles begin to overlap in Fourier space). The data was collected by displaying the input images on an LCD monitor with the sensor mounted solidly in front of the monitor. One laptop was used to display the images and another was used to control the sensor and collect the sensor’s output.

Figure 5-1. Trapezoidal Imager. Image Courtesy of Centeye, Inc.

89 Figure 5-2. Example of a radial sine wave (left) and its Fourier transform (right).

The collected data set was processed to produce both rectangularly and hexagonal- ly sampled images from each trapezoidal image by appropriately summing groups of four trapezoidal units to form the individual pixels. Each image was then cropped down to the center 128 × 128 image to maintain square images with side lengths equal to a power of two. The Fourier transform of the resultant images was calculated and plotted side-by-side. Recall that both images have the same number of pixels and are imaging the same scene, the only difference is the sampling geometry. The results clearly show that the hexagonally sampled images have a higher bandwidth than the rectangularly sampled images. Figure 5-3 shows a few of the frames from the image sequence. The frequency at which the circles in Fourier space begin to touch is the onset of aliasing and hence the bandwidth of the sampling approach. Notice that the ratio of bandwidths √ is very close to the theoretical limit, 0.268/0.309 ≈ 0.867 ≈ 3/2 ≈ 0.866. Therefore, this experiment demonstrates that one can realize the full benefit of hexagonal sampling with real imagers and improve bandwidth by approximately 13.4%.

90 91

Figure 5-3. Experimental results from the trapezoidal imager. Frequency (F) is given in radians per pixel. CHAPTER 6 CONCLUSION

6.1 Summary of Accomplishments

Array set addressing (ASA) is a novel coordinate system for addressing the points of a hexagonal grid. The following fundamentals of ASA have been developed:

1. Coordinate definition

2. Efficient coordinate conversion between ASA and Cartesian space

3. Definition of distance measures using ASA

4. Definition of basic ASA vector operations (addition, negation, subtraction, scalar multiplication)

5. Proof that ASA satisfies the 8 properties of a Z-module

6. Definition of efficient and compact digital memory storage of ASA images. Additionally, several basic image processing techniques have been developed for ASA and shown to be efficient. A method for performing convolution on an ASA image was developed and shown to require the exact same number of operations as a convolution on a rectangularly sampled image for the same size convolution mask. However, it was also shown that hexagonal neighborhoods are actually smaller than rectangular ones, so a convolution mask for a neighborhood operation is smaller in ASA representation than its rectangular counterpart, making convolution in ASA faster than rectangular convolution. The speedup is further improved since an image can be represented in ASA with 13.4% fewer pixels than in rectangular coordinates. The convolution masks for local gradient estimation were derived directly using a least-squares fit of a plane to the image intensity values in a local hexagonal neighborhood. Since local gradient estimation is a neighborhood operation and it can be performed via convolution, gradient estimation in ASA is faster than it would be using rectangular coordinates. It was also shown that the performance of common edge detection techniques is the same, or slightly better in some cases, using ASA than when using rectangular representation.

92 Additionally, since edge dection techniques rely heavily on gradient estimation and convolution, the operation should be faster in ASA. A method for image downsampling, used to create image pyramids for multi-scale operations, was developed and shown to be efficient. ASA was compared to HIP addressing (a spiral addressing approach) using several basic operations and shown to be significantly faster. The hexagonal discrete Fourier transform (HDFT) was reformulated using ASA coordinates and shown to have a separable kernel with ASA. A hexagonal fast Fourier transform (HFFT) was developed and shown to require fewer than twice the number of operations required for a standard radix-2 decimation-in-space FFT performed on a rectangularly sampled image of the same size. Equations for both the hexagonal discrete space Fourier transform (HDSFT) and the hexagonal Z-transform were developed in ASA coordinates.

It was shown that discrete wavelet transforms (DWTs) can be efficiently performed on hexagonally sampled images using ASA. And, because of the higher symmetry of the hexagonal grid, DWT filter banks for hexagonally sampled images can be designed that perform better than filter banks designed for rectangularly sampled images. Lastly, an experiment was conducted using a real imaging sensor which demonstrated that hexagonal sampling leads to an approximately 13.4% improvement in bandwidth over rectangular sampling in practice.

This document has introduced the array set addressing approach to addressing hexagonally sampled images. The advantages of hexagonal sampling for digital images have been presented and the shortcomings of the existing addressing methods have been pointed out. It has been shown that ASA can be used to perform common image processing tasks on hexagonally sampled images with no more complexity than would be required for a rectangularly sampled image. The separability of the Fourier kernel in ASA coordinates corroborates the elegance of this solution for the processing of hexagonally sampled digital images. With this efficient and convenient addressing

93 scheme, the advantages of hexagonal sampling for digital images can now be fully realized.

6.2 Significance of Accomplishments

The impact that the introduction of an efficient addressing scheme for hexagonal grids will have on the world of digital imaging is unclear. Since hexagonal sampling is the optimal approach for isotropically band-limited images, and assuming that the lack of an efficient addressing scheme has kept companies from using hexagonal

sampling in their products in the past, one would expect the introduction of ASA to have a huge impact. However, it will be difficult to overcome the inertia that rectangular

sampling has after decades of use. Fortunately, digital imaging devices (e.g. cameras and displays) typically have a relatively short life span before obsolescence, so a new technology could theoretically take root and become the norm fairly rapidly. ASA is already amenable to processing on standard processors due to the fact that the data structure is comprised of arrays, leaving only the sensors and displays to be redesigned to accommodate hexagonal sampling. However, it could also be the case that the inertia of rectangular sampling is already too great and the promising advantages of hexagonal

sampling fail to convince companies to make the switch. Regardless, it is hoped that ASA is at least embraced by other researchers and used to build an even stronger case for the use of hexagonal sampling in digital imaging applications.

6.3 Final Thought - Rectangular Component Conjecture

Three-dimensional (3-D) imaging is important for various applications such as medical imaging. The optimal sampling lattice (for isotropically band-limited signals) in n- D space is considered to be the dual of the tightest possible packing of hyperspheres in n-D Fourier space. For example, in 3-D the tightest possible packing of spheres results from the face-centered cubic (FCC) lattice. Its dual is the body-centered cubic (BCC) lattice, which is therefore the optimal sampling lattice in 3-D space. The advantage of sampling on the optimal lattice in 3-D is even more dramatic than in 2-D, providing an

94 approximately 30% improvement in sampling efficiency over rectangular sampling [65]. Note that the BCC lattice can be represented by a set of two rectangular components (3-D Cartesian grids) and the FCC lattice can be represented by four rectangular components. Therefore, the same idea that is exploited by the ASA method can be applied to 3-D imaging as well. The lattice points of both the BCC and FCC lattices can be addressed by using two binary and three integer coordinates, i.e. (b0, b1, i, j, k) ∈ {0, 1} × {0, 1} × Z × Z × Z (note that the BCC lattice is completely contained in one of the binary dimensions). It is suspected that this 3-D version of ASA addressing will prove to be a convenient and efficient coordinate system for processing optimally sampled 3-D data. Extrapolating this idea to even higher dimensions leads to the following conjecture:

The optimal sampling lattice in n-D space and its dual in Fourier space (for n- D spaces that have a known optimal sampling lattice) can be represented by

the superposition of, at most, 2dlog2 ne rectangular components (n-D Cartesian

grids). Therefore, a coordinate system which uses dlog2 ne binary coordinates and n integer coordinates can be used to efficiently address the lattice points and process the data.

95 APPENDIX A PROOFS OF ASA Z-MODULE PROPERTIES

A.1 Preliminaries

Let dk = k mod 2 for k ∈ N.

A.1.1 Domains

pi , q ∈ ASA

k, j ∈ N

A, B, C ∈ {0, 1}

A.1.2 Identities ¥ ¦ k k−dk 1. 2 = 2 2. (Ak) mod 2 = A ∧ k mod 2 3. (k mod 2 + j mod 2) mod 2 = (k + j) mod 2

4. (k + j) mod 2 = dk ⊕ dj

5. (kj) mod 2 = dk ∧ dj 6. A ⊕ B = (A + B) − (2)(A ∧ B) 7. (A ∧ B) ⊕ (A ∧ C) = A ∧ (B ⊕ C) 8. A ∧ B = (A)(B) A.2 Commutativity of Addition

Theorem A.1. If pi ∈ ASA, then p1 + p2 = p2 + p1.

Proof.      a1 ⊕ a2   a2 ⊕ a1          p1 + p2 =  r1 + r2 + (a1 ∧ a2)  =  r2 + r1 + (a2 ∧ a1)  = p2 + p1.     c1 + c2 + (a1 ∧ a2) c2 + c1 + (a2 ∧ a1)

96 A.3 Identity Element of Addition

Theorem A.2. If p ∈ ASA, then p + 0 = p for 0 = (0, 0, 0)T ∈ ASA.

Proof.          a   0   a ⊕ 0   a                  p + 0 =  r  +  0  =  r + 0 + (a ∧ 0)  =  r  = p.         c 0 c + 0 + (a ∧ 0) c

A.4 Inverse Elements of Addition

Theorem A.3. If p ∈ ASA, then p + q = 0 for some q ∈ ASA, where 0 = (0, 0, 0)T ∈ ASA.

Proof. Let    a      p =  r    c then,    a      q =  −r − a    −c − a and,        a ⊕ a   0   0              p + q =  r + (−r − a) + (a ∧ a)  =  r + (−r − a) + a  =  0  .       c + (−c − a) + (a ∧ a) c + (−c − a) + a 0

A.5 Distributivity of Scalar Multiplication (w.r.t. Vector Addition)

Theorem A.4. If p, q ∈ ASA and k ∈ N, then k (p + q) = kp + kq.

97 Proof.    a1 ⊕ a2      k(p1 + p2) = (k)  r1 + r2 + (a1 ∧ a2)    c1 + c2 + (a1 ∧ a2)    ((a1 ⊕ a2)(k)) mod 2   ¥ ¦  =  k  (A–1)  kr1 + kr2 + (k)(a1 ∧ a2) + (a1 ⊕ a2) 2   ¥ ¦  k kc1 + kc2 + (k)(a1 ∧ a2) + (a1 ⊕ a2) 2

and      (ka1) mod 2   (ka2) mod 2   ¥ ¦   ¥ ¦  k k kp1 + kp2 =   +    kr1 + (a1) 2   kr2 + (a2) 2   ¥ ¦   ¥ ¦  k k kc1 + (a1) 2 kc2 + (a2) 2    ((ka1) mod 2) ⊕ ((ka2) mod 2)   ¥ ¦  =  k  . (A–2)  kr1 + kr2 + (a1 + a2) 2 + ((ka1) mod 2 ∧ (ka2) mod 2)   ¥ ¦  k kc1 + kc2 + (a1 + a2)) 2 + ((ka1) mod 2 ∧ (ka2) mod 2)

The equality of Equations A–1 and A–2 is proven by considering the coordinates separately. First,

((a1 ⊕ a2)(k)) mod 2 = (a1 ⊕ a2) ∧ (k mod 2) (Identity 2)

= ((k mod 2) ∧ a1) ⊕ ((k mod 2) ∧ a2) (Identity 7)

= ((ka1) mod 2) ⊕ ((ka2) mod 2) (Identity 2).

Second, ¥ ¦ k (k)(a1 ∧ a2) + (a1 ⊕ a2) 2 =

98 ¥ ¦ k (k)(a1 ∧ a2) + ((a1 + a2) − (2)(a1 ∧ a2)) 2 ) (Identity 6) ¥ ¦ ¥ ¦ k k = (a1 + a2) 2 + (k − (2) 2 )(a1 ∧ a2) ¥ ¦ k = (a1 + a2) 2 + (dk )(a1 ∧ a2) (Identity 1) ¥ ¦ k = (a1 + a2) 2 + (a1 ∧ k mod 2) ∧ (a2 ∧ k mod 2) ¥ ¦ k = (a1 + a2) 2 + ((ka1) mod 2) ∧ ((ka2) mod 2). (Identity 2) Thus ¹ º ¹ º k k kr + kr + (k)(a ∧ a ) + (a ⊕ a ) =kr + kr + (a + a ) 1 2 1 2 1 2 2 1 2 1 2 2

+ ((ka1) mod 2 ∧ (ka2) mod 2).

Finally, c1, c2 can be substituted for r1, r2 yielding ¹ º ¹ º k k kc + kc + (k)(a ∧ a ) + (a ⊕ a ) =kc + kc + (a + a ) 1 2 1 2 1 2 2 1 2 1 2 2

+ ((ka1) mod 2 ∧ (ka2) mod 2).

A.6 Identity Element of Scalar Multiplication

Theorem A.5. If p ∈ ASA, then 1p = p.

Proof.      (a(1)) mod 2   a   ¥ ¦    1p =  1  =   = p.  (1)r + (a) 2   r   ¥ ¦    1 (1)c + (a) 2 c

99 APPENDIX B PYTHON SOURCE CODE FOR ASA TO HIP RUNTIME COMPARISON import time import math import random import array import scipy from Hexint import Hexint tha = array.array(’f’) taa = array.array(’f’) ths = array.array(’f’) tas = array.array(’f’) thd = array.array(’f’) tad = array.array(’f’) thn = array.array(’f’) tan = array.array(’f’) thm = array.array(’f’) tam = array.array(’f’) thc = array.array(’f’) tac = array.array(’f’) tac_ab = array.array(’f’) tac_hb = array.array(’f’) thc2 = array.array(’f’) tac2 = array.array(’f’) e = 3 n = 10000 for i in range(0,n):

d1 = random.randint(0,7**e) d2 = random.randint(0,7**e)

digits = "0123456" s1 = "" while 1: r = d1 % 7 s1 = digits[r] + s1 d1 = d1 / 7 if d1 == 0: break

s2 = ""

100 while 1: r = d2 % 7 s2 = digits[r] + s2 d2 = d2 / 7 if d2 == 0: break h1 = Hexint(int(s1)) h2 = Hexint(int(s2)) a1 = (random.randint(0,1),random.randint(0,7**e),random.randint(0,7**e)) a2 = (random.randint(0,1),random.randint(0,7**e),random.randint(0,7**e))

# compare addition # HIP t1 = time.clock() h3 = h1+h2 t2 = time.clock() tha.append((t2-t1)*1000000.0)

# ASA t1 = time.clock() z = a1[0]&a2[0] a3 = ((a1[0]+a2[0]) % 2,a1[1]+a2[1]+z,a1[2]+a2[2]+z) t2 = time.clock() taa.append((t2-t1)*1000000.0)

# compare subtraction # HIP t1 = time.clock() h3 = h1-h2 t2 = time.clock() ths.append((t2-t1)*1000000.0)

# ASA t1 = time.clock() a2neg = (a2[0],-a2[1]-a2[0],-a2[2]-a2[0]) z = a1[0]&a2neg[0] a3 = ((a1[0]+a2neg[0]) % 2,a1[1]+a2neg[1]+z,a1[2]+a2neg[2]+z) t2 = time.clock() tas.append((t2-t1)*1000000.0)

# compare Euclidean Distance # HIP t1 = time.clock()

101 r1 = h1.getReal() r2 = h2.getReal() dh = math.sqrt((r1[0]-r2[0])**2 + (r1[1]-r2[1])**2) t2 = time.clock() thd.append((t2-t1)*1000000.0)

# ASA t1 = time.clock() da = math.sqrt((1.0/3)*((a1[0]-a2[0])+(2)*(a1[2]-a2[2]))**2 + \ ((a1[0]-a2[0])+(2)*(a1[1]-a2[1]))**2) t2 = time.clock() tad.append((t2-t1)*1000000.0)

# compare Finding Neighbors # HIP x1 = Hexint(1) x2 = Hexint(2) x3 = Hexint(3) x4 = Hexint(4) x5 = Hexint(5) x6 = Hexint(6) t1 = time.clock() n1 = h1+x1 n2 = h1+x2 n3 = h1+x3 n4 = h1+x4 n5 = h1+x5 n6 = h1+x6 t2 = time.clock() thn.append((t2-t1)*1000000.0)

# ASA a = a1[0] r = a1[1] c = a1[2] t1 = time.clock() n1 = (a,r,c+1) n2 = (1-a,r-(1-a),c+a) n3 = (1-a,r-(1-a),c-(1-a)) n4 = (a,r,c-1) n5 = (1-a,r+a,c-(1-a)) n6 = (1-a,r+a,c+a) t2 = time.clock()

102 tan.append((t2-t1)*1000000.0)

# compare scalar multiplication # HIP k=random.randint(1,7**e) t1 = time.clock() h3 = k*h2 t2 = time.clock() thm.append((t2-t1)*1000000.0)

# ASA k=random.randint(1,7**e) t1 = time.clock() a3 = ((a1[0]*k) % 2,k*a1[1]+math.floor(k/2)*a1[0],\ k*a1[2]+math.floor(k/2)*a1[0]) t2 = time.clock() tam.append((t2-t1)*1000000.0)

# compare conversion from Cartesian # HIP x=random.randint(1,7**e) y=random.randint(1,7**e) t1 = time.clock() h1.getNearest(x,y) t2 = time.clock() thc.append((t2-t1)*1000000.0)

# ASA - Yang Bin t1 = time.clock() xs = math.sqrt(3)*x ys = y xt = math.floor(xs) yt = math.floor(ys) m = 1/3 if bool(xt%2)^bool(yt%2): yp = yt+(1/2) xp = xt+(1/2) line = (ys-yp)-m*(xs-xp) if line <= 0: xt = xt+1 else: yt = yt+1 else: yp = yt+(1/2)

103 xp = xt+(1/2) line = (ys-yp)+m*(xs-xp) if line >= 0: xt = xt+1 yt = yt+1 a = yt%2 r = (yt-a)/2 c = (xt-a)/2 t2 = time.clock() tac.append((t2-t1)*1000000.0)

# ASA - ASA Bin t1 = time.clock() sr3 = math.sqrt(3) pi = math.pi p = round(sr3*x) q = round(y) if bool(p%2)^bool(q%2): if q-y>=0: theta = math.atan2(y-(q-1/3),x-p/sr3) if (theta > -5*pi/6) and (theta <= -pi/6): q = q-1 elif (theta > -pi/6) and (theta <= pi/2): p = p+1 else: p = p-1 else: theta = math.atan2(y-(q+1/3),x-p/sr3) if (theta > -pi/2) and (theta <= pi/6): p = p+1 elif (theta > pi/6) and (theta <= 5*pi/6): q = q+1 else: p = p-1 a = p%2 r = (p-a)/2 c = (q-a)/2 t2 = time.clock() tac_ab.append((t2-t1)*1000000.0)

# ASA - Hybrid Bin t1 = time.clock() xs = math.sqrt(3)*x ys = y

104 xr = round(xs) yr = round(ys) m = 1/3 if bool(xr%2)^bool(yr%2): if xs >= xr: if ys > yr: yp = yr + (1/3) line = (ys-yp)-m*(xs-xr) if line <= 0: xr = xr+1 else: yr = yr+1 else: yp = yr - (1/3) line = (ys-yp)+m*(xs-xr) if line >= 0: xr = xr+1 else: yr = yr-1 else: if ys > yr: yp = yr + (1/3) line = (ys-yp)+m*(xs-xr) if line <= 0: xr = xr-1 else: yr = yr+1 else: yp = yr - (1/3) line = (ys-yp)-m*(xs-xr) if line >= 0: xr = xr-1 else: yr = yr-1 a = yr%2 r = (yr-a)/2 c = (xr-a)/2 t2 = time.clock() tac_hb.append((t2-t1)*1000000.0)

# compare conversion to Cartesian # HIP t1 = time.clock() x,y = h1.getReal() t2 = time.clock()

105 thc2.append((t2-t1)*1000000.0)

# ASA t1 = time.clock() x = (1/math.sqrt(3))*(a1[0]+2*a1[2]) y = a1[0]+2*a1[1] t2 = time.clock() tac2.append((t2-t1)*1000000.0) print ’\nAddress (Vector) Addition’ print ’HIP takes %0.5f us (std = %0.5f)’ \ % (scipy.mean(tha), scipy.std(tha)) print ’ASA takes %0.5f us (std = %0.5f)’ \ % (scipy.mean(taa), scipy.std(taa)) print ’Runtime Ratio (HIP/ASA): %2.2f’ \ % (scipy.mean(tha)/scipy.mean(taa)) print ’\nAddress (Vector) Subtraction’ print ’HIP takes %0.5f us (std = %0.5f)’ \ % (scipy.mean(ths), scipy.std(ths)) print ’ASA takes %0.5f us (std = %0.5f)’ \ % (scipy.mean(tas), scipy.std(tas)) print ’Runtime Ratio (HIP/ASA): %2.2f’ \ % (scipy.mean(ths)/scipy.mean(tas)) print ’\nCalculate Euclidean Distance’ print ’HIP takes %0.5f us (std = %0.5f)’ \ % (scipy.mean(thd), scipy.std(thd)) print ’ASA takes %0.5f us (std = %0.5f)’ \ % (scipy.mean(tad), scipy.std(tad)) print ’Runtime Ratio (HIP/ASA): %2.2f’ \ % (scipy.mean(thd)/scipy.mean(tad)) print ’\nCalculate All Six Nearest Neighbor Addresses’ print ’HIP takes %0.5f us (std = %0.5f)’ \ % (scipy.mean(thn), scipy.std(thn)) print ’ASA takes %0.5f us (std = %0.5f)’ \ % (scipy.mean(tan), scipy.std(tan)) print ’Runtime Ratio (HIP/ASA): %2.2f’ \ % (scipy.mean(thn)/scipy.mean(tan)) print ’\nScalar Multiplication’ print ’HIP takes %0.5f us (std = %0.5f)’ \ % (scipy.mean(thm), scipy.std(thm))

106 print ’ASA takes %0.5f us (std = %0.5f)’ \ % (scipy.mean(tam), scipy.std(tam)) print ’Runtime Ratio (HIP/ASA): %2.2f’ \ % (scipy.mean(thm)/scipy.mean(tam)) print ’\nConversion from Cartesian - Yang Bin’ print ’HIP takes %0.5f us (std = %0.5f)’ \ % (scipy.mean(thc), scipy.std(thc)) print ’ASA takes %0.5f us (std = %0.5f)’ \ % (scipy.mean(tac), scipy.std(tac)) print ’Runtime Ratio (HIP/ASA): %2.2f’ \ % (scipy.mean(thc)/scipy.mean(tac)) print ’\nConversion from Cartesian - ASA Bin’ print ’HIP takes %0.5f us (std = %0.5f)’ \ % (scipy.mean(thc), scipy.std(thc)) print ’ASA takes %0.5f us (std = %0.5f)’ \ % (scipy.mean(tac_ab), scipy.std(tac_ab)) print ’Runtime Ratio (HIP/ASA): %2.2f’ \ % (scipy.mean(thc)/scipy.mean(tac_ab)) print ’\nConversion from Cartesian - Hybrid Bin’ print ’HIP takes %0.5f us (std = %0.5f)’ \ % (scipy.mean(thc), scipy.std(thc)) print ’ASA takes %0.5f us (std = %0.5f)’ \ % (scipy.mean(tac_hb), scipy.std(tac_hb)) print ’Runtime Ratio (HIP/ASA): %2.2f’ \ % (scipy.mean(thc)/scipy.mean(tac_hb)) print ’\nConversion to Cartesian’ print ’HIP takes %0.5f us (std = %0.5f)’ \ % (scipy.mean(thc2), scipy.std(thc2)) print ’ASA takes %0.5f us (std = %0.5f)’ \ % (scipy.mean(tac2), scipy.std(tac2)) print ’Runtime Ratio (HIP/ASA): %2.2f’ \ % (scipy.mean(thc2)/scipy.mean(tac2))

107 REFERENCES [1] J. Crettez, “A pseudo-cosine transform for hexagonal tessellation with an heptarchical organization,” in Proc. IEEE Int. Conf. on Pattern Recognition and Image Processing, 1980, pp. 192–194.

[2] C. A. Curcio, J. K. R. Sloan, O. Packer, A. E. Hendrickson, and R. E. Kalina, “Distribution of cones in human and monkey retina: Individual variability and radial asymmetry,” Science, vol. 236, no. 4801, pp. 579–582, 1987. [3] D. P. Petersen and D. Middleton, “Sampling and reconstruction of wave-number- limited functions in n-dimensional Euclidean spaces,” Inf. Control, vol. 5, no. 4, pp. 279–323, Dec. 1962.

[4] B. H. McCormick, “The Illinois pattern recognition computer: ILLIAC III,” IEEE Trans. Electron. Comput., vol. EC-12, no. 6, pp. 791–813, Dec. 1963. [5] M. Golay, “Hexagonal parallel pattern transformations,” IEEE Trans. Comput., vol. C-18, no. 8, pp. 733–740, Aug. 1969.

[6] E. S. Deutsch, “Thinning algorithms on rectangular, hexagonal, and triangular arrays,” Comm. ACM, vol. 15, no. 9, 1972. [7] R. M. Mersereau, “The processing of hexagonally sampled two-dimensional signals,” Proc. IEEE, vol. 67, no. 6, pp. 930–949, Jun. 1979. [8] P. J. Burt, “Tree and pyramid structures for coding hexagonally sampled binary images,” Comput. Graph. Image Process., vol. 14, pp. 271–280, 1980.

[9] J. W. van Roessel, “Conversion of Cartesian coordinates from and to generalized balanced ternary addresses,” Photogramm. Eng. Remote Sens., vol. 54, no. 11, pp. 1565–1570, 1988. [10] D. Lucas and L. Gibson, “A system for hierarchical addressing in Euclidean space,” Jan. 1980, interactive Systems Corporation. [11] L. Gibson and C. Lenzmeier, “A hierarchical pattern extraction system for hexagonally sampled images,” Interactive Systems Corporation, Tech. Rep., 1981, final Report for Contract F49620-81-C-0039, U.S. Air Force Office of Scientific Research.

[12] L. Middleton and J. Sivaswamy, Hexagonal Image Processing: A Practical Ap- proach. London: Springer, 2005. [13] P. Sheridan, “Spiral architecture for machine vision,” Ph.D. dissertation, Univ. of Technology Sydney, 1996.

[14] I. Her, “A symmetrical coordinate frame on the hexagonal grid for computer graphics and vision,” J. Mech. Des., vol. 115, no. 3, pp. 447–449, 1993.

108 [15] W. E. Snyder, H. Qi, and W. Sander, “A coordinate system for hexagonal pixels,” in Proc. SPIE Medical Imaging: Image Processing, vol. 3661, 1999, pp. 716–727.

[16] X. He, H. Wang, N. Hur, W. Jia, Q. Wu, J. Kim, and T. Hintz, “Uniformly partitioning images on virtual hexagonal structure,” in Proc. 9th Int. Conf. Control, Automation, Robotics and Vision. Los Alamitos, CA: IEEE Computer Society Press, 2006, pp. 891–896. [17] X. He, W. Jia, N. Hur, Q. Wu, and J. Kim, “Image translation and rotation on hexagonal structure,” in Proc. 6th Int. Conf. Computer and Information Technology, Seoul, Korea, Dec. 2006, pp. 141–146.

[18] X. He, W. Jia, Q. Wu, and T. Hintz, “Description of the cardiac movement using hexagonal image structures,” Comput. Med. Imag. Grap., vol. 30, no. 6-7, pp. 377–382, 2006.

[19] X. He, J. Li, and T. Hintz, “Comparison of image conversions between square structure and hexagonal structure,” Lect. Notes Comput. Sci., vol. 4678, pp. 262–273, 2007.

[20] L. Condat, D. Van De Ville, and M. Unser, “Efficient reconstruction of hexagonally sampled data using three-directional box-splines,” in Proc. IEEE Int. Conf. Image Processing, Atlanta, US, Oct. 2006. [21] L. Condat and D. Van De Ville, “Three-directional box-splines: Characterization and efficient evaluation,” IEEE Signal Process. Lett., vol. 13, no. 7, pp. 417–420, Jul. 2006.

[22] ——, “New optimized spline functions for interpolation on the hexagonal lattice,” in Proc. IEEE Int. Conf. Image Processing, San Diego, CA, Dec. 2008, pp. 1256–1259. [23] L. Condat, B. Forster-Heinlein, and D. Van De Ville, “H2O: Reversible hexagonal-orthogonal grid conversion by 1-d filtering,” in Proc. IEEE Int. Conf. Image Processing, vol. 2, San Antonio, TX, Sep. 2007, pp. II–73 – II–76.

[24] L. Condat, D. Van De Ville, and B. Forster-Heinlein, “Reversible, fast and high-quality grid conversions,” IEEE Trans. Image Process., vol. 17, no. 5, pp. 679–693, May 2008. [25] L. Condat and D. Van De Ville, “Fully reversible image rotation by 1-D filtering,” in Proc. IEEE Int. Conf. Image Processing, San Diego, CA, Dec. 2008, pp. 913–916.

[26] L. Condat, B. Forster-Heinlein, and D. Van De Ville, “A new family of rotation-covariant wavelets on the hexagonal lattice,” in Proc. SPIE Optics Pho- tonics 2007 Conf. Mathematical Methods: Wavelet XII, vol. 6701, San Diego, CA, Aug. 2007, pp. 67 010B–1/67 010B–9.

109 [27] Q. T. Jiang, “FIR filter banks for hexagonal data processing,” IEEE Trans. Image Process., vol. 17, no. 9, pp. 1512–1521, Sep. 2008.

[28] ——, “Orthogonal and biorthogonal FIR hexagonal filter banks with sixfold symmetry,” IEEE Trans. Signal Process., vol. 52, no. 12, pp. 5861–5873, Dec. 2008. √ [29] ——, “Orthogonal and biorthogonal 3-refinement wavelets for hexagonal data processing,” IEEE Trans. Signal Process., vol. 57, no. 11, pp. 4304–4313, Nov. 2009. [30] F. Faille and M. Petrou, “Bio-inspired, invariant image reconstruction from irregularly placed samples,” in Proc. BMVC 2008, Leeds, UK, Sep. 2008.

[31] B. Gardiner, S. Coleman, and B. Scotney, “A design procedure for gradient operators on hexagonal images,” in Proc. Irish Machine Vision and Image Pro- cessing Conf., Kildare, Ireland, Sep. 2007, pp. 47–54. [32] ——, “Multi-scale feature extraction in a sub-pixel virtual hexagonal environment,” in Proc. Irish Machine Vision and Image Processing Conf., 2008, pp. 47–54.

[33] M. Puschel¨ and M. Rotteler,¨ “Algebraic signal processing theory: 2-D hexagonal spatial lattice,” IEEE Trans. Image Process., vol. 16, no. 6, pp. 1506–1521, 2007. [34] ——, “Algebraic signal processing theory: Cooley-Tukey type algorithms on the 2-D hexagonal spatial lattice,” Appl. Algebra Eng. Comm. Comput., vol. 19, no. 3, pp. 259–292, 2008. [35] A. Vince and X. Zheng, “Computing the discrete Fourier transform on a hexagonal lattice,” J. Math. Img. Vis., vol. 28, no. 2, pp. 125–133, 2007.

[36] K. S. Yang, “Toward refinements of spatial smoothers: Attending to details in computationally intensive settings,” Ph.D. dissertation, George Mason Univ., 1999. [37] J. R. D. Barbour, “Reduction of complexity in path finding using grid based methods,” Master’s thesis, Univ. of Regina, 2008. [38] R. C. Boyd, “Generalized Euclidean line-of-sight on regular hexagonal grids,” Master’s thesis, Univ. of Colorodo Denver, 2008.

[39] M. Guenette, “Triangulation of a hierarchical hexagon mesh,” Master’s thesis, Queen’s Univ., 2009. [40] G. Zhou, “Fourier analysis and truncation error estimates of multigrid methods and conservative Jacobians on hexagonal grids,” Ph.D. dissertation, Clarkson Univ., 2009.

[41] X. Zheng, “Efficient Fourier transforms on hexagonal arrays,” Ph.D. dissertation, Univ. of Florida, 2007.

110 [42] C. Mead, Analog VLSI and Neural Systems. Reading, MA: Addison-Wesley, 1989.

[43] R. Hauschild, B. J. Hosticka, S. Muller,¨ and M. Schwarz, “A CMOS optical sensor system performing image sampling on a hexagonal grid,” in Proc. 22nd European Solid-State Circuits Conf., 1996, pp. 304–307.

[44] A. P. Fitz and R. J. Green, “Fingerprint classification using a hexagonal fast Fourier transform,” Pattern Recogn., vol. 29, no. 10, pp. 1587–1597, 1996.

[45] P. Halasz. (2008) Pixel geometries. Wikimedia Commons. (accessed January 5, 2010). [Online]. Available: http://commons.wikimedia.org/wiki/File: Pixel geometry 01 Pengo.jpg

[46] E. Luczak and A. Rosenfeld, “Distance on a hexagonal grid,” IEEE Trans. Comput., vol. 25, pp. 532–533, 1976. [47] N. I. Rummelt and J. N. Wilson, “Array set addressing: making the world safe for hexagonal imaging,” in Proc. SPIE/IS&T Electronic Imaging, vol. 7532, 2010, p. 75320D.

[48] J. M. S. Prewitt, “Object enhancement and extraction,” in Picture Processing and Psychopictorics, B. S. Lipkin and A. Rosenfeld, Eds. New York: Academic Press, 1970. [49] N. Otsu, “A threshold selection method from gray level histograms,” IEEE Trans. Syst., Man, Cybern., vol. SMC-9, pp. 62–66, 1979. [50] D. C. Marr and E. Hildreth, “Theory of edge detection,” Proc. Roy. Soc. Lond., vol. B207, pp. 187–217, 1980. [51] J. F. Canny, “A computational approach to edge detection,” IEEE Trans. Pattern Anal. Mach. Intell., vol. PAMI-8, pp. 679–697, 1986.

[52] Y. Morikawa, H. Hamada, and N. Yamane, “Separation of kernel in the hexagonal discrete Fourier transform,” Electron. Comm. Jpn., vol. 64-A, no. 7, pp. 16–24, Jul. 1981. [53] U. R. Alim and T. Moller,¨ “A discrete Fourier transform for the hexagonal and body-centered cubic lattices,” SFU Computing Science, Tech. Rep., 2008.

[54] A. V. Oppenheim and R. W. Schafer, Digital Signal Processing. Prentice-Hall, 1975. [55] J. D. Allen, “Perfect reconstruction filter banks for the hexagonal grid,” in Proc. 5th Int. Conf. Information, Communications. and Signal Processing, Dec. 2005, pp. 73–76.

[56] E. Simoncelli and E. Adelson, “Non-separable extensions of quadrature mirror filters to multiple dimensions,” Proc. IEEE, vol. 78, no. 4, pp. 652–664, Apr. 1990.

111 [57] S. Tanimoto and T. Pavlidis, “A hierarchical data structure for picture processing,” Comput. Graph. Image Process., vol. 4, pp. 104–119, 1975.

[58] P. J. Burt, “Fast filter transforms for image processing,” Comput. Graph. Image Process., vol. 16, pp. 20–51, 1981. [59] A. B. Watson, “Ideal shrinking and expansion of discrete sequences,” NASA, Tech. Memorandum 88202, Jan. 1986.

[60] A. B. Watson and J. A. J. Ahumada, “A hexagonal orthogonal-oriented pyramid as a model of image representation in visual cortex,” IEEE Trans. Biomed. Eng., vol. 36, no. 1, pp. 97–106, Jan. 1989.

[61] E. A. Adelson, E. Simoncelli, and R. Hingorani, “Orthogonal pyramid transforms for image coding,” SPIE Vis. Commun. Image Process., vol. 845, pp. 50–58, 1987. [62] X. H, W. S. Lu, and A. Antoniou, “A new design of 2-D non-separable hexagonal quadrature-mirror-filter banks,” in Proc. CCECE, Vancouver, BC, Canada, Sep. 1993.

[63] J. D. Allen, “Coding transforms for the hexagonal grid,” Ricoh Calif. Research Ctr., Menlo Park, CA, Tech. Rept. CRC-TR-9851, Aug. 1998. [64] G. Strang and T. Nguyen, Wavelets and Filter Banks. Wellesley-Cambridge Press, 1996. [65] A. Entezari, “Optimal sampling lattices and trivariate box splines,” Ph.D. dissertation, Simon Fraser University, Vancouver, Canada, July 2007.

112 BIOGRAPHICAL SKETCH Nicholas I. Rummelt was born in Michigan and attended Michigan State University (MSU) where he received a Bachelor of Science degree in computer engineering. After graduation, he began working for the Air Force in Dayton, Ohio, but soon returned to MSU to get his Master of Science degree in electrical engineering. Returning to the Air Force, but seeking a change of scenery, he eventually found a position with the Air Force Research Laboratory in Florida, where he continues to work.

113