A General Scheme for Dithering Multidimensional Signals, and a Visual Instance of Encoding Images with Limited Palettes

Journal of King Saud University – Computer and Information Sciences (2013) xxx, xxx–xxx

King Saud University Journal of King Saud University – Computer and Information Sciences www.ksu.edu.sa www.sciencedirect.com

A General scheme for dithering multidimensional signals, and a visual instance of encoding images with limited palettes

Mohamed Attia a,b,c,*,1,2, Waleed Nazih d,3, Mohamed Al-Badrashiny e,4, Hamed Elsimary d,3 a The Engineering Company for the Development of Computer Systems, RDI, Giza, Egypt b Luxor Technology Inc., Oakville, Ontario L6L6V2, Canada c Arab Academy for Science & Technology (AAST), Heliopolis Campus, Cairo, Egypt d College of Computer Engineering and Sciences, Salman bin Abdulaziz University, AlKharj, Saudi Arabia e King Abdul-Aziz City for Science and Technology (KACST), Riyadh, Saudi Arabia

Received 12 March 2013; revised 30 August 2013; accepted 5 December 2013

KEYWORDS Abstract The core contribution of this paper is to introduce a general neat scheme based on soft Digital signal processing; vector clustering for the dithering of multidimensional signals that works in any space of arbitrary Digital image processing; dimensionality, on arbitrary number and distribution of quantization centroids, and with a comput- Dithering; able and controllable quantization noise. Dithering upon the digitization of one-dimensional and Multidimensional signals; multi-dimensional signals disperses the quantization noise over the frequency domain which renders Quantization noise; it less perceptible by signal processing systems including the human cognitive ones, so it has a very Soft vector clustering beneﬁcial impact on vital domains such as communications, control, machine-learning, etc. Our extensive surveys have concluded that the published literature is missing such a neat dithering scheme. It is very desirable and insightful to visualize the behavior of our multidimensional dithering scheme; especially the dispersion of quantization noise over the frequency domain. In general, such visualization would be quite hard to achieve and perceive by the reader unless the target multidimensional signal itself is directly perceivable by humans. So, we chose to apply our

* Corresponding author at: Luxor Technology Inc., Oakville, Ontario, Canada. Tel.: +1 6475349166. E-mail addresses: [email protected], [email protected] (M. Attia). 1 www.RDI-eg.com. 2 www.AAST.edu. 3 www.sau.edu.sa. 4 www.KACST.edu.sa. Peer review under responsibility of King Saud University.

Production and hosting by Elsevier

multidimensional dithering scheme upon encoding true-color images – that are 3D signals – with palettes of limited sets of colors to show how it minimizes the visual distortions – esp. contouring effect – in the encoded images. ª 2013 King Saud University. Production and hosting by Elsevier B.V. All rights reserved.

1. Introduction for human perception; e.g. digital audio, or may turn into a serious source of error if the digitized signal is forwarded to The main contribution of this paper is to introduce a general some further processing; e.g., machine learning, control neat scheme for the dithering of multidimensional signals that systems, etc. is able to deal with arbitrary dimensionality, arbitrary number For example, consider an audio signal of a single tone – i.e. and distribution of quantization centroids, and with comput- a purely sinusoidal wave – at 500 Hz. In the frequency domain, able and controllable noise power. In order to proceed with this analog signal shows a single impulse at 500 Hz and noth- presenting this novel multidimensional dithering scheme, it is ing elsewhere. When, this audio signal is digitized via 16-bit 16 necessary first to formally review one-dimensional signal quantization; i.e. 2 = 65,536 quantum levels, the resulting digitization, quantization noise, and dithering. digital signal in the frequency domain seems (almost) the same The digitization of an analog one dimensional signal – as the original analog one as illustrated by the blue curve at the known as Analog-to-Digital (‘‘A-to-D’’ or ‘‘A2D’’) conversion top of Fig. 2. On the other hand, when the same audio signal is 6 – aims at mapping any given sample of the signal within its digitized via 6-bit quantization; i.e. 2 = 64 quantum levels, the resulting digital signal in the frequency domain shows a dynamic range qmin 6 q 6 qmax to one element of a pre-defined major peak at 500 Hz but also other considerable harmonics set of quantum levels {c1, c2, ..., ci, ..., cL}; qmin 6 ci 6 qmax,- L P 2. In order to minimize the digitization error, this like the one around 4500 Hz as illustrated by the red curve mapping is typically done through the minimum-distance crite- at the middle of Fig. 2. These obtrusive harmonics mean that rion; i.e. the signal sample is mapped to the nearest quantum the digitized signal is not corresponding any more to a pure level, which can be formulated as follows: single tone, but is corresponding to a composite one where irritating false whistles are superimposed on the original pure A-to-D q ! i0 : i0 ¼ arg min fdðq; ckÞg; ð1Þ tone (Pohlmann, 2005). 8k;1kL Researchers and engineers had realized since decades that where d(q1, q2) is any legitimate distance criterion between this problem is caused by the concentration of the digitization 1 q1; q2 2 R . The digitization of a given signal sample in the errors within narrow bands of the signal, and has accordingly 1D space is reduced into a simple selection of one of – at most realized that dispersing the digitization errors over wider bands – the two quantum levels enclosing that signal sample in the frequency domain would produce a better digitized (Roberts, 2007; Widrow and Kolla´r, 2008) as illustrated by signal where obtrusive artifacts are less conspicuous. With Fig. 1 below. signals digitized this way, humans would perceive a better qual- The sum of the squared digitization errors of all the emerg- ity, and digital signal processing systems would perform more ing signal samples make the quantization noise which is formu- robustly. Dispersing the digitization errors over wider bands is lated as follows (Roberts, 2007; Widrow and Kolla´r, 2008): typically achieved through adding controlled noise to the X X 2 2 2 analog signal just before the A-to-D conversion (Petri, 1996; E ¼ e ðqÞ¼ ðq ci Þ : ð2Þ q 8q q 8q 0 Schuchman, 1964). This process is popularly known as The distribution of the set of quantum levels over the ‘‘dithering’’ whose simplest – and also most commonly used dynamic range of the signal may be regular that – variant adds to each analog signal sample q some ± random qmaxqmin value whose amplitude is half the distance between the two ci ¼ qmin þði 1Þ L and is then called regular quantization. When the distribution of emerging signal samples to be enclosing quantum levels ci and ci1. Digitization with this kind of dithering may be formulated as follows: digitized is significantly irregular, the distribution of the quan- no tum levels may be designed to track that irregular one of ci1 ci ci ci1 5 i0 ¼ argmin dqþ rand ; ; ck : ð3Þ emerging samples, and is then called adaptive quantization. 8k:i1ki 2 2 Adaptive quantization aims at minimizing the quantization noise for any given number L of quantum levels (Roberts, Digitization with dithering of a given signal sample as 2007; Widrow and Kolla´r, 2008). described by Eq. (3) is still a selection of one of the two quan- Increasing L obviously decreases both the digitization er- tum levels enclosing that signal sample; however, unlike Eq. (1) rors and quantization noise; however, there are hardware this selection is a stochastic process where the chances of and/or computational cost limitations on the size of L to be de- attributing the sample to each of the two quantum levels are ployed in a given digitization scheme. When L is not large en- given by: ough to adequately capture the resolution of the analog signal, Dithering c q Dithering Pq!i 1 ¼ i ;Pq!i the digitized signal suffers from obtrusive artifacts that render c c i i1 its information content into a significantly distorted version Dithering q c i1 from that carried by the original analog signal. This may turn ¼ 1 Pq !i 1 ¼ : ð4Þ ci ci1 into a serious drop of quality if the digitized signal is destined The quantization noise of digitizing some population of signal 5 The distribution of the quantum levels in Fig. 1 is assumed to samples with dithering is obviously larger than that of the belong to this second kind of quantization. quantization noise of the same population without dithering.

Figure 2 Dithering for the dispersion of quantization noise over the frequency domain.

This increase of quantization noise is the price paid to disperse the quantization noise over wider frequency bands (Petri, 1996; Schuchman, 1964). Applying this dithering model to the digitization of our Figure 1 Dithering in 1D space; only the two enclosing quantum exemplar single-tone audio signal via 6-bit quantization, the levels compete for the given point. resulting digital signal in the frequency domain – illustrated

Please cite this article in press as: Attia, M. et al., A General scheme for dithering multidimensional signals, and a visual instance of encoding images with limited palettes. Journal of King Saud University – Computer and Information Sciences (2013), http://dx.doi.org/10.1016/j.jksuci.2013.12.002 4 M. Attia et al. at the bottom of Fig. 2 – shows white noise all over the signal spectrum and shows also some obtrusive harmonics but now with significantly lower amplitudes than those resulting from the digitization via 6-bit quantization without dithering. Apparently, the total quantization noise power in the white noise plus the obtrusive harmonics has increased; however, this noise has also been much more dispersed over the frequency spectrum. With the whiz of the white noise plus some much fainter obtrusive whistles, the original tone is much clearly identified in the digitized audio via 6-bit quantization with dithering than in the same audio digitized via 6-bit quantization without dithering where higher obtrusive whistles are irritatingly obscuring the original tone (Vanderkooy and Lipshitz, 1987). Multidimensional signals – which arise in countless ad- vanced modern applications of vital fields like control, communications, electronics, machine learning, image processing...etc. – need also be digitized before being digitally processed, and dithering is then an indispensible operation for alleviating the vagarities of digitization errors especially when Figure 3 Dithering in two dimensional (or multi dimensional) multidimensional signals are digitized via limited sets of space with regular quantization; only the quantum levels on the ‘‘quantum points’’6 that are not large enough for capturing corners of the enclosing rectangle (or hyper rectangular prism) the multidimensional resolution of these signals. compete for the given point. So, the next section discusses why the dithering of multidimensional signals is qualitatively more challenging than that of one-dimensional signals. Then, Section 2 proceeds to present our general neat scheme of dithering signals in any space of tion with dithering of q that stochastically attributes it to one of the corners of its enclosing rectangle ci;j as follows: arbitrary dimensionality, and with arbitrary number and dis- 0 0 c c c c tribution of quantization centroids. Section 3 provides a quan- 1i1 1i 1i 1i1 i0 ¼ argmin dq1 þ rand 2 ; 2 ; c1k titative analysis of the quantization noise due to our dithering 8k:i1kino c c c c : ð5Þ scheme compared to that resulting from quantization without 2j1 2j 2j 2j1 j0 ¼ argmin dq2 þ rand 2 ; 2 ; c2k dithering. 8k:j1kj It is very desirable and insightful to visualize the behavior This procedure is extensible and applicable to any regular of our multidimensional dithering scheme; especially the dis- quantization setup in a space of any number of dimensions persion of quantization noise over the frequency domain. In to attribute q ¼ðq ; q ...q Þ2RD; D P 1 to one of the 2D general, such visualization would be quite hard to achieve 1 2 D corners of its enclosing hyper rectangular prism. and perceive by the reader unless the target multidimensional In such a regular quantization setup, dividing the dynamic signal itself is directly perceivable by humans. While sound sig- range of the signal over each dimension into (‘ 1) intervals nals are one-dimensional, images are three-dimensional ones d requires an overall number of centroids that is equal to: and make an ideal – and actually rare – instance for the sought YD visualization. As a means of visualizing the dynamic behavior D of our novel multidimensional dithering scheme, we apply this L ¼ ‘d; ‘d2f1;2:::Dg 2 ) L ¼ ‘ ; ‘d2f1;2:::Dg ¼ ‘: ð6Þ scheme upon encoding true-color images with palettes of lim- d¼1 ited sets of colors to show how it minimizes the visual distor- This formula of exponential nature is prohibitively expensive tions – esp. contouring effect – in the encoded images. This with high dimensionality which is very common in real-life application is discussed in detail in Section 4. Finally, Section 5 applications that need to digitize vectors of up to several presents and discusses the comparative results of our experi- tens of components; e.g. machine-learning feature-vectors. mentation with this application to a catalogue of miscellaneous If, for example, we have to digitize a vectorial signal in a true-color images. 20-dimensional space via a regular quantization setup where each dimension is to be modestly divided into 3 intervals, an 20 12 2. The dithering of multidimensional signals immense number of 4 1.1 · 10 centroids would be needed! Due to their hyper-linear computational complexity Consider Fig. 3 where the set of centroids in 2D space – or in patterns, real-life signal processing systems tend to deal with multidimensional space in general – are regularly distributed small sets of centroids and can at most deal with a couple- over the dynamic range of the signal in each dimension. of-thousand centroids. Therefore, regular quantization Given an emerging signal sample q ¼ðq ; q Þ2R2 – 1 2 schemes are seldom deployed and adaptive quantization located inside the rectangle whose four corners are schemes are instead resorted to. Vector clustering algorithms; c ; c ; c ; c that all belong to R2 – to be digitized i1;j1 i;j1 i1;j i;j e.g. k-means and LBG,7 are used to obtain an optimum set within this regular quantization setup, it is straightforward of centroids that are adaptive to the multidimensional signal to augment Eq. (3) to two-dimensional space for the digitiza-

7 LBG is the acronym denoting the famous vector clustering 6 A quantization point in a multidimensional space is called centroid. algorithm developed by Linde, Buzo, and Gray (Linde et al., 1980).

The function f(di) must obey the following conditions:

1. f ðdiÞ0 8di 0; 2. f(di) is continuous "di P 0, 3. f(di) is a monotonically decreasing function "di P 0, Soft VQ Soft VQ 4. di ¼ 0 ) P q ! i ¼ 1 ^ P q ! k–i ¼ 0:

In addition to satisfying the four conditions mentioned above, it is much desirable for the design of the function f(x) to have the following properties:

Figure 4 Dithering in two dimensional (or multi dimensional) 1. Simplicity. space with irregular/adaptive quantization: all the quantum levels 2. Tuning parameters to control the probability vanishing compete for the given point. speed with increasing distance. 3. Analytic computability of the quantization noise energy of the resulting Soft VQ with respect to that of Hard VQ. to be digitized (Linde et al., 1980; Gray, 1984; Gray and Neu- hoff, 1998; Jain et al. 2008). Fig. 4 illustrates an exemplar While the third of these desirable properties is subject to a adaptive quantization setup: detailed discussion over the next section, we select for our dith- The hard-deciding (i.e. deterministic) minimum-distance ering scheme the inverse power-function that realizes all the criterion for the digitization of any given signal sample in a necessary conditions and the ﬁrst two desirable properties multidimensional space with any distribution of centroids; reg- above. Our f(x) is then deﬁned as: ular or irregular, can be simply formulated as the vectorial fðd Þ¼dm; m > 0: ð9Þ form of Eq. (1) as follows8: k k

Hard VQ q ! i0 : i0 ¼ argminfdðq; ckÞg: ð7Þ 8k;1kL 3. Quantization noise of Soft VQ based dithering On the other hand, the dithering formula of Eq. (5) in the Dithering disperses the quantization noise over the frequency case of multidimensional regular quantization cannot be spectrum of the digitized signal, which is a great gain for the per- applied to the general case of multidimensional adaptive quan- ceived signal quality and for the robustness of any subsequent tization because no enclosing rectangles (or ‘‘hyper rectangular digital signal processing as well. However, the price of this gain prisms’’) can be decisively identiﬁed within irregular adaptive is the increase of total quantization noise, which might ruin the quantization setups like the exemplar one illustrated in Fig. 4. perceived signal quality or volatilize the stability of subsequent In order to go around this hurdle, and to preserve the max- digital signal processing. Therefore, this section is devoted to a imum compatibility with the basic dithering scheme dissected quantitative analysis of the additional quantization noise due previously, our new dithering scheme for the general case of to our general dithering scheme of multidimensional signals in vector quantization in multidimensional spaces has to comply order to see how safe its application to such signals is. with the following three principal constraints: Digitized through Hard VQ formulated by Eq. (7) above, the local participation of each signal sample to quantization 1. As hyper rectangular prisms enclosing a given q 2 RD are noise energy is given by: absent in the general case, we allow all the centroids c RD to compete for replacing q in the digitized 2 2 j2f1;2;...;Lg 2 eHard VQ ¼ min ðdðq; ckÞÞ ¼ dmin: ð10Þ signal representation. 8k;1kL 2. This competition is arbitrated stochastically through what The total quantization noise energy due to the digitization we call Soft VQ so that: of a given population of signal samples of size N via Hard Soft VQ P Soft VQ 8i 2f1; 2; ...; Lg : P q ! i > 0 and L P q ! i ¼ 1. VQ is then given by Petri (1996) and Schuchman (1964): i¼1 X N 2 3. With respect to a given q, a more distant centroid ci1 has a EHard VQ ¼ min ðdðqn; ckÞÞ : ð11Þ lower chance of replacing q than a less distant centroid c ; n¼18k;1kL i2 Soft VQ Soft VQ Digitized through Soft VQ formulated by Eq. (8) above, the i.e. dðq; ci Þ > dðq; ci Þ)P q ! i1 < P q ! i2 . 1 2 local participation of each signal sample to quantization noise energy is given by: Putting together all these constraints, our general dithering P scheme based on Soft VQ can be formulated as: XL L 2 2 Soft VQ k¼1ðdk fðdkÞÞ eSoft VQ ¼ d P q ! k ¼ P : ð12Þ k L fðd Þ k¼1 k¼1 k 2 2 2 In Eq. (12): dk dmin ¼ðdðq; ci0 ÞÞ 8k; 1 k L is 8 The digitization of vectorial signal samples is known also as Vector weighted by probabilities P 0, which results together with Quantization or VQ for shorthand. Eqs. (10) and (11) into:

Please cite this article in press as: Attia, M. et al., A General scheme for dithering multidimensional signals, and a visual instance of encoding images with limited palettes. Journal of King Saud University – Computer and Information Sciences (2013), http://dx.doi.org/10.1016/j.jksuci.2013.12.002 6 M. Attia et al. e E 1=m 2 Soft VQ Soft VQ 2 L m 2 2 L m eHard VQ eSoft VQ ) 1 r rmax ) 1 rmax: b eHard VQ EHard VQ lim a ¼ ; lim rmax ¼ : ð22Þ L!1 m 2 L!1 m m 2 ð13Þ Otherwise ba is obtained numerically by finding the peak of The question is then to study how big the ratio r is, or at Eq. (16) constrained by Eq. (17) like the exemplar charts illus- least how big its upper-bound rmax could be. Our selection of trated in Fig. 5, Fig. 6, Fig. 7, and Fig. 8 below. inverse power-function probability distributions proves to be quite useful at simplifying this task by substituting Eq. (9) in Eq. (12) to get: 4. Encoding true-color images with a limited palette via Soft VQ P L 2m k¼1ðdk Þ eSoft VQ ¼ P ; ð14Þ As mentioned in Section 1 previously, dithering is a common L dm k¼1ð k Þ useful operation to accompany the fundamentally vital A-to- Then by substituting Eqs. (14) and (10) in Eq. (13), we get: D conversion of one-dimensional signals. Dithering upon P P digitizing multidimensional signals is also useful for the same L 2m L m eSoft VQ k¼1ðdk Þ k ðdk Þ reason it is useful upon the digitization of one-dimensional sig- r ¼ 2 eHard VQ d nals; as it disperses quantization noise over the frequency do- P min L m2 main which renders it less perceptible by signal processing ðdmin=dkÞ ¼ Pk¼1 : ð15Þ systems – including the human cognitive ones – embedded in L ðd =d Þm k¼1 min k countless number of applications in vital domains such as com- Eq. (15) can be re-written more conveniently as: munications, control, machine-learning... etc. P It is very desirable and insightful to visualize the behavior L m2 1 þ k¼1 a – k k i0 dmin ¼ di0 of our multidimensional dithering scheme; especially the dis- r ¼ P ; ak 6 1; ak 2½0; 1: ð16Þ L m 1 þ k¼1 a dk persion of quantization noise over the frequency domain. In – k k i0 general, such visualization would be quite hard to achieve Attia et al. (2010, 2011) investigate in detail the maximiza- and perceive by the reader unless the target multidimensional tion of Eq. (16) – whose full derivation is unfolded in Appen- signal itself is directly perceivable by humans. While sound sig- dix I at the end of this paper – and we present their findings as nals are one-dimensional, images are three-dimensional ones follows: and make an ideal – and actually rare – instance for the sought For m <2r can grow infinitely huge; i.e. rmax = 1 regard- visualization. As a means of visualizing the dynamic behavior less to the value of L. So, this dithering scheme should not be of our novel multidimensional dithering scheme, we apply this used in this range in order to avoid the risk of producing dev- scheme upon encoding true-color images with palettes of lim- astating noise in the digitized signal. ited sets of colors to show how it minimizes the visual distor- For m =2 rmax = L which may be a proper operating tions – esp. contouring effect – in the encoded images. point for a small number of centroids; e.g. L =2 or L =3, Color images are typically modeled as a three-dimensional but turns to be risky for a large number of centroids; e.g. signal in some color space like RGB, CIE-XYZ, or CIE-Lab L = 256 or L = 1, 024 . (Hunt, 1998). The term ‘‘true-color image’’ denotes a digitized For m > 2 there is one and only one maximum of r = rmax image where each color dimension is independently divided that occurs at a ¼½ba1; ba2 ...bak ...baL which – according to into 255 intervals or more. Therefore, each pixel in such an im- (Attia et al., 2012, 2011) – are both obtained via: age is represented by 3 · log 2(255 + 1) = 24 bits or more. b b The classic problem in this regard is to represent such images akj8k–i ¼ a; ð17Þ 0 with an arbitrarily limited set of colors (L P 2) – called palette m 2 – with the minimum loss of quality (Dixit, 1991). r ¼ : ð18Þ max m ba2 Virtually all the up-to-date high-end digital displays can manipulate true-color images; however, this has never been ...where ba is the solution of the following polynomial: the case 15 or more years ago. However, there is still a need to m 1 m 2 1 deal with devices and setups with a limited – sometimes very lim- bam þ ba2 ¼ 0; 2 L 1 2 L 1 ited – color display capabilities that each pixel can only be switched to one of a small set of colors. Here are some examples m > 2; L 2; ba 2½0; 1: ð19Þ of such devices and setups that may still be in use on a wide scale: Only for m e {4, 6, 8, 10} this polynomial has closed-form exact solutions (Jacobson, 2009); for example: for m = 4 the 1. Displays of low-end of digital gadgets; e.g. watches, calcu- solution is: ffiffiffiffi lators, wireless/cell phones..., especially when such gadgets p are connected to the World Wide Web. b2 ffiffiffiffi1 L þ 1 a ¼ p ; rmaxjm 4 ¼ : ð20Þ 2. Printing devices with limited color capabilities such as m¼4 L þ 1 ¼ 2 monochrome printers, and fax machines. For m fi 1 regardless to the value of L Soft VQ 3. Transferring images over slow/very slow Internet approaches the behaviour of Hard VQ, and we get: connections. lim r ¼¼ 1: ð21Þ 4. Certain display modes of operating systems e.g. MS-Win- m!1 dows’ safe mode, certain image formats; e.g. GIF, fast For L fi 1 – which is interpreted in practice as L >>1; previewing of archived images; e.g. while browsing... etc. e.g. L = 256 or L = 512 ... etc. – we have the following excel- Such situations allow only a 16-color or 256-color palette- lent approximate solution that: oriented image display.

Figure 5 Quantization noise energy of our soft vector clustering relative to that of the hard clustering at two centroids (L = 2), and at various values of the power m (written in bold over each curve).

Figure 6 Quantization noise energy of our soft vector clustering relative to that of the hard clustering at four centroids (L = 4), and at various values of the power m (written in bold over each curve).

Figure 7 Quantization noise energy of our soft vector clustering relative to that of the hard clustering at eight centroids (L = 8), and at various values of the power m (written in bold over each curve).

Figure 8 Quantization noise energy of our soft vector clustering relative to that of the hard clustering at thirty two centroids (L = 32), and at various values of the power m (written in bold over each curve).

5. Sizeable mosaic and mosaic-like image compositions. et al., 1998); however, it is also the most mathematically sophis- 6. Old fashioned sizeable electronic ad boards serving in ticated and computationally demanding one, because it is not a places where it might be too expensive to replace them. mere dithering algorithm but is actually an intricate solution of a combinatorial cost minimization problem. True-color images suffer from loss of image quality when en- On the other hand, Fig. 9 depicts our much simpler solution coded via Hard VQ with a limited palette as the original colors of to this problem that is based on the direct application of our some image details are replaced by different ones in the palette. general dithering scheme presented in Section 3 of this paper. This may not only confuse the image colors, but may also obscure The input digital image is assumed to be a true-color image the very characteristic features of the image. Such an encoding in the RGB space which is the most common color space used with a predeﬁned palette produces more severe distortion than for the representation of digital images on electronic displays. the case of an adaptive palette of the same size. Moreover, for However, the Euclidean distance between two different points the same palette acquisition and encoding methods, smaller pal- (i.e. colors) in the RGB space does not correspond to the dif- ettes produce more severe image distortion than larger ones. ference in the human perception of these two colors (Hunt, The oldest method to address this issue is half-toning that 1998). So, the ﬁrst module M1 in our system converts the input has been invented and deployed in the pre-digital era to display image from the RGB color space to the CIE-Lab color space gray-scale analog photographs on old fashioned printed press where the Euclidean distance between two different color vec- produced via monochrome (i.e. black and white) printing. tors does correspond to the perceived visual difference of these Half-toning prints a dot with a black area proportional to two colors (Hunt, 1998). the darkness of the gray level of each point in the image; i.e. Module M2 then applies the LBG vector clustering algo- spatial resolution is traded for color resolution. rithm (Linde et al., 1980; Gray, 1984; Gray and Neuhoff, With the emergence of the digital age, more involved 1998) to the population of the CIE-Lab color vectors of the variants of half-toning have been developed to utilize the pixels in the output image from M1 in order to infer the opti- capabilities of monitors and digital graphic cards that can dis- mal palette of a given size L P 2. play multiple gray levels beyond the binary ones (Floyd and The CIE-Lab color vector of each pixel in the image is then Steinberg, 1976). New variants of image dithering for colored dithered via our Soft VQ with inverse power-function images then emerged with the prevalence of color monitors distributions detailed in Section 3 above. So, given a power and graphics cards with wider storage. Earlier such variants m P 2, the module M3 stochastically attributes each worked on each color dimension independently (Gentile CIE-Lab color vector q to one of the palette colors 6 6 et al., 1990), and then more sophisticated algorithms have been ci;1 i L with probabilities calculated according to Eqs. devised to simultaneously acquire the palette and perform the (8) and (9). image digitization so that the loss of image quality in the digi- Module M4 converts the output of M3 from the CIE-Lab tized image is minimal (Kollias and Anastassion, 1991; Flohr color space back into the RGB color space as the output of and T., 1993; Ketterer et al., 1998; Cheng et al., 2009). The algo- our system. The output of M4 is then a digital image encoded rithm that provides the best results in this regard is commonly with a limited palette that are compliant with digital image dis- known as scolorq (for Spatial Color Quantization) (Ketterer plays with any color depth P log2(L).

Figure 9 The overall structure of our system for optimally encoding true-color images with a limited palette.

Figure 10 Original true-color image (upper left corner), image encoded with a 16-color palette (upper right corner) via LBG clustering and hard-deciding nearest-distance VQ, image encoded via LBG and Soft VQ with m = 2 (lower left corner), and ﬁnally the image encoded via LB and Soft VQ with m = 3 (lower right corner).

Figure 11 Original true-color image (upper left corner), image encoded with a 16-color palette (upper right corner) via LBG clustering and hard-deciding nearest-distance VQ, image encoded via LBG and Soft VQ with m = 2 (lower left corner), and ﬁnally the image encoded via LB and Soft VQ with m = 3 (lower right corner).

Figure 12 Original true-color image (upper left corner), image encoded with a 16-color palette (upper right corner) via LBG clustering and hard-deciding nearest-distance VQ, image encoded via LBG and Soft VQ with m = 2 (lower left corner), and ﬁnally the image encoded via LB and Soft VQ with m = 3 (lower right corner).

Fig. 10 shows an illustrative example on the performance of noise within tight frequency bands as discussed in Section 1 our solution for encoding true-color images with a limited of this paper. Such a poor distribution does not only palette through Soft VQ with inverse power-function distribu- obscure the original content from the human viewers, but tions. The upper left corner of the ﬁgure shows the input also creates false details that were never present in the origi- original true-color photographic image. nal image. The upper right corner shows the result of encoding the The lower left and lower right corners of Fig. 10 show the original image via Hard VQ with an optimal (in the sense results of applying our dithering scheme based on Soft VQ of minimum quantization noise) 16-color palette obtained with (L = 16, m = 2) and (L = 16, m = 3) respectively. through the LBG vector clustering algorithm. A severe obtru- Obviously, the contouring effect has been alleviated at the sive price of higher quantization noise in both of these two images contouring effect9 is quite apparent as a manifestation of the than that of the image obtained via Hard VQ in accordance parasitic harmonics due to the concentration of quantization with Eq. (13). It is also apparent that the quantization noise with m = 3 is lower than that with m = 2 in accordance with Eq. (16). The visual quality with m = 3 is a more balanced

9 compromise of both alleviating the obtrusive effects and limit- Those false edges arising from abrupt transitions between disjoint ing the quantization noise compared with the two other cases color shades inadequately representing an area of a color gradient.

Figure 13 Original standard test-image of ‘‘Peppers’’. Figure 14 Original standard test-image of ‘‘Mandrill’’ (also known as ‘‘Baboon’’). in Fig. 10; the one at the upper right corner with too much 3D signals of true-color digital images but is also consistent contouring effect, and the one at the lower left corner with with multidimensional signals in general. too much quantization noise. Two more such examples like that of Fig. 10 are shown in 5. Experiments and assessment Fig. 11 and Fig. 12. This illustrates how our dithering scheme is working in action on this kind of 3D signals by trading sig- In order to learn in depth about its actual behavior, we have nal to quantization noise ratio for smoother distribution of applied our dithering scheme depicted in Fig. 9 above to a this noise over the frequency spectrum of the signal. This catalog containing tens of miscellaneous photos. For each behavior of algorithm is not speciﬁc to the self-visualizing photo; we archived the true-color photo, the one encoded with

Figure 15 Mandrill image represented by 8-color palette via LBG and our Soft-VQ based dithering with m = 2.5 (upper left corner), then via scolorq (upper right corner). Peppers image represented by 16-color palette via LBG and our Soft-VQ based dithering with m = 2.5 (lower left corner), then via scolorq (lower right corner).

Table 1 Mean Opinion Score (MOS) of the image quality of our color encoding compared to scolorq. Average MOS of original Average MOS of standard test Average MOS of standard standard test images images encoded via scolorq test images encoded via LBG vector clustering and Soft VQ 4.89 from 5 4.47 from 5 3.98 from 5

16-color palette via Hard VQ, and three ones encoded with 16- implement, and seems to be less computationally complex.12 color palette via Soft VQ with m e {2, 3, 4}.10 This archive is It has to be restated that the scope of the presented multidi- downloadable at http://www.rdi-e.g.com/downloads/Color- mensional dithering scheme is wider and more profound than Reduction-Project_2012.rar. Each processed image confirms the color-reduction problem that we used as an easy-to-visual- the behavior of our dithering algorithm as described in the ize instance of application, and the benefits of this dithering end of the previous section. We have also empirically found scheme to other vital domains as communications, control, that for 8 6 L 6 16 the most visually accepted image encoding machine-learning... etc. are great. via Soft VQ are obtained at m e [2.5, 2.8].11 The experimental results presented in what follows are 6. Conclusion meant only to measure how successful our dithering scheme is with dispersing the quantization noise of the 3D signals of The paper started with a quick formal review of dithering upon true-color images which is perceived by human viewers as alle- analog-to-digital (A-to-D) conversion of one-dimensional sig- viating the contouring effects. To put this measurement on a nals along with its benefits to the perceived quality of the dig- significant scale we compare the human judgments on the dith- itized signals. We then identified the difficulties of generalizing ered images via our scheme with both the original true-color the simple dithering schemes applied to the one-dimensional images and with the same images processed by scolorq signals to multidimensional signals. The paper proceeded with (Ketterer et al., 1998) which is the ultimate algorithm for the its main contribution of introducing a general neat scheme for representation of true-color images with a limited color the dithering of multidimensional signals that is able to deal palette. with arbitrary dimensionality, arbitrary number and distribu- With color images, scolorq goes much beyond than signal tion of quantization centroids, and with computable and con- dithering as it produces the optimal color-reduced images via trollable noise power. formally solving a combinatorial cost minimization problem In order to visualize the dynamic behavior of our presented in both the selection of the palette’s colors and simultaneously multidimensional dithering scheme, the paper then introduced the assignment of each pixel to one of these colors. Obviously, a visually perceivable instance of applying this scheme via this intricate optimization is computationally expensive. projecting it on the problem of encoding the 3D signals of So, even if the images processed our way realize compara- true-color images for optimal viewing – in the sense of mini- ble Mean-Opinion-Score (MOS), this would be a considerable mum visual distortion – on displays with limited sets of colors. visual indication that our multidimensional signal dithering We formally reviewed and chronologically surveyed the his- works so successfully at alleviating the contouring effect; i.e. tory of the significant approaches to tackle this problem along dispersing concentrated noise around narrow bands in the with the cons and pros of each. Then, the dithering scheme is 3D frequency domain, that the human viewers do not find a instantiated for this specific 3D signal processing problem and great quality difference between its output and the output of incorporated into the architecture of our new solution of this a quite elaborate technique as scolorq that takes care of many problem. This solution is then run on a variety of images so other aspects of color images. that the optimal operating settings are empirically determined. Therefore, we picked five of the most famous bench-mark- The experimental results are meant only to measure how ing images used to assess image processing algorithms to com- successful our dithering scheme is with dispersing the quantiza- pare our true-color image dithering with scolorq. While Fig. 13 tion noise of the 3D signals of true-color images which is per- and Fig. 14 show the true-color version of two of these five ceived by human viewers as alleviating the contouring effect. images, Fig. 15 shows the reduced color outputs of our image To put this measurement on a significant scale we compare dithering and of scolorq for both images at m = 2.5. the human judgments on the dithered images via our scheme We asked 13 volunteers independently to judge the with both the original true-color images and with the same visual quality of the output of both algorithms for each of images processed by scolorq which is the ultimate algorithm the 5 input bench-marking images on a scale of five marks for the representation of true-color images with a limited color {Very_Poor = 1, Poor = 2, Passable = 3, Good = 4, palette. Excellent = 5}. The average results are presented in Table 1. The results shown in the table suggest that the visual quality produced by scolorq are better – but not by far – than that 12 The inventors of scolorq (Ketterer et al., 1998) did not provide a produced by our method; however, we should also consider formula of the order of complexity of their sophisticated algorithm; that ours is much mathematically simpler to perceive and however, we estimate their full-fledged optimization approach produces some hyper-linear (yet sub-exponential) order of complexity in the product of the image size and the size of reduced colors set. On 10 Very few photos in this archive are encoded with L =8. the other hand, our dithering scheme tends to be linear in this product 11 For large palettes (e.g.L = 256; higher powers should be used to and might be reduced further through several optimizations in the preserve a reasonable rmax according to eq. No. (22). implementation.

So, even if the images processed our way realize less yet ...where s is the number of aj’s that are exactly equal to one. comparable mean-opinion-scores (MOS), this is a considerable This shows that the quantization noise energy of our proposed visual indication that our multidimensional signal dithering Soft VQ with the power m growing larger is approaching the works so successfully at alleviating the contouring effect – that one of the hard-deciding VQ; however its distributions are also is concentrated noise around narrow bands in the frequency turning less smooth and more similar to those of the hard- domain – that the human viewers does not ﬁnd a great quality deciding VQ. difference between its output and the output of a quite elabo- P L m2 rate technique as scolorq that takes care of many other aspects 1 þ j¼1 lima – j j i0 aj!0 1 þðL 1Þ0 of color images. lim r ¼ ¼ ¼ 1; ð27Þ 8a !0;j–i P j 0 L m 1 þðL 1Þ0 1 þ j¼1 lima – j j i0 aj!0 Acknowledgement

...which occurs only when q ¼ ci0 . The work presented in this paper has been achieved through the 12-month research project (27/A.H./1432) funded by the P L m2 1 þ j¼1 lima – j Deanship of Scientiﬁc Research at Salman bin Abdulaziz Uni- j i0 aj!1 1 þðL 1Þ1 lim r ¼ ¼ ¼ 1; ð28Þ versity www.sau.edu.sa, AlKharj, Kingdom of Saudi Arabia 8a !1 P j L m 1 þðL 1Þ1 1 þ j¼1 lima (KSA) from October 2011 up to September 2012. – j j i0 aj!1

6 6 ...which occurs when dðq; ciÞ is exactly the same "i;1 i L . Only for these three special cases r = 1, otherwise r >1.It Appendix I. is crucial to calculate the maximum value of r = rmax; i.e. the worst case, which – according to Eq. (13) presented in Section 3 The whole range of the positive power m in Eq. (16) – pre- above – is an upper bound of the ratio between the total quan- sented in Section 3 above – splits into three intervals with each tization noise energy of the proposed Soft VQ to that of the defining one of the following three cases: conventional hard VQ. For 0 < m < 2, it is obvious that the numerator of Eq. (16) To obtain rmax, the (L 1) dimensional region within b grows indefinitely faster than the denominator for arbitrarily aj–i0 2½0; 18j; 1 j L has to be searched for those aj–i0 infinitesimal values of some ak; k 2 X f1; 2; ...; Lg so that: where that maximum is produced. This is analytically achiev- able via solving the following set of (L - 1) equations: P @r=@akj – ¼ 0; 1 k L: ð29Þ m2 8k i0 1 þ nN þ 8k2X lim ak j0

In order for Eq. (31) to hold true, all ba – must be equal so L k i0 rj ¼ P ; 1 6 j 6 L; ð24Þ that: m¼2 L 2 1 þ j¼1 a – j j i0 ba ba; 33 kj8k–i0 ¼ ð Þ ...and one can easily guess that: ...which reduces formula No. (30) into: A ¼ 1 þðL 2Þbam2; B ¼ 1 þðL 2Þbam ) rmax ¼ maxðrj Þ¼ lim ðrj Þ¼L: ð25Þ m¼2 – m¼2 aj!08j i0;1jL A þ bam2 1 þðL 1Þbam2 m 2 r ¼ ¼ ¼ ba2 ð34Þ max B þ bam 1 þðL 1Þbam m For m > 2, the following three special cases of Eq. (16) can easily be noticed: Re-arranging the terms of (34), we get the polynomial equation: P L m2 m 1 m 2 1 1 þ j¼1 lim aj bm b2 j–i 1 þ s þ 0 a þ a ¼ 0; lim r ¼ P 0 m!1 ¼ ¼ 1; ð26Þ 2 L 1 2 L 1 m!1 L m 1 þ s þ 0 1 þ j¼1 lim a – j b j i0 m!1 m > 2; L 2; a 2½0; 1: ð35Þ

Please cite this article in press as: Attia, M. et al., A General scheme for dithering multidimensional signals, and a visual instance of encoding images with limited palettes. Journal of King Saud University – Computer and Information Sciences (2013), http://dx.doi.org/10.1016/j.jksuci.2013.12.002 A General scheme for dithering multidimensional signals, and a visual instance of encoding images 15 ! 1 1 For any m > 2, Eq. (35) can be shown to have one and only T3 2 L 2 ðL 1Þ one real solution in the interval ba 2½0; 1 through the following lim ¼ lim = ¼1; L!1 T1 L!1 m 2 m 2 three-step proof: 2=m b b2 1 T2 m 2 L 1- Put b ¼ a ; c ¼ L1, and re-write Eq. (35) lim ¼ = lim ¼ 0; L L b bm=2 m b m2 !1 T1 m 2 !1 m 2 as:gðbÞ¼b þ 2 c b 2 c ¼ 0: b 2- gðb ¼ 0Þ¼m2 1 < 0 2=m 2 L1 T2 m 2 L b 2L2þmmþ2 L lim ¼ = lim ¼ 0; L!1 L!1 gðb ¼ 1Þ¼ 2ðL1Þ ¼ L1 > 0 T3 m 2 m 2 ...which confirms the validity of Eq. (38) as an approximation ) g b has roots e [0, 1] ð Þ at L 1. b b m bm m 3- * dgðbÞ=dðbÞ¼ b =21 þ > 0 2 2 ðL 1Þ References b ) gðbÞ is a monotonically increasing function. Roberts, M.J., 2007. Fundamentals of Signal Processing. McGraw b 4- From steps 2 & 3, gðbÞ has only one root e [0, 1]. Hill. Widrow B., Kolla´r, I., 2008. Quantization Noise: Roundoff Error in A closed-form solution of Eq. (35) is algebraically extract- Digital Computation, Signal Processing, Control, and Communi- able only for (m/2)e {2, 3, 4, 5}. (Jacobson, 2009) cations, Cambridge University Press, pp. 485-528: Chapter 19 When m = 4, for example; Eq. (35) turns into essentially a Dither, http://oldweb.mit.bme.hu/books/quantization/dither.pdf b b quadratic equation of the form b2 þ 2 c b c ¼ 0 whose Cambridge, UK, 2008. Pohlmann, K.C., 2005. Principles of Digital Audio. McGraw-Hill, well known closed-form solution is: ISBN 0071441565. ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi p ffiffiffiffiffiffiffiffiffiffiffiffi Petri, D., 1996. Dither signals and quantization. Elsevier Measur. 19 b 2 c 4 c2 þ 4 c p b ¼ ba2 ¼ ¼ c2 þ c c; (3–4), 147–157. 2 Schuchman, L., 1964. Dither Signals and Their Effect on Quantization Producing: Noise. IEEE Trans. Commun. Technol. 12 (4), 162–165, ISSN: pffiffiffi 0018-9332. ba2 L1 ; r 1 pLffiffiffi1 jm¼4 ¼ L1 maxjm¼4 ¼ 2 L1 Vanderkooy, J., Lipshitz, S.P., 1987. Dither in digital audio. J. Audio pffiffiffiffi ð36Þ Eng. Soc. 35 (12), 966–975. b2 p1ffiffiffi 1 lim a jm¼4 ¼ ; lim rmaxjm¼4 ¼ L L!1 L L!1 2 Linde, Y., Buzo, A., Gray, R.M., 1980. An algorithm for Vector Quantization Design. IEEE Trans. Commun. COM-28, 4– As another example, when m = 6, Eq. (35) turns into 95. b6 b2 a þ 3 c a 2 c ¼ 0 which is a 3rd order equation of the Gray, R.M., 1984. Vector quantization. IEEE Signal Process. Magaz., b3 b form b þ g1 b þ g0 ¼ 0 whose closed-form solution is given 4–29. (according to Jacobson, 2009) by: Gray, R.M., Neuhoff, D.L., 1998. Quantization. IEEE Trans. Inf. sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi Theory 44 (6), 2325–2383. qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi Jain, A.K., 2008. Data Clustering: 50 Years Beyond K-means http:// b 1 3 1 b ¼ 27 g 27 g2 þ 4 27 g3 biometrics.cse.msu.edu/Presentations/FuLectureDec5.pdf. Plenary 3 2 0 0 1 sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi Talk at The IAPR’s 19th International Conference on Pattern qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi Recognition http://www.icpr2008.org/, Tampa, FL USA, Decem- 1 3 1 2 3 ber 2008. 27 g0 þ 27 g0 þ 4 27 g1 ; 3 2 Attia, M., Almazyad, A., El-Mahallawy, M., Al-Badrashiny, M.,

2 Nazih, W., 2011. Soft vector quantization with inverse power- ba j and rmaxj are then given by: m¼6 rffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffim¼6 rffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi function distributions for machine learning applications, first ed.. qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi In: Lecture Notes on Electrical Engineering: Intelligent Automa- 3 1 3 1 1 þ 1 þ L1 1 1 þ L1 tion and Systems Engineering (Chapter 26), vol. 103 Springer- b2 ffiffiffiffiffiffiffiffiffiffiffiffi ; Verlag, Berlin Heidelberg, pp. 339–351, www.SpringerOnline.com, a jm¼6 ¼ p3 L 1 ISBN 978-1-4614-0372-2, http://www.Springer.com/engineering/ ffiffiffiffiffiffiffiffiffiffiffiffi circuits+%26+systems/book/978-1-4614-0372-2. p3 ð2=3Þ L 1 Attia, M., Al-Mazyad, A., El-Mahallawy, M., Al-Badrashiny, M., r j rffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi rffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ; max m¼6 qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi Nazih, W., 2010. Post-clustering soft vector quantization with 3 1 3 1 1 þ 1 þ L1 1 1 þ L1 inverse power-function distribution, and application on discrete hmm-based machine learning. In: Proceedings of the International rffiffiffiffi rffiffiffiffi Conference on Signal Processing and Imaging Engineering (ICS- b2 3 2 2 3 L PIE’10) ISBN: 978-988-17012-0-6/World Congress on Engineering lim a jm¼6 ¼ ; lim rmaxjm¼6 ¼ : ð37Þ L!1 L L!1 3 2 and Computer Science 2010 (WCECS 2010). San Francisco, USA, For (m/2) R {2, 3, 4, 5}: one can only derive an expression October 2010, pp. 574–580 http://www.iaeng.org/publication/ WCECS2010/WCECS2010_pp574-580.pdf. for ba2 and r at L fi 1; i.e. with a large codebook. From max Jacobson, N., 2009, second ed.. In: Basic Algebra, vol. 1 Dover, ISBN Eqs. (36) and (37), one can speculate the generalization that: 978-0-486-47189-1. 1=m 2 m Hunt, R.W., 1998. Measuring Colour, third ed. Fountain Press, UK, b 2 L m 2 2 L lim a ¼ ; lim rmax ¼ ð38Þ ISBN 0-86343-387-1. L!1 m 2 L!1 m m 2 Dixit, S., 1991. Quantization of color images for display/printing on Substituting that guess in the terms of Eq. (35) gives: limited color output devices. Comput. Graph. 15 (4), 561–567.

Floyd, R.W., Steinberg, L., 1976. An adaptive algorithm for spatial Ketterer, J., Puzicha, J., Held, M., Fischer, M., Buhmann, J.M., grey scale. Proc. Soc. Inf. Display 17, 75–77. Fellner, D., 1998. Computer vision – ECCV’98. In: On Spatial Gentile, R., Walowit, E., Allebach, J., 1990. Quantization and Quantization of Color Images, Lecture Notes on Computer Science multilevel halftoning of color images for near original image (LNCS) Book Series, vol. 1406. Springer-Verlag, Berlin, Heidel- quality. In: SPIE Human Vision and Electronic Models, Methods, berg, pp. 563–577, www.SpringerOnline.com. and Applications, vol. 1249, pp. 249–259. Cheng, Cheuk-Hong, Au O.C., et al., 2009. Low color bit-depth image Kollias, S., Anastassion, D., 1991. A uniﬁed neural network approach to enhancement by contour region dithering. In: Proceedings of the digital image halftoning. IEEE Trans. Signal Process. 39 (4), 980–984. IEEE Paciﬁc Rim Conference on Communications, Computers and Flohr, T., Kolpatzik, B., Balasubramanian, R., Carrara, D., Bouman, Signal Processing (PacRim 2009), Victoria, British Columbia, C., Allebach, J., 1993. Model based color image quantization. In: Canada, 23–26 August 2009. Proceedings of the SPIE: Human Vision, Visual Processing, and Digital Display IV, vol. 1913, pp. 265–270.