Degree Project

colorXtractor – a technical aid for people with colour blindness

Stefan Hochwarter 2010-08-04 Subject: Computer Science Level: Bachelor Course code: 2DV00E

SE-351 95 Växjö / SE-391 82 Kalmar Tel +46-772-28 80 00 [email protected] Lnu.se

Abstract The aim of this thesis is to develop an technical aid () to help people with colour blindness. Colour blind people have difficulties to differentiate between certain colours, so the implemented software will name a selected colour. The software is implemented as a extension and also uses a XPCOM component. Furthermore can the user select different colour databases and change the displaying properties. Keywords: colour blindness, Mozilla Firefox extension, Colour algorithms, CIELAB, accessibility

iii Contents

1. Introduction 1 1.1. Problem discussion ...... 1 1.2. Problem analysis ...... 2 1.3. Paper structure ...... 2

2. Background 3 2.1. Colour blindness ...... 3 2.1.1. Classification ...... 3 2.1.2. Diagnosis ...... 4 2.1.3. Treatment ...... 4 2.2. Colour blindness as a handicap ...... 4 2.2.1. Daily life ...... 4 2.2.2. Internet barriers ...... 5 2.3. Colour management ...... 7 2.3.1. Colour spaces ...... 7

3. Implementation 9 3.1. Features ...... 9 3.2. Used technologies ...... 10 3.2.1. XUL - XML User Interface Language ...... 11 3.2.2. CSS - Cascading Style Sheet ...... 11 3.2.3. JavaScript ...... 12 3.2.4. XPCOM – Cross Platform Component Object Model ...... 13 3.3. Colour algorithms ...... 15 3.3.1. RGB to CIELAB conversion ...... 15 3.3.2. Colour comparison algorithm ...... 16 3.4. Online Publication ...... 17

4. Conclusion 18 4.1. Analysis ...... 18 4.2. Final Discussion ...... 18 4.3. Future Work ...... 18

References 20

A. List of Abbreviations 22

B. List of Figures 23

C. List of Listings 24

D. Source code 25 D.1. The Mozilla Firefox extension ...... 25

iv D.2. The XPCOM Component ...... 42

v 1. Introduction This chapter of the thesis shall give a first overview of the paper and the topic, which will be discussed more detailed later on. It focuses mostly on the problem, which had to be solved.

1.1. Problem discussion The Internet is a very colourful place but for people suffering from colour blindness this fact could be a barrier. There is no treatment for this, but an aid could improve the accessibility of the Internet. Today there is no free and open-source software, which supports people suffering from colour blindness. About five percent of the population are suffering from colour blindness.[1] This means that they have problems to differentiate between certain colours and it leads to new Internet barriers when it comes to colours. Online-shopping, pie-charts or coloured status icons are just a few examples, so there is a need for an aid for those people, which can compensate the colour blindness. Figure 1.1 shows a symbolic representation. The left side illustrate the situation without any aid while at the right side an aid has been inserted and the information (e.g. a colour) can reach the user.

Figure 1.1: symbolic representation of an aid [30]

Actually I found the idea of this bachelor thesis originally from the “Project:Possibility Wiki” during my search for a topic: When shopping for clothes online, it’s hard for people with colourblindness to discern between different colours. Develop a browser-plugin (or a native app) that can help with this shopping scenario. This could also be useful when using Google Image Search.[16] The first decision to make was how it should be implemented. Finally I decided to implement a Mozilla Firefox extension. One reason for this decision was that Mozilla Firefox is the second most popular (behind Microsoft Internet Explorer).[3] So it is the most used open source browser which offers a great documentation and com- munity for writing extensions. Another reason for solving this problem as an extension is that the user does not need to run a stand alone software. The extension is nicely integrated in the graphical user interface of Firefox. Furthermore the extension can be designed platform independent (apart from the XPCOM component).

1 1.2. Problem analysis So I decided to create a technical aid for people with colour blindness, which is imple- mented as a Mozilla Firefox extension. Before I started to write the extension, I set some goals which it should fulfil.

1. The basic feature of the extension is to name a selected colour.

2. Follow the KISS principle. KISS is a design principle and stands for “keep it simple and stupid”. The extension should be easy to use and not distract the user from his/her actual work.

3. The software should be platform independent. In other words, everywhere where Mozilla Firefox runs, the extension should work as well.

4. colorXtractor should be open source, so the source code can be reused.

5. i18n – internationalisation should be realised.

1.3. Paper structure This paper exists in two main parts. The first part will focus on the problem, namely on defective colour vision and how it affects people who are suffering from it. The second part will concentrate on the problem solving. This is mostly about the written extension, the used technologies and some interesting algorithms.

2 2. Background This chapter will provide the essential background information for writing the aid for people with colour blindness. The first part is about colour blindness in general and how it can be classified. As the extension should help people, this section will also show, why and in which way colour blindness can be a handicap. For the implementation it is also important, to know some colour spaces - each with its advantages and disadvantages.

2.1. Colour blindness Colour blindness was first scientificly documented by the English chemist, John Dalton. After he realised that he suffered from colour blindness, he published a paper on this subject in 1798: ”Extraordinary facts relating to the vision of colours”. Because of this work, colour blindness is often called Daltonism. Nowadays, Daltonism stands only for one certain type of colour blindness (deuteranopia).[1]

2.1.1. Classification Colour blindness can be classified by clinical appearance. Generally it can be differed between total colour blindness (achromatopsia) and partial colour blindness or defective colour vision.

• total colour blindness (achromatopsia)

• partial colour blindness – red / green colour deficit ∗ anomalous trichromacy (protanomaly and deuteranomaly) ∗ dichromatopsia (protanopia and deuteranopia) – blue / yellow colour deficit ∗ anomalous trichromacy (tritanomaly) ∗ dichromatopsia (tritanopia)

Total colour blindness Persons who suffer from total colour blindness can only experience different light inten- sities but no colours. It is very rare, only 0.01 % people are affected. Possible causes are cone aplasia, cone dystrophy, retina diseases, albinism or cortex diseases.[1]

Partial colour blindness This quite common disease affects mostly men (8% men, 1% woman). Beside the classifi- cation of clinical appearance you can also differentiate between inheritance and acquired colour blindness. Anomalous trichromacy is a reduced sense for red (protanomaly), green (deutera- nomaly, 50 % of the inheritance cases) or blue (tritanomaly, rare).

3 The absence of one of the three retinal cones, which are essential for colour vision, is called dichromatopsia. It leads to the typical colour confusions. This is discerned again by the three colours red, green and blue. If the missing colour is red, the disease is called protanopia. This leads to red / green colour deficit. Deuteranopia also leads to red / green colour deficit, but the missing colour is green. When blue is missing, it is called tritanopia and it leads to blue / yellow deficit. Tritanopia is very rare. When two colours are missing, it is called monochromaticity.[1]

2.1.2. Diagnosis To diagnose colour blindness, qualitative clinical tests are used. The most common tests are the Stilling colour test, the Ishihara colour test and the Farnsworth panel D15 test.[1]

(a) Ishihara colour test [6] (b) Farnsworth D15 colour test [10]

Figure 2.1: Colour blindness tests

2.1.3. Treatment There is no medical treatment to cure people suffering from colour blindness. However, there exists some aid for people with colour blindness. Gnome offers a library, which provides a pixel filter, according to the user’s colour blindness.[14] Another non-technical aid are contact lenses designed for people with colour blindness.[4]

2.2. Colour blindness as a handicap When it comes to daily life, colour blindness can be a big handicap. In the following section I will list some examples.

2.2.1. Daily life As my extension focuses on web accessibility, I will keep this section short. But I want to deliver insight into the world of people with colour blindness. This is best done with examples. On the first picture you can see bananas and it is clear to see which are ripe.

4 The second picture is the same, but deuteranopia is simulated. As you can see it is impossible to differentiate between ripe and unripe bananas.

(a) original (b) deuteranopia

Figure 2.2: Comparision of bananas [5]

Another example are LED lights, used in devices to indicate something. The left picture is again the original and in the right picture protanopia was simulated.

(a) original [18] (b) protanopia

Figure 2.3: Comparison of LED lights

2.2.2. Internet barriers As the goal of the extension is to help people with colour blindness in the web, I want to show examples where colours can be a barrier. For people suffering from colour blindness, it can be hard or impossible to interpret a statistic when colours are the only source for matching the value to the graph. Figure 2.4 shows a pie chart, once original and once with deuteranopia simulated.

5 (a) original [2] (b) deuteranopia

Figure 2.4: Comparison of a pie chart

Online-shopping is another example where people with colour blindness may have some difficulties. When they want to order some furniture which should fit with their interior, they need to order it in a certain colour. It is also the same with clothes. Figure 2.5 shows a screenshot of an online-shop, on the right again with deuteranopia simulated.

(a) original [24] (b) deuteranopia

Figure 2.5: Comparison of an online shop

Also status icons, which indicate if a user is online, are often just coloured shapes. For people suffering from colour blindness it can be hard to say if a user is on- or offline. See figure 2.6.

6 (a) original [11] (b) deuteranopia

Figure 2.6: Comparison of icq status icons

2.3. Colour management I worked mostly with two colour models, the RGB and the CIELAB colour model. This section will give a insight into those models.

2.3.1. Colour spaces The extension explains two different colour spaces, namely the RGB colour space and the CIELAB colour space.

RGB colour model RGB stands for red, green and blue and the model adds those three colours to produce a certain colour. The RGB colour model is a additive colour model and it is based on a Cartesian coordinate system. It is a very common system for displaying images in electronic systems. The disadvantage of the RGB colour model is, that it is device- dependent. This means that the presentation of a RGB value may vary on different devices. Figure 2.7 illustrates the RGB colour model. [17] [18]

Figure 2.7: RGB colour model [19]

CIELAB CIE L*a*b* is a colour opponent space and it describes all colours visible to the human eye. L describes the lightness, a is the position on the red-green axis and b the position on the yellow-blue axis. Unlike the RGB colour model, it is device-independent. Figure 2.8 shows the CIE L*a*b* colour space. [20] [13]

7 Figure 2.8: CIE L*a*b* colour space [22]

8 3. Implementation The project behind this thesis is called colorXtractor. This Mozilla Firefox extension helps people with colour blindness to distinguish colours. Figure 3.1 shows colorXtractor in action. At the left bottom edge is the extension’s icon. Next to the icon is the calculated colour name.

Figure 3.1: colorXtractor - screenshot

Because this is my first Mozilla Firefox extension, I had to learn how to use the technologies and how they interact. Therefor I read mostly on the Mozilla Develeoper Center and used the extension "Colorzilla"[21] as a template for starting. Unfortunately there was no good and up-to-date tutorial / document about a XPCOM extension, so this part was very time consuming.

3.1. Features The main feature of the extension is to display a colour under the mouse cursor, after clicking on the extension’s icon. The first step is to get the pixel’s colour code, format it and find the closest matching colour from a colour database. The user can also change some options. He or she can select between different colour databases and format the colour string. The RGB, HSL and CIELAB colour values and the x-y coordinates can be displayed.

9 Figure 3.2: colorXtractor - options

3.2. Used technologies A Mozilla Firefox extension is basically a package of files, that influences the appear- ance and behaviour of Firefox. This package is simply created by zipping all files and changing the the file extension to “xpi”. The package of colorXtractor has the following structure.[9]

Listing 3.1: Package structure |-- chrome.manifest // chrome registration manifest |-- components // component and XPT files |-- content // files for overlay and JavaScripts |-- defaults | ‘-- preferences // default preferences |-- install.rdf // extension install manifest |-- locale // i18n |-- platform // platform specific directory ‘-- skin // skin directory (CSS)

For building a simple Firefox extension, there are only three basic technologies needed. CSS (Cascading Style Sheets) is used for skinning, JavaScript for behaviour and XUL (XML User Interface Language) for the user interface. For more complex extension it is necessary to use XPCOM (Cross Platform Component Object Model), which provides additional functionality.[23]

10 Figure 3.3: role of each technology in Firefox [15]

3.2.1. XUL - XML User Interface Language XUL was developed by Mozilla for describing the Graphical User Interface (GUI) of Firefox. As it is XML based, it uses tags like

or . Most of Firefox’s GUI is described by XUL. This means, if an extension adds a new GUI-component, it is done by overlaying the browser’s XUL file. This is also called XUL overlay.[29] The following code listing shows a part of colorXtractor’s XUL overlay.

Listing 3.2: colorxtractorOverlay.xul

This XUL file inserts a box in the status-bar of Firefox, which will be for mated by CSS, with a popup menu. This context menu has a entry for showing the about-dialogue.

3.2.2. CSS - Cascading Style Sheet Cascading Style Sheets describes the presentation of HTML and XML files. The CSS files are situated at the skin directory of the extension.[7] For this project, it is used minimal,

11 because there is no need to change the default style. It is only used for inserting the icon at the status bar, as described in the following CSS file.

Listing 3.3: colorxtractor.css #colorxtractor-status-bar-color { -moz-appearance: none !important; background-image: url("chrome://colorxtractor/skin/main-icon.png"); border: 1px inset; background-repeat: no-repeat; background-position: top left; padding: 0; margin: 0; }

3.2.3. JavaScript A good way to describe JavaScript with one simple sentence is found at the Mozilla Developer Center.

JavaScript is a small, lightweight, object-oriented, cross-platform scripting language.[12]

In an extension, JavaScript is mostly used for behaviour, algorithms and for connecting components. When a user interacts with an XUL element, e.g. a button, nothing would happen without a JavaScript function. The easiest way for interaction is to call a function, when an action has proceeded. This is the case in the following code.

Listing 3.4: colorxtractorOverlay.xul

Listing 3.5: colorxtractorUtils.js var win = crGetBrowserWindow(); if (win) win.getBrowser().selectedTab = win.getBrowser().addTab(url); // we are here because we either wanted to open in new // window, or wanted to open in tab, but couldn’t find browser window else openDialog("chrome://browser/content/browser.xul", "_blank", " chrome,all,dialog=no", url, null, null); }

Another way for interactions with the user are listeners. In the extension colorXtractor a listener is used for watching the mouse and toggling the colour monitor.

12 3.2.4. XPCOM – Cross Platform Component Object Model Mozilla’s XPCOM stands for Cross Platform Component Object Model and it is similar to Microsoft’s COM. C and C++ can be accessed out-of-the-box because the XPCOM Layer is written in those languages. Beside C/C++, XPCOM supports several language bindings such as JavaScript, Java, Python, Perl and Ruby.[26] Because JavaScript is not able to get a pixel’s colour, colorXtractor uses a XPCOM component, written in C++. Figure 3.4 shows a overview of the extension’s architecture.

Figure 3.4: the extension’s architecture

XPIDL is Mozilla’s modified, cross-platform version of IDL (Interface Description Lan- guage). This is used to describe the interface in a language- and machine-independent way. The tool xpidl, which is included in the XULRunner, generates C++ header files and XPConnect typelib files (.xpt) based on the XPIDL interface description. The generated header files include a template for a C++ implementation of the interface.[28] The following code listing shows the XPIDL interface description of the extension’s component. It has two input parameters, x and y, and one return parameter. Everytime the mouse position is changing the monitoring is activated, the function getPixelColor will be called with the new mouse-coordinates (x, y). As a result, the colour of the pixel under the mouse-cursor will be returned.

Listing 3.6: IColorReader.idl #include "nsISupports.idl"

[scriptable, uuid(b146175f-a35d-4f59-9d2c-7eb828879299)] interface IColorReader : nsISupports

13 { unsigned long getPixelColor(in long x, in long y); };

After using the xpidl tool, the XPConnect typelib file (IColorReader.xpt) and the header file (IColorReader.h) is generated. The next step is to write the component header file (ColorReader.h), which defines the component (contract id, classname, GUID) and the interface. A template of the interface definition is included in the generated header (IColorReader.h). There exists also a template for the implementation file. The next listing shows the implementation of the function getPixelColor, which has been defined in IColorReader.idl.

Listing 3.7: ColorReader.cpp NS_IMETHODIMP ColorReader::GetPixelColor(PRInt32 x, PRInt32 y, PRUint32 *_retval NS_OUTPARAM) { // GetDC: The handle the device context for the specified window’s client area indicates success. NULL indicates failure HDC dc = GetDC(0); // Get the Color (RGB) at (x,y) if (dc != NULL){ COLORREF color = GetPixel(dc, x, y); *_retval = (unsigned long) color; return NS_OK; } *_retval = NULL; return NS_OK; }

The last file, that needs to be written is the module definitions file (ColorReaderMod- ule.cpp). Then the component is complete and can be built. As a result a dynamic library is created (ColorReader.dll). XPConnect is the bridge between JavaScript and XPCOM.[27] The next listing shows the registering of the component.

Listing 3.8: colorxtractorUtils.js function crInitComponent(){ try{ const cid = "@sourceforge.net/projects/ColorReader;1"; gCRComponent = Components.classes[cid].createInstance(); gCRComponent = gCRComponent.QueryInterface(Components.interfaces. IColorReader); } catch(err) { alert("Could not get object: " + err + "\n" + cid); return; } }

14 Now the C++ function can be easily accessed by gCXComponent.getPixelColor(x,y).

3.3. Colour algorithms This section shows two interesting colour algorithms, which are essential for the exten- sion.

3.3.1. RGB to CIELAB conversion It is necessary to convert the RGB values to CIELAB, to get a good result when com- paring two colours. This conversion is done in two steps. The first conversion is from RGB to XYZ (also a CIE colour space).

X r   Y  = M g Z b The standard colour model of Windows is sRGB, so we can use this information for the calculation.

 R/12.92 R ≤ 0.04045 r = ((R + 0.055)/1.055)2.4 R > 0.04045

 G/12.92 G ≤ 0.04045 g = ((G + 0.055)/1.055)2.4 G > 0.04045

 B/12.92 B ≤ 0.04045 b = ((B + 0.055)/1.055)2.4 B > 0.04045 Using D65 as reference white, the following matrix can be used for M.

0.4124564 0.3575761 0.1804375   M = 0.2126729 0.7151522 0.0721750 0.0193339 0.1191920 0.9503041 Now the XYZ value can be converted to CIELAB.

L = 116fy − 16

a = 500(fx − fy)

b = 200(fy − fz) where: √  3 xr xr > ε fx = κxr+16 116 xr ≤ ε  √ 3 yr yr > ε fy = κyr+16 116 yr ≤ ε

15 √  3 zr zr > ε fz = κzr+16 116 zr ≤ ε X Y Z xr = yr = zr = Xr Yr Zr 216 ε = Intent of the CIE Standard 24389 24389 κ = Intent of the CIE Standard 27 This transformation is done in the function cxGetLab(colour).[25]

3.3.2. Colour comparison algorithm The colour lists are stored in arrays in the following format.

[colourname, 0xRRGGBB]

This is because the RGB colour space is very common and many colour lists in this format are available online, which can then be used easily. When a colour list is selected for comparison, the array will also be filled with the CIELAB values (L, a, b), because the calculation of the difference between two colours is done in the L*a*b* colour space. A selected colour list for comparison are looks like this:

[colourname, 0xRRGGBB, L, a, b]

To compare the colour under the mouse cursor, the function crGetColorName is called. This function transforms the selected colour to CIELAB. For finding the closest colour, the euclidean distance is calculated.

∆E = p2 (∆L)2 + (∆a)2 + (∆b)2 The name of the colour from the list, with the smallest euclidean distance to the colour under the mouse cursor, is the return value of the function cxGetColorName. Listing 3.9 shows the colour comparison algorithm.

Listing 3.9: colorxtractorUtils.js function cxGetColorName(color){ var lab = cxGetLab(color); var l = lab[0], a = lab[1], b = lab[2]; var dif = -1, ndif = 0, cl = -1; for (var i = 0; i < gCxnames.length; i++) { if (color == gCxnames[i][1]) return gCxnames[i][0]; ndif = Math.pow(Math.pow(l - gCXnames[i][2], 2) + Math.pow(a - gCXnames[i][3], 2) + Math.pow(b - gCXnames[i][4], 2),1/2); if (dif < 0 || dif > ndif){

16 dif = ndif; cl = i; } } return gCXnames[cl][0]; }

3.4. Online Publication As colorXtractor is a Mozilla Firefox extension, it can be downloaded at the official add- ons repository of Mozilla. But the extension and the complete source code is also hosted on sourceforge, which is the largest Open Source application and software directory. Also a project-homepage exists (http://colorxtractor.sourceforge.net), where all re- sources are listed. The project-wiki, also hosted on sourceforge, is divided in documen- tation for users and for developers. Users can get more specific support with the feature “Support Request”. Bugs can be reported on sourceforge and an ideaTorrent offers a simple way to suggest new features for up-coming versions.

17 4. Conclusion This last chapter discusses the success of the project and reviews the reached goals. As the time was limited, there is still some work to do and also many ideas, for improving the extension.

4.1. Analysis The project achieved the following goals (from 1.3 Problem analysis):

1. Colour naming: The extension is able to name any colour under the mouse cursor inside the Mozilla Firefox window. The user can select a colour databases for comparison. They differ in the amount of colours and the colour names.

2. KISS-principle: After the installation of the extension, the user is not being an- noyed by any pop-ups or an overloaded user interface. The only change of the web browser is a small icon at the status bar. Activating and deactivating of the colour naming is implemented by a single click.

3. Platform independent: During the developing process this specification turned out to be tricky. For receiving the colour code it is necessary to use libraries of the operating system. At present it is only implemented for , but the requirements for platform independency are created.

4. Open source: The whole project, including the XPCOM component, is open source and published under Mozilla Public License. Comments assure that other people can easily understand the algorithms. The colorXtractor Wiki includes a docu- mentation for developer, which is a great place for beginning to develop.

5. i18n is realised by DTD (Document Type Definition) for XUL and properties for JavaScripts. A tutorial for i18n is available at the colorXtractor Wiki.

4.2. Final Discussion According to the section 1.1, the problem is successfully solved. The extension colorX- tractor assists people with colour blindness to distinguish colours. It is easy to install, simple to use and does not cost anything. Furthermore, the source code is published under Mozilla Public License, so everybody is allowed to use, reuse and modify the extension’s code as long as the outcome is also released under MPL.

4.3. Future Work After implementing the basic feature of the extension, I published a beta version for testing. At the same time I launched a bug report manager and a support request manager for getting feedback. Furthermore, a person who suffers from defective colour vision (red-green colour blindness) was involved in the developing process since the project-idea. All these different sources for feedback or ideas, guarantees that there will be new ideas for the extension. Those ideas will be implemented in the next versions.

18 The development of the extension will not close after finishing this degree project. There are already some ideas for new features, such as allowing the user to create his / her own colour database. Also a text-to-speech feature may be an upcoming feature. Another idea is to change the colours of the web-content, according to the user’s colour blindness. There is already the first entry in the ideaTorrent, which has some good ideas for the future. One request is a keyboard-shortcut to trigger the colour picker and another one for the options. Another idea is an on-screen-display for displaying the colour at the mouse-cursor’s position and also for zooming in this area. Moreover a very simple colour database could be realised for helping children with colour blindness. The current state of colorXtractor only supports Windows because of the XPCOM component. Together with Mozilla Firefox 3.7, a new technology will be supported, namely JSctypes.[8] This allows JavaScript to call into native C code without using XPCOM and without compiling a line of code. There is already the possibility to use JSctypes ( 1.9.3) for development. Instead of writing binary components for and Mac OS X, I will use this technology for providing platform independency.

19 References [1] Dr. Martina Bach. Pschyrembel - Klinisches Wörterbuch. De Gruyter, 2007. [2] Bar And Pie Charts With Raphael JS. http://masonoise.wordpress.com/ 2009/12/29/bar-and-pie-charts-with-raphaeljs/, Date of access: 2010/07/12. [3] Browser Market Share, April 2010. http://marketshare.hitslink.com/ browser-market-share.aspx, Date of access: 2010/05/03. [4] ChromaGen. http : / / www . dyslexia - help . co . uk/, Date of access: 2010/07/12. [5] Colblindor - Living with Color Blindness. http://www.colblindor.com/20 10/03/30/living-with-color-blindness/, Date of access: 2010/07/12. [6] Color Blindness. http://en.wikipedia.org/wiki/Colour_blindness, Date of access: 2010/07/12. [7] CSS. https : / / developer . mozilla . org / en / CSS, Date of access: 2010/05/05. [8] Extension authors, browser hackers, meet js-ctypes. http://blog.mozilla. com/dwitte/2010/03/12/extension-authors-browser-hackers- meet-js-ctypes, Date of access: 2010/05/09. [9] Extension Bootcamp:Zero to Hello World! in 45 Minutes. http://design- challenge.mozilla.com/resources, Date of access 2010/05/05. [10] Farnsworth D15 Color Test (Good-Lite Shop). http://www.good-lite.com/ Details.cfm?ProdID=374, Date of access: 2010/07/12. [11] ICQ. http://www.icq.com, Date of access: 2010/07/12. [12] JavaScript. https://developer.mozilla.org/en/JavaScript, Date of access: 2010/05/05. [13] Lab color space. http://en.wikipedia.org/wiki/Lab_color_space, Date of access: 2010/05/07. [14] libcolorblind. http://packages.debian.org/lenny/libcolorblind0, Date of access: 2010/07/12. [15] MDC - Technologies used in developing extensions. https : / / developer . mozilla . org / En / Firefox _ addons _ developer _ guide / Technologies _ used _ in _ developing _ extensions, Date of access: 2010/07/12. [16] Project Possibility - A Software Collaboration for Persons with Disabilities. http: //www.projectpossibility.org, Date of access: 2010/05/03. [17] Richard E. Woods Rafael C. Gonzalez. Digital Image Processing. Prentice Hall International, 2008.

20 [18] RGB color model. http://en.wikipedia.org/wiki/RGB, Date of access: 2010/05/07. [19] RGB Color Space. http://en.wikipedia.org/wiki/RGB_color_space, Date of access: 2010/07/12. [20] János Schanda. Colorimetry: understanding the CIE system. John Wiley & Sons, 2007. [21] Alex Sirota. ColorZilla. http://www.colorzilla.com/firefox, Date of access: 2010/05/25. [22] TechExchange. http : / / www . techexchange . com / thelibrary / DTPColorMgmt_RIPS.html, Date of access: 2010/07/12. [23] Technologies used in developing extensions. https://developer.mozilla. org/En/Firefox_addons_developer_guide/Technologies_used_ in_developing_extensions, Date of access: 2010/05/05. [24] Urban Outfitters. http://www.urbanoutfitters.co.uk/, Date of access: 2010/07/12. [25] Useful color equations. http://www.brucelindbloom.com, Date of access: 2010/05/08. [26] XPCOM. https://developer.mozilla.org/en/XPCOM, Date of access: 2010/05/06. [27] XPConnect. https://developer.mozilla.org/en/XPConnect, Date of access: 2010/05/06. [28] XPIDL. https://developer.mozilla.org/en/XPIDL, Date of access: 2010/05/06. [29] XUL. https : / / developer . mozilla . org / En / XUL, Date of access: 2010/05/05. [30] Prof. Wolfgang Zagler. Elektronische Hilfsmittel für behinderte und alte Menschen. Technical University of Vienna, 2008.

21 A. List of Abbreviations CIE International Commission on Illumination

CIELAB CIE 1976 (L*, a*, b*) Colour Space

CSS Cascading Style Sheets

DTD Document Type Definition

GUI Graphical User Interface

HSL Hue, Saturation, Sightness

HTML HyperText Markup Language

KISS Keep It Simple and Stupid

MSCOM Microsoft’s Component Object Model

RGB Red, Green, Blue

XML Extensible Markup Language

XPCOM Cross Platform Component Object Model

XPConnect Cross Platform Connect

XPIDL Cross Platform Interface Description Language

XUL XML User Interface Language

22 B. List of Figures 1.1. symbolic representation of an aid [30] ...... 1 2.1. Colour blindness tests ...... 4 2.2. Comparision of bananas [5] ...... 5 2.3. Comparison of LED lights ...... 5 2.4. Comparison of a pie chart ...... 6 2.5. Comparison of an online shop ...... 6 2.6. Comparison of icq status icons ...... 7 2.7. RGB colour model [19] ...... 7 2.8. CIE L*a*b* colour space [22] ...... 8 3.1. colorXtractor - screenshot ...... 9 3.2. colorXtractor - options ...... 10 3.3. role of each technology in Firefox [15] ...... 11 3.4. the extension’s architecture ...... 13

23 C. List of Listings 3.1. Package structure ...... 10 3.2. colorxtractorOverlay.xul ...... 11 3.3. colorxtractor.css ...... 12 3.4. colorxtractorOverlay.xul ...... 12 3.5. colorxtractorUtils.js ...... 12 3.6. IColorReader.idl ...... 13 3.7. ColorReader.cpp ...... 14 3.8. colorxtractorUtils.js ...... 14 3.9. colorxtractorUtils.js ...... 16 D.1. Folder structure ...... 25 D.2. chrome.manifest ...... 25 D.3. install.rdf ...... 25 D.4. about.xul ...... 26 D.5. colorxtractorClickListener.js ...... 26 D.6. colorxtractorColorDB.js ...... 28 D.7. colorxtractorOverlay.js ...... 28 D.8. colorxtractorOverlay.xul ...... 32 D.9. colorxtractorUtils.js ...... 34 D.10.colorxtractor.js ...... 41 D.11.about.dtd ...... 41 D.12.colorxtractor.dtd ...... 41 D.13.colorxtractor.properties ...... 41 D.14.colorxtractor.css ...... 41 D.15.Folder structure ...... 42 D.16.IColorReader.idl ...... 42 D.17.IColorReader.h ...... 44 D.18.ColorReader.h ...... 46 D.19.ColorReader.cpp ...... 46 D.20.ColorReaderModule.cpp ...... 47

24 D. Source code D.1. The Mozilla Firefox extension

Listing D.1: Folder structure . |-- chrome.manifest |-- install.rdf |-- components | ‘-- IColorReader.xpt |-- content | |-- about.xul | |-- colorxtractorClickListener.js | |-- colorxtractorColorDB.js | |-- colorxtractor.jpg | |-- colorxtractorOverlay.js | |-- colorxtractorOverlay.xul | ‘-- colorxtractorUtils.js |-- defaults | ‘-- preferences | ‘-- colorxtractor.js |-- locale | ‘-- en-US | |-- about.dtd | |-- colorxtractor.dtd | ‘-- colorxtractor.properties |-- platform | ‘-- WINNT | ‘-- components | ‘-- ColorReader.dll ‘-- skin |-- colorxtractor.css ‘-- main-icon.png

Listing D.2: chrome.manifest content colorxtractor content/ locale colorxtractor en-US locale/en-US/ skin colorxtractor classic/1.0 skin/ overlay chrome://browser/content/browser.xul chrome://colorxtractor/ content/colorxtractorOverlay.xul

Listing D.3: install.rdf [email protected] colorXtractor 0.1

25 Stefan Hochwarter colorXtractor is an extension for Mozilla Firefox, which is written for people with color-blindness. The extension shows the color as text of a selected area in text. http://colorxtractor.sourceforge.net chrome://colorxtractor/content/about.xul chrome://colorxtractor/content/colorxtractor.jpg {ec8030f7-c20a-464f-9b0e-13a3a9e97384} 3.5 3.6.*

Listing D.4: about.xul

Listing D.5: colorxtractorClickListener.js // the listener for monitoring the mouse

26 var gCXClickListener; function ColorXtractorClickListener() { }

ColorXtractorClickListener.prototype = { mbListening: false, mTargetDocs: null,

startClickListener: function() { this.mbListening = true; // add a click listener to all documents this.mTargetDocs = this.getAllDocuments();

for (var i = 0; i < this.mTargetDocs.length; ++i) { this.mTargetDocs[i].addEventListener("click", CXMouseClickListener, true); } },

stopClickListener: function(aTarget) { this.mbListening = false; cxToggleColorMonitor(); },

removeClickListener: function() { if (this.mTargetDocs == null) { // we never installed the listeners return; }

// remove all listeners for (var i = 0; i < this.mTargetDocs.length; ++i) { this.mTargetDocs[i].removeEventListener("click", CXMouseClickListener, true); } },

// currently, returns only the main window getAllDocuments: function() { return [window]; }, }

// Listener Objects

27 var CXMouseClickListener = { handleEvent: function(aEvent) { var target = aEvent.target; if (aEvent.type == "click") {

aEvent.stopPropagation(); aEvent.preventDefault();

gCXClickListener.stopClickListener(target); gCXClickListener.removeClickListener();

} } }; gCXClickListener = new ColorXtractorClickListener();

Listing D.6: colorxtractorColorDB.js

// The selected color database var gCXnames = null;

// The color database names var gCXdbnames = ["gCXcolors01", "gCXcolors02", "gCXcolors03", " gCXcolors04", "gCXcolors05"];

// i sorted the arrays after the efficiency (subjective)

// color databse from http://web.njit.edu/~kevin/rgb.txt.html var gCXcolors01 = [ ["Grey","545454"], ["Grey, Silver","C0C0C0"],

// ... and so on - over 3000 lines ...

Listing D.7: colorxtractorOverlay.js // initial method function cxInitOverlay() { // register the listener addEventListener("mousemove", cxOnMouseMove, false); // register the XPCom component - see colorxtractorUtils.js cxInitComponent(); }

//handles a click on the icon function cxIconClicked(event){ // left mousebutton

28 if (event.button == 0) { cxToggleColorMonitor(); } }

// starts or stops listening function cxToggleColorMonitor() {

gCXColorMonitor = !gCXColorMonitor;

// change cursor - still buggy cxSetCursorToCrosshair(gCXColorMonitor); window.setCursor(gCXColorMonitor ? ’crosshair’ : ’auto’);

// start listening if (gCXColorMonitor) { window.setCursor(’crosshair’); gCXClickListener.startClickListener();

} // stopped to listen, change the mouse-cursor back again else { window.setCursor(’auto’); }

}

// handles mouse movement function cxOnMouseMove(event) { //if monitoring is active, get the color under the cursor if (gCXColorMonitor) { cxGetColorUnderCursor(event); } }

// setx the x,y coordinates and updates color function cxGetColorUnderCursor(event){

gCXSelectedElement = event.target;

gCXSelectedX = event.screenX; gCXSelectedY = event.screenY;

cxRefreshSelectedColor(); }

// calls the update-method for the color function cxRefreshSelectedColor() {

if ( (gCXSelectedX == null) || (gCXSelectedY == null) ){ return;

29 }

// todo: gCRSelectedColor = crGetScreenColor(gCRSelectedX, gCRSelectedY); ? cxUpdateColorPanel(); }

// updates color function cxUpdateColorPanel() { // sets the color and formats the string for the status-bar cxSetStatusBarString(); // set the text cxSetStatusText(gCXStatusBarString); }

// place str at the status-bar function cxSetStatusText(str) { var statusTextFld = document.getElementById("statusbar-display"); statusTextFld.label = str; }

// used for changing the mouse-cursor function cxGetAllDocuments(frame, documents) { var frames = frame.frames;

documents.push(frame.document);

for (var i = 0; i < frames.length; i++) { cxGetAllDocuments(frames[i], documents); }

return documents; }

// change the mouse-cursor (from colorzilla) function cxSetCustomCursorHTML(bSet, cursor) { var contentsRule = "BODY, BODY * { cursor: " + cursor + " !important; }";

var allDocuments = cxGetAllDocuments(window.content, new Array()); for (var i = 0; i < allDocuments.length; i++) { var doc = allDocuments[i]; if (bSet) { var headElements = doc.getElementsByTagName("head"); var styleElement = doc.createElement("style"); styleElement.setAttribute("id", "colorxtractor-custom-cursor- style-" + cursor); styleElement.setAttribute("type", "text/css"); styleElement.innerHTML = contentsRule; if (headElements.length > 0) {

30 headElements[0].appendChild(styleElement); } else { doc.documentElement.childNodes[0].appendChild( styleElement); } } else { var styleElement = doc.getElementById("colorxtractor-custom- cursor-style-" + cursor); if (styleElement != null) { styleElement.parentNode.removeChild(styleElement); } } } }

// change the mouse-cursor (from colorzilla) function cxSetCursorToCrosshair(bSet) { var chromeRule = "window * { cursor: crosshair ! important; }"; // get the last style sheet var sheet = null; try { sheet = document.styleSheets[document.styleSheets.length - 1]; } catch (err) { dump("Couldn’t get document style sheet\n"); } var pos = sheet.cssRules.length;

// handle empty stylesheet if (bSet && (pos == 0)) { sheet.insertRule(’#colorxtractor-dummy { }’, pos); pos = sheet.cssRules.length; }

// HTML part: cxSetCustomCursorHTML(bSet, ’crosshair’);

if (bSet) { // browser part: if (sheet != null) { sheet.insertRule(chromeRule, pos); } } else { // browser part: if (sheet != null) { sheet.deleteRule(pos - 1); pos--; // make sure that we remove all the rules we added // (if we added two identical rules by mistake) while (sheet.cssRules[pos - 1].cssText == chromeRule) { sheet.deleteRule(pos - 1); pos--; }

31 } } }

// adds the event listener addEventListener("load", cxInitOverlay, false);

// color monitor activated? var gCXColorMonitor = false;

// last selected coordinates and element var gCXSelectedX = null; var gCXSelectedY = null; var gCXSelectedElement = null;

// ColorXtractor C++ library Component var gCXComponent = null;

// The string displayed in the statusbar, Format: // color | R: ##, G: ##, B: ## | H: ##, S: ##, L: ## | x: ##, y: ## var gCXStatusBarString = null;

Listing D.8: colorxtractorOverlay.xul