Master Reference

Master

A Heuristic Evaluation of Multilingual Lom ba rdy : Museums' Web Sites

MINACAPILLI, Carmen Ambra

Abstract

The compliance with localisation standards and guidelines ensures the usability and suitability of web sites to multilingual and multicultural audiences. Considering the importance of cultural tourism in Italy, a web site that is usable and suitable to both local and international users can have a positive impact on visitors. From the general perception that localised versions present some limitations stems the decision to analyse and compare Italian and English home pages of museums web sites in one Italian region-Lombardy. The aim of this paper is to test whether Italian and English home pages respect usability guidelines and whether localisation has a negative impact on usability. Through the heuristic evaluation methodology, this paper presents a descriptive and critical analysis of aspects related to internationalisation, localisation, search in the web site, SEO and web accessibility in the two linguistic versions. Results show that over half of the home pages respect most of the usability guidelines checked, with differences between the two linguistic versions.

Reference

MINACAPILLI, Carmen Ambra. A Heuristic Evaluation of Multilingual Lom ba rdy : Museums' Web Sites. Master : Univ. Genève, 2018

Available at: http://archive-ouverte.unige.ch/unige:112073

Disclaimer: layout of this document may differ from the published version.

1 / 1

Carmen Ambra Minacapilli

A Heuristic Evaluation of Multilingual Lombardy

Museums’ Web Sites

Directrice : Lucía Morado Vázquez

Juré : Paolo Canavese

Mémoire présenté à la Faculté de traduction et d’interprétation (Département de traitement informatique multilingue) pour l’obtention de la Maîtrise universitaire en traduction, mention technologies de la traduction.

Université de Genève Année académique 2017-2018 Août 2018

J’affirme avoir pris connaissance des documents d’information et de prévention du plagiat émis par l’Université de Genève et la Faculté de traduction et d’interprétation (notamment la Directive en matière de plagiat des étudiant-e-s, le Règlement d’études de la Faculté de traduction et d’interprétation ainsi que l’Aide-mémoire à l’intention des étudiants préparant un mémoire de Ma en traduction).

J’atteste que ce travail est le fruit d’un travail personnel et a été rédigé de manière autonome.

Je déclare que toutes les sources d’information utilisées sont citées de manière complète et précise, y compris les sources sur Internet.

Je suis consciente que le fait de ne pas citer une source ou de ne pas la citer correctement est constitu- tif de plagiat et que le plagiat est considéré comme une faute grave au sein de l’Université, passible de sanctions.

Au vu de ce qui précède, je déclare sur l’honneur que le présent travail est original.

Nom et prénom : Minacapilli, Carmen Ambra

Lieu / date / signature : Varese, le 14 juillet 2018

Acknowledgements First and foremost, I offer my sincere gratitude to my supervisor, Lucía Morado Vázquez, who has en- couraged me with her kindness through the learning process of this Master thesis and guided my work with her knowledge and expertise, whilst allowing me the freedom to develop my own research. Her engagement in localisation and her genuinely positive attitude towards students made me happy to be guided by her in the final accomplishment of my career as a student.

I would like to thank my co-supervisor, Paolo Canavese, for the time and dedication in reading and correcting my work, and Jeanette Pidanick on behalf of Nielsen Norman Group for allowing me to reproduce the figure from Nielsen (1993). My gratitude also goes to Violeta Seretan, for her precious help and guidance in both statistics and academic writing.

This Master thesis is heartily dedicated to my father Filippo and my mother Teresa, for their example of love, courage and hope. I do not have words to express my thank you for your unconditional support and caring attentions, despite all we have been through this year. Thank you is just a word to tell you that I truly owe you everything. If my thanks can be of any use, I would like to thank my dad for teaching me courage and optimism and my mum for teaching me hope and gratitude, however difficult life may seem.

In my daily work, I have been blessed with friendly and cheerful family and friends. My special thanks goes to my cousin Giada, because she makes everything seem possible, and my friend Sofia, because she constantly reminds me what dreams are made of.

I would also like to thank the international mobility office of University of Geneva and Moscow State Linguistic University for giving me the opportunity to study in Moscow, “a true moment of blissful happiness” in Dostoevsky’s words.

As conclusion, this work was written in the loving memory of my aunt Barbara, my first and most sincere friend, whose love taught me that trees must develop deep roots to let flowers grow.

Abstract The compliance with localisation standards and guidelines ensures the usability and suitability of web sites to multilingual and multicultural audiences. Considering the importance of cultural tourism in Italy, a web site that is usable and suitable to both local and international users can have a positive impact on visitors. From the general perception that localised versions present some limitations stems the decision to analyse and compare Italian and English home pages of museums web sites in one Italian region— Lombardy. The aim of this paper is to test whether Italian and English home pages respect usability guidelines and whether localisation has a negative impact on usability. Through the heuristic evaluation methodology, this paper presents a descriptive and critical analysis of aspects related to internationalisation, localisation, search in the web site, SEO and web accessibility in the two linguistic versions. Results show that over half of the home pages respect most of the usability guidelines checked, with differences between the two linguistic versions. The usability problems identified in this paper show that English pages present more errors than Italian pages. In both versions, there is room for improvement for what concerns the aspects studied.

Keywords Localisation – usability – usability guidelines – heuristic evaluation – museum – web site – home page – Lombardy

Table of Contents Chapter 1: Introduction ...... 6 Chapter 2: The Literature Review ...... 9 2.1 From Translation to Localisation ...... 9 2.1.1 Translation: Definition and Strategies ...... 9 2.1.2 Localisation: Components and Definitions ...... 10 2.2 Web Localisation: Process and Tendencies ...... 16 2.2.1 The Web Localisation Process (GILT): Globalisation and Internationalisation ...... 16 2.2.2 Web Localisation Tendencies: Standardisation vs. Customisation ...... 18 2.3 Web Design: Usability and Acceptability ...... 21 2.3.1 Web Usability: Definition and Components ...... 24 2.3.2 Usability Assessment Methods: Web Usability Testing ...... 26 Summary ...... 30 Chapter 3: Presentation of Research Method and Tool ...... 31 3.1 Research Questions and Hypotheses ...... 31 3.2 Heuristic Evaluation Method ...... 35 3.2.1 From broad Heuristics to Guidelines and Standards ...... 39 3.3 Heuristic Guidelines: A Tool for Heuristic Evaluation ...... 41 3.3.1 Group 1. Aspects related to source code ...... 42 3.3.2 Group 2. Aspects related to localisation ...... 46 3.3.3 Group 3. Aspects related to visual elements ...... 53 3.3.4 Group 4. Aspects affecting search in the web site ...... 56 3.3.5 Group 5. Aspects affecting the web sites ranking in search engines ...... 59 3.4 Limitations of the Heuristic Tool ...... 65 3.5 Scientific Research Terminology applied to a Usability Function Test ...... 68 Summary ...... 70 Chapter 4: Application of Research Method and Tool ...... 72 4.1 Data Collection ...... 72 4.1.1 Sample Selection: Purposive Sampling ...... 77 4.2 Analysis of the Data ...... 81 4.2.1 Group 1. Aspects related to source code ...... 81 4.2.2 Group 2. Aspects related to localisation ...... 99 4.2.3 Group 4. Aspects affecting search in the web site ...... 111 4.2.4 Group 5. Aspects affecting the web sites ranking in search engines ...... 115 4.3 Summary of the results ...... 140 4.3.1 Group 1. Aspects related to source code ...... 141

4.3.2 Group 2. Aspects related to localisation ...... 142 4.3.3 Group 4. Aspects affecting search in the web site ...... 143 4.3.4 Group 5. Aspects affecting the web sites ranking in search engines ...... 144 Summary ...... 145 Chapter 5: Discussion of the Results ...... 146 5.1 First Hypothesis and Research Question: Compliance with Guidelines ...... 146 5.1.1 Group 1. Aspects related to source code ...... 148 5.1.2 Group 2. Aspects related to localisation ...... 151 5.1.3 Group 4. Aspects affecting search in the web site ...... 152 5.1.4 Group 5. Aspects affecting the web sites ranking in search engines ...... 154 5.1.5 Guidelines Ranking...... 155 5.2 Second Hypothesis and Research Question: Comparison between Error Frequencies ...... 157 5.2.1 Group 1. Aspects related to source code ...... 159 5.2.2 Group 2. Aspects related to localisation ...... 160 5.2.3 Group 4. Aspects affecting search in the web site ...... 160 5.2.4 Group 5. Aspects affecting the web sites ranking in search engines ...... 161 5.2.5 A List of Usability Problems ...... 162 5.3 Statistical significance ...... 162 5.3.1 Chi-Square Goodness of Fit Test ...... 163 5.3.2 Spearman’s Rank Correlation Coefﬁcient ...... 169 Summary ...... 170 Chapter 6: Conclusions ...... 171 6.1 Summary of the Experimental Set-up ...... 171 6.2 Impact ...... 172 6.3 Limitations ...... 174 6.4 Future work ...... 176 Summary ...... 178 Bibliography ...... 180 Annexes ...... 187

Chapter 1: Introduction In a world of instant global reach and immediate ability of interaction with consumers, web design of multilingual web pages plays a central role. In fact, a well-designed web site can help build trust and loyalty among users (Singh and Pereira, 2012, pp. 1–2). If users feel smart and comfortable on a web site, they tend to stick around. When users feel discomfort in using a web site, they will most likely interrupt the navigation. In other words, the success of a web site depends on its usability—“how quickly and how well users can get things done on Web sites” (Nielsen and Loranger, 2006, p. xix). This users-oriented perspective of web design is closely related to the objectives of web usability, “which examines the reception of websites by means of empirical studies whose findings result in guidelines for web development” (Jiménez-Crespo, 2013, p. 36). People expect a lot from web sites today, and they are less and less tolerant of difficult sites and bad design (Nielsen and Loranger, 2006, p. xvii): “Ten years ago the Web was exciting to people. Today it’s routine. It’s a tool. If it’s convenient, they will use it; if not, they won’t” (Ibid., xv). In this context, usability has become more important than ever, as every design flaw may cause a negative impact on users and a lost business (Ibid.). In this Master thesis, I focus on the importance of usability and localisation in the domain of cultural tourism. More specifically, I analyse and compare Italian and English home pages of museums web sites in my region, Lombardy, Italy. In other words, I measure and evaluate the degree of usability and localisation in a sample of multilingual home pages.

The choice to address this research topic in my Master thesis comes from a personal experience. During my studies abroad, since I am an Italian native speaker, I was frequently asked from foreign friends, mostly Russian and Chinese, to help them plan a trip to Italy. Since their trips were mostly cultural, they used to ask me suggestions about the country’s cultural attractions, from the most famous ones to the less conventional, which remain off the beaten tourist track. However, when checking the web sites of Sicilian museums with two Chinese girls, it was extremely difficult to find user-friendly web sites with up to date design localised into English. The help of a native speaker was essential even to accomplish ordinary tasks, such as, for example, tickets purchase. Nonetheless, due to the bad and old design of some web sites, even I experienced difficulty orienting myself within the site. These facts made me curi- ous about the Italian cultural offer on the web and the level of web sites accessibility to foreign users. From these first observations, I received a negative impact about the level of usability and localisation of some web sites. I therefore decided to investigate this aspect in my Master thesis. My initial idea was to measure and evaluate the degree of usability and localisation of Italian museums web sites. However, this idea was too general and I needed 1. to generate specific research questions to narrow down my topic; 2. to develop a clear framework for examining the research problem. I decided to focus on museums web sites in my region, Lombardy, first because I know the territory and the cultural heritage of the region, and second because it was easier to reach tourism info-points for the data sample collection. Instead of

6 analysing whole web sites, I decided to focus on the home page. The reasons behind this choice are explained in Chapter 3 (Section 3.1). After defining the area of interest, I started reading about usability testing methodologies. The works of Nielsen (1993, 1995, 1995a, 1995b) provided me with knowledge about the discount usability engineering method (Chapter 2, section 2.3.2 and Chapter 3, section 3.2). Among all the methods outlined by Nielsen, I chose the heuristic evaluation method, which “involves having a small set of evaluators examine the interface and judge its compliance with recognised usability principles (the ‘heuristics’)” (Nielsen, 1995a). In this Master thesis, I, acting as a single evaluator, examined a sample of home pages and checked if they respect recognised usability guidelines. The heuristics on which my work is based are outlined by Andreu-Vall and Marcos in their article Evaluación de sitios web multilingües: metodología y herramienta heurística (2012). Hypotheses and research questions, as well as the theoretical framework of my analysis are explained in Chapter 3 (Section 3.1 and 3.5. See Table 2 and 10).

Localisation industry is engaged in producing an agreed set of standards. The importance of guidelines and standards in both usability and localisation does not have to be underestimated. Standards can be traced in ISO (International Organisation for Standardisation) and W3C (World Wide Web Consortium), as well as in the existing literature. Guidelines ensuring usability allow users to easily and efficiently complete a task, and determine whether the web site succeeds or fails. Localisation, which is defined as the adaptation of digital contents and products for sale and use in another locale (Jiménez-Crespo, 2013, p. 17), ensures the effective reception of the product in the target market: “Establishing localisation guidelines can increase the quality, accuracy, and user-friendliness of the international product version. More- over, it can significantly reduce the cost of localising your application into different languages” (Mi- crosoft, 2017). Hence, the compliance with recognised standards, guidelines and conventions concerning usability and localisation improves comprehension, usability and satisfaction in multicultural audiences.

Given the importance of cultural tourism in the region, following these guidelines becomes crucial for users, who are most likely also visitors of Lombard museums. According to the statistics of MiBACT— the Italian Ministry of Cultural Heritage and Activities and Tourism, in the last four years, visitors of Italian state museums (inbound and outbound) have increased from 38 million in 2013 to 50 million in 2017. This means that visitors increased by 31% (12 million more in four years), which lead to a 53% increase in earnings (70 million euro more). The proceeds are currently being used for scientific and didactic activities, as well as maintenance of museums collections (MiBACT, 2018). If we look at the regional figures, Lombard museums are the fifth most visited in 2017, after those of Lazio, Campania, Tuscany and Piedmont. Moreover, visitors in Lombardy have grown with a rate of +1.1%, from 1,830,495 visitors in 2016 to 1,850,605 in 2017 (Ibid). These figures are telling and show the importance of cultural tourism in both the country and the region.

In this Master thesis, I focus on 102 Italian and English home pages of Lombard museums web sites. My aim is to see if home pages respect the usability guidelines defined by Andreu-Vall and Marcos (2012), concerning internationalisation, localisation, search, SEO and web accessibility aspects. This Master thesis is organised as follows. In Chapter 2, I overview the state of the art about web localisation and usability. In Chapter 3, I introduce the research questions and present the research method and tool chosen for the analysis. In Chapter 4, I describe the process of sample collection and analyse the sample of home pages. I describe the emerging characteristics of the sample and critically analyse suitable and unsuitable items within home pages. I also focus on the error distribution, distinguishing between suitable and error- free home pages respecting usability guidelines and unsuitable home pages, presenting at least one error and not respecting the guidelines. In Chapter 5, I evaluate and discuss the results obtained. I also answer to my research questions and test the statistical significance of results. Eventually, in Chapter 6, I draw my main conclusions, considering the limitations and impact of my work.

Chapter 2: The Literature Review The present chapter provides a theoretical framework to contextualise the web sites usability function test carried out in this Master thesis. This chapter overviews the existing literature about localisation and focuses on the discipline main aspects. Firstly, it provides a distinction between translation and localisation (2.1), by resorting to a description of localisation main components (2.1.1). In the attempt of describing these components, it provides a comparison between different definitions of localisation, based on different perspectives and approaches to the subject — Translation Studies and localisation industry (2.1.2). It then focuses on web localisation (2.2), addressing to the dimensions of globalisation and internationalisation (2.2.1), as well as standardisation and customisation (2.2.2). It finally describes web usability and acceptability (2.3), focusing on usability definition (2.3.1) and studies on web usability testing (2.3.2).

2.1 From Translation to Localisation 2.1.1 Translation: Definition and Strategies Before giving a definition of localisation, I would like to attempt a definition of translation to clarify why these two disciplines are different and why many hesitate to define localisation as a subset of translation. Instead, they would rather define localisation as a different discipline.

If we decide to read the novel “One Hundred Years of Solitude” (Cien años de soledad) by Colombian author Gabriel Garcia Márquez, we can reasonably expect to find it translated from Spanish into our language on the shelf of every bookshop and library in our city. What we do not expect is the story of the Buendía family to be moved from Colombia to another country. We can read the novel in a language different from Spanish, but we cannot find the story adapted to the cultural context we live in (Microsoft, 2015). Translation is the process of converting written text to another language. It requires the full meaning of the source text to be rendered in the target language “with special attention paid to cultural nuance and style” (Esselink, 2000, p. 4). The translation of a text is not a univocal process, as different strategies and techniques can be applied by the translator. Applying different techniques means obtaining different versions, hence different translations of the same text, depending on historical constraints, audience expectations, and editorial requirements. In any case, the final product of a translation is something different from the original, since “translation is the most obviously recognisable type of rewriting” — as stated by Belgian linguist André Lefevere (Lefevere, 2017, p. 7). In fact, the image we get of an author heavily depends on the translation made of his work. For example, as Professor Hewson stated during the course “Critique des traductions”, we think to read Kafka in French, but what we read is only one of Kafka’s French versions (Hewson, 2018).

In his work “The Translator’s invisibility” (1995), the American theorist and translator Lawrence Venuti identifies two translation methods: foreignisation and domestication. Domestication is the strategy of making a text closely conform to the target culture, which may involve a loss of information from the source text (target-oriented translation). Foreignisation is the strategy of retaining information from the source text, deliberately breaking the conventions of the target language to preserve its meaning in the source culture (source-oriented translation).

2.1.2 Localisation: Components and Definitions Since localisation was born, discussions about its definition and delimitation have interested both scholars and professionals. Different parties with an interest in its definition (developers, industry experts, translation scholars, computational linguists and professional translators) have tried to define this term, that is why its definition varies depending on the context and perspective used (Jiménez-Crespo, 2013, p. 11).

In the following paragraphs, by referring to those elements proper of localisation, I discuss the notions of adaptation, locale, distribution and localisation process. Furthermore, I overview the definitions of localisation that mostly reached consensus among experts.

Adaptation: A Domestication Strategy

Adaptation is a specific feature of localisation. In a very broad sense, localisation can be defined as “the adaptation and translation of a text to suit a particular reception situation” (Pym, 2004, p. 1). I started this chapter mentioning Marquez’s novel: the text can be translated in different languages, while the context remains untouched. And yet, examples of context adaptation and cultural transposition of content can be found in translation as well, showing that adaptation certainly is a specific feature of localisation, but, as it concerns translation as well, it cannot be considered alone an element to distinguish between the two disciplines.

For example, the most famous translation of Lewis Carroll’s novel “Alice in Wonderland” into Russian is the one made by Vladimir Nabokov in 1923. For his translation of “Alice in Wonderland” (Anja v strane čudes), Nabokov opted for domestication strategies, realised in the text in the form of substitution and adaptation, which led to an almost complete russification of Carroll’s novel. Its young Russian readers could then identify themselves with the story and understand the complex world created by Carroll (Vid, 2008, p. 220). For example, the name of the main character, Alice, is rendered with Anja, a typical Russian name for little girls. Cultural references like “tarts” are substituted with typical Russian pies, “pirožki”; the flower daisy is substituted with dandelion, apparently a more typical Russian field flower (Vid, 2008, p. 224). Another example of cultural adaptation of this novel for a different public is the one made by the Catalan poet Josep Carner in 1927. For his Alícia en Terra de Meravelles, Carner’s as well recurs to domestication strategies, so much that Wonderland is not a foreign land, but Catalunya itself. In the 10 illustrations of the book by Lola Anglada, the Catalan Mediterranean landscape is soon recognisable, as well as the Spanish seeds of the cards — goblets, coins, sticks and swords instead of hearts, diamonds, spades and clubs (Chaparro, 2010, pp. 20–23; 26–27).

Both Nabokov and Carner’s works could be considered examples of localisation of the novel. They cannot be regarded as mere target-oriented translations, as they are both a transposition of language and context into another linguistic and cultural environment.

When defining localisation, the term “adaptation” is the most pervasive in both Translation Studies (TS) and localisation industry (Jiménez-Crespo, 2013, p. 15). In fact, industry definitions focus on “adapting the ‘look and feel’ of non-translated products so that they are accepted as local productions by users” (Ibid., 18). On the other hand, TS as well identify adaptation as “the performative action of the localisation process” (Ibid., 15).

The use of an extreme domestication strategy is certainly one of the main traits of localisation. We can state that “[l]ocalisation is therefore conceptualised as a target-oriented translation type” (Ibid., 18).

However, localisation can be distinguished from an extreme domesticated translation for other reasons: the reception situation, the type of product distributed, and its process-based nature.

Locale: The Reception Situation

The reception situation is usually referred to with the term “locale”, from which the word localisation comes from. In general terms, we can state that translation involves languages (even if it cannot be considered as a study of mere language problems), while localisation involves locales. However, the term “locale” does not simply indicate a combination of language and culture. This term, spread by the localisation industry, denotes a particular variety of language, used in a particular geographical area, with its local conventions regarding currency, date and hour setting, presentation of numbers and symbolic colour coding (Pym, 2004, pp. 1-2).

The borders of a locale are not easy to draw, as it is hard to define where one locale ends and another starts. According to Pym, (2004, pp. 22-23), the concept of locale is especially empirical:

From this empirical perspective, the limits of a locale can be defined as the points where texts have been (intralingually or interlingually) localised. That is, if a text can be adequately be moved without localisation, there is no new locale (Ibid., p. 22).

Therefore, according to Pym, locale show up only under certain circumstances: the need of a new localisation, due to the resistance to some process of distribution. Hence, localisation depends on distribution.

Pym (2004, p. 13) refers to the term “distribution” in the way it is used in marketing. For these reasons, neither “language” nor “community” can provide sufficient criteria to denote the kinds of places involved in distribution. This is the reason that explains the need to recur to the term “locale”. In fact, languages alone are insufficient to describe the places involved in localisation, as there are no natural borders between languages (Ibid., pp. 21-22).

Moreover, considering that a language can be spoken in more than one country, texts can be distributed from one community to another and yet not require localisation, because that language is able to find its appropriate receivers (Ibid., p. 21). In conclusion, the concept of “locale” cannot be equated neither to the concept of “language”, nor to the concept of “community”, “culture” or “state”. As afore-mentioned, localisation is not approached in social or political terms, but firstly in economic terms, recurring to the concept of distribution:

Distribution and localisation concern situation of contact and exchange, not lineal separations. Instead of using preconceptions about locales in order to form preconceptions about localisations, we can now use facts about distribution in order to describe contacts and differences between locales (Ibid., p. 23).

Today a locale is defined as “the combination of a sociocultural region and a language in industrial setting” (Jiménez-Crespo, 2013, p. 12). In software engineering, the term “locale” indicates the local market requirements where the product will be distributed, and it can be expressed as a language-country pair: e.g. French-Canada is one locale, French-France is another (Dunne, 2015, p. 551)

Definitions of the term focus on cultural conventions such as date, time and currency formats. Jiménez- Crespo (2013) provides another definition of locale, focusing on ISO language and country code:

A locale is expressed by the combination of the language code included in the international standard ISO 639, followed by the country code as stated in the standard 3166 (Jiménez-Crespo, 2013, p. 13).

ISO 639 is the international standard for language codes. Each language has a two-letter (639-1) and three-letter (639-2 and 639-3) code, designed as a lowercase abbreviation of the language (ISO, 2010). “The language codes are open lists that can be extended and refined. The job of maintaining these standards has been given to bodies known as Registration Authorities” (Ibid.). ISO 3166 is the international standard for country codes, designed as an uppercase abbreviation of the country name: Using codes saves time and avoids errors as instead of using a country’s name (which will change depending on the language being used) we can use a combination of letters and/or numbers that are understood all over the world (ISO, 2013).

For example, de–DE means German of Germany, while de–AT means German of Austria.

Cross-cultural Digital Product: The Object of Distribution

The concept of distribution and the definition of market places are used by Pym to create knowledge about localisation and its definition (2004, p. 29). We have seen so far that localisation is the adaptation of a product, so that it can be distributed in another locale.

Among the many definitions of localisation, it should be mentioned the one given by the Localisation Industry Standards Association (LISA),1 born in 1990:

Localisation involves taking a product and making it linguistically and culturally appropriate to the target locale (country/region and language) where it will be used and sold (LISA 2003, cited in Jiménez-Crespo, 2013, p. 13).

This definition also focuses on the notion of “locale”, but it refers to a product, and not text. In fact, the object of localisation is a material product that is being moved, or transferred (Pym, 2004, p. 13).

The kind of product that is distributed is better specified in the definitions given by Schäler (2010) and Dunne (2015):

[Localisation is] the linguistic and cultural adaptation of digital content (Schäler, 2010, p. 209).2

Localisation is an umbrella term that refers to the processes whereby digital content and products developed in one locale are adapted for sale and use in one or more other locales (Dunne, 2015, p. 550).

Adaptation is seen as the “additional component that localisation provides, as opposed to the textual or wordly nature of translation”. Another element that helps us distinguish between localisation and translation is the nature of the product analysed, as localisation addresses “non-textual components of products in addition to strict translation” (LISA, 2007, p. 11).

Furthermore, as localisation concerns digital products, even textual elements differ from the static concept of text. The text is not printed, but shown on a screen, it can be defined then as an “interactive digital text” (Jiménez-Crespo, 2013, p. 37). Consequently, the product is localised to be used and not only to be read. The receivers shift from being readers to being users.

[In web environments,] research has shown that reading slows down by 25% to 50%, and users do not read web text but rather scan the pages in search of the information that might draw their attention (Nielsen 2001, cited in Jiménez-Crespo, 2013, p. 37).

1 LISA ceased to exist in 2011. 2 Schäler also refers to localisation industry’s requirements to deliver multilingual and cross-cultural digital content (2008, p. 195).

The web site success is measured by its so called ‘stickiness’: “the ability to attract new and repeat visitors and keep them on a site” (LISA 2004, cited in Jiménez-Crespo, 2013, p. 35).

In this sense, localisation is certainly a multi-modal process, as it operates at different levels.

Localisation Process: The GILT Cycle

In both the afore-mentioned definitions (Schäler 2010, Dunne 2015), localisation refers to digital content and products. Moreover, in other definitions, localisation is referred to a series of processes, which is another important element to take in consideration when defining localisation.

Localisation, in fact, involves different activities, as it “does not exist in isolation, but forms part of a much wider complex of interrelated processes known as GILT (Globalisation, Internationalisation, Localisation and Translation)” (Jiménez-Crespo, 2013, p. 24).

In the 1980s, localisation could be considered as subset of translation where a domestication approach was applied. In fact, initially it was mainly a “translation on the computer for computer” (Van der Meer 1995, cited in Dunne, 2015, p. 550), “a matter of translating software” (Dunne, 2015, p. 550) for international markets, using an extreme domestication strategy (Jiménez-Crespo, 2013, p. 14).

The practice of translation remained relatively unchanged from the dawn of writing until the commoditiza- tion of the PC and the advent of mass market software ushered in the digital revolution in the 1980s (Dunne, 2015, p. 550).

With the growth of the Internet and the emergence of personal computing and software programmes (late 1970s – early 1980s), US software publishers started to widen their horizons, and target international users to grow sales abroad. Many US-based companies (e.g. Sun Microsystems, Oracle and Microsoft) had already become popular in domestic markets, and wanted to distribute their products in other countries, providing software programmes in languages other than English. They targeted Japan and Europe as their next major markets, and they firstly focused on the so-called FIGS countries (France, Italy, Ger- many and Spain). However, for these products to be accepted in the international market, software programmes had to be translated and adapted for the new users (Esselink, 2000, p. 5; Jiménez-Crespo, 2013, p. 8; Dunne, 2015, p. 550). As Dunne explains, it was necessary “to convert the software, so that users saw a product in their own language and firmly based in their own culture” (Dunne, 2015, p. 550). How- ever, “it soon became clear to practitioners that this work was related to, but different from and more involved than, translation” Lieu 1997 (cited in Dunne, 2015, p. 550).

Translation is only one of the different activities that form the localisation process. In addition to translation, localisation process include adapting icons, symbols, converting date and time formats, as well as currencies, expanding spaces for translated text and modifying navigation bars.

[Localisation is] the process by which digital content and products developed in one locale (defined in terms of geographical area, language and culture) are adapted for sale and use in another locale. Localisation involves: (a) translation of textual content into the language and textual conventions of the target language, (b), adaptation of non-textual content (from colours, icons and bitmaps, to packaging, form factors etc.) as well as input, output and delivery mechanisms to take into account the cultural, technical and regulatory requirements of that locale. In sum, localisation is not much about specific tasks as much as it is about the processes by which products are adapted […] (Dunne 2006a, cited in Jiménez-Crespo, 2013, p. 7).

Thus, localisation can ultimately be defined as a series of processes by which an object (digital product) is adapted (in all its parts – textual and not textual elements) to be distributed in another locale (a geographical area defined in terms of language, culture and industrial setting). Dunne’s definition refers to all the components discussed in this chapter (type of object, domestication approach, locale identification, industrial distribution, and the process-based nature of localisation, known as GILT. Localisation emerges then as part of a wider complex of processes, where each stage refers to different activities.

Despite the popularity reached by LISA definition, I would like to conclude the section with the definition of localisation provided by GALA (Globalisation and Localisation Association), as it enriches the definitions so far discussed by listing all the different activities included in the localisation process:

Localisation (also referred to as “l10n”) is the process of adapting a product or content to a specific locale or market. Translation is only one of several elements of the localisation process. In addition to translation, the localisation process may also include:

 Adapting graphics to target markets  Modifying content to suit the tastes and consumption habits of other markets  Adapting design and layout to properly display translated text  Converting to local requirements (such as currencies and units of measure)  Using proper local formats for dates, addresses, and phone numbers  Addressing local regulations and legal requirements

The aim of localisation is to give a product the look and feel of having been created specifically for a target market, no matter their language, culture, or location (GALA, 2011).

2.2 Web Localisation: Process and Tendencies Localisation was originally born to localise software programmes. Nowadays it encompasses a wider range of digital products, such as web sites, videogames, smart phone apps, and web search engines. Jiménez-Crespo (2013, p. 28) identifies five localisation types, ranked by business volume:  Web localisation  Videogame localisation  Software localisation  Small device localisation  Multimedia localisation This Master thesis focuses on web localisation. Web localisation could emerge some years after the in- vention of the World Wide Web (WWW), created by Tim Berners-Lee in the 1980s, thanks to technical innovations, such as Hypertext Mark-up Language (HTML). Web localisation developed after years of significant efforts in software products, and in the early 2000s overcame the market share of software localisation (Jiménez-Crespo, 2013, pp. 9–11).

2.2.1 The Web Localisation Process (GILT): Globalisation and Internationalisa- tion As afore-mentioned about localisation in general, web localisation as well can be regarded a process, involving a series of steps, known with the acronym GILT (Globalisation, Internationalisation, Localisa- tion and Translation). As translation and localisation have already been defined, the other two components are analysed in the following section.

Globalisation In general terms, globalisation is “the process by which a company breaks free of the home markets to pursue business opportunities wherever its customers may be located” (Esselink, 2000, p. 4). This process regards localisation industry as well. A definition of globalisation in localisation industry is the provided by LISA: Globalisation addresses the business issues associated with taking a product global. In the globalisation of high-tech products this involves integrating localisation throughout a company, after proper internationalisation and product design, as well as marketing, sales, and support in the world market (LISA, cited in Esselink, 2000, p. 4; Pym, 2004, p. 30) On the basis of this definition, we can state that there is one wide process called ‘globalisation’, of which ‘internationalisation’ and ‘localisation’ are parts (Pym, 2004, p. 30. In other terms, “globalisation covers both internationalisation and localisation” (Esselink, 2000, p. 4).

When the process starts, the aim is distributing a product globally (globalisation). First, the product needs to be rendered general (global). This process is what is called internationalisation. As next step, the product can be adapted to specific target markets, or locales. This process is what is called localisation (Pym, 2004, p. 30). With regards to web localisation, globalisation is referred to as […] the process of creating local or localised versions of a web sites, which we will refer to as “web site globalisation. Web sites globalisation refers to enabling a web site to deal with non-English speaking visitors […] (Esselink, 2000, p. 4).

Internationalisation Despite the name chosen to indicate this process, nations have little to do with internationalisation. The conceptual focus remains the notion of locale (Pym, 2004, p. 30). Internationalisation is “the process whereby the culture specific features are taken out of a text in order to minimise the problem of later distributing that text to a series of locales” (Ibid., p. 31). It can be defined as the process of removal of all those cultural specific components in a digital product, in order to make it available for localisation and the subsequent distribution in target locales.

If internationalisation means generalising a product (leveraging) for locales distribution, then, as suggested by Pym (2004, 30), a better term to define it could be “interlocalisation (since locales would be the conceptual focus throughout)” or “delocalisation (since the processes basically involve the taking out of local elements).” In this sense, it may appear as a “decontextualization”, the opposite of localisation, or a “reverse localisation” (Schäler 2008c, cited in Jiménez-Crespo, 2013, p. 32). This last term should be used with caution, as it can also be used to denote an increasing trend, “in which products developed in small markets are localised for sales in the U.S. and other traditional sources of localisation” (LISA, 2007, p. 35). The main reasons for internationalising products are the following (Esselink, 2000, p. 25; Pym, 2004, p. 30):  Reduction of overall expenses, if the home product is first prepared for localisation  Assurance of the product functionalities and acceptability in international markets  Assurance of the product localisability  Smoothness of localisation process

The main purpose of internationalisation is to make localisation faster, easier, higher in terms of quality, and more cost-effective (Cadieux and Esselink, 2004).

Internationalisation implies avoiding some problems during the localisation process, which is the reason why is frequently associated with the term “enablement”. Furthermore, it implies leveraging the product,

17 so that it can be exposed to foreseeable changes for every target locale. The problem is that the combination of internationalisation and localisation is usually just called localisation (Pym, 2004, p. 38). How- ever, as localisation is a cycle, it splits into globalisation, internationalisation, the actual localisation, where products are culturally adapted, and finally translation, where text is translated. To overcome ambiguities during the translation phase, the language must be as clear as possible. The bulk of localisation is performed in English, and this language must be somehow “controlled”, even if it is not easy for a writer whose native language is English to “write globally” (Esselink, 2000, p. 27). Nonetheless, “the text must be written with translation in mind” (Ibid.), creating what is called an “internationalised text” (Pym, 2004) or “writing for translation”, either known as “writing for the web”. As stated by Esselink (2000, p. 27), internationalisation has “international users in mind”. That is why language as well must be internationalised. Idiomatic expressions and jargon should be avoided, preferring a plainer and simpler language and style. On this matter, consistent phrases and terminology should be used, sentences should be clear and grammatical relations between words should always be explicit. Furthermore, images and symbols should be avoided, as well as references to religion, seasons, holidays etc.

Furthermore, translated text is about 30% longer when translating into languages that use Latin-based alphabets (Rockwell, cited in Singh and Pereira, 2012, p. 39). Translation agency Kwintessential reports that when translating from English into French or Spanish, text can expand about 15% up to 30%. When translating into Dutch or German, text can be even 35% longer (Kwintessential, 2018). Similarly, localisation company SDL recommends that is better to allow a large section of text to expand by 30% (Singh and Pereira, 2012, p. 39). On the other hand, Singh and Pereira (Ibid.) report that text can reduce to 10% in size when translated from English into Chinese. If there are length restrictions, localisers will have to resort to shorter synonymous words, or adapt the size of the text. In ideographic languages, like Chinese and Japanese, vertical expansion may also be required. Similarly, Semitic languages, such as Arabic and Hebrew, may need bidirectional text and navigation bars moving from right to left.

2.2.2 Web Localisation Tendencies: Standardisation vs. Customisation Standardisation

As explained above, web globalisation includes both internationalisation and localisation. Further, it concerns other issues rather than mere language problems. Singh and Pereira provide a definition of both internationalisation and localisation:

Web site internationalisation is the process through which back-end technologies are used to create modular, extendable, and accessible global web site templates that support front-end customisation, and web site localisation is the process of the front-end customisation, whereby web sites are adapted to meet the needs of specific international target markets (Singh and Pereira, 2012, p. 7).

When we speak of web localisation, we generally focus on two tendencies: customisation of web sites for speciﬁc markets across the world, and standardisation of web sites to minimise cultural distance (Ibid., p. 5). These tendencies “coexist to varying degrees in every localised web site” (Canavese, 2017, p. 229), and one does not necessarily exclude the other.

Standardisation is the process of developing technical standards. Standards are “specifications for products, services and systems, to ensure quality, safety and efficiency” (ISO, 2018). International standards are “instrumental in facilitating international trade” (Ibid.).

Localisation industry is engaged in producing an agreed set of standards. Web designers should respect these standards to ensure the product usability and acceptability. However, standards are not always respected and, information about localisation standards is dotted around different organisations because of “the lack of a centralised steering authority” (Wright, 2006, p. 244). Localisation standards can be classified in different categories: content creation, terminology, locale specification, project management, “as well as basic standards, such as those that govern language and character codes” (Ibid.).

Sue Ellen Wright (2006, 244-245) provides a clear categorisation of standards: industrial standards (such as product quality, consumer protection, interoperability of different types of products and consistency across international borders etc.), de facto standards (not promulgated by standards organisations or professional groups – e.g. Microsoft), and language standards. This last category is explained by Wright and McClure (Ibid.). They classify language standards into seven major groups:

 Base standards (SGML, XML, HTML, etc.),  Content creation, manipulation, and maintenance,  Translation standards,  Terminology and lexicography standards,  Taxonomy and ontology standards,  Corpus managements standards,  Language and locale-related standards.

For the usability function test conducted in this Master thesis, I refer to language and locale related standards, as well base standards to check if the content of elements and attributes in a sample of HTML documents complies with an agreed set of standards and guidelines. For example, I refer to the following ISO standard and W3C (World Wide Web Consortium) guideline on the use of character encoding: ISO

106463, as explained in Wright (2006, pp. 541-542), and W3C (2008) guideline on Unicode. Standards and guidelines considered throughout the analysis are explained in Chapter 3.

Customisation

A number of studies during the years have tried to support the idea that web sites must be culturally customised, stressing the importance of web design in a world of instant global reach and immediate ability of interaction with consumers. Web design of multilingual web pages plays a central role. In fact, a well-designed web site can help build trust and loyalty among users (Singh and Pereira, 2012, pp. 1–2). According to Singh and Pereira, cultural adaptation of web content and design positively affect navigation and rating of web sites. Providing empirical evidence, they state that “customers are more comfortable with, and also exhibit a more positive attitude toward, web sites that are consistent with their cultures and languages” (Ibid., 18).

The debate over standardisation versus customisation, started by Elinder in the 1960s and Levitt in the 1980s, has been complicated by the emergence of the World Wide Web, “as it is a global communication medium where technology makes mass customisation or adaptation possible, while forces of global integration and the emergence of transnational web style […] justify the use of a standardised web marketing and communication strategy” (Ibid., p. 5):

The advocates of Standardisation approach” argue that, in the era of globalisation, “cultural distance will be minimised, leading to convergence of national cultures into a homogeneous global culture” (Ibid.). On the other hand, the advocates of Customisation affirm that “country-specific web content enhances usability, reach, and web site interactivity, leading to more web traffic and business activity on the Web (Ibid.).

An interesting attempt to shed light on the relation existing between standardisation and customisation is the study carried out by Canavese (2017), as he measures the degree of standardisation and customisation in a corpus of 49 web sites for the language pair Italian-German. He recurs to seven localisation categories to analyse the content of the “About us” page (from standardised content to adapted content), and to five localisation strategies to analyse the macro- and microstructure of the page (from adapted structure to standardised structure).

As shown by the above-mentioned study, web localisation could rely on both standardisation and customisation approaches. Further, one technique is not applied at the expense of the other, but they could coexist on different levels. As can be seen from his study, the coexistence of these two tendencies, standardisation and customisation, is reflected in his corpus in the attempt to localise content, but standardise structure, or vice versa, to varying degrees in every localised website.

3 ISO 10646 is the international standard for Universal Coded Character Set (UCS). 20

In practice, standardisation is an essential step to overcome localisation problems, whose fixing is costly and time consuming. In an email to the author (March 2018), Canavese clarified the usefulness of standardisation and why it tends to be more common than customisation. In order to ensure standardisation, web masters recur to content management systems (CMS), software programs used to create and manage digital content. CMS do not allow to modify the structure of a web page, language content does not exhibit special features, so that it can be easily rendered into another language. International companies and organisations can decide to standardise or localise web content to varying degrees and for different reasons. For example, a company can decide to make its product look “exotic” using a foreignisation strategy, or one may want to underline the origin of a product because it could help grow sales. It is the case, for example, of Italian food market. If a situation of business-to-consumer (B2C) occurs, it could bring more benefits to localise content, while in a business-to-business (B2B) situation, it may not be the case.

Taking all these considerations into account, and without forgetting that localisation activity heavily depends on time, human, economic resources and market audience (Jiménez-Crespo, 2013, p. 33), research- ers distinguish among different levels in the amount of customisation. For example, Singh and Pereira (2012) propose a classification of web sites in five levels, from absence of localisation to a “total immer- sion in target locale” (Ibid., 35): 1. Standardised web sites, 2. Semi-localised web sites, 3. Localised web sites, 4. Extensively localised web sites, 5. Culturally adapted web sites. Before this classification, Mi- crosoft distinguished into three different levels of localisation (Ibid., 34): 1. Enabled products, 2. Local- ised products, 3. Adapted products.

Highly customised web sites are usually related to big companies operating in the international market, as they have the resources to provide their web sites with a high level of localisation and customisation.

As people around the globe not only speak different languages, but have different visions of the world, according to Yunker (2003, p. 309), it is important to understand “the visual languages of the world — not only how people speak, but how they see”. In such a context, web design plays a major role. Accord- ing to Yunker, the prerequisite for any web site is to be usable and acceptable. The notions of usability and acceptability are explained in the following section.

2.3 Web Design: Usability and Acceptability “Usability” is the term chosen by Nielsen in the early 1990s to indicate what is commonly denoted by the term “user-friendly”. Jakob Nielsen, principal of Nielsen Norman Group and “world’s leading expert on Web usability” (U.S. News & World Report, cited in Nielsen Norman Group 2018), believes the term “user-friendly” to be inappropriate for two reasons: 1. The term is “anthropomorphic”, as “users don’t

21 need machines to be friendly with them” (1993, p. 23); 2. “It implies that users’ needs can de described along a single dimension by systems that are more or less friendly” (Ibid.).

Professionals have coined other terms in recent years: e.g. CHI (computer-human interaction), HCI (human computer interaction, preferred by those who want to put human first), UCD (user-centred design), MMI (man-machine interface), HMI (human-machine interface), UID (user interface design), HF (human factors), ergonomics, and others (Ibid.).

Among all these terms, Nielsen chooses to use the term “usability” and defines it, as “a quality attribute that assesses how easy user interfaces are to use. The word ‘usability’ also refers to methods for improving ease-of-use during the design process” (2012). Thus, web usability concerns “the reception of web sites by users by means of empirical studies […]” (Jiménez-Crespo, 2013, p. 36).

In Yunker (2003, p. 280) usability is defined in comparison with acceptability. Usability means that the web site has to be usable:

Before people around the world can use your web site, they first have to find your web site. And when they get there, the site has to be usable, no matter what language they speak or how sophisticated their web skills (Ibid.).

On the other hand, acceptability means that the web site has to be acceptable, hence culturally sensitive: “People have hard time accepting web sites that they don’t trust. The site itself may be usable, but if it isn’t culturally sensitive, it still might not get the sale” (Ibid.).

From Yunker’s perspective, usability and acceptability are the essential requirements for any multilingual web site. Usability concerns a web site availability on the Internet, its ability to quickly load on browsers and communicate an intended meaning through visual elements. Acceptability concerns a design being able to be culturally sensitive (thus, locally acceptable), and to create positive impression, credibility and trust.

Usability and acceptability appear to be two parallel and essential aspects of a web site, whose aim is to target users in other locales. However, Nielsen ascribe usability and acceptability in a wider paradigm, shown in the figure below.4

4 The permission to use this image from Nielsen (1993, p. 25) was accorded to me by Jeanette Pidanick (Project Coordinator at the Nielsen Norman Group) by e-mail (22 February 2018). 22

Figure 1. “A model of the attributes of system acceptability” from Nielsen (1993, 25). Nielsen (1993, p. 24) describes usability as “a narrow concern compared to the larger issue of acceptability”.

Acceptability investigates “whether the system is good enough to satisfy all the needs and requirements of the users and other potential stakeholders, such as the users’ clients and managers” (Ibid.). Overall acceptability is a combination of a social dimension and a practical dimension.

Social acceptability concerns the audience attitude towards a web site feature, how people react to it, whether they consider a feature socially desirable or offending. While practical acceptability concerns different categories such as cost, support, reliability, compatibility with existing systems, and usefulness, described as an “issue of whether the system can be used to achieve some desired goal” (Ibid.). Usefulness splits in two categories: utility and usability. The first is “the question of whether the functionality of the system in principle can do what is needed”, while the second is “the question of how well users can use that functionality” (Nielsen, 1993, p. 25).

In Nielsen (2012) utility, which refers to the design’s functionality, investigates whether a design responds to users’ needs, while usability studies how easy and pleasant design features are to use. Usefulness of a web design is a combination of both utility and usability:

It matters little that something is easy if it’s not what you want. It’s also no good if the system can hypo- thetically do what you want, but you can’t make it happen because the user interface is too difficult (Nielsen, 2012).

Whether acceptability is seen as a superior or parallel category compared to usability, Yunker provides a definition of acceptability stressing the importance of what Nielsen calls social acceptability, and high- lights the role of user’s culture, which influences the perception of web design. On this matter, Singh and Pereira (2012) provide a method to test social acceptability and analyse design reception in different cultures. As afore-mentioned, they stress the importance of cultural adaptations. By recurring to Hofstede

(1980) and Hall’s (1976) classifications of cultural dimensions, they provide a cultural values framework for web design (Singh and Pereira, 2012, pp. 54–55). Five cultural dimensions (power distance, collectiv- ism vs. individualism, femininity vs. masculinity, uncertainty avoidance, and low- vs. high context) have been used to study the user’s perception of web sites in different countries (Ibid.).

The role of cultural context in multilingual web site has been also analysed by Hillier (2003), highlighting the importance of considering cultural context in the process of multilingual web site development. Ac- cording to Hillier, “the usability of a user interface refers to the fluency or ease with which a user is able to interact with a system without ‘thinking’ about it. This implies they can do so ‘naturally’ or without feeling ‘discomfort’, either physical or mental” (2003, p. 9). Therefore, interacting with a web page is regarded as a “communicative action” influenced by cultural factors (2003, p. 10).

In broad terms usability occurs when there is a ‘meeting of minds’ between the designer (through their creation) and the user. In terms of a web site, this means the user is able to understand and comprehend the web site implicitly, in that there are no ‘uncomfortable’ elements. This discomfort could be caused through inappropriate, offensive or misinterpreted elements (Hillier, 2003, p. 11).

His study aims to provide evidence that a web site should conform to the norms of the culture in which that language is based.

Another study analysing the impact of language and culture on perceived web site usability is the one conducted by Nantel and Glaser (2008). They assume that translation creates cultural distance. In a web environment, they provide evidence that “perceived usability increases when the web site [is] originally conceived in the native language of the user” (2008, p. 112).

In conclusion, Nielsen’s definition of acceptability, and his distinction into social and practical acceptability (1993, 2012), allows us to focus, besides social and cultural references, on all those practical aspects “of a system with which a human might interact […]” (1993, p. 25).

2.3.1 Web Usability: Definition and Components [Usability is] a quality attribute relating to how easy something is to use. More specifically, it refers to how quickly people can learn to use something, how efficient they are at using it, how memorable it is, how error- prone it is, and how much users like using it (Nielsen and Loranger, 2006, p. xvi).

In Nielsen (1993, 2006, 2012), usability is defined by five components: learnability, efficiency, memorability, errors, and satisfaction.

Learnability is the ability of users to perform basic tasks the first time they see a design (2012):

The system should be easy to learn so that the user can rapidly start getting some work done with the system. […] Learnability is in some sense the most fundamental usability attribute, since most systems need to be 24

easy to learn, and since the first experience most people have with a new system is that of learnability (1993, pp. 26-27).

Efficiency is related to the speed taken to cover these tasks, once users have learnt the design (2012):

The system should be efficient to use, so that once the user has learned the system, a high level of produc- tivity is possible (1993, p. 26).

Efficiency refers to the expert user’s steady-state level of performance at the time when learning curve flattens out (1993, p. 30).

Memorability is the ability of users to remember the design and re-establish proficiency, after a period of not using the web site (2012):

The system should be easy to remember, so that the casual user is able to return to the system after some period of not using it, without having to learn everything all over again (1993, p. 26).

Error “is any action that does not accomplish a desired goal, measured by counting the number of such actions” (Ibid., p. 32). This category takes into account the number and severity degree of errors made by users (2012):

The system should have a low error rate, so that users make few errors during the use of the system, and so that if they do make errors they can easily recover from them (1993, p. 26).

Ultimately, satisfaction indicates the degree of comfort and amusement of users when using the web site (2012):

The system should be pleasant to use, so that users are subjectively satisfied when using it; they like it (1993, p. 33).

Satisfaction is particularly important for non-work environment (e.g. creative painting, interactive fiction, games).

As afore-mentioned, we currently live in a world of instant global reach, where information and products can be reached by a simple click. In such a context, usability plays a major role in web design. As previously explained, as web sites are conceived to be used, their success is measured by their ability of attract- ing new visitors (stickiness) and keeping them on the web site (Jiménez-Crespo, 2013, p. 37):

If a website is difficult to use, people leave. If the homepage fails to clearly state what a company offers and what users can do on the site, people leave. If users get lost on a website, they leave […]. There are plenty of other websites available; leaving is the first line of defence when users encounter a difficulty (Nielsen, 2012).

As the web sites should be as clear as possible, I have stressed the importance of a simple and easy-to- translate language. Structural and design elements as well need to follow certain requirements to improve a web site usability. As afore-mentioned, the existing literature on web localisation offers guidelines on web writing (Esselink, 2000; Nielsen and Loranger, 2006; European Commission, 2016b), and so does with the more cultural aspects, providing publications on the role of cultural context (Hillier, 2003; Nantel and Glaser, 2008) and cultural conventions (Nord 1997; Krug 2006, cited in Jiménez-Crespo, 2013, p. 38). Guidelines and standards on more practical aspects, such as usability, are also largely provided by organisations (I mentioned standards and guidelines from ISO and W3C), as well as authors like Nielsen himself: “Current collections of usability guidelines typically have on the order of a thousand rules to follow and are therefore seen as intimidating by developers” (Nielsen, 1993, p. 19).

Nielsen (1993, p. 91) proposes a categorisation of guidelines, distinguishing between general guidelines (applicable to all user interfaces), category specific guidelines (for the kind of system being developed), and product specific guidelines (for the individual product).

The number of ways in which usability can be measured is also extensive. Usability is measured by having a number of test users (Nielsen, 1993, 1996a, 1996b, 2009). Such users can be distinguished in three categories: expert, casual, and novice (1993, p. 31). A casual user can be distinguished from an expert user or a novice depending on frequency use of a web site:

Casual users are using a system intermittently rather than having the fairly frequent use assumed for expert users. However, in contrast to novice users, casual users have used a system before […] (Ibid.).

Usability testing is therefore conceived by Nielsen with a focus on users. During usability testing on a web site, usability professionals are asked to measure the five above-mentioned categories. The following section is entirely devoted to a description of different methods to assess usability, where usability tests conducted on web sites are just one of these methods.

2.3.2 Usability Assessment Methods: Web Usability Testing In his book “Usability Engineering”, Nielsen (1993, p. 17) explains that usability specialists often propose using “the best possible methodology”, in order to assess usability. However, according to the author, the best methods not necessarily provide the best results, as he believes that “better results can be achieved by applying more careful methodologies” (Ibid.). For this reason, he prefers focusing in achiev- ing good results “with respects to having some usability engineering work performed” (Ibid.).

He proposes what he calls a “discount usability engineering method”, based on the use of four techniques: user and task observation, scenarios, simplified thinking aloud and heuristic evaluation. User and

26 task observation consists in observing users while they perform given tasks without interference. Scenar- ios are a reduction of level of functionality and number of features. They are defined as a “cheap kind of prototype” (Ibid. p. 18), which is a refinement of a product based on cutting on down the number of features (vertical prototyping), and reducing the level of functionalities (horizontal prototyping) (Ibid. pp. 94-95). Simplified thinking aloud method involves observing one user a time, while he uses the system thinking aloud. It is based on verbalised thought of users while performing a task. Ultimately, heuristic evaluation “is done by looking at an interface and trying to come up with an opinion about what is good and bad about the interface” (Ibid., p. 155). Individual evaluators can conduct a heuristic evaluation according to certain rules, such as those mentioned in guidelines documents. This method is further described in the next chapter, as the function usability test conducted in this Master thesis relies on its principles.

Discount usability engineering method by Nielsen is summarised in the following table:

Method Characteristics User and task observation Observation of users as they perform given tasks. Scenarios Reduction of number of features and level of functionalities. Simplified thinking aloud Observation of one user a time as he thinks aloud while performing given tasks. Heuristic evaluation Systematic inspection of user interface design to find usability problems, performed following usability guidelines.

Table 1. Summary of Discount usability engineering method by Nielsen (1993, 17). According to Nielsen, user testing on real users is the most important method to assess usability, as it gives direct information about concrete problems encountered by users. Other authors as well — such as Dumas and Redish (1999, p. 22), believe that usability testing, rather than other techniques based on the usability engineering approach, provide two unique benefits: they can be used to actually improve the usability of the product, as well as the process “by which products are designed and developed” (Ibid.). This implies that someone must use the results obtained from a usability test: “A usability test is successful only if it helps to improve the product that was tested and the process by which it was developed” (Ibid., p. 25).

In Dumas and Redish (Ibid., p. 22), a usability test is defined by five characteristics:

 Improvement of the product usability,  Participants representing real users, 27

 Participants doing real tasks,  Observation and recording of what participants do and say,  Data analysis, diagnosis of real problems, and recommendations to fix problems.

If one of the above-mentioned characteristic is missing, there is no usability test. In fact, these five characteristics allow us to distinguish usability testing from other usability assessment methods, based on the usability engineering approach, for e.g. heuristic evaluation, and research studies, where the existence of a certain phenomenon is investigated (Ibid.). Moreover, these characteristics allow us to differentiate a usability test from a quality assurance (QA) test or a function test, “which has a goal of assessing whether the product works according to its specifications” (Ibid.). Different goals and concerns allow us to differentiate one test from another.

Usability testing applies to all kinds of products, and it brings additional benefits, besides those already mentioned: it changes people’s attitude about users and, it “can break down the wall between those who create the product and those who use it” (Ibid., p. 33).

Usability testing relies on both quantitative and qualitative data from “the participants together with tester’s observations and users’ comments” (Ibid., p. 25). Quantitative data means data based on numbers, while qualitative data includes all non-numeric data, such as words, images and sounds (Oates, 2006, pp. 245–266). Dumas and Redish observe that in many cases if quantitative data are not provided, recommendations from experts can be rejected: “They sometimes base major decisions on just a few of the numbers from a usability test without considering the range of data that a usability test provides” (Ibid., p. 35).

Usability testing and scientific research are both empirical methodologies: they focus on observations and experience. However, we can distinguish one from the other because of the following differences: goals, participants, variables, weighing of observations, and data analysis (Ibid.).

Goals: The goals of usability testing have been already explained. The goals of scientific research are to improve and extend our scientific knowledge in a specific field. Within this context, the goal of empirical research is to test the validity of a hypothesis by performing an experiment that will confirm or reject the hypothesis. According to Dumas and Redish (1999, p. 36), the goal of a research study is to test whether a phenomenon exists or not, and provide evidence of its existence or inexistence.

In a usability environment, a usability test aims to uncover usability problems to improve a product usability. That is why a test aiming to demonstrate the existence of a particular feature could not be defined as a usability test.

Participants: Furthermore, usability testing focuses on users. The data sample in usability testing is represented by these users, participants representing real users selected on the basis of a certain criterion, or no criterion at all (random sampling). A usability test is conducted on a convenience sample: “people from the appropriate population whom you happen to find and who are available for you” (Ibid., p. 37). In statistics, population refers to the total set of data (individuals) with shared characteristics, which can be observed and tested.

In scientific research, users are not necessarily considered for the analysis, as the data sample can be of any nature. On this purpose, Saldanha and O’Brien (2013) distinguish among product-oriented research, process-oriented research, participant-oriented research and context-oriented research (case study).

Further, data can be selected according to different criteria, e.g. random sampling, stratified sampling, cluster sampling, etc.

Variables: For what concerns variables, they can be independent (they will not vary during the study), or dependent (they will vary during the study when an input is applied).

The dependent variable is the core concept we are trying to assess in our research question. We expect it to change when it is exposed to varying treatment. The independent variables, on the other hand, are things that we manipulate in order to see what the effect is on our dependent variable (Saldanha and O’Brien, 2013, p. 25).

In research studies, the variables that want to be studied must be defined and isolated. Whereas, in a usability test, they cannot be usually isolated, and the independent variable is usually the interface (Dumas and Redish, 1999, p. 37)

Weighing of observations: In both a scientific study and usability test, data are collected and measured. However, in a usability test “observations of the test team and the comments of test participants are given more weight in diagnosing problems than they are in a research study” (Ibid.).

Data analysis: Ultimately, usability tests include descriptive statistics, even if inferential statistics can also be considered: Dumas and Redish discuss the use of inferential statistics, such a t-tests (Ibid., p. 38). In research studies, the results are frequently tested and discussed by means of inferential tests, such as t-tests or chi-squared tests.

To conclude, according to the reasons explained above recurring to Nielsen (1993), Dumas and Redish (1999) clarifications, the object of this Master thesis cannot be properly defined a usability test. Instead, it can be described as a usability function test conducted by a single more or less expert reviewer, based on the heuristic evaluation method.

Summary This chapter aims to compare different definitions of localisation, and critically reflect on them. Locali- sation components are analysed to highlight the specific traits of the discipline. More specifically, I explained that localisation relies on a domestication strategy and aims to adapt products, not mere texts, to distribute them from a source to a target locale for mainly economic reasons. Further, in order to reach the international market (globalisation), such products need first to be rendered neutral (internationalisation), then they can be locally adapted to client markets (localisation). Focusing on web localisation, two tendencies have been analysed. Even if one does not automatically exclude the other, they are presented in the reviewed literature nearly as two opposite tendencies. On the one hand, standardisation allows avoiding problems and ensures the product usability. On the other hand, customisation wants to provide culturally acceptable products. The role of culture in web localisation and usability has been analysed. Lastly, the dimensions of usability and acceptability have been described recurring to Nielsen and Yunker’s definitions. To conclude the chapter, a distinction between different methods of usability assessment has been provided to introduce the function test carried out in this Master thesis.

Chapter 3: Presentation of Research Method and Tool The aim of this chapter is to describe the heuristic method and tool chosen for the usability function test carried out in this Master thesis. First, I introduce the hypotheses and research questions (3.1). Second, I explore the main characteristics of the heuristic evaluation method (3.2), focusing on the definition of standards and guidelines (3.2.1). Third, I describe the tool chosen for my heuristic evaluation on multilingual web sites (3.3). Lastly, I explain the use of the heuristic tool, i.e. how to check the compliance with specific guidelines and how to calculate the severity scores, but I also introduce the limitations related to the tool (3.4). To end this chapter, I summarise the concepts I am subscribing to in this Master thesis (3.5).

3.1 Research Questions and Hypotheses The present research begins with theoretical assumptions. My choice of methodology depends on these assumptions as well as on my research questions and hypotheses. Hypotheses are “specific types of research questions that are not phrased as questions but as statements about relationships” (Saldanha and O’Brien, 2013, p. 18). A research question is a question the researcher tries to answer by interrogating the data. Thus, the research question can sometimes be rephrased as a hypothesis: one might express a hypothesis in relation to a question (Ibid.).

My research questions arose from empirical observations: I started to explore the home pages of museums web sites in Lombardy, and ask myself if those home pages could be considered in accordance with the standards. Different dimensions of web localisation (web design, accessibility, usability etc.) focus on different categories of standards and guidelines. I decided to focus on one dimension — usability, and to evaluate usability aspects of the home pages collected. However, I did not want to base my work on a test made on users. I did not evaluate users’ reactions and performances because I was more interested in evaluating those usability attributes that are independent from specific goals and users. Hence, the decision to inspect usability attributes by myself using the heuristic evaluation method and referring to a checklist of guidelines (section 3.3). Nonetheless, a test on users could be considered for future studies that would enrich and complete the one carried out in this Master thesis. In fact, “[i]t is empirically proved that the most effective evaluation (i.e., the highest amount of usability problems discovery) is achieved by systematically combining evaluation and empirical testing” (Garzotto, 2004). Likewise, better results can be achieved when more evaluators are involved. Nielsen (1995b) recommends “to use three to five evaluators since one does not gain that much additional information by using larger numbers”. Empirical evidence suggests that on average five evaluators identify 75-80% of usability problems (Lapin, 2017). Nonetheless, due to time and budget constraints, the usability function test carried out in this Master thesis is performed by a single evaluator. In the future, further research efforts could include more evaluators to increase its external validity. 31

My sample consists of 102 HTML documents of home pages referring to major Lombard museums web sites. More specifically, 51 home pages are in Italian, whereas the other 51 are their English localised counterpart. The geographical area concerned is Lombardy, an Italian region in the north-west of the country.5 The figure below shows the position of the museums whose web sites are considered for the analysis (from lombardia.abbonamentomusei.it/Musei).6 Further details on the data sample are provided in chapter 4.

Figure 2. A map of Lombard museums. After quickly scanning the home pages as a preliminary step, I asked myself a question: do multilingual home pages in my sample respect usability guidelines?7 From this research question, I made some assumptions: I expected my sample of home pages to respect usability guidelines in 51% of cases. I chose this percentage because if at least 51% of the home pages respect usability guidelines, it means that more than half of the sample is consistent to guidelines. If the sample respects usability guidelines in more than 51% of pages, then results will be better than expected and can be considered positive. Otherwise, less than 51% of pages means that results can be considered negative. A guideline is considered respected if at least 51% of pages follow that guideline.

A secondary research question is to test if the usability results are the same across the two versions, i.e. usability results can differ between the Italian (IT) and English (EN) versions. Usability results are most likely to vary in the two versions. In fact, I expect to find less errors in the Italian version of the home

5 Data sampling is explained in chapter 4 (4.1 Data Collection). 6 The web site has been last accessed on 30 April 2018. 7 The usability guidelines I am referring to are explained in section 3.3 (Heuristic Guideline: A Tool for Heuristic Evaluation), where I present the heuristic evaluation tool chosen to test the hypothesis and answer this question. Whereas, section 3.2.1 (From broad Heuristics to Guidelines and Standards) clarifies the difference between standards and guidelines. 32 pages, as these pages are originally designed in Italian. In other terms, I expect error frequency to be higher in the English home pages. If we rephrase these statements in the form of a question, the second research question of my study is the following: is there any difference in the number of errors (error frequency) across the two versions? Is error frequency in the Italian pages lower than in the English ones? Results concerning the number of errors in the two versions are compared to check if they respect usability guidelines in different proportions.

I chose to focus on the home page because it is the first web page users see when visiting a web site. Since it is the first, it reflects the image of the product, like the front page of a newspaper. Hence, I expected home pages to be dedicated more resources in terms of web usability:

Homepages are the most valuable real estate in the world. Millions of dollars are funnelled through a space that’s not even a square foot in size. […] The homepage is the most important page on any website, getting more page views than any other page. Of course, users don’t always enter a website from the homepage, […]. However, one of the first actions these users will take after arriving at a new site is to go to the homepage. […] The homepage is the most important page on your site and we feel it warrants extra attention […] (Tahir and Nielsen, 2002, p. 1).

The home page certainly gets more visits than any other web page—according to Nielsen and Loranger, (2006, p. 28) a home page gets 40% of time users spend on a web site. However, even if users spend almost half of their time on the home page, this time is quite limited. According to Nielsen and Loranger, (2006, p. 30) users spend from 25 to 35 seconds on the home page, depending on their experience. Hence, the home page has to introduce a whole web site in half a minute. More precisely, the home page has to communicate what the site is about, what product or service is offered, who is the company or institution or person behind the web site and, finally, how to get to the most relevant sections of the web site (Ibid.). Nonetheless, the time spent on the home page decreases with each subsequent visit (Ibid., p. 32).

Therefore, I chose to analyse the home page not only for its importance, but also because I assumed it was the easiest page to scan: since it is supposed to convey a standardised form of information, I assumed errors were more detectable. Also, in terms of time, as the time spent on the home page decreases with the number of visits, I thought I could analyse usability aspects properly in a convenient amount of time.

After defining the object of my analysis and my expectations, I asked myself another question: how can I assess consistency and conformance to standards and guidelines? I was then keen on finding a set of criteria I could refer to, and a method to measure usability attributes. I found both a set of guidelines and a method to test my data sample in the article “Evaluación de sitios web multilingües: metodología y herramienta heurística” (2012, pp. 254-260) by Mar Andreu-Vall and Mari-Carmen Marcos (section 3.3). The method (3.2) and the tool (3.3) chosen are described in the following sections of this chapter.

On the basis of my observations and my perceptions, I formulated a hypothesis (H1): the sample of home pages respects usability guidelines in at least 51% of cases (H1= µ ≥ 51%). The null hypothesis

(H01) is that home pages respect usability guidelines in less than 51% of cases (H0 = µ < 51%).

The second hypothesis concerns the number of errors in the two linguistic versions. I assumed that IT home pages present less errors than EN home pages (H2): error frequency in the IT pages is lower than error frequency in the EN pages (EFIT < EFEN). The null hypothesis (H02) is that error frequency in the

EN pages is higher than error frequency in the IT pages (EFIT > EFEN).

Hypotheses generation is followed by hypotheses testing. As aforementioned, this chapter describes the method and tool chosen for testing. While the actual testing is described in the following chapter (4). The test conducted on the sample will confirm the hypotheses (H1, H2) rejecting the null hypotheses (H01,

H02) or it will reject the hypotheses (H1, H2) confirming the null hypotheses (H01, H02). I am interested in disproving the null hypotheses, and demonstrating the opposite, i.e. the alternative hypotheses (H1, H2) are true.

For what concerns variables, as already mentioned in Chapter 2 (Section 2.3.2), they can be independent or dependent. For what concerns the first hypothesis and research question, the independent variable is the sample of 102 multilingual home pages, while the dependent variable is the compliance with the heuristic guidelines, presented in section 3.3. If the home page or an item inside the home page respects the guideline, that home page or item is considered suitable. If the home page or an item inside the home page does not respect the guideline, that home page or item is considered unsuitable. For what concerns the second hypothesis and research question, the independent variable is the sample of 102 multilingual home page, but the analysis focuses on the Italian and English home pages separately. Whe dependent variable is the number of errors (error frequency).

H1 Domain Region Object Dependent variable Results (Independent variable) Localisation Lombardy 102 multilingual home Compliance with heuristic guide- Yes (suitable) / Usability pages (Total sample) lines No (unsuitable) Tourism (Are heuristic guidelines respected in at least 51% of pages?) H2 Domain Region Object Dependent variable Results (Independent variable) Localisation Lombardy 51 Italian home pages Error frequencies Yes/No Usability 51 English home pages (Do IT home pages present less er- Tourism rors than EN home pages?)

Table 2. Research variables. 3.2 Heuristic Evaluation Method Heuristic evaluation is outlined at great length by Nielsen (1993; 1995b, 1995a; 1995; 2009), and, as explained in the previous chapter, it is one of the four techniques which constitute the discount usability engineering method.

More specifically, heuristic evaluation is a usability engineering method carried out by observing a user interface design for finding usability problems, and trying to form an opinion about the interface strengths and weaknesses. It can be performed by a single evaluator or teams of evaluators, who conduct the evaluation test according to certain rules (Nielsen, 1993, p. 155; 1995b).

Heuristic evaluation involves having a small set of evaluators examine the interface and judge its compliance with recognised usability principles (the ‘heuristics’) (Nielsen, 1995a).

In Nielsen 1993, heuristic evaluation is described as a “systematic inspection” of a user interface design for usability (1993, p. 155). Usability inspection consists in a set of methods based on a user interface inspection performed by evaluators. The goal of usability inspection is to find usability problems in the design. However, some methods also focus on issues like the severity of usability problems or the inspection of user interface specifications that have not yet been implemented (Nielsen, 1995).

Besides heuristic evaluation, usability inspection includes the following types of evaluations:

Heuristic estimation, where “inspectors are asked to estimate the relative usability of two (or more) designs in quantitative terms” (Ibid.).

Cognitive walkthrough, “a more explicitly detailed procedure to simulate a user’s problem-solving process at each step through the dialogue, checking if the simulated user’s goals and memory content can be assumed to lead to the next correct action” (Ibid.). Pluralistic walkthrough, which consists of “group meetings where users, developers, and human factors people step through a scenario, discussing each dialogue element” (Ibid.). Feature inspection, which “lists sequences of features used to accomplish typical tasks, checks for long sequences, cumbersome steps, steps that would not be natural for users to try and steps that require extensive knowledge/experience in order to assess a proposed feature set” (Ibid.). Consistency inspection, having “designers who represent multiple other projects inspect an interface to see whether it does things in the same way as their own designs” (Ibid.). Standards inspection, having “an expert on an interface standard inspect the interface for compliance” (Ibid.). Formal usability inspection, which “combines individual and group inspections in a six-step procedure with strictly defined roles to with elements of both heuristic evaluation and a simplified form of cognitive walkthroughs” (Ibid.). While heuristic evaluation, heuristic estimation, cognitive walkthrough, feature inspection, and standards inspection are normally performed by a single evaluator at a time, pluralistic walkthrough and consistency inspection are “group inspection methods” (Ibid.).

In this context, heuristic evaluation remains “the most informal method and involves having usability specialists judge whether each dialogue element follows established usability principles (the ‘heuristics’)” (Ibid.). As seen so far, heuristic evaluation is one of the four techniques on which the discount usability engineering method is based, as well as a systematic inspection method. The relationship among heuristic evaluation and the other usability methods mentioned in this Master thesis is shown in the following diagram:

Figure 3. Heuristic evaluation among other usability assessment methods. 36

If we try to focus on the meaning of the term “heuristic evaluation”, we can agree that evaluation means to consider something in order to make a judgement about it. Whereas, the meaning of the world “heuristic” becomes less obscure once we analyse its etymology: it comes from the ancient Greek verb “εὑρίσκω”, which means “to find” or “to discover”. In simple terms, heuristic evaluation is an evaluation whose aim is to discover problems. In a web usability context, the goal of heuristic evaluation, as afore- mentioned, is “to find the usability problems in a user interface design […]” (Nielsen, 1993).

In principles, heuristic evaluation is performed by having each individual evaluator inspect the interface alone. When all evaluations have been completed, evaluators are allowed to communicate their findings and have them aggregated with the findings of other evaluators. This procedure ensure unbiased evaluations (Nielsen, 1993; J. Nielsen, 1995b).

Results are recorded as written reports. Otherwise, observers take notes of evaluators’ comments as they inspect the interface. Written reports have the advantage to be formal records, even though they need a manager to read and aggregate them. Nonetheless, an observer’s intervention can be of help in case the evaluators have little expertise, but “adds to the overhead of each evaluation session” (Nielsen, 1993, p.157; 1995b).

Three main differences between heuristic evaluation and usability user testing can be outlined (Nielsen, 1993, pp. 157–158, 224; 1995b):

In a user test, the observer, called the experimenter, has to interpret users’ actions, making it possible to conduct a test even if users do not know anything about web design. On the other hand, the observer in a heuristic evaluation can be of help in case a user is in trouble, but they normally only have to record comments, without any interpretation. In a user test, observers are reluctant to give evaluators more hints than necessary, as the goal of the test is to uncover problems, and users are asked to answer questions about the interface. In contrast, in a heuristic evaluation, observers can answer questions as they could be of help to evaluators “to better assess the usability of the user interface” (Nielsen, 1993, p. 158). Heuristic evaluation uncovers individual usability problems and can address expert user issues. On the other hand, user testing involves real users, making it possible to discover their real needs (Nielsen, 1993, p. 224). As explained above, a single evaluator can perform a heuristic evaluation, as each evaluator works separately. However, as aforementioned, heuristic evaluation is generally difficult for a single individual to do, since obviously more evaluators have more probabilities to uncover all usability problems in a user interface, and different people find different usability problems (Nielsen, 1993, p. 158; 1995b).

In a heuristic evaluation session, “the evaluator goes through the interface several times and inspects the various dialogue elements and compares them with a list of recognised usability principles”, the afore- mentioned “heuristics” (J. Nielsen, 1995a). Heuristics can be defined “as general rules that seem to describe common properties of usable interfaces” (Ibid.). Heuristics can be presented in the form of a checklist. In addition to the checklist, “the evaluator is also allowed to consider any additional usability principles or results that come to mind that may be relevant for any specific dialogue element” (Ibid.).

In principle, evaluators are free to decide how they want to conduct the inspection. A general recommendation (Ibid.) would be to first get the look and feel of the interface to get acquainted with the flow of the interaction and the general scope of the system. The second step would be to focus on specific elements keeping in mind how they all fit into the larger whole.

Further, “since the evaluators are not using the system as such (to perform a real task), it is possible to perform heuristic evaluation of user interfaces that exist on paper only and have not yet been implemented” (Ibid.).

Lastly, “the output from using the heuristic evaluation method is a list of usability problems8 in the interface with references to those usability principles that were violated by the design in each case in the opinion of the evaluator” (Ibid.). It is not enough for evaluators to simply say that they do not like something. They should list each problem separately; they should refer to the various usability principles that where violated, and explain why each wrong aspect of the interface is a usability problem (Ibid.). In fact, heuristic evaluation “is based on combining inspection reports from a set of independent evaluators to form the list of usability problems” (Nielsen, 1995).

The main characteristics of heuristic evaluation, as explained in this section, are summarised in the following table:

Goal: Heuristic evaluation aims to find usability problems in a user interface.

Method: Interface is inspected according to some principles, called “heuristics”.

Participants: Heuristic evaluation is performed by having each individual evaluator inspect the interface on his own.

8 It is worth mentioning that heuristic evaluation does not provide a systematic way to fix the usability problems, nor does it provide a way to assess the quality of any possible redesigns (Nielsen, 1993, p. 159). 38

Results: Findings from heuristic evaluation are aggregated and recorded as written reports.

Ultimate goal: Heuristic evaluation ends when a list of usability problems is produced.

Advantages: Heuristic evaluation finds individual usability problems and can address expert user issues.

Limitations: Heuristic evaluation does not involve reals users and does not directly test their needs.

Table 3. Heuristic Evaluation in summary. 3.2.1 From broad Heuristics to Guidelines and Standards As aforementioned, heuristic evaluation relies on principles, called “heuristics”. Jakob Nielsen identifies ten general principles for interaction design. They are called “heuristics” because “they are broad rules of thumb and not specific usability guidelines” (J. Nielsen, 1995a):

1. Visibility of system status: “The system should always keep users informed about what is going on” (Ibid.);

2. Match between system and the real world: “The system should speak the users’ language, with words, phrases and concepts familiar to the user” (Ibid.);

3. User control and freedom: When users choose system functions by mistake, they need a clearly marked “emergency exit” to leave the web site or the web page “without having to go through an extended dialogue” (Ibid.);

4. Consistency and standards: “Users should not have to wonder whether different words, situations, or actions mean the same thing” (Ibid.);

5. Error prevention: Design should prevent a problem from occurring in the first place. Either eliminate error-prone conditions or check for them and present users with a confirmation option before they com- mit to the action (Ibid.).

6. Recognition rather than recall: “Minimise the user's memory load by making objects, actions, and options visible” (Ibid.);

7. Flexibility and efficiency of use: Accelerators can speed up the interaction for the expert user and “allow users to tailor frequent actions” (Ibid.);

8. Aesthetic and minimalist design: Dialogues should not contain information which is irrelevant or rarely needed. Every extra unit of information in a dialogue competes with the relevant units of information and diminishes their relative visibility (Ibid.).

9. Help users recognise, diagnose, and recover from errors: “Error messages should be expressed in plain language (no codes), precisely indicate the problem, and constructively suggest a solution” (Ibid.).

10. Help and documentation: “Even though it is better if the system can be used without documentation, it may be necessary to provide help and documentation. Any such information should be easy to search, focused on the user's task, list concrete steps to be carried out, and not be too large” (Ibid.).

In practice, these so-called heuristics refer to broad principles, while usability standards and guidelines refer to rules that are more specific.

According to Nielsen (Nielsen, 1993, p. 91), “guidelines list well-known principles for user interface design which should be followed in the development project”. He distinguishes among general guidelines (applicable to all users interfaces), category-specific guidelines (for the kind of system being developed), and product-specific guidelines (for the individual product). An example of general guideline could be to “provide feedback” to user about a system’s state. This guideline can become category-specific if intended for graphical user interfaces (“ensure that the main objects of interest to user are visible on screen and that the most important attributes are shown”). It would be possible to develop this guideline into a product-specific one, for example if the design of a graphical file system is considered (“use different icons to represent different class of objects”) (Nielsen, 1993, p. 92).

According to the same author, “a standard ensures that your users can understand the individual interface elements in your design and that they know where to look for what features” (Nielsen, 1999). A standard does not ensure that users will know how to manage the interface features. In addition, it does not ensure that the system will have the features users expect to find (Ibid.).

Standards can be national or international, industry standards, and in-house standards. International standards have already been mentioned in the previous chapter, when discussing internationalisation. They are “instrumental in facilitating international trade” (ISO, 2018). The afore-mentioned ISO standards are an example of international standards. Industry standards are “promoted by various operating system and window system vendors”, and specify “the look and feel of user interfaces in great detail” (Nielsen, 1993, p. 232). Industry standards can differ to varying degrees, for such reason developers should take care when changing platforms, as details from other industry standards can show up (Ibid.). In-house standards are “developed locally with an organisation”, it is therefore possible “to aim for a high degree of usability of the standard itself” (1993, p. 233).

Further, the difference between standards and guidelines is explained by Nielsen (1999, p. 92):

Standards specify how the interface should appear to the users, while guidelines provide advice about the usability characteristics of the interface. […] Hopefully, a given standard follows most of the traditional usability guidelines so that the interface designed according to the standard will also be as usable as possible.

3.3 Heuristic Guidelines: A Tool for Heuristic Evaluation As discussed so far, heuristic evaluation implies a design interface inspection according to some general principles, called heuristics, and more specific principles: guidelines and standards. The aim of such evaluation is to uncover usability problems in a user interface. In our case, the heuristic evaluation is carried out using a heuristic tool: a checklist proposed by Mar Andreu-Vall and Mari-Carmen Marcos in their article “Evaluación de sitios web multilingües: metodología y herramienta heurística” (2012, pp. 254-260).

Andreu-Vall and Marcos (2012) proposed a methodology for web sites evaluation to multilingual and multicultural audience. First, standards and validated recommendations have been checked by authors, then they have been listed in what is called a heuristic guideline. “The results is a checklist for evaluating websites with international audiences, both monolingual (with or without globalisation) and multilingual (with or without localisation)” (2012, p. 254).

I decided to use their heuristic tool because it agrees with the rules discussed by Nielsen (1999): 1. It is a clear and extensive checklist; 2. In many cases, it provides examples of string code that can be checked when assessing usability; 3. It complies with most popular standards, such as ISO and W3C web standards; 4. In addition, it is supported by a template (a Google document),9 where such standards and guidelines are listed, and supplemented with hypertext links to the direct source online; 5. Andreu-Vall and Marcos’ checklist has already been used for a post-graduate work by former colleague Federica Quazzico (2016) in a case study on web accessibility.

The heuristic guide presented by Andreu-Vall and Marcos (2012) consists of 24 guidelines, classified in five categories (or groups):

 Group 1: Aspects related to source code  Group 2: Aspects related to localisation  Group 3: Aspects related to visual elements  Group 4: Aspects affecting search in the web site  Group 5: Aspects affecting the web sites ranking in search engines

9 The heuristic tool is available in the form of a Google document at http://goo.gl/LJTR5. I exported it in an XLSX document. A template with the checklist in form of questions, scores for error frequency, non-compliance, and sources consulted by authors is available (Andreu-Vall and Marcos, 2012a). 41

A score showing the usability impact of non-compliance is assigned to every guideline. Scores can vary from 1 to 3 (Ibid., p. 255):

 1 indicates a minor error: it doesn’t prevent user’s interface performing a task  2 indicates a medium error: it makes it more difficult for user’s interface to perform a task  3 indicates a major error: it prevents user’s interface performing a task

The heuristic tool for evaluating web sites usability is explained in the sub-sections below, while an insight on its limitations is provided at section 3.4 and Chapter 6 (section 6.3).

3.3.1 Group 1. Aspects related to source code This category refers to the elements of the source code HTML. This section covers the needs of internationalisation. More specifically, these guidelines are thought to ensure that web sites support the needs of many people around the world who use different writing systems:

If you internationalise, you design or develop your content, application, specification, and so on, in a way that ensures it will work well for, or can be easily adapted for, users from any culture, region, or language (W3C, 2016d).

For example, many countries have characters sets that are not confined to Latin alphabet. If a web site aims to cover the needs of internationalisation, it has to show a number of characteristics. The analysis of the elements listed in this section is carried out by inspecting the source code: after com- piling a corpus of 102 web pages (51 IT, 51 EN), I exploited it using Notepad ++.10 The advanced search functionality option of this tool was then used to inspect each file and determine the compliance to the usability guidelines described below.

1.1 Unicode codification (charset UTF-8): This guideline specifies the Universal Coded Character Set (UCS), which must be UTF-8 (8-bit Unicode transformation format). A character set is a group of characters (Yunker, 2003, p. 37). The mapping of characters is known as encoding, which means that each character must be assigned a number for the computer to read it as a code (Ibid.). A character set can have more than one encoding, which means that it can be encoded in several ways—for example, when working with Japanese, one character set can have more encodings. The most popular encodings for Japanese are Shift-JIS and JP-EUC, which must be specified in the charset attribute of the meta element (Ibid., p. 42).

10 Data collection (Chapter 4, section 4.1) covers a period going from November 2017 to March 2018. Data analysis (Chap- ter 4, section 4.2) covers a period going from 11 April 2018 to 9 May 2018. 42

Unicode is “the mother of all character sets” (Ibid., p. 43), as it can display 94,140 characters, which is enough to represent the majority of human languages. In fact, the ability to display that amount of characters exceeds the number of characters in any human language currently known” (Makki and Leppert, 2006, p. 8). Unicode is a character set, but it can have different encodings. The most popular encoding is UTF-8, which allows to display languages such as Chinese, Arabic, English, Russian and Japanese in one web page, and it is therefore said to be “universal” (Ibid., p. 42-43). UTF-8 (Unicode) encoding “is the standard that enables worldwide communication, without restrictions imposed by the language that the user uses or region that they live in”, and “provides a solid foundation for processing all text worldwide” (W3C, 2008c). For the analysis of this attribute, I had to inspect the HTML document of each home page, find the charset attribute, and check if the content of the charset attribute is UTF-8 (W3C, 2008b; Andreu- Vall and Marcos, 2012, p. 255; ISO, 2017):

The non-compliance score for this category is 3 (major error).

1.2 Consistency between url language and content language: A Web address is used “to point to a resource on the Web such as a Web page.” (W3C, 2008a). This guideline specifies that the words used as part of a web address (url) must be in the same language of the page that the url is directing to (Andreu-Vall and Marcos, 2012, p. 255). For this category, I checked if the url of different localised versions and content language are the same. In other terms, I checked if language codes are visible in the url, and if they match with the language of the home page. For the analysis of this attribute, I scanned the home page and inspected the top navigation bar.

An example of translated url is provided in Figure 4, showing the “Plan your visit” page of Uffizi Museum in Spanish and Russian:

ES  http://www.uffizi.org/es/planea-tu-visita/11

RU  http://www.uffizi.org/ru/посещение-музея/

11 The web pages in Spanish and Russian have been last accessed on 28 March 2018. 43

Figure 4. “Plan your visit” page of Uffizi Museum in Spanish and Russian. The non-compliance score for this category is 2 (medium error).

1.3 Declaration of content language in every page heading: The language in which the page is written must be specified at the top of the HTML document. The code for the language attribute to the element is specified in the standard ISO 639 (ISO, 2010).

Language tags are used to indicate the language of text or other items in HTML and XML documents. Use the lang attribute to specify language tags in HTML, and the xml:lang attribute for XML (W3C, 2014b).

If a particular variant of the language is used, the country code must be indicated, as reported by ISO 3166 (ISO, 2013). One should look at the content language declaration at the top of the HTML document. For example, the code for Spanish of Spain will be indicated as follows: . Focusing on the page heading, I checked the lang attribute by inspecting each HTML document. The non-compliance score for this category is 3 (major error).

1.4 Change of content language in the same web page: When a language change occurs inside the web page, the change must be indicated. For example, if the web page is in Spanish, but a paragraph is written in Italian, this change should be indicated with

(the tag for the paragraph element), followed by the language attribute lang (Andreu-Vall and Marcos, 2012, p. 256):

For this guideline, I checked if there is textual content in a language different from that declared in the lang attribute of the element. The focuses of the analysis are first textual strings in a foreign

44 language, and second the lang attribute. By scanning the home page, I could detect foreign words, sentences and paragraphs. After finding such words, sentences and paragraphs, by inspecting the HTML document, I checked if the language change was signaled by the use of the lang attribute in the element referring to that textual content. If the language was not indicated, this was considered as one error.

Another way to indicate which part of the text should be treated in another language is applying the lang attribute to other elements whose content is textual. The following example may help clarify this statement (W3C, 2014b):

En janvier, toutes les boutiques de Londres affichent des panneaux

SALE, mais en fait ces magasins sont bien propres!

When textual content is hooked in a element, it can be styled using CSS (Cascade Style Sheets) (w3schools, 1999c). For example, the following CSS code :lang(en-GB) { color: red; } colours all English text red, as long as the language attribute is used to specify the language: lang="en-GB" (W3C, 2014b). The machine can therefore recognise the English text (SALE) thanks to the lang attribute and colour it red, as indicated in the CSS code.

The declaration of language change is particularly important for visual impaired people who have the page read by a screen reader. If the language change is not declared, the screen reader keeps on reading in one language.

The non-compliance score for this category is 3 (major error).

1.5 Text expansion: As discussed in the previous chapter, text expansion must be considered when localising a web site from one language to another. Since the length of a target text is likely to be different from that of the source text, the layout of a web page should be flexible. “This phenomenon, known as text expansion or text contraction, occurs because translation is not a one-to-one process” (Yunker, 2003, p. 176). It depends on both the verbosity of the translator and the target language (Ibid.). A study on the world “view” in the Flickr user interface shows that this world expands of 260% when translated into French (consultations), and 300% when translated into Italian (visualizzazioni) (W3C, 2007c; Ishida, 2014).

Text expansion is checked here by counting characters with an online tool.12 The focus of the analysis is the number of characters. The non-compliance score for this category is 1 (minor error).

1.6 Language change specification in the anchor tag of a link: This rule specifies that when adding text to a link, the language of the web page that the link is directing to must be pointed out in the

12 The tool used to count words is the “Online Character Count Tool”, available at http://www.charactercountonline.com/. However, any online tool for words counting might be used to perform the same task. 45 anchor tag using the hreflang attribute (Andreu-Vall and Marcos, 2012, p. 257). The tag defines a hyperlink, linking one page to another. This guideline “may help the reader avoid wasted time linking to pages they can’t read”, but it may also “become out-of-date and so give incorrect information” (W3C, 2007b). For example, a web page in Spanish with a link to a page in English must be indicated as follows (Andreu-Vall and Marcos, 2012, p. 257):

Otra página

To analyse the hreflang attribute to hyperlink elements, I checked the content language of the web pages linked to my home pages through hyperlinks. By scanning other pages, I detected the language. As a second step, I checked if the language change was specified by inspecting the HTML document and searching the hreflang attribute.

The non-compliance score for this category is 1 (minor error).

3.3.2 Group 2. Aspects related to localisation This category aims at supporting the adaptation of different elements of the web page to users in the target locale (Andreu-Vall and Marcos, 2012a, p. 257). Some elements can be analysed as seen for Group 1, while others cannot be assessed looking at the source code. In this case, I had to check the presence of certain features online.

2.1 Date format: Different cultures around the world have different ways to write and read the date. For example, 10/03/2018 is read in Italian using the cardinal number ten (not the ordinal like in English), saying the name of the month, and using the cardinal number 2018. In Russian, the same date is read as follows: ordinal numeral for the day of the month in neuter nominative, month in genitive, ordinal numeral for the year in masculine genitive, and год (year) in genitive. Reading numbers certainly depends on language constraints,13 but the world does not agree not even in the way dates are written with numbers. For example, most European countries use the format day/month/year, China and Japan prefer the format year/month/day, while the format month/day/year is unique to the United States (W3C, 2007a). Date formats can become ambiguous in cases like 01/03 – imagine you want to book a flight for 1 March, and the system does a reservation for 3 January (Andreu-Vall and Marcos, 2012, p. 257).

To avoid ambiguities, “ISO 8601 describes an internationally accepted way to represent dates and times using numbers” (ISO, 2012):

13 Ordinals in English are created adding suffix –th to the number, while in Italian, as in Russian, ordinal numbers are adjec- tives, and as such they have a gender. Some languages like Russian also use cases to express what other languages express by using prepositions. Grammatical rules obviously affect the way dates are read. 46

ISO 8601 specifies a format of YYYY-MM-DD. 2003-04-02 is clearer than 03/04/02. (Some prefer to modify ISO 8601 by using an abbreviation for the month to make it more clear, for example 2003-Apr-02, but then it is no longer locale neutral.) (W3C, 2007a). The advantages of using ISO 8601 is that it is computer friendly and unambiguous. However, people are more comfortable with their “natural” date formats (W3C, 2007a). Another option proposed by W3C is to make the month and year obvious (Ibid.), by using a name for the month (abbreviated or not) and use 4 digits for all Gregorian year numbers (e.g. 2 April 2003; 2 Apr 2003). This method is completely unambiguous and people friendly, but it is less computer friendly, and it takes more space (Ibid.): In some locales even the abbreviation for a month name may be longer than three characters. (In French the first three letters of June and July are the same, juin and juillet). Allowing extra space for this exacerbates the space problem (Ibid.). For such reasons, ISO 8601 remains the most computer friendly and transparent method, as it recurs to numerical digits and avoid cross-cultural ambiguities. For this section, one is supposed to check if date formats are either respecting ISO 8601 or adapted according to target language rules.

I expect date formats to comply with ISO 8601 date format. “Dates should be formatted by the following format: YYYY-MM-DD”, as reported in the Annex to the supplementing Regulation (EU) No 600/2014 of the European Parliament and of the Council on markets in financial instruments with regard to regulatory technical standards (European Commission, 2016a). This is an example of how standardisation ensures transparency.

However, since I do not want to privilege standardisation over customisation, I also consider appropriate date formats that refer to local conventions. The importance of culturally customised web sites has already been discussed. For such reason, customised date formats are also considered appropriate. Further, Andreu-Vall and Marcos (2012, p. 257) as well agree to consider appropriate the most familiar form to users.

The languages considered in this analysis are Italian and English.

Italian: As web sites taken in consideration are localised from Italian into foreign languages (mostly English), I do not expect to find errors in date formats. Nonetheless, I here provide examples of how date should be written in Italian (Lesina, 1986; Petricola, 2011).

According to the Italian manual of style by Lesina (1986, p. 340), dates are expressed in the sequence day month year (3 novembre 1949), and they can be preceded by the day of the week (lunedì 3 novembre 1949). The names of days and months can be abbreviated (3 nov. 1949 or lun. 3 nov. 1949), though it is

47 preferable to avoid abbreviations. When dates are expressed in figures, numbers must be separated by hyphens (-) without space, as in 3-11-1949. The year should always be written out in full, since in shorten forms, such as 3-11-49, the saving is minimum. Lesina points out that some cultures may intend it as March 11th 1949. However, the date format dd/mm/yyyy is the most familiar to Italian language. He also mentions ISO 8601, but he says it is unsuitable for domestic use (1949-11-03).14 The following table summarises all the possible ways to write the date both in letters and figures (Lesina, 1986, p. 340; Petri- cola, 2011). The example chosen is Monday, 5 October 2017:

Index Italian date formats Notes

1. 5 ottobre 2017 2. 5 ottobre ‘17 Abbreviation of the year should be avoided. 3. 5 ott. 2017 4. lunedì 5 ottobre 2017 5. lun. 5 ott. 2017 6. 5.10.2017 7. 5-10-2017 8. 5/10/2017 9. 5.10.17 Abbreviation of the year should be avoided. 10. 5-10-17 Abbreviation of the year should be avoided. 11. 5/10/17 Abbreviation of the year should be avoided. 12. 5.X.2017 Less frequent. 13. 5-X-2017 Less frequent. 14. 5/X/2017 Less frequent. Table 4. Italian date formats. According to previous considerations, to avoid ambiguities only elements number 1, 3, 4, 5, 6, 7, and 8 are considered fully appropriate for the purposes of this study. As the non-compliance score for this category is 3, compliance score (0) is assigned only in case the date is presented as in the afore-mentioned cases.

English: For what concerns English, I make a distinction between British English and American English. The main source for British English guidelines on date formats is Oxford University: “New Hart’s rules”

14 Months expressed in Roman numbers are not common, but it is still a possibility (XI = November) (Petricola, 2011). The 9 August could then be written as follows: 9.VIII.2010, 9-VIII-2010, 9/VIII/2010 (Petricola, 2011). Dots and slashes can be used instead of hyphens. In special events, the day as well can be written using Roman numbers (il IV Novembre) (Lesina, 1986, p. 340). 48

(Ritter, 2005), adapted from “The Oxford guide to style” by R. M. Ritter, and “University of Oxford Style Guide” online (University of Oxford, 2014). The main source for American English is “The MLA Style Manual” (Achtert and Gibaldi, 1985), published by The Modern Language Association of America.

When writing dates, figures are used for days and years. Using cardinal numbers is preferable to using ordinal numbers: e.g. 12 August 1960 (Achtert and Gibaldi, 1985, p. 83; Ritter, 2005, p. 189). The form 12th August 1960 should be avoided, unless quoted verbatim: it is better not to use ordinal numbers (-st, -nd, -rd, -th) with dates, though they are read as such.

It is suggested to write just the number and month – and never precede the number with “the” (e.g. 13 April) (University of Oxford, 2014). “An incomplete reference may be given in ordinal form”, e.g. on the 18th (Ritter, 2005, p. 189). Days of the week are added to emphasise or avoid confusion. When comma is used “a terminal second comma is required if the date is worked into a sentence” (Ibid.):

e.g. The Modern Superstitions conference is on Friday 14 April (University of Oxford, 2014).

On Tuesday, 2 November 1993, the day dawned frosty (Ritter, 2005, p. 189).

Comma is not used between month and year: e.g. in June 1831 (Ibid.)

When using letters, in British English, date format is shown in the order day/month/year, without inter- nal punctuation: 2 November 2003 (Ibid.). Whereas in American English, the preferred order is month/day/year: e.g. November 2, 2003 (Ibid.; Achtert and Gibaldi, 1985, p. 83): “Be consistent in writing dates: use either ‘22 July 1986’ or July 22, 1986” but not both” (Achtert and Gibaldi, 1985, p. 83).

Names of days and months, both in British and American English, should be written in full. However, upper-case abbreviations can be used in notes or for space issues (Ritter, 2005, p. 179). When dates are expressed in all-figure forms, British English uses the format dd/mm/yyyy, using dots or slashes. The year can be given in full or shortened: e.g. 2.11.2003 or 2/11/03 (Ritter, 2005, p. 190). Whereas, in Amer- ican English the preferred format is mm/dd/yyyy separated by slashes rather than dots: e.g. 11/2/03 (Ritter, 2005, p. 179).

The ISO date format (yyyy-mm-dd) is separated by hyphens, and even if its popularity is increasing in technical and computer context, it remains unfamiliar to many (Ibid.). Therefore, texts intended for users who speak different varieties of English should spell out the name of the month or use the standard upper-case abbreviation to avoid ambiguity (Ritter, 2005, p. 410). The following table summarises all the possible ways to write the date both in letters and figures (Achtert and Gibaldi, 1985; Ritter, 2005; Uni- versity of Oxford, 2014). The example chosen is 10 April 2011:

Index British English dates American English dates Notes

1. 10 April 2011 10 April 2011 2. 10 Apr. 2011 10 Apr. 2011 3. April 10, 2011 4. Apr. 10, 2011 5. 10/4/2011 4/10/2011 Confusing 6. 10.4.2011 7. 10/4/11 4/10/11 Confusing 8. 10.4.11

Table 5. British and American date formats. To avoid ambiguities only elements number 1, 2, 3 and 4 are considered fully appropriate for the purposes of this study, unless the locale is specified.

The non-compliance score for this category is 3 (major error).

2.2 Time format: Time formats as well vary according to the locale. ISO 8601 defines as well a standardised way of presenting time: hh:mm:ss (Kuhn, 1999), where two-digit hour (hh), two-digit minute (mm), two-digit second (ss) and one or more digits for the decimal fraction of second(s) are used.

The complete format for time expression according to ISO 8601 is represented as follows: YYYY-MM- DDThh:mm:ss.sTZD (e.g. 1997-07-16T19:20:30.45+01:00), TZD represents the time zone designator (Z or +hh:mm or -hh:mm) (Wolf and Wicksteed, 1997). This standard “helps cut out the uncertainty and confusion when communicating internationally” (ISO, 2012). In fact, some cultures use the 12 hours format, while others prefer to use the 24 hours format (e.g. 5 PM, vs. 17:00):

The 24h time notation specified here has already been the de-facto standard all over the world in written language for decades. The only exception are a few English speaking countries, where still notations with hours between 1 and 12 and additions like “a.m.” and “p.m.” are in wide use. The common 24h international standard notation is widely used now even in England (e.g. at airports, cinemas, bus/train timetables, etc.). Most other languages do not even have abbreviations like “a.m.” and “p.m.” and the 12h notation is certainly hardly ever used on Continental Europe to write or display a time. Even in the U.S., the military and computer programmers have been using the 24h notation for a long time (Kuhn, 1999).

For this section, one is supposed to check if date formats are adapted according to target language rules.

For what concerns Italian language, 24h time notation is considered appropriate. The choice of time format is a matter of editorial style. However, when there is no need to tell the exact time, 19:00 can be replaced by 19, and 06:42 by 6:42, in a less formal way (Lesina, 1986, p. 142). Hours are separated from

50 minutes by colons or dot. While the use of a comma is not considered appropriate (Ibid.). The table below shows several ways to indicate time in Italian. Examples are taken from the Italian manual of style by Lesina (1986, p. 142).

Informal ways Formal ways 19 19:00 le sette (di sera) 19.00 6:42 06:42 6.42 06.42 le sei e quarantadue 6:42 e 3 secondi 06:42:03 6.42 e 3 secondi

Table 6. Italian time formats. Even for English, different styles are more or less appropriate depending on the context. When time is expressed in letters, it is customary not to use hyphens, and use “o’clock” only when referring to exact hours (e.g. four o’clock; a quarter to four). When time is expressed in figures, it is customary to use the 12h time notation with a.m. (ante meridiem = before noon) and p.m. (post meridiem = after noon). For a round hour, the decimal point and zeros do not need to be specified (e.g. 4 p.m.). The twenty-four- hour clock is also considered appropriate, and avoids using a.m. and p.m. (e.g. 16.00). In British English, it is customary to use the dot as separator, while in Northern America the dot is replaced by a colon. The afore-mentioned rules are taken from Ritter (2005, p. 187). Examples are summarised in the following table:

British/American English British English American English A quarter to four. 3.45 a.m. / p.m. 3:45 a.m. / p.m. Four o’clock. 4 a.m. / p.m. 4 a.m. / p.m. 3.45 / 15.45 3:45 / 15:45 4.00 / 16.00 4:00 / 16:00

Table 7. British and American time formats. The non-compliance score for this category is 3 (major error).

2.3 Number format: As seen for previous categories, numbers used in time expressions are treated differently in different languages. It is important for the target user that number format matches with target language rules, so that the information displayed to them look familiar. For example, the number format for one thousand two hundred and thirty four point five in England is 1,234.5, while in Germany

51 is 1.234,5 (W3C, 2014c). At the same time, “if you are paying 1.003 Omani Rials for an item on a website”, you should be informed whether you are paying one rial or approximately one thousand rials (Ibid.).

Resolution 10 in the Text of Resolutions adopted by the 22nd General Conference on weights and measures “declares that the symbol for the decimal marker shall be either the point on the line or the comma on the line, [and] reaffirms that ‘numbers may be divided in groups of three in order to facilitate reading; neither dots nor commas are ever inserted in the spaces between groups’, as stated in Resolution 7 of the 9th CGPM, 1948” (22nd General Conference on Weights and Measures, 2003).

In this category, it must be checked whether number formats are used in accordance with target language rules.

In Italian, the comma is used as decimal separator (e.g. 0,2 or 630,25). In writing large numbers, Italian uses a space to separate thousands, millions, etc. and tends to group numbers in groups of three (e.g. 26 312 600). Space is the most unambiguous method to separate thousands, as some cultures use either the comma or the dot, creating confusion with decimal numbers (Lesina, 1986, pp. 131-132).

Both British and American English use a dot to separate decimal numbers (e.g. 12.66). While a comma is used to separate thousands, millions, etc, (e.g. 1,000,000,000 = 1 million). However, in technical and foreign-language work it is suggested to use a thin space (e.g. 14 785 652) (Ritter, 2005, pp. 182, 185).

When symbols of currencies are used, they precede the figures both in English and Italian (Lesina, 1986, p. 144; Ritter, 2005, p. 186).

The non-compliance score for this category is 3 (major error).

2.4 Contacts section: This category specifies that contacts section in the localised version of a web site must be adapted to target users. For example, telephone numbers must be included specifying the area code, and no other number can be included between the area code and the telephone number (Andreu- Vall and Marcos, 2012, p. 257). 15

The non-compliance score for this category is 3 (major error). However, as the focus of this heuristic evaluation is the homepage, this category is not considered, unless contact information is given in the home page

2.5 Content update in every localised version: One of the main purposes of a web site is to carry out information about a service, a product, an institution or a company. However, information is not always

15 Information about the area code, 2-digit and 3-digit ISO code are available on the web site CountryCode.org. 52 updated in every localised version. It may happen that the content is more frequently updated in a language, while other linguistic versions may not be so up to date. Ideally the content of a web site should be updated in all linguistic versions at the same time (Andreu-Vall and Marcos, 2012, p. 257).

For this category, I check if the content is updated in all linguistic versions. In this specific case, the compliance of this guideline is done by reading and comparing the content of the Italian and English web site. The non-compliance score for this category is 2 (medium error).

2.6 Access to language selection from any page: This usability guideline specifies that users should be able to change language in every web page while surfing a web site, without starting the navigation again from the homepage

Sometimes when users click on the language selector in a web page, they are redirected to the homepage, the only page where they are allowed to actually switch language. This forces them to interrupt their reading, losing time and getting off-track.

If language choice is supported by a site, I recommend providing a link to the choice on every single page since users often go directly to pages from search services or bookmarks without passing through the home page. Some sites put up a language choice page before the user can reach the home page, but I recommend against this if it is possible to determine a default language that will be used by a very large proportion of the users (the Louvre Museum in Paris is a good example: fair enough to start in French) (Nielsen, 1996b).

For this guideline, I should check if access to language selection is available from any page. Since this analysis focuses on the homepage, this guideline does not apply to my evaluation.

The non-compliance score for this category is 3 (major error).

3.3.3 Group 3. Aspects related to visual elements This section covers all those visual aspects, defined by Makki and Leppert with the term “cultural markers” (2006). As previously discussed, cultural differences can represent a difficulty in websites design, as web sites not only need to be multilingual, but they also have to be multicultural (Hars, cited in Makki and Leppert, 2006, p. 8). In fact, these visual elements can have different cross-cultural meanings:

Studies show that different cultures react differently to certain visual stimuli and organisation techniques on websites. These stimuli can be simple pictures, icons representing a certain culture, colours, or even simply textual shapes (Makki and Leppert, 2006, p. 8).

Cultural markers must be taken into account for the design of multilingual and multicultural web sites. Users from target countries may “experience dissatisfaction when viewing sites which are not designed

53 specifically for their culture” (Ibid.), and this dissatisfaction may cause “a feeling of discomfort, lowering the site’s usability” (Ibid.).

This section takes into account the use of icons, colours, images and text direction, and generally applies to web sites localised for distant target cultures. Considering that Italian and Anglo-Saxon cultures are both Western, this category does not apply to my study. I do not expect to find any major errors concerning this group. However, only a reviewer from one or several Anglo-Saxon countries could confirm my assumption. As users are not involved, and the two cultures are relatively close, the following guidelines are not checked. Nonetheless, they are explained below to provide a general overview of the tool.

3.1 Icons: Icons “resemble the object they represent” (Singh, 2011, p. 161). In other terms, “an icon is a sign whose form suggests its meaning” (Ibid.). An icon can be illustrative or diagrammatic, as the “no- smoking” sign, figuring a red circle and line through a lit cigarette (Ibid.). When designing a web site with multicultural audience in mind, particular attention must be paid to the use of icons:

Customers in the United Kingdom found the trash can icon in the Apple Macintosh ‘Trash can’ very confusing, because it was a cylindrical bin, shaped exactly like mailboxes in Britain. In this case it was particularly difficult, because mail was mistakenly being sent to the trash can (Ibid.).

For this category, I should check if icons are adapted to different cultures. Sometimes icons are improp- erly used to display language selectors. Language selectors should avoid the use of icons, for examples flags. In fact, a country can have more than one ofﬁcial language (ex. Switzerland), and a language can be spoken in more than one country (ex. Spanish). Thus, the best practice is to present the language options in the spelling of the original language (Yunker, 2003, p. 89; Roturier, 2015, p. 58). Since the non-compliance score cannot be given only considering the use of language selectors icons, this guideline cannot be considered checked.

The non-compliance score for this category is 2 (medium error).

3.2 Colours symbolism: As stated by Singh and Pereira, “different colours mean different things to different people in different cultures” (2012, p. 44). For example, in Western cultures white is associated with marriage, as it reminds of a bride’s dress, while in India the same colour is worn by widows. Instead, green is the colour of fertility in Egypt, while it symbolises safety in the US, criminality in France, and danger in Malaysia (Ibid.). Even colours combination have their own meaning: in China black text on red background represents happiness, and it is widely used on wedding invitations (Ibid.). Thus, the use of specific colours and colours combinations on the web needs to comply with target users’ needs and

54 expectations (Ibid.). Andreu-Vall and Marcos (2012, p. 258) propose a chart representing colours meaning in different cultures in a simple visual format. The chart is available on the web site “Information is beautiful” (https://informationisbeautiful.net/).16

The non-compliance score for this category is 2 (medium error).

3.3 Images: When designing a web site, particular attention must be paid even in the choice of images and symbols, as some pictures and symbols may be considered appropriate for a culture, but offensive for another. For example, according to Singh and Pereira (2012, pp. 42-43), pictures of women and ani- mals are disliked in Arabic cultures. Whereas, nature symbolism (e.g. mountains, rivers, birds, etc.) is particularly appreciated by Eastern cultures like Japan and China. Further, religious symbols, animal figures, hand gestures, and forbidden food are to use cautiously (Singh and Pereira, 2012, pp. 42-43).

The non-compliance score for this category is 2 (medium error).

3.4 Text direction: As mentioned in the previous chapter, ideographic languages, like Chinese and Jap- anese, are traditionally written vertically in columns going from top to bottom. Whereas, Semitic languages, such as Arabic and Hebrew, are written from right to left. Web pages should be then designed to serve different audiences.

Spatial orientation is defined by Singh and Pereira (2012, p. 33) as the way web content is structured. In Arabic cultures, a left-justified web page might have a negative impact on users’ visual perception, affecting the web site usability. Whereas, in oriental cultures central composition is a “visual norm” when depicting graphics (Mooij, cited in Singh and Pereira, 2012, p. 33). For such reasons, web content structure should be presented in a layout visually appealing for users. Richard Ishida (2014) gives us an insight on how some tools17 that can be used when dealing with spatial orientation and justification of non-Latin text. Justification becomes in fact more complicated when it comes to other scripts.18

16 The web site has been last accessed on 28 March 2018. 17 In Arabic, text is right-aligned going from right to left across the page. Received wisdom says that to do justification in Arabic it is not necessary to stretch the spaces in between the words, as it is required in English. In Arabic, one should stretch the words themselves. However, not all of the words are stretched in the same amount: “One way you may hear that this can be done is by using a special baseline extension character in Unicode, U+0640 ARABIC TATWEEL” (Ishida, 2014). “Baseline extension tatweel is a tool that helps to produce text in a nice typographic style whatever your language or your script is” (MultilingualWeb, 2014). However, one of the biggest problems with this approach is that rules for the placement of baseline extension are complex, as extensions can only appear between certain characters, and are forbidden around others. Furthermore, different font styles have different rules, producing different visuals. Lack of consensus about both rules and justification make this task difficult, so that there is still work to be done on this matter. 18 The Unicode Consortium’s glossary defines a script as “a collection of symbols used to represent text in one or more writing systems” (W3C, 2003). Ishida (W3C Internationalization Activity Lead) presented on “New Internationalization Develop- ments” at the World Wide Web Consortium in Madrid, 2014 (Ishida, 2014). His presentation focuses on a specific aspect of W3C work: layout requirements for digital publishing, like justification in CSS for non-Latin scripts. 55

As I am dealing with Italian and English text, I do not expect to find any errors. However, more clarifications on this category have been provided in the notes because it is an important usability issue for which requirements are still being implemented.

When speaking of text direction in culturally customised web sites, particular attention should also be paid also on navigation modes. Navigation bars should follow the direction of the text. Therefore, a web site specifically designed for Arab users should include navigation bars moving from right to left. For example, the Swedish company Ikea “has taken note of this cultural preference” in its Kuwait web site (Singh and Pereira, 2012, p. 34), as seen in Figure 5.

Figure 5. Ikea Kuwait web site: ar.ikea.com/kw/ar/ (last accessed on 20 March 2018). The non-compliance score for this category is 3 (major error).

3.3.4 Group 4. Aspects affecting search in the web site A “search” on the web, more accurately defined with the term “query”, is defined as “a question or request for information” on the web (Thurow and Musica, 2009, p. 19). In large web sites, users may need to filter the information they need without browsing from page to page. Search on the web site should work in every language into which the web site has been localised:

Nielsen (1996) suggests having the computer to perform multi-lingual searches automatically by understanding the meaning of the search terms in several languages. Search options are checked directly on the web site.

4.1 Search in all linguistic versions: The searcher should be able to search for information in all the web pages (Andreu-Vall and Marcos, 2012, p. 259) in all linguistic versions. In our case, the search options should not be limited just to Italian web pages.

The search has been tested by entering the following search queries— 3 English words (“tickets”, “opening hours” and “special offers”) and 3 Italian words (“biglietti”, “orario apertura” e “offerte speciali”)19 without selecting the language first. These words have been deliberately chosen by me, as I think these words can probably be among the most searched words by users looking for information on a museum web site. I expected the system to automatically recognise the language and show results in that language. I hereby provide a practical example to show how I carried out the test.

Item 13a: IT  home page tested: www.museoscienza.org/

Item 13b: EN  home page tested: www.museoscienza.org/english/

Item 13a:

When I enter search queries in Italian, I expect the search to show me results in Italian. When I enter search queries in English, I expect the search to show me results in English, even if the language of the home page is Italian, i.e. regardless of language selection. If results match my expectations, they can be considered suitable. Otherwise, they are considered unsuitable.

Figure 6 shows the results obtained when entering the Italian word “biglietti” in the searcher of the IT home page. Results are in Italian. The test is repeated with the words “orario apertura” and “offerte speciali”. Results are in Italian as well.

Figure 7 shows the results obtained when entering the English word “tickets” in the searcher of the IT home page. Results are in English. The test is repeated with the words “opening hours” and “special offers”. Results are in English as well.

As in both cases results comply with the guideline, they can be considered suitable.

19 In “orario apertura” the preposition “di” (orario di apertura) is omitted. Prepositions are often omitted when searching information in a web site. 57

Figure 6. Results from search test in the IT sample. Biglietti. Figure 7. Results from search test in the IT sample. Tickets. Item 13b:

When I enter search queries in English, I expect the search to show me results in English. When I enter search queries in Italian, I expect the search to show me results in Italian, even if the language of the home page is English, i.e. regardless of language selection. If results match my expectations, they can be considered suitable. Otherwise, they are considered unsuitable.

Figure 8 shows the results obtained when entering the English word “tickets” in the searcher of the EN home page. Results are in English. The test is repeated with the words “opening hours” and “special offers”. Results are in English as well.

Figure 9 shows the results obtained when entering the Italian word “biglietti” in the searcher of the EN home page. Results are in Italian. The test is repeated with the words “orario apertura” and “offerte speciali”. Results are in Italian as well.

As in both cases results comply with the guideline, they can be considered suitable.

Figure 8. Results from search test in the EN sample. Tickets. Figure 9. Results from search test in the EN sample. Biglietti. The non-compliance score for this category is 2 (medium error).

4.2 Advanced settings for search in a language: The user should be able to choose if he wants to perform a search in all linguistic versions or in a particular language. Moreover, he should be able to choose in which language he wants to search the information he needs. To check the guideline, I verified if there are advanced settings for search in a language. When such settings are not present in the web site, I selected English language, and then I wrote the above-mentioned words in the searcher, expecting the computer to display the information in English only.

If all of the information has been replicated in every language, then there is no need to search more than one language. In this case, the search interface should know of the user’s preferred language and only display hits in that language. Unfortunately, it is often not possible to translate all documents, so many sites require searches of several languages if the user needs complete coverage of the available information (Nielsen, 1996).

The non-compliance score for this category is 3 (major error).

4.3 Language of informative texts in results page: When users perform the task of searching content using the search button, they expect the computer to display results in the language they have chosen. Further, if the search results lead to other informative texts or documents, these texts should be available in the language selected for the search (Andreu-Vall and Marcos, 2012, p. 259).

The object of the analysis is the language of informative documents given as search results. The afore- mentioned search queries have been entered in the searcher. Search results have been analysed, and no informative documents has been found. A further query has been added (map; mappa) to increase the chance of coming across informative documents. The non-compliance score for this category is 2 (medium error).

3.3.5 Group 5. Aspects affecting the web sites ranking in search engines As suggested by Iler (2006, p. 1), “studies have shown visibility on search engines is one of the best ways to promote online content”. However, most companies fail to hit potential customers, as they do not adequately promote their web sites through search engines:

The problem is even more common for multilingual sites. Many well-localised web sites are not visited as frequently by international customers as they potentially could be because the sites do not appear in searches for that country/region or language (Ibid.).

In fact, multilingual web sites need to face some obstacles, such as language and regional filters, text display issues, translators not using the correct terminology etc. (Ibid.).

Iler (2006) explains which factors are affecting international searches, and proposes some good practises to follow to achieve international visibility. Based on Iler (2006), Andreu-Vall and Marcos (2012, p. 259)

59 list the main aspects affecting a web site ranking in search engines. To achieve global visibility, a web site “must appear in the results for searches conducted in languages other than English, and for searches that are restricted based on country or region” (Iler, 2006, p. 4). It should be mentioned that a web page ranking in search engines is different according to the language, as each linguistic version privilege different aspects of SEO — search engine optimisation (Andreu-Vall and Marcos, 2012, p. 259).

5.1 Translation of the content of element (head): The <title> element defines a title in the browser toolbar (w3schools, 1999f). “On an actual web page in a web browser, the value of the <title> element displays at the top of a screen” (Thurow and Musica, 2009, p. 23). The <title> element also provides a title for the page, and displays a title for the page in search engine results pages, as clickable headlines for a given result (w3schools, 1999f). It plays therefore a significant role in how people discover and locate web content (Thurow and Musica, 2009, pp. 2–4), as the <title> element content is “the call to action” in each search listing (Ibid., p. 23). As recommended by Iler (2006, p. 9), “page title tags carry a lot of weight in search engine algorithms”, so web masters should make sure the title includes the keywords they are targeting. Further, title tags must be translated so that they can be identified by both users and web crawlers (spiders). Web crawlers are an important component of web search engines, where they are used to collect the corpus of web pages indexed by the search engine. […] In order to crawl a substantial fraction of the “surface web” in a reasonable amount of time [web indexing or web spidering], web crawlers must download thousands of pages per second, and are typically distributed over tens or hundreds of computers (Najork, 2009). If the content of the element <title> is correctly translated, web crawlers of search engines are able to index titles in the language in which the web page has been written. It is worth mentioning that if the value of the <title> element contains multilingual text, it is not possible to mark up part of the text for different languages. The same goes for multiple languages in <title> attribute values. There is no good solution for multiple languages in the <title> element/attribute values at the moment (W3C, 2014a). For the analysis of this attribute, I had to inspect the HTML document of each home page, find the <title> element, and check if it respects the guideline. The non-compliance score for this category is 3 (major error). 5.2 Translation of the content of description and keywords attributes: Description and keywords are attributes of the <meta> element, which provides metadata about the HTML document. Metadata is data (information) about data and is not displayed on the page, but it is machine parsable (w3schools, 1999b). The <meta> element can have a name attribute, which sets document metadata (W3C, 2012). Document metadata is “expressed in terms of name-value pairs, the name attribute on the <meta> element giving the name, and the content attribute on the same element giving the value” (Ibid.). 60 5.2.1 Translation of the content of description attribute: The description attribute gives the description of a web page. The description of the web page is not visible on the page itself. Nonetheless, web crawlers of search engines can retrieve and index these tags from source code to display a summary of web page content in the results list. The description attribute of the <meta> element is at the top of the page in the <head> section of the code (Andreu-Vall and Marcos, 2012, p. 259). The description attribute is specified in the name tag. The value of the description attribute is specified in the value of the content tag, which should be translated. For the analysis of this attribute, I had to inspect the HTML document of each home page, find the description attribute, and check if it respects the guideline. 5.2.2 Translation of the content of keywords attribute: If a web page contains keywords, web search engines can generate a search listing for query words and a snippet (description) of the page content (Thurow and Musica, 2009, pp. 24-25). The keywords that people type into search engines are defined “scent of information”. In other words, “keywords are a user-generated scent of information” (Ibid., p. 19). [The keyword description attribute] offers you the opportunity to influence the Search Engine Results Page (SERP) description. If the phrase the searcher uses in a search appears in your meta keyword description tag, that description very likely will be used on the SERP. If your site ranks lower than your competitors’ sites, but your description is much more attractive and “clickable”, you might beat your competitors and get the click (Iler, 2006, p. 11). Search engine marketers and usability professionals can get a higher ROI (Return of Investment) using this type of information. This knowledge can also be used to help building more user-friendly web sites. The keywords attribute to the <meta> element can have a name attribute, which sets document metadata (W3C, 2012). Document metadata is “expressed in terms of name-value pairs, the name attribute on the <meta> element giving the name, and the content attribute on the same element giving the value (Ibid.). “The value must be a set of comma-separated tokens, each of which is a keyword relevant to the page” (Ibid.) The value of the keywords attribute should be translated. An example of description and keywords attributes is shown in the following figure.20 20 The figure is a screenshot from the web site https://www.w3schools.com/tags/tag_meta.asp (last accessed on 13 April 2018). 61 Figure 10. Description and keywords attributes to the <meta> element. For the analysis of this attribute, I had to inspect the HTML document of each home page, find the keywords attribute, and check if it respects the guideline. If the values of description and keywords attributes are correctly translated, web crawlers of search engines can display a summary of the web page content in the appropriate language. The non-compliance score for this category is 3 (major error). 5.3 Translation of the content of alt and title attributes in images: Images cannot be indexed by web crawlers of search engines, since they are not readable textual content. Nonetheless, the title attribute gives a title to the image transferring visual meaning into words, which are recognisable by web crawlers. On the other hand, the alt attribute (alt text) provides “a clear text alternative of the image for screen reader users” (Pennsylvania State University, 2014a): Alt attributes of images “are invisible descriptions of images which are read aloud to blind users on a screen reader. Adding ALT text allows authors to include images, but still provide the content in an alternative text based format”(Pennsylvania State University, 2014b). The alt and title attributes should always be included in the source code, and translated into the target language. For this category, I checked if the alt and title images attributes are included and translated. The non-compliance score for this category is 3 (major error). 5.3.1 Translation of the content of alt attributes in images: The alt attribute contains the text of an image and is important not only for web indexing, but also for web accessibility reasons: The alt attribute provides a text alternative to images for visually impaired users. Since screen readers cannot read graphical text, alternate text must be specified in the value of the alt attribute to be read by the screen reader (Nielsen and Loranger, 2006, p. 247). A translation of the alt attribute value should be provided in localised pages. When an image is decorative and people do not need to know about it, the alt attribute value can be empty (alt=""), but the attribute still needs to be indicated. This type of alt attribute is 62 called null alt (W3C - Web Accessibility Initiative, 2015). The screen reader does not read the null attributes, but it recognises them. The W3C Web Accessibility Initiative (WAI) represents the most important effort for setting guidelines to improve accessibility on the Web. The WAI’s work focuses on the production of Web Content Ac- cessibility Guidelines (WCAG), and has two main goals. The first one is “producing contents that must be perceivable and operable”, such as “using a simple and clear language, as well as defining navigation and orientation mechanisms for supporting content access and browsing” (Matera, Rizzo and Carughi, 2006, pp. 6–7). The second goal is “ensuring access alternatives: this means that pages must be designed and coded so as they can be accessed independently from the adopted browsing technologies and devices, and from the usage environment” (Ibid.). For the analysis of this attribute, I had to inspect the HTML document of each home page, and find the <img> element. For each <img> element, I checked if the alt attribute respected the guideline. To ensure that the analysis was done as accurately as possible, I inspected the HTML code on the browser using the inspect element tool to find images and see them on screen. 5.3.2 Translation of the content of title attributes in images: Title text or title attribute, instead, contains the text of an image, which is seen by the user as tooltip text when hovering the mouse cursor on the image (w3schools, 1999e). For the analysis of this attribute, I had to inspect the HTML document of each home page, and find the <img> element. For each <img> element, I checked if the title attribute respected the guideline. To ensure that the analysis was done as accurately as possible, I inspected the HTML code on the browser using the inspect element tool to find images and see them on screen. The following figure21 provides an example of image element (<img>) with src, alt and title attributes, where src refers to the location of external resource (w3schools, 1999d). The figure below wants to show both the <img> element with its attributes in the HTML document, and the visual result on the web page: the title Juventus is shown when hovering on the logo image, so that a user who does not know the logo of the football team can read this information. The alt-text allows the screen reader, as well, to recognise this text and read it in a computerised voice. 21 The figure is a screenshot from the web site http://www.juventus.com/it/ (last accessed on 12 April 2018). 63 Figure 11. Alt and title attributes to the <img> element. 5.4 Translation of the content of title attributes in links: The title attribute of a link is used to describe the purpose of a link by providing descriptive text as the content of the <a> element (W3C, 2016b). As for all the other categories of this group, the value of a link title attribute should be translated. In fact, it is visible to users as they move the mouse cursor on the link, and it is parsable by web crawlers (Andreu-Vall and Marcos, 2012, p. 259). The figure22 below shows how the title attribute is used in the HTML document and the visual result on the web page. When hovering on the Facebook Messenger logo, the tooltip text appears. 22 The figure is a screenshot from the web site http://www.repubblica.it/ (last accessed on 12 April 2018). 64 Figure 12. Title attribute to the <a> element. Iler (2006, p. 9) points out that “having your keywords in the link text is one of most important elements of search algorithm”. In fact, “the words used as link text (sometimes referred to as ‘anchor text’) play a critical role in search engine rankings” (Iler, 2006, p. 10). Further, “if the supplementary information provided through the title attribute is something the user should know before following the link, such as a warning, then it should be provided in the link text rather than in the title attribute” (W3C, 2016c). For the analysis of this attribute, I had to inspect the HTML document of each home page, and find the <a> element. For each <a> element, I checked if the title attribute respected the guideline. To ensure that the analysis was done as accurately as possible, I inspected the HTML code on the browser using the inspect element tool to find hyperlinks. The non-compliance score for this category is 2 (medium error). 3.4 Limitations of the Heuristic Tool As already mentioned, Andreu-Vall and Marcos provide a practical checklist in the form of a Google document, which is available at http://goo.gl/LJTR5 (Andreu-Vall and Marcos, 2012a).23 This checklist must be used as follows: after checking each point in Column A (checkpoints are listed as questions), the evaluator is asked to give a score for the error frequency in Column B (0 = never, 1 = sometimes, 2 = many times or always. After evaluating how many times an error occurs, in Column C the evaluator is asked to indicate the impact score of each guideline when it is not respected. I previously 23 The web site has been last accessed on 28 March 2018. 65 referred to the impact score as non-compliance score (1 = minor error, 2 = medium error, 3 = major error). While the error frequency is determined by the evaluator, the impact score for non-compliance is determined by the authors, and has been indicated while explaining each guideline. In Column D, the evaluator is asked to calculate a severity score, resulting from the multiplication of the error frequency (0, 1, 2) score by the impact score (1, 2, 3). Severity score indicates the priority for error correction. Its value can vary from 0 to 6. Table 8 shows different combinations of error frequency and impact, and the resulting severity score: Frequency Impact Severity 0 1 0 = No error 2 0 = No error 3 0 = No error 1 1 1 = Very minor error 2 2 = Minor error 3 3 = Medium error 2 1 2 = Minor error 2 4 = Priority error 3 6 = High-priority error Table 8. Severity errors. Translated from Andreu-Vall and Marcos (2012, p. 260). The main limitation of the tool proposed by Andreu-Vall and Marcos is that some of the features that need to be inspected can only occur once, while others can occur many times. For example, content language declaration (Guideline 1.3) and Unicode codification (Guideline 1.1) can appear only once, while alt attributes in images (Guideline 5.3.1) and date formats (Guideline 2.1) can appear more than once within the same home page. It is not clear from the authors’ paper how to treat them in order to obtain results that could be compared. Further, the authors explain how to compute both the impact and the frequency of the errors, but they only include the distinction of frequency in three categories: 0 = never, 1 = sometimes, 2 = many times or always (Andreu-Vall and Marcos, 2012, p. 259). This categorisation, however, is fuzzy and strictly subjective. First, some issues have Boolean nature, as afore-mentioned they can be present or not, only once; while other issues can appear multiple times. Second, these categories of frequency are not meas- urable in quantitative terms. The frequency “never” and “always” could be measured, but “sometimes” 66 and “many times” remain strictly subjective. Therefore, it is not clear how the impact can vary according to such categorisation. Even if we measure the actual number of errors, no detail is provided about how to deal with different amounts of errors per page, i.e. with non-Boolean data type. For example, when accessing the declaration of content language in the head of the HTML document (Guideline 1.3), there is only one item to check, the content of the lang attribute in the <html> element: <html lang="it">. This means that error frequency can vary only from 0 to 1. On the other hand, when checking the alt attribute, error frequency can vary according to the number of images per page. Yet, in the checklist proposed by Andreu-Vall and Marcos, the non-compliance with these guidelines has in both cases an impact score of 3. Due to the limitations of the error frequency categorisation proposed by the authors, I preferred to rate usability problems on the basis of the actual error frequency (how many errors did I encounter?), and impact (how much trouble does the problem cause?), as explained by Nielsen and Loranger (2006, p. 124). However, when computing the severity score multiplying impact by error frequency, the scale of severity for usability problems was not actually useful, as the severity score did not show the real impact of errors. It was difficult to rank severity scores in nominal categories to define the potential damage to the sample for two reasons. First, I could not adjust the impact for multiple errors, as I did with the error frequency. I do not have the knowledge nor the expertise to decide the impact of an error to implement the tool. Second, the actual damage on the home page and the priority intervention for error fixing can only be determined by a usability test. As discussed by the authors (2012, p. 260), heuristic evaluation can only detect the most serious and frequent problems, but they suggest to complete the study with a usability test involving real users. On this purpose, Nielsen (1995c) explains that the priority intervention for error fixing can hardly be determined by heuristic evaluation, especially if it is carried out by a single evaluator. The third column of the table proposed by the authors (2012, p. 260) provides a severity rating for usability problems. However, as afore-mentioned, due to the diverse nature of the data analysed, it is difficult to rank severity scores. High-severity problems indicate high-priority intervention for error fixing. However, any attempt to prioritise usability problems can only be done by evaluating user experience, which is not the goal of heuristic evaluation. It is difficult to get good severity estimates from the evaluators during a heuristic evaluation session when they are more focused on finding new usability problems. Also, each evaluator will only find a small number of the usability problems, so a set of severity ratings of only the problems found by that evaluator will be incomplete. […] Severity ratings from a single evaluator are too unreliable to be trusted (Nielsen, 1995c). 67 Due to the lack of a framework to prioritise usability problems and to the limitations of heuristic tool, I only measured error frequency, as my second research question aims to compare the number of errors across the two linguistic versions. Given the heterogeneity (Boolean vs. non-Boolean nature) of the data analysed, to answer the first research question, I applied a binary logic: when the error frequency is 0 the home page is suitable (score 0); when the error frequency is equal to or greater than 1, the home page is unsuitable (score 1). 3.5 Scientific Research Terminology applied to a Usability Function Test The previous section offers a presentation of the tool used for the usability function test conducted in this Master thesis. I deemed important to provide sufficient theoretical and practical knowledge about the method and tool chosen, before actually explaining how I conducted the test. Before concluding the chapter, in this section I consider it appropriate to summarise the research dimensions on which my research is based. Saldanha and O’Brien (2013, pp. 12-14) provide a definition of the most frequently used terms in scientific research. Even if other members of the scientific community may not unanimously agree with these definitions, they represent the basis of scientific research. Before conducting a research, one should have clear in mind what is the difference between a framework and a model, a method and a tool, and how these concepts are related to each other. Definitions of common research terms are given below (Table 9). 68 Term Definition Model “A model is a representation of the ‘reality’ of your research topic or domain” (Saldanha and O’Brien, 2013, p. 12). Framework “A framework is a set of ideas and approaches that can be used to view and gather knowledge about a particular domain” (Saldanha and O’Brien, 2013, p. 12). Theory “A theory organises sets of concepts to define and explain some phenomenon” (Saldanha and O’Brien, 2013, p. 13). Methodology “A methodology is a general approach to studying a phenomenon” (Saldanha and O’Brien, 2013, p. 13). Method “A method is a specific research technique” (Saldanha and O’Brien, 2013, p. 13). Tool A tool measures the concept the researcher wishes to measure (Saldanha and O’Brien, 2013, p. 12). Table 9. Research Terminology (Saldanha and O’Brien, 2013, p. 12). My multilingual web sites evaluation is a function usability test. By performing this test using a specific tool and method, I test the validity of my hypotheses. Hypothesis testing to reject or confirm the hypothesis is the goal of empirical research. Thus, since my work can be regarded as empirical research and the heuristic evaluation method can be ascribed to the frame provided by Saldanha and O’Brien (2012), I thought it could be useful to use their frame to think about the other categories they refer to. This is also an attempt to visually summarise the concepts I am subscribing to. Starting from the bottom, the tool I chose to use is the heuristic guideline (checklist) for multilingual web sites evaluation proposed by Andreu-Vall and Marcos (2012). The method chosen is the heuristic evaluation method, as explained in Nielsen (1993; 1995b, 1995a; 1995; 2009). The methodology, which will be technically described in the next chapter, is empirical and relies on quantitative data analysis. Considering that methodology is a general approach to studying a phenomenon, the general approaches to which heuristic evaluation can be ascribed to are “discount usability engineering method” and “usability inspection”. The first is described as a “method based on the use of four different techniques” (Nielsen, 1993, p. 17). However, in Nielsen 1993, when proposing new 69 “more careful methodologies”, the terms methodologies and methods are used as synonyms (Ibid.). Thus, I think that discount usability engineering method can be seen as a methodology consisting of four different methods. Further, heuristic evaluation is also one of the many methods based on user interface inspection performed by evaluators. This set of methods is referred to as “usability inspection” (Nielsen, 1995). For what concerns the theory, I find it hard to define a theory, as I am not trying to explain some phenomenon. My multilingual web sites heuristic evaluation is done referring to heuristic principles, specific standards and guidelines. If I have to find a theory that organises the sets of concepts I am referring to, I would certainly cite usability theory, which encompasses a group of ideas about user’s interaction with the web page (TCBOK, 2014). For what concerns the other research categories, I do not recur to some particular framework nor model. However, I would like to mention that according to the definition of usability, it is a quality attribute of the UI [User Interface], as it “assesses how easy user interfaces are to use” (Nielsen, 2012). Thus, to contextualise my work I would refer to the domain of web user interface. The research concepts afore-mentioned are summarised in the following template (Table 10). Model - Framework - Theory Usability Theory Methodology Discount Usability Engineering Method + Usability Systematic Inspection Empirical/Experimental methodology Quantitative analysis Methods Heuristic Evaluation Tools Heuristic guideline (check-list) for evaluating monolingual (with or without globalisation) and multilingual (with or without localisation) web sites Table 10. Research terminology and concepts applied to my usability function test. Summary The aim of this chapter is to introduce the research questions, the hypotheses tested, as well as the method and the tool chosen for the function usability test carried out in this Master thesis. Heuristic evaluation method aims to find usability problems in a user interface, according to usability principles (heuristics), as well as validated standards and guidelines. It can be performed by one or more evaluators, inspecting the user interface alone. Users’ participation is not requested. Different evaluations 70 are finally recorded as written reports to produce a list of known usability problems, without further attention to fixing. The tool chosen to perform the heuristic evaluation is a heuristic guide created by Andreu-Vall and Marcos (2012) in the form of a checklist consisting of 24 guidelines, classified in five categories. Each category concerns different aspects: aspects related to source code (1), localisation (2), visual elements (3), aspects affecting search in the web site (4), and the web sites ranking in search engines (5). For each category, I provide information about standards and recommendations, and details on how I checked the compliance with each guideline. Finally, I discuss the limitations related to the heuristic tool. The chapter ends with a frame proposed by Saldanha and O’Brien (2013) to summarise the research concepts applied to my usability function test. 71 Chapter 4: Application of Research Method and Tool This chapter is dedicated to the analysis of the data, the presentation of the results, and the description of the sample characteristics emerging from the analysis. In the chapter, I present the data collected (4.1), and describe the process of sample selection (4.1.1). I explain the experimental set-up put in place to verify the hypotheses (4.2). I describe the data analysis, but I also present and compare the results concerning the guidelines compliance: Group 1 (4.2.1), Group 2 (4.2.2), Group 4 (4.2.3), Group 5 (4.2.4). At the end of the chapter, I provide a summary of the results in terms of suitable and unsuitable home pages (4.3) for every group of guidelines (Group 1 Section 4.3.1, Group 2 Section 4.3.2, Group 4 Section 4.3.3, and Group 5 Section 4.3.4). 4.1 Data Collection After defining the hypotheses and variables (Chapter 3, Section 3.1), I here explain the process of data collection. My aim was to collect a sample of multilingual home pages, so that I could check the guidelines compliance as explained in the previous chapter (Section 3.3). On the open data portal of Lombardy region, I found a list of Lombard museums, officially recognised by the regional authorities (Musei riconosciuti da Regione Lombardia, 2018). The list reported 253 museums and it was downloadable in a XLSX file. After filtering the displayed items by province on Ms Excel, I decided to focus on Milanese museums (Musei riconosciuti - Comuni Città Metropolitana Mi- lano, 2018) to get a general view on the data. I noticed that not every museum reported a functional web address (url) to its ofﬁcial web site. Furthermore, some museums had the same url indicated, while others did not have any url mentioned. For those without any url indicated, I found the right one checking on the search engine Google. In many cases, the official source of information for these museums was a web page inside the web site of the municipality. For those with the url indicated, I started to check if the url indicated was correct and functioning. As aforementioned, some urls were listed several times. In fact, different museums can be part of one structure, that is why they had only one url indicated. In Italy, it is not unusual to find museums which are part of a bigger complex. In fact, in 2014 the Italian Ministry of Cultural Heritage and Activities and Tourism (Ministero dei Beni e delle Attività Culturali e del Tur- ismo—MiBACT) implemented a number of reforms aimed at promoting Italian cultural heritage through the reorganisation of Italian State Museums. Regional “poli museali” (museum clusters) have then been created to favour exchange and collaboration (Il Sole 24 Ore Digital Edition, 2015). 72 The Sforza Castle in <a href="/tags/Milan/" rel="tag">Milan</a> hosts seven museums, a historical archive, a photo archive and two libraries (Figure 13). This means that one web site refers to several cultural attractions. Table 11 shows the museums which refer to the web site www.milanocastello.it,24 according to the open data portal (Musei riconosciuti - Comuni Città Metropolitana Milano, 2018). Hence, it is not surprising that the list of Milan museums reported 46 museums, but actually counted 36 urls. Figure 13. The Sforza castle and its museums: www.milanocastello.it (last accessed on 6 April 2018). URL POLO MUSEALE MUSEUM www.milanocastello.it CIVICHE RACCOLTE GRAFICHE E Civico Gabinetto dei Disegni - FOTOGRAFICHE Piazza Castello Civica Raccolta delle Stampe A. Bertarelli - Piazza Castello Civico Archivio Fotografico MUSEI ARCHEOLOGICI Museo Egizio - Piazza Castello Museo di Preistoria e Protostoria - Piazza Castello RACCOLTE ARTISTICHE DEL CA- Raccolte d'Arte Antica STELLO Raccolte d'Arte Applicata (o Museo delle Arti Decorative) Museo degli Strumenti Musicali Gabinetto Numismatico e Meda- gliere Table 11. Museums referring to the web site of the Sforza castle (www.milanocastello.it). The process of finding missing web sites and matching multiple web sites to each museum was hindering the process of data collection. Further, even if the list had been last modified on June 2017, some of the 24 The web site was last accessed on 30 April 2018. 73 urls indicated were outdated. It is the case for example of Pinacoteca Ambrosiana, the Ambrosian Art Gallery. The list reported the web site of Biblioteca Ambrosiana (www.ambrosiana.it),25 the historic Am- brosian library, as shown in Figure 14. Since the Biblioteca Ambrosiana is also housing the Pinacoteca Ambrosiana, they can both be found on a new web site (www.leonardo-ambrosiana.it),26 as shown in Figure 15. Figure 14. The old url of the web site of Pinacoteca Ambrosiana as shown in the XLSX document downloaded from Musei riconosciuti - Comuni Città Metropolitana Milano, 2018. Figure 15. The new and the old web site of the Pinacoteca Ambrosiana. 25 The web site was last accessed on 30 April 2018. 26 The web site was last accessed on 9 May 2018. 74 Nonetheless, after the last update in January 2018, the web site indications are no longer available on the open data portal of Lombardy region (Figure 16). The column that is supposed to show the url of the web sites is empty (3rd column SITO_WEB). All the considerations above are based on the XLSX file downloaded27 from the above mentioned sources before January 2018. Figure 16. Museums list from the open data portal of Lombardy region accessed on 6 April 2018. For such reasons, I had to select my web sites sample from another database, slightly different from the official one. The tourism info-point of Varese suggested me to visit the web site of “Abbonamento Musei Lombardia Milano” (lombardia.abbonamentomusei.it/).28 “Abbonamento Musei”, born in 1995 in the city of Turin and developed as a subscription to Turin and Piedmont museums, represents an attempt to regroup in one ticket the regional cultural offer. Since 1998, “Abbonamento Musei” is a project developed and supported by the local authorities of the Piedmont region and the city of Turin allowing visitors to access the extensive cultural heritage and museums of Piedmont through a museum pass. In 2014, Lombardy joined the project with “Abbonamento Musei Lombardia Milano”. The museum pass for Lombardy and Milan offers a single ticket to get to know the diverse cultural offer of the region (Lombardia/Abbonamento Musei, 2018). The web site of the project reports updated information such as web sites, opening hours and tickets prices of the major museums in the region. According to the employee who suggested me the web site of “Abbonamento Musei Lombardia Milano” at the Varese info-point, this web site contains updated 27 The list “Musei riconosciuti - Comuni Città Metropolitana Milano” has been downloaded in a XLSX file from the following web address: www.dati.lombardia.it/Cultura/ Musei-riconosciuti-Comuni-Citt-Metropolitana-Milan/ seen-dpws (last accessed on 10 December 2017). 28 The web site was last accessed on 9 May 2018. 75 information about Lombardy museums, and it is therefore proposed to tourists at the local info-points of the region. On this web site, I found a list of 129 museums sorted by province. Before describing the process of data collection, I should explain that Lombardy comprises the provinces of Bergamo (BG), Brescia (BS), <a href="/tags/Como/" rel="tag">Como</a> (CO), Cremona (CR), Lecco (LC), Lodi (LO), Mantua (MN), Milan (MI), Monza and Brianza (MB), <a href="/tags/Pavia/" rel="tag">Pavia</a> (PV), Sondrio (SO) and Varese (VA), the latter being my hometown. This figure (129 museums) is almost the half if compared to the 253 museums officially recognised. It is important to point out, however, the impossibility to conduct a function test on the global population of web sites. “Abbonamento Musei” already removes the smaller museums, whose web sites are most probably unlocalised,29 and therefore not useful for the purposes of this study. The results probably will not be generalisable to the global population of web sites. The lack of generalisability, however, does not necessarily invalidate the study (Saldanha and O’Brien, 2013, p. 36) carried out in this Master thesis. Using the database “Abbonamento Musei”, I expected to find a considerable number of multilingual web sites. My aim was to check every museum’s web site, gather these web sites together and select an accurate sample of multilingual home pages to measure in the usability function test. In conclusion, the web sites indicated in “Abbonamento Musei” seemed updated to me. For example, the new web site of the Pinacoteca Ambrosiana has been picked from here (Figure 17. cf. Figure 15). “Abbonamento Musei” has therefore been chosen for data sampling. 29 I did not compare the web sites reported in “Abbonamento Musei Milano Lombardia” with those reported in Musei riconosciuti da Regione Lombardia. However, from a previous study for another course at the University of Geneva, I obtained the following results. According to Musei riconosciuti da Regione Lombardia, there are 46 museums in Milan, but only 36 web sites, as in some cases, more museums are part of the same group. Only 18 web sites out of 36 have the home page localised at least in one language. On the other hand, according to “Abbonamento Musei Milano Lombardia”, there are 36 museums in Milan, but 32 web sites, and only 21 out of 36 have the home page localised. This means that Musei riconosciuti da regione Lombardia reports more museums (46), but has less localised home pages (18). While, “Abbonamento Musei” reports less museums (36), but has more localised home pages (21). By generalising the results and extending them to the whole region, I assumed that “Abbonamento Musei” reports less museums (129), but has more localised home pages (51) than “Musei riconosciuti”. From these considerations, I came to the assumption that “Abbonamento Musei” removes those web sites that are most probably unlocalised. 76 Figure 17. Abbonamento Musei Lombardia. The Pinacoteca Ambrosiana. 4.1.1 Sample Selection: Purposive Sampling Purposive sampling is a common technique used in research studies to select a data sample. As stated by Saldanha and O’Brien (2013, p. 180), purposive sampling involves selecting a sample “on the basis of principled criteria so as to cover the key aspects of the research question”. In purposive sampling data are selected because they share certain pre-defined parameters. On this purpose, web sites were selected on the basis of the following criteria: 1. web sites needed to be up to date, 2. web sites needed to have the home page localised at least into English. My purpose was to obtain a sample consisting of pairs of home pages, available in Italian and English. Such criteria allowed me to analyse a sample applying the heuristic tool prosed by Andrey-Vall and Marcos (2012). The sample selection has been done on the basis of three annotations. First annotation: After collecting a list of museums, I needed to get a list of web sites from it. I classified the 129 museums by province in an XLSX document. For each museum, I reported the url as indicated in “Abbonamento Musei” (Annex 1). However, the relation between web site and museum is not univocal. In fact, in some cases, one url is reported several times as it refers to different museums or museums clusters. Second annotation: After inspecting the home page of each web site, the following considerations have been taken into account: 67 web sites are not localised, some are not indicated and some direct to different home pages. The following table tries to show the typologies of web site found. For the province of Lodi (LO), there is no museum indicated in the list. 77 Category Item Notes Unlocalised web site Item 93, 99, 109 By clicking on the language selector, you do not have access to the localised version of the web site. For what concerns items 93 and 99, language selectors direct to a web page with tourism information in English. While for what concerns item 109, they direct to a downloadable document in English displaying the museum’s map. As the home pages are not localised, they are not taken into account. Url not indicated Item 5, 58, 60 New web site Item 66, 69, 70, The web site indicated redirects to the new one. The latter is taken into account for 71 the analysis. Same url and same web Item 35, 36, 18, Web sites appearing more times are counted just once. site 36, 42, 43, 47, 48, 56, 61, 64, 70, 71, 74, 75, 76, 82, 83, 84, 95, 96, 97, 98, 99, 100, 111, 119, 127, 128 Similar url, but different Item 17 It may seem that the web site indicated (www.comune.milano.it/museostorianatu- web site rale) is the same as for item 4 (www.comune.milano.it/museoarcheologico), i.e. two different web pages in the same web site. Instead, they do refer to different home pages (Figure 18), as the latter redirects to the web site (www.museoarcheologicomi- lano.it). Item 64 The indication provided is a web page (www.bresciamusei.com/pinacoteca.asp), taken from the web site www.bresciamusei.com. Since the home page of the web site is already taken into account, this web page is not analysed. Different url, same web Item 68, 69 During the analysis, I found out that two different url actually refer to the same web site site (www.palazzote.it/index.php/it/ and www.museodellacitta.mn.it/index.php/it/). Only the first web site has been considered for the analysis. See Figure 20. Splash page Item 28 The web site indicated (www.onde.net/desenzano/citta/museo/) directs to a splash global gateway,30 hence users have to select the language (Italian or English) before starting the navigation. The home pages of the Italian and English version are taken into account for the analysis: www.onde.net/desenzano/citta/museo/index_ITA.htm www.onde.net/desenzano/citta/museo/refresh/ENG/INDEX/index_ENG.htm Item 34 The web site indicated (www.museosetacomo.com/) directs to a splash page, hence users have to click on “Enter website” before accessing the home page. The home pages of the Italian and English versions are taken into account for the analysis: www.museosetacomo.com/home.php www.museosetacomo.com/eng_home.php Web site under con- Item 38 Since the new web site is under construction, the old one is taken into account. struction Item 62 Not taken into account. 78 Item 57 Not taken into account. Broken link Item 45 Since the link proposed is broken, I retrieved the web site using Google suggestions. Multilingual web site Item 106 The multilingual web site of the same complex is chosen for the analysis. International web site Item 121, 127, The international web site is chosen for the analysis. The FAI – Fondo Ambiente 128 Italiano (National Trust for Italy) has two web sites. The web site indicated (www.fondoambiente.it) is the national one and it is available in Italian only. Further, every attraction has a different web page dedicated to it (e.g. www.fondoambiente.it/luoghi/villa-della-porta-bozzolo). None of them is localised. FAI, after all, has an international web site available in Italian, English and French (fai-international.org/). Since the latter is localised, it is considered for the analysis (Figure 19). Table 12. Considerations on Second Annotation. Figure 18. The home page of the web sites of “Civico Museo Archeologico” and “Museo di Storia Naturale di Milano”. 30 Language selectors, also called “global gateways” (Yunker, 2003; Roturier, 2015, p. 58), are a “strategy for directing users to their localised web pages” (Yunker, 2003, p. 168). By clicking on a small device, usually located at the top of the page, users are allowed to switch language. A “splash global gateway” is “a web page that users see before progressing on to the home page” (Yunker, 2010, p. 28). This means that the first thing users see before accessing the content of a web site is actually a page where they are asked to choose the language. The splash global gateway is also referred to as international or landing page (Ibid.). 79 Figure 19. The FAI web sites. Items 121, 127 and 128. Figure 20. Musei Civici di Mantova. Civic museums of Mantua. Therefore, to summarise, for the purpose of my analysis, I took into account only those web sites whose home pages were localised (IT-EN), usually preferring updated to outdated versions. When an official multilingual or international version of the web site was available for the same museum or organisation, the multilingual or international version was taken into account. In case one web site was reported more times, it was counted just once. After these considerations, I obtained a list of 98 web sites in total (Annex 2). 80 Third annotation: After obtaining a sample of web sites, my objective was to focus on the home page to collect a sample of multilingual home pages. Results from third annotation show that 51 out 98 web sites are localised in English, at least for what concerns the home page (Annex 3). Final consideration on the data sampling: I chose to analyse the home pages of these 51 web sites in two languages. Therefore, as I took into account only home pages for the linguistic pair IT-EN, I obtained a list of 102 home pages (Annex 4). My final sample consists of 102 home pages, 51 in Italian and 51 in English, where items are listed as 1IT, 1EN, 2IT, 2EN etc. The choice to focus on the home page has been already explained in the previous chapter. According to (Nielsen and Loranger, 2006, pp. 30–32), experienced users spend less than 30 seconds on the home page of a web site, and users in general spend “less and less time on the home page with each subsequent visit” (Ibid. p. 32). After scanning 102 home pages, the scanning became quicker and finding “dismissing things” became easier. This was due to the “standardised nature” of the home page content. With this statement, I do not mean to say that home pages are all similar. I only mean that we all have a general idea of what a home page looks like, and this is due to the “load of mental baggage, accumulated from prior visits to a thousand of home pages” (Tahir and Nielsen, 2002, p. 37). In fact, “users have accumulated a generic mental model of the way home pages are supposed to work, based on their experience on other web sites” (Ibid.). 4.2 Analysis of the Data After describing the process of data collection, I hereby explain how the data collected have been analysed applying the method of heuristic evaluation. Home pages have been inspected by a single evaluator, with the aim to check if the guidelines proposed by Andreu-Vall and Marcos (2012) were respected or not, and to what extent. 4.2.1 Group 1. Aspects related to source code 1.1 Unicode codification (charset UTF-8) The guideline concerns the use of Unicode codification for the representation of characters set. Andreu- Vall and Marcos (2012) remind us that UTF-8 (8-bit Unicode transformation format) should be used as character encoding (Section 3.3, Guideline 1.1). Results referring to the total sample show that 84 out of 102 items (82% of total) respect the guideline, whereas 18 (18%) do not respect the guideline. Results referring to the IT sample show that 42 out of 51 items (82%) respect the guideline, whereas 9 (18%) do not respect the guideline. The same results were found in the analysis of the EN sample. Results are shown in Chart 1. 81 Items with unsuitable encoding use the ANSI (Windows-1252) Character Set. The charset attribute appears as follows: <meta http-equiv="Content-Type" content="text/html; charset=windows-1252"> ANSI is the default character set in Windows up to Windows 95 (w3schools, 1999g), and it is a 1 byte encoding of the Latin alphabet. As only Latin alphabet is used, characters are nonetheless correctly displayed, but, as such items do not comply with the guideline, they have been considered unsuitable. Unicode codification 100% 18% 18% 18% 80%60%40% 82% 82% 82%20%0% Total IT ENSuitable UnsuitableChart 1. Unicode codification. No difference was found between IT and EN home pages. This is due to the fact that Unicode character set can display all the necessary characters to represent the majority of human languages (Yunker, 2003, p. 42). UTF-8 encoding allows to display languages such as Chinese, Arabic, English, Russian and Japa- nese in one web page, therefore it can easily process both Italian and English text. ANSI as well can display both Italian and English characters. There was therefore no need to use a different encoding since only Latin characters are used. In this case the error distribution is symmetrical (9 errors were found in both the IT and EN sample), which means that if the guideline is not respected in the Italian home page, the error is repeated in EN localised home page. In a Boolean logic, the score 1 has been assigned to home pages not respecting the guideline (unsuitable), while the score 0 has been assigned to home pages respecting the guideline (suitable). 1.2 Consistency between url language and content language This guideline refers to the language of the url, which must be specified using language codes (ISO 639) and must be the same language used for textual content. Results referring to the total sample show that 67 out of 102 items (66% of total) respect the guideline, whereas 35 items (34%) do not respect the guideline. Results referring to the IT sample show that 28 out of 51 items (55%) do not respect the guideline, whereas 23 (45%) respect the guideline. Results referring to the EN sample show that 7 out of 82 51 items (14%) do not respect the guideline, whereas 44 (86%) respect the guideline. Results are shown in Chart 2. The score 1 has been assigned to home pages not respecting the guideline (unsuitable), while the score 0 has been assigned to home pages respecting the guideline (suitable). Consistency between url and content language 100% 14%80% 34% 55%60%40% 86% 66%20% 45%0% Total IT ENSuitable UnsuitableChart 2. Consistency between url and content language. If we compare the results, there are more suitable items (40% more) in the EN sample. This may be due to the fact that these home pages are most likely to be localised from Italian into English, therefore I assume that the language code for English appears in the url because it is considered the foreign version (e.g. Item 25b: www.lacarrara.it/en/). Whereas the language code for Italian is omitted, as it is considered the original version (e.g. Item 25a: www.lacarrara.it/). Nonetheless, the guideline specifies that the language code in multilingual web sites should be indicated in the url. For example, the home page of the house-museum Casa Milà, known as La Pedrera (The stone quarry) can be considered an example of good practice for this guideline, despite the degree of localisation, as each home page (English, Catalan and Spanish) respects the aforementioned guideline (Figure 17). 83 Figure 21. La Pedrera. Consistency between url language and content language: www.lapedrera.com/en (accessed on 6 April 2018). 1.3 Declaration of content language in every page heading In every page heading, the lang attribute to the <html> element should be specified to define the language in which the textual content is written. The following example sets the default language to English: <html lang="en"> As already discussed in section 2.1.1 and section 3.3 (Guideline 1.3), the language codes are defined by ISO 639. Results referring to the total sample show that 70 out 102 items (68.5%) respect the guideline, whereas 10 (10%) do not respect the guideline. Further, in 22 cases (21.5%) the lang attribute is missing. We can refer to Chart 3 to see the difference between suitable, unsuitable and missing attributes. Declaration of content language 100.0% 21.5% 21.5% 21.5% 80.0% 10% 10% 10%60.0%40.0% 68.5% 68.5% 68.5%20.0%0.0% Total sample IT sample EN sampleProper Improper MissingChart 3. Declaration of content language. An attribute can be considered suitable or unsuitable depending on the language of the home page (IT or EN). That is why, for example, <html lang="it"> is considered suitable when found in an Italian 84 home page, but unsuitable when found in an English home page. To get a more precise idea of the actual composition of the sample, I hereby describe the results found for the IT and EN sample. Results referring to the IT sample show that 35 out of 51 items (68.5%) respect the guideline, whereas 5 (10%) do not respect the guideline. Further, in 11 cases (21.5%) the lang attribute is missing. Results referring to the IT sample reveal the following composition: Category Number Language attribute Notes Suitable attributes 18 <html lang="it"> 12 <html lang="it-IT"> locale specification 5 <html lang="it-it"> locale specification* 35 Total Unsuitable attrib- 4 <html lang="en"> utes 1 <html lang="it,en"> 5 Total Missing attributes 11 Total Table 13. Declaration of content language (IT). Results referring to the EN sample are the same as the IT sample: 35 out of 51 suitable attributes (68.5%), 5 (10%) unsuitable attributes, and 11 (21.5%) missing attributes. The composition though is different: Category Number Language attribute Notes Suitable attributes 21 <html lang="en"> 10 <html lang="en-US"> locale specification 4 <html lang="en-gb"> locale specification* 35 Total Unsuitable attrib- 2 <html lang="it"> utes 1 <html lang="it,en"> 1 <html lang="it-IT"> 1 <html lang="it-it"> 5 Total Missing attributes 11 Total Table 14. Declaration of content language (EN). As the results found for both the IT and EN sample are the same, they do not need to be compared. In other terms, when the language attribute is missing in the IT home page, it is also missing in the EN 85 home page. If we have a look at suitable declarations, in most cases when the locale is specified in IT, it is also specified in EN (See Figure 22). Figure 22. Declaration of content language. Symmetrical composition. Only in one case (Item 41), the content of the language attribute is asymmetrical: IT. Item 41a: <html lang="it-IT"> EN. Item 41b: <html lang="en"> On the other hand, the number of cases when the declaration of content language does not match the actual language of the home page is the same in IT and EN: IT. Item 37a: <html lang="it-IT"> EN. Item 37b: <html lang="it-IT"> IT. Item 14a: <html lang="en"> EN. Item 14b: <html lang="en"> In such cases, the default language of the template is not localised. A point that needs to be clarified is the locale specification. As seen above, in some cases, the content of the lang attribute specifies both the language and the locale in the form of codes. According to the W3C, locale specification is not necessary, in fact, when creating language attributes, it is better to keep them as short as possible. Region, script and other “subtags” should be avoided, “expect where they add useful distinguishing information” (W3C, 2014b). For example, the language code for Japanese ja should be used instead of ja-JP, “unless there is a particular reason that you need to say that this is Japanese as spoken in Japan, rather than elsewhere” (Ibid.). In my sample, the specification it-IT may refer to the Italian spoken in Italy, rather than in Switzerland, since the museums are Italian. The en-US or en-gb specification, does not seem to refer to any locale in particular. In fact, there are no elements in the home page pointing out the use of a specific variety of English. For example, item n. 9 (museobagattivalsecchi.org/en/) has the following language declaration lang="en-US", but the image used as language selector is the UK flag. All these elements led me to think that there was no connection between the locale specification and the content language. Nonetheless, by checking the language used in the entire web site, in the web page “Visit the Museum”, the language used actually corresponds to American English: 86 A personalized visit to the museum is possible, accompanied by the heir of the Bagatti Valsecchi family, Pier Fausto. The audio guides, realized by Acoustiguide, are available free-of-charge with entrance ticket. The audio guides are available in Italian, English, Chinese, French, Japanese, German and Russian. Table 15. Item n.9. “Visit the Museum” page in English. Section Audio-guides. Therefore, in order to define if the locale specification is correct or not, the entire web site should be checked, as the home page inspection may not be enough. Further, in some cases locale specification is marked with an asterisk (*). In fact, the value of the lang attribute should identify the language in ISO 639-1 format, and optionally the region in ISO 3166-1 Alpha 2 format” (Google, 2018). In my sample, language is specified using language codes (ISO 639), the locale, however, is not always specified using country codes (ISO 3166-1 Alpha 2 format), as two lowercase letters are used: <html lang="it-it"> <html lang="en-gb"> As stated by Yunker (2003, p. 51), by conventions the language code is written in lowercase, while the country code is written in uppercase. The country code is also informally known as region subtag, as the value of the lang attribute (tag) can be made of different parts (subtags), each specifying an object (script, dialect, region etc.). For example, az-Latn means Azerbaijani, written in Latin script, as Azerbai- jani can also be written using the Arabic script (W3C, 2014b). The way each subtag must be written is determined in the guidelines defined by organisations such as W3C, IANA (Internet Assigned Numbers Authority), IETF (Internet Engineering Task Force) etc. The use of language tags and subtags is regulated by the IETF, whereas an extensive list of all the codes used as subtags of the lang attribute is provided by the IANA Language Subtag Registry.31 Only one region subtag can appear in a language attribute. The region code follows the language code. The region subtag can be either a two-letter alpha code, i.e. country codes made by the ISO 3166-1 (e.g US in en-US for American English) or a 3-digit numeric code (e.g. es-005 for South American Spanish). Once again, region subtags should only be used if they are necessary to make a distinction (W3C, 2014b). 31 “Language tag syntax is defined by the IETF’s BCP 47. BCP stands for ‘Best Current Practice’, and is a persistent name for a series of RFCs [Request for Comments] whose numbers change as they are updated. The latest RFC describing language tag syntax is RFC 5646, Tags for the Identification of Languages” (W3C, 2011). The document “Tags for identifying languages” is available at tools.ietf.org/html/bcp47#section-4.1 (last accessed on 18 May 2018), and it explains the use of language tags and subtags. Whereas, the IANA Language Subtag Registry, available at www.iana.org/assignments/language-subtag-registry/language-subtag-registry (last accessed on 18 May 2018), lists all the types of language tags and subtags. 87 The code en-gb is not written neither using a 3-digit numeric code nor a two-letter alpha code (ISO 3166-1 Alpha 2 format). However, according to the rules of the HTML syntax, the attribute names are “case-insensitive”, which means that they can be written “with any mix of lowercase and uppercase letters” Therefore, both en-gb and en-GB can be considered correct to specify the use British English (W3C, 2014b). The use of the code gb instead of UK made me question the correctness of this code. However, the W3C confirms the use of such code: “According to BCP 47 en can be said to match en- GB” (W3C, 2014b), referring to the locale of United Kingdom. For the purpose of this analysis, the value of the lang attribute has been considered suitable every time the language code actually matches with the content language, regardless of locale specification, which should be first proven to be necessary. The scores 0 and 1 have been assigned to home pages according to the aforementioned criteria. The error sum is given by the sum of unsuitable and missing declarations. In this case the error distribution is symmetrical (16 errors were found in both the IT and EN sample), which means that if the guideline is not respected in the Italian home page, the error is repeated in EN localised home page. 1.4 Change of content language in the same web page Any time that a content on the page is in a different language from that declared in the lang attribute of the <html> element, language attributes should be used on elements surrounding that content to signal the new language (W3C, 2014a). By doing so, that content can be processed differently, for example, when screen readers are used by visually impaired people (Section 3.3, Guideline 1.4). Given the non-Boolean nature of the data analysed, i.e. multiple errors can be found within the same home page, the score 1 has been assigned to home pages not respecting the guideline (unsuitable) – 1 error makes the home page unsuitable. While the score 0 has been assigned to home pages respecting the guideline (suitable), hence only when the error frequency is equal to 0. Results are shown in the following table. They are presented according to the number of errors found and the number of home pages presenting a specific range of errors (e.g. from 1 to 5 errors). 88 2616 21116<31-0 errorsSample- - - -- 0 ScoreScore 1 Score10 10HomeErrors pages5 errors 15 20 25300 errors errors errors errors errors errorsIT 355 7 29 8 3 2 1 1 0 51 7 44 14% 86% EN 449 9 23 5 4 5 0 0 5 51 9 42 18% 82% Tot. 784 16 52 13 7 7 1 1 5 102 16 86 16% 84% Table 16. Change of content language: error distribution. As shown in Annex 6, which indicates the exact number of errors per page, I found 7.7 errors per page on average, with significant variance from IT and EN home pages (6.6 vs. 8.8). In general, I found more errors in the EN home pages (449) than in the IT home pages (335). But, there are exceptions such as Item 26—the Italian home page presents 4 errors, while its English counterpart presents only 2 errors. In the Italian home page, I found 3 words in English used as text string in the <a> element, and 1 paragraph in Latin, as shown in the table below: Item 26a Example 1) <a href="http://www.grottedicatullo.beniculturali.it/index.php?it/21/news" title="News">News</a> 2) <a href="http://www.grottedicatullo.beniculturali.it/index.php?it/148/links" title="Links">Links</a> 3) <a href="http://www.grottedicatullo.beniculturali.it/index.php?it/15/disclaimer" title="Disclaimer">Disclaimer</a> 4) Paene insularum, Sirmio, insu- larumque ocelle, quascumque in liquentibus stagnis marique vasto fert uterque Neptunus, quam te libenter quamque laetus in- viso, vix mi ipse credens Thyniam atque Bithynos liquisse campos et videre te in tuto. ...Salve, o venusta Sirmio, atque ero gaude: gaudete vosque, o Lydiae lacus undae: ridete, quicquid est domi cachinnorum! Table 17. Item 26a. Words in another language. In the English home page, I found 2 errors—one error is the text shown in the heading, which reports the title of the home page in Italian, the other error is a paragraph left in Italian, as shown below: Item 26b Example 1) <h2 id="HomeText-Title">Grotte di Catullo</h2> 2) Area archeologica delle Grotte di Catullo e Museo di Sirmione Table 18. Item 26b. Words in another language. 89 Chart 4 shows the error distribution across the total, IT an EN sample, as well as the distinction between suitable and unsuitable homepages. Change of content language 100.0% 5.0% 3.9% 1.0% 6.0% 1.0% 3.9% 2.0% 7.0% 5.9% 2.0% 9.8% 7.0% 80.0% 15.7% 7.9% 12.5% 9.8% 60.0%56.9% 40.0% 51.0% 45.0%20.0%17.6% 15.5% 13.7% 0.0% Total sample IT sample EN sample 0 errors 1-5 errors 6-10 errors 11-15 errors 16-20 errors 21-25 errors 26-30 errors <30 errorsChart 4. Change of content language. For what concerns the total sample, nearly 16% of home pages can be considered suitable, as they do not present any error. On the other hand, 84.5% of home pages can be considered unsuitable (from 1 to +30 errors). Among unsuitable items, we can distinguish a further 51% presenting only a few errors (from 1 to 5 errors), and a 5% of items that most probably require urgent intervention (+30 errors). In the middle, we can find 28.5% of the items presenting from 6 errors to 30 errors. For what concerns the IT sample, nearly 14% of home pages can be considered suitable (0 errors), 86.3% of home pages can be considered unsuitable (from 1 to+30 errors), among which 3.9% of the items present +30 errors. For what concerns the EN sample, nearly 18% of home pages can be considered suitable (0 errors), and 82.4% can be considered unsuitable (from 1 to +30 errors), among which 6% of home pages present +30 errors. The difference between unsuitable home pages in the two sub-samples is not significant—nearly 4% more unsuitable home pages in the English sample. Error frequency is high in both samples, which mostly present unsuitable home pages. Results concerning English home pages may not be surprising, 90 since content text is most likely translated from Italian into English, some parts of text may remain unlocalised. It is also not surprising that unlocalised portions of text might not be signalled due to time constraints and monetary budget limitations. Nonetheless, I did not expect such a high error frequency in the Italian sample. And yet, the Italian text abounds with anglicisms, whose language is not indicated in the source code. Due to the large amount of data, I do not provide a categorisation of error typologies. Nonetheless, an insight on anglicims and italianisms found in the sample is provided in the next chapter (Section 5.1.1). To conclude, I would like to mention that the greatest limitation to the analysis was to retrieve textual content in the HTML document to check the foreign language specification in the code. If I had a pro- gram to extract textual strings, one that does not remove elements and attributes, this task would have been much easier, as I could concentrate on the content immediately without retrieving it manually from the source code. However, it should also be mentioned that the idea of using a text extractor was considered only at the end of the analysis, which could not be repeated due to my time constraints. None- theless, the use of a text extractor could be considered for future studies on this specific guideline to help the evaluator focus on the textual string. 1.5 Text expansion Since the length of a target text is likely to be different from that of the source text, textual content can expand or contract, depending on the target language. Therefore, the lay-out of a web page should be flexible in view of possible contractions and expansions. I checked if there are any overlaps in the textual content. No overlap was found. Using an online tool (Chapter 3, Section 3.3, Note 12), I measured the number of characters used in the IT and EN homepages. In an XLSX document, I recorded the numbers obtained for the IT and EN items. My aim was to see in how many cases text expands or contracts, and by comparing IT and EN results, I could verify if English text generally contracts in my data sample. For each item, I subtracted the number of characters obtained in EN from the number of characters obtained in IT (IT-EN); the result obtained shows the difference in terms of characters from IT and EN text. Then, using the formula (EN/IT)-1, I obtained the percentage of text expansion or contraction for each item. Negative numbers represent a text contraction; positive numbers represent a text expansion. The following table may help clarify the process: 91 Item 3 858 - 711= 147 -17.13% 17.13% contraction in IT: 858 characters (711/858)-1= -0.1713 English EN: 711 characters Item 4 867 - 902= -35 4.04% 4.04% expansion in Eng- IT: 867 characters (902/867)-1= 0.040 lish EN: 902 characters Table 19. Calculations to determine text expansion and text contraction. Results referring to the total sample show that in 36 out of 51 cases32 (70.5%), English text contracts (from -1.6% to -86.6%), in 13 out of 51 cases (25.5%), English text expands (from 0.2% to 301.3%), and in 2 cases (4%), the text length does not change (0% and 0.03%). The results obtained are shown in Chart 5. 100.0% 3.9%25.5% 80.0%60.0%40.0% 70.6%20.0%0.0% EN sampleText contraction Text expansion No changesChart 5. Text expansion/contraction. The percentages of text contraction (from -1.6% to -86.6%) and text expansion (from 0.2% to 301.3%) reflect the different degrees of localisation. In fact, text contraction and expansion, in my sample, do not only depend on the verbosity of the translator or the target language, but to the varying degrees of localisation. In 12 out of 51 cases, the home pages have less content and less characters in English because they are partially localised, and some spaces remain empty. It is the case, for example, of item 12 (Figure 23). 32 The divider is 51 because we do not consider the total number of items (102), but the number of pairs IT-EN (51). 92 Figure 23. Item 12. EN-IT home pages. Content update. In 4 cases, for example in items 29 and 30, the English page in under construction, and text contraction measures -82.9% and -86.6%. These pages are not localised (Figure 24). Using the classification proposed by Singh and Pereira (2012, p. 35), some home pages cannot even be considered partially localised, but standardised web pages. For example, in item 39, the English version is actually a template for further localisation and the content is standardised text used as place holder. This means that the comparison between EN and IT characters cannot be considered completely reliable. Better results could have been achieved by excluding unlocalised and standardised pages. However, different degrees of localisation were not taken into account for the analysis of text expansion. Due to time constraints, text expansion and text contraction of the data were not evaluated on the base of the form and content of the home pages, but it would be interesting to do consider this aspect in further future analyses. An insight on the degrees of localisation is, nonetheless, proposed in section 4.2.2 (Guideline 2.5). Figure 24. Item 29. EN-IT homepages. Content update. In this case, the focus of the analysis is not the error frequency. The test carried out shows that text length does not vary in almost 4% of pages, while in the greatest majority of pages (71%), English text 93 contracts, as expected when discussing Guideline 1.5 (Chapter 3, section 3.3). In a significant proportion (25.5%), English text expands. 1.6 Language change specification in the anchor tag of a link This guideline concerns the language change in a link, which means that when a link directs to a page, whose language is different from that of the source page, the language change should be indicated using the hreflang attribute in the <a> element. “The value of the hreflang attribute identifies the language (in ISO 639-1 format) and optionally the region (in ISO 3166-1 Alpha 2 format) of an alternate URL”(W3C, 2014b). Further, “the reserved value "x-default" is used for indicating language selectors/redirectors which are not specific to one language or region, e.g. your homepage showing a clickable map of the world (Ibid.). In this sub-section, I explain the sample composition for what concerns the total, IT and EN samples, I have analysed 427 hyperlinks to other web pages. In 316 cases (75.4%), the hreflang attribute was not found. In 6 cases (1.4%), a variant is used. In 64 cases (15.3%), the hreflang attribute was suitablly indicated. Further, 13 hyperlinks directed to partially localised pages (3.1%), 7 hyperlinks were broken links (1.7%), 10 hyperlinks directed to empty pages (2.4%), and 3 hyperlinks directed to downloadable documents in Italian (0.7%). Considering that the downloadable documents are in Italian, when referring to the IT sample, there is no need for language change declaration. When referring to the EN sample, they can be considered unsuitable, as the language of the destination page is different from that of the home page. The following table summarises the composition of the IT sample, i.e. the typologies of attributes out of 108 hyperlinks directing to pages in another language. It also shows whether the attributes found have been considered suitable or unsuitable: Categories Number of Percentage Correctness items Missing attribute 69 64% ✕ Unsuitable Unsuitable at- Empty pages 3 ✕ tributes Partially localised pages 2 ✕ Broken links 2 ✕ Total 7 6% ✕ Unsuitable Suitable attrib- hreflang="en-US" 5 ✓ utes hreflang="en-GB" 1 ✓ hreflang="fr-FR" 1 ✓ hreflang="de-DE" 2 ✓ 94 Categories Number of Percentage Correctness items hreflang="es-ES" 1 ✓ hreflang="en-us" 1 ✓ hreflang="en" 7 ✓ hreflang="x-de- 5 ✓ fault" hreflang="zh" 1 ✓ hreflang="zh-CN" 1 ✓ 1 ✓ 1 ✓ 1 ✓ 1 ✓ 1 ✓ 1 ✓ 1 ✓ Total 32 30% ✓ Suitable Table 20. Language change in hyperlinks (IT). The variant <a class="en-lang"> has been considered unsuitable. In fact, the class attribute to links “is used to point to a class in a style sheet” (w3schools, 1999a). As seen in the previous chapter, the following “pseudo language attribute” :lang(en-GB) { color: red; } can be used in the CSS code to highlight English text in red, as long as the lang attribute is used to specify the language of textual content hooked in an element. In this case, the class attribute may point to the class "en-lang" in the CSS, but it is not a matter of style. The most appropriate way to declare language in a link is using the hreflang attribute. The lang attribute may also be used as follows (W3C, 2014a): <a lang="es" title="Spanish" href="qa-html-language-declarations.es">Español</a> The hreflang attribute, in fact, “provides user agents with information about the encoding of data at the other end of the link”, i.e. “the language of a resource at the end of a link, just as the lang attribute provides information about the language of an element’s content or attribute values” (W3C, 2018b). The hreflang attribute, though, is specific to links, whereas the lang attribute is supported by almost all element (W3C, 2018a). 95 The indication hreflang="x-default" is used for language selectors and it has been considered suitable. In fact, “the reserved value "x-default" is used for indicating language selectors/redirectors which are not specific to one language or region”(Google, 2018). “This markup tells Google’s algorithm to consider all of these pages as alternate versions of each other […]. To indicate to Google that you want the German version of the page to be served to searchers using Google in German, the en-us version to searchers using google.com in English, and the en-gb version to searchers using google.co.uk in Eng- lish, use rel="alternate" hreflang="x" to identify alternate language versions” (Ibid.). In 7 cases the lang attribute to the element is used. The element “is used to group inline-elements in a document [and] provides a way to add a hook to a part of a text or a part of a document” (w3schools, 1999c). Table 21 shows the <a> element containing the href attribute (where the link’s destination is specified), the element used to add language specification through its own lang attribute, and the <img> element used to specify the image displayed on screen. Example are taken from the data sample. According to the considerations above, the language specification has been considered suitable. In fact, in order to specify the language of some content when there is no markup around it, it is suggested to use an element such as or <div> around the content (W3C, 2014a). Here is an example (Ibid.): You’d say that in Chinese as 中国科学院文献情报中心. In the example provided, the lang attribute could have been added to the element, as well as in the data analysed the hreflang attribute could have been added to the <a> element. However, I do not know the reason behind the choice to use the element. Since it is not an error, it has been considered suitable. <a href="http://www.museofrancescogonzaga.it/int/eng- lang="en"> English</a> (1) <a href="http://www.museofrancesco- many.png"> Deutsch</a> (1) <a href="http://www.museofrancescogon- sia.png"> Русский</a> (1) Table 21. The element and the language attribute. From the analysis of the EN sample, I could draw the same considerations. The same criteria have been applied for the data analysis. In the EN sample, I found 319 hyperlinks directing to pages in another language. In most of those cases (255, i.e. 80%) the hreflang attribute was not found. The rest had 96 some formatting issues (details can be checked in the table below. Table 22). The following table summarises the composition of the EN sample, where I found 319 hyperlinks directing to pages in another language. It also shows whether the attributes found have been considered suitable or unsuitable: Categories Number of Percentage Correctness items Missing attribute 255 80% ✕ Unsuitable Unsuitable at- Empty pages 8 ✕ tributes Partially localised pages 13 ✕ Broken links 13 ✕ Variants 3 ✕ Downloadable docu- 3 ✕ ments in Italian Total 40 12.5% ✕ Unsuitable Suitable attrib- hreflang="it-IT" 6 ✓ utes hreflang="fr-FR" 1 ✓ hreflang="de-DE" 2 ✓ hreflang="es-ES" 1 ✓ hreflang="it-it" 2 ✓ hreflang="it" 6 ✓ hreflang="x-de- 5 ✓ fault" hreflang="zh" 1 ✓ hreflang="zh-CN" 1 ✓ 1 ✓ 1 ✓ 1 ✓ 1 ✓ 1 ✓ 1 ✓ 1 ✓ Total 32 7.5% ✓ Suitable Table 22. Language change in hyperlinks (EN). 97 Missing attributes and unsuitable variants have been considered errors. Further, the list of errors could be implemented by including links directing to partially localised pages, empty pages, and downloadable documents in a different language, and broken links. These typologies of links have been counted as errors. In the IT sample, 30% of the attributes are suitable, while 70% of the attributes are unsuitable. In the EN sample, 7.5% of the attributes are suitable, while 92.5% of the attributes are unsuitable (Chart 6). Language change in links 100%80%64% 60% 76% 80%40% 6%20% 10% 30% 12.5% 14% 7.5% 0% Total IT ENSuitable Unsuitable MissingChart 6. Language change in links. First of all, I would like to point out that there are more links directing to pages in another language in EN than IT. This is may be due to the fact that we are considering Italian museums, whose home pages are most probably localised from Italian into English, it is then obvious why these home pages are pointing out to web sites of Italian public and private entities, such as local authorities or banks: E.g. Item 15b: Banca Intesa San Paolo <a href="http://www.intesasanpaolo.com/" target="_blank" title=""></a> In both cases, the majority of hyperlinks has no hreflang attribute, which means that in the sample this attribute is quite rare to be found, i.e. the majority of links directing to web pages in another language does not signal the language change. The amount of missing attributes might reveal a lack of care in the specification of language change in hyperlinks. The number of suitable cases in the IT sample (30%) is greater than that in the EN sample (7.5%), which means that in the IT home pages the language change in the <a> element is most probably specified in the correct way. Further, the number and type of errors found in the EN sample (40 errors, 12.5% of unsuitable cases)—wrong variants, partially localised pages, 98 broken links, downloadable documents in Italian might explain a general lack of care in the localisation of English home pages. In the IT sample, these errors are a half less frequent (6%). For what concerns the error distribution, considering the non-Boolean nature of the data analysed, the score 1 has been assigned to home pages not respecting the guideline in at least one occasion (unsuitable); while the score 0 has been assigned to home pages respecting the guideline (suitable), hence only when the error frequency is equal to 0. The table below shows the number of home pages presenting from 1 to 5 errors, from 6 to 10 errors etc. This categorisation represents an attempt to normalise the results obtained. Home pages Home16116<20 1-0 errorsSample- - -Score 0 ScoreScore 1 Score10 10Errors5 errors 15 20 errors errors errors errorsIT 76 35 16 0 0 0 0 51 35 16 69% 31% EN 295 11 24 11 3 1 1 51 11 40 21.5% 78.5% Tot. 371 46 40 11 3 1 1 102 46 56 45% 55% Table 23. Language change specification in links: error distribution. The table shows that in the total sample 46 home pages (45%) have 0 errors, hence they are suitable; while 56 (55%) present from 1 to +20 errors, which means that they are unsuitable. In the IT sample, 35 home pages (69%) are suitable, while 16 home pages (31%) are unsuitable. To conclude, in the EN sample, only 11 home pages (21.5%) are suitable, while the majority (78.5%) is unsuitable. 4.2.2 Group 2. Aspects related to localisation 2.1 Date format Date formats should be adapted according to the locale conventions. By scanning the home page, I detected date formats. Date formats respecting the stylistic rules summarised in Table 4 and Table 5 have been considered suitable. Results referring to the total sample show in 58 out 102 home pages (57%), date formats are suitable, while in 24 home pages (24%), date formats are unsuitable. In 19 home pages (19%), there is no date format indicated in the home page. Results are summarised in Chart 7, which shows the percentage of suitable, unsuitable and missing date formats. 99 Date formats 100% 19% 16% 22% 80% 10% 24% 60% 39%40% 74% 57% 20% 39%0% Total IT ENSuitable Unsuitable MissingChart 7. Date formats. Results referring to the IT sample show that in 38 out 51 pages (74%), date formats are suitable, while in 5 (10%) pages, date formats are unsuitable. In 8 (16%) pages, there is no date format indicated in the home page. The unsuitable cases reveal error typologies: item 1a presents 16 calques from English, item 34a and 42a present 2 and 27 untranslated date formats, and item 46a uses ordinal numbers, even if their use is not suggested by the manual of style consulted for the analysis (Lesina, 1986; Petricola, 2011). Further, two errors of inconsistency have been found (items 37a and 46a). In item 37a, one date format is inconsistent within the text: in one case, slashes are used as number separators, whereas in all the other dates the dot is used. In item 46a, numeral and ordinal numbers are used in the same home page. Whether inconsistency should be considered an error could be discussed. In fact, it is not mentioned in the checklist proposed by Andreu-Vall and Marcos (2012). However, the error typology guidelines developed by TAUS (Translation Automation User Society) consider inconsistency a stylistic error in the category of translation specific errors.33 For the purpose of this analysis, inconsistency has been considered an error. Even if it does not affect the comprehensibility of the text, the use of different date formats can be considered not stylistically correct. I am not referring to abbreviation of date formats (e.g. 5/10/2017 instead of 5 ottobre 2017), that could depend on lay-out constraints, but to inconsistency within the text once a stylistic format is used (e.g. 1° Aprile al 1° Novembre; GIOVEDI 26 APRILE 2018; Mercoledì 25 Aprile). Through the analysis of unsuitable cases, 49 errors have been found. In this case, the absence of date formats in the home page cannot be considered an error. Error categories and composition are summarised in the following tables. 33 TAUS list of error types and categories is available on the following web site www.taus.net/knowledgebase/index.php?title=Error_type#Style (last accessed on 20 May 2018). 100 Error typology Number of error Example Error Use of English format (e.g. 16 Item 1a: http://casami- 16 dic, 2017 (x4) Dec. 16, 2017). The use of lan.acmilan.com/it 31 mar, 2018 (x2) comma in date formats is not 15 apr, 2018 (x3) proper to Italian. There is no 06 mag, 2018 (x4) reference to the use of 08 apr, 2018 (x2) comma in the manual con- 24 mag, 2017 - 31 ott, 2017 (x1) sulted. Date in English 29 Item 34a: 09 May 2018 http://www.museoseta- 04 May 2018 como.com/index.htm Item 42a: http://www.ta- January 1, 1913 zionuvolari.it/it/ Inconsistency between 2 Item 37a: 13/05 numbers separators and http://www.museoseta- 15.04.2018 formats garlate.it/ 25.02.2018 Use of ordinal numbers 2 Item 46 a: Dal 1° Aprile al 1° Novembre http://www.rossiniart- 25 Aprile, 26 APRILE 2018 site.com/ Table 24. Date formats (IT). Results referring to the EN sample show that in 20 out of 51 pages date formats are suitable (39%), while in 20 pages date formats are unsuitable (39%). In 11 pages, there is no date format indicated in the home page (22%). Through the analysis of unsuitable pages, 105 errors have been found. The error composition is explained in the following table. Error typology Number of error Example Not translated date 70 Item 51b: Sabato 5 Maggio 2018 formats Ordinals numbers 29 Item 7b: 30th of January, 7th of May, 14th of April, 9th of May 2018 in date formats Item 26b: January 1st, May 1st, December 25th Typographic error 1 Item 20b: Celebrating the Centenary of His Birth: 1918-20188 Inconsistency 3 Item 36b: 01 January 2018 - 31 December 2020 March 13 - June 12 April 7 - May 19 Missing date trans- 2 Item 37b: IT: 15.04.2018 lation EN: missing Table 25. Date formats (EN). 101 Results show that there are more suitable items (35% more) in the IT sample. This may be due to the fact that IT home pages are mostly likely written by Italian native speakers who are aware of stylistic guidelines when writing date formats. On the other hand, we do not have information about the localisers who transposed those dates in English; they could be translators but not English native speakers, they could be Italian web masters with some knowledge of English. In any case, there are 30% more errors in the English sample, showing that English home pages obtained poorer results in this guideline compliance. I would like to point out that this guideline has been checked referring to the stylistic rules explained in Chapter 3 (Section 3.3, Guideline 2.1). Having at disposal different stylistic manuals where more rules are discussed, I could have obtained different results. The figures referring to missing date formats (16% IT, 22% EN) show that date formats are not always localised in the English home pages, as a result they are missing causing a lack of information for users. For what concerns error distribution, the following table shows the number of errors (groups of error) found per home page. The score 1 has been assigned to home pages not respecting the guideline (unsuitable), where at least one error was found. The score 0 has been assigned to home pages respecting the guideline (suitable), and to home pages where no date formats was found. In this case, since date formats are missing, they could not be evaluated. Nonetheless, not having a date format in the home page does not represent an error per se, therefore these home pages are given the score 0. 16116<20 1-0 errorsSample- - -ScoreScore 1 Score10 10Errors Home pages5 errors 15 20 errors errors errors errors0IT 49 46 3 0 0 1 1 51 46 5 90%* 10% EN 105 31 15 2 1 2 0 51 31 20 61%** 39% Tot. 154 77 18 2 1 3 1 102 77 25 75.5%*** 24.5% Table 26. Date formats: error distribution. * 90% = 74% suitable + 16% missing ** 61% = 39% suitable + 22% missing ***75.5% = 57% suitable + 19% missing 102 2.2 Time format Time formats should be adapted according to the locale conventions. By scanning the home page, I detected time formats. Time formats respecting the stylistic rules summarised in Table 6 and Table 7 have been considered suitable. Results referring to the total sample show that in 42 out of 102 pages (41%) time formats included are suitable, while in 10 cases (10%), time formats are unsuitable. In 50 pages (49%), there is no time expression indicated, which means that time formats are not a common feature in the home pages analysed. Results are summarised in Chart 8. Time formats 100%80% 49% 49% 49% 60% 10% 6% 40% 16%20% 41% 45% 35% 0% Total IT ENSuitable Unsuitable MissingChart 8. Time formats. Results referring to the IT sample, show that in 23 out of 51 pages (45%), time formats indicated are suitable, while in 3 pages (6%), time formats included are unsuitable (10 errors in total). In 25 pages (49%), no time format is indicated in the home page. The results found are shown in Table 27. As these home pages are most likely written in Italian by native speakers, the great difference between suitable and unsuitable items might be due to this reason. Number of pages Correctness Error category Example 23 Suitable ✓ / Item 10a: 13.00 – 17.45 3 Unsuitable ✕ / (10 errors) 9 Use of comma Item 8a: ore 15,30 (1 error) as number sepa- Item 32a: rator. ORARI DI APERTURA : Note legali lunedì, martedì, giovedì, venerdì dalle 9,00 alle 12,30 e dalle 14,30 alle 15,30 mercoledì dalle 9,00 alle 12,30 sabato (solo Anagrafe) dalle 9,00 alle 12,30. 103 1 Inconsistency Item 23a: 9.00 - 13.00, 15:00 -18:00 (1 er- between num- ror) ber separators. Table 27. Time formats (IT). Results referring to the EN sample, show that in 18 pages out of 51 (35%), time formats included are suitable, while in 8 pages (16%), time formats included (20 errors in total) are unsuitable. In 25 cases (49%), no time format is indicated in the home page. Error categories and results are shown in Table 28. Number of pages Correctness Error category Example 18 Suitable ✓ / Item 22a: h. 21.00 8 Unsuitable ✕ / (20 errors) 2 Inconsistency Item 18b: Sunday: Open 8.30 > 19.15 (24h/12h) Ticket office closes at 6.40 p.m. 5 Unfamiliar (abbrevi- Item 8b: Last admission 17 ated form) every Tuesday from 14 1 Partially translated Item 20b: 3.30 pm - (Italiano) 19.30 11 Not translated. Item 40b: dalle ore 20.30 1 Missing translation. Item 37b: IT: 15.00 -16.00 Visita guidata EN: / Table 28. Time formats (EN). When comparing errors, results show that there are more suitable occurrences (10% more) in the IT sample. As afore-mentioned, this may be due to the fact that IT home pages are mostly likely written by Italian native speakers who are aware of stylistic guidelines when writing date formats. If we compare Chart 8 to the previous chart, the difference between proper items in the IT and EN sample is not so neat. This means that time formats are more likely to respect the guideline than date formats. This may be due to the fact that time formats can be written using the 12h or 24h format, while there are way more formats for dates. The figure referring to missing attributes shows that time formats are quite rare to be found in the home page. The scores 0 and 1 have been assigned to home pages according to the criteria mentioned for guideline 2.1. The error distribution is shown in the table below. 104 16116<201-0 errorSample- - -Score 0 ScoreScore 1 Score10 10Errors Home pages5 15 20 errors errors errors errors errors sIT 10 48 2 1 0 0 0 51 48 3 94%* 6% 3EN 20 43 7 1 0 0 0 51 43 8 84%** 16% Tot. 30 91 9 2 0 0 0 102 91 11 90%*** 10% Table 29. Time formats: error distribution. * 94% = 45% suitable + 49% missing ** 83% = 35% suitable + 49% missing *** 90% = 41% suitable + 49% missing 2.3 Number format Number formats should be adapted according to the locale conventions. By scanning the home page, I detected number formats. Number formats respecting the stylistic rules explained in the previous chapter (Section 3.3, Guideline 2.3) have been considered suitable. The stylistic rules are here summarised: IT: comma  decimal separator; space  thousands separator EN: dot  decimal separator; comma  thousands separator Results referring to the total sample show that in 83 out of 102 pages (81%), there is no number indicated. In 12 pages (12%), number formats are suitable, while in 7 (7%) number formats are unsuitable (7%). Results are summarised in Chart 9. 105 Number formats 100% 18% 80%60% 81% 80%40% 82%20% 7% 14% 12% 0% 6% Total IT ENSuitable Unsuitable MissingChart 9. Number formats. Results referring to the IT sample show that in 42 out of 51 pages (82%), number formats are suitable, while in 9 pages (18%), there is no number indicated. There are no unsuitable cases in the IT sample. This may be due to the fact that these home pages are most likely written in Italian by native speakers. Results referring to the EN sample show that in 41 out of 51 pages (80%), there is no number indicated, which means that numbers are very rare to be found in the pages analysed. In 3 pages (6%), number formats are suitable, while in 7 (14%) number formats are unsuitable. Considering that 82% of IT home pages have suitable number formats, and 80% of EN home pages do not have number at all, I assume that those numbers have been omitted in the EN home pages, producing a lack of information for users, as in Item 37b (Table 28). In EN home pages, only 6% has proper number format, the rest of information is missing or poorly localised. I found 23 errors: in 22 cases, the text is translated, but the number format is not localised, i.e. comma is used as decimal separator (e.g. Item 21b: Reduced: € 8, 00). In one case, both text and number formats are not translated (e.g. Item 36b: Biglietto intero: 10 € (tariffe ridotte e altre info)). The scores 0 and 1 have been assigned to home pages according to the aforementioned criteria. The error distribution is shown in the table below. 106 16116<20 errors1-0 errorsSample- - -Score 0 ScoreScore 1 Score10 errors 10Errors Home pages5 errors errors 15 20 errorsIT 0 51 0 0 0 0 0 51 51 0 100%* 0% EN 23 44 6 1 0 0 0 51 44 7 86%** 14% Tot. 23 95 6 1 0 0 0 102 95 7 93%*** 7% Table 30. Number formats: error distribution. * 100% = 82% suitable + 18% missing ** 86% = 6% suitable + 80% missing *** 93% = 12% suitable + 81% missing 2.4 Contact section The contact section should be adapted according to the locale conventions. Telephone numbers must be indicated specifying the area code. The object of the analysis is the textual content given as contact information in the home page. I did not analyse the contacts web page, but the contact information given in the home page. By scanning the home page, the contact information has been detected and evaluated. Results referring to the total sample show that in 29 out of 102 pages (28.5%), there is no contact information in the home page, in 44 pages (43%), contacts are suitable, while in 29 (28.5%), contacts are unsuitable. Results are summarised in Chart 10. 107 Contact section 100% 28.5% 27.5% 29% 80%60% 28.5%40% 57% 72.5%20% 43% 14% 0% Total IT ENSuitable Unsuitable MissingChart 10. Contact section. Results referring to the IT sample show that in 14 out of 51 pages (27.5%), contact information is not given in the home pages, whereas in 37 (72.5%), contacts are suitable. There are no unsuitable contacts in the IT sample. This may be due to the fact that these home pages are most likely written in Italian by native speakers. Results referring to the EN sample show that in 15 out of 51 pages (29%), contact information is not given in the home page. In 7 pages (14%), contacts are suitable, whereas in 29 (57%), contacts are unsuitable. If we compare the IT and EN results, we can conclude that when the contact information is given, it is always correct in Italian. However it is poorly rendered in English in 57% of pages. In the EN sample, I found 66 errors. The error composition is summarised in the following table: Number of errors Error category Example 12 City not translated Item 1b: Via Aldo Rossi, 8 20149 Milano 20 Area code missing Item 22b: Tel. 035 962780 25 Not translated Item 40b: Viale Te, 13 - 46100 Mantova Biglietteria e informazioni Telefono +39 0376 323266 8 Country not translated Item 23b: Via San Tomaso, 53 24121 Bergamo Italia 1 Missing contact information in Item 34b: English IT: Museo didattico della Seta Via Castelnuovo, 9 - 22100 Como - Italia Tel/Fax: 031 303180 info@museosetacomo.com 108 press@museosetacomo.com EN: / Table 31. Contacts section (EN). The scores 0 and 1 have been assigned to home pages according to the aforementioned criteria. The error distribution is summarised in the table below. 16116<20 errors1-0 errorsSample- - -Score 0 ScoreScore 1 Score10 errors 10Errors Home pages5 errors errors 15 20 errorsIT 0 0 0 0 0 0 0 51 51 0 100%* 0% EN 66 22 28 1 0 0 0 51 22 29 43%** 57% Tot. 66 22 28 1 0 0 0 102 73 29 71.5%*** 28.5% Table 32. Contact section: error distribution. * 100% = 72.5% suitable + 27.5% missing ** 43% = 14% suitable + 29% missing *** 71.5% = 43% suitable + 28.5% missing 2.5 Content update in every localised version Content should be updated in every localised version. The object of the analysis is textual content and layout. The analysis focuses on the content of localised home pages (EN), which is compared to the content of IT home pages. Localised home pages (51 items - EN) are inspected and compared to the IT home pages. IT home pages are the frame of reference for the analysis, and they are considered suitable by default. EN home pages are classified according to their degree of localisation. As the IT sample is the frame of reference (gold standard), results refer to the EN sample only. Results show that in 26 out of 51 cases (51%), there is no difference in terms of content and lay out between IT and EN home pages, i.e. EN home pages are totally localised. These 26 cases are considered suitable because represent the best available solution of guideline compliance. In 25 cases (49%), the localisation of EN home pages cannot be considered complete. These 25 cases are considered unsuitable because neither content is updated nor localisation is complete. More specifically, in 4 cases, EN home pages are under construction. In 8 cases, EN home pages are poorly localised (most content remains in Italian or is missing). In 12 cases, EN home pages are partially localised (most content is in English, but some 109 relevant content is still missing). In 1 case, the localised version of the home page is not available, EN content is given in an info page for tourists with different information from the IT home page. The different degrees of localisation of EN home pages can be summarised in the following table: Percentage Number of pages Degree of localisation Correctness 51% 26 Total localisation Suitable ✓ 49% 25 Unsuitable ✕ 12 Partial localisation ✕ 8 Poor localisation ✕ 4 Page under construction ✕ 1 Tourist info page ✕ Table 33. Degrees of localisation. As conclusion, if we compare EN home pages to the IT ones, 51% of items is suitable, i.e. just over half of EN home pages has up to date content. On the other hand, 49% of home pages needs an update intervention for users to access the same information available in IT home pages. By generalising, we can say that half of EN home pages requires intervention, while the other half does not. Results are shown in the following Chart 11. Content update 100%80% 49% 60% 100% 40% 51% 20%0% IT ENStandard frame Up to date content Not up to date contentChart 11. Content update. The score 0 has been assigned to all Italian home pages by default, as they represent the standard frame. In the EN sample, the score 0 has been assigned to the 26 suitable home pages (51%), while the score 1 has been assigned to the 25 unsuitable home pages (49%). 110 4.2.3 Group 4. Aspects affecting search in the web site 4.1 Search in all linguistic versions: “Search on the home page should search the entire site by default” (Tahir and Nielsen, 2002, p. 20), i.e. the searcher should be able to search information in all the web pages regardless of the language. I expected the searcher to find results in all linguistic versions, matching the language of the input given with the language of the results. Details on how the test has been carried out are provided in Chapter 3 (Section 3.3, Guideline 4.1). Results referring to the total sample show that in 51 out of 102 cases (50%), there is no searcher in the home page. In 13 cases (13%), search results are considered suitable, i.e. search results are consistent with the language of the search query regardless of previous language selection. In 38 cases (37%), search results are considered unsuitable, i.e. search results only refer to the web pages of one linguistic version. Results are shown in Chart 12. Search in all linguistic versions 100%80% 50% 49% 51% 60%40% 37% 37% 37% 20% 13% 14% 12% 0% Total IT ENSuitable Unsuitable MissingChart 12. Search in all linguistic versions. Results referring to the IT sample show that in 25 out of 51 cases (49%), there is no search feature in the home pages. In 7 cases (14%), the searcher has shown results regardless of language selection. Hence, these cases are considered suitable. In 4 cases (8%), the searcher does not work. These cases are considered unsuitable. In 15 cases (29%), the searcher has shown results in one linguistic version only. These cases are considered unsuitable. Suitable cases represent 14% of the total, while unsuitable cases represent 37% of the total. Results referring to the EN sample, show that in 26 out of 51 cases (51%), there is no searcher in the home page. In 6 cases (12%), the searcher has shown results regardless of language selection. These cases are considered suitable. In 4 cases (8%), the searcher does not work. These cases are considered unsuitable. In 15 cases (29%), the searcher has shown results in one linguistic version only. These cases are 111 considered unsuitable. Suitable cases represent 12% of the total, while unsuitable cases represent 37% of the total. The sample composition is quite symmetrical. In fact, in both linguistic versions, in 8% of home pages the searcher does not work, and in 29% does not respect the guideline, i.e. unsuitable cases represent 37% of the total. If we compare proper cases, the difference is minimal (2% more in Italian). In both samples, almost 50% of items does not have the searcher in the home page. The score 1 has been assigned to home pages not respecting the guideline (unsuitable), while the score 0 has been assigned to home pages respecting the guideline (suitable). It should be mentioned that the score 0 has been assigned also when the search feature is absent from the home page. In fact, it was not considered an error because the search could be dedicated its own separate page. Even if it would not be correct to say that the majority of home pages respect the guideline, for the purposes of this Master thesis the score 0 has been assigned also to home page that do not have the search feature in the home page, as the absence of this feature cannot be considered an error. Home Sample Errors 0 error 1 error Score 0 Score 1 pages IT 19 32 19 51 32 19 63%* 37% EN 19 32 19 51 32 19 63%** 37% Tot. 38 64 38 102 64 38 63%*** 37% Table 34. Search in all linguistic versions: error distribution. * 63% = 14% suitable + 49% missing ** 63% = 51% suitable + 12% missing *** 63% = 50% suitable + 13% missing 4.2 Advanced settings for search in a language: The user should have the possibility to choose the language of search results through advanced settings for search in a language. By inspecting the search feature, I verified if there are advanced settings for search in a language. Results referring to the total sample show that in 6 out of 102 cases (6%), the searcher has advanced settings for search in a language correctly working, whereas in 92 cases (90%), the searcher does not have advanced settings. In 4 out of 102 cases (4%), the searcher has advanced settings for search in a language, but they do not work correctly. These cases have been considered unsuitable. To be more precise, in 51 112 cases (50%), the home page has no searcher at all, which means that in 41 cases (40%), there is a searcher in the home page, but without advanced options for search in a language. If the home pages do not have a searcher in home page, it cannot be considered an error, as the web site may have a separated search page. However, in 40% of cases, a searcher in the home page without the possibility to set the search language is considered an error (Andreu-Vall and Marcos, p. 259). Results are shown in Chart 13. Advanced options for search in a language 100%80% 50% 49% 51% 60%40% 44% 45% 43% 20%0% 6% 6% 6% Total IT ENSuitable Unsuitable No searcherChart 13. Advanced options for search in a language. Results referring to the IT sample show that in 3 out of 51 cases (6%), the searcher has advanced settings for search in a language correctly working, whereas in 25 cases (49%), there is no searcher in the home page. In 21 cases (41%), there is a searcher in the home page but without advanced options for search in a language, and in 2 cases (4%), advanced settings do not work correctly. Suitable cases represent 6% of the total, while unsuitable cases represent 45% of the total. Results referring to the EN sample show that in 3 out of 51 cases (6%), the searcher has advanced settings for search in a language, whereas in 26 cases (51%), the home page has no searcher; in 20 cases (39%), there is a searcher in the home page but without advanced options for search in a language. Further, in 2 cases (4%), advanced settings do not work correctly. Suitable cases represent 6% of the total, while unsuitable cases represent 43% of the total. Results show that in both cases only 6% of home pages respect the guideline. Items 8, 42 and 43 have advanced options for search in a language in both IT and EN home pages. The great majority of home pages, instead, does not have any advanced options for search. Over the half has no searcher at all. The score 1 has been assigned to home pages not respecting the guideline (unsuitable), while the score 0 has been assigned to home pages respecting the guideline (suitable). It should be mentioned that the score 0 has been assigned also in case there is no search in the home page, as the absence of this feature 113 cannot be considered an error per se. The score 1, instead, has been assigned in case advanced settings for search in a language are not present in the home page, as Andreu-Vall and Marcos (2012, p. 259) state that the user should be able to select the language search according to their needs. Home Sample Errors 0 error 1 error Score 0 Score 1 pages IT 23 28 23 51 28 23 55%* 45% EN 22 29 22 51 29 22 57%** 43% Tot. 45 57 45 102 57 45 56%*** 44% Table 35. Advanced settings for search in a language: error distribution. * 55% = 6% suitable + 49% missing ** 57% = 6% suitable + 51% missing *** 56% = 6% suitable + 50% missing 4.3 Language of informative texts in results page: When search results lead to informative texts or documents, these texts and documents should be available in the language selected for the search. If there are no advanced options for search in a language, these texts should be available in the language of the home page. Results referring to the total sample show that only in 1 case (1%), 1 informative document was found. It is a downloadable document showing a map of Como in English. In 101 out of 102 cases (99%), no informative document was found. Actually, as aforementioned, in 51 out of 102 cases (50%), the home pages has no searcher at all. No informative document was found in the IT sample. The only document found was retrieved in the EN sample (2%). In 50 out of 51 cases (98%), no document was found (98%). Since only 1 informative text in English was retrieved in the EN sample, no error was found. Thus, the score 1 was never assigned. This means that the score 0 has been assigned to all home pages. The fact that the search queries tested do not provide any document for the analysis cannot be considered an error. Thus, for the purposes of this Master thesis, 100% of home pages is considered suitable in both the IT and EN sample. However, it should be mentioned that there is not enough material to state that this guideline is respected in 100% of cases, because only 2% of home pages was actually able to provide the material that needs to be tested for this guideline. Further tests where the results of other or different search queries are tested could shed light on this guideline compliance and alter the results obtained. 114 4.2.4 Group 5. Aspects affecting the web sites ranking in search engines 5.1 Translation of the <title> element content The value of the <title> element indicates the title of a web page and it should therefore be translated. Results referring to the total sample show that in 64 out of 102 cases (63%), the <title> element is suitable, i.e. its content is in the language of the home page. In 38 cases (37%), the <title> element is unsuitable, i.e. some errors have been found. Results are shown in the following chart. Results referring to the IT sample show that in 48 out of 51 cases (94%), the <title> element is suitable (94%), whereas in 3 cases (6%), the <title> element is unsuitable. Results referring to the EN sample show that in 16 out of 51 cases (31%), results are suitable, while in 35 cases (69%), results are unsuitable. Given the Boolean nature of the data analysed, the score 0 has been assigned to home pages with suitable attribute, while the score 1 has been assigned to home pages with an unsuitable attribute. Translation of title element100% 6%80% 37%69% 60% 94% 40% 63% 20% 31%0% Total IT ENSuitable UnsuitableChart 14. Translation of the <title> element. If we compare the results, almost every title in the IT home pages respect the guideline (94%). This figure drastically reduces to 31% in the EN home pages. In fact, Italian titles are most likely written by Italian native speakers. Instead, titles in the English version are not translated, showing a significant lack of care for proper translation of the title. In the following paragraphs, I explore the error typologies. IT: In one case, the content of the element is half in English (item 18a): the expression “Official Web- site” is preferred to the Italian “Sito ufficiale”. It could be a deliberate choice on the part of localisers. However, since the Italian expression exists, I consider the use of English in this case as an error. In another case, the content of the element is “index” (item 28a), and in the last case the content of the element is “home” (item 38a). Considering what said above, the value of the <title> element should 115 contain the title of the page. “Index” and “home” are too generic and could refer to every home page. Moreover, they do not add any informative meaning to the type of web page. Both item 28a and 38a refer to museums with a name: Civico Museo Archeologico “Giovanni Rambotti” and Villa Monastero. The error composition is summarised in the following table. Error category Number of errors Example Use of English 1 Item 18a: <title>Pinacoteca di Brera | Official Web- site No title (Generic) 1 Item 28a: index No title (Generic) 1 Item 38a: Home Total 3 Table 36. Translation of the element (IT). EN: The analysis of unsuitable cases shows that there are 34 errors in the home pages evaluated. The error composition is summarised in the following table. Error category Number or errors Example Not translated. 25 Item 20b: <title>La Triennale di Milano Item 13b: Museo Nazionale della Scienza e della Tecnologia "Leonardo da Vinci" Page not localised 5 Item 28b and 29b: Home - MIBAC demo Tourism info page. 1 Item 32b: Tourism No title (Generic). 2 Item 28b: index Item 38b: Home Inconsistent with 2 Item 46: Italian text. Rossini Art Site | Un luogo da vivere Home Page | Rossini Art Site Total 35

Table 37. Translation of the element (EN). 116 In 25 out of 35 errors, the content of the title element is not translated, i.e. it is the same as in Italian. As most titles indicate museums’ proper names, it is quite hard to establish whether the omitted translation could be considered an error. When checking guideline 1.4, proper names were excluded from the analysis, except from museums’ names. In the English informative content, where the museum is presented, the name of the museum can be rendered or, at least, explained using synonymous words. When assessing the title textual value, it is hard to establish a baseline between correct and incorrect. In fact, the choice to avoid the translation of museums names could be due to the will to emphasise the Italian museum tradition (foreignisation). This may be the reason why <title>Teatro alla Scala (item 15) remains the same in the two versions. Comparing this title with the title of another famous theatre, I could notice that the Russian and English versions are different.34

Большой театр – официальный сайт35

The Bolshoi Theatre Official Website36

Like in the case of the Bolshoi Theatre, a proposition for the value in our case could be La Scala Theatre. In fact, just because Italian and English use the same alphabet or some words are intellegible (the Italian word “teatro” resembles the English word “theatre”, likewise “museo” resembles “museum”), this does not mean that the translation can be omitted. The language proximity could be enough to make it understandable. Moreover, it could be a foreignisation choice made on purpose (Chap- ter 2, Section 2.1.1). In fact, translating the name implies that it may not be on public signs nor known by the locals, especially in less tourist destinations. Hence, it would be difficult to find it if tourists ask for directions. However, if the sample included localised home pages into Russian or Chinese, an omitted translation could be considered a severe error, hindering users’ comprehension. Language proximity can be examined, but I think it should not be taken into account when evaluating the translation of a textual content. For the purpose of this analysis, I decided to consider the omitted translation of the title content an error, on the basis of what generally learnt during the translation classes at our faculty: the client wants to obtain clarifications from the translator. Without translation, the reader, or user in our case, is forced to interpret alone. In some cases, in fact, the title could be considered a little obscure. It may not be the case for the afore- mentioned case, nor for item 5b (<title>Duomo di Milano - Homepage), but it may be for item 20b (La Triennale di Milano). In fact, I do not think that an English speaker would understand from this title what the museum is about. A title like The Triennale of

34 This example is taken from the web page www.bolshoi.ru/en/ (last accessed on 22 May 2018).

36 This example is taken from the web page www.bolshoi.ru/ (last accessed on 22 May 2018). 117

Milan Design Museum may be more explicative. In any case, I hereby explain only how I distinguished suitable from unsuitable items. My considerations about what is clear and what is unclear should be verified by a test on users (native and non-native English speakers). In fact, the English version of these home pages is a source of information about museums also for those people who speak English as a second language (ESL). I think that this point should not be underestimated, since the majority of these home pages are not localised into other languages. Considering the role of English “as the major language of international communication” (Hewson, 2009, p. 110), in other terms the impact of English as global language or English as lingua franca (ELF), the omitted translation of titles could force the user to interpret alone. As said above, a test on users could reveal to what extent English titles are clearer than Italian titles. In this case, language proximity could be an interesting variable to evaluate.

For the purpose of this analysis, items 5b, 15b, 20b and similar have been considered unsuitable. The main reason is that the Italian title is the same, i.e. localisation might not have taken place. Again, this assumption should be verified by a questionnaire to web masters and localisers. Further, the sample itself provides criteria for the evaluation. In some cases, the title is localised, while in others localisation is missing, but required to understand the meaning, as shown in the table below. On this purpose, a test on users would reveal in how many cases localisation of the title value is necessary for comprehension.

Sample Type Example IT Title in Italian Item 48a: Zavattarello, la perla verde dell'Oltrepò <a href="/tags/Pavese_(territory)/" rel="tag">Pavese</a> EN Title translated in Eng- Item 48b: Zavattarello, the green pearl of lish Oltrepò Pavese IT Title in Italian Item 26a: Home - Grotte di Catullo EN Title in Italian Item 26b: Home - Grotte di Catullo Table 38. Title elements types. In 5 cases, the English home pages are not localised, neither is the title value. Both the home page content and title are standardised templates: e.g. Home - MIBAC demo. In 1 case, the English home page does not exist. English information is displayed in a web page for tourists. In 2 cases, the title is too generic. As aforementioned, it does not contain informative content on the museum. In 2 cases, the English title is not an actual translation of the Italian title. In this case, as human evaluator, I assessed whether the translation was accurate or not. In item 46b, the English title is suitable, but “Un luogo da vivere” (A place to live) is not translated.

118

IT Title in Italian Rossini Art Site | Un luogo da vivere EN Inaccurate translation Home Page | Rossini Art Site

Table 39. Translation of the element. Item 46. In item 41, the English title is a list of keywords, that cannot be considered a suitable translation. IT Title in Italian <title>Benvenuti al museo - Museo Francesco Gon- zaga EN Inaccurate translation Francesco Gonzaga Museum, Mantua, It- aly

Table 40. Translation of the element. Item 41. 5.2 Translation of the content of description and keywords attributes 5.2.1 Translation of the description attribute content The value of the description attribute, specified in the content tag, should be translated. Results referring to the total sample show that in 53 out of 102 cases (52%), the description attribute was not found. In 35 cases (34%), the description attribute is suitable, as the value of the description attribute is in the language of the home page. In 14 cases (14%), the description attribute is unsuitable. In 3 cases, the value of the description attribute is empty, which means that it does not contain information snippet; while in 11 cases, the value is not translated. Missing attributes are considered unsuitable, as well. In fact, if these attributes are missing snippets cannot be created. Results are shown in Chart 15. The score 0 has been assigned to those home pages with suitable description attribute, while the score 1 has been assigned to home pages with an unsuitable or missing attribute. Description attribute 100%80% 52% 57% 51% 60%14% 40% 23.5%43% 20% 34% 25.5% 0% Total IT ENSuitable Unsuitable MissingChart 15.Translation of the description attribute. 119 Results referring to the IT sample show that in 27 out of 51 cases (53%), the description attribute was not found. In 22 cases (43%), the <description> attribute is suitable, as the value of the <description> attribute is in Italian. In 2 cases (4%), while searching the <description> attribute, I came across the open graph protocol suitablety called “description”. The open graph protocol (suitablety="og:description") “enables any web page to become a rich object in a social graph”, by adding properties to the meta element (Open Web Foundation Agreement, Version 0.9., 2017). For example, it is “used on Facebook to allow any web page to have the same functionality as any other object on Facebook”, adding the chance to like and comment (Ibid.). Since the open graph to the meta element has a different function, and it is not used to the replace the description attribute, its use cannot be considered unsuitable for the aforementioned reason. On the other hand, when the description attribute is missing, it is considered an error. The sample composition is summarised in the following table: Category Number of Percentage Example Correctness pages Not found 27 53% / Unsuitable ✖ Open 2 4% Item 48a: / graph <meta suitablety="og:description" content="Si informano i visitatori che Rossini Art Site resterà chiuso nella giornata di giovedì 26 aprile 2018."> Suitable 22 43% Item 6a: Suitable ✓ <meta name="description" content="Innovativo museo e centro di ricerca dedicato a Leonardo da Vinci"> Table 41. Translation of the description attribute (IT). Results referring to the EN sample show that in 26 out of 51 cases (51%), the description attribute was not found. In 13 cases (25.5%), the description attribute is suitable, i.e. the value of the description attribute is translated into English. In 12 cases (23.5%), the description attribute is unsuitable. In 9 cases, the value of the description attribute is not translated, while in 3 cases, the value is empty. If the value is not translated, users searching for those items in English will not be able to read the description of the web site in English. If the value is empty, the snippets summarising the page content cannot be displayed. The EN sample composition is summarised as follows: 120 Category Number Percentage Subcategory Example Correctness of pages Not found 26 51% / / ✖ Suitable 13 25.5% Translated Item 9b: ✓ value <meta name="description" content="The Bagatti Valsecchi Mu- seum is the fruit of the extraordinary collecting experience of two brothers at the end of the 19th century. Unsuitable 12 23.5% / / ✖ 9 / Not trans- Item 21b: ✖ lated value <meta name="description" content="Davanti al Cenacolo Vinciano a Milano apre un nuovo museo: La Vigna di Leonardo da Vinci nella Casa degli Atellani. Benvenuti nel sito ufficiale."> 3 / Empty value Item 32b: ✖ <meta content="" name="description"> Table 42. Translation of the description attribute (EN). If we compare the IT and EN home pages, we can notice that in the IT sample there are not unsuitable items, in other terms the error percentage is given by missing attributes (57%). In the IT sample, almost 60% of home pages has no description attribute, otherwise the attribute is in Italian (43%). Missing attributes represent the greatest error category in the EN sample as well (51%), since only 23.5% is mistakenly not translated into English or empty. We can conclude then that over the half that home pages have no description attribute.121 5.2.2 Translation of the keywords attribute content The value of the keywords attribute, specified in the content tag, should be translated. Results referring to the total sample show that in 68 out of 102 cases (67%), the attribute was not found. In 21 cases (21%), the attribute is suitable. In 11 cases (13%), the value of the description attribute is unsuitable. Missing attributes are considered unsuitable as well. In fact, the keywords attribute contains the words that will be retrieved by search engines, and used to display the search engine results page (SERP). If the attribute is untranslated, missing or empty, search engines cannot find out what words customers use in the “keywords research” on search engines (Nielsen and Loranger, 2006, p. 166). Results are shown in Chart 16. The score 0 has been assigned to those home pages with suitable keywords attribute, while the score 1 has been assigned to home pages with an unsuitable or missing attribute. Keywords attribute 100%80%67% 67% 67% 60%40% 4% 13% 20% 22% 29% 21% 12% 0% Total IT ENSuitable Unsuitable MissingChart 16. Translation of the keywords attribute. Results referring to the IT sample show that in 34 out of 51 cases (67%), the attribute was not found. In 15 cases (29%), the attribute is suitable. In 12 suitable cases, the value of the attribute is in Italian, while in 2 cases the value of attribute is multilingual. In one case (item 39a), the value uses a different syntax, i.e. token are not separated by commas. Nonetheless, since keywords are indicated in Italian in the content tag, and keywords queries on the search engine Google show the museum’s web site on the first search engine results page, this case was considered suitable. In 2 out of 51 cases (4%), the value of the attribute is unsuitable. In one case, the value of the attribute is empty, which means that it cannot be used by search engines or other web services. In another case (item 16a), the value is “parole chiave”, which means “keywords” in Italian, hence it does not explain what the web site is about. The reason behind 122 this mistake may be that web masters left the text written by default in the template they used for the web site. The IT sample composition is summarised in the following table. Category Number Error per- Subcategory Example Correctness of pages centage Not found 34 67% / / ✖ Suitable 15 29% ✓ 12 Italian value Item 31a: ✓ <meta name="keywords" content="Parco Nazionale delle Incisioni Rupestri, Capo di Ponte, Brescia, archeologia, incisioni rupestri"> 2 Multilingual Item 15a: value <meta name="keywords" content="La Scala,Mailand,Mi- lano,Milano Musica,Teatro alla Scala,opera,tickets"> Item 33a: <meta name="keywords" content="como, lago di como, scoprire, vivere, lago, villa olmo, visitcomo"> 1 Italian value Item 39a: ✓ + different <meta name="keywords" con- syntax tent="MUSEO ARCHEOLOGICO NA- ZIONALE MANTOVA"> Unsuitable 2 4% ✖ 1 Not pertinent Item 16a: ✖ <meta name="keywords" content="parole chiave"> 1 Empty value Item 49a: ✖ <meta name="keywords" content=""> Table 43. Translation of the keywords attribute (IT). Results referring to the EN sample show that in 34 out 51 cases (67%), the attribute was not found. In 6 cases (12%), the attribute is suitable. The value of the attribute is translated into English, yet multilingual text is included in the value. Multilingual text is considered suitable, and it might have been included to reach users from different locales. Text can be multilingual in both the original page and the localised page. While, in 11 cases (21%), the value of the attribute is unsuitable. In 5 cases, it is not translated, while in 6 cases, it is empty. When comparing these results to the IT results, we can notice that the proportion 123 of missing attributes is the same, as when the attribute is missing in the original home page, it is also missing in the English localised version. However only 12% of EN home pages has keywords in English, compared to 29% of IT home pages having keywords in Italian. Multilingual text is not considered a mistake, however if the keywords in the EN home pages remain in Italian (10%) it is considered a lack of localisation in the first place. Second, keywords are user-generated scent of information. “A query formulation is an opportunity to present searchers with desired content” (Thurow and Musica, 2009, p. 19). Hence, if results do not match the querying language, users could experience frustration and confusion. Category Number of Error percent- Subcategory Example Correctness pages age Not found 34 67% / ✖ Suitable 6 12% Translated value Item 6b: ✓ (Multilingual) <meta name="keywords" content="Leonardo da vinci, museum, exhibitions, machines, inventions, paintings, the last supper, secrets, milan, piazza della scala, Leo- nardo3, rent, codex, gallery, レオナルド, ダヴィンチ, レオナルド３, 美術館,"> Unsuitable 11 21% 5 Not translated Item 40b: ✖ <meta name="keywords" content="musei civici, musei civici del comune di mantova, palazzo te, palazzo san sebastiano, San Sebastiano, Te, Giulio Romano, Vasari, Federico II di Gonzaga, Gonzaga"> 6 Empty value Item 26b: ✖ <meta name="keywords" content=""> Table 44. Translation of the keywords attribute (EN). 124 5.3 Translation of the content of alt and title attributes in images 5.3.1 Translation of the alt attribute content The value of the alt attribute of an image should always be specified, so that the screen reader can recognise it and read it aloud for visually impaired people. The value of the alt attribute should be written using a simple and clear language, and translated for the localised versions of a web page. Null- alt is used for decorative images. Further details on this attribute are provided in Section 3.3, Guideline 5.3.1. Results are shown in Diagram 17: Results referring to the total sample show that out of 3190 images, 2341 attributes are indicated, but 849 are missing (27%). Among indicated attributes, 1798 are suitable (56%), while 534 are unsuitable (17%). Alt attribute 100% 27% 26% 27% 80% 10% 17% 60% 25%40% 64% 56% 20% 48%0% Total IT ENSuitable Unsuitable MissingChart 17. Translation of the alt attribute. I found 1792 images in the IT home pages, and 1234 alt attributes. This means that 468 images do not have the alt attribute, i.e. the alt attribute is missing (26%). Among the 1234 alt attributes (74%), 64% are suitable: 356 values are empty, 764 values are in Italian, 8 are in language other than Italian or English. On the other hand, 10% are unsuitable: 140 are in English, 37 are numeric, 3 are bilingual (IT-EN), and 16 are not grammatically correct. The IT sample composition can be summarised in the following table. 125 Category Number of Percentage Subcategory Example Correctness items Missing at- 468 26% Item 3a: Suitable at- 1128 64% ✓ tributes 764 Italian text Item 1a: 356 Empty text Item 15a: 8 Text in a an- Item 8a: Unsuitable 196 10% ✖ attributes 140 English text Item 13a: Numeric text 37 Item5a: alt="DSC06001 MI ✖ Duom Terraz Vsud R X8j Merc240x110 Lit6k4 3X2k2017" Not gram- 16 Item 51a: Bilingual text 3 Item 46a: 126 site.com/wp-content/uploads/2016/03/logoweb- 340.png" alt="Rossini Art Site" class="fusion-logo- 1x fusion-standard-logo" data-logo-height="100" data-logo-width="126" style="height: 100px;" width="" height=""> Table 45. Translation of the alt attribute (IT). As aforementioned, missing attributes are considered unsuitable. When the value contains English text, the attribute is considered unsuitable because it is not translated. When the value contains bilingual text, it is considered unsuitable because it is partially translated. When the value contains numeric characters or alpha-numeric characters, it is considered unsuitable because it can be ambiguous for impaired users. However, this assumption should be verified by a test on screen reader users. When the value contains not grammatically correct text, it is considered unsuitable because it is a linguistic error and it can be ambiguous for screen reader users. An example is provided from Item 51a: the word “mesuo” does not exist in Italian—it is a misspelling of the word “museo”. Another example is provided by Item 25a: alt="Twiiter <a href="/tags/Accademia_Carrara/" rel="tag">Accademia Carrara</a>", where the word “Twitter” is misspelled. In this case, as the word it is used in Italian as well, it is considered Italian text. When the value contains Italian text, the attribute is considered suitable. When the alt attribute value contains text in another language, but the image refers to language selectors, the attribute is considered suitable. The textual content of language selectors should, in fact, have the name of the actual languages in their own language or ISO 639 codes. When the alt attribute value is empty, the attribute is considered suitable, as it signals decorative images. My evaluation of alt attribute tries to assess whether the alt texts were helpful or well-written. However, this master thesis is “not the place for an in-depth discussion of how to improve usability for users with disabilities” (Tahir and Nielsen, 2002, p. 49), and it does not provide an insight on accessibility guidelines. Results referring to the EN sample are hereby explained. I found 1358 images in the EN home pages, and 1017 alt attributes. This means that 381 images do not have the alt attribute, i.e. the alt attribute is missing (27%). Among the 1017 alt attributes, 48% are suitable: 278 values are empty, 384 in English, 8 represent images for languages selectors. On the other hand, 25% are unsuitable: 280 values are in Italian, 36 are numeric, 7 are bilingual (IT-EN), 9 are in language other than Italian or English, among which only, 14 are not grammatically correct, and 1 value is a punctuation mark. The EN sample composition can be summarised in the following table. 127 Category Number Percentage Subcategory Example Correctness of items Missing at- 381 27% Item 3b: Suitable at- 670 48% ✓ tributes 384 English text Item 13b: 278 Empty value Item 4b: alt="" ✓ 8 Lang. selectors Item 3b: alt="FR" ✓ Unsuitable 347 25% attributes 280 Italian text Item 12b: 36 Numeric text Item 25b: alt="16x9" ✖ 14 Not grammati- Item 25b: alt="Twiiter Acca- ✖ cally correct demia Carrara" text 9 Text in lan- Item 13b: alt="Oil prêt-à-por- ✖ guage other ter" than IT or EN 7 Bilingual text Item 35b: 1 Punctuation Item 47b: alt="." ✖ mark Table 46. Translation of the alt attribute (EN). 128 As aforementioned, missing attributes are considered unsuitable. When the value contains Italian text, the attribute is considered unsuitable because it is not translated. When the value contains bilingual text, it is considered unsuitable because it is partially translated. When the value contains numeric characters, it is considered unsuitable because it can be ambiguous for screen reader users. However, this assumption should be verified by a test on screen reader users. When the value contains text in a language other that IT or EN and does not refer to language selectors, it is considered unsuitable. When the value contains a punctuation mark it is considered unsuitable. When the value contains not grammatically correct text, it is considered unsuitable because it is a linguistic error and it can be ambiguous for screen reader users. However, these assumptions should be verified by a test on users. When the value contains English text, the attribute is considered suitable. When the alt attribute value contains text in another language, but the image refers to language selectors, the attribute is considered suitable. When the alt attribute value is empty, the attribute is considered suitable, as it signals decorative images. My evaluation of alt attribute tries to assess whether the alt texts were helpful or well-written. However, this master thesis is “not the place for an in-depth discussion of how to improve usability for users with disabilities” (Tahir and Niel- sen, 2002, p. 49), and it does not provide an insight on accessibility guidelines. If we compare the IT and EN home pages, IT images respect the guideline more than EN images (16% more). The EN error composition is 15% greater and more diverse than the IT one. Given the non-Boolean nature of the data analysed, the score 1 has been assigned to home pages when the alt attribute is missing or unsuitable, while the score 0 has been assigned when the alt attribute is suitable, hence only when the error frequency is equal to 0. For what concerns the error distribution, the following table shows the number of errors (groups of errors) per home page. Home pages Home16116<20 errors1-0 errorsSample- - -Score 0 ScoreScore 1 Score10 errors 10Errors5 errors errors 15 20 errorsIT 664 4 21 5 3 6 12 51 4 47 8% 92% EN 728 3 16 8 6 5 13 51 3 48 6% 94% Tot. 1392 7 37 13 9 11 25 102 7 95 7% 93% Table 47.Translation of the alt attribute: error distribution. A maximum of 57 errors can be found in item 12b and 44a. In this case, the category measuring error frequency <20 errors may seem reductive and too general. In fact, the average errors frequency for this 129 category is 34.6 errors in IT, and 34.2 in EN. Nonetheless, the same number of error frequency categories has been applied to all the guidelines for coherency reasons. Results referring to the total sample show that 7 pages present 0 errors (7%), hence they are suitable; while the over majority of home pages (44) present at least one error (93%). Similarly, only 6% and 7% of IT and EN home pages are suitable, while the greatest majority are unsuitable (94% and 93%). These figures show that only a small percentage of home pages is free from errors. Since the alt attribute is a recurring feature in all the home pages analysed, the error frequency varies according to the number of images. Error frequency is very high, however, the insight provided on the error composition show that in the total and IT sample, most images have suitable attributes. Only in the EN sample, a little less of the half (48%) of images retrieved have suitable attributes. This is a clear example of guideline referring to non-Boolean data, hence the error composition and the error distribution across the samples lead to different, and yet interesting results. In fact, the results from the analysis of each image show that alt attribute is mostly suitable. But, when checking how many pages follow the guideline, only a few can be considered suitable and completely error-free. In fact, one single error compromises the home page in terms of guideline compliance. How- ever, whether these errors actually affect the home page in terms of navigation or user satisfaction should be verified by a test on user for every single home page. 5.3.2 Translation of the title attribute content The title attribute can contain the text of an image, which is seen by the user as tooltip text when hovering the mouse cursor on the image (w3schools, 1999e). The value of the title attribute should be translated. Results referring to the total sample show that out of 3190 images, attributes are indicated in 367 cases (11%), and they are missing in 2823 cases (89%). Among indicated attributes, 281 are suitable (9%), while 86 are unsuitable (3%). Results are shown in Diagram 18. 130 Images title attribute 100%80%60% 89% 88% 90%40%20% 3% 1% 4% 9% 11% 0% 6% Total IT ENSuitable Unsuitable MissingChart 18. Translation of the title attribute in images. In the IT home pages, I found 216 title attributes. Considering that in the IT sample there are 1792 images, missing attributes are 1576 (88%). Nonetheless, since the consequences of missing title attributes are not as serious as those of missing alt attributes, missing attributes in this case are not regarded as errors. In fact, the presence of both title and alt attributes is important for SEO (Andreu-Vall and Marcos, 2012, p. 259), but while, for example, decorative images should always be indicated with a null alt, the title can be omitted. The W3C (Techniques for WCAG 2.0 - H67) specifies that for each image that should be ignored, title attribute is either absent or empty, while the alt attribute is present and empty (W3C, 2016a). Therefore, absent title attributes are not considered an error. Among the 216 title attributes (12%), 192 attributes are suitable (11%)—9 empty values, 174 Italian texts, 1 suitable name, 8 language selectors. On the other hand, 24 attributes are unsuitable, as their value contains English text (1%). The sample composition can be summarised as follows: 131 Category Number of Percentage Subcategory Example Correctness items Missing 1576 88% ✓ attributes Suitable 192 11% ✓ 174 Italian text Item 12a: title="Leggi ✓ le recensioni su Trip Advisor" 9 Empty values Item 12a: title="" ✓ 8 Language selec- Item 9a: 1 Proper name in Item 51a: ti- ✓ German tle="Lindte" Unsuitable 24 1% ✖ 24 English text Item 37a: title="Twit- ✖ ter settings iframe" Table 48. Translation of the title attribute in images (IT). In the EN home pages, I found 151 title attributes. Considering that in the EN sample there are 1398 images, missing attributes are 1247 (90%). Among the 151 title attributes (10%), 89 attributes are suitable (6%)—3 empty values, 76 English texts, 1 proper name, 9 language selectors. On the other hand, 62 attributes are unsuitable (4%)—1 punctuation, 1 bilingual text and 60 Italian textual contents. If value of the title attribute is a punctuation mark, it is considered unsuitable, as it is not explicative. Partially and not localised content values are also considered unsuitable. The sample composition can be summarised as follows: 132 Category Number Percentage Subcategory Example Correctness of items Missing at- 1247 90% ✓ tributes Suitable 89 6% ✓ 76 English text Item 26b: ✓ title="Wildlife & An- imal lovers" 9 Language selec- Item 26b: ✓ tors ✓ title="" 1 Proper name in Item 51b: ✓ German title="Lindte" Unsuitable 62 4% ✖ 60 Not translated. Item 8b: ✖ Italian text title="Particolare della Torre del Filarete" 1 Punctuation Item 47b: ✖ mark title="." 1 Partially trans- Item 37b: ✖ lated. Bilingual title="Permalink to 15.04.2018 BAMBINI AL MU- SEO: Sogni di seta." Table 49. Translation of the title attribute in images (EN). Results (Chart 17) show out that the greatest majority of images do not have a title. The figures do not vary much between the IT and EN sample. Nonetheless, the majority of titles indicated remain in Italian (11% IT). If we have a closer look to the EN sample attributes, untranslated titles are not rare to be found—76 titles are in English, while 60 are in Italian. 133 For what concerns the error distribution, the score 0 has been assigned to home pages with both missing and suitable attributes; whereas, the score 1 has been assigned to home pages with unsuitable attributes. The table below shows the number of errors per page. Home pages Home16116<20 errors1-Sample- - -Score 0 ScoreScore 1 Score0 error errors 10Errors5 errors errors 15 20 err orsIT 24 39 12 0 0 0 0 51 39 12 76% 24% EN 62 31 15 5 0 0 0 51 31 20 61% 39% Total 86 70 27 5 0 0 0 102 70 32 69% 31% Table 50. Translation of the title attribute in images: error distribution. In the total sample, the majority of home pages (70 home pages) present 0 errors. In other terms, 69% of pages are suitable. While the remaining 31% are unsuitable, as 27 home pages present from 1 to 5 errors (26%), while 5 home pages present from 6 to 10 errors (5%). In the IT sample, 39 home pages (76%) are suitable; while 12 (24%) present from 1 to 5 errors, hence they are unsuitable. In the EN sample, the majority of home pages as well (61%) are suitable (31 home pages), while 15 home pages (29%) present from 1 to 15 errors, while only 5 (10%) present from 6 to 10 errors. No home page presents more than 10 errors, which is actually a good figure in terms of possible future error fixing. In this case, the analysis shows that in all the three samples, titles in images are mostly missing. Since its absence does not negatively affect the sample, the majority of home pages respect the guideline in the all three cases.5.4 5.4 Translation of the title attribute content in links The title attribute can contain the text of a hyperlink, which is seen by the user as tooltip text when hovering the mouse cursor on the link. The value of the title attribute should be translated. Results referring to the total sample show that there are 4173 hyperlinks in the sample: in 3076 cases (74%), the title attribute is missing, in 848 cases (20%), the title attribute is suitable, and in 249 cases (6%), the title attribute is unsuitable. Results are shown in Chart 18. 134 Links title attribute 100%80%73% 60% 74% 74%40%4% 20% 6% 9% 20% 22% 18% 0% Total IT ENSuitable Unsuitable MissingChart 19. Translation of the title attribute in links. In this case, it was not immediately clear whether to consider missing attributes suitable or unsuitable. Since they are the greatest category found in the sample, they can have a huge influence on the results when measuring error frequency. As the url of the destination is generally not sufficiently descriptive, the title attribute allows the user distinguish this link from other links in the web page and helps the user determine whether to follow the link (W3C, 2016b). However, whether its absence has a strong negative impact on usability could be debated. Title attribute to links is not mentioned in the design guidelines explained by Tahir and Nielsen (2002, p. 18). The W3C explains that when the content of a link contains both text and images, if the text is sufficient to describe the purpose of the link, the images may have an empty text alternative, in other terms a null alt. Nevertheless, if the text alternative of the <img> element is empty or the <a> element only contains text, the text of the link should describe the purpose of the link. Due to time constraints, I could not carry out such a detailed analysis, and missing title attributes have been generally considered suitable. An article from Nielsen (2016), “Using the Title Attribute to Help Users Predict Where They Are Going” played a decisive role in deciding the score (See Table 55). According to the world’s major usability expert, “the link title attribute can be used to provide additional details for mouse users, but should not be relied on as the main source of information scent” (Nielsen, 2016). Link titles should be short, less than 80 characters, and should only rarely go above 60 characters (Ibid.). Further, not every link actually need a title: If it is obvious from the link name and its surrounding context where the link will lead, then a tooltip will increase clutter and ultimately reduce usability. (Even if users may not actually purposefully hover on the 135 link, the tooltip may be displayed while the user moves the mouse.) A link title is superfluous if it simply repeats the same text that is already shown in the link label displayed on the page (Ibid.). To conclude, different browsers may display link titles in different ways, and most touchscreen browsers do not display the link title at all (Ibid.). For such reasons, I did not consider missing title links as an error, even if a more precise analysis, based on Nielsen 2006 and W3C recommendations, could lead to very interesting results in possible future studies. IT: In the IT home pages, I found 2344 hyperlinks, and 605 title attributes. Missing attributes are 1739 (74%). Among the 605 title attributes (26%), 524 attributes are suitable (22%)—107 empty values, 407 Italian texts, 1 Latin text, 9 language selectors. On the other hand, 81 attributes are unsuitable (4%)—48 English text, 15 ambiguous text, 8 bilingual text, 10 predefined IT text. In this case, missing attributes are not considered as an error. The composition of suitable attributes can be summarised in the following table: Category Number Percentage Subcategory Example Correctness of items Suitable 524 22% ✓ attributes 407 Italian text Item 47a: <a ✓ href="/user" title="Area riservata">Area riservata</a> 107 Empty value Item 23a: title="" ✓ 9 Language se- Item 44a: <a ✓ lectors href="/en" class="language- link en" xml:lang="en" title="English"> 1 Latin Item 43a: ti- ✓ tle="Art Bonus" Table 51. Translation of the title attribute in links (IT). Suitable cases. The composition of unsuitable attributes can be summarised in the following table: 136 Category Number Percentage Subcategory Example Correct- of items ness Unsuitable 81 4% ✖ attributes 48 English text Item 12a: title="Close Win- ✖ dow" 15 Ambiguous Item 46a: title="RSS" ✖ text 10 Predefined Item 40a: <a ✖ IT text href="http://www.palazzote .it/index.php/it/?layout- Mode=default" rel="nofollow" title="Questa è la modalità predefinita" id="zf--default-layout- link" class="grey-button- style rounded-corners show-tooltip">Predefin- ita. Questa è la mo- dalità predefinita.</a> 8 Bilingual text Item 37a: ✖ title="Permalink to 15.04.2018 BAMBINI AL MU- SEO: Sogni di seta." Table 52. Translation of the title attribute in links (EN). Unuitable cases. In the EN home pages, I found 1829 hyperlinks, and 492 title attributes. Missing attributes are 1337 (73%). Among the 492 title attributes (27%), 324 attributes are suitable (18%)—59 empty values, 256 English text, 9 language selectors. On the other hand, 168 attributes are unsuitable (9%)—143 Italian text, 11 ambiguous text, 4 bilingual text, 10 predefined EN text. In this case, missing attributes are not considered as an error. The composition of suitable attributes can be summarised in the following table: Category Number Percentage Subcategory Example Correctness of errors Suitable 324 18% ✓ attributes 256 English text Item 23b: <a ✓ href="http://www.gamec.it/e 137 n/frontpage#exhib" alt="Ex- hibitions" title="Exhibi- tions" class="menuitem menuitem-simple">Exhibi- tions</a> 59 Empty value Item 3b: <a ✓ href="https://www.stelline.it/en/stelline-foundation/exhibitions/current- exhibitions" title="">CUL- TURAL ACTIVITIES</a> 9 Language se- Item 44b: <a href="/it" ✓ lectors class="language-link it" xml:lang="it" title="Ital- iano"> </a> Table 53. Translation of the title attribute in links (EN). Suitable cases. The composition of unsuitable attributes can be summarised in the following table. Category Number Percentage Subcategory Example Correct- of errors ness Unsuitable 168 9% ✖ attributes 143 Italian text Item 12b: title="Link ✖ Corso di Calligrafia Ci- nese" 11 Ambiguous Item 46b: title="RSS" ✖ text 10 Predefined EN Item 40b: <a ✖ text href="http://www.palazzo te.it/index.php/en/?lay- outMode=full-access" rel="nofollow" title="This layout is suitable for screen readers, obsolete browsers and keyboard only navigation" id="zf-- full-access-link" class="grey-button-style 138 rounded-corners show- tooltip">Full access 4 Bilingual text Item 33b: <a ✖ href="https://twitter.com/visitcomo" rel="external" target="_blank" title="un- defined (Questo link si apre in una nuova pagina)">Twit- ter</a> Table 54. Translation of the title attribute in links (EN). Unsuitable cases. To conclude, title attribute in links is less rare than in images. Nonetheless, the greatest majority of links does not have a title. The number of errors found is slightly bigger in the EN sample (9%), while the number of suitable attributes is greater in the IT sample (22%), categories do not vary significantly. The score 0 has been assigned to home pages with both missing and suitable attributes; whereas, the score 1 has been assigned to home pages with unsuitable attributes. The table below shows the number of errors per page: 16116<20 errors1-0 errorsSample- - -Score 0 ScoreScore 1 Score10 errors 10Errors Home pages5 errors errors 15 20 errorsIT 81 30 16 3 2 0 0 51 30 21 59% 41% EN 168 24 16 4 5 2 0 51 24 27 47% 53% Total 249 54 32 7 7 2 0 102 54 48 53% 47% Table 55. Translation of the title attribute in links: error distribution. According to the results obtained, in the total sample 54 out of 102 home pages are suitable (53%), while 48 are unsuitable (47%). In the Italian sample, 30 out of 51 home pages are suitable (59%), while 21 are 139 unsuitable (41%). On the other hand, in the English sample, 24 out of 51 home pages are suitable 47%, while 27 are unsuitable (53%). Even in this case, the title attribute is missing in the majority of hyperlinks. However, since the lack of title does not negatively affect the sample, the majority of home pages in the total and IT sample respect this guideline. Whereas, in the EN sample, home pages mostly present unsuitable attributes. 4.3 Summary of the results After explaining and discussing the analysis of the results and error frequencies, I hereby summarise the results obtained. In Chapter 3 (Section 3.1), I presented the hypotheses and research questions: H1: the sample of home pages respects guidelines in at least 51% of cases (H1 = μ ≥ 51%) H2: error frequency in the IT pages is lower than error frequency in the EN pages (EFIT < EFEN) These tables summarise the results obtained, and will be used to test the hypotheses in the following chapter. For what concerns H1, I take into account the figures referring to the number of home pages, not the total number of errors found. This means that for guidelines 1.4, 1.6, 5.3.1, 5.3.2 and 5.4, I refer to the percentages shown in the tables referring to the error distribution (suitable and unsuitable home pages). For guidelines 2.1, 2.2, 2.3, 2.4, 4.1, 4.2 and 4.3, I could not measure the whole data-set because the features is absent from home pages in several cases. Nonetheless, since in these cases the absence of the feature does not represent an error, missing and suitable home pages were both assigned the score 0. Thus, for the purposes of this Master thesis, they are both considered suitable. I refer to the tables showing the percentages of error distribution in these cases too. In all the other cases (1.1, 1.2, 1.3, 2.5, 5.1, 5.2.1 and 5.2.2), I refer to the percentages shown in the charts of this chapter (Section 4.2.1), as the number of home pages corresponds to the number of occurrences. Since missing features can be either suitable or unsuitable depending on the guideline, home pages as well can be either suitable (score 0) or unsuitable (score 1). The tables below aim at clarifying this aspect:  Percentages referring to suitable home pages are coloured green (score 0).  Percentages referring to unsuitable home pages are coloured red (score 1).  Percentages referring to home pages with missing features are coloured red when they are unsuitable (score 1), i.e. the lack of that feature represents an error. On the other hand, they are coloured green when they are suitable (score 0). For what concerns H2, I take into account the number of errors referring to the Italian and English samples. As explained in Chapter 3 (Section 3.4), I do not evaluate the impact nor the severity of the errors. I only compare the number of errors to verify which sample presents the highest number of errors. 140 4.3.1 Group 1. Aspects related to source code Index Guideline Suitable (Score 0) Unsuitable (Score 1) Missing (Score 1) 1.1 Unicode codification Tot. 82% Tot. 18% / IT 82% IT 18% EN 82% EN 18% EFIT 9 EFEN 9 1.2 Language url and Tot. 66% Tot. 34% / content language IT 45% IT 55% EN 86% EN 14% EFIT 28 EFEN 7 1.3 Declaration of con- Tot. 68.5% Tot. 10% Tot. 21.5% tent language IT 68.5% IT 10% IT 21.5% EN 68.5% EN 10% EN 21.5% EFIT 16 EFEN 16 1.4 Change of content Tot. 15.5% Tot. 84.5% / language IT 13.7% IT 86.3% EN 17.6% EN 82.4% EFIT 335 EFEN 449 1.5 Text expansion Text contraction in English 70.6% Text expansion in English 25.5% No changes 3.9% 1.6 Language change in Tot. 45% Tot. 55% / links IT 69% IT 31% EN 21.5% EN 78.5% EFIT 76 EFEN 295 Table 56. Summary table. Group 1. Results concerning aspects related to source code. 141 4.3.2 Group 2. Aspects related to localisation Index Guideline Suitable (score 0) Unsuitable (score 1) Missing (score 0) 2.1 Date formats Tot. 57% Tot. 24% Tot. 19% IT 74% IT 10% IT 16% EN 39% EN 39% EN 22% EFIT 49 EFEN 105 2.2 Time formats Tot. 41% Tot. 10% Tot. 49% IT 45% IT 6% IT 49% EN 35% EN 16% EN 49% EFIT 10 EFEN 20 2.3 Number formats Tot. 12% Tot. 7% Tot. 81% IT 82% IT 0% IT 18% EN 6% EN 14% EN 80% EFIT 0 EFEN 23 2.4 Contacts session Tot. 43% Tot. 28.5% Tot. 28.5% IT 72.5% IT 0% IT 27.5% EN 14% EN 57% EN 29% EFIT 0 EFEN 66 2.5 Content update Tot. / Tot. / / IT 100% IT 0% EN 51% EN 49% EFIT 0 EFEN 25 Table 57. Summary table. Group 2. Results concerning aspects related to localisation. 142 4.3.3 Group 4. Aspects affecting search in the web site Index Guideline Suitable (score 0) Unsuitable (score 1) Missing (score 0) 4.1 Search in all linguistic Tot. 13% Tot. 37% Tot. 50% versions IT 14% IT 37% IT 49% EN 12% EN 37% EN 51% EFIT 19 EFEN 19 4.2 Advanced settings for Tot. 6% Tot. 44% Tot. 50% search in a language IT 6% IT 45% IT 49% EN 6% EN 43% EN 51% EFIT 23 EFEN 22 4.3 Language of informa- Tot. 1% Tot. 0% Tot. 99% tive texts in results IT 0% IT 0% IT 0% page EN 2% EN 0% EN 98% EFIT 0 EFEN 0 Table 58. Summary table. Group 4. Results concerning aspects related to search in the web site. 143 4.3.4 Group 5. Aspects affecting the web sites ranking in search engines Index Guideline Suitable (score 0) Unsuitable (score 1) Missing (score 1) 5.1 Translation of the Tot. 63% Tot. 37% / <title> element IT 94% IT 6% content EN 31% EN 69% EFIT 3 EFEN 35 5.2.1 Translation of the Tot. 34% Tot. 14% Tot. 52% description attrib- IT 43% IT 0% IT 57% ute content EN 25.5% EN 23.5% EN 51% EFIT 27 EFEN 48 5.2.2 Translation of the Tot. 21% Tot. 13% Tot. 67% keywords attribute IT 29% IT 4% IT 67% content EN 12% EN 22% EN 67% EFIT 36 EFEN 45 5.3.1 Translation of the Tot. 7% Tot. 93% / alt atrtibute content IT 8% IT 92% EN 6% EN 94% EFIT 664 EFEN 728 5.3.2 Translation of the Tot. 69% Tot. 31% / title attribute con- IT 76% IT 24% tent in images EN 61% EN 39% EFIT 24 EFEN 62 5.4 Translation of the Tot. 53% Tot. 47% / title attribute con- IT 59% IT 41% tent in links EN 47% EN 53% EFIT 81 EFEN 168 Table 59. Summary table. Group 5. Results concerning aspects related to web sites ranking in search engines. 144 Summary The chapter describes how I analysed the data gathered and presents the results obtained. First, I explain the process of data collection, the sources used, and the underlying logic behind the sample selection. Applying the technique of purposive sampling, I selected a sample of 102 home pages, (51 IT, 51 EN) from the urls indicated on the web-based resource “Abbonamento Musei Lombardia Milano”. Second, by inspecting both the HTML document and the home page on the browser, I verified the presence of elements and attributes in the source code (Group 1, Group 5), and evaluated aspects related to localisation (Group 2) and search in the web site (Group 4). Finally, I present results recurring to charts and tables, and describe the sample composition, i.e. the distribution of certain shared characteristics found in the data sample. This chapter also focuses on error distribution, distinguishing between the number of errors found in the two linguistic versions. For each set of guidelines, I computed error frequency in the total sample and in the two sub-samples (IT-EN), distinguishing between suitable (score 0) and unsuitable (score 1) home pages. Results concerning the percentage of suitable and unsuitable home pages and the error frequency are finally summarised in tables. These results are used to confirm or reject the hypotheses and answer the research questions in the following chapter. 145 Chapter 5: Discussion of the Results The following chapter tries to chart the results to answer the research questions and hypotheses of my study. After testing the first hypothesis (5.1), I discuss the results obtained reflecting on the consequences that the non-compliance with guidelines have on the sample in terms of internationalisation (5.1.1), localisation (5.1.2), search in the web site (5.1.3), SEO and web accessibility (5.1.4), and comment on the most critical (unfollowed) guidelines (5.1.5). After testing the second hypothesis (5.2), I compare the errors found in the Italian and English home pages for each set of guidelines (5.2.1, 5.2.2, 5.2.3, 5.2.4). I ultimately provide a list of usability problems ranked by error frequency (5.2.5). After the discussion, the last section of the chapter offers an insight on statistical testing (5.3). More specifically, I apply the chi- square goodness of fit test (5.3.1) and Spearman’s rank correlation coefficient (5.3.2). 5.1 First Hypothesis and Research Question: Compliance with Guide- lines In Chapter 3 (Section 3.1) I presented the hypotheses and research questions. The first research question concerns the total sample: do multilingual home pages in my sample respect usability guidelines? From this research question, I made some assumptions: I expected my sample of home pages to respect usability guidelines in 51% of cases. I chose this percentage because I thought that if at least 51% of the home pages respect usability guidelines, it means than more than half of the sample is consistent to guidelines. If the sample respects each usability guideline in more than 51% of pages, then results are better than expected and are considered positive. Otherwise, less than 51% of pages means that results are considered negative. H1: The sample of home pages respects guidelines in at least 51% of cases (H1 = μ ≥ 51%). The table below shows in which cases H1 is confirmed. Percentage of home pages Guideline Hypothesis confirmation respecting the guideline 1.1 Unicode ✔ 82% 1.2 Url and content language ✔ 66% 1.3 Content language declara- ✔ 68.5% tion 1.4 Change of content language ✕ 15.5% 1.5 Text expansion ✔ - 1.6 Language change in links ✕ 45% 146 2.1 Date format ✔ 76%37 2.2 Time format ✔ 90% 2.3 Number format ✔ 93% 2.4 Contacts section ✔ 71.5% 2.5 Content update ✔ 51% (EN) 4.1 Search in all linguistic ver- ✔ 63% sions 4.2 Advanced settings for ✔ 56% search 4.3 Informative texts ✔ 100% 5.1 Title element ✔ 63% 5.2.1 Description attribute ✕ 34% 5.2.2 Keywords attribute ✕ 21% 5.3.1 Alt attribute ✕ 7% 5.3.2 Title attribute in images ✔ 69% 5.4 Title attribute in links ✔ 53% Table 60. H1 confirmation. Out of 20 guidelines, in 15 cases (75%) H1 is confirmed, which means that in 75% of cases at least 51% of home pages respect the guideline. In other terms, most guidelines are respected in over half of the sample. In 5 cases (25%) H1 is rejected, which means that five usability guidelines are followed only by half or less than half of home pages. In other terms, in 25% of cases, less than 51% of home pages respect the guidelines. Results are summarised in the following chart (Chart 20). To generalise the results obtained, we could state that most usability guidelines (75% of guidelines) are respected in over half of the home pages analysed. However, for guidelines in Group 2 (2.1, 2.2, 2.3 and 2.4) and Group 4 (4.1, 4.2 and 4.3), I could not actually test the guidelines in the whole sample, as the features that need to be tested were missing. Further, results vary across the two sub-samples, i.e. Italian and English home pages respect the guidelines in different proportions as discussed in the previous chapter (Section 4.2). 37 For what concerns guidelines of Group 2 (2.1, 2.2, 2.3 and 2.4) and Group 4 (4.1, 4.2 and 4.3), the percentage of home pages respecting the guideline was computed adding up the number of suitable home pages (score 0) and the number of home pages with missing features not representing an error (score 0). They are both coloured green in the tables that summarise the results. For example, 76% is given by the sum of 57% (suitable pages, score 0) and 19% (pages with missing features not representing an error, score 0). See Table 57 and 58. 147 H125%75%Confirmed RejectedChart 20. H1 confirmation. To answer the first research question in general terms (do multilingual home pages in my sample respect usability guidelines?), results demonstrate that in most cases (75%), multilingual home pages respect usability guidelines in over half of the sample. These results are positively higher than expected. In the following sections I present and discuss the results obtained for each group of guidelines. 5.1.1 Group 1. Aspects related to source code This category concerns some elements in the HTML code, whose suitability ensures meeting the needs of internationalisation, i.e. the needs of many people around the world using different languages and writing systems. In fact, content developers and web masters must create web pages prepared to deal with linguistic and cultural issues. As discussed in Chapter 2 (Section 2.2.1), internationalisation covers aspects related to the preparation of the source content for subsequent localisation. Internationalisation implies avoiding some problems during the localisation process. The aspects analysed in this category ensure that technology supports text in any writing system of the world (Guideline 1.1 on Unicode codification) and recognises the default language of the text in the page (Guideline 1.3 on Content language declaration). When the page contains content in another language, the language attribute to the element surrounding that content enables the machine to recognise the language (Guideline 1.4 Change of content language and Guideline 1.6 Language change in links). Further, the words used a part of a web address should be consistent with the content language (Guideline 1.2 Consistency between language url and content language). To conclude, as text in different languages require different amounts of space, the web page should be prepared to face these differences, avoiding overlays or cuts of text (Guideline 1.5 Text expansions/contraction). The chart below summarises the results obtained for this category. 148 Group 1 100%80%60%40%20%0% Tot. IT EN Tot. IT EN Tot. IT EN Tot. IT EN Tot. IT EN Unicode Language url and Declaration of Change of content Language codification content language content language language change in linksSuitable Unsuitable MissingChart 21. Group 1. Results. The figures show that the sample does not present serious problems for what concerns Unicode codification (82%). The url language and content language is generally consistent (66%), and content language is declared in most cases (68.5%). Aspects related to source code are generally fine. However, guidelines referring to language change in links and textual content are more problematic. This might be attributable to localisers, who might not have taken language change in consideration. These guideline are in fact less automatic and require a lot of time and painstaking work. However, error fixing can be even more costly and time consuming, which is why these errors may not be fixed at all. Consequences may not seem very serious compared to the costs, but the impact on users is nonetheless negative. This last statement should be verified by a test on users. However, my analysis evaluated the suitability of home pages to multilingual and multicultural audiences, and figures show that the home pages analysed are mostly unsuitable for what concerns language change both in links and textual content. For what concerns language change in textual content, only a few home pages have the language change specified in the source code (15.5%). Because of the great number of errors found and my time constraints, I could not categorise them as I did for the other guidelines. Nonetheless, one of the recurring characteristic found in the Italian home pages is the use of anglicisms. An Anglicism is “a word or idiom that is recognisably English in its form (spelling, pronunciation, morphology, or at least one of the three), but is accepted as an item in the vocabulary of the receptor language (Görlach 2003 in Furiassi, 2010, p. 34). Generally speaking, considering the “unprecedent influx of English loan-words in contemporary Italian and its subsequent influence on Italian vocabulary” (Rando, 1970, p. 129), I am not surprised by the amount of anglicisms found in the Italian sample. The terms “home page”, “news” and “newsletter” are among the most frequent. The word “news” is widely used in Italian and it has become part of Italian 149 language since 1989 (Devoto G., Oli G.C., 2006). The Italian dictionary Devoto-Oli (2007) defines news as “notizie di cine- o telegiornale”, i.e. broadcasting news (Ibid.). Nonetheless, the word is widely used in web language as well, where anglicisms are also frequent. Word such as “news”, “newsletter” and “store” have displayed the Italian “notizie”, “bollettino” and “negozio” on the web. The word “home page” does not even have an equivalent in Italian”. And what’s even more astonishing is that it would seem odd to find Italian words in such context. However, given that a word’s objective is to convey a meaning, the English variants are not always able to reach the entire population. Because this Master thesis is not the place for an in-depth terminological discussion on anglicisms in Italian web language, I would like to remark that, generally speaking, English words with no language specification are considered an error. Even if these words are part of Italian language, the screen reader needs to know how to pronounce the word, therefore the language should be specified. On this purpose, we could go a little further and propose even another assumption. We could also argue that anglicisms may not need a language specification in the source code, as these words are accepted as part of the language in their form and pronunciation. We could assume that when the screen reader translates these words by voice, it could probably be even better if the English language is not signalled, as the screen reader could pronounce these words following the rules of Italian phonology. An Italian pronunciation could actually be clearer for screen reader users using the web site in Italian. However, these are just theoretical assumptions, as these statements should be verified by a test on real users and should be based on supplementary research about the technology used for speech recognition. Since my analysis is mostly based on the written text, English words with no language specification are considered an error for the purposes of this analysis. However, as afore-mentioned, a test on users could provide more reliable results in practical terms. In this sense, my analysis relies on theoretical principles and it is based on the inspection of written text. Further research on anglicisms and false anglicisms in Italian could also shed light on this guideline compliance. For what concerns English home pages, many Italian words were found. These words are either italianisms used to describe Italian art and architecture in English or untranslated words from the source home page. The most frequent case is that of untranslated names of museums. As discussed in Chapter 4 (Section 4.2.1, Guideline 5.1), it could be a foreignisation strategy of translation or an omitted translation. In any case, since the language change is not indicated, for the purposes of this study, these cases have been considered as errors. For what concerns text expansion and contraction, I could not determine how many home pages respect the guideline. In fact, the list proposed by Andreu-Vall and Marcos does not really propose a method to check the guideline. I could only measure in how many cases text expansion or contraction occurred. As expected, English text contracts in most cases (70.5%). 150 5.1.2 Group 2. Aspects related to localisation This category concerns aspects related to localisation. As discussed in Chapter 3 (Section 3.3) these guidelines aim at checking the adaptation of different elements of the web page to suit the target locale standards. More specifically date, time and number format (Guidelines 2.1, 2.2, 2.3), and contacts section (Guideline 2.4) are checked to verify if they are adapted to target users. Further, web content should be up to date in both linguistic versions (Guideline 2.5). The chart below summarises the results obtained for this category. Group 2 100%80%60%40%20%0% Tot. IT EN Tot. IT EN Tot. IT EN Tot. IT EN Tot. IT EN Date format Time format Number format Contacts section Content updateSuitable Unsuitable MissingChart 22. Group 2. Results. The figures show that the sample does not present frequent errors for what concerns date formats (57%). When date formats appear in the Italian pages, they respect the stylistic guidelines considered for this study. While date formats appearing in the English pages are suitable or unsuitable to equal proportions (39%). For what concerns time formats, they are rarely indicated in the home page. When they appear, they are generally suitable. However, since time expressions are missing in almost half of the sample (49%), this guideline could be actually be checked in 51% of the home pages. More specifically, the hypothesis is confirmed for the purposes of this Master thesis, as home pages with score 0 are considered suitable. But, it should be pointed out that this does not mean that at least half of home pages has suitable time formats, as only 51% of pages could actually be tested. However, since the fact of not indicating time in the home pages is not an error, these pages are considered suitable as they do not violate the guideline. Numbers as well are rare to find in the sample, as most home pages (81%) do not present number formats. However, in this case, they are more frequent in the Italian version, since they are not localised in English, causing a lack of information for users. As for the previous guideline, only a small percentage of home pages (19%) could actually be tested. The contact section in these home pages is either missing or suitable in Italian, while is mostly unsuitable in English, as the contact information is 151 either partially translated or omitted in English. According to our scoring criteria, 90% of home pages has suitable contact information. But, 28.5% of home pages could not be tested. For what concerns content update, we can state that information is not equally distributed. Web sites are often subject to changes both from a content and design perspective. As explained by Esselink (2000, p. 37), “[w]eb pages are often generated dynamically using templates, scripts, and databases”. Hence, maintaining web sites in several languages can be a tedious process (Ibid.). Nonetheless, ideally updates on multilingual web sites should be published simultaneously in all the linguistic versions. In our sample, almost half of English home pages present not up to date or missing content (49%). On the other hand, slightly more than half of the sample present up to date information (51%). Results borne out the expectations, however there is still much room for improvement, as multilingual audience using the web site in English still does not have access to a large part of home page content. In conclusion, for what concerns the total sample, 5 out of 5 guidelines are considered to be respected for the purposes of this Master thesis. However, as already mentioned, I could not evaluate the whole sample of home pages, as in many cases the features were missing. Further, when comparing the Italian and English versions, localisation aspects might have been understated by web masters and localisers, who probably did not completely ensure that their versions would be as truthful to the original as possible. As a result, localised versions may not be familiar and enjoyable for foreign users. This assumption should be verified by a test on users. In this sense, home pages could be improved and errors could be fixed to better fit the local market standards. Localisation is certainly constrained by limited time, human and economic resources. Hence, the degree of localisation depends on the importance or size of the local market (Jiménez-Crespo, 2013, p. 33). I do not have information about the resources spent on localisation of Lombard museums web sites. How- ever, comparing the two versions and taking into account the incomes resulting from museum tourism, I think that there is still work to be done to improve the localisation of these home pages. 5.1.3 Group 4. Aspects affecting search in the web site This category concerns aspects related to search in the web site. Results demonstrate that 50% of the home pages does not have a search feature. This means that half of the sample has actually been excluded from the analysis. The following percentages refer to those home pages that do have search capabilities. In 13% of the home pages, the search feature works in all linguistic versions, so users can type a query in either English or Italian and access the results page in that language. In 6% of home pages, the search feature has advanced settings for search in a language, so users can choose in which language they want results to be displayed. In 1% of cases, the language of informative text matches the home page language content. Results concerning the search feature are low, mainly because of the lack of this feature in the home page. Nonetheless, in all the three cases (Guideline 4.1, Guideline 4.2, Guideline 4.3), H1 is confirmed because not having the search function does not mean that the guideline was not followed. More 152 details can be found in the previous chapter (Section 4.2.3). However, it would be rash to state that half or over half of the sample respects these three guidelines without adding some considerations. In fact, half of the sample does not have a search feature that can be tested (Guidelines 4.1 and 4.2) and only 1 suitable informative text was retrieved (Guideline 4.3). In this case, only 1% of the sample actually follows the guideline. Since the lack of informative texts does not represent an error, the guideline is considered to be followed in 100% of cases. We can conclude that our sample of home pages is not ready to be fully tested and provide information in different languages. This does not apply to the web sites, as they could present the search feature in a different page, whose testing could provide different results. This aspect could be analysed in future studies focusing on search in the web site. Group 4 100%80%60%40%20%0% Tot. IT EN Tot. IT EN Tot. IT EN Search in all linguistic Advanced options Informative texts versionsSuitable Unsuitable MissingChart 23. Group 4. Results. When focusing on unsuitable cases, these guidelines violation may cause frustration and great dissatisfaction to users expecting to access results page in both Italian and English. These considerations obviously apply to the home page, which is the focus of my analysis. If the search has its own web page, it is not an error. However, “two things users often look for first on a home page are the logo and the search feature. People need to know where they are and they often want to search” (Tahir and Nielsen, 2002, p. 41). The same holds true for advanced settings for search in a language. Nielsen and Tahir argue that “usability is almost always best with simple search because scoped search offers too many opportunities for users to make errors and search the wrong scope” (Ibid., p. 42). However, users should be given the possibility to choose in which language they want to see the results, otherwise they might not be able to read it. They specify that “whether advanced search is necessary depends on the complexity of the site, […]. People who enter advanced search do so at their own risk” (Ibid.). The test carried out, which is described Section 3.3, Guideline 4.1, has been created by the evaluator to determine and measure the search functionality of these home pages. The results obtained 153 were then reviewed to check the home pages compliance with guidelines. However, a test with different queries could have produced different results. 5.1.4 Group 5. Aspects affecting the web sites ranking in search engines This category concerns aspects related to web sites ranking in search engines. Search-friendly web sites receive more attention and benefit both visitors and businesses (Thurow and Musica, 2009, p. 13). If the aspects analysed in this category are suitable, they help make web pages more “search-engine-friendly”. The translation of the web page title, description and keywords provide more effective and accessible information scents for both visitors and search engines (Guidelines 5.1, 5.2.1 and 5.2.3). The translation of images alt-text is beneficial for screen reader users (Guideline 3.1), while images and links title attributes provide additional details to mouse users when hovering the mouse on the image or the link (Guide- line 3.2 and 3.4). The chart below summarises the results obtained for this category. Group 5 100%80%60%40%20%0% Tot.IT EN Tot.IT EN Tot. IT EN Tot. IT EN Tot. IT EN Tot. IT EN Title Description Keywords Images alt Images title Links title translation attribute attribute attribute attribute attribute translation translationSuitable Unsuitable MissingChart 24. Group 5. Results. The figures show that the title element is suitable in 63% of home pages, results referring to the Italian pages are nonetheless higher than those referring to the English ones. Title elements in Italian home pages are almost 100% suitable, while in English they do not even reach 40%. The great disparity is mainly due to the fact that Italian titles are presumably mostly written by Italian native speakers. For what concerns title elements in English home pages, as discussed in Chapter 4 (Section 4.2.1, Guideline 5.1), it could be either a foreignisation strategy of translation or an omitted translation. In any case, for the purposes of this Master thesis, considering the role of English as a lingua franca, when the title is not translated it is viewed as an error. The description and keywords attributes are suitable in only 34% and 21% of cases, without great differences across the samples. These aspects, as aforementioned, can 154 negatively affect the “search-engine friendliness” of the home pages analysed, since in most cases description and keywords attribute are missing. In the sample analysed, search usability is probably ignored or minimised, which can result in substantial costs in terms of customer satisfaction, lost sales and advertising expenses. In fact, “if people find it difficult to locale or discover their desired content”, they will most likely feel like they are wasting their time and abandon the web site (Thurow and Musica, 2009, p. 13). The lowest result concerns web accessibility, only 7% of home pages have suitable alternative text for images. The incompliance with this guideline has a negative impact on screen reader users. Results concerning title attributes in images and links are positively higher: 69% of home pages have suitable title attribute to images and 55% of home pages have suitable title attribute to links. The great difference between the results is explained by the way missing attributes are treated: if the alt attribute is missing, it is an error; but if the title attribute is missing, it cannot be considered an error. 5.1.5 Guidelines Ranking To conclude, if we were to answer the first research question (do multilingual home pages in the sample respect usability guidelines?), we could state that over half of Lombard museums home pages, in the sample analysed, respect usability guidelines in 75% of cases. This result is positively higher than expected, as I assumed that guidelines were respected in at least 51% of pages. My assumptions were related to the compliance with each specific guideline separately, as well as the entire checklist of guidelines. Results are summarised in the following chart, which shows the percentages of suitable home pages from the lowest to highest. Bars in light green are higher that 51%, which means that over the majority of home pages respect that specific guideline according to our scoring criteria. Bars in light orange are less than 51%, which means that less than the majority of home pages respect that specific guideline according to our scoring criteria. The colour of the label indicates the group of guidelines—Group 1 in blue, Group 2 in green, Group 4 in lilac and Group 5 in pink. 155 Scale of most respected guidelines5.3.1 Alt attribute 7% 1.4 Change of content language 15.5% 5.2.2 Keywords attribute 21% 5.2.1 Description attribute 34% 1.6 Language change in links 45% 2.5 Content update 51% 5.4 Title attribute in links 53% 4.2 Advanced settings for search 56% 4.1 Search in all linguistic versions 63% 5.1 Title element 63% 1.2 Url and content language 66% 1.3 Content language declaration 68.5% 5.3.2 Title attribute in images 69% 2.4 Contacts section 71.5% 2.1 Date format 76% 1.1 Unicode 82% 2.2 Time format 90% 2.3 Number format 93% 4.3 Informative texts 100% 0% 20% 40% 60% 80% 100% 120%Chart 25. Scale of most respected guidelines. Home pages are suitable for what concerns localisation aspects (date, time and number format and contacts section) and search in the web site, even if these guidelines could not be measured in the whole data-set. Guidelines concerning Unicode codification, title attribute in images, content language declaration, consistency between url and content language, title attribute of HTML document, title attribute in links and content update are respected to different extents. Guidelines concerning the source code are mostly respected (Guideline 1.1 on Unicode, Guideline 1.3 on Content language declaration and Guideline 1.2 on url and content language). These guidelines are basic and probably the easiest to follow. However, they also have a remarkable impact on the home page. In fact, if, for example, language declaration is missing or unsuitable in the HTML document (Guideline 1.3) the screen reader does not recognise anything and might not process the text. Otherwise, if the Unicode codification is not used, the home page could never recognise any possible future text in different writing systems, nor any possible future localisation of the home page into other languages, like Chinese or Arabic. Unicode codification is one of the pillars of web internationalisation. Its success at unifying character sets has led to its wide- spread and predominant use in the localisation industry. Guidelines concerning web site raking are scat- tered across the scale. Some aspects, like title attribute in images, are generally respected, while others, like alt-text, are not followed and may require intervention. Further, these aspects are categorised in one guideline in the Andreu-Vall and Marcos’ checklist (2012). I do not see how these aspects could be evaluated together, since the results found are so different. 156 At the bottom of the scale, we can find guidelines concerning the alt-text (5.3.1), change of content language (1.4) and keywords attribute (5.2.2). The guideline concerning alt-text is the most critical, as only 7% of home pages respect the guideline. The alt attribute is unsuitable (17%) or missing (27%) in 44% of cases. This means that most attributes are suitable (56%). Nonetheless, when counting the errors (1392 in total), we become aware of the fact that, even if the over majority of images has a suitable alt attribute, the errors found are frequent. When focusing on the error distribution, we could estimate that only 7% of home pages is free from errors. This is a clear example of guideline where the number of errors does not correspond to the number of home pages. In fact, in one home page, there can be nu- merous images and alt attributes. In this case, the high error frequency is mostly due to the number of occurrences. This makes the comparison among guidelines difficult, as they refer to different occurrences (in some guidelines error frequency can vary from 0 to 1, while in others from 0 to ∞). Nonetheless, alt- text seems to be the most minimised aspect in the set of guidelines with negative consequences for screen reader users. Another critical guideline is the one concerning the change of content language. The violation of this guideline can negatively affect screen reader users, as well. While the error fixing for these guidelines can be costly and time-consuming, the error fixing of description and keywords attributes may be less challenging (only one occurrence per page), and help users to discover and locate content. Guidelines referring to description and keywords attributes are not followed in the sample analysed, as missing and unsuitable features prevail over suitable features. 5.2 Second Hypothesis and Research Question: Comparison between Error Frequencies The second research question concerns the number of errors, i.e. the comparison of error frequency across the two linguistic versions. In the previous chapter, I showed the number of errors found for each guideline and classified them according to error frequency per page. In this section, I compare IT and EN error frequencies. I expected the error frequency in the Italian home pages to be lower than error frequency in the English home pages (H2). The reason behind this assumption is that home pages refer to Italian museums, whose web content is most likely written by Italian native speakers. From my first general inspection of the museums of home pages on the browser, I could notice a general lack of care in the English home pages, if comparing for example localisation aspects in the two versions. From these considerations, I could formulate the second hypothesis tested in this Master thesis: H2: Error frequency in the IT pages is lower than error frequency in the EN pages (EFIT < EFEN). This hypothesis implies that I expected localisation to have a negative impact on usability. When a product is modified, the error risk inevitably increases. Nonetheless, users should be able to complete tasks 157 and access information despite their language of navigation. The second research question of my study tries to investigate whether English-speaking users are presented a product with more frequent limitations in terms of usability. The actual severity of these problems and whether they actually have a negative effect on users can only be tested by addressing users. The second research question is formulated as follows: is there any difference in the number of errors (error frequency) across the two versions? Is error frequency in the Italian pages lower than in the English ones? Does localisation have a negative impact on usability? The table below shows in which cases H2 is confirmed. Error Error Guideline Frequency Comparison Frequency H2 confirmation IT EN 1.1 Unicode 9 < 9 = 1.2 Url and content language 28 < 7 ✕ 1.3 Content language declaration 16 < 16 = 1.4 Change of content language 335 < 449 ✔ 1.6 Language change in links 76 < 295 ✔ 2.1 Date formats 49 < 105 ✔ 2.2 Time formats 10 < 20 ✔ 2.3 Number formats 0 < 23 ✔ 2.4 Contacts section 0 < 66 ✔ 2.5 Content update 0 < 25 ✔ 4.1 Search in all linguistic versions 19 < 19 = 4.2 Advanced settings for search 23 < 22 ✕ 4.3 Informative texts 0 < 0 = 5.1 Title attribute 3 < 35 ✔ 5.2.1 Description attribute 27 < 48 ✔ 5.2.2 Keywords attribute 36 < 45 ✔ 5.3.1 Alt attribute 664 < 728 ✔ 5.3.2 Title attribute in images 24 < 62 ✔ 5.4 Title attribute in links 81 < 168 ✔ Table 61. H2 confirmation. 158 H2 is confirmed in 13 out 19 guidelines, which means that generally speaking, Italian home pages obtained positively higher results than English home pages. In other terms, in 68% of cases, Italian pages have less errors than English pages. This also means that in most cases localisation has a negative impact on usability. In 32% of cases H2 is rejected — in 21% of cases, Italian and English home pages have the same number of errors, i.e., localisation has no impact on usability. In the remaining 11%, English pages have less errors than Italian pages, i.e., localisation has a positive impact on the usability of the home pages analysed. Results are summarised in the following chart. H221%11% 68%Confirmed (IT < EN) Rejected (IT > EN) Rejected (IT = EN)Chart 26. H2 confirmation. The reasons behind the differences in the two samples composition are already discussed in Chapter 4, when comparing Italian and English results. Therefore, I here provide a summary of the comparison between error frequencies for each guideline. 5.2.1 Group 1. Aspects related to source code In the guidelines concerning the source code, H2 is generally rejected. In two cases, Italian and English home pages have the same number of errors. In Guideline 1.1 (Unicode codification), as already discussed in Chapter 4 (Section 4.2.1), UTF-8 encoding allows to display languages such as Chinese, Arabic, Eng- lish, Russian and Japanese in one web page, therefore it can easily process both Italian and English text. Windows-1252 as well can display both Italian and English characters. There was therefore no need to use a different encoding since only Latin characters are used. In Guideline 1.3 (Declaration of content language) the sample composition is symmetrical. As discussed in Chapter 4 (Section 4.2.1), when the language attribute is missing in the IT home page, it is also missing in the EN home page, while when the locale is specified in IT, it is also specified in EN. Further, in one case, English home pages have less errors than Italian home pages. In Guideline 1.2 (Consistency between url and content language), Italian home pages have surprisingly more errors than English home pages. The reason might be because these home pages are most likely localised from Italian 159 into English. Therefore, I assume that the language code for English appears in the url because it is considered the foreign version. In two cases, H2 is confirmed, as Italian home pages have less errors than English home pages. In the case of Guideline 1.6 (Language change in links), this could be due to the fact that, surprisingly, there are more links in the English home pages, hence more probabilities to find errors. In the case of Guideline 1.4, this may be due to the fact that these home pages are most likely localised from Italian into English. Therefore, while Italian web content is most likely written by Italian native speakers, we have no details about localisers of English home pages. They might not be English native speakers. Another hypothesis is that, due to financial reasons, localisation is partial, hence some parts remain untranslated and language change is not signalled. This aspect could be in-depth analysed in future studies by analysing the localisation process of these home pages and contacting the localisers. 5.2.2 Group 2. Aspects related to localisation In the guidelines concerning localisation aspects, H2 is generally confirmed. In all the guidelines, Italian home pages have less frequent errors than English home pages. This result is not unexpected. In fact, localisation aspects were the first ones to be noticed when firstly inspecting the home pages. From these first general observations, I formulated the second hypothesis. For what concerns date format, time format, number format, contacts section and content update, Italian home pages obtained positively higher results. Again, the reason behind these findings is that they are most likely written by Italian native speakers, while we do not know anything about localisers of English pages. It is interesting to note that localisation has a negative impact on the usability of the home pages analysed. As afore-mentioned, the extent to which users are actually affected by problems should be verified by a test on users. Nonetheless, the differences in terms of error frequency show that aspects related to localisation could be revised and improved. The central aim of further localisation is to adapt these home pages to the target language/locale at a greater level. 5.2.3 Group 4. Aspects affecting search in the web site In the guidelines concerning search in the web site, H2 is generally rejected. In 2 out of 3 guidelines, Italian and English home pages have the same number of errors. In Guideline 4.1 (Search in all linguistic versions), as discussed in Chapter 4 (Section 4.2.3), the majority of home pages in both samples have unsuitable or missing search feature. In Guideline 4.3 (Language of informative texts in results page), no error was found, mainly because only one suitable document was retrieved. In Guideline 4.2 (Advanced settings for search in a language), Italian home pages present more errors than the English ones. The difference is very subtle (69 errors in IT vs. 66 errors in EN). In Guideline 4.2, as discussed in the previous chapter (Section 4.2.3), the majority of home pages in both sample have unsuitable or missing settings in 160 the search feature. Actually, half of the home pages in both samples does not have a search feature at all. As already mentioned, the search feature should be visible at the top of the home page (Tahir and Nielsen, 2002, p. 41). Nonetheless, the missing features are not regarded as errors because the search could be dedicated its own separate page. Despite Tahir’s and Nielsen recommendation, I was not asked to check if the search feature was present in the home page, but to evaluate if queries give results in all linguistic versions. In this sense, the guideline explained by Andreu-Vall and Marcos (2012, p. 259) could be implemented by specifying how to deal with cases when the search feature is absent. The analysis carried out in Chapter 4 allowed me not only to shed light on the characteristics of the sample chosen, but also to create a discussion about what should be considered suitable or unsuitable. On this purpose, the extensive research on usability (and localisation in general) helped me to clarify my doubts and choose a criterion. The criterion chosen is based on different recommendations, standards and guidelines, quoted throughout the text. However, since the decisions are made by one evaluator alone, they could be discussed. For example, when alt-text is missing, localisation experts agree that it is an error. In the case of the search feature, it is debatable. To conclude, for what concerns Group 4, suitable features prevail over unsuitable features, but guidelines could not be checked in the whole data-set. 5.2.4 Group 5. Aspects affecting the web sites ranking in search engines In the guidelines concerning web sites ranking in search engines, H2 is generally confirmed. In all the six guidelines, English home pages have more errors than Italian home pages. Nonetheless, this category counts the greatest amount of errors in both linguistic versions, affecting both web sites ranking and web accessibility. As discussed in Section 5.1.4, if people find it difficult to locate or discover their desired content, including both mouse and screen reader users, they will most likely abandon the web site. The highest amount of errors was found when checking the alt-text in images. For example, in the EN sample, I found 1358 images and 1017 alt attributes. In the IT sample, I found almost 2000 images (1792) and 1234 alt attributes. As afore-mentioned, the high error frequency depends on the high number of occurrences that need to be evaluated to check the guideline. To conclude, to answer my second research question (is there any difference in the number of errors across the two versions? Is error frequency in the Italian pages lower than in the English ones?), I can state that according to the data analysed there is a difference between error frequencies in the two sub-samples. More specifically, after comparing the results, I can state that Italian home pages present less errors than English home pages, in 68% of cases. These findings suggest that localisation negatively affect the usability of the home pages analysed in most cases. 161 5.2.5 A List of Usability Problems The output of heuristic evaluation is to produce a list of usability problems. The list below represents an attempt to rate the errors encountered according to their frequency. Since the actual impact of the guideline non-compliance should be verified by a test on users, the severity of the problems encountered was not calculated. The list below shows the most frequent errors. IT EN ERROR ERROR GUIDELINE GUIDELINE FREQUENCY FREQUENCY 5.3.1 664 5.3.1 728 1.4 335 1.4 449 5.4 81 1.6 295 1.6 76 5.4 168 2.1 49 2.1 105 5.2.2 36 2.4 66 1.2 28 5.3.2 62 5.2.1 27 5.2.1 48 5.3.2 24 5.2.2 45 4.2 23 5.1 35 4.1 19 2.5 25 1.3 16 2.3 23 2.2 10 4.2 22 1.1 9 2.2 20 5.1 3 4.1 19 2.3 0 1.3 16 2.4 0 1.1 9 2.5 0 1.2 7 4.3 0 4.3 0 Table 62. A list of usability problems. The heuristic evaluation carried out in this Master thesis allowed me to identify a considerable number of usability problems in the multilingual home pages inspected. I examined each home page judging its compliance with recognised usability principles. Nonetheless, the results obtained should be compared to the reports of other evaluators to avoid subjectivity. Despite the difficulties and the limitations due to a single evaluation, the present study may provide a loophole for other types of interface testing. 5.3 Statistical significance After discussing the results found, I applied two statistical tests to see if the results obtained in terms of suitable/unsuitable home pages (H1) and error frequencies (H2) are statistically significant. In other words, after discussing the results, I would like to prove that they did not occur by chance. As afore- mentioned, my research study started with the formulation of two hypotheses: H1: The sample of home pages respects guidelines in at least 51% of cases (H1 = μ ≥ 51%). 162 H2: The error frequency of Italian home pages is lower than the error frequency of English home pages (EFIT < EFEN). The first hypothesis is tested using the Chi-Square Goodness of Fit test (Section 5.4.1). I chose to recur to this test because the experimental variables are nominal (suitability/unsuitability), and because this test allows me to compare my expectations in terms of suitable/unsuitable home pages to the actual results obtained from the analysis. The second hypothesis is tested using Spearman’s Rank Correlation Coef- ficient (Section 5.4.2) because the experimental variables are ordinal. Further, Spearman’s coefficient allows me to verify if there is a correlation between error frequencies in the IT and EN sample. 5.3.1 Chi-Square Goodness of Fit Test Chi-Square Goodness of Fit test is used to determine whether sample data are consistent with a hypoth- esised data distribution, where Oi is the observed distribution (Obs.), and Ei is the expected distribution (Exp.). In other words, this test looks at whether two distributions (Ei and Oi) are associated to a significant level—“it’s unlikely an apparent relationship is down to chance” (Oates, 2006, p. 259). Equation: χ2 = Σ [(Oi−Ei)2 / Ei] In this research, a chi-square (χ2) test is carried out to verify if the results obtained from first hypothesis testing (Table 61. H1 confirmation) occurred by chance. A result is statistically significant if it is very unlikely to have occurred by chance or, in other words, given to the null hypothesis. For this test, we consider H1 = there is a difference between the distributions, and H01 = there is no difference between the distributions. In fact, the null hypothesis implies that the observations appear to be due to chance (Gonick, 2016, p. 140). I assumed that the sample of multilingual home pages respects guidelines in at least 51% of cases, however results do not always match my expectations. This test allows me to determine whether the findings concerning each guideline are the result of a genuine difference between the expected and observed distributions, or whether they are just due to chance. Hence, the chi-square test allows me to see if results are statistically significant and to confirm or reject the hypothesis that there is a difference between the observed and expected distribution. The chi-square test “performs a set of calculations using the difference between the expected values […] and the actual values” (Oates, 2006, p. 260). Thanks to this test, we can find out “whether we can have confidence that the observed relationship in the sample is not down to chance” (Ibid.). For example, for what concerns guideline 1 (Unicode codification), results show that 82% of home pages respect the guideline, while 18% do not respect the guideline. The test is applied only to results referring to the total sample, because H2 is dedicated to the comparison EN-IT. 163 The observed distribution (Obs.) and the expected distribution (Exp.) are summarised in the table below, where “yes” is used for suitable home pages (score 0), while “no” for unsuitable home pages (score 1). Yes 51 No 49 Exp. Yes 82 No 18 Obs. 2 χ = 38.4553 Table 63. Chi-square goodness of fit test. By applying the formula χ2 = Σ [(Oi−Ei)2 / Ei], I obtained the chi-squared value of 38.4553. χ2 = Σ [(Oi−Ei)2 / Ei] = [(82-51)2/51+(18-49)2/49] = 38.4553 This value must then be compared to a critical value. Critical values are shown in the table38 below: Figure 25. Percentage points of the chi-square distribution. “Tables of critical values for chi-squared typically show degrees of freedom against probability” (Oates, 2006, p. 261). Degrees of freedom are given by k-1, where k is the number of categories (yes, no), which in our case is 2. Considering that 2 - 1 is equal to 1, we focus on the first row. Looking down the column for 1 degree of freedom, we see that for significance at 0.05 level, chi-squared would have to be at least 3.84. In our case, 38.45 > 3.84. Hence, we can assert that the probability that our results are down to chance is less than 5 in 100 (p<0.05), in other words there is less than 5% of probability that results occurred by chance. We can conclude that our results are statistically significant. The test is repeated for every guideline. Results are shown in the table below. 38 The table is a screenshot from passel.unl.edu/pages/informationmodule.php?idinformationmodule=1130447119&top- icorder=8&maxto=16&minto=1 (last accessed on 30 July 2018). 164 Sample Yes No Chi-Square Comparison p<0.05 Statistical Significance 1.1 Total Yes No Obs 82 18 38.4553 > 3.84 YES Exp 51 49 1.2 Total Yes No Obs 66 34 9.00360 > 3.84 YES Exp 51 49 1.3 Total Yes No Obs 68.5 31.5 12.2549 > 3.84 YES Exp 51 49 1.4 Total Yes No Obs 15.5 84.5 50.4301 > 3.84 YES Exp 51 49 1.6 Total Yes No Obs 45 55 1.44057 > 3.84 NO Exp 51 49 Table 64. Chi-square test. Group 1. To summarise, a chi-square test of goodness-of-fit was performed to determine whether my sample of multilingual home pages respect internationalisation guidelines. For what concerns guideline 1.1, considering that χ2 (1, N = 102) = 38.45, p < 0.05: the difference between the observed and expected distributions is not due to chance. There is less than 5% of probability that results are down to chance. Actually, if we look at the table, p is less than 0.01, and even less than 0.001 (Carr, 2013), which means that the difference is very statistically significant. Hence, H0 (there is no difference between the distributions) is rejected, and H1 is confirmed (there is a difference between the distributions). However, this does not imply that the initial hypothesis is confirmed. I assumed that 51% of home pages respect the guideline, instead 88% of home pages respect it. This is a very good result, since Unicode codification is the first step for web site internationalisation and localisation. However, since its importance in the global web, results are not unexpected. H1 confirmation for each guideline has been already discussed in Section 5.1 together with the first research question. 165 For what concerns guideline 1.2, a chi-square test of goodness-of-fit was performed to examine the relation between suitable and unsuitable home pages. The relation between these variables is significant, χ2 (1, N = 102) = 9.00, p > 0.05. Again, there is less than 5% of probability that results are down to chance. For what concerns guideline 1.3, the same test was performed to examine the relation between suitable and unsuitable homepages. Results are statistically significant, χ2 (1, N = 102) = 12.25, p < 0.05. The same holds true for guideline 1.4, χ2 (1, N = 102) = 50.43, p < 0.05. For what concerns guideline 1.6, results are not statistically significant, χ2 (1, N = 102) = 1.44, p < 0.05. There is great probability (95%) that this result is down to chance. Sample Yes No Chi-Square Comparison p<0.05 Statistical Significance 2.1 Total Yes No Obs 76 24 25.0100 > 3.84 YES Exp 51 49 2.2 Total Yes No Obs 90 10 60.8643 > 3.84 YES Exp 51 49 2.3 Total Yes No Obs 93 7 70.5882 > 3.84 YES Exp 51 49 2.4 Total Yes No Obs 71.5 28.5 16.8167 > 3.84 YES Exp 51 49 2.5 Total Yes No Obs 51 49 0 > 3.84 NO Exp 51 49 Table 65. Chi-square test. Group 2. For what concerns guidelines of group 2, the same test was performed. For what concerns guideline 2.1, 2.2, 2.3 and 2.4, results are statistically significant, χ2 (1, N = 102) = 25.01, p < 0.05; χ2 (1, N = 102) = 60.86, p < 0.05; χ2 (1, N = 102) = 70.58, p < 0.05; χ2 (1, N = 102) = 16.81, p < 0.05. For what concerns 166 guideline 2.5, results are not statistically significant, as the observed and expected distributions are the same, χ2 (1, N = 102) = 0, p < 0.05. There is great probability (95%) that this result is down to chance. Sample Yes No Chi-Square Comparison p<0.05 Statistical Significance 4.1 Total Yes No Obs 63 37 5.76230 > 3.84 YES Exp 51 49 4.2 Total Yes No Obs 56 44 1.00040 > 3.84 NO Exp 51 49 4.3 Total Yes No Obs 100 0 96.0784 > 3.84 YES Exp 51 49 Table 66. Chi-square test. Group 4. For what concerns guideline 4.1 and 4.3, results are statistically significant, χ2 (1, N = 102) = 5.76, p < 0.05; χ2 (1, N = 102) = 96.07, p < 0.05. For what concerns guideline 4.2, results are not statistically significant, χ2 (1, N = 102) = 1.00, p < 0.05. There is great probability (95%) that this result is down to chance. Sample Yes No Chi-Square Comparison p<0.05 Statistical Significance 5.1 Total Yes No Obs 63 37 5.7623 > 3.84 YES Exp 51 49 5.2.1 Total Yes No Obs 37 63 7.84313 > 3.84 YES Exp 51 49 5.2.2 Total Yes No Obs 27 73 23.0492 > 3.84 YES 167 Sample Yes No Chi-Square Comparison p<0.05 Statistical Significance Exp 51 49 5.3.1 Total Yes No Obs 7 93 81.0324 > 3.84 YES Exp 51 49 5.3.2 Total Yes No Obs 69 31 12.9651 > 3.84 YES Exp 51 49 5.4 Total Yes No Obs 53 47 0.16006 > 3.84 NO Exp 51 49 Table 67. Chi-square test. Group 5. For what concerns guidelines of group 5, in guideline 5.4, considering that χ2 (1, N = 102) = 0.16, p < 0.05, results cannot be considered statistically significant. As previously mentioned for guideline 1.6, 2.5 and 4.2, there is great probability (95%) that these results are down to chance. This is due however to the fact that the test measures the relationship between observed and expected frequencies. If these frequencies happen to be similar or equal, the test assumes that there is high probability that results occurred by chance. However, this does not mean that they are not reliable: If the agreement between the observed (Oi) and expected frequencies (Ei) is close, then the differences will be small and consequently χ2 will be small. Otherwise, it will be large. The larger the value of χ2 the more likely is that the observed frequencies did not come from the population under H0. This means that the test is always right-sided. It can be shown that for large samples the sampling distribution of χ2 under H0 follows chi-square distribution with (k − 1) d. f. The approximation holds good if every Ei ≥5. In case there are some Ei>5, we have to combine adjacent classes till the expected frequency in the combined class is at least 5. Then k will be the actual number of classes used in computing χ2 (Sahu, Pal and Das, 2015, p. 147) For what concerns guideline 5.1, 5.2.1, 5.2.2, 5.3.1 and 5.3.2, considering that χ2 (1, N = 102) = 5.76, p < 0.05; χ2 (1, N = 102) = 7.84, p < 0.05; χ2 (1, N = 102) = 23.04, p < 0.05; χ2 (1, N = 102) = 81.03, p < 0.05; χ2 (1, N = 102) = 12.96, p < 0.05; results are statistically significant. 168 Out of 19 guidelines, results are statistically significant in 15 cases. We can then conclude that we can trust the results obtained. In four cases, they may not be 100% reliable, but generally speaking, we can state that there is much less than 5% of probability that results occurred by chance. 5.3.2 Spearman’s Rank Correlation Coefficient After analysing the observed and expected distribution of suitable and unsuitable home pages, the following test focuses on the error frequencies in the IT and EN samples. I would like to measure the correlation between IT and EN rankings of error frequency, i.e. I would like to understand whether there is an association between error frequencies in the two samples. Using Spearman’s rank correlation coefficient, I calculated the correlation between error frequencies (IT-EN). I calculated the deviation, which is given by the difference between EN ranking values and IT ranking values. I calculated the square of the deviation values. Then, I calculated the sum of squared deviation values. Using Spearman’s rank correlation efficient, I calculated the ρ value (rho), n=19 using the formula ρ = 1 – 6 x Σd2 / [n (n2 – 1)]: ρ = 1 – 6 x 289.96 / [19 (361 – 1)] = 0.7456 To evaluate the result obtained, we need to compare it with the critical value (n = 19, p < 0.005). Degrees of freedom are given by n-2 = 17, as show in the table39 below: Figure 26. The Spearman rank correlation coefficient. 39 The table is a screenshot from www.york.ac.uk/depts/maths/tables/spearman.pdf (last accessed on 30 July 2018). 169 Results of the Spearman correlation indicate that there is a significant positive association between the IT and EN error frequencies, (ρ (19) = 0.66, p < 0.005). In fact, Spearman’s rho (ρ) is greater than the critical value (0.61), hence the correlation is positive. The ρ value is a number from 0 to 1. On a scale from 0 to 100, error frequencies are correlated in a measure of 74.56%. As shown in Table 63, the rating scales of error frequencies are quite similar. Thanks to this test, we can state that when we rank the number of errors from the higher to the lower, the correlation between error frequencies in the two samples is 74.6%. This also means that the probability that this result occurred by chance is less than 0.5%. Summary This chapter charts the results obtained from the analysis and comments on the impact and limitations related to these results. Results concerning the compliance with guidelines of the total sample show that the over majority of home pages (51% or higher) follow most of the usability guidelines (75%) checked for the purposes of this Master thesis. Results are positively higher than expected, even if there are differences between the two linguistic versions. Results concerning the comparison between error frequencies in the two sub-samples IT-EN show that English home pages present more errors in 68% of cases, highlighting the negative impact of localisation on the usability of English home pages. This chapter also provides a list of usability problems, in which error frequencies are ranked in a scale from the higher to the lower. At the end of the chapter, I apply two statistical tests. Thanks to Spearman’s rank correlation coefficient, error frequencies are proven to be correlated in a measure of 74.56%, with the probability that this result occurred by chance being less than 0.5%. The chi-square goodness-of-fit test proves that most results (expressed as percentage) concerning suitable and unsuitable home pages are statistically significant (p < 0.05), which means that there is less than 5% of probability that results are down to chance. 170 Chapter 6: Conclusions In this chapter, after summarising the experimental set-up put in place to verity the hypotheses (6.1), I reflect on the impact (6.2), and limitations (6.3) of my work. I finally present an insight on eventual future studies (6.4). 6.1 Summary of the Experimental Set-up My work aims to evaluate a sample of multilingual home pages (IT-EN), to determine whether they respond to the demands of international audiences. More specifically, I measured whether they comply with guidelines concerning aspects related to internationalisation (Group 1), localisation (Group 2), search (Group 4) and web site ranking in the search engines (Group 5). In order to determine the suitability of these home pages, I based my research on two hypotheses (H1, H2), and by testing these hypotheses, I could assess the suitability of the sample for the twenty guidelines taken into account and measure if localisation has a negative impact on usability. The tool that allowed me not only to test the hypotheses, but also to shed light on the sample composition, is the checklist of guidelines proposed by Andreu-Vall and Marcos (2012). Considering that I analysed only the linguistic pair IT-EN, I focused on groups 1, 2, 4 and 5, checking only those guidelines that were deemed to be relevant for the case. Results show that both hypotheses are confirmed. In other terms, the over majority of home pages (51% or higher) follow most of the usability guidelines (75%) checked, with overall EN home pages presenting more errors than IT home pages. In the first case, results are positively higher than expected, even if for some guidelines of Group 2 (2.1, 2.2, 2.3 and 2.4) and all guidelines of Group 4, I could not evaluate the whole data-set. In the second case, results borne out my initial assumptions. I expected the EN home pages to present more errors than the IT ones, mainly because web sites refer to Italian museums, i.e. they are most likely to be localised from Italian into English. Results show that localisation has a negative impact for what concerns the guidelines analysed in 68% of cases. Finally, to prove that results did not occur by chance, I applied the chi-square goodness of fit test and Spearman’s rank correlation coefficient. The chi-square test proved that results are mostly statistical significant. Hence, the results concerning the distribution between suitable and unsuitable home pages are not down to chance in 15 of the guidelines checked. On the other hand, Spearman’s rank correlation coefficient proved that there is a positive correlation of 74.56% between error frequencies. Hence, error frequencies in the two samples (IT-EN) did not occur by chance, but there is a positive correlation between them. 171 6.2 Impact “Italian museums and archaeological sites have been experiencing renewed vitality for the last couple of years. Former minister of Cultural Heritage and Activities and Tourism, Dario Franceschini revealed “the country’s latest museum attendance figures, which surpassed its goal of 50 million visitors in 2017, with admissions fees ringing in over 200 million euro” (La Stampa, 2018). According to the Ministry’s statistics (MiBACT - Ministry of Cultural Heritage and Activities and Tourism) on the state of national museums, the five most visited cultural heritage sites are the Colosseum (over 7 million visitors), Pompeii (3.4 million), Uffizi (2.2 million), Florence Academy (1.6 million) and Castel Sant’Angelo (1.1 million), while the three most popular regions were Lazio, Campania and Tuscany (Ibid.). Tourism is a great source of income for Italy, even if it is discussed whether tourism is exploited to its full potential and in the right way. Professors and historians from the major Italian academies signed a petition denouncing Franceschini’s reform (2016), claiming that cultural treasures are often poorly run and badly promoted. Nonetheless, in the last four years (2013-2017), Italian earnings from state museums and archeological sites have been increasing by 53% (MiBACT, 2018). If we focus on the statistics concerning Lombardy in 2015, we notice that “The Last Supper” in Milan was the museum with the greatest income: over 2.2 million euros. The museum registering the lowest income was “Palazzo Besta of Teglia” with about 5.2 thousand euros. Results are shown in the chart below (Statista, 2018). It is worth mentioning that Palazzo Ducale Museum, Pinacoteca di Brera, Grottoes of Catullus, Castello Scaligero, Ar- cheological Park, Villa Romana and Antiquarium are included in “Abbonamento musei” and in the sample of home pages analysed. Figure 27. Income in euros of museums managed by MiBACT in the Italian region of Lombardy in 2015 (Statista, 2018), accessed on 16 July 2018. 172 Within two years (2016-2017), the number of visitors in Lombard museums has increased by 1.1%, from 1,830,495 to 1,850,605 visitors (MiBACT, 2018), making Lombardy the fifth most visited region in 2017 for what concerns cultural and heritage tourism. These figures reflect the importance that museums have in the Italian and Lombard economy, despite critics on the administration of museums and archeological sites. Thus, I think that the results obtained from my analysis are important. Results concerning the suitability of home pages are generally positive, but the usability problems identified in this function test show that there is room for improvement for what concerns the aspects studied. The home page is the like a magazine cover: it defines by example the content, style and so forth inside the web site (Tahir and Nielsen, 2002, p. 2). Considering the reputation and notoriety of some museums (Teatro alla Scala, Duomo di Milano, Pinacoteca di Brera, just to mention some), the home page is their “face to the world”, in Nielsen and Tahir’s words (Ibid.). Given the importance today of web and technologies, these home pages truly give an image of the museum to potential customers from all over the world. It is therefore important to invest in both the visual and interaction design of the home page and the entire web site, because “on the web you get only one chance to make a good impression” (Ibid.). Nonetheless, some could disagree with my opinion, claiming that some museums are so well known that they will receive visitors despite the impression gave from the web site. It is true that some touristic attractions such as the “Duomo di Milano” do not need advertise. On the contrary, they may need and want to reduce overtourism in the high season. Nonetheless, I do not mean to claim that there is a link between a successful web site and the number of tourists. I only report what web design experts say about the importance of investing on the web site. Localisation, internationalisation and usability standards that have been analysed in this Master thesis may not have any connection to the number of tickets sold to enter the museum. Nonetheless, they can be used to evaluate the home page performance and compliance with major standards. In fact, these guidelines are based on guidelines and recommendations taken from the literature (Nielsen; Iler, 2006; Singh and Pereira, 2012, Thurow and Musica, 2009 and others), as well as W3C and ISO standards. I think that it is in the interests of both customers and service providers to receive/present a digital product meeting the standards required. Lombard museums may have to deal with more relevant problems concerning facilities, prices and employees. Nonetheless, I think it is important for the museum to be introduced to the world from a web site/home page that respects international standards. Again, some museums do not need any introduction. But, for smaller museums, a well-designed and performing web site can have a positive impact on visitors, perhaps leading to the curiosity to visit it personally. Certainly, a usable, accessible and localised web site can increase the number of visits and the online visibility of the web site. After evaluating the results, I can generally state that over the half of home pages of “Abbonamento musei” in Lombardy respect most of the guidelines checked, concerning internationalisation, localisation, 173 search, SEO and web accessibility standards, with differences between the two linguistic versions. Fur- ther, the function usability test carried out in this Master thesis shows that localised pages present more frequent limitations. The compliance with these guidelines is important because they allow to present users usable, accessible and localised products. In fact, it should be mentioned that, according to the data published by “Travel appear”, an Italian startup working in the field of digital communication and big data analytics, Italian museums are “web friendly, but not too much” (La Stampa, 2017). Working on a MiBACT project to measure the digitalisation of 30 national museums, “Travel appear” wrote a report for these museums web sites. Results from their report show that museums are “web friendly, but with reservations”—they are active on social networks such as Twitter, Facebook and Instagram, but their web sites are not mobile- friendly (in 4 out 10 cases design is too old), user interfaces are not intuitive and technical aspects are poor (Ibid.). The Minister Dario Franceshini supported the project, stating that results show which aspects should be improved, and he would like to increase their presence on the web (Ibid.). After evaluating my results, I could be able to produce a list of the most frequent usability problems. However, for this list to be effective, the error severity and degree of priority intervention for error fixing should be determined by a test on users. The heuristic evaluation that I carried in this Master thesis shows that there is room for improvement for what concerns the aspects studied in both versions. 6.3 Limitations One of the greatest limitations of my work is determined by the heuristic evaluation method per se. In fact, heuristic evaluation does not provide recommendations to fix problems and does not test users’ real needs and expectations. Further, even if I tried to be as meticulous as possible basing the choice of whether to consider something an error or not on extensive literature and recognised standards, the work contains a subjectivity component. In fact, the output of heuristic evaluation is a list of usability errors with references to those guidelines that were violated by the design in each case in the opinion of the evaluator-s. As I carried out the analysis alone, the lack of comparisons with other evaluators’ opinions makes my work inevitably subjective. Further, I based my work on human evaluation only, thus human judgement cannot be purely objective by definition. Nonetheless, human evaluation allowed me to shed light into the sample characteristics and composition. Another important limitation dictated by the method is the lack of involvement of real users. Heuristic evaluation does not directly test their needs and expectations. As I explain in the next paragraph (7.4 Future work), this study could be enriched by a test on real users, which could allow me to verify if the problems encountered do actually have a negative impact on users. 174 After analysing the consequences of errors with a usability test, it would be interesting in the future to spend some time in error fixing to compare the old and fixed home pages, for example by measuring the degree of appreciation with some tests on users. It would also be interesting to investigate the causes of such errors. Following usability guidelines can help improve these home pages and make them successful. However, without understanding the reasons why errors occurred and how they affect users, the figures obtained from this analysis cannot tell much about how to improve the overall home page usability and the fixing cost-benefit balance. On this purpose, questionnaires to web masters and localisers could help understand why errors occurred, and if they are part of a localisation strategy. Unfortunately, I could not investigate the reasons why errors occurred. Nonetheless, I tried to provide a descriptive analysis of the sample composition basing my work on in-depth research. I also tried to assume why some features were unsuitable, but I do not provide certain answers. Many assumptions throughout this Master thesis should be verified by tests on users. The limitations and difficulties encountered are immediately discussed in the previous chapters in the sections to which they relate. For example, when checking the compliance with localisation guidelines (Group 2 – 2.1, 2.2, 2.3), I distinguish between suitable and unsuitable formats according to the stylistic rules reported in the sources consulted. Perhaps, having at disposal different stylistic manuals where more rules are discussed, I could have obtained different results. Generally speaking, results could be enriched and implemented by further analyses focusing on one single guideline or group of guidelines. Moreover, being a single evaluator, one of the greatest difficulties encountered was related to the decision on whether to consider an item suitable or unsuitable. For example, when checking guideline 5.1 (Translation of the <title> element), Italian proper names of museums have been generally considered an error in the English sample because they are not translated. This choice is debatable, but I decided to consider the omitted translation of the <title> element content an error, on the basis of some reflections on the role of English as a global language, despite the language proximity between Italian and English. I am aware that the choice to leave a museum’s name untranslated could be a foreignisation strategy. On this purpose, the opinion of other evaluators, tests on users—native and non-native speakers of English, to measure their prefer- ences and questionnaires to web masters to investigate localisation strategies could have helped my evaluation. In this sense, my work could be improved and extended by specific tests and questionnaires focusing on debatable issues. When reflecting on the challenging aspects of my work, one issue that hindered my work was the sample selection. In fact, as explained in Chapter 4 (Section 4.1), I could not access the official list of recognised museums in Lombardy. Moreover, even if I could, I should have created a list of web sites myself, as the only information I was acquainted with were lists of museums, not web sites. The choice to resort to the 175 database “Abbonamento musei” certainly simplified the sample selection because of the clear url specification and up to date information, but I still had to rank web sites in a list. Further, as explained in section 4.1.1, my final sample was not simply obtained by ranking web sites indicated in the database. In order to select a sample of multilingual home pages (IT-EN), I had to check if each one of the urls indicated was correct, functioning, and if the home page was localised. After careful consideration, I could identify museums clusters, as well as inaccuracies and repetitions concerning the urls. Even if this process slowed down the experimental set-up, I could get an idea of the sample and carry out a preliminary inspection. I could notice some characteristics of the sample that helped me define my hypotheses and research questions. All the aspects above mentioned certainly represent limitations to my research. However, the greatest limitation concerns the heuristic tool chosen—the checklist of guidelines by Andreu-Vall and Marcos (2012). As anticipated in Chapter 3 (Section 3.4), the checklist does not provide indications on how to evaluate multiple errors. Some guidelines, like Guideline 1.1 on Unicode codification or Guideline 1.3 on declaration of content language can be checked by inspecting one item in the source code in a binary or Boolean logic (0 suitable, 1 unsuitable). While others, like Guideline 5.3.1, 5.3.2 and 5.4 on alt-text and title attributes in images and links, require the analysis of more than 1000 items in one page. Hence, some guidelines can present maximum 1 error per page, while others can have countless errors per page. Given the heterogeneity (Boolean vs. non-Boolean nature) of the data analysed, the severity of the problems encountered could not be measured. To conclude, some guidelines could not be fully tested, as the feature that needed to be analysed was absent from the home page. The absence of an attribute, like the keywords or the description attribute, is an error. These home pages were considered unsuitable (score 1). However, if the search feature or a time expression cannot be found in the home page, it cannot be considered an error. Hence, these home pages were considered suitable (score 0). In some guidelines of Group 2 (2.1, 2.2, 2.3 and 2.4) and all the guidelines of Group 4 (4.1, 4.2 and 4.3), we could not evaluate the whole data-set. As a result, 7 out of 20 guidelines could not be fully tested. The heuristic evaluation carried out in this Master thesis tests guidelines concerning different aspects of usability. However, further studies focusing on a specific set of guidelines could shed light on one specific aspect of usability. 6.4 Future work My heuristic evaluation assessed the compliance with usability guidelines, thus my results could probably provide some feedback to the web designers of “Abbonamento musei” museums home pages. However, being a single evaluator at her first heuristic evaluation, I faced some difficulties during the analysis, mainly because heuristic evaluation requires knowledge and experience to apply and check the guidelines. 176 My work is mostly descriptive, but it also tried to identify usability problems found in the sample. It has a subjective component, but it is based on extensive research about recognised usability standards and recommendations. Nonetheless, my work could be implemented by aggregating the results of other evaluators and combining my evaluation with other usability testing methodologies to further examine potential issues. At present, my Master thesis shows that at least 51% of home pages of “Abbonamento Musei Lombardia Milano” respect 75% of the usability guidelines tested. English home pages present more errors, which can hinder the home page usability. The results of my analysis are true unless changes are adopted after the date of the analysis (09 May 2018) and they are proven to be statistically significant in most cases. In the previous section, we have seen that to make my work more expendable and usable, it could be implemented by a test on users to check if the problems encountered are really bothering them. We have mentioned that heuristic evaluation does not provide a systematic way to fix usability problems and it does not assess users’ experience. Therefore, through a usability test, whose goal is to improve the usability of the product tested, I could compare the problems detected by heuristic evaluation to those detected by a usability test, diagnose real problems and recommend changes to fix those problems. In fact, when we speak of usability the ultimate word usually goes to users. Heuristic evaluation provides a report on usability problems through user interface inspection, but it is the users experience that makes the difference (Thurow and Musica, 2009, p. 168). It would be also meaningful to ask users if they perceive the difference between “suitable” and “unsuitable” items and analyse their responses. It could also be interesting to ask them which aspects they perceive as problems and compare the results with the error frequency ratings presented in Chapter 5 (Section 5.1.5). Further, as explained in the previous section, surveys and questionnaires to the web masters could help improve my work by analysing the causes of errors, which could be dictated by time or budget constraints, lack of localisers, as well as deliberate choice of localisation. However, the aspect that most interests me would be to carry out other heuristic evaluations on museums home pages of other Italian regions and compare the results obtained. A similar study could determine whether results concerning Lombardy can be considered positive or negative compared to other regions. It would be interesting to analyse the results obtained in a region like Veneto—which was the most visited Italian region in 2017 (Unicredit, 2017), and receives similar funds. It would also be interesting to analyse the results of regions like Sicily or Campania, where tourism rate is high but resources are less. A similar comparative study would provide interesting results about the differences among regions, and shed light on the level of usability of a part of the country. Considering that heterogeneity in terms of visitors, funds and advertise that Italian regions receive, it would be interesting to see if such heterogeneity also hits the web. Further, even if the idea of analysing the level of usability of the entire web sites is interesting, I 177 would rather keep my focus on the home page, but compare the results of different regions. It could also be interesting to explore one usability aspect through more specific research, focusing on the compliance with one guideline or set of guidelines. On this purpose, a study on accessibility (Guideline 5.3.1) could probably make the most significant contribution. It would also be interesting to focus on the degree of localisation and content update (Guideline 2.5). However, even in this case, I would prefer to stay on the home page and compare results in terms of accessibility or content update with other museums in different regions. Another aspect that I would like to deepen is localisation (Group 2). Thanks to this work, I have come to realise that localisation aspects are not as easy to analyse as expected. Further studies on localisation aspects would certainly interest me, but I saw how difficult they are to assess. When the guideline is clear and the border between suitable-unsuitable item is neat, it is easier to provide a clear evaluation (as seen with the guidelines concerning internationalisation—Group 1). But, when speaking of localisation of date, time and number formats the border becomes more blurred. The set of stylistic rules I recurred to certainly helped me simplify the evaluation. Nonetheless, I think complementary research in this field and the judgement of users could shed light on the results obtained. It is worth saying that, if focusing on localisation only, with the help of a native speaker evaluator, I would really like to analyse home pages in another linguistic version, like Russian for example. To conclude, another aspect that would highly interest me is to analyse the home pages of other European museums. When in doubt over the suitability of an item, I liked to recur to examples taken from other European museums. For example, in Guideline 1.2 (Consistency between url language and content language), I compare my results to my expectations in terms of suitability and consistency. I provide an example from the home page of the house-museum Casa Milà in Barcelona, which can be considered an example of good practice for this guideline because each localised version has the content language specified in the url. It would be interesting to see if the same guideline obtains different or similar results when comparing Italian museums home pages to Spanish museums home pages. In another case, Guide- line 5.1 (Translation of title attribute content), the entire discussion on whether to consider untranslated titles in the English version suitable relies on an example taken from the home page of Bolshoi Theatre in Moscow. On this purpose, it would be interesting to enrich the discussion on this guideline by analysing the title attribute of museums home pages across Europe to study the most recurring tendencies and strategies of localisation. Summary This chapter ends my Master thesis with a summary of the experimental set-up and a reflection on both the impact and limitations of my work. My heuristic evaluation shows at least 51% of Italian and English 178 home pages of “Abbonamento Musei Lombardia Milano” respect 75% of the usability guidelines tested, with differences between the two linguistic versions. Considering the figures in terms of visitors and incomes, my work reflects on the importance that web usability can have in the context of cultural tourism in the region. Results from my study show that there is still work to be done to improve the usability of these home pages, especially on the English localised versions. Despite the effort of producing an objective evaluation, my work presents some limitations—such as using only one evaluator, lack of integration with usability tests on users, and heterogeneity of the data for what concerns the aspects analysed. The chapters ends with a brief review of possible future studies. My heuristic evaluation could be implemented by questionnaires to web masters and tests on users to assess the causes and consequences of errors. However, my cultural interests would rather lead me to enrich my work with comparative studies about home pages of other Italian and European museums. 179 Bibliography 22nd General Conference on Weights and Measures (2003) ‘Text of the Resolutions adopted by the 22nd General Conference on Weights and Measures, Resolution 10’. Available at: https://www.bipm.org/utils/en/pdf/Resol22CGPM-EN.pdf (Accessed: 8 February 2018). Achtert, W. S. and Gibaldi, J. (1985) The MLA Style Manual. New York: The Modern Language Associ- ation of America. Andreu-Vall, M. and Marcos, M. C. (2012a) Indicadores_heurística_multilingüismo, Google Docs. Available at: https://docs.google.com/spreadsheets/d/1rC-uyNV-ufZKmuK28MK5aaQA1- DVVM6bclArjOhN5GY/edit?authkey=CJiU880N&authkey=CJiU880N&usp=embed_facebook (Ac- cessed: 9 March 2018). Andreu-Vall, M. and Marcos, M.C. (2012) ‘Evaluación de sitios web multilingües: metodología y herramienta heurística’, El Profesional de la Información, 21(3), pp. 254–260. doi: 10.3145/epi.2012.may.05. Cadieux, P. and Esselink, B. (2004) ‘GILT: Globalization, Internationalization, Localization and Trans- lation’, Globalization Insider, XI(1.5), pp. 1–5. Canavese, P. (2017) ‘Localizzazione di contenuti web tra standardizzazione e adattamento’, International Journal of Translation (2017) 19, pp. 229–250. doi: 10.13137/2421676317361. Carr, S. M. (2013) Table D1 - Chi-square. Available at: https://www.mun.ca/biology/scarr/4250_Chi- square_critical_values.html (Accessed: 4 July 2018). Chaparro, A. (2010) ‘Translating the Untranslatable: Carroll, Carner and Alícia en Terra Catalana?’, Jour- nal of Iberian and Latin American Studies, 6(1), pp. 19–28. doi: 10.1080/13507490050020923. Devoto G., Oli G.C. (2006) ‘Il Devoto Oli. Vocabolario della lingua italiana’. Firenze: Le Monnier. Dumas, J. S. and Redish, J. C. (1999) A practical guide to usability testing. Revised. Exeter-Portland: Intel- lect. Dunne, K. J. (2015) ‘Localization’, in Chan, S. (ed.) Routledge Encyclopedia of Translation Technology. New York: Routledge, pp. 1–2, 13, 22–23, 30, 38. Esselink, B. (2000) A Practical Guide to Localization. Revised. Edited by Shiera O'Brien, Arjen-Sjoerd de Vries. Amsterdam-Philadelphia: John Benjamins. European Commission (2016a) ‘C(2016) 2711 final ANNEX 1. ANNEX to the Commission Delegated Regulation (EU) 2017/577 of 13 June 2016 supplementing Regulation (EU) No 600/2014 of the Euro- pean Parliament and of the Council on markets in financial instruments with regard to regulatory technical standards on the volume cap mechanism and the provision of information for the purposes of transparency and other calculations’. Available at: https://ec.europa.eu/transparency/regdoc/rep/3/2016/EN/3-2016-2711-EN-F1-1-ANNEX-1.PDF (Accessed: 8 February 2018). European Commission (2016b) The EU Internet Handbook. Multilingual Publishing. Available at: http://ec.europa.eu/ipg/content/multilingualism/index_en.htm (Accessed: 8 February 2018). Furiassi, C. (2010) False Anglicisms in Italian. Milano: Polimetrica. GALA (2011) What is localization? Available at: https://www.gala-global.org/industry/intro-language- industry/what-localization (Accessed: 4 February 2018). 180 Garzotto, F. (2004) MiLE (Milano Lugano Evaluation Method). A systematic approach to usability evaluation, http://hoc.elet.polimi.it/garzotto/pdf/MiLE_1_dic.pdf. Available at: http://hoc.elet.polimi.it/garzotto/pdf/MiLE_1_dic.pdf (Accessed: 19 April 2018). Gonick, L. (2016) Les statistiques en BD. Paris: Larousse. Google (2018) Use hreflang for language and regional URLs - Search Console Help. Available at: https://support.google.com/webmasters/answer/189077?hl=en (Accessed: 18 May 2018). Hewson, L. (2009) ‘Brave New Globalized World? Translation Studies and English as a Lingua Franca’, Brave New Globalized World? Translation Studies and English as a Lingua Franca. (Revue française de linguis- tique appliquée), XIV(1), pp. 109–120. Hewson, L. (2018) ‘Critique des traductions. Introduction’, Course code BTM0005: Critique des traductions. Available at: https://chamilo.unige.ch/home/main/document/show- inframes.php?cidReq=BTM0005&id_session=0&gidReq=0&id=511 (Accessed: 22 February 2018). Hillier, M. (2003) ‘The role of cultural context in multilingual website usability’, Electronic Commerce Re- search and Applications, 2(1), pp. 2–14. doi: 10.1016/S1567-4223(03)00005-X. Il Sole 24 Ore Digital Edition (2015) Italian Culture Minister Franceschini’s reform boosts directors’ managerial autonomy, Italian Culture Minister Franceschini’s reform boosts directors’ managerial autonomy. Available at: http://24o.it/KZrsCC (Accessed: 17 May 2018). Iler, H. (2006) Maximizing Visibility for Multilingual Web Sites, wintranslation.com. Available at: http://www.wintranslation.com/wp-content/uploads/localsearchwhitepaper.pdf (Accessed: 28 March 2018). Ishida, R. (2014) New Internationalization Developments at the W3C. Available at: https://www.w3.org/In- ternational/talks/1405-madrid/slides.pdf (Accessed: 9 March 2018). ISO (2010) ‘Language Code – ISO 639’. Available at: www.iso.org/iso-639-language-codes.html (Ac- cessed: 10 December 2017). ISO (2012) ISO 8601 Date and time format. Available at: https://www.iso.org/iso-8601-date-and-time- format.html (Accessed: 10 March 2018). ISO (2013) Country Codes – ISO 3166. Available at: https://www.iso.org/iso-3166-country-codes.html (Accessed: 10 December 2017). ISO (2017) ISO/IEC 10646:2017 - Universal Coded Character Set (UCS). Available at: https://www.iso.org/standard/69119.html (Accessed: 10 December 2017). ISO (2018) All about ISO. About us. Available at: https://www.iso.org/about-us.html (Accessed: 8 March 2018). Jiménez-Crespo, M. Á. (2013) Translation and web localization. London: Routledge. Kuhn (1999) International standard date and time notation. Available at: http://www.cl.cam.ac.uk/~mgk25/iso-time.html (Accessed: 11 March 2018). Kwintessential (2018) Translation Text Expansion: How it affects Design. Available at: http://www.kwintessential.co.uk/blog/translation/translation-text-expansion-how-it-affects-design/ (Accessed: 23 Febru- ary 2018). 181 La Stampa (2017) Le pagelle dei musei italiani, tra siti web mai aggiornati e personale scortese, LaStampa.it. Avai- lable at: http://www.lastampa.it/2017/06/26/tecnologia/le-pagelle-dei-musei-italiani-tra-siti-web-mai- aggiornati-e-personale-scortese-QBHnvPFxgvOiMtk8TEc5VN/pagina.html (Accessed: 7 July 2018). La Stampa (2018) Italy’s museums draw record number of visitors with over 50 million, LaStampa.it. Available at: http://www.lastampa.it/2018/01/06/esteri/italys-museums-draw-record-number-of-visitors-with- over-million-GpuhFKcuuPfPUn7hH3Bu3L/pagina.html (Accessed: 7 July 2018). Lapin, K. (2017) Heuristic evaluation - Lecture 8. Available at: http://web.vu.lt/mif/k.lapin/files/2016/04/8_Heuristical_evaluation2016a.pdf (Accessed: 19 April 2018). Lefevere, A. (2017) Translation, rewriting and the manipulation of literary fame. New York: Routledge. Lesina, R. (1986) Il nuovo manuale di stile: guida alla redazione di documenti, relazioni, articoli, manuali, tesi di lau- rea. Bologna: Zanichelli. LISA (2007) ‘Globalization Industry Primer’. Available at: https://liberalarts.utexas.edu/tlc/_files/con- ferences/heritage/lisa_globalization_primer.pdf (Accessed: 12 March 2018). Lombardia/Abbonamento Musei (2018). Available at: https://lombardia.abbonamentomusei.it (Accessed: 6 April 2018). Makki, K. S. and Leppert, G. (2006) ‘Factors of Usability Design for Multilingual and Multicultural Websites’, in 2006 IEEE International Conference on Information Reuse Integration. 2006 IEEE International Conference on Information Reuse Integration, pp. 6–10. doi: 10.1109/IRI.2006.252379. Matera, M., Rizzo, F. and Carughi, G. T. (2006) ‘Web Usability: Principles and Evaluation Methods’, in Mendes, E. and Mosley, N. (eds) Web Engineering. Berlin/Heidelberg: Springer-Verlag, pp. 143–180. doi: 10.1007/3-540-28218-1_5. MiBACT (2018) 2017. Tutti i numeri dei musei italiani. Available at: http://www.beniculturali.it/mibac/ex- port/MiBAC/sito-MiBAC/Contenuti/visualizza_asset.html_249254064.html (Accessed: 11 July 2018). Microsoft (2015) Developer Center sulla globalizzazione delle applicazioni. Verso la globalizzazione. Available at: https://msdn.microsoft.com/it-it/goglobal/bb688145 (Accessed: 19 January 2018). Microsoft (2017) Establishing Localization Guidelines - Globalization. Available at: https://docs.microsoft.com/en-us/globalization/localization/establishing-localization-guidelines (Accessed: 11 July 2018). MultilingualWeb (2014) Richard Ishida (W3C): New Internationalization Developments at the World Wide Web Consortium (MLW 7). Available at: https://www.youtube.com/watch?v=iHU3M_7U7fE (Accessed: 10 March 2018). Musei riconosciuti - Comuni Città Metropolitana Milano (2018). Available at: https://www.dati.lombardia.it/Cultura/Musei-riconosciuti-Comuni-Citt-Metropolitana-Milan/seen-dpws (Accessed: 5 April 2018). Musei riconosciuti da Regione Lombardia (2018) Open Data Regione Lombardia. Available at: https://www.dati.lombardia.it/Cultura/Musei-riconosciuti-da-Regione-Lombardia/3syc-54zf (Ac- cessed: 5 April 2018). Musei/Lombardia/Abbonamento Musei (2018) Abbonamento Musei. Available at: https://lombardia.abbonamentomusei.it/Musei (Accessed: 6 April 2018). 182 Najork, M. (2009) Web Crawler Architecture. Available at: https://www.microsoft.com/en-us/research/wp-content/uploads/2009/09/EDS-WebCrawlerArchitecture.pdf (Accessed: 29 March 2018). Nantel, J. and Glaser, E. (2008) ‘The impact of language and culture on perceived website usability’, Journal of Engineering and Technology Management, 25(1–2), pp. 112–122. doi: 10.1016/j.jengtec- man.2008.01.005. Nielsen, J. (1993) Usability Engineering. San Diego: Academic Press. Nielsen, J. (1995a) Nielsen Norman Group. 10 Usability Heuristics for User Interface Design. Available at: https://www.nngroup.com/articles/ten-usability-heuristics/ (Accessed: 25 February 2018). Nielsen, J. (1995b) Nielsen Norman Group. How to Conduct a Heuristic Evaluation. Available at: https://www.nngroup.com/articles/how-to-conduct-a-heuristic-evaluation/ (Accessed: 25 February 2018). Nielsen, J. (1995c) Severity Ratings for Usability Problems: Article by Jakob Nielsen, Nielsen Norman Group. Available at: https://www.nngroup.com/articles/how-to-rate-the-severity-of-usability-problems/ (Ac- cessed: 24 July 2018). Nielsen, J. (1995) Nielsen Norman Group. Summary of Usability Inspection Methods. Available at: https://www.nngroup.com/articles/summary-of-usability-inspection-methods/ (Accessed: 26 Febru- ary 2018). Nielsen, J. (1996a) Nielsen Norman Group. International Usability Testing. Available at: https://www.nngroup.com/articles/international-usability-testing/ (Accessed: 5 February 2018). Nielsen, J. (1996b) Nielsen Norman Group. International Web Usability. Available at: https://www.nngroup.com/articles/international-web-usability/ (Accessed: 5 February 2018). Nielsen, J. (1999) Nielsen Norman Group. Do Interface Standards Stifle Design Creativity? Available at: https://www.nngroup.com/articles/do-interface-standards-stifle-design-creativity/ (Accessed: 25 Feb- ruary 2018). Nielsen, J. (2009) Nielsen Norman Group. Discount Usability: 20 Years. Available at: https://www.nngroup.com/articles/discount-usability-20-years/ (Accessed: 5 February 2018). Nielsen, J. (2012) Nielsen Norman Group. Usability 101: Introduction to Usability. Available at: https://www.nngroup.com/articles/usability-101-introduction-to-usability/ (Accessed: 5 February 2018). Nielsen, J. (2016) Using the Title Attribute to Help Users Predict Where They Are Going, Nielsen Norman Group. Available at: https://www.nngroup.com/articles/title-attribute/ (Accessed: 17 June 2018). Nielsen, J. and Loranger, H. (2006) Prioritizing web usability. Berkeley: New Riders. Oakes, M. P. (1998) Statistics for corpus linguistics. Edinburgh: Edinburgh University Press (Edinburgh textbooks in empirical linguistics). Oates, B. J. (2006) Researching information systems and computing. London-Thousand Oaks: SAGE. Open Web Foundation Agreement, Version 0.9. (2017) Open Graph protocol. Available at: http://ogp.me/ (Accessed: 22 May 2018). 183 Pennsylvania State University (2014a) Image ALT Tag Tips for HTML, Accessibility at Penn State. Available at: http://accessibility.psu.edu/images/imageshtml/ (Accessed: 29 March 2018). Pennsylvania State University (2014b) Image ALT Text, Accessibility at Penn State. Available at: http://accessibility.psu.edu/images/alttext/ (Accessed: 29 March 2018). Petricola, P. (2011) Date [prontuario] in ‘Enciclopedia dell’Italiano’. Available at: http://www.treccani.it//enciclopedia/date-prontuario_(Enciclopedia-dell’Italiano) (Accessed: 13 March 2018). Pym, A. (2004) The moving text : localization, translation and distribution. Amsterdam-Philadelphia: John Ben- jamins. Available at: http://data.rero.ch/01-R003649543/html?view=GE_V1. Quazzico, F. (2016) Localisation et accessibilité du Web à partir de l’étude de cas de la RTS et la RSI. University of Geneva. Available at: https://archive-ouverte.unige.ch/unige:88302 (Accessed: 9 March 2018). Rando, G. (1970) ‘The Assimilation of English Loan - Words in Italian’, Italica, pp. 129–142. Ritter, R. M. (2005) New Hart’s rules: the handbook of style for writers and editors. Oxford: Oxford University Press. Available at: http://data.rero.ch/01-R004259575/html?view=GE_V1. Roturier, J. (2015) Localizing apps: a practical guide for translators and translation students. New York: Routledge. Sahu, P. K., Pal, S. R. and Das, A. K. (2015) Estimation and Inferential Statistics. New Delhi: Springer In- dia. doi: 10.1007/978-81-322-2514-0. Saldanha, G. and O’Brien, S. (2013) Research Methodologies in Translation Studies. New York: Routledge. Schäler, R. (2008) ‘Linguistic Resources and Localization’, in Yuste Rodrigo, E. (ed.) Topics in Language Resources for Translation and Localization. Amsterdam-Philadelphia: John Benjamins, p. 195. Schäler, R. (2010) ‘Localization and Translation’, in Gambier, Y. and Doorslaer, L. (eds) Handbook of Translation Studies. Amsterdam-Philadelphia: John Benjamins, p. 209. Singh, N. (2011) Localization Strategies for Global E-Business. Cambridge University Press. Singh, N. and Pereira, A. (2012) The culturally customized web site: customizing web sites for the global marketplace. London: Routledge. Statista (2018) Income of museums managed by Mibact in Lombardy 2015 - Italy, Statista. Available at: https://www.statista.com/statistics/734296/income-of-museums-managed-by-mibact-in-lombardy- italy/ (Accessed: 7 July 2018). Tahir, M. and Nielsen, J. (2002) Homepage usability: 50 websites deconstructed. Indianapolis: New Riders. TCBOK (2014) ‘Usability Theory – Technical Communication Body of Knowledge (TCBOK)’. Availa- ble at: https://www.tcbok.org/wiki/usability-theory/ (Accessed: 3 April 2018). Thurow, S. and Musica, N. (2009) When Search Meets Web Usability. Berkeley: New Riders. Unicredit (2017) Rapporto sul Turismo 2017 realizzato da UniCredit in collaborazione con il Touring Club Italiano, UnicreditGroup.eu. Available at: http://www.unicreditgroup.eu/it/press-media/press-releases/2017/rapporto-sul-turismo-2017-realizzato-da-unicredit-in-collaborazi.html (Accessed: 10 July 2018). 184 University of Oxford (2014) ‘University of Oxford Style Guide’. Available at: https://www.ox.ac.uk/sites/files/oxford/media_wysiwyg/University%20of%20Ox- ford%20Style%20Guide.pdf (Accessed: 16 March 2018). Vid, N. (2008) ‘The Challenge of Translating Children’s Literature: Alice’s Adventures in Wonderland Translated by Vladimir Nabokov’, ELOPE: English Language Overseas Perspectives and Enquiries, 5(1–2). doi: 10.4312/elope.5.1-2.217-227. W3C (2003) Script direction and languages. Available at: https://www.w3.org/International/questions/qa- scripts (Accessed: 23 March 2018). W3C (2007a) Date formats. Available at: https://www.w3.org/International/questions/qa-date-format (Accessed: 10 March 2018). W3C (2007b) Internationalization Best Practices: Specifying Language in XHTML & HTML Content. Available at: https://www.w3.org/International/geo/html-tech/tech-lang.html#ri20050128.152033553 (Ac- cessed: 10 March 2018). W3C (2007c) Text size in translation. Available at: https://www.w3.org/International/articles/article- text-size (Accessed: 9 March 2018). W3C (2008a) An Introduction to Multilingual Web Addresses. Available at: https://www.w3.org/Interna- tional/articles/idn-and-iri/ (Accessed: 9 March 2018). W3C (2008b) Internationalization techniques: Authoring HTML & CSS, W3C Internationalization (i18n) Mak- ing the World Wide Web worldwide! Available at: https://www.w3.org/International/techniques/authoring-html (Accessed: 9 March 2018). W3C (2008c) Migrating to Unicode. Available at: https://www.w3.org/International/articles/unicode-mi- gration/ (Accessed: 9 March 2018). W3C (2011) Choosing a language tag. Available at: https://www.w3.org/International/questions/qa- choosing-language-tags (Accessed: 18 May 2018). W3C (2012) 4.2.5 The meta element — HTML5. Available at: https://dev.w3.org/html5/spec-pre- view/the-meta-element.html (Accessed: 22 May 2018). W3C (2014a) Declaring language in HTML. Available at: https://www.w3.org/International/questions/qa-html-language-declarations (Accessed: 18 May 2018). W3C (2014b) Language tags in HTML and XML. Available at: https://www.w3.org/International/articles/language-tags/ (Accessed: 9 March 2018). W3C (2014c) Locale-based forms - Internationalization. Available at: https://www.w3.org/Interna- tional/wiki/Locale-based_forms (Accessed: 14 March 2018). W3C (2016a) H25: Providing a title using the title element | Techniques for WCAG 2.0. Available at: https://www.w3.org/TR/WCAG20-TECHS/H25.html (Accessed: 23 May 2018). W3C (2016b) H30: Providing link text that describes the purpose of a link for anchor elements | Techniques for WCAG 2.0. Available at: https://www.w3.org/TR/WCAG20-TECHS/H30.html#H30-disclaimer (Accessed: 17 June 2018). 185 W3C (2016c) H33: Supplementing link text with the title attribute | Techniques for WCAG 2.0. Available at: https://www.w3.org/TR/WCAG20-TECHS/H33.html (Accessed: 17 June 2018). W3C (2016d) Internationalization - W3C. Available at: https://www.w3.org/standards/webdesign/i18n (Accessed: 9 March 2018). W3C (2018a) Index of the HTML 4 Attributes. Available at: https://www.w3.org/TR/html4/index/attributes.html (Accessed: 19 May 2018). W3C (2018b) Links in HTML documents. Available at: https://www.w3.org/TR/html4/struct/links.html (Accessed: 19 May 2018). W3C - Web Accessibility Initiative (2015) Easy Checks - A First Review of Web Accessibility. Available at: https://www.w3.org/WAI/EO/Drafts/eval/checks#images (Accessed: 23 May 2018). w3schools (1999a) HTML Global class Attribute, w3schools.com. Available at: https://www.w3schools.com/tags/att_global_class.asp (Accessed: 19 May 2018). w3schools (1999b) HTML meta tag, w3schools.com. Available at: https://www.w3schools.com/tags/tag_meta.asp (Accessed: 29 March 2018). w3schools (1999c) HTML span tag, w3schools.com. Available at: https://www.w3schools.com/tags/tag_span.asp (Accessed: 19 May 2018). w3schools (1999d) HTML src Attribute, w3schools.com. Available at: https://www.w3schools.com/tags/att_src.asp (Accessed: 13 April 2018). w3schools (1999e) HTML title Attribute, w3schools.com. Available at: https://www.w3schools.com/tags/att_title.asp (Accessed: 13 April 2018). w3schools (1999f) HTML title tag, w3schools.com. Available at: https://www.w3schools.com/tags/tag_ti- tle.asp (Accessed: 28 March 2018). w3schools (1999g) HTML Windows-1252 Reference, w3schools.com. Available at: https://www.w3schools.com/charsets/ref_html_ansi.asp (Accessed: 17 May 2018). Wolf, M. and Wicksteed, C. (1997) Date and Time Formats. Available at: https://www.w3.org/TR/NOTE-datetime (Accessed: 11 March 2018). Wright, S. E. (2006) ‘The creation and application of language industry standards’, in Dunne, K. J. (ed.) Perspectives on localization. Amsterdam-Philadelphia: John Benjamins, pp. 241–245. Available at: http://data.rero.ch/01-R004170943/html?view=GE_V1. Yunker, J. (2003) Beyond borders: Web globalization strategies. Indianapolis: New Riders. Yunker, J. (2010) The Art of the Global Gateway: Strategies for Successful Multilingual Navigation. Ashland: Byte Level Research. Available at: https://books.google.it/books?id=qu6QB2tvAAQC. 186 Annexes Annex 1. First annotation. List of web sites as indicated in “Abbonamento Musei Lombardia Milano”. IN- PROV PROVINCE WEB SITE DE INCE X MI MILANO - AREA METROPOLITANA 1 1 ACQUARIO CIVICO www.acquariocivicomilano.eu 2 2 CASA MILAN - MUSEO MONDO MILAN casamilan.acmilan.com 3 3 CASA MUSEO BOSCHI DI STEFANO www.fondazioneboschidistefano.it 4 4 CIVICO MUSEO ARCHEOLOGICO www.comune.milano.it/museoarcheologico > http://www.museoarcheologi- comilano.it 5 5 CRIPTA DI SAN GIOVANNI IN CONCA not indicated 6 6 FONDAZIONE STELLINE www.stelline.it 7 7 GALLERIA D’ARTE MODERNA www.gam-milano.com 8 8 GALLERIE D’ITALIA – PIAZZA SCALA www.gallerieditalia.com 9 9 GRANDE MUSEO DEL DUOMO www.duomomilano.it 10 10 LEONARDO3 MUSEUM www.leonardo3.net 11 11 MUBA MUSEO DEI BAMBINI MILANO www.muba.it 12 12 MUSEI DEL CASTELLO SFORZESCO www.milanocastello.it 13 13 MUSEO BAGATTI VALSECCHI www.museobagattivalsecchi.com 14 14 MUSEO DEL NOVECENTO www.museodelnovecento.org 15 15 MUSEO DELLA BASILICA DI SANT'EU- www.chiostrisanteustorgio.it STORGIO 16 16 MUSEO DELLE CULTURE: COLLEZIONE www.mudec.it/ita/collezione-PERMANENTE permanente/ 17 17 MUSEO DI STORIA NATURALE www.comune.milano.it/muse- ostorianaturale 18 18 MUSEO DIOCESANO CARLO MARIA MAR- www.chiostrisanteustorgio.it TINI 19 19 MUSEO D’ARTE E SCIENZA museoartescienza.com 187 20 20 MUSEO INTERATTIVO DEL CINEMA www.cinetecamilano.it/museo 21 21 MUSEO NAZIONALE DELLA SCIENZA E www.museoscienza.org DELLA TECNOLOGIA LEONARDO DA VINCI 22 22 MUSEO POLDI PEZZOLI www.museopoldipezzoli.it 23 23 MUSEO TEATRALE ALLA SCALA www.teatroallascala.org 24 24 PALAZZO LEONE DA PEREGO cultura.legnano.org/sale/46/ 25 25 PALAZZO MORANDO | COSTUME MODA www.costumemodaimmag-E IMMAGINE ine.mi.it 26 26 PALAZZO MORIGGIA - MUSEO DEL RI- www.museodelrisorgi-SORGIMENTO mento.mi.it 27 27 PARCO DELL’ANFITEATRO ROMANO E www.parcoanfiteatro-ANTIQUARIUM “ALDA LEVI” milano.beniculturali.it 28 28 PINACOTECA AMBROSIANA www.leonardo-ambrosiana.it 29 29 PINACOTECA DI BRERA www.pinacotecabrera.org 30 30 PIRELLI HANGARBICOCCA www.hangarbicocca.org 31 31 STUDIO MUSEO FRANCESCO MESSINA www.fondazionemessina.it 32 32 TRIENNALE DI MILANO www.triennale.org 33 33 VIGNA DI LEONARDO www.vignadileonardo.com 34 34 WOW SPAZIO FUMETTO www.museowow.it BG BERGAMO 35 1 CAMPANONE www.bergamoestoria.it 36 2 CONVENTO DI SAN FRANCESCO - MU- www.bergamoestoria.it SEO STORICO – SEZIONE MOSTRE 37 3 GALLERIA DELL’ ACCADEMIA TADINI www.accademiatadini.it 38 4 GALLERIA D’ARTE MODERNA E CON- www.gamec.it TEMPORANEA – GAMEC 39 5 MUSEO ADRIANO BERNAREGGI www.fondazionebernareggi.it 40 6 MUSEO CIVICO ARCHEOLOGICO www.museoarcheologicober- gamo.it 41 7 MUSEO CIVICO DI SCIENZE NATURALI www.museoscienzebergamo.it “E. CAFFI” 42 8 MUSEO DONIZETTIANO www.bergamoestoria.it 43 9 MUSEO E TESORO DELLA CATTEDRALE www.fondazionebernareggi.it 188 44 10 ORTO BOTANICO "LORENZO ROTA" www.ortobotanicodibergamo.it 45 11 PALAZZO DEL PODESTA’ - MUSEO STO- www.palazzodelpodesta.it > RICO – ETA’ VENETA – IL ‘500 INTERAT- http://www.palazzodelpode- TIVO sta.it/museo_storico_eta_ve- neta.aspx 46 12 PINACOTECA DELL’ACCADEMIA CAR- www.lacarrara.it RARA 47 13 ROCCA - MUSEO STORICO – SEZIONE ‘800 www.bergamoestoria.it 48 14 TORRE DEI CADUTI www.bergamoestoria.it BS BRESCIA 49 1 AREA ARCHEOLOGICA DELLA VILLA www.villaromanadesen-ROMANA zano.beniculturali.it 50 2 AREA ARCHEOLOGICA DELLE GROTTE www.grottedicatullo.benicul-DI CATULLO E MUSEO ARCHEOLOGICO turali.it DI SIRMIONE 51 3 BRIXIA. PARCO ARCHEOLOGICO DI BRE- www.bresciamusei.com SCIA ROMANA 52 4 CASTELLO - RICETTO DI DESENZANO www.comune.desenzano.brescia.it/italian/castello.php 53 5 CASTELLO SCALIGERO www.architettonicibrescia.beniculturali.it 54 6 CIVICO MUSEO ARCHEOLOGIO "GIO- www.onde.net/desen-VANNI RAMBOTTI" zano/citta/museo 55 7 MUSA - MUSEO DI SALÒ www.museodisalo.it 56 8 MUSEI DEL CASTELLO – MUSEO DEL RI- www.bresciamusei.com SORGIMENTO E MUSEO DELLE ARMI “LUIGI MARZOLI” 57 9 MUSEO ARCHEOLOGICO NAZIONALE www.museoarcheologico.valca-DELLA VALLE CAMONICA monicaromana.beniculturali.it 58 10 MUSEO DI SANTA GIULIA not indicated 59 11 MUSEO NAZIONALE DELLA PREISTORIA www.parcoarcheologicocivi- DELLA VALLE CAMONICA date.valcamonicaromana.beniculturali.it 60 12 PARCO ARCHEOLOGICO DEL SANTUA- not indicated RIO DI MINERVA 189 61 13 PARCO ARCHEOLOGICO DEL TEATRO E www.parcoarcheologicocivi- DELL’ANFITEATRO date.valcamonicaromana.beniculturali.it 62 14 PARCO ARCHEOLOGICO NAZIONALE www.parcoarcheologico.massi-DEI MASSI DI CEMMO dicemmo.beniculturali.it 63 15 PARCO NAZIONALE DELLE INCISIONI www.parcoincisioni.capodi-RUPESTRI ponte.beniculturali.it 64 16 PINACOTECA TOSIO MARTINENGO www.bresciamusei.com/pinacoteca.asp 65 17 VILLA ROMANA DI TOSCOLANO MA- www.comune.toscolanoma-DERNO derno.bs.it CO COMO 66 1 MUSEO ARCHEOLOGICO “PAOLO GIO- cultura.comune.como.it > VIO” http://visitcomo.eu/it/ 67 2 MUSEO DEL CICLISMO MADONNA DEL www.museodelghisallo.it GHISALLO 68 3 MUSEO DIDATTICO DELLA SETA www.museosetacomo.com 69 4 MUSEO STORICO “GIUSEPPE GARI- cultura.comune.como.it BALDI” 70 5 PINACOTECA CIVICA DI PALAZZO VOLPI cultura.comune.como.it 71 6 TEMPIO VOLTIANO cultura.comune.como.it 72 7 VILLA CARLOTTA – MUSEO E GIARDINO www.villacarlotta.it BOTANICO CR CREMONA 73 1 MUSEO ARCHEOLOGICO “SAN LO- musei.comune.cremona.it RENZO” 74 2 MUSEO CIVICO DELLA CIVILTA' CONTA- musei.comune.cremona.it DINA "IL CAMBONINO VECCHIO" 75 3 MUSEO CIVICO DI STORIA NATURALE musei.comune.cremona.it 76 4 MUSEO CIVICO “ALA PONZONE” – PINA- musei.comune.cremona.it COTECA 77 5 MUSEO DEL BIJOU www.museodelbijou.it 78 6 MUSEO DEL VIOLINO www.museodelviolino.org 79 7 MUSEO DIOTTI www.museodiotti.it LC LECCO 190 80 1 CIVICO MUSEO DELLA SETA ABEGG www.museosetagarlate.it 81 2 PALAZZO BELGIOJOSO – MUSEO AR- www.comune.lecco.it/in-CHEOLOGICO, MUSEO STORIA NATU- dex.php/scoprire-lecco/musei RALE, MUSEO STORICO 82 3 PALAZZO DELLE PAURE – GALLERIA www.comune.lecco.it/in-COMUNALE D’ARTE: SEZIONE DI ARTE dex.php/scoprire-lecco/musei CONTEMPORANEA, GRAFICA E FOTO- GRAFIA, SPAZI ESPOSITIVI TEMPORANEI 83 4 TORRE VISCONTEA www.comune.lecco.it/index.php/scoprire-lecco/musei 84 5 VILLA MANZONI - MUSEO MANZO- www.comune.lecco.it/in-NIANO E GALLERIA COMUNALE D’ARTE dex.php/scoprire-lecco/musei 85 6 VILLA MONASTERO www.villamonastero.eu LO LODI - MN MANTOVA 86 1 CHIESA DELLA BEATA VERGINE INCO- www.iatsabbioneta.org RONATA 87 2 MUSEO ARCHEOLOGICO DELL'ALTO www.museocavriana.it MANTOVANO 88 3 MUSEO ARCHEOLOGICO NAZIONALE DI www.museoarcheologicoman-MANTOVA tova.beniculturali.it 89 4 MUSEO CIVICO DI PALAZZO TE www.palazzote.it 90 5 MUSEO DELLA CITTA’ – PALAZZO SAN www.museodellacitta.mn.it SEBASTIANO 91 6 MUSEO DIOCESANO FRANCESCO GON- www.museodiocesanoman-ZAGA tova.it 92 7 MUSEO TAZIO NUVOLARI www.tazionuvolari.it 93 8 PALAZZO DELLA RAGIONE E TORRE www.comune.mantova.gov.it DELL’OROLOGIO 94 9 PALAZZO DUCALE www.mantovaducale.beniculturali.it 95 10 PALAZZO DUCALE – SABBIONETA www.iatsabbioneta.org 96 11 PALAZZO GIARDINO E GALLERIA DE- www.iatsabbioneta.org GLI ANTICHI 97 12 SINAGOGA www.iatsabbioneta.org 191 98 13 TEATRO ALL'ANTICA www.iatsabbioneta.org 99 14 TEATRO SCIENTIFICO BIBIENA www.comune.mantova.gov.it 100 15 TEMPIO DI SAN SEBASTIANO www.cittadimantova.it > www.comune.mantova.gov.it MB MONZA-BRIANZA 101 1 CAPPELLA ESPIATORIA www.turismo.monza.it/it/cosa- fare/48-cappella-espiatoria 102 2 MUSEI CIVICI DI MONZA – CASA DEGLI www.museicivicimonza.it UMILIATI 103 3 MUST MUSEO DEL TERRITORIO VIMER- www.museomust.it CATESE 104 4 PALAZZO ARESE BORROMEO www.comune.cesano- maderno.mb.it 105 5 ROSSINI ART SITE www.rossiniartsite.com 106 6 VILLA REALE DI MONZA www.villarealedimonza.it > http://www.reggiadimonza.it/ PV PAVIA 107 1 CASTELLO DI ZAVATTARELLO E MUSEO www.zavattarello.org D'ARTE CONTEMPORANEA 108 2 MUSEI CIVICI "L. BARNI" - MUSEO INTER- www.museocalzaturavi-NAZIONALE DELLA CALZATURA PIETRO gevano.it BERTOLINI 109 3 MUSEI CIVICI DEL CASTELLO VISCON- www.museicivici.pavia.it TEO 110 4 MUSEI CIVICI “L. BARNI” - MUSEO www.comune.vige- DELL'IMPRENDITORIA VIGEVANESE vano.pv.it/contenuti/cultura/sottopagine/musei-1/musei 111 5 MUSEI CIVICI “L. BARNI” - PINACOTECA www.comune.vige- CIVICA “C. OTTONE” vano.pv.it/contenuti/cultura/sottopagine/musei-1/musei 112 6 MUSEO ARCHEOLOGICO NAZIONALE DI www.museoarcheologico.vige-VIGEVANO vano.beniculturali.it 113 7 MUSEO DELLA CERTOSA DI PAVIA www.museo.certosadipavia.beniculturali.it 192 SO SONDRIO 114 1 MUSEO VALTELLINESE DI STORIA E www.comune.sondrio.it ARTE 115 2 PALAZZO BESTA E MUSEO ARCHEOLO- www.lombardiabenicultu- GICO rali.it/architet- ture/schede/1n120-00087/ VA VARESE 116 1 AREA ARCHEOLOGICA E ANTIQUARIUM www.antiquarium.castelse-DI CASTELSEPRIO prio.beniculturali.it 117 2 CASA MUSEO LODOVICO POGLIAGHI www.casamuseopogliaghi.it 118 3 CIVICI MUSEI DI VILLA MIRABELLO E RI- www.varesecultura.it SORGIMENTO / MUSEO ARCHEOLO- GICO E SEZIONE RISORGIMENTALE 119 4 CIVICO MUSEO DI ARTE MODERNA E www.varesecultura.it CONTEMPORANEA – CASTELLO DI MA- SNAGO 120 5 MA*GA www.museomaga.it 121 6 MONASTERO DI TORBA www.visitfai.it/monasteroditorba > fai-international.org/ 122 7 MUSEO BAROFFIO E DEL SANTUARIO www.museobaroffio.it DEL SACRO MONTE SOPRA VARESE 123 8 MUSEO CASTIGLIONI www.museocastiglioni.it 124 9 MUSEO CIVICO ARCHEOLOGICO DI SE- www.comune.sesto-ca-STO CALENDE lende.va.it 125 10 MUSEO CIVICO PREISTORICO DELL’ ISO- https://www.isolinovirginia.it LINO VIRGINIA 126 11 MUSEO DELLA CERAMICA G. GIANETTI www.museogianetti.it 127 12 VILLA DELLA PORTA BOZZOLO www.visitfai.it/villadellaporta- bozzolo > fai-international.org/ 128 13 VILLA E COLLEZIONE PANZA www.visitfai.it/villapanza/ > fai-international.org/ 129 14 VOLANDIA PARCO E MUSEO DEL VOLO www.volandia.it 193 Annex 2. Second annotation. Selection of web sites based on the considerations listed in Table 12. IN- PRO INDEX MUSEUM WEB SITE LOCALI- NOTES DE VINC PER PROV- SATION X E INCE (05/04/201 8) 1 MI 1 ACQUARIO www.acquariocivi- not local-CIVICO comilano.eu ised 2 MI 2 CASA MILAN - casamilan.acmi- localised MUSEO lan.com MONDO MI- LAN 3 MI 3 CASA MUSEO www.fonda- localised BOSCHI DI zioneboschidiste-STEFANO fano.it 4 MI 4 CIVICO www.comune.mi- not local- redirec- MUSEO AR- lano.it/museoarcheo- ised tion CHEOLOGICO logico > http://www.museoar- cheologicomilano.it 5 MI 6 FONDAZIONE www.stelline.it localised STELLINE 6 MI 7 GALLERIA www.gam-mi- localised D’ARTE lano.com MODERNA 7 MI 8 GALLERIE www.galleriedita- not local-D’ITALIA – PI- lia.com ised AZZA SCALA 8 MI 9 GRANDE www.duomomilano.it localised MUSEO DEL DUOMO 9 MI 10 LEONARDO3 www.leonardo3.net localised MUSEUM 10 MI 11 MUBA MUSEO www.muba.it localised DEI BAMBINI MILANO 11 MI 12 MUSEI DEL www.milanocastello.it localised CASTELLO SFORZESCO 12 MI 13 MUSEO www.museobagatti- localised BAGATTI valsecchi.com VALSECCHI 13 MI 14 MUSEO DEL www.museodelnove- localised NOVECENTO cento.org 14 MI 15 MUSEO www.chiostrisanteus- not local-DELLA BASI- torgio.it ised LICA DI SANT'EU- STORGIO 15 MI 16 MUSEO www.mu- localised DELLE CUL- dec.it/ita/collezione- permanente/ 194 TURE: COLLE- ZIONE PER- MANENTE 16 MI 17 MUSEO DI www.comune.mi- not local- STORIA NAT- lano.it/museosto- ised URALE rianaturale 17 MI 19 MUSEO museoartesci- localised D’ARTE E SCI- enza.com ENZA 18 MI 20 MUSEO IN- www.cinetecami- not local-TERATTIVO lano.it/museo ised DEL CINEMA 19 MI 21 MUSEO NA- www.museosci- localised ZIONALE enza.org DELLA SCIENZA E DELLA TEC- NOLOGIA LEONARDO DA VINCI 20 MI 22 MUSEO POLDI www.museopoldipez- localised PEZZOLI zoli.it 21 MI 23 MUSEO www.teatroallascala.or localised TEATRALE g ALLA SCALA 22 MI 24 PALAZZO LE- cul- not local- ONE DA PER- tura.legnano.org/sale ised EGO /46/ 23 MI 25 PALAZZO MO- www.costume- not local-RANDO | CO- modaimmagine.mi.it ised STUME MODA E IMMAGINE 24 MI 26 PALAZZO MO- www.museodelrisor- not local-RIGGIA - MU- gimento.mi.it ised SEO DEL RI- SORGIMENTO 25 MI 27 PARCO www.parcoanfiteatro- localised DELL’ANFI- milano.beniculturali.it TEATRO RO- MANO E AN- TIQUARIUM “ALDA LEVI” 26 MI 28 PINACOTECA www.leonardo-am- localised AMBROSIANA brosiana.it 27 MI 29 PINACOTECA www.pina- localised DI BRERA cotecabrera.org 28 MI 30 PIRELLI www.hangarbi- localised HANGARBI- cocca.org COCCA 195 29 MI 31 STUDIO www.fondazionemes- not local-MUSEO FRAN- sina.it ised CESCO MES- SINA 30 MI 32 TRIENNALE www.triennale.org localised DI MILANO 31 MI 33 VIGNA DI LE- www.vignadileo- localised ONARDO nardo.com 32 MI 34 WOW SPAZIO www.museowow.it not local- FUMETTO ised 33 BG 1 CAMPANONE www.bergamoesto- not local- ria.it ised 34 BG 3 GALLERIA www.accademiata- localised DELL’ ACCA- dini.it DEMIA TADINI 35 BG 4 GALLERIA www.gamec.it localised new web D’ARTE MO- site un- DERNA E der con- CONTEMPO- struction RANEA – GA- MEC 36 BG 5 MUSEO ADRI- www.fondazioneber- not local-ANO BERNA- nareggi.it ised REGGI 37 BG 6 MUSEO CIV- www.museoarcheo- not local-ICO ARCHEO- logicobergamo.it ised LOGICO 38 BG 7 MUSEO CI- www.museoscien- not local-VICO DI zebergamo.it ised SCIENZE NA- TURALI “E. CAFFI” 39 BG 10 ORTO BOTAN- www.ortobo- localised ICO "LO- tanicodibergamo.it RENZO ROTA" 40 BG 11 PALAZZO DEL www.palazzodelpode- not local- Broken PODESTA’ - sta.it > www.palaz- ised link, I MUSEO STO- zodelpodesta.it/mu- retrived RICO – ETA’ seo_storico_eta_ve- the web VENETA – IL neta.aspx site ‘500 INTERAT- TIVO 41 BG 12 PINACOTECA www.lacarrara.it localised DELL’ACCA- DEMIA CAR- RARA 42 BS 1 AREA AR- www.villaromana- not local- CHEOLOGICA desenzano.benicul- ised DELLA VILLA turali.it ROMANA 196 43 BS 2 AREA AR- www.grottedica- localised CHEOLOGICA tullo.beniculturali.it DELLE GROTTE DI CATULLO E MUSEO AR- CHEOLOGICO DI SIRMIONE 44 BS 3 BRIXIA. www.bre- localised PARCO AR- sciamusei.com CHEOLOGICO DI BRESCIA ROMANA 45 BS 4 CASTELLO - www.comune.desen- not local- RICETTO DI zano.brescia.it/ital- ised DESENZANO ian/castello.php 46 BS 5 CASTELLO www.architettonici- not local-SCALIGERO brescia.beniculturali.it ised 47 BS 6 CIVICO MU- www.onde.net/desen- localised SEO ARCHEO- zano/citta/museo LOGIO "GIO- VANNI RAM- BOTTI" 48 BS 7 MUSA - www.museodisalo.it not local- MUSEO DI ised SALÒ 49 BS 8 MUSEO AR- www.museoarcheolo- localised under CHEOLOGICO gico.valcamonicaro- construc-NAZIONALE mana.beniculturali.it tion DELLA VALLE CAMONICA 50 BS 9 MUSEO NA- www.parcoarcheolo- not local- ZIONALE gicocividate.valcamo- ised DELLA PREI- nicaromana.benicultu-STORIA rali.it DELLA VALLE CAMONICA 51 BS 10 PARCO AR- www.parcoarcheolo- localised under CHEOLOGICO gico.massidi- construc-NAZIONALE cemmo.beniculturali.it tion DEI MASSI DI CEMMO 52 BS 11 PARCO NA- www.parcoinci- localised ZIONALE sioni.capodiponte.be-DELLE INCI- niculturali.it SIONI RUPE- STRI 53 BS 12 VILLA RO- www.comune.toscola- localised MANA DI TO- nomaderno.bs.it SCOLANO MA- DERNO 197 54 CO 1 MUSEO AR- cultura.co- localised redirec- CHEOLOGICO mune.como.it > visit- tion “PAOLO GIO- como.eu/it/ VIO” 55 CO 2 MUSEO DEL www.muse- not local-CICLISMO MA- odelghisallo.it ised DONNA DEL GHISALLO 56 CO 3 MUSEO DI- www.museoseta- localised DATTICO como.com DELLA SETA 57 CO 7 VILLA CAR- www.villacarlotta.it localised LOTTA – MU- SEO E GIAR- DINO BOTA- NICO 58 CR 1 MUSEO AR- musei.comune.cre- not local- CHEOLOGICO mona.it ised “SAN LO- RENZO” 59 CR 5 MUSEO DEL www.museodelbijou.it not local- BIJOU ised 60 CR 6 MUSEO DEL www.museodelvio- localised VIOLINO lino.org 61 CR 7 MUSEO www.museodiotti.it not local- DIOTTI ised 62 LC 1 CIVICO MU- www.museosetagar- localised SEO DELLA late.it SETA ABEGG 63 LC 2 PALAZZO www.co- not local- BELGIOJOSO mune.lecco.it/in- ised – MUSEO AR- dex.php/scoprire-CHEOLO- lecco/musei GICO, MUSEO STORIA NA- TURALE, MU- SEO STORICO 64 LC 6 VILLA MON- www.villamonas- localised ASTERO tero.eu 65 MN 1 CHIESA www.iatsabbion- not local-DELLA BEATA eta.org ised VERGINE IN- CORONATA 66 MN 2 MUSEO AR- www.museocav- not local-CHEOLOGICO riana.it ised DELL'ALTO MANTOVANO 67 MN 3 MUSEO AR- www.museoarcheo- localised CHEOLOGICO logicomantova.be-NAZIONALE niculturali.it DI MANTOVA 198 68 MN 4 MUSEO CI- www.palazzote.it localised VICO DI PA- LAZZO TE 69 MN 5 MUSEO www.museodel- localised DELLA CITTA’ lacitta.mn.it – PALAZZO SAN SEBA- STIANO 70 MN 6 MUSEO DIOC- www.museodioce- localised ESANO FRAN- sanomantova.it CESCO GON- ZAGA 71 MN 7 MUSEO TAZIO www.tazionuvolari.it localised NUVOLARI 72 MN 8 PALAZZO www.comune.man- not local- tourism DELLA RA- tova.gov.it ised english GIONE E version TORRE DELL’OROLO- GIO 73 MN 9 PALAZZO DU- www.mantovadu- localised CALE cale.beniculturali.it 74 MB 1 CAPPELLA ES- www.tur- localised PIATORIA ismo.monza.it/it/cos a-fare/48-cappella-espiatoria 75 MB 2 MUSEI CIVICI www.museicivici- not local-DI MONZA – monza.it ised CASA DEGLI UMILIATI 76 MB 3 MUST MUSEO www.museomust.it localised DEL TERRITO- RIO VIMERCA- TESE 77 MB 4 PALAZZO www.comune.cesano- not local-ARESE BOR- maderno.mb.it ised ROMEO 78 MB 5 ROSSINI ART www.rossiniart- localised SITE site.com 79 MB 6 VILLA REALE www.vil- localised multilin- DI MONZA larealedimonza.it > gual web www.reggiadimonza.it site chosen 80 PV 1 CASTELLO DI www.zavattarello.org localised ZAVATTA- RELLO E MU- SEO D'ARTE CONTEMPO- RANEA 81 PV 2 MUSEI CIVICI www.museocal- not local-"L. BARNI" - zaturavigevano.it ised 199 MUSEO IN- TERNAZIO- NALE DELLA CALZATURA PIETRO BER- TOLINI 82 PV 3 MUSEI CIVICI www.museicivici.pa- not local- when DEL CA- via.it ised clicking STELLO VI- on lan- SCONTEO guage selector, download the museum map in Eng 83 PV 4 MUSEI CIVICI www.comune.vige- not local- “L. BARNI” - vano.pv.it/conte- ised MUSEO nuti/cultura/sottopa-DELL'IM- gine/musei-1/musei PRENDITORIA VIGEVANESE 84 PV 6 MUSEO AR- www.museoarcheolo- not local- CHEOLOGICO gico.vigevano.benicul- ised NAZIONALE turali.it DI VIGEVANO 85 PV 7 MUSEO www.museo.certosa- not local-DELLA CER- dipavia.beniculturali.it ised TOSA DI PA- VIA 86 SO 1 MUSEO VAL- www.comune.son- not local-TELLINESE DI drio.it ised STORIA E ARTE 87 SO 2 PALAZZO BE- www.lombardiabeni- not local- STA E MUSEO culturali.it/architet- ised ARCHEOLO- ture/schede/1n120-GICO 00087/ 88 VA 1 AREA AR- www.antiquarium.ca- not local- CHEOLOGICA stelseprio.benicultu- ised E ANTIQUA- rali.it RIUM DI CA- STELSEPRIO 89 VA 2 CASA MUSEO www.casamuseopogli- not local-LODOVICO aghi.it ised POGLIAGHI 90 VA 3 CIVICI MUSEI www.varesecultura.it not local- DI VILLA MI- ised RABELLO E RISORGI- MENTO / MU-200 SEO ARCHEO- LOGICO E SE- ZIONE RISOR- GIMENTALE 91 VA 5 MA*GA www.museomaga.it localised 92 VA 6 MONASTERO www.visitfai.it/mon- localised interna- DI TORBA asteroditorba > tional http://fai-interna- web site tional.org/ considered 93 VA 7 MUSEO BA- www.museobaroffio.it not local- ROFFIO E DEL ised SANTUARIO DEL SACRO MONTE SO- PRA VARESE 94 VA 8 MUSEO www.muse- not local-CASTIGLIONI ocastiglioni.it ised 95 VA 9 MUSEO CI- www.comune.sesto- not local-VICO AR- calende.va.it ised CHEOLOGICO DI SESTO CA- LENDE 96 VA 10 MUSEO CI- www.isolinovirginia.it not local- VICO PREI- ised STORICO DELL’ ISO- LINO VIRGI- NIA 97 VA 11 MUSEO www.museogianetti.it not local- DELLA CERA- ised MICA G. GIA- NETTI 98 VA 14 VOLANDIA www.volandia.it localised PARCO E MU- SEO DEL VOLO Annex 3. Third annotation. Localised home pages. INDEX PER LOCALISA- IN- PROV- PROV- TION DEX INCE INCE MUSEUM WEB SITE (05/04/2018) CASA MILAN - MUSEO 1 MI 2 MONDO MILAN casamilan.acmilan.com localised CASA MUSEO BOSCHI DI 2 MI 3 STEFANO www.fondazioneboschidistefano.it localised 3 MI 6 FONDAZIONE STELLINE www.stelline.it localised 201 GALLERIA D’ARTE 4 MI 7 MODERNA www.gam-milano.com localised GRANDE MUSEO DEL 5 MI 9 DUOMO www.duomomilano.it localised 6 MI 10 LEONARDO3 MUSEUM www.leonardo3.net localised MUBA MUSEO DEI BAM-7 MI 11 BINI MILANO www.muba.it localised MUSEI DEL CASTELLO 8 MI 12 SFORZESCO www.milanocastello.it localised MUSEO BAGATTI VALSEC-9 MI 13 CHI www.museobagattivalsecchi.com localised 10 MI 14 MUSEO DEL NOVECENTO www.museodelnovecento.org localised MUSEO DELLE CULTURE: www.mudec.it/ita/collezione-perma-11 MI 16 COLLEZIONE PERMANENTE nente/ localised 12 MI 19 MUSEO D’ARTE E SCIENZA museoartescienza.com localised MUSEO NAZIONALE DELLA SCIENZA E DELLA TECNOLO-13 MI 21 GIA LEONARDO DA VINCI www.museoscienza.org localised 14 MI 22 MUSEO POLDI PEZZOLI www.museopoldipezzoli.it localised MUSEO TEATRALE ALLA 15 MI 23 SCALA www.teatroallascala.org localised PARCO DELL’ANFITEATRO ROMANO E ANTIQUARIUM www.parcoanfiteatromilano.benicul-16 MI 27 “ALDA LEVI” turali.it localised 17 MI 28 PINACOTECA AMBROSIANA www.leonardo-ambrosiana.it localised 18 MI 29 PINACOTECA DI BRERA www.pinacotecabrera.org localised 19 MI 30 PIRELLI HANGARBICOCCA www.hangarbicocca.org localised 20 MI 32 TRIENNALE DI MILANO www.triennale.org localised 21 MI 33 VIGNA DI LEONARDO www.vignadileonardo.com localised GALLERIA DELL’ ACCA-22 BG 3 DEMIA TADINI www.accademiatadini.it localised GALLERIA D’ARTE MO- DERNA E CONTEMPORA-23 BG 4 NEA – GAMEC www.gamec.it localised ORTO BOTANICO "LO-24 BG 10 RENZO ROTA" www.ortobotanicodibergamo.it localised PINACOTECA DELL’ACCA-25 BG 12 DEMIA CARRARA www.lacarrara.it localised AREA ARCHEOLOGICA DELLE GROTTE DI CATULLO E MUSEO ARCHEOLOGICO 26 BS 2 DI SIRMIONE www.grottedicatullo.beniculturali.it localised BRIXIA. PARCO ARCHEOLO-27 BS 3 GICO DI BRESCIA ROMANA www.bresciamusei.com localised CIVICO MUSEO ARCHEOLO- GIO "GIOVANNI RAM- www.onde.net/desenzano/citta/mu-28 BS 6 BOTTI" seo localised MUSEO ARCHEOLOGICO NAZIONALE DELLA VALLE www.museoarcheologico.valcamoni-29 BS 8 CAMONICA caromana.beniculturali.it localised 202 PARCO ARCHEOLOGICO NAZIONALE DEI MASSI DI www.parcoarcheologico.massidi- 30 BS 10 CEMMO cemmo.beniculturali.it localised PARCO NAZIONALE DELLE www.parcoincisioni.capodiponte.be-31 BS 11 INCISIONI RUPESTRI niculturali.it localised VILLA ROMANA DI TOSCO- www.comune.toscolanoma- 32 BS 12 LANO MADERNO derno.bs.it localised MUSEO ARCHEOLOGICO cultura.comune.como.it > visit- 33 CO 1 “PAOLO GIOVIO” como.eu/it/ localised MUSEO DIDATTICO DELLA 34 CO 3 SETA www.museosetacomo.com localised VILLA CARLOTTA – MUSEO 35 CO 7 E GIARDINO BOTANICO www.villacarlotta.it localised 36 CR 6 MUSEO DEL VIOLINO www.museodelviolino.org localised CIVICO MUSEO DELLA SETA 37 LC 1 ABEGG www.museosetagarlate.it localised 38 LC 6 VILLA MONASTERO www.villamonastero.eu localised MUSEO ARCHEOLOGICO www.museoarcheologicoman-39 MN 3 NAZIONALE DI MANTOVA tova.beniculturali.it localised MUSEO CIVICO DI PALAZZO 40 MN 4 TE www.palazzote.it localised MUSEO DIOCESANO FRAN- www.museodiocesanomantova.it > 41 MN 6 CESCO GONZAGA www.museofrancescogonzaga.it/ localised 42 MN 7 MUSEO TAZIO NUVOLARI www.tazionuvolari.it localised 43 MN 9 PALAZZO DUCALE www.mantovaducale.beniculturali.it localised www.turismo.monza.it/it/cosa- CAPPELLA ESPIATORIA 44 MB 1 fare/48-cappella-espiatoria localised MUST MUSEO DEL TERRI-45 MB 3 TORIO VIMERCATESE www.museomust.it localised 46 MB 5 ROSSINI ART SITE www.rossiniartsite.com localised www.villarealedimonza.it > www.reg- VILLA REALE DI MONZA 47 MB 6 giadimonza.it localised CASTELLO DI ZAVATTA- RELLO E MUSEO D'ARTE 48 PV 1 CONTEMPORANEA www.zavattarello.org localised 49 VA 5 MA*GA www.museomaga.it localised www.visitfai.it/monasteroditorba > MONASTERO DI TORBA 50 VA 6 http://fai-international.org/ localised VOLANDIA PARCO E MU-51 VA 14 SEO DEL VOLO www.volandia.it localised Annex 4. Final list of Italian and English home pages. INDEX HOME PAGE (IT-EN) 1 IT http://casamilan.acmilan.com/it 1 EN http://casamilan.acmilan.com/en 2 IT http://www.fondazioneboschidistefano.it/ws/ 203 2 EN http://www.fondazioneboschidistefano.it/ws/en/ 3 IT https://www.stelline.it/it 3 EN https://www.stelline.it/en 4 IT http://www.gam-milano.com/it/home/ 4 EN http://www.gam-milano.com/en/home/ 5 IT https://www.duomomilano.it/it/ 5 EN https://www.duomomilano.it/en/ 6 IT http://www.leonardo3.net/ 6 EN http://www.leonardo3.net/en 7 IT https://www.muba.it/ 7 EN https://www.muba.it/en 8 IT https://www.milanocastello.it/ 8 EN https://www.milanocastello.it/en 9 IT https://museobagattivalsecchi.org/ 9 EN https://museobagattivalsecchi.org/en/ 10 IT http://www.museodelnovecento.org/it/ 10 EN http://www.museodelnovecento.org/en/ 11 IT http://www.mudec.it/ita/ 11 EN http://www.mudec.it/eng/ 12 IT http://museoartescienza.com/ 12 EN http://museoartescienza.com/en/ 13 IT http://www.museoscienza.org/ 13 EN http://www.museoscienza.org/english/ 14 IT http://www.museopoldipezzoli.it/#!/it/scopri 14 EN http://www.museopoldipezzoli.it/#!/en/discover 15 IT http://www.teatroallascala.org/it/index.html 15 EN http://www.teatroallascala.org/en/index.html 16 IT http://www.parcoanfiteatromilano.beniculturali.it/ 16 EN http://www.parcoanfiteatromilano.beniculturali.it/index.php?en/1/home 17 IT http://www.leonardo-ambrosiana.it/ 204 17 EN http://www.leonardo-ambrosiana.it/en/ 18 IT http://pinacotecabrera.org/ 18 EN http://pinacotecabrera.org/en/ 19 IT http://www.hangarbicocca.org/ 19 EN http://www.hangarbicocca.org/en/ 20 IT http://www.triennale.org/ 20 EN http://www.triennale.org/en/ 21 IT https://www.vignadileonardo.com/it 21 EN https://www.vignadileonardo.com/en 22 IT http://www.accademiatadini.it/ 22 EN http://www.accademiatadini.it/index_en.aspx 23 IT http://www.gamec.it/it/frontpage 23 EN http://www.gamec.it/en/frontpage 24 IT http://www.ortobotanicodibergamo.it/ 24 EN http://www.ortobotanicodibergamo.it/en/ 25 IT http://www.lacarrara.it/ 25 EN http://www.lacarrara.it/en/ 26 IT http://www.grottedicatullo.beniculturali.it/ 26 EN http://www.grottedicatullo.beniculturali.it/index.php?en/1/home 27 IT https://www.bresciamusei.com/ 27 EN https://www.bresciamusei.com/ 28 IT http://www.onde.net/desenzano/citta/museo/index_ITA.htm http://www.onde.net/desenzano/citta/museo/refresh/ENG/INDEX/in- 28 EN dex_ENG.htm 29 IT http://www.museoarcheologico.valcamonicaromana.beniculturali.it/ http://www.museoarcheologico.valcamonicaromana.beniculturali.it/in- 29 EN dex.php?en/1/home 30 IT http://www.parcoarcheologico.massidicemmo.beniculturali.it/ http://www.parcoarcheologico.massidicemmo.beniculturali.it/in- 30 EN dex.php?en/1/home 31 IT http://www.parcoincisioni.capodiponte.beniculturali.it/ 205 http://www.parcoincisioni.capodiponte.beniculturali.it/in- 31 EN dex.php?en/1/home 32 IT http://www.comune.toscolanomaderno.bs.it/ 32 EN http://www.comune.toscolanomaderno.bs.it/en/ 33 IT http://www.visitcomo.eu/it/ 33 EN http://www.visitcomo.eu/en/index.html 34 IT http://www.museosetacomo.com/home.php 34 EN http://www.museosetacomo.com/eng_home.php 35 IT http://www.villacarlotta.it/home.php?lang_id=1 35 EN http://www.villacarlotta.it/home.php?pag_id=9&sez_id=13&lang_id=2 36 IT http://www.museodelviolino.org/it/ 36 EN www.museodelviolino.org/en/ 37 IT http://www.museosetagarlate.it/ 37 EN http://www.museosetagarlate.it/en/ 38 IT http://www.villamonastero.eu/index.php/it/ 38 EN http://www.villamonastero.eu/index.php/en/ 39 IT http://www.museoarcheologicomantova.beniculturali.it/ http://www.museoarcheologicomantova.beniculturali.it/in- 39 EN dex.php?en/1/home 40 IT http://www.palazzote.it/index.php/it/ 40 EN http://www.palazzote.it/index.php/en/ 41 IT http://www.museofrancescogonzaga.it/ 41 EN http://www.museofrancescogonzaga.it/int/english.php 42 IT http://www.tazionuvolari.it/it/ 42 EN http://www.tazionuvolari.it/en/ 43 IT http://www.mantovaducale.beniculturali.it/it/ 43 EN http://www.mantovaducale.beniculturali.it/en/informazioni/welcome 44 IT http://www.turismo.monza.it/it/cosa-fare/48-cappella-espiatoria http://www.turismo.monza.it/en/things-to-do/49-cappella-espiatoria-ex- 44 EN piatory-chapel 45 IT http://www.museomust.it/drupal/ 206 45 EN http://www.museomust.it/drupal/en 46 IT http://www.rossiniartsite.com/ 46 EN http://www.rossiniartsite.com/en/ 47 IT http://www.reggiadimonza.it/ 47 EN http://www.reggiadimonza.it/en 48 IT http://www.zavattarello.org/ 48 EN http://www.zavattarello.org/eng_index.html 49 IT http://www.museomaga.it/ 49 EN http://www.museomaga.it/en/ 50 IT http://fai-international.org/?lang=it 50 EN http://fai-international.org/ 51 IT http://volandia.it/ 51 EN http://volandia.it/en/ Annex 5. Guideline 1.4 (Change of content language). Errors per page. Index Errors Index Errors Index Errors Index Errors 1 IT 69 14 IT 0 27 IT 6 40 IT 5 1 EN 102 14 EN 0 27 EN 10 40 EN 17 2 IT 12 15 IT 5 28 IT 6 41 IT 3 2 EN 21 15 EN 5 28 EN 4 41 EN 0 3 IT 3 16 IT 1 29 IT 4 42 IT 3 3 EN 9 16 EN 11 29 EN 1 42 EN 1 4 IT 0 17 IT 0 30 IT 4 43 IT 2 4 EN 6 17 EN 4 30 EN 1 43 EN 20 5 IT 57 18 IT 7 31 IT 7 44 IT 2 5 EN 36 18 EN 3 31 EN 1 44 EN 0 6 IT 2 19 IT 11 32 IT 2 45 IT 2 6 EN 1 19 EN 0 32 EN 2 45 EN 11 7 IT 0 20 IT 12 33 IT 4 46 IT 3 7 EN 1 20 EN 12 33 EN 0 46 EN 0 8 IT 1 21 IT2 0 34 IT 4 47 IT 6 8 EN 40 21 EN 5 34 EN 1 47 EN 18 9 IT 2 22 IT 0 35 IT 4 48 IT 0 9 EN 1 22 EN 3 35 EN 2 48 EN 1 10 IT 4 23 IT 10 36 IT 5 49 IT 6 207 10 EN 6 23 EN 16 36 EN 28 49 EN 4 11 IT 8 24 IT 4 37 IT 1 50 IT 16 11 EN 0 24 EN 14 37 EN 1 50 EN 1 12 IT 1 25 IT 2 38 IT 2 51 IT 2 12 EN 0 25 EN 0 38 EN 2 51 EN 6 13 IT 18 26 IT 4 39 IT 3 13 EN 1 26 EN 2 39 EN 18 208 </div> </article> </div> </div> </div> <script type="text/javascript" async crossorigin="anonymous" src="https://pagead2.googlesyndication.com/pagead/js/adsbygoogle.js?client=ca-pub-8519364510543070"></script> <script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.6.1/jquery.min.js" crossorigin="anonymous" referrerpolicy="no-referrer"></script> <script> var docId = 'e76ff09f6914a8a8ad68edba089bcea2'; var endPage = 1; var totalPage = 210; var pfLoading = false; window.addEventListener('scroll', function () { if (pfLoading) return; var $now = $('.article-imgview .pf').eq(endPage - 1); if (document.documentElement.scrollTop + $(window).height() > $now.offset().top) { pfLoading = true; endPage++; if (endPage > totalPage) return; var imgEle = new Image(); var imgsrc = "//data.docslib.org/img/e76ff09f6914a8a8ad68edba089bcea2-" + endPage + (endPage > 3 ? ".jpg" : ".webp"); imgEle.src = imgsrc; var $imgLoad = $('<div class="pf" id="pf' + endPage + '"><img src="/loading.gif"></div>'); $('.article-imgview').append($imgLoad); imgEle.addEventListener('load', function () { $imgLoad.find('img').attr('src', imgsrc); pfLoading = false }); if (endPage < 7) { adcall('pf' + endPage); } } }, { passive: true }); </script> <script> var sc_project = 11552861; var sc_invisible = 1; var sc_security = "b956b151"; </script> <script src="https://www.statcounter.com/counter/counter.js" async></script> </html>