Usable Security and E-Banking: Ease of Use vis-à-vis Security

Data collected in an evaluation of six Danish web-based electronic banking systems (DRAFT)

Morten Hertzum, Niels Christian Juul, Niels Jørgensen, Mie Nørgaard. Roskilde University, Denmark email: {mhz,ncjuul,nielsj,mnl}@ruc.dk Table of Contents Preface...... 3 Introduction...... 4 How to read the tables...... 4 Browser messages...... 5 Heuristics...... 6 A taxonomy of ease-of-use problems in e-banking...... 7 PC and browser configuration...... 8 Installation...... 11 Danske Bank...... 11 Nordea...... 14 Jyske bank...... 17 Skandiabanken...... 20 Sydbank (certificate solution)...... 23 Sydbank (“key card” solution)...... 26 Basisbank...... 31 Summary of tables...... 34 Every day log on...... 36 Danske Bank...... 36 Nordea...... 38 Jyske bank...... 40 Skandiabanken...... 41 Sydbank...... 44 Basisbank...... 47 Log off...... 50 Danske bank...... 50 Nordea...... 52 Jyske bank...... 54 Skandiabanken...... 56 Sydbank...... 58 Basisbank...... 60 Money transfer...... 61 Danske Bank...... 61 Nordea...... 64 Jyske bank...... 66 Skandiabanken...... 68 Sydbank...... 70 Basisbank...... 72 Weak passwords...... 74 Danske Bank...... 74 Nordea...... 78 Jyske bank...... 80 Skandiabanken...... 81 Sydbank...... 83 Basisbank...... 85 References...... 86 Preface

This report contains data collected in an evaluation of six Danish web-based electronic banking systems. The report supplements an academic paper in which we interpret and discuss the evaluation results and motivate the method used to obtain them: Hertzum et al. Usable Security and E-banking: Ease of Use vis-à-vis Security. Proceedings of OzCHI 2004, Wollongong, Australia. The data was collected during June-August, 2003, and the draft version of the present report was completed November 17, 2004. Introduction

The evaluation comprised four tasks that a user of an e-banking (web-based electronic banking) system must or is like to carry out: installation, log on, log off, and money transfer. In addition we tested the systems' response to weak passwords. The remainder of this introduction explains how the read the tables in which the evaluation results are presented, and describes the hardware and software configuration of the PC used for the evaluation.

How to read the tables

The walk-throughs are documented in tables of four or five columns. The columns are as follows: 1. column shows the number of the current page to reveal if any pages are revisited. For brewity, we record the display of a window displaying a standard browser message using the abbreviation "Security alert", "Security warning", or "Security warning". These three types of messages are described below in the section Browser messages. 2. column holds the information offered by the system. Text shown in italics is the original text in Danish as offered by the system. If an English term is not followed by an italized (ie. Danish) term, the system has offered the English term. 3. column describes the correct user-action on the current page. 4. column holds the heuristic evaluation (HE) comments. The comments are based on a set of 12 heuristics presented below in the section Heuristics. 5. column holds the number of the category in which we put the problem. The categories are defined in the section A taxonomy of ease-of-use problems in e-banking. (Only problems found during walk-through of the installation task were assigned categories). After each table we add a few illustrative screen dumps as examples and some additional HE comments – usually more general ones regarding the system as a whole. Browser messages

During the walk-throughs some of the system responses come from the browser. Please note that these messages are not part of the Internet banking system itself, but may be believed so by the novice user. The messages from the browser are not counted as independent pages in the tables below. The page number in the table refers to the system page on which the message pops up. The following three types of messages are generated by the browser: “Security alert” is used to describe the browser message: “Security alert: You are about to view pages over a secure connection. Any information you exchange with this site cannot be viewed by anyone else on the Web”. Which includes the check box: “In the future, do not show this warning” and the buttons: “OK” and “More info”. When leaving a secure connection the browser will provide a similar Illustration 1 Browser pop-up message: Security message. Alert.

“Security Warning” describes a browser pop-up message such as: “Security warning: Do you want to install and run “C:\DOCUME~1\INF- M_~1\LOCALS~1\Temp\JVMD0.tmp” signed on an unknown date/time and distributed by: Bankernes EDB Central. Publisher authenticity verified by VeriSign Class 3 Code Signing 2001-4 CA. Caution: Bankernes EDB Central asserts that this content is safe. You should only install/view this content if you trust Bankernes EDB Central to make that assertion. SIGNED WITH PERMISSIONS – Full permissions.”. The message includes the check box such as: Illustration 2 Browser pop-up message: Security “Always trust content from Bankernes EDB Central” warning. and the buttons “Yes”, “No” and “More info”.

“Security information” describes the browser message such as: “Security information. This page contains both secure and non secure items. Do you want to display the non secure items?” The message includes the buttons: “Yes”, “No” and “More info”.

Illustration 3 Browser pop-up message: Security information.

The browser messages inform the user that danger lies ahead, for example, that his actions may cause him to loose data. Therefore, the system's misuse of a warning sign be regarded as a usability problem, and will mentioned in the tables below. One browser message may cause several usability problems. We have, however, chosen to count each browser message as one whole problem. In the final count (see “Summary of tables” below) one browser message thus equals one usability problem each time it appears in a sequence. Heuristics

As mentioned above we estimate breakdowns and problems on the basis of a collection of heuristics. Heuristics 1, 2, 3, 4, 5, 6, 7, and 8 are sampled from Nielsen and Molich (1990). Heuristics 3, 4, 5, 8, 9, 10, and 11 are sampled from Shneiderman (1998). Finally we added one heuristic, Heuristic 12, that we find could be of importance to our main goal. Thus we make our usability evaluation on the basis of the following heuristics:

No. Heuristic How we use the heuristic in the investigation 1 Simple and Dialogues should not contain irrelevant or rarely needed information. natural dialogue Every extraneous unit of information in a dialogue competes with the relevant units of information and diminishes their relative visibility. All information should appear in a natural and logical order. 2 Speak the user's The dialogue should be expressed clearly in words, phrases and language concepts familiar to the user rather than in system oriented terms or specific terms used in banking circles. 3 Minimize user The user should not be required to remember information from memory load previous sites or tasks. Neither should he be obliged to remember predefined codes. 4 Be consistent Consistent sequences of actions should be required in similar situation; identical terminology should be used in prompts, menus etc. Consistency in colour, fonts layout etc. should also be employed. 5 Provide For every action there should be a system feedback. For instance informative when logging off the user should receive information that he has feedback indeed succeeded in doing so. 6 Provide clearly A system should never capture users in situations that have no visible marked exits escape. Users often choose system functions by mistake and will need a clearly marked “emergency exit” to leave the unwanted state without having to go through an extended dialogue. 7 Good error Reading an error message should enable the user to correct the error messages rather then just informing him that an error has occurred.1 8 Offer error The system should preferably be designed so that users cannot make prevention and errors. For example by preferring menu selection to form fill in. simple error handling 9 Permit easy As much as possible actions should be reversible. This feature reversal of relieves anxiety since the user knows that errors can be undone thus actions encouraging unfamiliar options.

1The heuristic about error messages is relevant only for the evaluation of the systems' response to weak passwords (see the chapter Weak Passwords). During the remainder of the evaluation we behaved “correctly” and did not trigger error messages. Note also that the browser messages discussed above in section Browser Messages are warnings, not error messages. 10 Design dialogs to Sequences of actions should be organized into groups with a yield closure beginning, middle and end. The informative feedback at the completion of a group of actions gives users the sense of accomplishment and satisfaction with the system such as( when making a money transfer) the system e.g. informs: “Finally you need to sign the order to transfer the amount”. 11 Support internal Experienced operators strongly desire the sense that they are in charge locus of control of the system and that the system responds to their actions. Make users initiators of actions rather than responders to actions. With regard to this heuristic we do accept that a system such as an e- banking system to some extent demands user response such as signing in with a password. We do find however that e.g. browser pop-up messages or pop-up windows that close automatically are in conflict with this heuristic. 12 Provide The system should provide the user with the possibility to obtain sufficient and thorough and sufficient help whenever it is needed. This could be as easily accessible e.g. examples of how to fill in dates or how to create a strong help password.

A taxonomy of ease-of-use problems in e-banking

The following table defines four categories of ease-of-use problems. The categories group ease-of- use problems according to how they may be fixed eg. easily or only at the cost of weakening security.

Name Definition Fixable but security- An aspect of the user interface that reduces ease of use, but where related modifying this part of the interface would weaken security. Straightforwardly Ease-of-use deficiences that we believe can be fixed without weakening fixable security or introducing other problems. Platform-related Ease-of-use problems that are due to the underlying platform, for example browser pop-up messages that may confuse the user and which are outside of the direct control of the bank. Undue simplification Ease-of-use problems likely to be experienced by the more informed user, who may be rightly confused if the user interface describes distinct features by the same name, for example both password and private key by the name "code".

Fixable but security-related ease-of-use problems include authentication mechanisms that require the user to remember a randomly generated eight-character password, where the use of a more easily remembered password would weaken security.

Straightforwardly fixable ease-of-use problems include confusing terminology such as labelling the button that a new user must press "Functions" rather than "New user", and where choosing the more user-friendly label seems seems not to have no undesirable consequences. The category also includes the use of different phrases such as "code" and "password" to denote the same item, something that can be fixed simply by using a consistent terminology. Platform-related deficiencies originate from the underlying software and as such, cannot be fixed by changing the bank's software. For example, the browser pop-up message "Security alert" is shown by the browser used on our test machine when a session that uses a secure socket connection is invoked by some user action. Possible solutions to platform-related deficiencies include:

• Eliminate whatever triggers the platform-related deficiency. For example, omit the use of a secure socket connection to eliminate the "Security alert" message. This solution radically challenges the whole idea of using the Internet as an infrastructure, and is discussed in more detail in ..

• Encourage the user to read a text that explains the relevant browser messages etc. prior to performing the installation. The benefit of this approach is to provide the user with an opportunity to obtain some degree of understanding of the security issues related to her use of the Internet banking system. The difficulty is that many users may simply choose to omit the reading and studying of such general, explanatory material.

• Provide on-the-fly explanation of the browser messages, for example "Next a pop-up message will appear saying that .. This means that .. You may safely ..". This solution may be difficult to implement, because different platforms (ie. combinations of browsers, versions, etc.) may give rise to different browser messages. Realistically, only a few, common platforms can be covered. If implemented, this type of solution should probably include an option for short-cutting the explanatory messages altogether, since when first read and understood, repeating them at subsequent sessions is irrelevant and increases the number of steps the user must go through. Undue simplification is distinct from the other types of deficiencies because it may reduce the amount of complex terminology presented to the user, and so in some sense facilitates ease of use. When we label a feature or aspect of the user interface as belonging to this category, it reflects our point of view that it is desirable that the user is assisted in building some minimal level of understanding of the relevant, basic security issues, such as the distinction between a private key stored on hardware and a password remembered by the user. Moreover, the informed user who already understands eg. the password/key distinction is likely to be confused if the interface describes them as if they were the same.

PC and browser configuration

The PC used for the evaluation was equipped with a Pentium III 866 MHz processor with Windows 2000 and the browser Internet Explorer 6.0. The Internet was accessed via a broadband connection. The browser's security settings are set to “Custom” and the privacy to “Medium” (see Illustration 1). Illustration 4 Browser security settings. The settings concerning cookies are set to medium.

The Browser is set to prefer English (see Illustration 2).

Illustration 5 Browser language settings. The browser prefers English to Danish.

The advanced security settings are shown in Illustration 3. Illustration 6 The advanced browser security settings.

The browser always check for newer versions of pages (see Illustration 4).

Illustration 7 The browser always checks for newer versions of the page

Finally, the browser's auto complete feature was used for web addresses, forms and user names and addresses on forms. Installation

The installation task in each e-bank is described in a separate section. Each section contains the subsections “Intent”, “Steps and actions”, “Examples”, and “Additional comments”. The subsection "Steps and actions" contains the core data in the form of a table. These four subsections will reappear in subsequent chapters as well, to describe the other tasks. However, since the installation of an Internet bank is for most solutions the largest and most time consuming task, installation is described also by means of two additional paragraphs: The paragraph “Preconditions and available Instructions” lists the items and information provided by the bank in order to make the installation possible. This is information such as pin code and initial password. The paragraph “Information vs. instruction” comments on the level of information, whether it be in the form of instructions or thorough information. The overall structure of the subsections describing the installation sequences is thus: “Intent”, “Preconditions and available Instructions”, “Steps and actions”, “Examples”, “Information vs. instruction” and “Additional comments”.

Danske Bank

Intent: To run Danske Netbank for the first time (24.6.2003).

Preconditions and available Instructions: Before the installation Danske bank provides the user with: 1. a letter including a 4 digit pin code, 2. a letter including a 6 character agreement number and 3. an instructional pamphlet. The information enclosed in the pamphlet is however neither sufficient nor correct. The information is very general, but Danske bank makes a fine attempt to explain basic system features in a user friendly manner. Since the pamphlet is incorrect we have not used it's instructions in the following sequence.

Steps and actions:

Pp System information/actions User actions Breakdowns or small Problem problems category 1 www.danskebank.dk Click on button “Log on” Not natural language – the user cannot log on – he is not 2 Several menu buttons including “Log on” yet initialized 1 Scroll menu unfold several links including Click on “Danske Netbank” “Danske Netbank” 1 Browser pop-up message: “Security alert” “OK” is clicked, check box The language is not user left unchecked friendly and may leave the 3 user worried No sense of internal locus of 3 control 2 Click “Functions” The term “Functions” is not Title: “Log on”(“Danske bank log-on”) 2 natural dialogue Fields: “Agreement number” and “Password” (“Aftalenummer” and “Kodeord”) The feedback is not very useful 2 Buttons: “OK”, “Cancel” and “Functions” (“OK”, “Annuller” and “Funktioner”) 2 A scroll menu unfolds: Choose “New agreement” The term “New user” should be preferred to “New 2 “New agreement”, “New pin code”, “Change password”, agreement” according to the “Recreate key”, “Delete key”, “Save key”, “Key on request for natural dialogue floppydisc”(“Ny aftale”, “Ny pinkode”, “Skift kodeord”, “Gendan nøgle”, “Slet nøgle”, “Gem nøgle”, The term “key” may not be 2 “Nøgle på diskette”) user language No help is offered 2 3 Pop up message: Fill in fields and press Though the system do hold “OK” information about how to Title: “New agreement” (“Ny Aftale”) create strong passwords this information is not accessible 2 Fields: “Agreement number”,“Pin code”, from page 3. This fact may “New password”, ”Confirm new password” cause the user mental (“Aftalenummer”, “Pinkode” , “Nyt overload kodeord”, “Bekræft nyt kodeord”) The user needs to fill in 2 fields with codes the bank has Buttons: “OK”, “Cancel” (“OK”, “Annuller”) 2 provided for him. This causes a heavy memory load

4 Title: “Information” (“Information”) Click “Yes” Terminology is inconsistent (mixes Danish and English 3 Text: “Danske Bank has updated your key. You should terms) save a copy. Do you wish to save a copy now?” (“Danske Bank har opdateret din nøgle. Du bør derfor The text holds difficult snarest tage en sikkerhedskopi. Ønsker du at tage en security terms which are not sikkerhedskopi nu?”) explained 1 Buttons: “Yes” and “No” 5 Pop up message: “Save as” (“Gem som”) Accepts default path A path name may not be user's language 3 A default path is shown in the field 6 Pop up message: Click “OK” The warning sign does not fit 2 the text Text: (warning sign ) “Information. The task is correctly fulfilled” (“Information. Funktionen er korrekt Which task (the Danish text gennemført”) uses the word funktion (function)) is the text 2 Button: “OK” referring to? Not a simple dialogue 7 Title: “View of account” (“Kontooversigt”) The welcome text is easily missed which results in a bad 2 Text: “This is your first log on” (“Det er første gang du closure logger på netbanken”)

Examples:

Illustration 8 Screen dump of the message on page 3. The user has to fill in agreement number, pin code and new password (x2). Information vs. instruction: The information level is at an instructional level.

Additional usability comments: 1. No examples are available to the user. 2. The system offers no way for the user to exit but to log out of the entire system or use the backtracking possibilities provided by the browser. 3. Help is very limited. The system does provide some extent of advice regarding the creation of strong passwords. It may, however, not be sufficient. Nordea

Intent: To run Nordea Netbank for the first time (23.6.2003).

Preconditions and available Instructions: Nordea provides the user with 1. one letter including a 4 digit pin code, 2. a letter including a 10 digit agreement number (which is not similar to the account number) and 3. a third letter which includes a few instructions about how to get the installation started. These instructions are however not sufficient nor entirely correct.

Steps and actions:

Pp System information/actions User actions Breakdowns ( ) or small Problem problems category 1 www.nordea.dk Click menu button: Neither button “Netbank” nor “Private customers” Several menu buttons including “Private customers” “Kom godt igang”exists on (“Privatkunder”) www.nordea.dk 2 (A letter from the bank instructs the user No help is offered 2 to chose “Internet Bank” (“Netbank”) There is no logic in using the label 2 followed by “Get started” (“Kom godt “Private customers” igang”)) 2 Title: “Private” (“Privat”) Click “Self service” There is no apparent logic in using the label “Self service” - everything the 2 Text: A side menu with 11 buttons including “Self user does on a web page could be service” (“Selvbetjening”). labeled “Self service” 2 Scroll menu unfolds 10 buttons including “Internet Click “Internet bank” (At this point the user Bank” (“Netbank”). recognizes the instructions from the letter) 2 Scroll menu unfolds 10 buttons including “Get Click “Get started” started” (“Kom godt igang”) 3 Title: “Get started” (“Kom godt igang”) Click “Log on” The amount of text is rather large and covers many different types of Text: (excerpt) “Under “Internet bank” you will information. choose “log on” followed by “new user”. On the page “new user” you have to fill in agreement The explanation of the many following 2 number, user name (e.g. your own name) and your steps may cause an increased memory five digit pin code. Hereafter you choose your load. 2 personal password” (“Under punktet Netbank skal du vælge Log på og derefter Ny Bruger. I det The term Log on does not cover the 2 billede, der hedder Ny Bruger, skal du taste dit action about to take place (installation). aftalenummer, brugernavn (fx. Dit navn) og din It causes confusion that this button does femcifrede pinkode. Herefter vælger du dit not open the actual log on site. personlige kodeord.”) 3 Browser pop-up message: “Security alert” “OK” is clicked, The language is not user friendly and 3 check box left may leave the user worried unchecked 3 No sense of internal locus of control 4 Title: “Internet bank” (“Netbank”) Click “New user” Help offers the opportunity to make a search among issues related to logging 2 Side menu includes button: “New user” (“Ny on bruger”) 5 Title: “New user” (“Ny bruger”) Fill in fields The term “key” (which in the Danish text actually says “Security key” is not Fields: “Fill in agreement number”, “Fill in Click “Log on ” user friendly language) username”, “location of key” (“Indtast aftalenummer”, “Indtast brugernavn” , Some confusion concerning “user “Sikkerhedsnøgle placering”) name” - how is it different from a 2 password? Button: “Browse” (“Gennemse”) 2 The user have to fill in two codes which Fields: “Fill in pin code”, “Fill in personal are issued by the bank and cause him a 1 password”, “Repeat personal password”, (“Indtast heavy memory load. pin kode”, “Indtast personligt kodeord”, “Gentag personligt kodeord”) Buttons: “Log on”, “Cancel” (“Log på”, “Annuller”) 5 Pop-up message: (warning sign) “Your key will be Click “OK” The warning sign is not appropriate for generated when pressing ok. This may take several this kind of message minutes.” (“Din sikkerhedsnøgle vil blive genereret 2 ved trk på ok. Vær opmærksom på at det kan tage The term “key” (which in the Danish flere minutter.”) text actually says “Security key” is not 2 user friendly language) Button: “OK” (“OK”) 6 Title: “Kontooversigt” (“view of account”) Text: “Velkommen NN” (“welcome NN”)

Examples:

Illustration 9 The many scroll menus on page 2. Note that there is a "Log on" buttonin addition to "Get started" in the last menu. Illustration 10 Screen dump of page 3. Many various types of information is presented on this page.

Information vs. instruction: Nordea Netbank provides the possibility to obtain general information and ensures understanding by displaying a variety of links and search fields. The search engine lets the user search between issues relating to the current task which to the novice user is a great help.

Additional usability comments: 1. The system offers an explanation of various difficult security expressions and provides examples of strong passwords. However the user is not presented with all of these explanations and may have to seek out this information by himself by using for instance the help function. 2. The system fails to provide easy reversal of actions as well as presenting obvious exits. Jyske bank

Intent: To run Jyske Netbank for the first time (6.6.2003). Jyske Netbank uses a "key card" (see picture below) and there is not installation procedure. The procedure the user must go through to run Jyske Netbank for det first time is exactly the same as for every-day log on. Preconditions and available Instructions: Initially, Jyske bank has provided the user with: 1. a letter including the “key card” (see picture below) and 2. limited written instruction on how to use it.

Illustration 11 This is what the “key card” looks like. It measures the size of a credit card. Picture from www.jyskebank.dk.

Steps and actions:

Pp System information/actions User actions Breakdowns ( ) or small Problem problems category 1 www.jyskebank.dk Click on menu button “Begin Jyske Several menu buttons including “Begin Jyske Netbank” Netbank” (“Start Jyske Netbank”) 1 Browser pop-up message: “Security alert” “OK” is clicked, check The language is not user friendly box left unchecked and may leave the user worried 3 No sense of internal locus of 3 control 2 Title: “Log on and user identification”, “Welcome to Fields are filled in For the novice or elderly user Internet bank” (“Tilslutning og brugeridentifikation”, the “key card” can be difficult to “Velkommen til Netbank”) “OK” clicked use. There is a lot of numbers in a rather small font which may Fields: “Fill in customer number (social security cause problems 1 number)”, “Fill in key-card number” (“Indtast dit kundenr. (CPR-nr.)”, “Indtast dt nøglekortnr.”) Buttons: “OK”, “Reset” (“OK”, “Nulstil”) 3 Title: “Log in and access control” (“Tilslutning og Key and password are For the novice or elderly user adgangskontrol”) filled in the “key card” can be difficult to use. There is a lot of numbers in Text: “Does key number 17.ME figure on your key-card “OK”, is clicked. a rather small font which may number XXXXXX?” (Findes nøglenr. 17.ME på dit cause problems nøglekortnr. XXXXXX?”) The user has to remember a 1 Fields: “Yes. Fill in the 4 characters of the key at 17.ME”, password which causes some “Fill in password”, “No. Do not fill in your key and memory load 1 password. Log out of Jyske Netbank and contact your branch of Jyske Bank ” (“Ja. Indtast den 4-cifrede nøgle der står ved 17.ME”, “Indtast dit kodeord”,“Nej. Du må ikke indtaste din nøgle og dit kodeord. Afslut Jyske Netbank og kontakt din Jyske Bank afdeling”) Buttons: “OK”, “Reset” (“OK”, “Nulstil”) 4 Title: “Accounts and payments” (“Konti og betalinger”) Menu button No accounts are shown at this “Authorization” is page even though the title Text: “If necessary press authorization to see which clicked clearly suggests it. The result is accounts you hold authority” ( “Tryk evt. på fuldmagt for confusing and disturbing. at se hvilke konti du har fuldmagt til”) 2 This step seems unnecessary and Left menu includes the button “Authorization” the user may wonder why he has 2 (“Fuldmagt”) to take additional actions in order to view his accounts. 2 The menu button “Authorization” is not clearly marked. 5 Title: “View of accounts” (“Oversigt over konti”) Lack of closure. 2 The accounts are shown.

Examples:

Illustration 12 Screen dump of page 2. Illustration 13 Screen dump of page 3. Information vs. instruction: The system offers no other introduction or information but the content of the web page. The system prefers to provide clear instructions and not understanding or thorough information. The user can obtain thorough information about specific security issues such as certificates but this information is however not on a novice level and very poorly marked in the menus.

Additional usability comments: 1. The user can obtain thorough and sufficient help using the “Help” button. Help includes many illustrative examples but is unfortunately not clearly marked. 2. As for easy reversal of actions the system offers a reset button in an attempt to prevent the user from using the browser navigation. 3. If using the browser backtracking abilities the user will (rather confusingly) be returned to the front page. Skandiabanken

Intent: To open Skandiabank Netbank for the first time (12.6.2003).

Preconditions and available Instructions: Before the installation Skandiabanken has provided the user with: 1. one letter including an 8 character pin code and 2. a letter including an 11 digit user number (which is similar to the account number). The last letter refers to an on line “step-by-step guide” to installation.

Steps and actions:

Pp System information/actions User actions Breakdowns ( ) or Problem small problems category 1 www.skandiabanken.dk “Internet bank” is clicked Menu includes button “Internet bank” (“Netbank”) 2 Title: “Welcome to Internet bank” (“Velkommen til “...is it the first log on click Netbank”) here” is clicked Menu includes text/button “...is it the first log on click here”, (“...er det første gang, så klik her”) 3 Adobe Reader opens “Guide to installation” Adobe reader is closed “Guide to installation” is (“Vejledning”) difficult to use unless in printed version. The text holds too much information only to 2 be read from the screen, and may cause a heavy mental load 2 It may send the signal, that installation is complicated and difficult 2 Title: “Welcome to Internet bank” (“Velkommen til Click “Log on Internet The language does not Netbank”) bank” correspond with the attempted action: installation 2 Menu includes button “Log on Internet bank”, (“Log på netbank her”) 2 Browser pop-up window: “Security alert” “OK” is clicked and check The language is not user box left unchecked friendly and may leave the 3 user worried 3 No sense of internal locus of control 3 Browser pop-up window: “Security warning” “Yes” is clicked and check Language is not user friendly box left unchecked 3 Warning sign is in contrast with content. 3 No sense of internal locus of 3 control 3 Title: “Log on” (“Log on”) User number filled in The user has to fill in a user leaving check box number which increases Field: “Fill in user number” (“Indtast unchecked mental memory load brugernummer”) Click “OK” 1 Check box: “Remember user number” (“Husk Brugernummer”) Button: “OK” (“OK”) 4 Title: “Log on” (“Log on”) Click radio button “I have No examples of how to create received a pin code from the strong passwords. Radio button: “Go on line with the bank” (“Gå bank. Create a new signature online med banken”) file” Using the expressions “Uderskriftskode” and Fields: “Password”, “Signature file” (a field Click “OK” “Underskriftsfil” for password proposes a default path to the “signature file”. and key may cause problems 1 Additionally there is a “view” button) to the novice user who cannot 1 (“Underskriftskode”, “Underskriftsfil”) distinguish between the two. Radio button: “I have received a pin code from the Furthermore the terms - bank. Create a new signature file” (“Har modtaget especially signature file - are pinkode fra banken. Dan ny underskriftsfil”) not “The user's language” Buttons: “OK”, Return” (“OK”, “Retur”) 5 Log on site: The 4 fields are filled in The pdf- file “Guidance” correctly offers very limited information Fields: “Pin code”, “Repeat pin-code”, “New about how to create a strong password”, “Repeat new password”, “Path to the “OK” is clicked using the password signature file” (a default is proposed. Additionally default path there is a “view” button), “Name of “signature file” The expressions “path” and (“Pinkode”, “Genindtast pinkode”, “Ny “signature file” are not 2 underskrift”, “Genindtast ny underskrift”, “Sti til necessarily “The user's 3 underskriftsfil”, “Navn på underskriftsfil”) language” 2 Buttons: “OK”, “Return” (“OK”, “Retur”) There is an inconsistent use of “Signature” and “Signature 1 code” (“Underskrift” and “Underskriftkode”) The user has to fill in the pin code which increases the mental memory load 6 Blank page. No user action required. Poor feedback. 2 Next window pops up Text: “Generating key” in lower left corner of the Lack of internal locus of automatically 2 browser window control 2 Nor user's language

5 Short re-appearance of page 5 No user action required. Poor system feedback 2 Page closes automatically after ½ a sec. No internal locus of control 2 7 Title: “View of accounts” (“Kontooversigt”) Lack of closure – there is no 2 “welcome message”

Examples: Illustration 14 Screen dump of page 3. The guide to installation is full of useful examples. Unless it is printed out on paper it is however useless since the user clearly cannot remember 4 pages of information. Information vs. instruction: The system provides a printable guide through the entire installation. It generally prefers instruction as opposed to understanding and thorough information (e.g. “Guidance” and “Instructions”).

Additional usability comments: 1. Help is accessible from the top of every page, but it may not be sufficient since the user has to search within various help topics or FAQs. 2. The only options to reverse actions or exit from a task are provided by the browser. However if using the browser navigation to backtrack the user is returned to the initial page “Welcome to the Internet bank”. This may be confusing system feedback. Sydbank (certificate solution)

Intent: To open Sydbank Netbank (using the certificate solution) for the first time (15.6.2003).

Preconditions and available Instructions: Before installation Sydbank has provided the user with: 1. a letter including an 8 character user number (which is different from the account number) and a 8 character password. The user has provided Sydbank with an initial phrase (max. 10 characters) which is to be used instead of a pin code.

Steps and actions:

Pp System information/actions User actions Breakdowns ( ) or Problem small problems category 1 www.sydbank.dk Click “Start Internet bank” Navigation is difficult. One button is called “Gammel Menu buttons include “Start Internet bank” (“Start Netbank” (“old Internet Netbank”) bank”) and another “Start 2 Netbank” (“Start Internet bank”). Both are difficult to find. 1 Browser pop-up message: “Security alert” Click “OK” The language is not user friendly and may leave the 3 user worried 3 No sense of internal locus of control 1 Browser pop-up message: “Security information” Click “Yes” Language is not user friendly 3 No sense of internal locus of 3 control 2 Title: “Welcome” (“Velkommen”) Click “Get started” Text: “If this is first time you use the new Internet bank it may be to your advantage to view our introduction which can be found under “Get started”” (“Hvis det er første gang, du skal til at bruge den nye NetBank, kan det være en fordel at gennemgå vores introduktionsforløb, som du kan finde under "Kom godt i gang””) Menu buttons include “Get started” (“Kom godt igang”) 2 Scroll menu presents a choice between: “Introduction Click “Get started” Clicking ”Get started” and technical issues”, “Signing up”, “Get started” and reveals a new button called “Investment” (“Introduktion og “Get started”. In this case 2 teknik”,”Tilmelding”, “Kom godt igang” and there should not be “Investering”) consistency between these two labels 3 Title: “Get started” (“Kom godt igang”) Fill in user number and Initially the two fields are disposable password difficult to spot Text: (A flash presentation of the Internet bank.) Click “Log on” Terminology can be Fields: “User number” and “Password” confusing. “Password” (“Brugernummer” and “Kodeord”) (“Kodeord”) may suggest to 2 Buttons: “Log on” and “Help” (“Log på” and the user a personal password “Hjælp”) but in this case it is a 1 disposable password from the bank 1 User has to fill in two codes provided by the bank – this causes increased memory load 4 Title: “Welcome -> change password -> read Click “Continue” “Security solution” may not certificate .-> accept terms” (“Velkommen-> skift be the users language kodeord -> indlæs certifikat -> accepter vilkår”) Text: “During this log on you will have your “security solution” to Sydbank opened. Hence the sequence is somewhat different from the ordinary one” (“Ved 2 denne pålogning skal du have åbnet din sikkerhedsløsning til Sydbank. Forløbet er derfor noget anderledes end det normalt vil være”) Buttons: “Continue” and “Cancel” “Fortsæt”and “Afbryd” 5 Tilte: “Welcome -> change password -> read Fill in initial phrase The user is required to certificate .-> accept terms” (“Velkommen-> skift remember his password and kodeord -> indlæs certifikat -> accepter vilkår”) Fill in password a initial phrase he chose Click “Continue” minimum a fourth night ago Text: “Now you have to fill in your initial phrase. The which increases memory initial phrase must be typed in precisely as you wrote load it on the form to Sydbank” (“Du skal nu indtaste dit mærke. Mærket skal angives præcist som du angav The field to fill in the initial det på blanketten til Sydbank”) phrase holds no way to prevent errors since the user Field: (to fill in initial phrase) himself chooses the length Text: “Here you need to change your password and and content of the initial 1 phrase you need to pick one which on one hand is easy to 1 remember to you but hard to guess to others. The Lack of continuity in button password should be minimum 8 characters long and terminology. “Afbryd” 2 preferably contain both ciphers and letters” (“Her (previous page) and 1 skal du ændre dit kodeord, og du skal vælge et “Annuller” are both used to kodeord, der på samme tid er let at huske for dig men describe cancellation svært at gætte for uvedkommende. Kodeordet skal være mindst 8 karakterer langt og helst bestå af både The system does not provide tal og bogstaver.”) understanding for strong passwords and delivers no Fields: “Fill in new password” and “Repeat new examples to guide the user password” (“Indtast nyt kodeord” and“Gentag nyt kodeord”) Buttons: “Continue” and “Cancel” (“Fortsæt”and “Annuller”) 6 Pop-up message: “You password has been changed. Click “OK” A word is missing in the The password you received from (-) cannot be used message, which results in a anymore however you still need to use the user strange unnatural dialogue number” (“Dit kodeord er nu ændret. Kodeordet som du modtog fra (-) kan ikke anvendes mere, men du The intent of the message is skal forsat anvende brugernummeret”) to tell the user not to throw 2 out the paper containing both 2 Button: “OK” (“OK”) initial password and user name. However it fails to do 2 so in an awkward dialogue that leaves the user puzzled No help is offered to understand this message 7 Title: “Welcome -> change password -> read Click “Continue” A path may not be user's certificate .-> accept terms” (“Velkommen-> skift language kodeord -> indlæs certifikat -> accepter vilkår”) Text: “To use the Internet bank you need to load a certificate onto your PC” (“For at benytte netbanken skal du have indlæst et certifikat på din PC”) (Further information about certificates) 2 “The certificate is saved on the following path” (“Certifikatet gemmes i nedenstående stiangivelse”): Field: (containing a default path) (Additional information about certificates) Buttons: “Continue” and “Cancel” (“Fortsæt” and “Afbryd”) 7 Browser pop-up message: “Security warning” Click “Yes” Language is not user friendly. Warning sign may 3 be in contrast with content 3 No sense of internal locus of control 8 Text: “The certificate is being read...” (“Certifikatet No action is required. This Not a clear response from er ved at blive indlæst...”) window automatically turns the system. The page is 2 into the next after a few almost identical to page 6 seconds 2 Lack of locus of control 9 Title: “Welcome -> change password -> read Click “Continue” (This is the anticipated certificate -> accept terms” (“Velkommen-> skift response to user actions kodeord -> indlæs certifikat -> accepter vilkår”) taken in step 6) Text: “The certificate which you need to log onto the Internet bank has been loaded to your PC. The certificate expires (...) Now you only need to...”(more text) (“Indlæsning af certifikatet på din PC som du skal bruge ved efterfølgende pålogning til Netbanken er nu fuldendt. Det indlæste certifikat udløber (...) Nu mangler du bare...”) Buttons: “Continue” and “Cancel” (“Fortsæt” and “Afbryd”) 10 Title: “Welcome -> change password -> read Fill in password There is no print version of certificate .-> accept terms” (“Velkommen-> skift the agreement. The user kodeord -> indlæs certifikat -> accepter vilkår”) Click “OK” must read a long text (which is, by the way, not prepared Text: The entire agreement between bank and for the current media) and customer (about 4 printed pages) remember about 4 pages 2 Field: “Confirm by filling in your password and before accepting. This step clicking OK” (“Bekræft ved at indtaste dit kodeord increases the load on the og klikke OK”) user's memory dramatically. Button: “OK” (“OK”) 10 Browser pop-up message: “Security warning” Click “Yes” Language is not user friendly. Warning sign may 3 be in contrast with content. 3 No sense of internal locus of control 11 Text: “Your password has been changed. In a moment No user action required. This This is no logic response to you can get started” (“Dit kodeord er blevet ændret. page closes automatically in a the user action of previous Det varer et øjeblik før du kan komme igang”) few seconds. steps. Password was changed 2 several steps ago. 2 Automatic closure results in missing internal locus of control 12 Title: “View of account” (“Kontooversigt”) Lack of closure. (There is a small welcome notice but it 2 drowns in the additional informations on the page.)

Examples: Unfortunately no examples are available.

Information vs. instruction: Sydbank succeeds in providing a thorough information about the meaning of various security concepts and how to use the system. Since this information consists of both a long piece of text as well as a flash application it is however likely to cause confusion to the user.

Additional usability comments: 1. The user can get help by clicking a “Help” button on top of every page. Help consists of both a search field and an index which provides a stepwise walk-through the installation. 2. Help provides an explanation of some but not all of the crucial security words. 3. Reversal of actions and exit is provided only by the browser. 4. The system provides the user with a fine sense of progression during the completion of the installation. Pages 4-10 shows how tasks have been grouped and hence leaves the user with a fine sense of closure when completing a certain task (see example of page 4 above). Sydbank (“key card” solution)

Since the installation failed and the task thus unaccomplished we have chosen not to include the 5th row: “Problem category” in the table below.

Intent: To run Sydbank Netbank (using the key card solution) for the first time (5.8.2003).

Preconditions and available Instructions: Before installation Sydbank has provided the user with: 1. a letter including the “key card”, 2. limited instructions about how to activate the card 3. a letter including an 8 character user number (which is different from the account number) and a 8 character password. The user has provided Sydbank with an initial phrase (max. 10 characters) which is to be used instead of a pin code. In order to use Sydbank's “key card solution” the user initially need to have a certificate solution running.

Steps and actions: Initially the user has to carry through the installation of the certificate solution according to the sequence above. The installation of the “key card” solution follows immediately:

Pp System information/actions User actions Breakdowns ( ) or small problems 1 Title: “View of accounts” (“Kontooversigt”) Click “Security” 7 links including “Security” (“Sikkerhed”) 2 Title: “Security solution” (“Sikkerhedsløsning”) Click “Activate key card solution” Text: (excerpt): “You have logged on using a certificate” (“Du er logget på med en certifikatløsning”) Links: “Change certificate solution”, “Activate key card solution” (“Ændr certifikatløsning”, “Aktiver nøglerkortløsning”) 3 Title: “Activation of key card solution” (“Aktivering Click “OK” No useful help is provided. There is no such af nøglekortløsning”) function as “Change key card solution” on the page Text: (excerpt) “When receiving the key card you have to activate the key card. You do this with the The feedback is confusing – the user has just function “Change key card solution”” (“Efter clicked “Activate key card solution” and is now modtagelse af nøglekortet skal du aktivere asked to click “Change key card solution” . nøglekortet. Det gør du i funktionen “Ændr Either there is a problem with consistency or nøglekortløsning””) there is a function missing Buttons: “OK”, “Cancel” (“OK” and “Fortryd”) 4 Text: “Your key card solution is now activated” Click “OK” Surprising feedback that does not respond to the (“Din nøglekortløsning er nu aktiv”) conducted action Button: “OK” (“OK”) 2 Title: “Security solution” (“Sikkerhedsløsning”) Click (“Log off”) Surprising feedback that does not respond to the action just made Text: (excerpt) “You have logged on using a certificate” (“Du er logget på med en The only sign of change of status is the slight certifikatløsning.”) change of words: “Change key card solution” in stead of “Activate key card solution” Links: “Change certificate solution”, “Change key card solution” (“Ændr certifikatløsning”, “Ændr nøglerkortløsning”) Menu includes button “Log off” (“Afslut”) 5 www.sydbank.dk Click “Start Internet bank” The top menu holds the button:“Start Internet bank” (“Start Netbank”) 5 Browser pop-up message: “Security alert” Click “OK” The language is not user friendly and may leave the user worried No sense of internal locus of control 6 Text: (excerpt) “Welcome to the new Internet bank” Fill in user number The help offered by the search engine is very (“Velkommen til den nye netbank”) limited Fill in password Fields: “User number” and “Password” Help offered by the dialogue is insufficient – (“Brugernummer” and “Kodeord”) Click “Log on” how should a user log on using a “key card solution”? Buttons: “Log on” and “Help” (“Log på” and “Hjælp”) The system does not offer to remember the user number. And since the user number cannot be changed the user may risk mental memory overload There is no information about using the key card or fields to fill in the numbers 7 Text: “Your password has been accepted. The system No user action required. The The language may not be user friendly is currently getting hold of your certificate. It will page closes automatically only take a moment before you are logged on” (“Dit No sense of internal locus of control kodeord er accepteret. Systemet er i øjeblikket i færd med at hente dit certifikat. Det varer kun et øjeblik, så er du logget på”) Text: “Note: After an idle period the system will automatically end your log in session and you will be asked to log on once again ” (“Efter nogen tid uden aktivitet på systemet, vil din login automatisk blive afsluttet og du vil blive bedt om at logge på igen”) Button: “Cancel” (“Afbryd”) 1 Browser pop-up message: “Security warning” Click “OK” The language is not user friendly and may leave the user worried No sense of internal locus of control 1 Title: “View of accounts” (“Kontooversigt”) Click “Daily finances” Top menu: “View”, “Daily finances”, “Investment”, “Housing”, “Contact”, “Guidance & appliance”, “Security” (“Overblik”, “Daglig økonomi”, “Investering”, “Bolig”, “Kontakt”, “Vejledning & tilmelding”, “Sikkerhed”) 1 Scroll menu unfolds 4 links including “Payments and Click “Payments and Not user's language: What is the actual transfers” (“Betalinger & overførsler”) transfers” difference between the two words? 1 Scroll menu unfolds 4 links including “Transfers” Click “Transfers” (“Overførsler”) 8 Title: “Transfers” (“Overførsler”) Click “Security” No way to reverse actions Text: “You have no account from which you can Poor error message withdraw money. Hence you cannot make payments using the Internet bank” (“Du har ingen konto at No sense of internal locus of control hæve på. Du kan derfor ikke foretage betalinger i No help or information netbanken”) Top menu: “View”, “Daily finances”, “Investment”, “Housing”, “Contact”, “Guidance & appliance”, “Security” (“Overblik”, “Daglig økonomi”, “Investering”, “Bolig”, “Kontakt”, “Vejledning & tilmelding”, “Sikkerhed”) 2 Title: “Security solution” (“Sikkerhedsløsning”) Click “Change key card No way to reverse actions solution” Text: (excerpt): “You have logged on using a certificate” (“Du er logget på med en certifikatløsning.”) Links: “Change certificate solution”, “Change key card solution” (“Ændr certifikatløsning”, “Ændr nøglerkortløsning”) 9 Title: “Changing key card solution” (“Ændring af Click “Activate new key card This is no simple and natural dialogue nøglekortløsning”) –...” No help is offered Radiobuttons: “Order new key card” and “Activate Fill in correct card number new key card – please note that your present key card will be erased and hence cannot be used any more” Click “OK” (“Bestil nyt nøglekort” and “Aktiver nyt nøglekort (vær opmærksom på at dit eksisterende nøglekor slettes og derfor ikke kan benyttes mere” ) Field: “Fill in key card number from the new key card” (“Indtast nøglekortnummer fra det nye nøglekort”) Buttons: “OK” and “Cancel” (“OK” and “Fortryd”) 9 Same text as previous, except from the text above the (Repeat filling in the correct No help offered (e.g. The number of the Hotline) title: key card number with the same result) Poor error message since “You have not filled in the key card number correctly. Please try again. If the number is not Eventually the attempts are accepted please contact the Hotline” (“Du har canceled and the user logs off. indtastet nøglekortnummer forkert. Prøv igen. Hvis nøglekortnummeret stadig ikke accepteres kontakt Hotline”)

Examples:

Illustration 15 Screen dumps of page 2. This page offers the possibility to activate the “key card”. Illustration 16 Screen dump of page 8. After re-log on the user is met with this message: “You do not have an account to withdraw from. Thus you cannot make payments using Netbanken”. Illustration 17 Screen dump of page 4. The “key card” is now active – but what does that mean. When trying to use the “key card” after a re-log on it is however clear that the system does not work where as the question as to what is wrong remains unanswered. See screen dump of page 8. Illustration 18 Screen dump of page 6. When trying to activate key card again this page appears. After several attempts it is clear that the number on our “key card” is not valid, and we are left to contact Hotline.

Information vs. instruction: The key card arrives with a short letter with instructions of how to initialize the solution. The information offered in this letter (and by the system as a whole) is however rather shallow and neither informative nor sufficiently instructive.

Additional usability comments: 1. Sydbank Netbank provides absolutely no useful help or information with regard to initializing the key card. 2. The system feedback and terminology seems sometimes random and the user is left without any feeling of internal locus of control before simply giving up installation sequence. Basisbank

Intent: To run Basisbank for the first time (29.7.2003).

Preconditions and available Instructions: Before the installation Basisbank has provided the user with: 1. a “pin-letter” including a 4 digit access code and a 8 character “key activating code” and 2. an email including a 7 digit user number (which is not identical with the account number). The email includes a few basic instructions about how to perform the installation.

Steps and actions:

Pp System information/actions User actions Breakdowns ( ) Problem or small category problems 1 www.basisbank.dk Click “Log onto the bank” The menu includes the button “Log onto the bank” (“Log på banken”) 1 Browser pop-up message: “Security alert” Click “OK” The language is not user friendly and may leave the user 3 worried 3 No sense of internal locus of control 2 Title “Front page” (“Forside”) Click “I have received new codes from the Fields: “Reg. and user no.” and “Access code” (“Reg. og bank” brugernr.” and “Adgangskode”) Links: 4 links including “I have received new codes from the bank” (“Jeg har modtaget nye koder fra banken”) Buttons: “OK” and “Cancel” (“OK” and “Fortryd“) 3 Title: “Activate on line access” (“Aktiver online adgang”) Fill in the 6 fields There is a pause for about 7 seconds with Fields: “Reg.no.”, “User no.”, “Key activating code”, “Email Click “OK” total lack of system address”, “Signature code”, “Confirm signature code” (“Reg. nr.”, response “Brugernr.”, ”Nøgleaktiveringskode”, “Email adresse”, Underskriftkode”, “Bekræft underskriftkode”) This page holds many fields to fill in Buttons: “OK” and “Cancel” (“OK” and “Fortryd”) and increases 2 memory load 1 It makes no sense to fill in email address, 2 since the bank has all ready got it 2 “Nøgleaktiveringsko 1 de” (“key activating code”) is not user's language Poor help with regard to creating a strong password 3 Pop-up message: (warning sign) “The key has been generated and Click “OK” The term “key” has saved to disc” (“Nøglen er nu genereret og gemt på disc”) never been introduced and the 2 Button: “OK” (“OK”) system offers no help to examine it 4 Text: “As this is the first time you log onto the bank you need to Fill in fields. User has to change your access code” (“Da dette er første gang du logger på remember access banken skal du ændre din adgangskode”) Click “OK” code which increases memory load Fields: “Fill in your access code - appears in your PIN-letter”, “new 1 access code (4 digits)” and “confirm new access code” (“Indtast din adgangskode – fremgår af dit PIN-brev”, “Ny adgangskode (4 cifre)” and “Bekræft ny adgangskode”) Buttons: “OK” and “Cancel” (“OK” and “Fortryd”) 4 Pop-up message: “Signing” (“Signering”) Fill in field User has to remember signature Field: “Fill in signature code” (“Indtast din underskriftskode”) Click “OK” code which increases 1 memory load Buttons: “OK” and “Cancel” (“OK” and “Fortryd”) 2 No sense of internal locus of control 5 Title: “Message”(“Meddelelse”) Click “Back” Text: “You have changed your access code. Remember to use the new code next time you log on” (“Du har nu ændret din adgangskode til banken. Husk at bruge den nye kode næste gang du logger på”) Button: “Back” (“Tilbage”) 2 Title “Front page” (“Forside”) Fill in the 2 fields Content: see top of table Click “OK” 2 Pop-up message: “Signing” (“Signering”) Fill in field User has to fill in two codes which Field: “Fill in signature code” (“Indtast din underskriftskode”) Click “OK” increases memory 1 Buttons: “OK” and “Cancel” (“OK” and “Fortryd”) load 6 “View of account” (“Kontooversigt”) Text: “Welcome” (“Velkommen”)

Examples:

Illustration 19 Screen dump of page 3. The registration number is filled in by default but the user has to fill in his email address even though he has already provided this information earlier. Information vs. instruction: Basisbank provides the account holder with an email that instructs him how to carry through the initial log-on. The information level aims at instructing the user and does not create an understanding of security issues concerning the system.

Additional usability comments: 1. The system does not provide any help (additional to the email instructions) with regard to the installation, whereas the sequence of an every-day log on (this sequence is included in the installation, see last three rows in the table above) provides a “Help” button. This inconsistency is inconvenient and uncalled for. 2. Furthermore the help provided is rather poor since the user must choose between only three issues “Your money and you” (“Du og dine penge”), “Products” (“Produkter”) and “Technical issues” (“Teknik”) or be able to perform a search filling in keywords. 3. The system fails to provide error prevention since it is possible to fill in an infinite amount of both letters and digits in fields such as “Access code” (“Adgangskode”). The access code used in Basisbank never requires anything but 4 digits. 4. The system fails to provide exits, and the user have to close the browser or retype a path to get away from the system. Opportunity to reverse actions is provided solely by the browser. 5. The system seems to require some information, which is not necessary. The user has previously provided his email address to the bank, and the registration number is the same to every customer in the bank. These requirements seems unnecessary. 6. At some points the system uses the registration number together with the “User number” hence signaling that they belong together as registration number and account number. This is not the case: The user number is a unique user identification number issued by the bank. Summary of tables

The table below summarizes the number of steps, secrets and security phrases the user encounters during the successful installations above. It concerns the number of user actions acquired in the installation of the 6 banks (row 1), the number of problems encountered during these installations (row 2), the amount of user input (row 3), and finally the number of security phrases the user encounters during the installation (row 4).

How to read the 4 rows

Comments to row 1; number of user actions We regard a user action as an occasion where the user is required to do something. On one page the user may for instance be required to fill in 3 fields and clicking ”OK”, thus resulting in a total of 4 user actions.

Comments to row 2; number of problems The problems in row 2 have been arranged in the three categories mentioned in the paragraph “A taxonomy of ease-of-use problems in Internet banking”. We find it necessary to point out that one problem can occur more than once in each sequence, and that such a problem is counted as one problem every time it occurs.

Comments to row 3; amount of user input The user input is subdivided into 5 categories: 1. Initial ID (used to ID the user during initial log on), 2. permanent ID (used to ID the user during any later log ons), 3. initial password (used verify user identity), 4. permanent password (used to access the key in later log ons) and 5. other (input provided by the user for other or unknown reasons). The input written in (parenthesis) in row 3 are optional for the user to remember and the notation (x2) indicates that the user has to type in the secret twice. With regard to the secrets to remember we have chosen to count the social security number (“CPR-nummer”), in spite it is a code which most Danes remembers alongside their names and as such does not put much load onto the user's memory.

Comments to row 4; number of security phrases As for Jyske Bank the security concepts “User identification” and “Access control” merely appear as headings. The term “Authorization” is a menu item, but not strictly related to the security-during installation-issue. All three have, however, been included in the count of security concepts. The browser pop-up messages include following difficult terms, of relevance to row 4: “Security alert”, “Secure connection”, “Security warning”, “Signed”, “Publisher authenticity”, “VeriSign Class 3 Code Signing 2001-4 CA. Caution”, “SIGNED WITH PERMISSIONS – Full permissions”, “Security information” and “Secure and non secure items”. These terms are considered to be difficult security phrases and are included in the total count “Total including browser messages” underneath the list of phrases. Danske bank Nordea Jyske bank Skandiabanken Sydbank, certificate 1 User actions 13 14 9 15 19 2 Problems 1 1 1 3 4 5 2 11 12 4 11 15 3 4 2 2 6 8 3 User input Initial ID 1. Agreement number 1. Agreement number 1. Customer number (CPR 1. User number 1. User number number) 2. “Key card number” Permanent (same as initial ID - the system 2. User name ( the system (same as initial ID) (same as initial ID – the (same as initial ID) ID remembers the ID) remembers the ID) system remembers the ID) Initial 2. Pin code 3. Pin code 3. 4 digit key 2. Pin code 2. Initial phrase password 4. Password 3. Initial password Permanent 3. Password (x2) 4. Password (x2) (same as initial password, both 3. Password (x2) 4. Password (x2) password key and password) Other 4. Path to copy of key 5. Location of key

4 Security phrases 1. “Agreement” 1. “User name” 1. “Customer number” 1. “User number” 1. “ 2. “Agreement number” 2. “Agreement number” 2. “CPR number” 2. “Signature code” 2. “ 3. “Pin code” 3. “Five digit pin code” 3. “Key card number” 3. “Signature file” 3. “ 4. “Password” 4. “Personal password” 4. “Key number” 4. “Pin code” 4. “ 5. “Key” 5. “Safety key” 5. “4 digit key” 5. “Signature” 5. “ 6. “Safety copy” 6. “Location of safety key” 6. “Password” 7. “User identification” 8. “Access control” 9. “Authorization” Total incl. browser messages: Total incl. browser messages: Total incl. browser messages: Total incl. browser messages: Total incl. browser messages: 8 8 11 12 14 Every day log on

In the sequence we test the every day log on and how the system reacts when the user types in the wrong password. The latter is not directly documented in the tables, but commented upon in the paragraphs “Additional usability comments”. From this point and on we only test Sydbank's certificate solution since we did not succeed in activating the “key card” solution.

Danske Bank

Intent: To log onto Danske Netbank (6.6.2003).

Steps and actions:

Pp System information/actions User actions Breakdowns ( ) or small problems 1 www.danskebank.dk Click on “Log on” (“Log på”) Menu buttons include “Log on” (“Log på”) 1 Scroll menu reveals buttons including “Danske Click on “Danske Netbank” Netbank” 2 Browser pop-up message: “Security alert” Click “OK” The language is not user friendly and may leave the user worried No sense of internal locus of control 3 Title: “Log onto Danske Netbank” (“Log på Danske No user action is Poor system feedback. The user has no time to Netbank”) required. Window closes read the content of the page and is left confused automatically in about 1 Some additional text is shown but the window closes sec. Automatic closure of window means lack of too fast to enable the user to read it. internal locus of control 4 Pop up message: Check agreement number User has to remember his password and may (presented in a scroll additionally have to choose between several Title: “Danske Bank Log on” menu) and fill in agreement numbers in case he has more than one Fields: “Agreement number” and “Password” password. Click “OK” account – this increases memory load (“Aftalenummer” and “Kodeord”) The label “Functions” may cause confusion – Buttons: “OK”, “Cancel” and “Functions” (“OK”, what is the use for this button during log on? “Annuller” and “Funktioner”) 5 View of accounts

Examples:

Illustration 20 Screen dump of pop up message on page 4. The system remembers the agreement number in a scroll menu. Illustration 21 Screen dump of page 5. After filling in the correct password the user enters the view of accounts. Additional usability comments: 1. Danske Netbank's log on sequence provides no help whatsoever. The button “Help” (“Hjælp”) is unaccessible from page 4 where a user might want help to perform the correct actions. 2. There seems to be a potential problem with memory load with regard to agreement number: The system remembers agreement numbers and presents them in a scroll menu but a user with many accounts may experience difficulties when trying to remember which account number is attached to which account. 3. No exits are offered. 4. Error messages are very helpful and are concerned with solving the problem rather than just informing that there is a problem. Nordea

Intent: To log onto Nordea Netbank (23.6.2003).

Steps and actions:

Pp System information/actions User actions Breakdowns ( ) or small problems 1 www.nordea.dk Click on “Log on” (“Log på”) Menu includes button “Log on” (“Log på”) 1 Scroll menu opens several buttons Click on “Internet bank” Surprising feedback on the previous action (clicking ”Log including “Internet bank” (“Netbank”) (“Netbank”) on”) 2 Browser pop-up message: “Security alert” Click “OK” The language is not user friendly and may leave the user worried No sense of internal locus of control 3 Field: “Choose username” (“Vælg Fill in password accepting The label “Log on” has been used previously to another brugernavn”) default user name. button Button: “Find key” (“Find Click “Log on” The button “Find key” is confusing. There is apparently sikkerhedsnøgle”) no need for it Field: “Fill-in password” (“Indtast Memory load is increased due to the password kodeord”) Buttons: “Log on”, “Cancel” (“Log på”, “Annullér”) 4 View of account Examples:

Illustration 22 Screen dump of the scrool menu on page 1.

Additional usability comments: 1. Help is easy accessible, but offers no thorough assistance. On page 3 the provided help to “Ordinary Log on” (“Almindelig Log på”) is as follows: “You have to choose user name and fill in your personal password if you all ready got Internet bank or mail” (original text: “Har du allerede Netbank eller Mail, skal du vælge brugernavn og indtaste dit personlige kodeord”). This information is quite obvious when examining page 3, whereas questions regarding “what is a personal password” or “I have forgotten my password are left unanswered. 2. Error messages with regard to wrongly typed password is informative and instructive: “There is an error in the typed password – you may try again. Now you got 4 attempts left before access is terminated. If you do not remember your personal password you can order a new pin code at your local branch. (Error 15756)” (original text: “Der er fejl i det indtastede kodeord – prøv evt. igen. Du har nu 4 forsøg tilbage før din adgang bliver spærret. Kan du ikke huske dit personlige kodeord kan du kontakte din egen filial for at bestille en ny pinkode. (Fejl 15756)”) Jyske bank

Every day log on is the same sequence as installation (see above). The system offers a short error message as response to the wrong “key card” number. At this point the system could have chosen to provide an example of where to find the desired code. It does however not.

Illustration 23 The system responds with this error message: "Error in typed in "key card" number. Please try again". The red sign blinks between the word "stop" and the warning sign ! Skandiabanken

Intent: To log onto Skandiabanken (29.7.2003)

Steps and actions

Pp System information/actions User actions Breakdowns ( ) or small problems 1 www.skandiabanken.dk Click on “Internet bank” Menu includes “Internet bank” (“Netbank”) 2 Title: “Welcome to Internet bank” Click on “Log onto the (“Velkommen til netbank”) Internet bank” Text: (exerpt) “You enter the Internet bank by clicking on the header “Log onto Internet bank”” (“Du går i netbank ved at klike på fanen “Log på Netbank”) 2 Browser pop-up message: “Security alert” Click “OK” The language is not user friendly and may leave the user worried No sense of internal locus of control 3 Browser pop-up message: “Security Click “OK” The language is not user friendly and may leave the user warning” worried No sense of internal locus of control 3 Title: “Log on” (“Log på”) Fill in fields The attempt to prevent errors by using a scroll menu to present the user's various account numbers may cause a Fields: “User number”, “Signature code” (a Click “OK” mental overload since it is hard to remember which number default path is shown), “Signature file” belongs to which account (“Brugernummer”,“Underskriftkode”, “Underskriftfil”) “Signature file” may not be user language Buttons: “Browse”, “OK”, “New user/pin Memory load is increased due to the signature code code”, (“Gennemse”, “OK” and “Ny bruger/pinkode”) 4 Title: “View of accounts” (“Kontooversigt”)

Examples: Illustration 24 If the user has unread messages the system opens on the "in box page". Otherwise the system opens on the view of accounts. Illustration 25 The error message is short but in-instructive. "Error message. User number or password has been filled in wrongly". Additional usability comments: 1. There seems to be a problem with consistency since the system does not always open on the same page. If there is messages from the bank the system opens on “Information form the bank” (“Informationer fra banken”). Otherwise it opens directly onto the “View of accounts” (“Kontooversigt”). This design is in conflict with the heuristic concerning internal locus of control. 2. Error messages simply states where the error may be. Sydbank

Intent: To log onto Sydbank (24.6.2003 and 29.7.2003)

Steps and actions: a) Unsuccessful try to log on (24.6.2003).

Pp System information/actions User actions Breakdowns ( ) or small problems 1 www.sydbank.dk Click “Start Internet bank” The top menu includes the button “Start Internet bank” (“Start Netbank”) 1 Browser pop-up message: “Security alert” Click “OK” The language is not user friendly and may leave the user worried No sense of internal locus of control 2 Text: (excerpt) “Welcome to the new Internet bank” Fill in user number The help offered by the search engine is very limited (“Velkommen til den nye netbank”) Fill in password. The system does not offer to remember the user number. Fields: “User number” and “Password” And since the user number cannot be changed the user (“Brugernummer” and “Kodeord”) Click “Log on” may risk memory overload Buttons: “Log on” and “Help” (“Log på” and “Hjælp”) 3 Text: “Your password has been accepted. The (After 17 minutes of No system response system is currently getting hold of your certificate. It no system response will only take a moment before you are logged on” the user is left with no No help provided (“Dit kodeord er accepteret. Systemet er i øjeblikket choice but to click The term “Certificate” is used without explanation which i færd med at hente dit certifikat. Det varer kun et cancel) øjeblik, så er du logget på”) may cause the user worries Text: “Note: After an idle period the system will Poor sense of internal locus of control automatically end your log in session and you will The promise to end the log in after an idle period is not be asked to log on once again ” (“Efter nogen tid fulfilled uden aktivitet på systemet, vil din login automatisk blive afsluttet og du vil blive bedt om at logge på igen”) Button: “Cancel” (“Afbryd”) Examples:

Illustration 26 Screen dump of page 3. The system informs the user that it is trying to locate the certificate and that it will only take a moment before the user is logged on. It may, however, easily take as long as 30 minutes. Steps and actions: b) Successful try to log on (29.7.2003).

Pp System information/actions User actions Breakdowns ( ) or small problems www.sydbank.dk Click “Start Internet bank” The top menu holds the button: “Start Internet bank” 1 (“Start Netbank”) 1 Browser pop-up message: “Security alert” Click “OK” The language is not user friendly and may leave the user worried No sense of internal locus of control 2 Text: (excerpt) “Welcome to the new Internet bank” Fill in user number The help offered by the search engine is very limited (“Velkommen til den nye netbank”) Fill in password The system does not offer to remember the user Fields: “User number” and “password” number. And since the user number cannot be changed (“Brugernummer” and “Kodeord”) Click “Log on” the user may risk memory overload Buttons: “Log on” and “Help” (“Log på” and “Hjælp”) 3 Text: “Your password has been accepted. The system is No user action No sense of internal locus of control currently getting hold of your certificate. It will only required. The page take a moment before you are logged on” (“Dit closes automatically kodeord er accepteret. Systemet er i øjeblikket i færd med at hente dit certifikat. Det varer kun et øjeblik, så er du logget på”) Text: “Note: After an idle period the system will automatically end your log in session and you will be asked to log on once again ” (“Efter nogen tid uden aktivitet på systemet, vil din login automatisk blive afsluttet og du vil blive bedt om at logge på igen”) Button: “Cancel” (“Afbryd”) 4 Browser pop-up message: “Security warning” Click “OK” The language is not user friendly and may leave the user worried No sense of internal locus of control 4 Title: “View of accounts” (“Kontooversigt”) Examples:

Illustration 27 Screen dump of the error message the user receives when typing in both wrong user number and password. The message seems to suggest that the password holds the error (which is not entirely correct) bur does not provide any thorough help. The message is a follows: “The information filled in is wrong. Please note that the system is case sensitive when it comes to the password”.

Additional usability comments: 1. Ability to reverse actions or exit specific tasks is provided solely by the browser 2. The error message is neither entirely correct nor very useful. Basisbank

Intent: To log onto Basisbank (29.7.2003)

Steps and actions:

Pp System information/actions User actions Breakdowns ( ) or small problems 1 www.basisbank.dk Click “Log on” Various top menu buttons e.g. “Log on” (“Log på banken”) 1 Browser pop-up message: “Security alert” Click “Yes” The language is not user friendly and may leave the user worried No sense of internal locus of control 2 Browser pop-up message: “Security Click “OK” The language is not user friendly and may leave the user warning” worried No sense of internal locus of control 2 Title “Front page” (“Forside”) Accept default account number User has to remember several codes which increases in scroll menu memory load Fields: “Reg. and user no.” (scroll menu) and “Access code” (“Reg. og brugernr.” and Fill in access code “Adgangskode”) Click “OK” Links: 4 links including “I have received new codes from the bank” (“Jeg har modtaget nye koder fra banken”) Buttons: “OK” and “Cancel” (“OK” and “Fortryd”) 2 Pop-up message: “Signing: Fill in password” Fill in password User has to remember a password which increases (“Signering: Indtast underskriftskode”) memory load Click “OK” Buttons: “OK” and “Cancel” (“OK” and “Fortryd”) 3 Title: “View of accounts” (“Kontooversigt”) Examples:

Illustration 28 Screen dump of page 2. “User number” (“Bruger nr.”) appears together with the registration number as if it was the account number. It is, however, not.

Illustration 29 Screen dump of pop-up message on page 2. When filling in the wrong password and clicking "OK" nothing happens. There is no system response whatsoever and the user must terminate the task by clicking "Cancel" or closing the window with the x-button in the upper right corner. Additional usability comments: 1. The system may not provide sufficiently help since the user has to look for e.g. explanations of terms within a search index of just 3 issues. 2. The system does not provide error messages when the user have typed in the wrong password. The system simply freezes which must be regarded as a very unfortunate lack of feedback. Log off

Danske bank

Intent: To log off Danske Netbank and attempt a quick re-log on (6.6.2003).

Steps and actions:

Pp System information/actions User actions Breakdowns ( ) or small problems 1 Title: “View of account” (“Kontooversigt”) Click on “Log off” Menu buttons include “Log off” (“Log af”) 2 Text: “Getting information...” (“Henter No user action required. Window Poor system feedback oplysninger...”) closes automatically in about 1 sec. Leaves the user without any sense of internal Additional text is displayed but the window closes to locus of control fast to enable the user to read it. Action cannot be reversed 2 Browser pop-up message: “Security alert” Click “Yes” The language is not user friendly and may leave the user worried No sense of internal locus of control 3 Text: “You have succeeded in logging off Danske (After clicking “Log onto Danske Netbank” (“Du er nu logget af Danske Netbank”) Netbank once more” an ordinary log on session follows. For details Link: “Log onto Danske Netbank once more” (“Log see table above) på Danske Netbank igen”)

Examples:

Illustration 30 Screen dump of page 3. Danske bank provides status message from the system (“You are logged off Danske Netbank”) as well as the opportunity to log on quickly again. Additional usability comments: There is no additional comments. Nordea

Intent: To log off Nordea Netbank and attempt a quick re-log on (23.6.2003).

Steps and actions:

Pp System information/actions User actions Breakdowns ( ) or small problems 1 Title: “View of account” (“Kontooversigt”) Click “Log off” Menu includes the button “Log off” (“Log af”) 2 Browser pop-up message: “Security alert” Click “Yes” The language is not user friendly and may leave the user worried No sense of internal locus of control Blank page No user action needed. This page No sense of internal locus of control 3 closes automatically in ½ a sec. Nordea web page entitled “Private” (“Privat”) (From this point follows an Poor system feedback – no information about containing various informations about products ordinary log on including user log off status etc. identification. For details see table above) User experiences no sense of closure This page also includes a button “Log on” (“Log 4 på”) No reversal of action

Examples:

Illustration 31 From page 1 the user clicks "Log off" to exit and ends on page 4 (next example). Illustration 32 Screen dump of page 4. There is no thorough information about log off status on this page.

Additional usability comments: No additional comments. Jyske bank

Intent: To log off Jyske Netbank and attempt a quick re-log on (4.7.2003)

Steps and actions:

Pp System information/actions User actions Breakdowns ( ) or small problems 1 Title: “View of accounts” (“Kontooversigt”) Click “Close Jyske The link is marked in clear yellow as if “Close Jyske Netbank” Netbank” is the title of the current page. This is not Several menu links including “Close Jyske Netbank” consistent with the rest of the system (“Afslut Jyske Netbank”) 2 Text: “Jyske Netbank has been closed correctly. Press Click “OK” No possibility to reverse actions OK if you wish to go to Jyske Banks home page” (“Jyske Netbank er nu afsluttet korrekt. Tryk på OK hvis du ønsker at gå til Jykse Banks hjemmeside”) Button: “OK” (“OK”) 2 Browser pop-up message: “Security alert” Click “Yes” The language is not user friendly and may leave the user worried No sense of internal locus of control 3 www.jyskebank.dk (From this point follows the ordinary Home page includes button “Start Jyske Netbank” log on including (“Start Jyske Netbank”) identification of the user. See table above) Examples:

Illustration 33 Screen dump of page 1. The link “Afslut Jyske Netbank” in the left side menu is marked with the colour yellow as if it was the current page. All other links are white.

Additional usability comments: 1. The system does only to some extent provide reverse and exit possibilities. 2. If the user uses the browser's navigation buttons he is returned to Jyske Banks homepage which may be a confusing system feedback. Skandiabanken

Intent: To log off Skandiabanken (23.6.2003)

Steps and actions:

Pp System information/actions User actions Breakdowns ( ) or small problems 1 Title: “View of account” (“Kontooversigt”) Click “Log off” The menu includes the button “Log off” (“Log af”) 2 Title: “Log off” (“Log af”) Click “Yes” This page is an unnecessary feedback upon the previous Text: ”Do you want to log off?” (“Skal der logges af?”) action (pressing “Log off”) Buttons: “Yes” and “No” (“Ja” and “Nej”) 3 Text: “You have logged off the system. Thank you for your visit” (“Der er Click link No reversal of action logget af systemet. Tak for besøget”) Link to Skandiabankens home page www.skandiabanken.dk (Hereafter follows ordinary log on, see table above) 4 Menu includes “Internet bank” (“Netbank”) Examples:

Illustration 34 Screen dump of page 2. The systems wants to make sure tha the user actually want to exit. There is no call for that extra check-up.

Additional usability comments: There is no additional comments. Sydbank

Intent: To log off Sydbank Netbank immediately followed by an attempt to re log-on (28.7.2003).

Steps and actions:

Pp System User actions Breakdowns ( ) or small problems information/actions 1 Title: “View of account” Click “End” (”Kontooversigt”) Menu includes button “End” (“Afslut”) 2 Home page: Click on “Start Internet Lack of closure www.sydbank.dk/sydbank_privat/ bank”) (At this point actions can sometimes still be reversed) Menu includes button “Start Internet bank” (“Start Netbank”) Poor system feedback – no system status 1 Title: “View of account” See below in “Additional usability comments” (”Kontooversigt”)

Examples:

Illustration 35 Screen dump of page 1. When clicking "Log off" ("Afslut" in the top right corner) the system turns directly to page 2 (next example). Illustration 36 Screen dump of page 2. The system offers no specific information that the user has successfully been logged off.

Additional usability comments: 1. At some occasions, like the sequence above, it seems to be possible to reenter “View of accounts” (“Kontooversigt”) without having to provide any identification. Though this problem is not directly related to the heuristics we use in the current examination it is however a big problem for security and hence the user's confidence in the system. Basisbank

Intent: To log off Basisbank followed by an attempt to re-log on (4.7.2003).

Steps and actions:

Pp System information/actions User actions Breakdowns ( ) or small problems 1 Title: “View of accounts” (“Kontooversigt”) Click “Log off the No reversal of actions bank” Top menu includes link: “Log off the bank“ (“Log af banken”) 1 Browser pop-up message: “Security alert” Click “Yes” 2 Text: “You have now been logged off” (“Du er nu logget af Click on “Log on” banken”) Menu includes “Log on” (“Log på banken”) 2 Browser pop-up message: “Security warning” Click “OK” 3 Title “Front page” (“Forside”) (Hereafter follows ordinary log on, see Fields: “Reg. and user no.” (scroll menu) and “Access code” (“Reg. table above) og brugernr.” and “Adgangskode”) Links: 4 links including “I have received new codes from the bank” (“Jeg har modtaget nye koder fra banken”) Buttons: “OK” and “Cancel” (“OK” and “Fortryd”)

Examples:

Illustration 37 Screen dump of page 2. Clear system feedback: "You have been logged off".

Additional usability comments: No additional comments. Money transfer

This task concerns transferring money between accounts in two different banks. We perform this task without previously participating in any kind of introduction/demo if such is present.

Danske Bank

Intent: To transfer money from an account in Danske Bank to an account in another bank (31.7.2003).

Steps and actions:

Pp System information/actions User actions Breakdowns ( ) or small problems 1 Title: “View of accounts” ( “Kontooversigt”) Click “Payments” 13 links including “Payments” (“Betalinger”) 2 Title: “Payments” (“Betalinger”) Click “Transfer” 9 links including “Transfer” (“Kontooverførsel”) 3 Title: “Transfer” (“Kontooverførsel”) Fill in fields The scroll menu concerning currency is in conflict with simple and natural Fields: “Choose account” (scroll), “Text to extract”, “Amount” Click “OK” dialogue. There is no need for it since (including currency scroll menu), “Date” (“Vælg konto”, “Tekst på only one currency can be chosen kontoudskrift”, “Beløb”, “Dato”) Error prevention could be improved. Check box: “Forward receipt” (“Tilsend kvittering”) The field “Amount” holds for instance Menu link: “Transfer on behalf of” (“Overførsel på vegne af”) room for as much as 20 characters (both letters and digits) Fields: “Choose recipient” (scroll), “Or fill in account number”, “Text on recipient's extract” (“Vælg modtager”, “Eller indtast The labels and titles of the fields could kontonummer”, “Tekst på modtagers kontoudskrift”) be clarified Check box and field: “Save as recipient” (“Gem som fast beløbsmodtager”) Menu link: “Message” (“Meddelelse”) Buttons: “OK” and “Clear” (“OK” and “Ryd felter”) 4 Text: “Getting information. Please wait...” (“Henter oplysninger – No user action is Poor feeling of internal locus of control vent venligst...”) needed. The window closes automatically in ½ a sec. 3 Title: “Transfer” (“Kontooverførsel”) No user action needed. Poor feeling of internal locus of control The following pop-up Fields: “Choose account” (scroll), Text to extract”, “Amount” message pops up (including currency scroll menu), “Date” (“Vælg konto”, “Tekst på automatically in ½ a kontoudskrift”, “Beløb”, “Dato”) sec. Check box: “Forward receipt” (“Tilsend kvittering”) Menu link: “Transfer on behalf of” (“Overførsel på vegne af”) Fields: “Choose recipient” (scroll), “Or fill in account number”, “Text on recipient's extract” (“Vælg modtager”, “Eller indtast kontonummer”, “Tekst på modtagers kontoudskrift”) Check box and field: “Save as recipient” (“Gem som fast beløbsmodtager”) Menu link: “Message” (“Meddelelse”) Buttons: “OK” and “Clear” (“OK” and “Ryd felter”) 3 Pop-up message: “Digital signature” (“Digital underskrift”) Fill in password The phrase “Digital signature” (“Digital underskrift”) has not been Text: (excerpt) “Do you accept the following?” (“Vil du acceptere Click “OK” used before følgende?”) The user has to remember a password Field: “Password” (“Kodeord”) which increases load on mental memory Buttons: “OK” and “Cancel” (“OK” and “Annuller”) After this point there is no easy reversal of actions 4 Text: “Getting information. Please wait...” (“Henter oplysninger – No user action is Poor feeling of internal locus of control vent venligst...”) needed. The window closes automatically in ½ a sec. 3 Title: “Transfer” (“Kontooverførsel”) Poor sense of internal locus of control Fields: “Choose account” (scroll), Text to extract”, “Amount” (including currency scroll menu), “Date” (“Vælg konto”, “Tekst på No sense of closure kontoudskrift”, “Beløb”, “Dato”) Check box: “Forward receipt” (“Tilsend kvittering”) Menu link: “Transfer on behalf of” (“Overførsel på vegne af”) Fields: “Choose recipient” (scroll), “Or fill in account number”, “Text on recipient's extract” (“Vælg modtager”, “Eller indtast kontonummer”, “Tekst på modtagers kontoudskrift”) Check box and field: “Save as recipient” (“Gem som fast beløbsmodtager”) Menu link: “Message” (“Meddelelse”) Buttons: “OK” and “Clear” (“OK” and “Ryd felter”)

Examples:

Illustration 38 Screen dump of pop up-massage on page 3. At this point the user has to sign the order. Illustration 39 Screen dump of page 3 (after the transfer). Note the diminutive feedback marked by green dot in the very bottom of the page: “The transfer has been executed”.

Additional usability comments: 1. In order to get proof that this task has indeed been accomplished the user has to enter “View of accounts” after the sequence. This is poor system feedback. Nordea

Intent: To transfer money from an account in Nordea to an account in another bank (7.8.2003).

Steps and actions:

Pp System information/actions User actions Breakdowns ( ) or small problems 1 Title: “View of accounts” (“Kontooversigt”) Click “Payments and transfers” 10 links including “Payments and transfers” (“Betalinger og overførsler”) 1 Scroll menu unfolds 6 links including “Transfer Click “Transfer money” money” (“Overfør penge”) 2 Title: “Transfer money” (“Overfør penge”) Fill in fields The number 30 in the message “Message to recipient (Max. 40 characters) 30” is Fields: “To be deducted from” (scroll menu of Click “Send” confusing accounts), “Text to own extract” (“Hæves på”, “Tekst til eget kontoudskrift”) It is possible to fill in 13 digits in field: “Amount” but the size of the field only Radio buttons: “Once” and “Several times” (“En allows the user to view 11 digits at a time – gang” and “Flere gange”) poor error prevention Fields: “Amount”, “Date of payment”, “Receiver”, If the user has several accounts choosing “Name”, “Registration number”, “Account number” between account numbers may increase (“Beløb”, “Betalingsdato”, “Beløbsmodtger”, memory load “Navn/Kendenavn”, “Reg.”, “Kontonr.” ) Check box. “Save recipient” (“Gem beløbsmodtager”) Field: “Message to recipient (Max. 40 characters) 30” (“Meddelelse til beløbsmodtager (Max. 40 tegn) 30”) Buttons: “Next transfer”, “Send”, “Cancel” (“Næste overførsel”, “Send”, “Annullér”) 3 Title: “Confirm transfer” (“Bekræft overførsel”) Fill in password At this point the transfer cannot be reversed (view of order) Click “Send” The user has to remember a password which increases memory load Field: “Fill in personal password” (“Indtast personlig kode”) Buttons: “Send”, “Cancel” and “Print” (“Send”, “Annuller”, “Udskriv”) 4 Title: “Transfers” (“Overførsler”) (Shows the order marked with a check mark. See example below) Examples.

Illustration 40 Screen dump from page 4. This is how Nordea effectively closes the sequence.

Additional usability comments: 1. The use of the term “Send” in stead of “OK” is problematic. It is not a natural choice of word to describe the action “acceptance”. Jyske bank

Intent: To transfer money from an account in Jyske Bank to an account in another bank (4.7.2003).

Steps and actions:

Pp System information/actions User Breakdowns ( ) or small actions problems 1 Title: “View of accounts” (“Kontooversigt”) Click “Transfer to others” 5 menu buttons including “Transfer to others” (“Overførsel til andre”) 2 Title: “Transfer to others” (“Overførsel til andre”) Fill in fields The information “Transfer from XXXXX current account XX.XX DKK” Field: “Old payments” (scroll menu) (“Gamle betalinger”) Click “OK” does not appear in a logic place of order Text: “Transfer from XXXXX current account XX.XX DKK” (“Overførsel fra kontonummer XXXXX Folio XX,XX kr”) Fields: “Fill in text to extract of account”, “Amount”, “Dare of transfer” (default: Today), “Transfer to”, “Fill in message to receiver” (“Indtast evt. Tekst til kontoudtog”, “Beløb”, Udførelsesdato”, “Overførsel til”, “Indtast evt. meddelelse til modtager”) Radiobuttons: (click either yes or no) “Do you wish to save payment for the future?”, “Do you wish to send the message to the receiver immediately”, “Do you wish to receive a receipt?” (“Ønsker du at gemme betalinger til senere brug”, “Ønsker du at mdtager får tilsendt en nota med meddelselsen med det samme?”, “Ønsker du at få tilsendt en kvittering for overførslen?”) Buttons: ”OK” and “Clear” (“OK” and “Nulstil”) 3 Title: “Interim confirmation of order” (“Foreløbig ordrebekræftigelse”) Fill in key Text: (the confirmation of oder) Click “OK” Text: “Confirm the new order with a key. Does your key card no. XXXXXX have a key no. XX.XX?” (“Bekræft din nye ordre med en nøgle. Finder nøglenr. XX.XX på dit nøglekort XXXXXX?”) Field: “Yes - fill in the 4 character key at XX.XX” (“Ja - indtast den 4 cifrede nøgle der står ved XX.XX”) Text: “No – do not fill in the key. Close Jyske Netbank and contact your branch” (“Nej – du må ikke indtaste din nøgle. Afslut Jyske Netbank og kontakt din Jyske Bank afdeling”) Buttons: “OK” and “Cancel” (”OK” and “Fortryd”) 4 Text: (excerpt) “The order has been carried through” (“Ordren er Click “OK” Action can no longer be reversed gennemført OK”) Button: “OK” (“OK”) 1 Title: “View of accounts” (“Kontooversigt”) 5 menu buttons including “Transfer to others” (“Overførsel til andre”)

Examples: Illustration 41 The screen dump of page 2 shows the fields the user has to fill in to transfer money between accounts. Additional usability comments: No additional comments. Skandiabanken

Intent: To transfer money from an account in Skandiabanken to an account in another bank (29.7.2003).

Steps and actions:

Pp System information/actions User Breakdowns ( ) or small problems actions 1 Initial page shows “View of account” (”Kontooversigt”) Click “Payments” Among the buttons in the left menu is “Payments” (“Betalinger”) 1 Scroll menu unfolds: Click “Transfer Inconsistent use of words to describe transfer (Danish between other words used in the original text: “Overførsel”, “Transfer between own accounts”, “Transfer between accounts” “Betaling” and “Oprettelse”) other accounts”, “Inpayment form”, “Future payments”, “Search payments” (“Overførsel - egne konti”, Overførsel - andre konti”, “Indbetalingskort”, “Kommende betalinger”, “Søg i betalinger” ) 2 Title: “Create extern transfer“ (“Opret ekstern Fill in fields Inconsistent use of words to describe transfer (Danish overførsel”) words used in the original text: “Overførsel”, Click “Next” “Betaling” and “Oprettelse”) Text: “Get standard payment” scroll menu containing previous payments) (“Hent standard betaling”) User may have to chose between several accounts from which to withdraw which increases memory load Fields: “Amount”, “Withdraw from account” (scroll menu), “To account – reg. no and account no.”, “Date of The terms payment and transfer are used inconsistently payment (ddmmyyyy) – blank mean as quickly as possible”, “Text for own extract (max 20 char.)”, “Text The system requires the user to fill in registration for receiver's extract” (choose between field and scroll number and account number in the same field which menu)) (“Beløb”, “Hæves på konto”, “Til konto – reg.nr. may cause confusion since there is no guidance whether og kontonr.”, “Betalingsdato (ddmmåååå) – blank the digits e.g. should be filled in hyphenated or not betyder hurtigst muligt”, “Tekst til egen kontoudskrift Error prevention with regard to the filling in of (max 20 tegn)”, “Tekst til modtagers kontoudskrift”) registration number and account number could be Check boxes: 6 boxes includes e.g. “Wishes to fill in improved by providing a separate 4 spaced field to more text” (“Ønsker at angive mere tekst”) registration number or providing a hyphen within the current field Button: “Next” (“Næste”) 3 Title: “Accept order” (“Godkend oprettelse”) Fill in signature Inconsistent use of words to describe transfer (Danish words used in the original text: “Overførsel”, Text: (shows a view of the order) Click “OK” “Betaling” and “Oprettelse”) Fields: “Signature code”, “Signature file”(shows the default path), “Underskriftkode”,“Underskriftsfil”) Button: “OK” (“OK”) 4 Text: “The transfer has been made” ( “Betalingen er Click “OK” No chance to reverse action oprettet”) Inconsistent use of words to describe transfer (Danish Button: “OK” (“OK”) words used in the original text: “Overførsel”, “Betaling” and “Oprettelse”) 2 Title: “Create extern transfer“ (“Opret ekstern Inconsistent use of words to describe transfer (Danish overførsel”) words used in the original text: “Overførsel”, “Betaling” and “Oprettelse”)

Examples: Illustration 42 Screen dump of page 2. Despite the fact that the account holder is only allowed to transfer 2.000 DKK each day he can actually type in 20 digits in the "Amount" field. Additional usability comments: 1. Due to the lack of “Clear”-buttons the system fails to provide easy reversal of actions with regard to e.g. filling in the fields. Reversal of actions and exit of specific tasks is provided solely by the browser. 2. With regard to error prevention the system provides several scroll menus where the user can choose standard payments or account numbers. The size of some of the fields may however cause problems since e.g. the field to fill in the amount has room for 20 digits and it seems unlikely that any ordinary account holder should desire to transfer 99,999,999,999,999,999,999 DKK from his PC. Especially since the bank has a limit to the amount of money the user can transfer in one day. Sydbank

Intent: To transfer money from an account in Sydbank to an account in another bank (30.7.2003).

Steps and actions:

Pp System information/actions User Breakdowns ( ) or small actions problems 1 Title: “View of accounts” (“Kontooversigt”) Click “Daily The labels are not sufficiently clear finances” Top menu: “View”, “Daily finances”, “Investment”, “Housing”, “Contact”, “Guidance & appliance”, “Security” (“Overblik”, “Daglig økonomi”, “Investering”, “Bolig”, “Kontakt”, “Vejledning & tilmelding”, “Sikkerhed”) 1 Scroll menu unfolds 4 links including “Payments and transfers” Click (“Betalinger & overførsler”) “Payments and transfers” 1 Scroll menu unfolds 4 links including “Transfers” (“Overførsler”) Click “Transfers” 2 Title: “Transfer” (“Overførsel”) Fill in fields, Lack of help to how to fill out “Date of leaving check payment” Fields: “Use template or transfer to own account” (chose from scroll boxes menu), “Withdraw from” (chose from scroll menu), “Text to own unchecked and Text uses “Transfer” and “Payment” statement of account”, “Receiver”, “Account number”, “Amount” (chose radio buttons inconsistently currency from scroll menu), “Date of payment”, “Text to receiver” unmarked (“Benyt gemt betaling eller overør til egen konto”, “Hæves på”, “Tekst The radio button “Create as a single til engen kontoudskrift”, “Beløbsmodtager”, “Kontonummer”, “Beløb”, Click “OK” transfer” could be removed to increase “Betalingsdato”, “Tekst til beløbsmodtager”) simplicity. Not marking the radio button “Create as multiple transfers” says exactly Extra fields: “On behalf of” and “Message” (“På vegne af” and the same. It causes unnecessary memory “'Meddelelse”) load Check boxes: “Save payment for the future”, “Transfer to User may have to chose between several unauthenticated payments and authenticate late”, “Receipt (5.00 DKK)” accounts numbers in order to chose from (“Gem betaling til fremtidig brug”, “Overfør til ubekræftede betalinger which account to make the withdrawal og bekræft senere”, “Kvittering (5,00 kr)”) Radio buttons: “Create as a single transfer” (default) and “Create as multiple transfers”) (”Opret som engangsoverførsel”, “Opret som flergangsoverførsel”) Buttons: “OK” and “Clear” (“OK” and “Nulstil”) 3 Title: “Transfer” (“Overførsel”) Fill in password Password increases cognitive memory load Text: “Confirm payment” (“Bekræft betaling”) Click “OK” Text uses “Transfer” and “Payment” inconsistently Text: (summary of the entire order) Field: “Confirm by filling in your password and clicking ok” (“Bekræft ved at indtaste kodeord og klikke ok”) Button: “OK” and “cancel” (“OK” and “Fortryd”) 4 Text: “Your password has been accepted. The system is currently getting (No user action Poor sense of internal locus of control hold of your certificate. It will only take a moment before you are logged required. on” ( “Dit kodeord er accepteret. Systemet er i øjeblikket i færd med at Window closes hente dit certifikat. Det varer kun et øjeblik, så er du logget på”) automatically in 1.5 seconds) Text: “Note: After an idle period the system will automatically end your session and you will be asked to log on once again ” (“Efter nogen tid uden aktivitet på systemet, vil din login automatisk blive afsluttet og du vil blive bedt om at logge på igen”) Button: “Cancel” (“Afbryd”) 2 Title: “Transfer” (“Overførsel”) Poor system feedback Fields: “Use template or transfer to own account” (chose from scroll Insufficiently information as to whether menu), “Withdraw from” (chose from scroll menu), “Text to own or not the transaction has occurred statement of account”, “Receiver”, “Account number”, “Amount” (chose currency from scroll menu), “Date of payment”, “Text to receiver” No opportunity to reverse action. (“Benyt gemt betaling eller overør til egen konto”, “Hæves på”, “Tekst Text uses “Transfer” and “Payment” til engen kontoudskrift”, “Beløbsmodtager”, “Kontonummer”, “Beløb”, inconsistently “Betalingsdato”, “Tekst til beløbsmodtager”) Extra fields: “On behalf of” and “Message” (“På vegne af” and “'Meddelelse”) Check boxes: “Save payment for the future”, “Transfer to unauthenticated payments and authenticate late”, “Receipt (5.00 DKK)” (“Gem betaling til fremtidig brug”, “Overfør til ubekræftede betalinger og bekræft senere”, “Kvittering (5,00 kr)”) Radio buttons: “Create as a single transfer” (default) and “Create as multiple transfers”) (”Opret som engangsoverførsel”, “Opret som flergangsoverførsel”) Buttons: “OK” and “Clear” (“OK” and “Nulstil”)

Examples:

Illustration 43 Screen dump of page 2. The fields to fill in in order to conduct a money transfer.

Additional usability comments: 1. In the end of this sequence the user will be quite confused as to whether or not the transfer did actually take place. Consulting the view of accounts will however clear out this confusion: Immediately after the transfer the view of account show that the right amount has been subtracted. It is however needless to say that this need for the user to check up on the status of the system is a big usability problem. Basisbank

Intent: To transfer money from an account in Basisbank to an account in another bank (31.7.2003).

Steps and actions:

Pp System information/actions User actions Breakdowns ( ) or small problems 1 Title: “View of accounts” (“Kontooversigt”) Click “Payments/transfer” Menus: “Deposit”, “Payments/transfer”, “Credit & loan”, “Basis invest”, “Your dialogue with us”, “Basisbank service” (“Indlån”, “Betalinger/overførsler”, “Kredit & Udlån”, “Basisinvest”, “Din dialog med os”, “Basis bankservice”) 1 Scroll menu unfolds 10 links including “Transfer amount to others” Click “Transfer amount (“Overfør beløb til andre”) to others” 2 Title: “Transfer to other accounts” (“Overfør til andre konti”) Fill in fields Poor error prevention in field “Amount”. The system separates kroner Fields: (From) “Account” (remembers the users different account Click “Continue” and ører into two fields, but accepts an numbers), “Amount”, “Date of transfer” infinite number of characters in the field ((Fra): “Konto”, “Beløb”, “Overførselsdato”) kroner (or at least 60 characters according to this investigation) Check box: “Get receipt (costs 25DKK)” (“Få tilsendt kvittering (koster 25 dk)”) User may have to chose between several sender account numbers which increases Fields: (To) “Account”, “Text to extract – max. 20 characters” memory load ((Til) “Konto”, “Tekst til kontoudskrift – maks. 20 tegn”) Menu links: “Save in recipient register”, “Send receipt to recipient (costs 25 DKK)” (“Gem i modtagerregister”, “Send kvittering til modtager (koster 25 kr)”) Buttons: “Reset” and “Continue” (“Nulstil” and “Videre”) 3 Title: “Check your information” (“Tjek dine oplysninger”) Click “Accomplish” The title is misleading Text: (overview of the entire transfer order) Buttons: “Correct” and (“Accomplish” (“Tilpas/ret” and “Gennemfør”) 3 Pop-up message: Fill in password User has to remember password which increases memory load “Signing – fill in password” (“Signering – Indtast Click “OK” underskriftskode”) After clicking “OK” there is not reversal of actions Buttons: “OK” and “Cancel” (“OK” and “Fortryd”) 4 Title: “Transfer to others” (“Overførsel til andre”) Not sufficient feedback Text: (overview of the entire transfer order) Poor sense of closure. The text does not state that the transfer has taken place Buttons: “Transfer more money”, “Print”, “Back to my front page” (“Overfør flere penge”, “Udskriv”, “Til bage til min forside”) Examples:

Illustration 44 Screen dump of page 4. This page does not state whether or not the transfer has been completed.

Additional usability comments: 1. The system does not provide an easily accessible information about whether or not the task is completed. To see if the transfer has indeed taken place the user will have to click the button “Back to my front page” (“Tilbage til min forside”) at page 4. He then returns to “View of accounts” where it is evident that the amount has been deducted from his account. This extra workload is a big usability problem. Weak passwords

At this point we examine how the system reacts to passwords that do not fulfill the system protocol (such as aaaaaaaa when the system requires small and capital letters or letters and digits. Additionally we will try to type in 7 characters when the system requires min. 8). We expect to receive some error messages and focus on the nature of these to decide whether they be simply stating the error or trying to help the user to solve the problem. In addition to the ordinary paragraphs such as “Intent” and “Steps and actions” we have added the paragraph “Protocol” below. “Protocol” documents how each system describes it's password requirements.

Danske Bank

Protocol: “The password has to be min. 8 and max. 16 characters. Choose a password that is a nonsense word that does not have any meaning. Use a mixture of digits and letters” (original text: “Kodeordet skal være på mindst 8 og maksimalt 16 karakterer. Vælg et kodeord, der er et vrøvleord som ikke umiddelbart giver mening. Anvend en blanding af tal og bogstaver”).

Intent: To change the pasword of the account in Danske Netbank (5.8.2003)

Steps and actions:

Pp System information/actions User Breakdowns ( ) or small problems actions 1 Title: “View of account” (“Kontooversigt”) Click “Settings” Consistency problem: The button is difficult to find since most other buttons is located in the side menu 9 top menu buttons including “Settings” (“Indstillinger”) 2 Title: “Settings” (“Indstillinger”) Click “Security” The text and title could be more specific Text: (excerpt) “Under 'Settings' you can change the general settings” (“Under indstillinger kan du skifte de generelle indstillinger”) 7 buttons in the side menu including “Security” (“Sikkerhed”) 3 Text: “Getting information. Please wait...” (“Henter No user action Poor feeling of internal locus of control oplysninger. Vent et øjeblik...”) needed. Window closes automatically in ½ a sec. 4 Title: “Security” (“Sikkerhed”) Click “Change The system does not provide sufficiently help to password” create strong passwords Text: (excerpt) “Here you can change your personal password which you use to log on or sign a transfer. Please note...” (followed by 2 pieces of advice concerning creation of password, 1 piece of advice concerning use of password from other PCs and 1 piece of advice concerning safety copy of key) (“Her kan du skifte dit personlige kodeord som du benytter til at logge på eller underskrive en transaktion. Bemærk: ...”) Buttons: “Change password” and “Save key” (“Skift kodeord” and “Gem nøgle”) 4 Pop-up message: “Change password” (“Skift kodeord”) Fill in present Having to remember present password increases password user's memory load Text: “Agreement number XXXXXX” (“Aftale nummer XXXXXX”) Fill in new 7 character Fields: “Present password”, “New password”, “Confirm password: password” (“Nuværende kodeord”, “Nyt kodeord”, aaaaaaa “Bekræft kodeord”) Click “OK” Button: “OK” and “Cancel” (“OK” and “Annuller”) 4 Pop-up message: “The password has to be min. 8 and max. Click “OK” Poor sense of internal locus of control 16 characters” (“Kodeordet skal være min. 8 og højst 16 tegn”) Button: “OK” (“OK”) 4 Pop-up message: “Change password” (“Skift kodeord”) Delete the old Having to remember present password increases versions of new user's memory load Text: “Agreement number XXXXXX” (“Aftale nummer password XXXXXX”) Re-type new 8 Fields: “Present password”, “New password”, “Confirm character password” (“Nuværende kodeord”, “Nyt kodeord”, password: “Bekræft kodeord”) aaaaaaaa Button: “OK” and ”Cancel” (“OK” and “Annuller”) Click “OK” 4 Pop-up message: “Information” (“Information”) Click “No” Poor feedback and lack of consistency: The user has clicked ”Change password” and is informed that Text: “Danske Bank has updated your key. You should soon the key has been changed save a security copy. Do you wish to save a copy now? (“Danske Bank har opdateret din nøgle. Du bør derfor snarest tage en sikkerhedskopi. Ønsker du at tag en sikkerhedskopi nu?”) Buttons: “Yes” and “No”. 4 Pop-up message: “Change password” (“Skift kodeord”) Text: “Agreement number XXXXXX” (“Aftale nummer XXXXXX”) Fields: “Present password”, “New password”, “Confirm password” (“Nuværende kodeord”, “Nyt kodeord”, “Bekræft kodeord”) Button: “OK” and “Cancel” (“OK” and “Annuller”) Examples:

Illustration 45 Screen dump of page 4. The system instructs the user in creating a 8-16 character non-sense password.

Illustration 46 Screen dump of pop-up message on page 4. (Translation: “Danske bank has updated your key. You should save a copy. Do you wish to save a cop now? Yes/No”)

Illustration 47 Screen dump of pop-up message on page 4. This message pops up when the user clicks F1 in the 2nd field of pop-up message “Change password” (“Skift kodeord”) on page 4. The message says: “The new password must be min. 8 and max. 16 characters. You can use letters, digits, special characters and combination of these. Note that the system is case sensitive. The password must be used with every log on”. This help is indeed a more thorough explanation of the advise offered at page 4 and one wonders why the user is not offered this help instantly. Additional usability comments: 1. It is not absolutely clear which part of the information about the protocol is concrete system requirements and which is just plain good advise. According to the sequence above the only real requirement is the one concerning the length of the password. This fact is more clearly explained in the help message of the pop-up message “Change password” (“Skift kodeord”) (see screen dump above). Nordea

Protocol: Nordea informs that passwords must be 8-32 characters, both letters and digits. The system is not case sensitive and accepts space but neither special characters such as full stop or the Danish letters æ, ø and å.

Intent: To change the password used for the account in Nordea (7.8.2003).

Steps and actions:

Pp System information/actions User actions Breakdowns ( ) or small problems 1 Title: “View of account” (“Kontooversigt”) Click “Security” Menu holds 10 links including “Security” (“Sikkerhed”) 1 Scroll menu unfolds. Click “Change personal password” Menu holds 5 links including “Change personal password” (“Skift personligt kodeord”) 2 Title: “Change personal password” (“Skift personligt Fill in present password The term “Send” on one of the buttons may kodeord”) not be a logical label. Fill in new password: aaaaaaa (7 Fields: “Fill in personal password”, “Fill in new x a) The user has to remember his password personal password”, “Repeat new personal which increases memory load password” (“Indtast personligt kodeord”, “Indatst Click “Send” nyt personligt kodeord”, “Gentag nyt personligt kodeord”) Buttons: “Send” and “Cancel” (“Send” and “Annuller”) Link: “How to make a strong personal password” (“Sådan her laver du et godt personligt kodeord”) 2 Additional remark upon page 2: (All fields have been erased) There is no need to delete all the content of all the fields Error message situated below the fields in question: Fill in present password “Personal password must be at least 8 characters” The error message simply states the problem (“Personligt kodeord skal være mindst 8 tegn”) Fill in new password: aaaaaaaa (8 x a) Click “Send” 2 Additional remark upon page 2: (All fields have been erased) Poor error message – no explanation of how to mend this error Error message situated below the fields in question: Fill in present password “The personal password does not fulfill the protocol” There is no need to delete all the content of (“Kodeordet lever ikke op til bankens regelsæt”) Fill in new password: hundhund all the fields Click “Send” 2 Additional remark upon page 2: (All fields have been erased) Poor error message – no explanation of how to mend this error Error message situated below the fields in question: Fill in present password “The personal password does not fulfill the protocol” There is no need to delete all the content of (“Kodeordet lever ikke op til bankens regelsæt”) Fill in new password: 123456789 all the fields Click “Send” 2 Additional remark upon page 2: (All fields have been erased) Poor error message – no explanation of how to mend this error Error message situated below the fields in question: Fill in present password “The personal password does not fulfill the protocol” There is no need to delete all the content of (“Kodeordet lever ikke op til bankens regelsæt”) Fill in new password: hundeflojte all the fields Click “Send” 2 Pop-up message: (warning sign ) “Personal password Click “OK” No notice of confirmation should hold a was changed” (“Personligt kodeord blev ændret”) warning sign Button: “OK” (“OK”) 2 Title: “Change personal password” (“Skift personligt kodeord”) Fields: “Fill in personal password”, “Fill in new personal password”, “Repeat new personal password” (“Indtast personligt kodeord”, “Indatst nyt personligt kodeord”, “Gentag nyt personligt kodeord”) Buttons: “Send” and “Cancel” (“Send” and “Annuller”) Link: “How to make a strong personal password” (“Sådan her laver du et godt personligt kodeord”)

Examples:

Illustration 48 Screen dump from link: “How to create a strong personal password” (“Sådan laver du et godt personligt kodeord”) on page 2. Nordea provides a thorough information about both strong and weak passwords.

1. Additional usability comments: 2. Though Nordea actually does provide a thorough information about both strong and weak passwords it seems to be a problem that this information is optional to the user. The information about passwords include important information about the system's protocol which every user should know about. The only way the user can get an explanation of the error message on for instance page 2 is if he examines the information about system's protocol. Thus the systems does not provide the user with sufficiently help. 3. Throughout the entire dialogue the system uses the phrase “Send” in stead of the more common “OK”. This is an unfortunate divergence from a consistent terminology upon which most systems seems to agree. Jyske bank

It is not possible to change the password issued by Jyske Bank. The user has to remember a pre- made 9 character password (letters and digits). This puts quite a lot of pressure on mental memory load. With regard to the nature of error messages that we wished to examine the investigation shows that error messages often occur with a blinking red warning sign on the screen. This may to some users imply system breakdown or fatal errors.

Examples:

Illustration 49 This screen dump from Jyske Netbank shows an example of the blinking red warning sign. Skandiabanken

Protocol: The system states that it requires a password of 8-16 characters both letters (but the Danish æ,ø and å) and digits.

Intent: To change the existing password (30.7.2003).

Steps and actions:

Pp System information/actions User Breakdowns ( ) or small problems actions 1 Title: “View of accounts” (“Kontooversigt”) Click“User “User functions” is not sufficiently clear to describe to functions” button Left menu includes the button “User functions” (“Brugerfunktioner”) 1 Scroll menu unfolds into 4 buttons including “Change Click “Change signature” (“Skift underskriftskode” ) signature” 2 Browser pop-up message: “Security warning” Click “OK” Not user's language No sense of internal locus of control 2 Title: “Change signature” (“Skift underskriftskode”) Fill in fields No examples and no help to create a strong password or explaining the importance of a password Fields: “Present signature code”, “New signature code”, New password “Repeat signature code” and “Signature file” (show the consists of 6 Possibility to reverse actions could be improved default path) (“Nuværende underskriftskode”, “Ny random characters underskriftskode”, “Genindtast underskriftskode”, both letters and “Underskriftsfil) digits Click “OK” 3 Title: “Error message” (“Fejlmeddelelse”) Click “OK” Text: “New signature code must be 8-16 characters” (“Ny underskriftkode skal være på 8-16 tegn”) Button: “OK” (“OK”) 2 Title: “Change signature” (“Skift underskriftskode”) Fill in fields. New No examples and no help to create a strong password or password explaining the importance of a password Fields: “Present signature code”, “New signature code”, consisting of 8 “Repeat signature code” and “Signature file” (show the random letters Possibility to reverse actions could be improved default path) (“Nuværende underskriftskode”, “Ny underskriftskode”, “Genindtast underskriftskode”, Click “OK” “Underskriftsfil) 3 Title: “Error message” (“Fejlmeddelelse”) Click “OK” Text: “New signature code must include both letters and digits” (“Ny underskriftkode skal indeholde både bogstaver og tal”) Button: “OK” (“OK”) 2 Title: “Change signature” (“Skift underskriftskode”) Fill in fields. New No examples and no help to create a strong password or password: explaining the importance of a password Fields: “Present signature code”, “New signature code”, 1aaaaaaa “Repeat signature code” and “Signature file” (show the Possibility to reverse actions could be improved default path) (“Nuværende underskriftskode”, “Ny Click “OK” underskriftskode”, “Genindtast underskriftskode”, “Underskriftsfil) 4 Title: “Change signature” (“Skift underskriftskode”) Text: “The signature file has been changed” (“Underskriftskoden er nu ændret”)

Examples: Illustration 50 Screen dump of page 2. This page provides some information about the consequences of changing a password. Additional usability comments: 1. Possibilities of exits and reversal of actions are provided solely by the browser. Sydbank

Protocol: Sydbank informs that passwords have to be min. 8 characters long.

Intent: To change the password accessing the account at Sydbank (5.7.2003)

Steps and actions:

Pp System information/actions User actions Breakdowns ( ) or small problems 1 Title: “View of accounts” (“Kontooversigt”) Click “Security” The term and label “Security” may not be sufficiently clear 7 links including “Security” (“Sikkerhed”) 2 Title: “Security solution” (“Sikkerhedsløsning”) Fill in old password Lacks information about password requirements or how to create strong passwords Text: (excerpt) “You are logged on using a certificate Fill in new password: solution...” (“Du er logget på med en certifikatløsning...”) aaaaaaa (7 x a) Separate information box: “Change of password” Click OK (“Ændring af kodeord”) Fields: “Old password”, “New password”, “Confirm new password” (“Gammelt kodeord”, “Nyt kodeord”, “Bekræft nyt kodeord”) Buttons: “OK”, “Cancel” (“OK” and “Fortryd”) 2 New text appears on top of the information box in Clear the previous question: typed “New password” “The new password must be at least 8 characters long” (“Det nye kodeord skal være mindst 8 karakterer langt”) Re-type “New password”: aaaaaaaa (8 x a) Click “OK” 2 Browser pop-up message: “Security warning” Click “Yes” Poor sense of internal locus of control Not the user's language 2 New text appears on top of the information box in No user action The feedback could be improved. The change in question: needed. The window text is not necessarily clear to the user closes automatically “The system is carrying out the desired changes. Please in 1 sec. wait...” (“Systemet er nu ved at foretage de ønskede ændringer. Vent et øjeblik...”) 2 New text appears on top of the information box in Click “OK” The feedback could be improved. The change in question: text is not necessarily clear to the user “The password has been changed” (“Kodeordet er There seems to no need for the “OK” button, which ændret”) complicates the dialogue unnecessarily Button: “OK” (“OK”) 2 Title: “Security solution” (“Sikkerhedsløsning”) Text: (excerpt) “You are logged on using a certificate solution...” (“Du er logget på med en certifikatløsning...”) Separate information box: “Change of password” (“Ændring af kodeord”) Fields: “Old password”, “New password”, “Confirm new password” (“Gammelt kodeord”, “Nyt kodeord”, “Bekræft nyt kodeord”) Buttons: “OK”, “Cancel” (“OK” and “Fortryd”) Examples:

Illustration 51 Screen dump of page 2. To change the password the user has to fill in the top box on the right.

Illustration 52 Screen dump from the help offered at page 2. No information about how to create strong passwords are offered.

Additional usability comments: 1. In between the many changes in the header of the information box on page 2 the page flicks as if changing. This is quite confusing to the user since there is hardly any noticeable change on the page before and after such a flick. This is a poor system feedback, which eventually results in a poor sense of internal locus of control. Basisbank

Protocol: The system informs that passwords need to be 8-12 characters long, and contain both letters (A-Z) and digits (1-9).

Intent: To change the password used to access Basisbank (4.7.2003).

Steps and actions:

Pp System information/actions User actions Breakdowns ( ) or small problems 1 Title: “View of accounts” (“Kontooversigt”) Click “Basisbank Not sufficient help. The label is not clear in service” it's terminology Menus: “Deposit”, “Payments/transfer”, “Credit & loan”, “Basis invest”, “Your dialogue with us”, “Basisbank service” (“Indlån”, “Betalinger/overførsler”, “Kredit & Udlån”, “Basisinvest”, “Din dialog med os”, “Basis bankservice”) 1 Scroll menu unfolds. 12 links including “Change password” Click “Change (“Skift underskriftskode”) password” 2 Title: “Change password” (“Skift underskriftskode”) Fill in present The system does not provide sufficient help password Fields: “Present password”, “New password”, “Repeat new The user has to remember his passwords password” (”Nuværende underskriftskode”, “Ny Fill in new passwords which increase memory load underskriftskode”, “Gentag ny underskriftskode”) consisting of 7 characters (6 letters Buttons: “OK”, “Cancel” (“OK” and “Fortryd”) and 1 digit) Click “OK” 2 Pop-up message: “The password has to be more than 7 Click “OK” characters. Please correct” (“Underskriftskoden skal være mere end 7 tegn lang. Ret venligst”) Button: “OK” (“OK”) 2 Title: “Change password” (“Skift underskriftskode”) Fill in new password: No error protection – the fields can hold 19 aaaaaaaaaaaaaaaaa characters in spite the system informs that Fields: “Present password”, “New password”, “Repeat new aa (19 x a) max.12 are accepted password” (”Nuværende underskriftskode”, “Ny underskriftskode”, “Gentag ny underskriftskode”) Click “OK” Buttons: “OK”, “Cancel” (“OK” and “Fortryd”) 3 Title: “Message: Your password has been changed” Lack of consistency between what input the (“Meddelselse: Din underskriftskode er nu blevet ændret”) system tells it requires and what it in fact does accept Button: “Back” (“Tilbage”)

Examples:

Illustration 53 Screen dump of pop-up error message on page 2.

Additional usability comments: No further comments. References

Nielsen, J., Molich; R., 1990. Heuristic evaluation of user interfaces. In: Proceedings of the ACM CHI ‘90 Conference on Human Factors in Computing systems, ACM Press, New York, pp. 249-256. Shneiderman, B., 1998. Designing the User Interface: Strategies for Effective Human-Computer Interaction, Third Edition, Addison Wesley, Reading, MA.