sign in sign up
<<
Home , Cognitive science

A survey on in light of Vision and Cognitive

Corresponding author: Jose F Rodrigues-Jr, [email protected] Luciana A M Zaina+, Maria C F de Oliveira, Bruno Brandoli, and Agma J M Traina

May 7, 2018

Abstract

Information Visualization techniques are built on a context with many factors related to both vision and , making it difficult to draw a clear picture of how data visually turns into comprehension. In the intent of promoting a better picture, here, we survey on vision, cognition, and Information Visualization organized in a theorization named Visual Expression Process. Our theorization organizes the basis of visualization techniques with a reduced level of complexity; still, it is complete enough to foster discussions related to design and analytical tasks. Our work introduces the fol- lowing contributions: (1) a Theoretical compilation of vision, cognition, and Information Visualization; (2) Discussions supported by vast literature; and (3) Reflections on visual-

arXiv:1505.07079v2 [cs.GR] 13 May 2016 cognitive aspects concerning use and design. We expect our contributions will provide further clarification about how users and designers think about InfoVis, leveraging the potential of and techniques.

Keywords— Information Visualization, InfoVis, Vision , Cognitive Science

1 1 Introduction

Understanding why and how visual representations work is an important issue addressed in many works on Information Visualization (Dill et al. 2012)(Thomas & Cook 2005). On the track of this issue, in this work, we assume that well-designed visualizations must stimulate visual and cognitive processes in a way that reasoning is amplified. The effective promotion of such reasoning depends on principles mastered in the sciences of vision and cognition: vision is the gate through which information derived from graphics reaches the ; cognition refers to the processing that is induced by such graphics. Vision and cognition are closely intertwined, a fact to be considered in the design of visualizations. Accordingly, understanding Information Visualization (InfoVis for short) in light of these sciences may improve design principles that, usually, are performed intuitively. To this end, we review the steps that take place during vision-cognition phenomena when the goal is data analysis. In survey fashion, we compile the literature introducing the following contributions:

• Theoretical compendium: we draw a descriptive relationship between vision, cognition, and Information Visualization;

• Discussions: we debate our rationalizations over extensive literature;

• Reflections: we provide study cases to revisit design practices.

We organize visualization concepts aiming at the needs pointed out by Johnson et al. (2006), who recommend the characterization of how and why visualizations work, and by Scaife & Rogers (1996), who stress the importance of the cognitive aspects underlying visualizations. Furthermore, we review principles for visual representations that, according to Card et al. (1999), are an initial step towards more effective visualization techniques, a demand defended by Wong et al. (2012). We note that, while we build an association between Information Visualization, vision, and cognitive sciences, we do not reach a definitive settlement. This is because neither vision nor cognition are yet fully understood. Rather, constrained to the current state of the art, we introduce an organizational process that discusses use and design.

2 2 Related work

The literature presents several models that lay the basis to Information Visualization. Bertin (1977/1981) introduced the of deriving visual structures from a set of fundamental components. Cleveland & McGill (1984), and Mackinlay (1986), conducted studies on the use- fulness of visual patterns in the form of frameworks for design. Card et al. (1999) follow Bertin by discussing the importance of the spatial substrate. Keim (2002) suggests a taxonomical space for quick referencing; Shneiderman (1996) about the possibilities of visualization and interaction; and Chi (2000) focuses on data transformations. In the realm of design, Buga- jska (2003) deals with spatial and abstract visualizations considering the design guidelines of Tweedie (1997). In the line of works that reflect about the visualization field, van Wijk (2006) systematically discusses visualizations based on a cost-oriented analysis; and Green et al. (2009) research many cognitive and perceptual aspects (Rensink 2000) to build a model and a set of guidelines for design. Patterson et al. (2014) introduces a framework based on vision and cog- nition sciences, similar to ours, focusing on top-down processes. Comparatively, we conduct a complementary bottom-up approach; we translate vision-cognition phenomena into an original vocabulary that may bring such science closer to the design practice. To organize our compilation on vision and cognitive sciences, we depart from the Visualiza- tion Pipeline of Card et al. (1999) to draw the Visual Expression Process, a sequence of events delineated by the possibilities of the visual-cognitive interplay. According to our organization, (1) vision phenomena (pre-attentive stimuli) determines a map of potential interesting objects. Then, attentive selection concentrates on one single element, part of a set of (2) analytical . Such perceptions go through a pattern-matching process that turns them into (3) abstract patterns that, in working , support cognition in combination with domain originated from long-term memory. Finally, leading to (4) cognitive decision sup- port. Nevertheless, our process does not explain the intercourse between vision, cognition, and Information Visualization; this is not feasible considering the current knowledge, and neither it would fit in a single article. The rest of the paper is organized as follows. Section 3 draws a detailed panorama of vision and cognition in the realm of Information Visualization. Section 4 introduces the Visual Expression Process, our organizational theorization; and the last section presents conclusive remarks.

3 3 Concepts on Vision, Cognition, and Visualization

According to Vision Science, the visual process has two stages, namely, the parallel extraction of low-level , called pre-attentive processing, followed by a slower detailed scan. The first stage promotes the major benefit of visualizations, that is, improved data comprehension (Triesman 1985). Meanwhile, the second stage addresses conventional reading practices that do not contribute towards faster cognition, but that are necessary for further analysis. In fact, Ware (2004) states that understanding what is processed pre-attentively is probably the most important contribution that Vision Science can bring to visualization. This two-stage process might be the underlying motivation for the broadly referenced “visu- alization mantra”: overview first, zoom and filter, then details on demand (Shneiderman 1996) – Schneiderman came to this conclusion by means of and empirical experiments, never drawing the vision/cognitive reasons of why this is the case. We discuss this process in the following sections according to our process, presented in section 4.

3.1 Maps of Saliences

In the early stages of vision, the brain deals with the problem of casting potential elements of interest; regions of the scene that should be considered for cognition. To cope with that, a com- plex process takes place so that certain characteristics pop out to the eyes. These characteristics appear as saliences over the scene. They define the so-called maps of saliences, exemplified in Figure 1; the first element of our theoretical organization.

Figure 1: (a) Example of a map of saliences over the Gapminder tool. In the scene, the bubbles with outstanding features stand for saliences corresponding to countries of interest. (b) The result is a map of potential targets that pop out due to their color, or their shape.

The principle of salience is to reinforce the of the areas in the scene whose visual

4 properties contrast with those of their surroundings (Itti & Koch 2001). It is a process that considers different visual aspects, such as position and color, and that ends as it combines them onto a single scene (Nothdurft 2000). The occurrence of salient features stems from their interplay with other stimuli, depending on a context that favors conspicuousness. The brain is highly trained to detect such configura- tions, being able to track salient features in parallel, in real-time, and covering the entire visual field. There is also evidence that this ability is strongly influenced by the task-at-hand, in a top-down (cognition-to-vision) process (Navalpakkam & Itti 2007) (Patterson et al. 2014).

3.2 and Attentive Selection

Although the brain perceives visual targets simultaneously in a map of saliences, it cannot process all of them in parallel. This is considered a prohibitively computational task even to the most sophisticated (Tsotsos 1991). Primates and other animals handle this by restricting the consideration of the objects presented to their eyes: their vision concentrates on small regions considering objects one after the other. This is a serialization process ruled by what is called attention. In other words, once a map of candidates is ready, it is necessary to “filter out” one of these candidates for attention. Attention, here, occurs in accordance with the task that the user is performing; the task determines the information demands and, consequently, what visual stimuli should be extracted from the scene. Trial and error is part of the process, which is iterative. This process of filtering has been modeled as a pyramidal neuronal structure – named selective tuning model (Cutzu & Tsotsos 2003) (Essen et al. 1992). This theoretical model predicts a broad layer of in its first level, narrowing down as it advances to upper layers. The layers intercommunicate through feed-forward and feedback connections and, according to Cutzu & Tsotsos (2003) (Tsotsos et al. 1995), a pyramid of neurons successively performs three stages of processing, illustrated in Figure 2: (a) bottom-up feed-forward, (b) top-down winner- take-all feedback (Lippmann 1987), and (c) bottom-up straight path. This process explains what is broadly referenced as filtering or attentive selection. Following this widely-accepted model, it is worthy to note that, as described, filtering occurs according to both bottom-up and top-down processes. The bottom-up process depends on the visual stimuli, while the top-down process depends on the task at hand and is prioritized (Wolfe 1994). Each process influences the other iteratively. Actually, according to Patterson (2012), the entire visual-cognition interplay in influenced by top-down mechanisms that emanate from

5 working memory, but that may be highly influenced by long-term memory (Woodman et al. 2013). In this work, we focus on the bottom-up process, but there are works that explore higher-level factors related to the top-down process (Patterson et al. 2014).

Figure 2: Three-stage pyramidal visual selection: (a) bottom-up feed-forward, (b) top-down winner-take-all feedback (Lippmann 1987), and (c) bottom-up straight path.

3.3 Cognition, Memory and Vision

After a target is selected, it is potentially useful for “details on demand”, or cognition. In general terms, cognition refers to the acquisition or use of knowledge Cohen (1985), and may occur analytically (conscious and slower) or intuitively (automatic and fast), as defended by the dual theory (Evans 2008)(Evans & Stanovich 2013). Cognition is of special importance in data visualization as it supports analogical reasoning (Patterson et al. 2009). That is, the transfer of inferences from a relationship of elements in one domain (the analogue) to a relationship of elements in another domain (the target). In any case, cognition is intermediated by the memory system. The relationship between memory and cognition is studied by works on Cognitive Architectures, such as ACT-R (Buttner 2010) and Soar (Young & Lewis 1999). In Soar and other theories, the structural configuration of memory roughly reflects the model of Baddeley & Hitch (1974). Working memory includes three components: the central executive module, the phonological loop, and the sketchpad. The central executive module determines the attention focus, guiding the visual system, for example, by top-down biasing the pyramidal selection mechanism. The phonological loop stores information related to sound. The sketchpad (also known as Visual Short-Term Memory – VSTM 1) is associated with the maps of saliences

1although, this is not a consensus

6 discussed in Section 3.1, storing information related to space and to visual features. Following these lines, memory comes to be the main element in supporting cognition as it allows complex mental operations. Consequently, it also supports the practice of data visual- ization. Miller’s (Miller 1956) states that the memory is limited to 7+/-2 elements; more recent, and accepted, works state that this limit is at 4 chunks of elements – a chunk refers to the grouping of elements into larger units based on their meaning (Cowan 2010). Alvarez & Cavanagh (2004) discuss memory limits considering the nature of the elements to be re- membered, demonstrating that, despite the consensus of a (very) limited resource, there is not an ultimate conclusion about the topic. In any case, these limitations are severe because the greater the capacity of an individual’s memory, the more information she/he has available for solving problems (Just & Carpenter 1992). The memory system supports cognition in two ways: by retaining a list of elements for quick referencing, and by assisting in the construction of mental models (Johnson-Laird 1983). According to Johnson-Laird (2010), mental models preserve the relationship between entities by defining that save on logical reasoning, one of the principles behind complex visualization techniques and interaction (Liu & Stasko 2010a). Following the study of Logie (1995), mental models are created in the visuo-spatial sketchpad, a specialization of VSTM. Stimuli encoded into VSTM, ruled by cognition, may activate enduring information known as long-term memory, including facts, meanings, relationships, skills, and procedures (Patterson et al. 2010) – for a deeper discussion on working memory and long-term memory, refer to the work of Rose et al. (2010). The concepts presented in the former sections rely on ideas posed by widely accepted the- ories, among several others, for the visual-cognitive process. The choice for this specific line of has been motivated by its intuitive coherence and scope of influence in the literature. Notwithstanding, other theories are widely referenced, such as the Spotlight (Eriksen & Hoff- man 1973) and the Gradient (Cheal et al. 1994) models. For an ample discussion, refer to the work of Squire (2009).

4 The Visual Expression Process

In this section, we review the practice of visualizing data by considering the concepts presented so far. We organize the relationship among visualization, vision, and cognition according to a framework named Visual Expression Process – Figure 3. Our theoretical organization has

7 four constituents: (1) pre-attentive stimuli, (2) analytical perceptions, (3) abstract patterns, and (4) decision support. For completeness of our survey, Table 1 summarizes previous models found in the literature. Our theorization is inspired by these previous works putting together concepts in a complementary point of view.

Table 1: Summary of previous models on InfoVis.

Model Summary Lohse (1993) An algebraic model to estimate the effort to answer questions based on a visualization. Card et al. A pipeline of how to conduct the visualization (1999) practice. van Wijk (2005) An economic model stated to evaluate efficacy and efficiency. (Ware 2005) A top-down (problem-solving) model that states that what we see in a visualization depends on what we are seeking for. (Hegarty 2011) A descriptive discussion of how visualizations work that culminates into a compilation of principles and perspectives. (Patterson et al. A vision-cognition model that explains visualization 2014) and how to improve it from a top-down perspective.

According to our theorization, (1) pre-attentive stimuli come from the neuronal reaction to light, determining a map of potential interesting objects, or saliences. Then, attention concentrates on one single element, part of a limited taxonomic vocabulary of (2) analytical perceptions. Such perceptions go through a pattern-matching process that turns them into (3) abstract patterns. Finally, abstract patterns in working memory support cognition in combination with domain knowledge originated from long-term memory, leading to (4) decision support. Notice, in the figure, that the arrow in between each pair of the four constituents is bidirectional to indicate that the interplay occurs both bottom-up and top-down, notably in response to the task at hand. Also, notice that after cognition, there might be new information demands, what leads to the redefinition of the visualization by means of new parameters for interaction – as depicted in blue in Figure 3. Following, we present further details.

8 Figure 3: The Visual Expression Process for Information Visualization specified with four constituents. The black bidirectional arrows refer to bottom-up/top-down visual-cognitive pro- cesses. In blue, new parameters of interaction alter the scene iteratively.

4.1 Pre-attentive Stimuli - channels for data encoding

Pre-attentive stimuli impel maps of saliences, as highlighted on the leftmost side of Figure 3. In the work of Rodrigues et al. (2008), the authors verified that such stimuli manifest through position, shape, color, and time. Here, we refer to these factors as channels for data encoding, in the that they encode data into visual stimuli. Although the consideration of four channels is a reductionist classification, it is supported by the literature. About color and texture, Watt (1995) affirms that, just like texture, color is the psychological response to the spectral characteristics of a surface; and that, different surfaces are perceived as having different colors. Furthermore, Motter (1994) observes that, early in visual processing, the incoming information is sorted and grouped according to the similarity of simple shape features, such as orientation or size, and of surface features, such as color, luminance, or texture. The features of each channel span to a large set, but Card et al. (1999) observe that just a limited number of the many existing graphical properties are used for Information Visualization. Healey & Enns (2012) present an extensive survey on pre-attentive features and visualization. A non-exhaustive list of such features is presented in Table 2.

9 Table 2: Classes/channels of pre-attentive features from the perspective of data representation. Pre-attentive class Features (data encoding channel) Position 1D/2D/3D position, stereoscopic depth; Shape line, area, volume, form, orientation, length, width, collinearity, size, curvature, marks, numerosity, con- vex/concave; Color hue, saturation, brightness and texture; Time movement, morphing, blinking, color/light intermit- tence.

Discussion

A well-designed visual representation must present a high overlap between its map of saliences and its (implicit) map of semantic relevance. That is, the design of visual representations is supposed to maximize pre-attentive effects. However, such maximization may not be possible without flexible human intervention over the channels of data encoding. In other words, inter- action, the active redefinition of the channels, is a mandatory need not fully explored in many designs. Liu et al. (2008) point out that many visualization systems are not sufficiently flexible to support user customization and appropriation. In fact, it is not difficult to find visualization tools that are limited in allowing the user to determine how to encode data. In these circum- stances, a user may ask “may I change the positioning order of the elements?”, “can I have each year represented with a different shape?”, “can I color the left group in red?”, or “can I see that animated?”. Each of these examples refers to a particular pre-attentive feature or, as we propose, to a data encoding channel. For instance, consider the seminal system GGobi (Cook & Swayne 2007), which introduced a large set of features if compared to its former version, system XGobi. Still, a brief analysis reveals that there is much to be improved: positioning of views is limited, except for Parallel Coordinates; shape is not an option for coding in the same way that color is; and animation is restricted to scatter plots through touring techniques. Those design issues contrast to what is observed in commercial systems like TIBCO’s Spotfire (http://spotfire.tibco.com/) and Google’s Gapminder (http://www.gapminder.org/), which present higher levels of freedom for each encoding channel, but that, still, do not achieve full appropriation of the scene the way we discuss. Accordingly, the design of visualizations must not only allow users to alter visual attributes,

10 they must incorporate mechanisms to emphasize each channel. This is necessary because a given stimulus (position, shape, color, or time) may fail to capture attention when it is surrounded by other stimuli – this is called . In such circumstance, this given stimulus fails in competing with other more interesting stimuli or, worse, it is ignored (Mack A 1998)(DJ 2000). “Blindness” occurs especially when the viewer has some previous expectation on the scene; in such situations, top-down processes can strongly influence on what is noticed. This issue has been noticed by Patterson et al. (2014), who claim that the design of a visualization technique must provide means to capture attention alerting users of changes. Furthermore, according to the limitations of the memory system, as presented in Section 3.3, memory overloading is a problem that might constrain problem-solving. In fact, according to Wickens (2008), multiple tasks in one same cognitive dimension might decrease performance. One possibility to lessen this drawback is to selectively turn the visual channels on and off. For example, it is possible to conceive a dispersion plot in which size is simply turned off, so to temporarily avoid overlapping – see Figure 4 (b). Similarly, it is possible to turn off color, presenting all the information with black marks, emphasizing the role shape contours – Figure 4(c). It is also possible to include intermittence, or movement, much the way that cyclists do with lights in urban traffic. This way, specific graphical elements can start to blink – Figure 4(e). We conjecture that this kind of selective constraining of channels not only alters attention mechanisms, as there will be fewer candidates for attention, it also reduces the memory load, as there would be fewer targets. Furthermore, it has the potential of affecting top- down processes by de-emphasizing features that are expected by the user – and that prevent her/him from noticing unexpected traits due to “blindness”. In these cases, to assist visualizations may be helpful, as advocated by Chen (2006). Such algorithms might monitor statistical features or mine characteristics of interest to suggest new encodings “on the fly”, enabling more intense appropriation of the scene. However, not much research has been conducted on this issue, which has potential for further developments.

4.2 Analytical Perceptions

Data encoding channels provide maps of potential targets for attention. Now, following vision theory, the next mechanism is attentive selection – see Figure 3. Biased by user intention, a subset of the prominent entities in a visualization will reach working memory. Once selected, the chosen visual stimuli will be the basis of the analogies that lead to mental models, see Section 3.3. Here, one question comes up – what perceptions are produced by the targets of

11 Figure 4: (a) Dispersion plot using channels of position, shape contour, shape size, and color. (b) Shape size turned off. (c) Color turned off. (d) Shape contour turned off. (e) Time movement turned on for automatically detected outliers. (f) Shape size turned on. attention in an information visualization design? To answer this question, we have extensively inspected the literature tracking the ways in which visual manifestation occurs when the intent is data analysis. We have found a limited set of possibilities, defining a visual taxonomic vocabulary whose elements appear recurrently. We refer to these elements analytical perceptions, depicted in the second part of Figure 3. In the realm of interaction, Yi et al. (2007) followed a similar procedure based on extensive literature inspection; they achieved results concerning the user intent, producing a limited set of recurrent possibilities. Analytical perceptions are the traits that any user attentively seeks for in a visual repre- sentation. Following the dual system theory presented in Section 3.3, such perceptions occur intuitively. Our investigation indicates that such elements include correspondence, differentia- tion, recognition, connectivity, arrangement, and variation in time. The most verified of these phenomena, correspondence and differentiation, are noted by Bertin (1977/1981) and by Card et al. (1999). The third analytical perception is presented by Mackinlay (1986) who states that the notion of relationship among graphical entities comes from the perception of connec- tivity. Meanwhile, arrangement arises from group positional configurations, largely studied by

12 the Gestalt (Koffka 1935) as observed, for example, in graph layouts (Dwyer et al. 2009)(Tollis et al. 1998). Recognition, in turn, takes place as a resemblance to previous knowl- edge and/or expertise, a concept studied in psychological models and Information Visualization models. As Liu & Stasko (2010b) point out, the concept of internalization involves the encod- ing of perceived information into long-term memory (enduring information or pattern). Lastly, variation, manifests only along time – not necessarily for temporal data – and in combination to the other five perceptions. The notion of analytical perceptions becomes evident when they are not found and the pipeline outlined in Figure 3 is broken, preventing Visual Expression – if none of the aforemen- tioned perceptions occur, the user is unable to make sense. As depicted in our theorization, analytical perceptions occur after pre-attention (Section 4.1) and before abstract patterns (Sec- tion 4.3), independently of the data domain. Hence, they bridge vision and data interpretation. Specifically, we discuss the analytical perceptions and how they relate to the data encoding channels in the following.

• correspondence: each position/shape/color has a direct correspondence to a referential map – discrete or continuous – that is part of the scene (explicit) or that is mental (im- plicit), defining analogical reasoning as explained in Section 3.3. Explicit maps include axes, geographical maps, shape/color dictionaries, and position/shape/color ranges. Im- plicit maps include known orderings and shape metaphors;

• differentiation: each position/shape/color discriminates graphical items. Differentiation is a correspondence achieved by the user, who creates a referential map in memory. Such map is limited in the number of elements (or differentiations) according to the limitations of memory;

• recognition: positions/shapes/colors whose decoding comes from the expertise of the user or from previous knowledge – recognition is a correspondence established from visual entities to concepts retained/learned in long-term memory;

• connectivity: shapes, mainly edges, that convey information about relationships among entities in memory;

• arrangement: Gestalt principles of organization – positional placements (closure, proxim- ity, and symmetry) that convey perception about group properties, for example, clusters and structural cues;

13 • variation in time: obtained when the parameters of position/shape/color are altered along time, inducing new perceptions for each of these channels.

Discussion

Our set of analytical perceptions is not an exhaustive listing, but a first reference that suggests that visualization designs tend to resort to the same basic set. For instance, the design of Google’s Gapminder tool, although contemporaneous, reproduces the same dispersion plots of statistical books a hundred years old. Nevertheless, as observed by Liu et al. (2014), works on new designs and on the evaluation of existing ones have not considered that there is a limited set of elements to instantiate in data representations. This fact could fruitfully support the def- inition of design and frameworks, which would benefit from recurrent constructs that lead to a limited set of analytical perceptions. Differently, current languages and frameworks rely on graphical patterns and it is up to the user to build the desired analytical perception; see Table 3 for a representative set.

Table 3: Previous works on languages and frameworks for visualization design.

Work Approach Elements Protovis Graphical Marks, shapes and layout (Bostock & Heer 2009)(Heer & Bostock 2010) D3 (Bostock Visualization Selection, operation, join, layout and et al. 2011) pipeline transformation Improvise Link and Variable, function and view (Weaver 2004) coordinate Prefuse (Heer High-level API Filter, layout, interaction, color and et al. 2005) size ggplot2 Domain specific Layer, scale, coordinate system and (Wickham facet 2009) Flexible Linked Linked axes Axes mapping, interaction, line and Axes (Claessen point & van Wijk 2011) This work Visual-cognitive Pre-attentive encoding channels and analytical perceptions

By considering the concepts reviewed so far, it is possible to conceive a design

14 whose approach is based on cognition, and whose elements are interactive encoding channels, and analytical perceptions – refer to Table 3 for comparison. For example, in this design language, one would be able to state a visualization by choosing channel color and perception of differentiation. This same visualization would demand a few more elements, as channel position and perception of correspondence. As in any language, these elements would receive parameters according to an extensible of data-to-marks mapping. In the case of relationships, one would be able to choose channel position with perception of arrangement, together with channel shape and perception of connectivity. Underlying algorithms would abstract undesired complexity, as an for outlier detection to support color differentiation; and a force- directed algorithm to support arrangement. As a design language, this approach would bring the benefit of discriminating the recurring elements of visualization techniques and having them in libraries for composition. This alternative approach contrasts with the usual practice of combining recurrent elements in ensembles assumed as new techniques.

4.3 Abstract Patterns

According to Hutchins (1996), tools – or externalizations (Hegarty 2004) – transform difficult tasks into in- manipulations of physical systems, or into pattern-matching problems (Giere 2002). Pattern matching, or pattern-recognition (Patterson et al. 2014), the association of a given stimulus to information retrieved from memory (Eysenck & Keane 2003), is one of the principles of data visualization and the second step of the Visual Expression Process – see Figure 3. According to our organization, once a user focuses on an analytical perception, she/he proceeds to match this perception to an abstract pattern. Based on the theory of vision – seen in Section 3.3, the generation of patterns is supported by memory, which is filled with data from the visual-sensorial system or from long-term memory. The efficiency of this intercourse is explained by the fact that the visual-sensorial system provides spatial information to memory at rates higher than that of long-term memory (Ware 2004), in a time ranging from 100 to 250 ms (Kieras & Meyer 1997). Hence, analytical perceptions in the visual-sensorial system work similarly to the images stored in long-term memory, providing efficient pattern-matching. According to the dual system theory, Section 3.3, this process is conscious and slow; but, with practice and experience, a given analyst can become a specialist. For specialists, pattern matching becomes intuitive (Evans 2008) – refer to the work of Patterson et al. (2010) for a thorough discussion. It is a straight conclusion, then, that a period of practice and experience

15 is necessary for most visualization techniques, and that this is an obstacle for use. A suggestive set of the patterns – third part of Figure 3 – that arise from the perception-to- pattern matching, includes: correlation, tendency, classification, relationship, order, summa- rization, outlier, cluster, structure, and reading. Tufte (1997) and Amar et al. (2005) provide more exhaustive listings that follow different rationalizations.

Discussion

In our literature review, we noticed that many systems disregard the fact that visualization techniques converge through abstract patterns. According to current practices, instead of a set of familiar patterns, the user has to choose among a set of visualization designs that, quite often, they have little experience with (Chen 2005). Thomas & Cook (2005) reinforce this notion; they state that abstract patterns correspond to the second factor of their four- steps analytical-reasoning process. Following their process – a pattern-to-construct sequence, users have constraints in relation to what they can search for in face of a given pattern. For example, suspiciousness tends to appear by means of tracking for outliers; while evidence of illegal lobbying practices emerge from clusters; and community detection in graphs is a task for relationship. Still, users are often offered a menu whose options are, for instance, dimensional stacking (LeBlanc et al. 1990), star coordinates (Kandogan 2000), and table lens (Rao & Card 1994); alternatives far from the pattern-to-construct task they have in their . A similar notion dates back to the 1990’s, in the work of Wehrend & Lewis (1990), who advocates in favor of more specific problem-oriented choices. We suspect that, because design-oriented interfaces neglect the more natural notion of abstract patterns, this is possibly one of the reasons why advanced visualization techniques have struggled to achieve a wider commercial dissemination, e.g, in office suites and in everyday spreadsheets. Take, for instance, the visualization technique Treemap (Shneiderman 1992), introduced for visualizing hierarchical data in general. In two decades, Treemap gained popularity at the , but, as a general hierarchical tool, it has failed in reaching a wider use. This is, possibly, because it has been criticized since its introduction (Barlow & Neville 2001, Cawthon & Moere 2007, Fabrikant & Skupin 2005), being accused of lacking cognitive plausibility, having poorly perceived aesthetic qualities, and presenting poor task-driven performance (Wood & Dykes 2008). Despite all, however, an especial design of the Treemap has remarkably succeeded. The SequoiaView system (www.win.tue.nl/sequoiaview) has achieved wide dissemination (check (van Wijk 2006) for some impressive numbers), far beyond the academic walls. But,

16 how can we have two flavors of the same technique evolve in different ways? Certainly, not one single aspect explains everything, but an outstanding factor comes up: the SequoiaView is domain and pattern-oriented, it is distributed to visualize the structure and the sizes of the files in your hard-drive, specifically. In accordance, users do not have to discover that the tool is good at doing this; instead, when they have this specific problem at hand, they are guided to SequoiaView, a more natural process. In designing systems, an alternative course of action would be to initially present the user with a set of patterns to choose from. After that, she/he would be offered a set of visualization techniques that better suit the pattern they are seeking for. Users may know what to look for by means of previous knowledge of the data domain, by means of known problems to be solved, by means of suspicious clues perceived along the data usage, and also by means of previous visual exploration of the data.

4.4 Cognitive Decision Support

After producing patterns, vision is no longer an active agent, neither pre-attentively nor atten- tively. Now, the analysis follows the widely accepted pattern-then-cognition process (Margolis 1990) (MacEachren 2004) to achieve decision support. Indeed, according to the analytical rea- soning of Thomas & Cook (2005), Information Visualization cannot ultimately provide decision support, which can only be achieved after interpreting the patterns in light of the data domain – rightmost side of Figure 3. That is, even though users can come up with patterns without considering the underlying data, these patterns are not of great use if the domain is not deeply understood. Insufficient domain knowledge leads to unsatisfied expectations in relation to InfoVis, cre- ating situations in which a user is presented to a supposedly insightful visualization but, then, everything one hears is “so what”? A disappointment that happens due to the enthusiasm according to which one can solve a wide range of problems just by looking at the data. How- ever, visualization tools can do little if the analyst is not well-prepared to assess what the data ultimately describes and potentially carries within.

Discussion

To prevent the aforementioned problems, InfoVis systems should define systematic means to aid the user in recording and accessing the domain knowledge related to the problems at hand. An interesting approach to overcome the gap of domain knowledge is to use annotations (or

17 analytic provenance (Xu et al. 2015)), either automatic or manual. As pointed out by Hullman et al. (2011), annotations help to direct the user’s attention and foreground particular insights, supporting the most efficient inferences. The work of Hullman et al. (2013) exemplifies this issue. Their work focuses on stock-price time series, which are hard to understand if one is not aware of the facts that influenced the behavior of the market. Their system solves this problem by identifying news that happened contemporaneously to outstanding patterns found in the plots, presenting them on demand – illustrated in Figure 5. Similar approaches (Dennis et al. 2003, Conesa et al. 2005) have been proposed for genomic data, an extreme case in which domain knowledge is necessary, otherwise no reading of the data (either visual or textual) will make sense.

Figure 5: Example of an annotated visualization produced with Contextifier – in the figure, labels provide domain knowledge. Reproduced from the work of Hullman et al. (2011).

An alternative is to use technique storytelling, as surveyed by Segel & Heer (2010). Plaisant (2005) defends that advanced interfaces need to address the long-term process of analysis that may require annotation, keeping, collaboration with peers, and the dissemination of results and procedures used. Storytelling not only attacks the problem of lacking domain knowledge, it also provides knowledge about new findings in the form of further “story chapters” interactively created. The visual analysis, potentially, becomes an incremental set of insights from multiple in the form of bookmarks, keyword tags, text comments, and audio

18 annotations. By bringing existing (factual) or produced (annotated) domain knowledge to the stage, abstract patterns can support reasoning by supporting conditional inferences (Johnson-Laird & Byrne 2002), possibly influenced by statistical regularities (Patterson et al. 2012). The result is the provision of inferences for decision making, notably, predictions, alternatives, and evaluations as depicted in Figure 3.

4.5 Remarks

In Sections 4.1 through 4.4, we have drawn conclusions by surveying accepted concepts of vision and cognition. Notwithstanding, we state these conclusions as conjectures with theo- retical evidence and with examples. This is because the validation of these hypotheses would encompass vast experimentation; producing material enough to spam a few papers or a book, to be conservative. Therefore, we leave our discussions both as contributions – to guide new systematizations; and, as future work – to drive further refinements and discoveries. Our conclusions also point to a challenging systematization. Rendering all the recommended aspects, together with multiple techniques and data domains, might involve a development ef- fort similar to that of huge software pieces, as office suites for instance. This is academically non-attractive, and economically risky. Possibly, the solution is to set a well-defined devel- opment framework for collaborative work, with ample acceptation, and standard interfacing. In the realm of machine , software Weka (Hall et al. 2009) has achieved great success in a similar endeavor. However, the graphical nature of InfoVis, and its data preprocessing techniques, imposes big challenges.

5 Conclusions

We reviewed concepts on vision, cognition, and Information Visualization by introducing the organizational theorization named Visual Expression Process, which proposes a course of action to explain visual data analysis. Over an extensive literature survey, the theorization provides comprehension and science for data graphical presentation. It proposes a new perspective to discuss and characterize techniques. From this perspective, for instance, it would be possible to “dissect” a given technique in terms of the channels, the analytical perceptions and the patterns that it supports, potentially revealing strengths and weaknesses. Our contributions are as follows:

19 • Theoretical compendium: we plotted the Visual Expression Process to interrelate vision, cognition, and Information Visualization;

• Discussions: we provided an extensive survey from different fields of science to serve as basis for relevant debate;

• Reflections: we revisited design practices supported by examples and study cases.

Overall, we have put together key concepts to introduce an insightful consideration of vi- sualizations. The reductionist perspective of our organization translates non-familiar concepts of vision and cognition into their corresponding effects with respect to design. In the form of a vocabulary taxonomically organized, we proposed a simplified comprehension of the fac- tors that define techniques and systems. With our contributions, we expect to foster a more comprehensive, accessible, and applied science of visualization.

References

Alvarez, G. A. & Cavanagh, P. (2004), ‘The capacity of visual short-term memory is set both by visual information load and by number of objects’, Psychological science 15(2), 106–111.

Amar, R., Eagan, J. & Stasko, J. (2005), Low-level components of analytic activity in infor- mation visualization, in ‘Proceedings of the Proceedings of the 2005 IEEE Symposium on Information Visualization’, INFOVIS ’05, IEEE Computer Society, Washington, DC, USA, pp. 15–. URL: http://dx.doi.org/10.1109/INFOVIS.2005.24

Baddeley, A. & Hitch, G. (1974), ‘Working memory’, The and motivation: advances in research and theory 8, 47–89.

Barlow, T. & Neville, P. (2001), A comparison of 2-d visualizations of hierarchies, in ‘Informa- tion Visualization, 2001. INFOVIS 2001. IEEE Symposium on’, pp. 131–138.

Bertin, J. (1977/1981), Graphics and Graphic Information-Processing, Walter de Gruyter, Berlin. 273 pages.

Bostock, M. & Heer, J. (2009), ‘Protovis: A graphical toolkit for visualization’, Visualization and , IEEE Transactions on 15(6), 1121 –1128.

20 Bostock, M., Ogievetsky, V. & Heer, J. (2011), ‘D3 data-driven documents’, IEEE Transactions on Visualization and Computer Graphics 17(12), 2301–2309. URL: http://dx.doi.org/10.1109/TVCG.2011.185

Bugajska, M. (2003), Spatial Visualization of Abstract Information, PhD thesis, ETH Zurich. 197 pp.

Buttner, P. (2010), Hello java: Linking act-r 6 with a java , in ‘Proceedings of the 10th International Conference on Cognitive Modeling’, pp. 289–290.

Card, S., Mackinlay, J. & Shneiderman, B. (1999), Readings in Information Visualization: Using Vision to Think, San Francisco, CA, EUA: Morgan Kaufmann.

Cawthon, N. & Moere, A. (2007), The effect of aesthetic on the usability of data visualization, in ‘Information Visualization, 2007. IV ’07. 11th International Conference’, pp. 637–648.

Cheal, M., Lyon, D. & Gottlob, L. (1994), ‘A framework for understanding the allocation of attention in location-precued discrimination’, Quarterly Journal of Experimental Psychology: Human Experimental Psychology 47, 699–739.

Chen, C. (2005), ‘Top 10 unsolved information visualization problems’, IEEE Computer Graph- ics and Applications 25(4), 12–16.

Chen, C. (2006), Information Visualization: Beyond the Horizon, Springer Science.

Chi, E. H. (2000), A taxonomy of visualization techniques using the data state reference model, in ‘IEEE Symposium on Information Visualization’, IEEE Computer Society Press, Salt Lake City, UT, USA, pp. 69–75.

Claessen, J. & van Wijk, J. (2011), ‘Flexible linked axes for multivariate data visualization’, Visualization and Computer Graphics, IEEE Transactions on 17(12), 2310–2316.

Cleveland, W. S. & McGill, R. (1984), ‘Graphical perception: Theory, experimentation and application to the development of graphical methods’, J. Am. Stat. Assoc. 79, 387 5, 531– 554.

Cohen, R. (1985), The Development of , Psychology Press.

21 Conesa, A., Gotz, S., Garcia-Gomez, J. M., Terol, J., Talon, M. & Robles, M. (2005), ‘Blast2go: a universal tool for annotation, visualization and analysis in functional genomics research’, 21(18), 3674–6.

Cook, D. & Swayne, D. F. (2007), Interactive and Dynamic Graphics for Data Analysis, Springer-Verlag.

Cowan, N. (2010), ‘The magical mystery four: How is working memory capacity limited, and why?’, Current directions in psychological science 19(1), 51–57.

Cutzu, F. & Tsotsos, J. (2003), ‘The selective tuning model of attention: psychophysical evi- dence for a suppressive annulus around an attended item’, Vision Research 43, 205–219.

Dennis, G., Sherman, B., Hosack, D., Yang, J., Gao, W., Lane, H. C. & Lempicki, R. (2003), ‘DAVID: for Annotation, Visualization, and Integrated Discovery’, Genome 4(5), P3+.

Dill, J., Earnshaw, R., Kasik, D., Vince, J. & Wong, P. (2012), Expanding the Frontiers of and Visualization, Springer-Verlag.

DJ, S. (2000), ‘Attentional capture and inattentional blindness’, Trends Cognitive Science 4(4), 147155.

Dwyer, T., Lee, B., Fisher, D., Quinn, K., Isenberg, P., Robertson, G. & North, C. (2009), ‘A comparison of user-generated and automatic graph layouts’, Visualization and Computer Graphics, IEEE Transactions on 15(6), 961 –968.

Eriksen, C. W. & Hoffman, J. E. (1973), ‘The extent of processing of noise elements during selective encoding from visual displays’, Perception and 14, 155–160.

Essen, D. V., Anderson, C. & Fellowman, D. (1992), ‘ in the primate visual system: An integrated systems perspective’, Science 255(5043), 419–422.

Evans, J. S. B. T. (2008), ‘Dual-processing accounts of reasoning, judgment, and social cogni- tion’, Annual Review of Psychology 59(1), 255–278.

Evans, J. S. B. T. & Stanovich, K. E. (2013), ‘Dual-process theories of higher cognition: Ad- vancing the debate’, Perspectives on Psychological Science 8(3), 223–241. URL: http://pps.sagepub.com/content/8/3/223.abstract

22 Eysenck, M. W. & Keane, M. T. (2003), : A Student’s Handbook, Taylor & Francis.

Fabrikant, S. & Skupin, A. (2005), Cognitively plausible information visualization, in ‘Amster- dam’, Elsevier Science.

Giere, R. N. (2002), Models as parts of distributed cognitive systems, in ‘Model-Based - ing’, Springer US, pp. 227–241.

Green, T. M., Ribarsky, W. & Fisher, B. (2009), ‘Building and applying a human cognition model for visual analytics’, Information Visualization 8(1), 1–13.

Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P. & Witten, I. H. (2009), ‘The weka software: An update’, SIGKDD Explor. Newsl. 11(1), 10–18. URL: http://doi.acm.org/10.1145/1656274.1656278

Healey, C. G. & Enns, J. T. (2012), ‘Attention and visual memory in visualization and computer graphics’, Visualization and Computer Graphics, IEEE Transactions on 18(7), 1170–1188.

Heer, J. & Bostock, M. (2010), ‘Declarative language design for interactive visualization’, IEEE Transactions on Visualization and Computer Graphics 16, 1149–1156.

Heer, J., Card, S. K. & Landay, J. A. (2005), Prefuse: A toolkit for interactive information visualization, in ‘Proceedings of the SIGCHI Conference on Human Factors in Computing Systems’, CHI ’05, ACM, New York, NY, USA, pp. 421–430. URL: http://doi.acm.org/10.1145/1054972.1055031

Hegarty, M. (2004), Diagrams in the mind and in the world: Relations between internal and external visualizations, in ‘Diagrams’, pp. 1–13.

Hegarty, M. (2011), ‘The cognitive science of visual-spatial displays: Implications for design’, Topics in Cognitive Science 3(3), 446–474.

Hullman, J., Adar, E. & Shah, P. (2011), ‘Benefitting infovis with visual difficulties’, IEEE Transactions on Visualization and Computer Graphics 17(12), 2213–2222. URL: http://dx.doi.org/10.1109/TVCG.2011.175

Hullman, J., Diakopoulos, N. & Adar, E. (2013), Contextifier: Automatic generation of anno- tated stock visualizations, in ‘Proceedings of the SIGCHI Conference on Human Factors in

23 Computing Systems’, CHI ’13, ACM, New York, NY, USA, pp. 2707–2716. URL: http://doi.acm.org/10.1145/2470654.2481374

Hutchins, E. (1996), Cognition in the Wild, The MIT Press.

Itti, L. & Koch, C. (2001), ‘Computational modeling of visual attention’, Nature Reviews 2(3), 194–203.

Johnson, C., Moorhead, R., Munzner, T., Pfister, H., Rheingans, P. & Yoo, T. (2006), NIH/NSF Visualization Research Challenges, Los Alamitos, CA, EUA: IEEE Press. 36 pages.

Johnson-Laird, P. N. (1983), Mental models: towards a cognitive science of language, inference, and , Harvard University Press.

Johnson-Laird, P. N. (2010), ‘Mental models and human reasoning’, Proceedings of the National Academy of Sciences 107(43), 18243–18250.

Johnson-Laird, P. N. & Byrne, R. M. (2002), ‘Conditionals: a theory of meaning, , and inference.’, Psychological review 109(4), 646.

Just, M. & Carpenter, P. (1992), ‘A capacity theory of comprehension: individual differences in working memory’, Psychological Review 99, 122–149.

Kandogan, E. (2000), Star coordinates: A multi-dimensional visualization technique with uni- form treatment of dimensions, in ‘IEEE Symposium on Information Visualization’, IEEE Press, Salt Lake City, pp. 4–8.

Keim, D. (2002), ‘Information visualization and visual data mining’, IEEE Trans. Visualization and Computer Graphics 8(1), 1–8.

Kieras, D. & Meyer, D. (1997), ‘An overview of the epic architecture for cognition and perfor- mance with application to human-computer interaction’, HCI 12, 391–438.

Koffka, K. (1935), Principles of Gestalt psychology, International library of psychology, philos- ophy and scientific method, New York : Harcourt, Brace and Company.

LeBlanc, J., Ward, M. & Wittels, N. (1990), Exploring n-dimensional , in ‘Visu- alization, 1990. Visualization ’90., Proceedings of the First IEEE Conference on’, pp. 230 –237.

24 Lippmann, R. P. (1987), ‘An introduction to computing with neural nets’, IEEE ASSP Maga- zine 4(2), 4–22.

Liu, S., Cui, W., Wu, Y. & Liu, M. (2014), ‘A survey on information visualization: recent advances and challenges’, The Visual Computer 30(12), 1373–1393.

Liu, Z., Nersessian, N. & Stasko, J. (2008), ‘ as a theoretical framework for information visualization’, Visualization and Computer Graphics, IEEE Transactions on 14(6), 1173 –1180.

Liu, Z. & Stasko, J. (2010a), ‘Mental models, visual reasoning and interaction in information visualization: A top-down perspective’, IEEE Transactions on Visualization and Computer Graphics 16(6), 999–1008. URL: http://dx.doi.org/10.1109/TVCG.2010.177

Liu, Z. & Stasko, J. (2010b), ‘Mental models, visual reasoning and interaction in informa- tion visualization: A top-down perspective’, Visualization and Computer Graphics, IEEE Transactions on 16(6), 999 –1008.

Logie, R. H. (1995), Visual-spatial working memory, Lawrence Erlbaum Associates.

Lohse, G. L. (1993), ‘A for understanding graphical perception’, Hum.-Comput. Interact. 8(4), 353–388.

MacEachren, A. M. (2004), How Maps Work: Representation, Visualization, and Design, The Guilford Press.

Mack A, R. I. (1998), ‘Inattentional blindness’, 5(3).

Mackinlay, J. (1986), ‘Automating the design of graphical presentations of relational informa- tion’, ACM Transactions on Graphics 5(2), 110–141.

Margolis, H. (1990), Patterns, Thinking, and Cognition: A Theory of Judgment, University Of Chicago Press.

Miller, G. A. (1956), ‘The magical number seven, plus or minus two: Some limits on our capacity for processing information’, Psychological Review 2(63), 34335.

Motter, B. C. (1994), ‘Neural correlates of attentive selection for color or luminance in extras- triate area v4’, The Journal of Neuroscience 14(4), 2178–2189.

25 Navalpakkam, V. & Itti, L. (2007), ‘Search goal tunes visual features optimally’, 53(4), 605–617.

Nothdurft, H. (2000), ‘Salience from feature contrast: additivity across dimensions’, Vision Research 40(10-12), 1183–1201.

Patterson, R. E. (2012), ‘Cognitive engineering, cognitive augmentation, and information dis- play’, Journal of the Society for Information Display 20(4), 208–213. URL: http://dx.doi.org/10.1889/JSID20.4.208

Patterson, R. E., Blaha, L. M., Grinstein, G. G., Liggett, K. K., Kaveney, D. E., Sheldon, K. C., Havig, P. R. & Moore, J. A. (2014), ‘A human cognition framework for information visualization’, & Graphics 42(0), 42 – 58. URL: http://www.sciencedirect.com/science/article/pii/S0097849314000387

Patterson, R. E., Pierce, B., Bell, H. H., Andrews, D. & Winterbottom, M. (2009), ‘Train- ing robust decision making in immersive environments’, Cognitive Engineering and Decision Making 3(4), 331–361.

Patterson, R. E., Pierce, B. J., Bell, H. H. & Klein, G. (2010), ‘Implicit learning, tacit knowl- edge, expertise development, and naturalistic decision making’, Journal of Cognitive Engi- neering and Decision Making 4(4), 289–303.

Patterson, R. E., Pierce, B. J., Boydstun, A. S., Ramsey, L. M., Shannan, J., Tripp, L. & Bell, H. (2012), ‘Training intuitive decision making in a simulated real-world environment’, Human Factors: The Journal of the Human Factors and Ergonomics Society p. 0018720812454432.

Plaisant, C. (2005), Chapter 3 - information visualization and the challenge of universal usabil- ity, in ‘Exploring Geovisualization’, International Cartographic Association, Elsevier, Oxford, pp. 53 – 82.

Rao, R. & Card, S. (1994), The table lens: Merging graphical and symbolic representation in an interactive focus+context visualization for tabular information, in ‘Human Factors in Computing Systems’, pp. 318–322.

Rensink, R. A. (2000), ‘The dynamic representation of scenes’, Visual Cognition 7(1-3), 17–42.

26 Rodrigues, J., Traina, A., Oliveira, M. & Traina, C. (2008), ‘The spatial/perceptual design space: a new comprehension for data visualization’, Palgrave Information Visualization 6, 261–279.

Rose, N. S., Myerson, J., Roediger III, H. L. & Hale, S. (2010), ‘Similarities and differences between working memory and long-term memory: evidence from the levels-of-processing span task.’, Journal of Experimental Psychology: Learning, Memory, and Cognition 36(2), 471.

Scaife, M. & Rogers, Y. (1996), ‘External cognition: how do graphical representations work?’, Int. J. Hum.-Comput. Stud. 45(2), 185–213.

Segel, E. & Heer, J. (2010), ‘Narrative visualization: Telling stories with data’, IEEE Transac- tions on Visualization and Computer Graphics 16(6), 1139–1148. URL: http://dx.doi.org/10.1109/TVCG.2010.179

Shneiderman, B. (1992), ‘Tree visualization with treemaps: A 2d space-filling approach’, ACM Trans. on Graphics 11(1), 92–99.

Shneiderman, B. (1996), The eyes have it: A task by data type taxonomy of information visualizations, in ‘IEEE Symp. Visual Languages’, IEEE CS Press, pp. 336–343.

Squire, L. R. (2009), ‘Memory and brain systems: 1969–2009’, The Journal of Neuroscience 29(41), 12711–12716.

Thomas, J. & Cook, K., eds (2005), Illuminating the Path: The Research and Development Agenda for Visual Analytics, IEEE Press.

Tollis, I. G., Di Battista, G., Eades, P. & Tamassia, R. (1998), Graph Drawing: Algorithms for the Visualization of Graphs, Prentice Hall.

Triesman, A. (1985), ‘Preattentive processing in vision’, , Graphics and Image Processing 31, 156–177.

Tsotsos, J. (1991), ‘Is complexity theory appropriate for analysing biological systems?’, Behav- ioral and Brain Sciences 14(4), 770–773.

Tsotsos, J., Culhane, S., Wai, W., Lai, Y., Davis, N. & Nuflo, F. (1995), ‘Modeling visual attention via selective tuning’, Artificial 78(1-2), 507–547.

27 Tufte, E. R. (1997), Visual Explanations: Images and Quantities, Evidence and Narrative, Graphics Press.

Tweedie, L. (1997), Characterizing interactive externalizations, in ‘ACM SIGCHI conference on Human factors in computing systems’, ACM Press, Atlanta, Georgia, USA, pp. 375–382. van Wijk, J. (2005), The value of visualization, in ‘Visualization, 2005. VIS 05. IEEE’, pp. 79– 86. van Wijk, J. (2006), ‘Views on visualization’, Visualization and Computer Graphics, IEEE Transactions on 12(4), 1000 –433.

Ware, C. (2004), Information Visualization: Perception for design, San Francisco, CA: Morgan Kaufman. 486 pages.

Ware, C. (2005), Knowledge and Information Visualization: Searching for Synergies, Springer Berlin Heidelberg, Berlin, Heidelberg, chapter Visual Queries: The Foundation of , pp. 27–35.

Watt, R. J. (1995), Some speculations on the role of texture processing in , in ‘Early vision and beyond’, Massachusetts Institute of Technology, Cambridge, MA, USA, pp. 59–67.

Weaver, C. (2004), Building highly-coordinated visualizations in improvise, in ‘Information Visualization, 2004. INFOVIS 2004. IEEE Symposium on’, pp. 159–166.

Wehrend, S. & Lewis, C. (1990), A problem-oriented classification of visualization techniques, in ‘Visualization, 1990. Visualization ’90., Proceedings of the First IEEE Conference on’, pp. 139–143, 469.

Wickens, C. D. (2008), ‘Multiple resources and mental workload’, Human Factors: The Journal of the Human Factors and Ergonomics Society 50(3), 449–455. URL: http://hfs.sagepub.com/content/50/3/449.abstract

Wickham, H. (2009), ggplot2: Elegant Graphics for Data Analysis, Springer, New York.

Wolfe, J. M. (1994), ‘Guided search 2.0 a revised model of visual search’, Psychonomic Bulletin & Review 1(2), 202–238.

28 Wong, P. C., Shen, H.-W., Johnson, C., Chen, C. & Ross, R. B. (2012), ‘The top 10 challenges in extreme-scale visual analytics’, Computer Graphics and Applications, IEEE 32(4), 63–67.

Wood, J. & Dykes, J. (2008), ‘Spatially ordered treemaps’, IEEE Transactions on Visualization and Computer Graphics 14(6), 1348–1355.

Woodman, G. F., Carlisle, N. B. & Reinhart, R. M. (2013), ‘Where do we store the memory representations that guide attention?’, Journal of vision 13(3), 1.

Xu, K., Attfield, S., Jankun-Kelly, T., Wheat, A., Nguyen, P. & Selvaraj, N. (2015), ‘Analytic provenance for sensemaking: A research agenda’, Computer Graphics and Applications, IEEE 35(3), 56–64.

Yi, J. S., ah Kang, Y., Stasko, J. T. & Jacko, J. A. (2007), ‘Toward a deeper understanding of the role of interaction in information visualization’, Visualization and Computer Graphics, IEEE Transactions on 13(6), 1224–1231.

Young, R. M. & Lewis, R. L. (1999), The soar and human working memory, in A. Miyake & P. Shah, eds, ‘Models of Working Memory: Mechanisms of Active Maintenance and Executive Control’, Cambridge University Press, pp. 224–256.

29