Cognitive Representations of Dos Commands As a Function of Expertise

COGNITIVE REPRESENTATIONS OF DOS COMMANDS AS A FUNCTION OF EXPERTISE Kathleen M. Snyder James R. Lewis IBM Corporation IBM Corporation Dept F3B, Div 20 T. J. Watson Research Center Room 12B174 Dept 564, Room Hl-E22A 360 Hamilton Avenue Route 9A White Plains, NY Hawthorne, NY ABSTRACT that would be impossibly difficult with anything less adequate. [I, The purpose of this study was to examine P. 221 the cognitive networks derived from the similarity rating of IBM Personal Computer Disk Operating System (IBM PC Although Winston's [l] comments were DOS) (TM) commands by computer users given in the context of the advantage of with varying levels of expertise. good representation in artificial Naive, novice, and intermediate networks intelligence, the argument is also were examined to determine which links applicable to the use of multivariate in their networks were also present in statistical methods in psychology. an expert network. Groups with a Different methods allow different greater level of DOS expertise had more representations of the structure hidden links in common with the experts. A in a complex dataset. Like any other core set of commands were identified craftsman, we must choose the which were linked in every network. As appropriate tools to achieve our the level of expertise changed, it was analytical goals. possible to show the order in which links in the experts' network became The goal of this study was to present in the novice and intermediate demonstrate the use of network analysis groups. for modeling cognitive representations of operating system commands as a function of expertise. In some uninteresting sense, all computer-based representations are Expert-Novice Studies equivalent. This is so because computer-based representations are A number of studies have been conducted embedded ultimately in the symbolic which have examined the cognitive structures available in a computer differences between experts and novices language like LISP and thence down (e.g., [2,3]). The findings of these into arrangements of bits in studies have been remarkably consistent. memory. Consequently, any representation that can be used to Chase and Simon [2] studied the ability represent arrangements of bits can of novice and expert chess players to indirectly bear the information in recall the board positions of chess any other representation. pieces. They found that experts outperformed novice players. The In a practical sense, however, some experts' advantage was attributed to representations emphasize things their ability to form larger chunks of that are important to solving a information in memory rather than above- class of problems. One scheme, average memory capacity. They also therefore;,.is more powerful than found that if the arrangement of chess another because it offers more pieces was random rather than a midgane convenience to the user even (and presumably meaningful) arrangement, though, theoretically, both can do then experts recalled no more piece the same work. Convenience, positions than novices. however, is perhaps too weak a word. In general, the much greater In an attempt to replicate these perspicuity and the inherent findings in a more applied setting, Egan thinking advantages of powerful and Schwartz [3] had both novice and representations enable progress expert electronics technicians attempt 447 0073-1129/89/0000/0447$01.W0 1989 EEE to recall briefly presented (5 - 10 sec) regression and discriminant analysis are circuit diagrams. Some diagrams were examples of dependence methods. meaningful and others were random re- Principal components analysis, factor constructions of the meaningful analysis, multidimensional scaling diagrams. As with the chess masters, (MDS), cluster analysis, and network the expert technicians recalled more of analysis are types of interdependence the meaningful diagrams, but had no methods. Dependence methods are used to performance advantage for the random explain or predict one or more dependent arrangements. Additional analyses variables as a function of a set of showed that neither more skilled independent variables. "Interdependence guessing by expert technicians nor methods, on the other hand, are less spatial proximity could account for the predictive in nature and attempt to experts' performance advantage. provide insights into the underlying structure of the data by simplifying the Alwood [4] has recently reviewed the complexities, primarily through data body of work examining the behavior of reduction" [7, p. 191. For this reason, novice programmers with computers, interdependence methods have been used contrasting their behavior with that of to assess the relationship among experts to illustrate that which is concepts in memory. typical of novices. In his review, he points out that the expert advantage for A recent addition to the collection of the recall of meaningful information has interdependence methods is a link- been replicated several times with weighted network analysis procedure novice and expert programmers recalling called Pathfinder [8, 91. It can be lines of code (e.g., [SI). applied to the same types of data typically analyzed by MDS or cluster These results have been explained by analysis, i.e, estimates of pairwise hypothesizing that experts can process distance between entities. With global display properties or code a Pathfinder, distances between entities single chunk whose relation to a more are represented as links in a network if general category is known. This the resulting links form the shortest conceptual knowledge would allow the possible path between the entities. expert to search visually and recall Pathfinder solutions are affected by two from memory more systematically than variables: the value of the path length novices. Therefore, a critical function (defined by the Minkowski r- difference between novices and experts metric) and the number of links examined would be the organization of concepts in in constructing the network. With long-term memory. ordinal level measurement, g should be set to infinity to ensure a unique According to Bateson, Alexander, and network structure. To avoid triangle Murphy [6], another explanation is that ineqvality violations, N-1 links should experts may use higher-level knowledge be examined when developing the network. to understand problems while novices focus on specific details. Experts may Since their development and availability also use high-level plan knowledge to on computer systems powerful enough to direct their activities. Bateson et al. implement them, multivariate divided 50 computer science majors into interdependence methods have been used novice and expert groups, and as tools for psychological research. administered tests to assess syntactic Rips, Shoben, and Smith [lo] memory, semantic memory, tactical skill, demonstrated that MDS could be used to and strategic skill. The best model semantic distance in memory, and predictors of programmer expertise were that these semantic distances could be semantic memory, tactical skill, and used to predict reaction times in a strategic skill. In particular, they categorization task. Schvaneveldt, concluded that semantic memory is an Durso, Goldsmith, Breen, and Cooke important cognitive factor in the [ll] used both MDS and Pathfinder to development of programming skill. This study the organization of flight-related research also supports the hypothesis concepts by fighter pilots of varying that the organization of concepts in degrees of expertise. They concluded memory should differ due to expertise. that both MDS and Pathfinder revealed the underlying structure of the data, but highlighted different aspects. Pathfinder focused more on local Multivariate Statistical Methods and relationships among concepts, while MDS Psychological Research provided more global information regarding the dimensions of the concept Dillon and Goldstein [7] have described space. In research comparing the serial two primary categories of multivariate recall of concept lists organized by statistics: dependence and ALSCAL-S MDS and Pathfinder interdependence methods. Multiple 448 representations, Cooke, Durso, and knowledge [15]. Their formal method Schvaneveldt [12j found that the network requires three stages: initial organization led to faster learning than elicitation of a set of concepts, the MDS organization. This implies that application of a psychological scaling the Pathfinder networks better captured technique (i.e., multivariate the relations important for recall. interdependence method), and the interpretation of the resulting These methods have also been applied in representation. For example, when using two areas of computer development: menu Pathfinder, the final stage is the organization and expert systems. interpretation of the nature of the McDonald, Stone, Liebelt, and Karat relationship between concepts, achieved [13] used MDS and cluster analysis to through either direct link-labeling or model experts' similarity ratings of cluster analysis of similarity ratings word-processing functions. They of the links (applying a second compared the performance of a different interdependence method to the analysis set of word-processing experts using a of the output of the first). menu based on experts' organization of the commands and one based on a random organization. Using a paired-associates Interdependence Analyses of Expert and learning paradigm, they found that fewer Novice Computer Users errors were made using the organized menu. Snyder, Paap, Lewis,

Load more