ARTEMIS: Learner-Centered Design of an Information Seeking Environment for K-12 Education

Raven Wallace, Elliot Soloway, Joseph Krajcik, Nathan Bos, Joseph Hoffman, Heather Eccleston Hunter, Dan Kiskis, Elisabeth Klann, Greg Peters, David Richardson, and Ofer Ronen University of Michigan School of Education School of Information 610 East University 550 East University Ann Arbor, MI 48104 Ann Arbor, MI 48104 313-647-7877 313-936-1562 [email protected] [email protected] ABSTRACT they find. Artemis is an interface designed in response to Learners use software for different reasons and with our classroom observations, to support K-12 students different skills and motivations than other users. Using access and use information in the University of Michigan concepts of learner-centered design (LCD), we developed Digital Library. a user interface for supporting learners as they use digital This problem is of particular interest for two reasons. information resources in inquiry-based science First, we are addressing a user population – K-12 learners classrooms. Learner needs are categorized in five areas: – which is often overlooked in the user-centered design of content knowledge, technology knowledge, strategic and general software tools. Second, the growing enthusiasm metacognitive knowledge, and motivation. Results of for connecting schools to the Internet, and the increasing research on problems encountered by students as they ubiquity of connected classrooms, create a pressing need engage in information seeking are used as the basis for for closer scrutiny of the design and use of tools which applying LCD, by identifying some specific problem contribute to educationally sound uses of the vast and areas learners encounter: engaging in a process, heterogeneous resources available on the Internet. This generating search terms, staying on task, and evaluating paper describes the rationale for the design of Artemis, information. Solutions offered through the Artemis including an overview of the UMDL; explanation of our interface are described. conception of learner-centered design; description of the Keywords contexts for initial implementation of Artemis and the Learner-centered design, information seeking, digital learning problems associated with those contexts; and libraries, K-12 Education explanation of the solutions Artemis provides to the INTRODUCTION problems of learning with digital information resources. In the fall of 1995, researchers from the University of The design which has been through three major revision Michigan Digital Library (UMDL) began observing use cycles, has followed an iterative process, with revision of of digital resources in 6th and 9th grade classrooms to the software based on observation of use in classrooms gain understanding of classroom uses and learner needs and test environments. for information seeking. Initially, students used Web THE UMDL/MYDL PROJECTS browsers and search engines in their science classrooms Artemis is part of the University of Michigan Digital to complete inquiry-based assignments. Our observations Library Project (UMDL), a project of the Digital Libraries convinced us that standard Web tools are not adequate for Initiative. The project's mission is to create an architecture learners as they engage in information seeking in and software infrastructure for development of a digital classrooms. For example, search engines return too many library open to multiple, heterogeneous collections. The hits, and too many of them are useless or irrelevant; structure and contents of collections are constrained only students lack a stable context in which to work and are by their ability to communicate with agents in the UMDL easily lost or distracted; and Web browsers are infrastructure about contents, search protocols, and other disconnected from software which lets students use what metadata. The UMDL project includes testbed collections in the domains of earth and space science. Wallace, R., et al, (1998). ARTEMIS: Learner-centered design of an information seeking environment for K-12 Within the agent architecture of UMDL, multiple education. Proceedings of CHI98: ACM Conference on interfaces are allowed, each with its own agent which Human Factors in Computing Systems, Los Angeles, CA, communicates with other agents in the UMDL pp. 195-202. infrastructure to find information, to negotiate prices for copyrighted information, to browse collections and Domain (content) knowledge: How can the software help the learner recall prior knowledge and build on what she knows? Technology (tool) knowledge: How can the software meet learners at varying levels of technological expertise in ways that keep the learner focused on substance, not on the tool? Strategic knowledge: How can the software help the learner use strategies he possesses, and learn new strategies appropriate to the task at hand? Metacognitive knowledge: How can the software assist the learner in keeping track of her goals, monitoring her progress, etc.? Motivation: How can the software help the learner stay engaged and motivated?

Table 1: Five Dimensions of Learner Needs thesauri, and to register materials. [1] Just as different Another area of difference between learners and other collections can be specialized by content domain or user users is in metacognitive knowledge. This refers to characteristics, so interfaces to UMDL can address students knowledge of strategies for monitoring their particular user groups. Artemis is an example of such an progress and thinking about their thinking. Metacognitive interface, designed particularly for K-12 students learning strategies include asking themselves such questions as through a process of inquiry. “Have I seen this before?” “Where am I in this process - Middle Years Digital Library (MYDL) focuses on what do I need to finish?” “Do I understand this?” [4] deployment of UMDL in middle school classrooms, Metacognitive knowledge implies the abilitiy to assess following the initial deployment of UMDL in high school one’s content knowledge and learning strategies. This is classrooms. Between 1995 and 1997, researchers clearly a type of knowledge which students lack in observed over 1000 students in over 30 sixth and 15 ninth varying degrees, and which teachers provide in the normal grade classrooms as they learned to find and use digital course of teaching. information resources. Classes began by using Web Finally, learners differ from other users in terms of their browsers and resources while UMDL was being motivation. While professionals in a field are committed developed, with initial tests of Artemis beginning in to their work and use software tools to accomplish a task, February, 1997. [2] students are often not similarly motivated. Technology must help sustain engagement by supporting complex TWO KEY ASPECTS: LEARNER NEEDS AND INFORMATION SEEKING activities so that students can focus on substantive Learner Needs cognitive issues and problem solving. The principles of learner-centered design (LCD) [3] These five dimensions are considered as they apply to the recognize the fact that students differ in a number of ways particular requirments of learners as they engage in from professionals who use computational software. information seeking in an electronic environment. Table 1 provides a summary of the dimensions of learner Information Seeking needs which require particular scrutiny as part of a Information Seeking in Inquiry process of learner-centered design (LCD). Each Our research has focused on providing tools, pedagogy, dimension is more fully explained below. and contexts for students to engage in inquiry, with Unlike professionals using software to do their job, information seeking as one part of the process. Inquiry learners do not initially know the content with which they can include many types of investigations from are engaging. They lack domain knowledge and must be experiments to observations, from using primary supported as they engage in inquiry, in ways that allow resources to reading what others have written. Key to the them to build coherently on any background knowledge concept of inquiry is that students engage in sustained they have. A related area of difference between learners investigations of questions of interest to them. [5, 6] The and other users is in their technology knowledge. major challenge of including digital information resources Students may need particular help in using new in inquiry-based learning is to provide tools which allow technologies to insure that the tool is the background, not students to embed information seeking in a sustained the purpose of the learning. In order to learn content they process. This means allowing for not only one-shot may need strategies particular to the type of work they are queries when the user is looking for a simple answer, but doing. For example, they may need to learn to read a also complex exploration of information spaces when the graph, or to interpret a map. Thus, students differ in user is trying to understand a complex problem. [7] terms of their strategic knowledge both generally (i.e., strategies for reading and comprehending text) and specifically within the domain of investigation (e.g., reading a map.) Information Dimensions of Learner Needs Seeking Domain knowledge Technology Strategic Metacognitive Motivation knowledge knowledge knowledge Engaging a in Persistent Workspace Driving Question Folders Past Results Persistent Workspace Driving Question Folders Past Results Recommendations process

Generating Broad Topics Overall design supports Broad Topic Broad Topics Broad Topics Collections growth in technology Past Resultss search terms knowledge by building on common design metaphors Staying on task Persistent Workspace Persistent Workspace Collections Collections Past Results Evaluating Driving Question Folders Driving Question Folders Driving Question Folders Recommendations Collections Collections Collections information Past Results Table 2: Artemis Features: Dimensions of Learner Needs and Information Seeking Typically, the information seeking process includes video allows fine-grained studies of students posing a question, exploring for information, refining the collaboratively searching for information. question, finding information, organizing and evaluating  Classroom observations follow focus groups of what is found, synthesizing information, and, finally, students over time in the course of their classroom using the information. [2, 7] These steps are not followed activity (which includes Artemis searching). Student linearly, but rather, feedback from each phase may used to groups are monitored across inquiry-based science reiterate a previous step, or to jump to a new step. For projects, as well as longitudinally over the year. example, the cycle of exploring, refining the question, and Following students over time allows study of looking for information may be repeated many times until students' developing competencies, and preferences- a satisfactory, focused question is formulated. For a user, over-time, and also allows us to assess the larger each piece of the information seeking process might be impact of Artemis-provided information in inquiry- thought of as a separate task, and different software tools based science. might be used for different pieces of the process. Historically, computer systems have provided tools for  Structured interviews of students and teachers were information seeking in pieces, focusing on limited aspects used in conjunction with other observations. of the process. For example, search engines address the Observers watch students in the course of a search phase of looking for information; word processors address recorded on process video, or over the course of a using information; Web indexes help with browsing; longer project, and question students or teachers on bookmarking programs are for organizing. However, for a aspects of their search, inquiry processes, and learner, whose purpose is to learn from the information attitudes toward the tools. encountered, not just to find a specific piece of During the one to three week projects, students posed information, the task is the entire process, and it must be questions and looked for information on the Web and in supported as a whole. The support should allow the their school libraries to complete their assignments, which student to focus on the content, not on the technology. For varied from writing a report to creating a multimedia students, especially at younger ages, simultaneous use of document. Detailed observations of focus students were multiple pieces of software is not an option. In addition to obtained by recording their on-line activities with process cognitive issues, the hardware available in schools often video. We followed ten pairs of middle school students cannot effectively support the information seeking doing four projects, and six pairs of high school students process through use of multiple programs. doing three projects during the 1996-7 school year. Empirical Basis for Understanding Information Seeking Analysis of the video records is ongoing, but initial results As a first step toward understanding how to support of analysis of these records, along with observations of information seeking for students, we implemented Web- similar projects during the 1995-6 school year, based units in 6th and 9th grade science classrooms. We highlighted four major aspects of information seeking in used three major sources of data: which students need support. [8-11] These are described below, and are related back to the five dimensions of  Process video uses a video recorder to capture learner needs. Table 2 gives an overview of these aspects everything that students see on their computer the information seeking as they related to the dimensions monitors through a video-out connection, and of learner needs, indicating the Artemis feature which combines this with audio recording of conversations addresses each dimension. between students stationed at the computer. Process Engaging in a process. Students often interpret the task of information seeking as one of getting one "right" answer or a perfect source. Rather than trying to solve a problem “ Scaffolding,” on the other hand, refers to features and increase their understanding of the content area, they which support learning, but which fade as the learner work to finish, reducing their goal to "schoolwork." This grows in knowledge and skills. [15] An example of is a problem related to all areas of student motivation and scaffolding might be an interactive help screen which knowledge. They need some base level content prompts the student through the steps of a process with knowledge to be able to ask an interesting question and to decreasing frequency as the student becomes successful. deal with content they encounter. They need strategic Scaffolding can also be passive, for example, knowledge of a variety of types. For example, they need functionality providing simplified views of a problem. strategies for reading on-line material, for planning their The distinguishing characteristic of scaffolding is that it progress toward completion of their project, and for fades over time, becoming unavailable or invisible to the keeping track of what they find. user as the user becomes more expert. The purpose of Generating search terms. Lacking background scaffolding is to enable the learner to do complex things knowledge, students often have trouble generating any on her own once the scaffolding has faded. A vivid keywords other than those used in their question. More example of scaffolding is training wheels on a bicycle: generally, keyword selection is problematic because of they provide support for the beginner, can be altered as the nature of language and of search systems: e.g., exact the child learns, and finally are removed altogether when string matching ignores meaning and choice of keywords no longer needed. exactly determines what is returned to the user. [12, 13, In the design of Artemis, we include learner supports, but, 14] An inability to generate synonyms, either because of a in the initial releases, no scaffolds. Our intention is to lack of background knowledge or because of a continue observation and analysis of Artemis in use, and misunderstanding of the significance of the particular consider addition of scaffolds as we understand more keyword chosen, can completely thwart a student’s about the information seeking needs and contexts of progress. This problem relates to student's content, learning of students in inquiry-based environments. technological, and strategic knowledge. Learner-centered design methodology employed in the Staying on task. Students can be easily distracted in an design of Artemis includes analyzing the learning context environment such as a computer-rich classroom or lab in and learner needs in light of research on learning and which accountability is low because of the variety of tasks teaching. In this instance, the context is information students are engaged in and the high ratio of students to seeking with digital resources in science classrooms. The teacher. Lack of success can decrease student interest in special problems of this context are described above. The staying on task, and lack of understanding of content can five dimensions of learner needs, along with the identified make it difficult for students to focus. This is a problem of aspects of information seeking, provide the basis for domain knowledge and motivation. understanding the task from the perspective of the learner. Evaluating information. Accustomed to the textbook Our methodology requires that we work closely with model, in which students merely look up the answer and teachers in classrooms to iteratively modify our designs take it as given, students may have neither skills, based on actual uses by students. knowledge, nor inclination to critically evaluate information they find. This is a problem of content, ARTEMIS DESIGN strategic, and metacognitive knowledge. Our observations of students using Web browsers for LEARNER-CENTERED DESIGN: ADDRESSING KEY inquiry confirm that information seeking is a complex ASPECTS process which is often not attended to in K-12 education. The incorporation of features which address the needs of [7,10,13] We designed classroom contexts and Web learners within the problem space of the learning at hand environments to support students’ reflective use of Web is central to LCD. These features can be of two types: resources, and yet students short-circuited our efforts as learner supports and scaffolds. We use the term “learner they sought to make sense of their assignments and of the supports” to refer to features that are inherent in the inherent difficulties of finding useful information in a software, but which are specifically included to support complex environment. The design of Artemis is in large learning. Examples of learner supports might be a spell part a response to the difficulties we observed in the four checker which helps students identify and correct general categories above, and in many small, specific misspellings, or a graphical overview of an information instances. Overall, we found that it is imperative for space which helps the learner understand the knowledge students to be able to accomplish multiple tasks within a domain in question. Learner supports may be useful to single computer environment so that their work does not professionals, but are in some sense essential for learners. become fragmented, and so they can pick up where they They are passive supports, in that they do not interactively left off when they return to the computer day after day. intervene in the student’s behavior, but rather are We sought to design learner supports which made it available as an integral part of the software. possible for students to use Artemis as part of a reflective approach to using digital information, concluding that Broad Topics Recommendations Driving (alternates with Past Question Results) Folders Collections

Figure 1: The Artemis Interface providing scaffolds early in the design cycle might be too The Persistent Workspace supports growth of students limiting. Our stance as educators led us to believe that the strategic and metacognitive knowledge, as well as helping role of the teacher along with the classroom and school them build content knowledge. contexts determines how even the best software is used, In classrooms, we observed a disconnect between and we opted not to try to predict what scaffolds would querying and actually looking at and evaluating become most important until we had extensive information, which students have trouble bridging. In fact, observation of the software in use in the classrooms of we have observed that students act as if anything they find expert teachers. The most important learning supports of about the subject they are searching for is acceptable: they the Artemis interface in its initial design, and their do very little evaluation of content. The idea of a relation to the needs of learners and the particular persistent workspace is to give them a way to retain problems of information seeking, are described below, information about what they have already done, and to shown in Figure 1, and summarized in Table 2. work with information as they need it. The persistence not Persistent Workspace only provides information over time, but also reinforces The Persistent Workspace provided through Artemis Folder Label retains student work from session to session. In particular, a record of searches with their results (in the Drag and drop links form of live links to actual documents) and the Driving Question Folders (described below) with live links to documents are stored on a UMDL server and are available through a user name from any computer with access to the Web. The Persistent Workspace is important because it can help students engage in information seeking as a sustained process over time, an important aspect of inquiry and one which is quickly forgotten if students focus only on query, submitting keywords and “getting good hits.” Even for reflective students, fifty minute class periods make it difficult to make progress in finding useful information, especially if they need to start over each day. Seeing what they have done before can not only help students avoid redundant searches, but can also reinforce the notion that they are engaged in a process of exploring, finding, and evaluating information, not racing to finish. Figure 2: Driving Question Folders Past Results (alternates ramble around in content without paying much attention with Broad Topics) to what they know, or what they are trying to learn. Strategic knowledge is supported by providing them with ready ways to organize what they found, as well as to refine their question. These are strategic skills which have proven difficult for students to use on the Web. Metacognitive knowledge is supported through the persistence of the Driving Question Folders, allowing later reflection on what they found, where they are in accomplishing their goals, and what they need to do next.

Figure 3: Past Results in ArtemisPast Results The Past Results window (Figure 3) keeps a "live" list of the idea that information seeking is more than just getting the student's searches. Clicking on an item in the Past the right keywords Results window shows the results in the Search window. The results are actually stored on the UMDL server as Driving Question Folders part of the student's persistent workspace, so what they Driving Question Folders (Figure 2) are created by see in the Search window is identical to their original students to store links to items they find interesting. They search. In addition, the broad topic and keywords used function like a bookmark file, but are a visible part of the when the search was submitted are shown, so the student interface. Items are placed in a folder by dragging and has a record of how he has searched, as well as what he dropping from the Search window. Students can make has found. multiple folders which can reflect different areas of interest, or refinements of an initial question. Because In observing student search behavior on the Web, we they are part of the Persistent Workspace, the folders are found that students sometimes submit the same query accessible day after day, from any computer the student is repeatedly, and are often unaware that they have done so. working on. One reason for their repetition may be that they come to see information seeking as a task of creating a perfect An important concept of inquiry-based learning is that query, one which gets them exactly what they want. This students work on questions of interest to them, sometimes is reinforced by the unmanageable number of sites described as "authentic" or "driving" questions. We use returned through Web search engines, leading to the the term driving question, defined by one middle schooler feeling that there is no point in looking at sites until you as "A question that drives you crazy until you get the are dealing with a manageable number. However, Web answer." In posing such a question, the student may start search engines typically offer little support for actually with a general area of interest, and narrow it down as she refining a query. finds interesting information. Or, alternatively, she may find that her interests change as she encounters new We also observed that students developed a strategy of information, and change her question altogether. These recording keywords as if they provided an index to Web folders allow flexibility in storing links, and may help sites. That is, if they found a good site, instead of writing students focus on the substance of what they find. They down the URL or creating a bookmark, they recorded the can create multiple folders, reflecting a refinement of their keywords they used to get to the site. This is an original question, or a new area of interest. The folders interesting strategy which works in the short term and are part of the persistent workspace, and thus can be used says much about their mental model of the Web. The across multiple work sessions, adding items or looking in Past Results feature in a sense formalizes their strategy by more detail at items already found. This provides support actually making their past searches into an index of what for engaging in a process and may help students stay on they found. task by making organizing easier. It also allows them to Past Results can help students avoid useless repetition, readily compare and evaluate resources about a specific contributing to their staying on task. It can also be used in driving question. In a later release, a note pad will be an appropriate pedagogical context to help them use available in the driving question folder. different keywords, by making what they have already done easy to review. Finally, this feature can contribute to students' understanding of information seeking as a Driving Question Folders support learning content process, by allowing them to see their own process over knowledge by helping students focus on the question. time. Past Results thus supports development of strategic Needing to put things in folders helps them think about and metacognitive knowledge, addressing all four aspects what information fits with their question and what other of information seeking on which we have focused. related questions they might usefully pursue. We have observed that without a strong focus, students often Broad Topics Encyclopedia of Science and Technology, and Grolier's The Broad Topics window (Figure 1), which alternates Encyclopedia Americana; and the “Recommendation with Past Results in one corner of the Artemis window, Collection” (which is described in the next section.) The consists of a list of topics organized by domain. The Web itself is also a collection of UMDL by virtue of topics serve two types of functions, one internal to the having a UMDL agent which searches the Web when UMDL system, and the other directed toward the learner. queries are submitted. Once a query is submitted, returns Within UMDL, broad topics are actually a controlled are grouped by collection, showing the number of returns vocabulary used to register materials in some collections within each collection. This provides an initial grouping within the library. As explained below, the broad topics of returns, helping make the return set manageable. shown were selected to register the particular collection The Web Collection consists of Web pages identified by designed by UMDL researchers for K-12 classrooms, and librarians as particularly suited for students in the pilot they are taken from the SEARS headings, a well known deployment of Artemis, 6th and 9th grade earth and space subject heading system for school libraries. Agents within science students. Librarians located, evaluated, and the UMDL architecture translate these broad topics to abstracted readable material with appropriate content. topics in other vocabularies (that is, other sets of terms They also looked specifically for information that was used to register collections), and can automatically designed to take advantage of the web environment, not broaden or narrow searches if no results are found. simply marked up textbook pages. They sought pages From the perspective of the user, broad topics present a with interactivity, animated illustrations, forms to hierarchy of terms which can be browsed or searched as communicate and respond with web authors. These sites the first step in creating a query. UMDL uses the broad are registered in a special UMDL collection using topics selected by the user as keywords when a search is controlled vocabulary from the Broad Topics list. The submitted. If specific topics are also entered, UMDL Web Collection is designed to insure that students have searches on those keywords as well. The topics are successful searches as a way to help motivate them to stay searchable - entering a word results in highlighting all on task. instances of that word in the Broad Topics window. Observations of students using general Web search From the perspective of learning, the Broad Topics engines indicate that students get frustrated by the window is intended to help students generate keywords numerous, often irrelevant, sites returned from even fairly and recall prior knowledge, and it gives them a view of specific searches. Irrelevant results also lead to the structure of the content area they are exploring. distraction from their task, as students follow useless Students who ask questions that interest them often end links. The Web Collection approach is similar to the tried up exploring content areas about which they know very and true approach of libraries, in which collections little. The result is that they can have a lot of trouble appropriate to the user population are made available. thinking of any terms outside of the language of the Each site in the Web Collection includes an abstract question they have posed. That is, if the question is, “how written by the librarians. The abstract appears when the hot is lava?” they may try to use the keywords “hot” and student picks the link from the search results list, allowing “lava” for their search. For them, Broad Topics works them to get a quick overview of the site without the time something like an on-line thesaurus, letting them explore cost of actually linking to the page. This feature is another related terms, jogging their memory and perhaps their way to help keep students on-task. In addition, it provides interest in related topics. an aid to their evaluation of the site. We see Broad Topics as a support for learning content knowledge as described above, and also for development of strategic and metacognitive knowledge, and helping Figure 4: Making a Recommendation with student motivation. It presents a strategy for generating search terms and gives students a concrete way The various feature of UMDL collections support students to think about what they know and what they need to in engaging in a process, staying on task, and evaluating know. As support for motivation, it can provide them information. with alternatives - different terms with which to search - Recommendations possibly avoiding some of the frustration of getting stuck The Recommendations feature lets students and teachers using search terms that aren’t working without being able recommend good resources. Figures 4 and 5 show how to think of alternatives. recommendations are set up, while the results of Collections recommendations are shown in the search window of UMDL consists of multiple collections, related by a focus Figure 1. on earth and space science. In the testbed are a journal In our classroom experiences with students using the collection from UMI (ProQuest Direct); the “Web Web, one approach which was beneficial was to create Collection,” a collection of Web sites created for K-12 our own Web pages with good starting points selected by students; a collection of reference materials including the UMDL librarians in collaboration with teachers. This experience resulted in the Web collections described students to go beyond simply answering teacher-given above, and the Recommender. The Recommender questions, supporting them as they engage in substantive automates the process of making targeted collections, and inquiry. incorporates them into the UMDL as a registered Artemis is one attempt at using learner-centered design collection. ideas to produce a learning environment that does support A teacher or student can recommend resources by using students in such inquiry activities. As the foregoing has the window shown in Figure 4. The recommended sites documented, the design is well rationalized; we have are compiled into a recommendation collection which is a attempted to take into consideration cognitive, social, and searchable collection within UMDL. Using the Profile motivational issues in the design of Artemis’ functionality window (Figure 5) to set criteria, a user making a query and supports. Our next round of data on the use of can ask for recommendations based on characteristics of Artemis will enable us to revise, yet again, the interface the person making the recommendation or the topic. For and better hone the learner-centered design guidelines and example, a class list of recommended resources can be processes. created by the teacher. She can have her class set their ACKNOWLEDGMENTS preferences to get her recommendations. When they enter The authors wish to thank the students and teachers who queries, the system searches her recommendations for participated in the UMDL/MYDL research projects from matches to their query, and these appear as search results 1995-1998. This work was supported by the in the Recommendation collection. NSF/DARPA/NASA Digital library Initiative Recommendations is another feature which can be used to (Cooperative Agreement IRI-9411287); by a grant from help students evaluate information they find. On the one NSF NIE for the Middle Years Digital Library (RED- hand, they can read evaluations from others about a site 9554205) and by the University of Michigan. they are interested in. On the other hand, they may be asked to make recommendations to others, a process which involves explicit evaluation. This supports both strategic and metacognitive knowledge. We also expect that being able to contribute recommendations, and see what peers and teachers think about sites will motivate students as they work.

Figure 5: User Profile for Recommendations EVALUATION Evaluation of the first release of Artemis as described in this paper is being conducted during the 1997-98 school year in middle and high schools. Log file data and process video records of student use are being collected and analyzed for system functionality and interface usability. For example, we are looking at patterns of use by high school students to determine proportion of time spent submitting and sorting through searches compared to time spent reading and evaluating material found; and we are looking for use of specific supports such as the Driving Question folders and Past Results. Feedback from field data is being used to rework the interface for a new release in the spring of 98. DISCUSSION There are very high expectations for the value of the digital resources, and particularly the Internet, in education. Unfortunately, currently the Internet is used, for the most part, as a mechanism to deliver electronic versions of textbooks: “go to the following Web site and answer the following 5 questions.” In order to truly realize the potential of the digital information resources for learning and teaching, learners will need software that supports them in their use of those resources, and enables REFERENCES 1. Atkins, D.E., Birmingham, W. P., Durfee, E. H., et al., Toward Inquiry-Based Education Through Interacting Software Agents. Computer, 1996. 29(5): p. 69-76. 2. Wallace, R., J. Krajcik, and E. Soloway, Digital Libraries in the Science Classroom: An Opportunity for Inquiry. D-Lib Magazine, 1996 (http://www.dlib.org/dlib/september96/umdl/09wallace.html). 3. Soloway, E., M. Guzdial, and K.E. Hay, Learner- centered design: The challenge for HCI in the 21st century. Interactions, 1994. 1(2): p. 36-48. 4. Paris, S.G. and P. Winograd, How metacognition can promote academic learning and instruction, in Dimensions of thinking and cognitive instruction, B.F. Jones and L. Idol, Editors. 1990, Lawrence Erlbaum Associates: Hillsdale, N. J. p. 15-51. 5. National Research Council, National Science Education Standards. 1996, Washington, DC: National Academy Press. 262. 6. Blumenfeld, P.C., et al., Motivating Project-based learning: Sustaining the doing, Supporting the Learning. Educational Psychologist, 1991. 26(3 & 4): p. 369-398. 7. Kuhlthau, C.C., The process of learning from information, in The virtual school library: Gateway to the information superhighway, C.C. Kuhlthau, Editor. 1996, Libraries Unlimited, Inc.: Englewood, CO. p. 95-104. 8. Wallace, R., On-line information seeking in a sixth grade classroom. In revision, 1997. 9. Hoffman, J.L., Kupperman, J., Wallace, R. On-Line learning materials for the science classroom: Design methodology and implementation. Paper presented at AERA 97, Chicago. 10. Wallace, R. and J. Kupperman. On-line search in the science classroom: Benefits and possibilities. Paper presented at AERA 97, Chicago. (ERIC # ED407252) 11. Soloway, E. and R. Wallace, Does the internet support student inquiry? Don't ask. Communications of the ACM, 1997. 40(5): p. 11-16. 12. Greene, S.L., Devlin, S.F., Cannata, P. E., Gomez, L. M., No IFs, ANDs, or ORs: A study of database querying. International Journal of Man-Machine Stuidies, 1990. 32: p. 303-326. 13. Furnas, G. W., Landauer, T. K., Gomez, L. M., Dumais, S. T. The vocabulary problem in human -system communications, Communications of the ACM, 1987. 30: p. 964--971 14. Marchionini, G., Information seeking in electronic environments. Cambridge Series on Human- computer Interaction, ed. J. Long. 1995, Cambridge: Cambridge University Press. 224. 15. Wood, D., J.S. Bruner, and G. Ross, The role of tutoring in problem-solving. Journal of Child Psychology and Psychiatry, 1975. 17: p. 89-100.