Redesigning a Communication Support System for Teachers

REDESIGNING A COMMUNICATION SUPPORT SYSTEM FOR TEACHERS

Master Project Report

Submitted to Master Program of Learning, Design, and Technology

at School of Education, Stanford University

ii ABSTRACT

I proposed to redesign a communication support system for teachers, which will provide an easy solution for both teachers and project administration staff to archive teachers’ scholarship of teaching, and to exchange their expertise in the teachers’ community. This system is being used under the project of The Carnegie Academy for the Scholarship of

Teaching and Learning (CASTL) at the Carnegie Foundation for the Advancement of

Teaching (CFAT).

Currently, CASTL participants use a Web-based communication system called Workspace, which provides a document posting area and other functions. Occasional users of

Workspace are not sufficiently satisfied with its functionality and current state of usability.

My challenges in redesigning this communication system are to investigate what functions are necessary in the system, what level of functionality and usability encourage users to frequently utilize this system and to continue to use it, and what role multimedia plays in archiving teachers’ expertise and practices, called the scholarship of teaching.

Informal interviews were conducted and former in-house discussions were analyzed. I proposed a new design of Workspace. In addition, I will assess the communication process and usability, and present a future direction for further improvement.

iii For my wife and son, Yukiko and Fumiyasu.

iv Acknowledgments

This project is supported in party by my internship work at the Carnegie Foundation for the

Advancement of Teaching. I could have not finished my master project work without any help from the Carnegie Foundation. Especially, Dr Toru Iiyoshi, the director of Knowledge

Media Laboratory as my internship supervisor, he gave me many opportunities and allowed me to use high-tech equipment. In addition, his gracious and frequent advice and support enabled me to deeply understand the site and to study in the right direction. Dr.

Tom Hatch, a senior scholar of the Carnegie Foundation, gave me insightful comments and excited me to move forward.

I would like to thank Decker Walker and Deborah Kim. Professor Decker Walker gave me useful advice and appropriate comments. Program coordinator Deborah Kim gave me practical comments and encouraged me to keep going. I also thank Professor James Greeno and Professor Brigid Barron for their suggestions and comments.

Comments and suggestions from Dr. Laurie Wayne and Dr. Lois Brooks, at the master project presentation, were very useful and inspired me to rethink the future direction. In addition, many visitors at the master project EXPO gave me valuable advice and comments.

Finally, I want to thank my immediate family. Yukiko, thank you for waiting patiently for me to overcome this long and crucial hurdle. Thanks to Fumiyasu. I was so lucky to have the little interrupting knocks on the door. His smile always encouraged me.

v Table of Contents

ABSTRACT...... III

ACKNOWLEDGMENTS...... V

TABLE OF CONTENTS...... VI

LIST OF TABLES...... VIII

LIST OF FIGURES...... IX

CHAPTER ONE...... 1

INTRODUCTION...... 1

BACKGROUND...... 1 CHAPTER TWO...... 4

SITE BACKGROUND...... 4

CARNEGIE FOUNDATION...... 4 CASTL...... 5 SCHOLARSHIP OF TEACHING (SOT)...... 6 CASTL COMMUNITY...... 6 KML...... 7 WORKSPACE...... 9 CHAPTER THREE...... 11

PROBLEMS AND NEEDS...... 11

ANALYSIS OF CASTL COMMUNITY...... 11 PROBLEMS...... 11 NEEDS...... 16 CHAPTER FOUR...... 17

DESIGN PRINCIPLES...... 17

TECHNOLOGY-ENHANCED SCHOLARSHIP OF TEACHING...... 17 MEDIA RICHNESS...... 18 INCREASE INTERACTIVITY...... 18 vi ENHANCE MOTIVATION...... 19 CONSIDER AUDIENCE DIVERSITY...... 20 CHAPTER FIVE...... 22

FEATURES OF REDESIGN...... 22

CHAPTER SIX...... 25

IMPLEMENTATION...... 25

CHAPTER SEVEN...... 29

EVALUATION & USER TESTING...... 29

EVALUATION...... 29 USER TESTING...... 29 CHAPTER EIGHT...... 31

DISCUSSION AND FUTURE WORK...... 31

LIMITATIONS...... 31 DISCUSSION...... 31 FUTURE WORK...... 32 APPENDIX A...... 33

SCREEN SHOTS OF THE CURRENT WORKSPACE...... 33 APPENDIX B...... 35

SAMPLE PROJECT GOAL...... 35 APPENDIX C...... 36

SCREEN SHOTS OF THE NEW WORKSPACE...... 36 APPENDIX D...... 37

RESUME...... 37 LIST OF PUBLICATIONS...... 39 Journal Articles...... 39 Refereed Conference Proceedings...... 39 Book Review...... 40 University Thesis...... 40 REFERENCES...... 41

vii viii List of TABLES Table 1: Classification of the Technology-enhanced Scholarship of Teaching...... 21 Table 2: User Testing Plan...... 30

ix List of figures Figure 1: Mailing List Traffic (1998 Participants)...... 13 Figure 2: Mailing List Traffic (1998 Participants)...... 14 Figure 3: Mailing List Traffic (1999 Participants)...... 15 Figure 4: Design Process...... 26 Figure 5: Development Process I ...... 27 Figure 6: Development Process II ...... 28 Figure 7: Listserv Archive...... 33 Figure 8: Scholars Workspace...... 34 Figure 9: Mockup demo of new Workspace...... 36

x Chapter One

Introduction

Background

Network-based communication, especially asynchronous systems including e-mail, newsgroup, and web-based forums conform well to the time-restricted schedules of teachers. Features such as time and place independence, multiple participants, fast searching, message storage and retrieval, and interactivity validate the ability of the computer network to foster communication among users (Hawkes 1999). Therefore, asynchronous learning tools have enhanced teacher education in many ways, including creating links with the K-12 community, providing professional development opportunities, encouraging collaborative relationships, and developing new ways of mentoring (McMullen 1998).

Specifically, a number of early attempts have met the users’ administrative needs, but not curricular needs. Besides, there have not been many research activities on the assessments of system integration. This project explores in depth one such system: Workspace, and its coverage of various teachers’ needs.

Boyer, the last President of CFAT, initially used the term “scholarship of teaching”.

Shulman, CFAT’s current President, has identified three requirements for transforming excellent teaching into the scholarship of teaching and learning: ”We must make documented teaching and learning theory and practice publicly accessible, open to peer

1 critical review, and available for reuse” (Cambridge 1999). Web-based computer systems can play an important role to fulfill Shulman’s three requirements and develop a practical system. Web-based communication systems enable a large audience to access the data in the system. Specifically, threaded discussions and real-time chats enhance peer reviews of teaching practices, even if participants are located in different places. In addition, a web- based database and search functionality increase the reusability. After teachers browse and retrieve teaching practices and multimedia data on this web-based system, they can recreate their own teaching materials and curriculum.

The Carnegie Academy for the Scholarship of Teaching and Learning (CASTL) is a project aiming to build a scholarship of teaching on a nation-wide scale. In this project, participant teachers, called Carnegie scholars, develop their scholarship of teaching and learning, which will: “(1) Foster significant, long-lasting learning for all students (2) Enhance the practice and profession of teaching (3) Bring to faculty members’ work as teachers the recognition and reward afforded to other forms of scholarly work in education.” (CFAT brochure, 1999). This project includes a two-year program for teachers and teacher educators from K-12 and a one-year program for teachers from higher education. They investigate and document their work on issues in the teaching and learning of their own field. Carnegie Scholars usually spend two 10-day summer sessions together near the campus of the Carnegie Foundation and additional time during the academic year. They work primarily in their own academic settings, which are located nation-wide. Given this opportunity, an effective communication system will become an important tool among

Carnegie Scholars and administration staff.

2 Workspace is a communication support system for teachers and administration staff in the

CASTL project. To satisfy the three requirements mentioned above, this web-based system provides e-mail, threaded discussion groups, and posting and sharing multimedia documents to promote mutual assessment.

Up to this point, Workspace has been used for one and a half years in the CASTL project, primarily for making announcements, posting teachers’ project proposals and final documentations, and for sharing all of these. However, the participants have not frequently used Workspace. Because there are few peer reviews through the threaded discussion forum, users lack feedback from the system. It is not clear whether they benefit much from this web-based system.

3 Chapter Two

SITE BACKGROUND

In this chapter, I describe a targeted communication support system and its organization and community. Carnegie Foundation has a professional development program called

CASTL (Carnegie Academy for the Scholarship of Teaching and Learning). The

Knowledge Media Laboratory have developed its communication support system, called

Workspace for CASTL community.

Carnegie Foundation

The Carnegie Foundation for the Advancement of Teaching (CFAT) is a not-for-profit corporation founded by Andrew Carnegie in 1905 and chartered by an act of Congress in

1906. CFAT has a long and distinguished history. It is an independent policy and research center, whose primary activities of research and writing have resulted in published reports on every level of education. Eight presidents have guided the Foundation through its history, each bringing unique shape to its work.

The mission of CFAT as expressed in the founding charter is: “to do and perform all things necessary to encourage, uphold, and dignify the profession of teaching.” The founders created institution dedicated not only to the advancement of teaching in general but also to the “profession of teaching” in particular. To this day, the Foundation is rooted in its original mission, strengthening the future of the profession of teaching and the calling of

4 educator. The Foundation fulfills its mission through its contributions to improvements in

education policy and practice.

CFAT has several programs. One of the most proactive programs is the Carnegie Academy

for the Scholarship of Teaching and Learning (CASTL). In the CASTL project, participant

teachers, called Carnegie scholars, will seek a scholarship of teaching and learning that

will: (1) Foster significant, long-lasting learning for all students (2) Enhance the practice

and profession of teaching (3) Bring to faculty members’ work as teachers the recognition

and reward afforded to other forms of scholarly work in education. This program includes a

two-year program for teachers and teacher educators from K-12 and a one-year program

for teachers from higher education.

CASTL

The Carnegie Academy for the Scholarship of Teaching and Learning (CASTL) seeks to

support the development of a scholarship of teaching and learning that will:

 Foster significant, long-lasting learning for all students

 Enhance the practice and profession of teaching

 Bring to faculty members' work as teachers the recognition and reward afforded to other

forms of scholarly work

5 CASTL entails work on several fronts. These include a Program for faculty in Higher

Education and a Program for K-12 Teachers and Teacher Educators who spend one or two years documenting some aspect of their teaching practice and partnerships with a variety of campuses, school reform networks and scholarly societies in both K-12 and Higher

Education.

Through these efforts CASTL hopes to help establish and refine scholarly standards for the critical review of work on teaching and learning; to establish new settings, forums and forms through which the scholarship of teaching and learning can be exchanged widely; and underline the character of teaching itself as a complex and challenging endeavor worthy of support and reward.

Scholarship of Teaching (SoT)

SoT includes three features. One is being public, which means that teaching practices and methodologies would become community property. The second open to critique and evaluation and the third is availability for reuse, which means SoT should be in a form that other people can build on.

Besides, there is a forth feature, which is implied by the above three, it involves question- asking, inquiry and investigation, particularly around issues of student learning.

6 CASTL Community

CASTL community has unique features. First, it is a closed community. When members are selected and fixed, no change has occurred throughout the program period. There is no addition and no subtraction of members. It is different from a community open to public.

Besides, participants are located across the country. Therefore, it is impossible to meet once a week or even once a month. Participants have face-to-face two-week meetings twice a year. The first one is held in mid-June, that is the beginning stage of this program, and then, members can know each other at that meeting. It is not a community which is totally virtual, or on-line. The size of this community is about from 20 to 30 in one program. It is relatively very small community. Therefore, they can make deeper connection with each other

Members are selected by their project proposals, and the selection process is independent of their technological skills. Therefore, it is different from a community in which all participants are interested in technology and computers.

Participants are from variety of subject areas. This year’s participants are from Chemistry,

Mathematics, English, History, Business, Psychology, and Performing arts.

KML

The Knowledge Media Laboratory (KML) is a multimedia laboratory in CFAT. KML provides resources and information to support and document the development and pursuit

7 of the scholarship of teaching. One of the main projects in KML is to develop the

Workspace, which provides such resources as a virtual workspace for teacher-scholars,

exhibitions of the scholarship of teaching in multiple formats and multimedia resources for

the support of the scholarship of teaching.

The Knowledge Media Laboratory (KML) is a multimedia laboratory, which is designed to

take advantage of emerging technologies to support the development of the scholarship of

teaching in both K-12 and Higher Education. Its functions include:

 Developing online communication tools, such as a virtual collaboration environment called

the Workspace for a small group of faculty who are documenting their teaching and sharing

what they are learning with others as part of the Carnegie Academy for the Scholarship of

Teaching and Learning

 Facilitating the efforts of those and other faculty who are attempting to use a variety of

media in order to present their investigations

 Developing resources that promote the understanding and pursuit of the scholarship of

teaching. These resources include articles, annotated bibliographies, materials related to

intellectual property, technical tutorials, etc. and are provided in a variety of formats such

as Website, CD-ROM, and DVD.

 Helping to build a larger network of faculty and organizations that are interested in

advancing collective understandings of teaching and learning.

8 While the development of the KML is still in the early stages, it seems likely that the

development of a scholarship of teaching in which teachers make their work public, subject

it to critical review, and share what they are learning with others may benefit substantially

from the opportunities to communicate and share resources with other faculty around the

country that come with participation in online environments like the Workspace. At the

same time, the further development of these environments depends on developing new

models of the scholarship of teaching, new forums for sharing reflections and

investigations of teaching across such environments, and more substantive connections

among many of the individuals and groups that are involved in related work.

Workspace

The Workspace is designed to enable the participants in the CASTL programs and

Carnegie staff to post and share project documents (drafts as well as final versions),

comments, and resources (articles, websites, and other online materials) that may be useful

in the completion of the participants’ projects. The Workspace is comprised of a set of

asynchronous and synchronous web-based communication tools and online resource

databases. There are eight components.

1. Scholar Workspace

Scholars’ Workspace displays a listing of users’ posted documents relating to their project and response documents.

2. Discussion Listserv 9 This database is used as an email repository for information relating to the Carnegie Foundation. This enables users to post, respond, and reply to various topics related to the Scholarship of Teaching.

3. News

This news forum is designed to serve as a “bulletin board” where any member of the project can post information about events, publication, important statements, presentation, etc.

4. Program Documents

This area is used as announcements from administration staff of CASTL program.

5. WWW Resources

This database contains related organization and project inside/outside of CASTL program.

6. Scholar Directory

This is a directory of each member’s contact information.

7. Email Speed Dial

This is a list of email addresses, which has hot link for launching users’ email software.

8. Calendar of Events

This is a calendar schedule, which can be posted only by the CASTL administration staff.

There are a few screen shots of the current version of the Workspace in Appendix A.

. 10 Chapter Three

Problems and Needs

Analysis of CASTL community

In the CASTL community, each participant has his/her own project. Therefore different people have different goals. Some one is trying to frame problem-based learning, and some one is trying to investigate project-based learning. No same goals are shared in this community. In this CASTL program, participants spend most of their time on their campuses of schools. Therefore, they would do self-learning or may learn from their colleagues or their students. Peer review is the main learning opportunity in this program.

Participants join two face-to-face meeting a year. They may directly email or call each other. In addition, they have possibilities to do peer review through the Workspace. This is my focus in this master project. One sample participant’s project goal is in Appendix B.

Problems

Like other systems, the current version of Workspace was designed and implemented without sufficient investigation of user needs or systematic analysis. It is somewhat preliminary as a system to start actual trials. Therefore, initially it was used mainly for such administrative communications as posting announcements and archiving proposals and final documents. In addition, this system is hindered by several functionality and usability problems, mainly because this system was quickly built and designed in an administrator-

11 oriented style, not teacher-oriented. In addition, there is a wide range of computer skills among the participant teachers from novice to expert. Consequently, it is very difficult to meet the needs of all users in one system. From the teachers’ viewpoint, there are two major problems: (1) Functionality issues such as confusing categorization and (2) Usability issues such as lack of consistency and difficult navigation. Therefore, participants do not always actively use this system,, then they have not received enough advantage through this communication system, such as peer review and understanding of other’s work. They might have a chance to develop their project more deeply when they had used the

Workspace effectively.

From an informal interview with administration staff, I found that it was very hard for administrators to organize and analyze the scholars’ activities, and difficult to present their products systematically. In addition, I have found the reasons of these problems. The current version of the Workspace allows users to post and share only with text, excluding attached documents. Besides, no appropriate communication tool is provided, when participants needed. Some usability issues such as lack of consistent navigation, and confusing categorization can be found, shown in the screen shots in Appendix A. For example, participants are easily confused with a few posting forums: where do I post?

Scholar Workspaces or News? Finally, in some respect, distrust of the new system and new technology reduce frequency of the usage.

Quantitative analysis of communication is not the best way, however this kind of analysis would give us some clues to identify and solve problems. Figure 1shows the traffic of

12 mailing list among 1998 participants. People who participated in 1998 CASTL program

can use this Workspace continuously and are encouraged to do so. Figure 1 shows the

number of e-mails sent by mailing list server month by month.

This mailing list started in December 1998. In January, there was a face-to-face meeting.

At that time, traffic is a little increase, then decreasing gradually. Jun was a due date of

project document, which was announced by the CASTL staff and the second face-to-face

meeting was held in mid June.. At this time, the traffic is going up, then, it is also

decreasing gradually. After that, the next meeting was January 2000. Therefore, the traffic

is rapidly growing and then decreasing.

Peer-to-peer e-mail or phone call cannot be tracked and communication among participants

cannot be exactly measured and cleared.

50 45

40 35 30 l i a

m 25 - E 20 15 10

5

0

8 9 9 9 9 9 9 9 9 9 9 9 9 0 0 0 0 0 /9 /9 /9 /9 /9 /9 /9 /9 /9 /9 /9 /9 /9 /0 /0 /0 /0 /0 2 1 2 3 4 5 6 7 8 9 0 1 2 1 2 3 4 5 1 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 13 Month Figure 1: Mailing List Traffic (1998 Participants)

In this mailing list, the number of responses and feedback are relatively small, and most of them are announcement.

14  In Figure 2, the same mailing list traffic as Figure 1 is represented in two types of mail

senders: program participants and program administrative staff.

1998 Scholars

40

35

30

25 l i

a Participants

m 20 - Staff E 15

10

5

0

8 9 9 9 9 9 9 0 0 /9 /9 /9 /9 /9 /9 /9 /0 /0 2 2 4 6 8 0 2 2 4 1 0 0 0 0 1 1 0 0 Month

Figure 2: Mailing List Traffic (1998 Participants)

Figure 3 represents mailing list traffic of 1999 participants. The traffic pattern is almost as

same as Figure 1. This mailing list did not also provide much interactive feedback about

participants’ project activities. The participants seem to post their message/documents,

stimulated by the CASTL administration staff.

15 35

30

25 l

i 20 a m -

E 15

10

5

0

9 9 9 9 9 9 9 9 0 0 0 0 0 /9 /9 /9 /9 /9 /9 /9 /9 /0 /0 /0 /0 /0 5 6 7 8 9 0 1 2 1 2 3 4 5 0 0 0 0 0 1 1 1 0 0 0 0 0 Month

Figure 3: Mailing List Traffic (1999 Participants)

16 Needs

Under the current Workspace, teachers consider this system low priority to utilize. In addition, they do not receive sufficient benefits from the activities related to Workspace. It seems to be an extra effort for users. However, fundamentally, a web-based system like

Workspace can provide many possibilities and bring teachers new benefits for teaching and learning, such as obtaining effective feedback from critical peer reviews and conceiving new ideas from other teachers’ suggestions anytime and anywhere.

On the other hand, teachers originally have the need to present their teaching expertise and practices in public, but they claim the lack of skills and methods in presenting their knowledge on the Web. If an appropriate communication system is provided, teachers will consider the system more beneficial, share other teachers’ accomplishments easily, and integrate and improve their own knowledge.

17 Chapter Four

Design Principles

Technology-enhanced Scholarship of Teaching

Table 1 shows the classification of the technology-enhanced scholarship of teaching

(SoT). Technologies, which are helpful and effective for each three functions of SoT, are

presented as well as challenges. Each function of SoT has its own goals. Technologies can

support and enhance each activity to pursuit the goals.

In many respects, online collaborative environments provide ideal opportunities for the

development and pursuit of SoT, opportunities that are not normally available to many

faculty in their own schools and on their own campuses:

1. They provide a means for faculty to make some of the work that they normally do in

isolated classrooms, behind closed doors, accessible to others.

2. They provide a forum in which faculty can get feedback from peers beyond their local

campuses.

3. They enable faculty to refer to, use and build on the activities, projects, rubrics, and

examples of student work of their peers.

Overall, these activities may make it easier to look across the work in different classrooms

and schools, to recognize the new ideas and understandings that are being achieved by

faculty, and to generalize and develop insights that can advance the profession as a whole. 18 This work is not without substantial challenges, however. These include many of the same challenges faced by many groups and projects that are also concerned with supporting teachers who are documenting and reflecting on their practice including limits on time and lack of appropriate incentives. They also include some of the well-known challenges of working in online environments including issues of access, technical expertise, intellectual property, and building critical mass.

Media richness

Generally, multimedia reduces ambiguity through proper media selection in human communication. In the current Workspace, teachers can collaborate only with text communication. Therefore, if teachers can have more possibilities to use more than text media, such as images, sound, movies, etc, the representation of teaching practices will reduce its vagueness and enhance its explanation.

Increase Interactivity

From the informal interview and discussions, interactivity is one of the big incentives for teachers to use this kind of online communication tool and to find effectiveness easily.

19 Enhance Motivation

As analyzed before, online communication is difficult to be kept continually active without any facilitating. Four activities are designed to solve these problems.

1. Orientation session in the first face-to-face meeting

In this session, as well as knowing prior knowledge of teachers, clear explanation and encouragement is one of the main purposes of this orientation.

2. Computer Lab Hours in the first meeting

This section is useful for teachers, who have interests with technology, but do not have enough technological skills. This hands-on activity will reduce teachers’ hurdles and encourage them to use the system.

3. Presenting appropriate examples for both novice and experienced teachers

Good examples are very attractive for experienced teachers, but they may discourage novice teachers to think that they are beyond their competence. Therefore, we will choose appropriate examples and show them in a big audience, then we may show a little complicated examples for teachers who want to know more.

4. Help function

An instant help will be provided for each communication tool and each web page. We have to avoid such situations that teachers reach dead ends and they do not know how to make a contact to solve problems. FAQ database, phone numbers of participants and administration staff, and resources of relevant web pages will be provided.

20 Consider audience diversity

Most users do not know enough technological expertise, however some have enough skills.

Therefore, we have to answer needs from novice to expert users. User-Interface should be as simple as we can. Some individualized adaptive technology will be integrated with the communication system.

Most teaches access to the Workspace from both their school and home. As well as home

Internet, some schools do not always have high-speed Internet connection. Therefore, we had better make web pages as small as we can. In addition, streaming technology is a better solution for both high and low speed users, rather than downloading huge movie files.

21 

Table 1: Classification of the Technology-enhanced Scholarship of Teaching

Functions Goals Tools, Resources, and Challenges Products Make teaching Create project Scholars’ homepages Developing a wide range of practice public documents and Electronic teaching formats, that faculty can use examples of the portfolios to represent their work. Scholarship of Multimedia examples of Providing technical support Teaching using the SoT for collection, analysis and multimedia presentation in the local contexts where faculty work. Subject it to peer Disseminate project Online discussion forums Developing common review documents and Listservs language and criteria for multimedia examples Chat Rooms discussing teaching across a of the SoT through Theme-based discussion range of groups and the Workspace forums contexts. Establishing formal and informal review routines with appropriate asynchronous and synchronous communication tools. Build on one Build and sustain the SoT resource databases Creating dynamic databases another’s work SoT knowledge-base Multimedia examples of and appropriate searching and networks the SoT strategies that make it Online research data possible for faculty to find archive relevant examples of the “Friends & Partners” scholarship of teaching web links easily. Going “beyond the link” to establish substantive online and organizational connections across campuses, school reform networks, scholarly societies and other CASTL partners.

22 Chapter Five

Features of Redesign Eleven key features are redesigned and their roles are reassigned by the new design feature.

1. My Workspace

My Workspace displays teacher’s profile information, including picture, address and contact information. This area also captures teacher’s postings regarding their projects, as well as any response documents relating to these projects. Scholar Workspace in the current Workspace is broken into My Workspace and Scholars Workspace. My Workspace is not a mere posting and viewing area. It allows teachers to present their work with multimedia. That is a kind of personal web pages.

2. Scholars Workspace

Scholars’ Workspace displays a listing of users (defaulting to group user belongs to) with relating project and response documents. This area also allows users to view each teacher’s profile information. It is an integration with “Email Speed Dial” in the current Workspace, because this email function is redundant and not used so frequently.

3. Listserv Archive

This database is used as an mailing list repository for information relating to the Carnegie

Foundation. This is a succession of the current function, because many users need this archive and it is frequently used. 23 4. Discussion Forum

This online threaded discussion forum is used as an “Special Interest Group” activity rather than for a whole CASTL group. This enables users to post, respond, and reply to various topics related to the Scholarship of Teaching. This is a new function, which is wanted by man teachers and administration staff.

5. Program Information

This database contains current news and program documents relating to the various educational groups within the Carnegie Foundation. This is a succession of the current function, “Program

Documents”.

6. Schedule of Events

This database displays news and event items relating to the various educational groups within

CASTL. This is a succession of the current function, “Calendar of Events”.

7. Resources

This database contains helpful online resources for teachers to understand and pursue the

Scholarship of Teaching. The database contains resources such as media files and URL links.

“WWW Resources” is integrated with this function.

8. Chat Rooms

24 Chat Rooms enable users to interact with each other in real time. There are three dedicated chat rooms for the purpose of online real-time discussion related to the Scholarship of

Teaching. It is a new feature, which several teachers really want to have.

9. Friends & Partners

This database contains information with links to various partner organizations (universities, foundations, etc.) associated with CASTL. It is a new feature.

10. Staff Directory

This database contains contact information of CASTL and KML related Carnegie staff. This area includes contact information, staff picture, address information, and email addresses. It is also a new feature.

11. Search

The tool provides a mechanism for searching for items throughout the entire website. This is a succession of the current system, but more advanced search options will be added.

25 Chapter Six

Implementation

In this chapter collaborative work within the Workspace development team is described.

There are three categories of people are concerned with this Workspace implementation.

CASTL staff, KML staff, and Software development team. In the design process, Figure 4 shows that who are responsible and join which process.

In Figure 5, a development process where Workspace components are developed by the software development team, is described. Figure 6 shows another development process, which includes components developed by KML. Discussion Forum, Chat Room,

Multimedia Resource Database will be developed by KML rather than by software development team because of some financial and political issues.

26 Figure 4: Design Process

CASTL Staff Soft Dev CASTL Staff KML KML KML Softw Dev.

Evaluation of Current Functionality Joint Meeting Workspace

Specification

27 Figure 5: Development Process I CASTL Staff KML KML KML Softw Dev. Softw Dev. Softw Dev.

Implementation Joint Meeting Spec Meeting Meeting

Improved Interim Semi-final Specification Specification Product Product

28 Figure 6: Development Process II

CASTL Staff KML KML

Joint Integration Meeting

Semi-final Self-developed Final Product Tools Product

29 Chapter Seven

Evaluation & User Testing

Evaluation

These are four evaluation criteria. First, usefulness and satisfaction of each tool. I would like to identify which tool is useful and which is not useful in terms of doing participants’ project work. Secondly, to evaluate each communication tool, we might investigate the traffic data for web access, e-mails, and so on. In addition, how frequent the interactions among participants were and how deeply they communicated with each other. Third, I would like to identify usability problems. What problems and difficulties did they face with while using the Workspace? Finally, we will find new needs. What additional help and tools are needed?

User Testing

User testing plan is described in this section. At this moment, they have not been done yet.

They will be scheduled and done in this year and early next year.

I am planning three possibilities to make user tests. First, observation, we will have an orientation session and computer lab hour in mid June at the face-to-face meeting. User

30 activities are monitored there and it will be identified that where users are stuck and what questions they raise.

After this meeting, sometime one-month or two months later, a simple web-based survey will be conducted to know users’ voice. Finally, I will observe web access and track user logs for several month. Moreover, I would like to identify what tool is most and less important for peer review.

Table 2 shows which user test activity corresponds to the evaluation criteria.

Table 2: User Testing Plan

Obse Question Trac rvati naire king on

Useful X X ness

Interact X X ion

Usabili X X ty

Needs X X

31 Chapter Eight

Discussion and Future Work

Limitations

This project focused on the analysis of participants’ interaction, and design and development of communication tools. The improvement or any change of the CASTL program itself is not included. The tools and the system were designed only for the

CASTL community.

Discussion

In chapter two, mailing list traffic was analyzed. However, peer-to-peer e-mail or phone call cannot be tracked and its quality and quantity cannot be identified either. At most, I might say that mailing list communication was motivated by the Carnegie Staff posting as a facilitator and asking participants to post documents. Such a function of a facilitator usually plays an important role in an online community.

I focused on the specific community and mentioned a limitation of my study above.

However, my design principles and communication tool integration would be applied to other community, which is partially or fully online. One of the key issues is how to evaluate communication among community effectively. The most difficult part is to measure interaction among teachers effectively, to select appropriate communication tools, and to build a consistent system as a whole.

32 Future Work

The implementation of the Workspace and system integration will be done by the end of

May, 2000. Then we will move all data from the current Workspace to the new one. In addition, CASTL meeting will be held in mid-June, 2000. I will join the Workspace introductory session and stay at the computer lab to observe and help participants. Web survey will be made a couple of month later after the meeting, and I will gather user- tracking data for about six months. Then, I will do some summative evaluation for the next millennium version of the Workspace. Some communication tools will be omitted and new tools will be added. There are some new features already discussed in Workspace design meeting. One possibility is a video annotation system. Its streaming technology enables even low-bandwidth users to share video files and put comments on the specific point of the video.

33 Appendix A Screen shots of the current Workspace Figure 7: Listserv Archive

34 Figure 8: Scholars Workspace

35 APPENDIX B Sample Project Goal Professor Stephen Chew, Psychology, Samford University

Title: “ Using Examples in Teaching: Progress and General Reflection”

She focused on the use of examples in teaching. Her three main questions are:

 What tare the properties of a good example

 What is the best way to use examples in teaching?

 What do students learn form examples?

36 APPENDIX C Screen shots of the new Workspace Figure 9: Mockup demo of new Workspace

37 Appendix D Resume Yasuhisa Kato

965 E. El Camino Real #924 Sunnyvale, CA 94087

[email protected]

Education

1999- Stanford University Graduate School of Education, Stanford, CA Master program of Learning, Design, and Technology (LDT) 1988-1990 Kyoto University Graduate School of Engineering, Kyoto, Japan Master of Engineering in Applied Systems Science, Mar. 1990 1984-1988 Kyoto University, Kyoto, Japan Bachelor of Engineering in Applied Mathematics and Physics, Mar. 1988

Work Experience

1999- Carnegie Foundation for the Advancement of Teaching, Palo Alto, CA Research Assistant, the Knowledge Media Laboratory 1990-1999 NTT (Nippon Telegraph and Telephone Corporation), Tokyo, Japan 1997-1998 NTT America Inc., New York, USA

1997-1999 Research Engineer, Contents Handling Project at Cyber Solutions Laboratories, Tokyo, Japan (from 1999), Tele-Education Project at Information and Communication Systems Laboratories, Tokyo, Japan (1998) and Technical Manager, NTT America Inc., New York (1997-1998)  Manage and promote the international education network project between NTT and Keio Academy.  Develop and program Web-based educational systems.  Design educational courseware.  Supervise three project members and three engineers.  Administer and manage the international experimental network.  Promote products of NTT’s Laboratories. (New York)

1996.1997 Research Engineer, Tele-Education Project in Information and Communication Systems Laboratories, Tokyo  Joined MAP (Multimedia Application Project); an international joint project of AT&T, KDD, and NTT to determine the feasibility of an international ATM network.  Designed a new simulation mechanism for an Intelligent Tutoring System.  Developed educational courseware.  Administered the experimental computer network shared by many laboratories.

38  Supervised one engineer.

1994-1996 Research Engineer, Multimedia Service Application Laboratory in High-speed Computing Laboratories, Tokyo  Started Media-Wonderland Project, now known as HyCLASS (Hyper Collaborative Learning Application in Shared Space)  Developed and programmed the sound/voice module of HyCLASS.  Instructed one graduate trainee from Switzerland.  Administered and managed the computer network of the laboratory.  Supervised one engineer.  Attended the summer workshop, Introduction to Psychoacoustics and Psychophysics: Audio and Haptic Components of Virtual Reality Design, at CCRMA, Stanford University. 1990-1994 Engineer, Voice Processing Service Laboratory in Message System Laboratories, Tokyo  Researched and developed the voice browsing and voice recognition system.  Supervised one engineer.  Instructed three undergraduate trainees.  Administered and managed the computer network of the laboratory.  Attended the European Summer School on Language and Speech Communication in Utrecht, the Netherlands.  Recruited new employees at Kyoto University.

Awards

1997 A special Laboratory President’s award for developing CALAT (Computer Aided Learning and Authoring System for Tele-Education)

1996 Outstanding paper award at ED-MEDIA’96 in Boston.

Skills

 Proficient in computer skills: MacOS, Windows, SunOS, FreeBSD, Linux, etc

 Proficient in programming languages: C, Java, Perl, HTML, etc

 Network management: Network design, administration, etc.

 Proficient in written and spoken English.

Interests

 Outdoor sports and activities: cycling, motorcycle touring, camping.

 Japanese taiko drumming, and playing ethnic music.

 Digital photography, especially of my two-year-old son.

 Visiting museums of art, science, and history, zoos, and aquariums. (I have visited more than 50 such institutions worldwide)

39 List of publications Journal Articles  K. Hosoya, Y. Kato, A. Kawanobe, S. Kakuta, and Y. Fukuhara, "A Collaborative Educational Environment Based on a Multi-User Virtual Space," Systems and Computers in Japan, Vol.28, No.8, pp.1-7 (1997)

 K. Hosoya, Y. Kato, A. Kawanobe, S. Kakuta, and Y. Fukuhara, “Collaborative Learning Environment Design methods using multi-user virtual world,” IEICE, Apr. (1997) (in Japanese)

 Y. Kato and Y. Yamamoto, “On learning of neural networks with feedback connections,”

 Systems/Information/Control, vol. 4, pp. 369-374 (1991) (in Japanese)

Refereed Conference Proceedings  H. Tsuchiya, N. Takahashi, Y. Kato, and H. Baba, “The Method of Learning Internet Literacy in English Lessons,” ED-MEDIA’99 (1999)

 Y. Kato, N. Takahashi, H. Tsuchiya, K. Nakabayashi, Y. Fukuhara, and H. Tokuda, “International Integrated Educational Network,” ED-MEDIA’98 (1998)

 T. Hoshide, H. Touhei, Y. Kato, K. Nakabayashi, and Y. Fukuhara, “A simulation Environment for an Intelligent Tutoring System on the WWWW,” ED-MEDIA’98 (1998)

 Y. Kato, J. Hirai, Y. Fukuhara and M. Yamada, “International Tele-Education Experiment using CAI system on the World-Wide Web,” WebNet’97 (1997)

 K. Nakabayashi, M. Maruyama, H. Touhei, Y. Kato, K. Takiuchi, and Y. Fukuhara, “An ITS on the WWW with Interactive and Adaptive Courseware,” ED-MEDIA’97 (1997)

 K. Hosoya, A. Kawanobe, S. Kakuta and Y. Kato, “Interactive CSCL System Based on Three-Dimensional Shared Space,” ED-MEDIA’97 (1997)

 S. Kakuta, A. Kawanobe, Y. Kato and K. Hosoya, “Interactive 3-D Physical Simulation Environment in a Virtual World,” ED-MEDIA’97 (1997)

 Kawanobe, Y. Kato, S. Kakuta and K. Hosoya, “The Proposal for the Management method of Session and Status in Shared Space,” International Conference on Virtual Systems and Multimedia (1996)

40  K. Hosoya, Y. Kato, S. Kakuta, A. Kawanobe, and Y. Fukuhara, “HyCLASS: Cooperative Learning Environment for Tele-Education,” TINA’96 Conference (1996)

 Y. Kato, S. Kakuta, A. Kawanobe, K. Hosoya, and Y. Fukuhara, “Advanced Collaborative Educational Environment using Virtual Shared Space,” ED-MEDIA'96 (1996)

 S. Kakuta, A. Kawanobe, M. Hisamatsu, Y. Kato and K. Hosoya, "Media Wonderland: User Oriented Multimedia Telecommunication Environment for Multiuser," Telecom 95 Technology Summit (1995)

 Y. Kato and K. Hosoya, “Voice Message Summary for Voice Services,” ISSIPNN'94 (1994)

 Y. Kato and K. Hosoya, “Fast voice message retrieving method,” IPSJ SIG-SLP (1993) (in Japanese)

 Y. Kato and K. Hosoya, “Message Browsing Facility for Voice Bulletin Board Service,” HFT'93 (1993).

 Y. Kato and K. Hosoya, “Fast Message Searching Method for Voice Mail Service and Voice Bulletin Board Service,” AVIOS'92 (1992)

Book Review  Y. Kato, Book Review of “Virtual Community” by Howard Reingold, Journal of Artificial Intelligence, vol.10 No.6 (1995) (in Japanese)

University Thesis  “ A learning algorithm of neural networks with feedback connections,” Master thesis, Graduate School of Engineering, Kyoto University, (1990) (in Japanese)

 “An approximation method of energy eigenvalues of one dimension Shrodinger equation using Lagrange interpolation,” Bachelor thesis, Kyoto University, (1988) (in Japanese)

41 References Applehans, W., Globe, A., and Laugero, G. (1999). Managing Knowledge: A Practical Web-Based Approach. Reading, MA: Addison Wesley Longman, Inc.

Black, J. (1997). Online Students Fare Better. http://teleeducation.nb.ca/anygood/

Borenstein, N. S. (1991). Programming as if people mattered: friendly programs, software engineering, and other noble delusions. Princeton, N.J.: Princeton University Press.

Boyer, E. (1990). Scholarship Reconsidered: Priorities of the Professorate. Princeton: Carnegie Foundation for the Advancement of Teaching.

Cambridge, D. (1999). Supporting the Development of a National Constellation of Communities of Practice in the Scholarship of teachings and Learning Through the Use of Intelligent Agents. Computer Support for Collaborative Learning.

Carnegie Foundation for the Advancement of Teaching. http://www.carnegiefoundation.org/

CTIE: Center for Technology Innovations in Education. http://www.ctie.missouri.edu/home.htm

CUE: Computer-Using Educators. http://www.cue.org/

Davis, M. (1997). Fragmented by Technologies: A Community in Cyberspace. Interpersonal Computing and Technology, 5, 1-2, 7-18.

EDUCATION WEEK. (1999) Technology Counts ’99.

Ess, C. (1996). Philosophical Perspectives on Computer-Mediated Communication. Albany, NY: State University of New York Press.

Flagg, B. (1990). Formative Evaluation For Educational Technologies. Hillsdale, NJ: LEA.

Fleming, J. (1998). Web Navigation: Designing the User Experience. Sebastopol: O’Reilly & Associates, Inc.

Garmer, K. and Firestone C. (1996). Creating a Learning Society: Initiatives for Education and Technology. A Report of The Aspen Institute Forum on Communications and Society.

Greenbaum, T. (1993). The handbook for focus group research. New York: Lexington Books

Hawkes, M. (1999). Exploring Network-Based Communication in Teacher Professional Development. Educational Technology. 39, 7, 45-52. 4 Intraspect. http://www.intraspect.com/

ISTE: International Society for Teacher Education. http://www.iste.org/

Johnson, D. (1997). Extending the Educational Community: Using Electronic Dialoguing to Connect Theory and Practice in Preservice Teacher Education. Journal of Technology and Teacher Education, 5, 2-3, 163-170.

Kim, A. J. (2000). Community Building on the Web. Berkley, CA: Peachpit Press.

Krathwohl, D.R. (1998). Methods of Educational and Social Science Research: An Integrated Approach (2nd edition). New York: Addison-Wesley

Lotus. Learning Space. http://www.lotus.com/home.nsf/welcome/learnspace

Lynch, P. J., & Horton, S. (1999). Web style guide: Basic design principles for creating web sites. New Haven, CT: Yale University Press.

Macmillan, B., Timmons, V., and Liu, X. (1997). Teachers, Computers, and the Internet: The First Stage of a Community-Initiated Project for the Integration of Technology into the Curriculum. Alberta Journal of Educational Research, 43, 4, 222-234.

McCollum, K. (1997). EXPERIMENT SHOWS STUDENTS DO BETTER ONLINE. http://teleeducation.nb.ca/anygood/

McMullen, David W.; Goldbaum, Howard; Wolffe, Robert J.; Sattler, Joan L. (1998). Using Asynchronous Learning Technology To Make the Connections among Faculty, Students, and Teachers. Annual Meeting of theAmerican Association of Colleges for Teacher Education.

Microsoft. Digital Dashboard. http://www.microsoft.com/DigitalNervousSystem/km/DigitalDashboard.htm

National Center for Education Statistics. http://nces.ed.gov/

Nielsen, Y. (2000). Designing Web Usability: The Practice of Simplicity. New Riders Publishing.

Norman, D. (1988). The design of everyday things. New York :Doubleday.

Norman, D. (1999). The Invisible Computer. Cambridge, MA: The MIT Press.

Office of Technology Assessment, US Congress. (1995). Teacher and Technology: Making the Connection. http://www.wws.princeton.edu/~ota/ns20/year_f.html

4 OISE, University of Toronto. Web Knowledge Forum. http://kf.oise.utoronto.ca/webcsile/help/navigate.html

Palloff, M. R. and Pratt, K. (1999). Building Learning Communities in Cyberspace. San Francisco: Jossey-Bass Inc.

Pfaffenberger, B. (1997). The Elements of Hypertext style. Boston, MA: Academic Press.

Roblyer, M. D. and Edwards, J. (1999). Integrating Educational Technology into Teaching. New Jersey: Prentice-Hall.

Rose, A., B., A.R, & Katheleen, F. (1999). Multiple Channels of Electronic Communication for Building a Distributed Learning Community. Computer Support for Collaborative Learning.

Rubin, H., & Rubin, I. (1995). Qualitative interviewing: the art of hearing data. Thousand Oaks, CA: Sage.

Ruhleder, K. (1999). Network Community: Virtual Space for Physical Bodies. Computer Support for Collaborative Learning.

Russell, T. (1999). No significance difference phenomenon. http://teleeducation.nb.ca/anygood/

Schlager, M. S., Fusco, J., and Schank, P. (1999). Evolution of an On-line Education Community of Practice. In Building virtual communities: Learning and change in cyberspace. NY: Cambridge University Press.

Schulman, L. (1998). Course Anatomy: The Dissection and Analysis of Knowledge Through Teaching. In The Course Portfolio: How Faculty Can Examine Their Teaching to Advance Practice and Improve Student Learning., edited by Hutchings, P. American Association for Higher Education.

Sergiovanni, J. T. (1994). Building Community in Schools. San Francisco: Jossey-Bass Inc.

Siegel, D. (1996). Creating Killer Web Sites. Indianapolis: Hayden Books.

SITE: Society for Information Technology and Teacher Education. http://www.aace.org/site/index.html

Spool, J.,M., Scanlon, T., Schroeder, W., Snyder, C., and Deangelo, T. (1999). Web Site Usability: A Designer’s Guide. San Francisco, CA: Morgan Kaufman Publishers, Inc.

SREB. Technology Standards for Teachers. http://www.sreb.org/Main/LatestReports/Tech/TechStandard/Tech_standars.htm

SRI International, Center for Technology in Learning. Tapped in. http://www.tappedin.org/

4 TeamWave Software. Workspace. http://www.teamwave.com

Tufte, E. (1983). The Visual Display of Quantitative Information. Cheshire, Conn.: Graphics Press. Tufte, E. (1990). Envisioning Information. Cheshire, Conn.: Graphics Press. Tufte, E. (1997). Visual Explanations. Cheshire, Conn.: Graphics Press.

U.S. Department of Education, Office of Educational Technology. http://www.ed.gov/Technology/

Wallace, P. (1999). The Psychology of the Internet. New York: Cambridge University Press.

Wenger, E. (1998). Communities of Practice: Learning, Meaning, and Identity. Cambridge University Press.

White House. Report to the President on the Use of Technology. http://www.whitehouse.gov/WH/EOP/OSTP/NSTC/PCAST/k-12ed.html

Winograd, T., (ed.). (1996). Bringing Design to Software. New York: Addison-Wesley.

Wetzel, K. (1999). Getting in the Technology Game. Learning & Leading Technology, 27, 9, 32-35.

4