sign in sign up
<<
Home , Alexandria, Catania

MIAMI UNIVERSITY The Graduate School

Certificate for Approving the Dissertation

We hereby approve the Dissertation

of

Charles A.

Candidate for the Degree:

Doctor of

______Director Dr. Sally Lloyd

______Reader Dr. Richard Quantz

______Reader Dr. Steven Thompson

______Graduate School Representative Dr. William Boone

ABSTRACT

Preparing Global Leaders: A Program Evaluation of a Train the Trainer Model for Children’s International Villages

by Charles A. Catania

Children’s International Summer Villages (CISV) identifies itself as a charitable, independent, non-political, volunteer organization promoting peace education and cross-cultural friendship. The specific focus of this research project is a program evaluation of the Train the Trainer (TTT) process as utilized by Children’s International Summer Villages (CISV) for the purpose of leadership training. This project uses a participatory mixed-method program evaluation design. The intent of this research was to study the impact of the Train the Trainer (TTT) process by gaining a deeper understanding of the experiences and perspectives of those who have participated in the TTT process within Children’s International Summer Villages. In addition, the study investigated the scope of the impact of the TTT process, not only on the direct participants, but also upon the organization as a whole. Data were gathered through a variety of strategies. Archival data were retrieved from the International office and provided historical information regarding CISV in general, and leadership training specifically. Information from previous leadership training evaluations was reviewed. Interviews were held with members of the international office, the executive committee, international committee chairs and the international pool of trainers. An online survey was created and distributed to the 250 individuals who had participated in the TTT process during the past three years. Results indicated that participants perceived an enhancement of attitudes, skills and knowledge regarding the concepts of the TTT. These results were similar across three regions worldwide and throughout the three years of implementation. Data indicated that the TTT is meeting the aims of the four guiding principles and is strongly aligned with the newly developed common goals and indicators of the organization. In addition, participants acknowledged that the TTT is an effective avenue for promoting common training practices, introducing new organizational initiatives, and creating networks across the organization. Implications for program improvement and future evaluation are discussed.

PREPARING GLOBAL LEADERS: A PROGRAM EVALUATION OF A TRAIN THE TRAINER MODEL FOR CHILDREN’S INTERNATIONAL SUMMER VILLAGES

A DISSERTATION

Submitted to the Faculty of

Miami University in partial fulfillment

of the requirements

for the degree of

Doctor of Education

Department of Educational Leadership

by

Charles A. Catania

Miami University

Oxford, Ohio

2011

Dissertation Chair: Dr. Sally Lloyd

©

Charles A. Catania

2011

Table of Contents

Page LIST OF TABLES ……………………………………………………………………………...v

DEDICATION ………………………………………………………………………………….vi

ACKNOWLEDGEMENTS ……………………………………………………………………..vii

Chapter

1. INTRODUCTION ……………………………………………………………………1

2. CHILDREN’S INTERNATIONAL SUMMER VILLAGES: LIVING ...5

CISV: The Beginning Years Training Leaders – Leadership Training CISV – Strategic Directions The Train the Trainer Model for CISV My Position as the Researcher

3. REVIEW OF LITERATURE ……………………………………………………….25

Leadership The Educational Foundations of Learning within CISV Program Evaluation Mixed Method Evaluation Designs

4. METHOD……………………………………………………………………………38

Interviews with Administrative Participants Review of Archival Documents Interviews with the International Pool of Trainers The Online Survey Observations of the International Pool of Trainers Meeting

5. RESULTS AND DISCUSSION …………………………………………………….46

Online Survey Results Interviews Feedback from the International Pool of Trainers

iii

6. KEY FINDINGS, RECOMMENDATIONS AND CONCLUSIONS ……………...78

Summary of Key Findings Recommendations to CISV Limitations of the Evaluation Design Future Steps in the Continuing Evaluation of TTT Conclusions

Appendices ……………………………………………………………………………………...91

A. Interview Consent Form B. Online Survey Consent Form C. Interview Questions D. International Pool of Trainers Questions E. Online Survey F. Charts G. Tables

References ……………………………………………………………………………………...111

iv

LIST OF TABLES

Table Page

1. Mean Responses to Online Survey Questions, Post-Training Assessment of Attitudes, Skills and Knowledge ………………………………………...105

2. ANOVA’s by Region on Post-Training Assessment of Attitudes, Skills and Knowledge ……………………………………………………………………106

3. ANOVA’s by Year on Post-Training Assessment of Attitudes, Skills and Knowledge ……………………………………………………………………107

4. CISV Common Goals and Indicators ……………………………………………………108

5. Comparison of Before and After Train the Trainer Course Responses to Attitudes, Skills, Knowledge, Overall Effects ………………………………………...109

v

Dedication

This research project is dedicated to my parents for inspiring and encouraging the pursuit of lifelong learning (and for really wanting a doctor in the ); to my wife, Michelle, for her incredible love and support through this long process and for tolerating all that was needed in order to complete this project; and lastly, to Doris Allen for sharing her dream and helping thousands of people worldwide for sixty years of building global friendships.

vi

Acknowledgements

This journey has been a long and winding road. It began long before the start of coursework for this degree. It began from the seed for learning and service that was planted by my parents. The spark was ignited by CISV. The drive was fueled by all those I met along my road less travelled. I owe my deepest gratitude to many people whose support and encouragement helped make the completion of this dissertation possible. A multitude of thanks goes to Dr. Sally Lloyd who, as dissertation chair, provided guidance, wisdom and support. She has shared a wealth of knowledge and experience along with a great perspective throughout this whole process. A big thank you to my committee: Dr. Richard Quantz, Dr. Steven Thompson and Dr. William Boone. They brought a diversity of experience, ideas and perspective that was very much appreciated. Thank you to the members of the Department of Educational Leadership. Some were my teachers. Some were my guides. Some were my cheerleaders. All were needed. Thank you to the members of the Department of Educational Psychology. Thank you for your advice, flexibility, support and continued nudges along this journey. I also owe gratitude to many people in CISV. Considerable thanks to Cathy Knoop who was not only influential in my involvement with USA National Training, but also influential in my involvement with the International Pool of Trainers. Thank you for your confidence, trust and friendship. Thanks go to Kiran Hingorani who, as Educational Officer, was influential in my involvement in this project. He has been an invaluable resource throughout the development and implementation of the TTT and throughout this research. Thanks to Gabrielle Mandell who, as Secretary General, helped make so much of the data collection for this project possible. Thank you for helping me to be a part of Writeshop 2010 and then having access to the archives while in Newcastle. Thanks also to Gaby and Arnie for their time and hospitality. Thanks to Jennifer Watson for sharing her extensive knowledge of research in CISV and for collaboration between

vii our two research projects. Thank you also for your hospitality. Thanks to the Executive committees and expanded executive committees for seeing the need for this research and participating in its implementation. Thanks to the International Pool of Trainers: Bjorn Samming, Rogelio Gutierrez, Mark Flores, Leah Levac, Mitzi Mapa, Mark Porter Webb, Caecilia van Peski, Alvin Parreno, Lorena Ballega, Siri Torgard, Florentine Versteeg and Alex Neuman. They are a creative, inspirational and dedicated group with whom to work. Thanks to the TTT participants for all they brought to the TTT and for the incredible participation in data collection for this project. Thanks to all in the CISV family who have inspired me with their energy and passion throughout the past twenty-five years. Special thanks to my family. Thanks to my parents, aunts and uncles for establishing my foundation. Thanks to my children, Clare and . They keep me going. They created quiet space when I needed to focus and were great distractions when I needed to clear my head. Thanks to my wife, Michelle. She has been supportive and encouraging. She has taken care of things when I was travelling. Thanks for the signs of love wherever I have turned.

viii

Chapter 1 Introduction “Leadership is the key to success.” (2004 CISV Strategic Plan) Leadership is a personal experience. Wren (1995 p. x) stated that: “Leadership in its full compass is neither a position or title nor the actions of an identifiable leader. It is instead an interactive process in which leaders and followers engage in mutual goals. Viewed in this light, leadership occurs at all levels of society and engages all humans. A people approach to leadership must acknowledge all elements of the process, not just the actions of the leader.” Leadership in this regard is personal and interpersonal, reflective and interactive. Traditional historical descriptions of leadership tend to describe it as one-way directives. Wren’s definition emphasizes the interaction between leader and follower as the key toward achieving mutual goals. Leaders are people often in a position or situation where there are others who follow in some capacity. In his study of how people lead, Gardner (1995 p. 8) defined leaders as “persons who, by word and/or personal example, markedly influence the behaviors, thoughts and/or feelings of a significant number of their fellow human beings.” Leadership is a cultural experience. Whenever individuals gather together, each person brings the diversity of their experiences, beliefs and traditions to the interaction. In sharing their own , these participants form a unique cultural dynamic representing this small group of people. A report by the Association of American Colleges and Universities (AACU 1995, p. 9- 10) describes diversity as “the variety created in any society (and within any individual) by the presence of different points of view and ways of making meaning which generally flow from the influence of different cultural and religious heritages, from the differences in how we socialize women and men, and from the differences that emerge from class, age, and developed ability.” The overall purpose of this dissertation is to evaluate a leadership training process within a unique intercultural organization, Children’s International Summer Villages (CISV). Children’s International Summer Villages identifies itself as “a charitable, independent, non- political, volunteer organization promoting peace education and cross-cultural friendship” (www.cisv.org/about/who.html). Children’s International Summer Villages has identified leadership as one of its key directives throughout the international organization. The annual report states “CISV provides a range of unique, educational group activities promoting cross-

1 cultural understanding in children, youth and adults from around the world. By encouraging respect for cultural differences and the development of self-awareness, CISV strives for each participant to incorporate these values into their lives as they become global citizens and strive for a more peaceful world” (www.cisv.org/about/what.html). The organization’s purpose is “to educate and inspire action for a more just and peaceful world” (CISV Passport 2009, p. 8). CISV aims at stimulating the lifelong development of friendships, effective communication skills, and co-operative abilities. CISV activities are planned to promote personal, cultural, intercultural and international learning. This enables individuals to develop positive attitudes towards others, and the skills and knowledge to live, work and play with them, irrespective of cultural background. Through participation, individuals come to better understand themselves and their own , as well as others and their cultures. In this way, they can then understand their similarities and differences (CISV Annual Review, 2009). In his book, Diversity Works (1997), Smith emphasizes that using a comprehensive framework for diversity encompasses the complexities of the community and also the differences within the individuals who make up that community. “It includes such important and intersecting dimensions of human identity as race, ethnicity, national origin, religion, gender, sexual orientation, class, age and ability. These dimensions do not determine or predict any one person’s values, orientation, choices or responses. But they are by definition closely related to patterns of societal experience, socialization and affiliation. They influence ways of understanding and interpreting the world” (Smith 1997, p. 7). Leadership occurs at all levels of an organization. For this reason, leaders must be culturally responsive. Children’s International Summer Villages is working to prepare culturally proficient leaders through a Train the Trainer process. Train-The-Trainer (TTT) is a widely acknowledged educational model across a variety of disciplines. The TTT process is structured in such a way that a small group of key trainers will provide a process training to national representatives who will then return to their national association (NA) to provide training in their area of content. The implementation of this process provides a unique opportunity to study the effectiveness of the preparation of culturally diverse leaders. CISV provides opportunities for non-formal education activities, which are opportunities to learn from direct hands-on experience. Non-formal activities are interactive, co-operative,

2 action orientated, participant-centered and fun. In addition, CISV’s activities are sensitive to personal and cultural differences and offer the opportunity to understand different cultures (CISV website). By engaging in these activities participants have the opportunity to develop intercultural competence. Intercultural competence is the ability to establish relationships; the ability to communicate with minimal loss or distortion; and the ability to work together toward a common task. CISV also aims to develop the participant’s competence to deal across cultures by developing awareness, fostering positive attitudes, increasing skills and acquiring knowledge of other cultures. The specific focus of this research project is a program evaluation of the Train the Trainer (TTT) process as utilized by Children’s International Summer Villages (CISV) for the purpose of leadership training. This project uses a participatory mixed-method program evaluation design. The intent of this research is to study the impact of the Train the Trainer process by gaining a deeper understanding of the experiences and perspectives of those who have participated in the TTT process within Children’s International Summer Villages. In addition, the study investigates the scope of the impact of the TTT process, not only on the direct participants, but also upon the organization as a whole. The specific purpose of this study is two-fold: to gain an understanding of the degree to which the TTT has met its goals of implementation to date, and to study the impact of the TTT process. This study uses a mixed-method program evaluation format in order to gain a deeper understanding of the experiences of those individuals who have participated in the Train the Trainer model as utilized within CISV. The first research question is: What are the stakeholders’ perceptions regarding the overall process of implementing the TTT? This question examines the components of TTT implementation. The examination of this research question draws upon the work of Kirkpatrick (1994) who identifies ten factors that are considered for creating effective training programs including: determining needs, setting objectives, determining subject content, selecting participants, determining schedule, selecting facilities, selecting instructors, selecting materials, coordinating the program, and evaluating the program. The second research question is: In what ways has the implementation of TTT had an impact upon leader training within the organization? There are four levels that represent a sequence of ways to evaluate programs: reaction, learning, behavior and results (Kirkpatrick,

3 1994). Reaction refers to the training participants’ perceptions of the program. Some refer to this as customer satisfaction. Learning relates to “the extent to which participants change attitudes, improve knowledge, and/or increase skill as a result of attending the program” (Kirkpatrick, 1994. p. 22). Behavior refers to any change in behavior that might have occurred as a result of participating in the training. Results refer to any changes in outcomes like production, quality, and return on investment as a result of the training. The remainder of this dissertation is organized as follows. Chapter two provides detailed background information on Children’s International Summer Villages (CISV), as well as, the Train the Trainer (TTT) process utilized by CISV to improve the leader training within the organization. Chapter Three contains the review of the literature. This chapter first examines issues of leadership and leadership training especially in the form of a Train the Trainer process. A review of experiential learning includes the work of such theorists as John Dewey, Kurt Lewin and David Kolb. This chapter concludes with a discussion of the use of program evaluation for the analysis of training and development of programs especially within non-profit organizations. Chapter Four describes the program evaluation methods utilized in the present study. Chapter Five lays out the results and discussion. Finally, Chapter Six provides concluding remarks including a discussion of key findings and recommendations for future steps in the continuing evaluation of the TTT process.

4 Chapter 2 Children’s International Summer Villages – Living History “The power of love is greater than the love of power.” (Doris Twitchell Allen) This chapter provides a chronicle of the growth and development of CISV. An historical perspective demonstrates the importance of research in CISV from the beginnings to present day. This chapter gives a detailed account of the development of the first CISV program in 1951, the process established for sustainability and the initial proposed structure for the organization. It goes on to show the evolution of leader preparation and strategic planning that ultimately led to the creation of the train the trainer process. CISV – The Beginning Years Dr. Doris Twitchell Allen was a visionary leader. She had a dream and knew that she couldn't make it happen alone. In order for the dream to grow and live on the leadership must be shared and passed from person to person. Dr. Allen was the founder and inspiration for Children's International Summer Villages (CISV). During the summer of 1946, Dr. Allen was a professor within the Psychology Department at the University of Cincinnati. Dr. Allen was well respected as a specialist in child growth and development. In 1946, the painful stories of World War II were vivid realities. Rusty, Dr. Allen's young son approached her one day and asked if he would have to go to war when he became an adult. Dr. Allen’s response sparked the beginning of this sixty-year adventure. Not too long following that conversation, Dr. Allen was on a three-week assignment in New York as a consultant to the Child Education Foundation. As the CISV story goes, she was sitting on a park bench eating lunch and reading the New York Times. An article written by Alexander Meiklejohn caught her attention in a very dramatic way. Following the devastation of World War II, many people were focusing on initiatives to build and maintain peace. Dr. Meiklejohn proposed a program of study that would act like an international graduate school for the Ph.D. level students of philosophy, history, political science, economics, physics and the social sciences. The students would meet at the United Nations and talk to each other. This project would be aimed at forming a leadership group focused upon building world peace. Dr. Allen knew that the world, still recovering from war, was eagerly searching for panaceas. The Meiklejohn article struck a chord with Dr. Allen both personally and professionally. This approach was diametrically opposed to her view of the world and how it must proceed. Dr.

5 Allen firmly believed that "We must start with the children!" (Matthews, 1991 p. 22). The idea was born. The idea developed into a definite plan to focus upon peace education by bringing young children together from all over the world in an informal camp-like setting to build toward future peace. Upon her return to Cincinnati, Dr. Allen shared her idea with everyone she could. She developed a paper that she presented in Philadelphia at the annual American Psychological Association (APA) convention. In the audience were individuals from the U.S. State Department who were impressed with the idea. She then presented at a meeting of social scientists in Maine. In this audience were renowned social scientists Gordon Allport and Margaret Meade. Both initiated continued dialogue. Dr. Allen traveled to to survey existing programs such as: the International Voluntary Work Camps at Berne, the International of Children's Communities (FICE) in , and the Pestalozzi Children's Village in . While in Europe, Dr. Allen presented a request for funding at the General Conference of UNESCO in Paris. The research idea was well received by UNESCO. While the interest in research into international work with children was strong, the money offered was minimal (Matthews, 1991 p. 25). The passion for this youth project began to grow and spread. As the vision became clearer, more definitive planning began to occur. Leaders stepped forward to develop plans for finances, program development and participant outreach. On June 3, 1951 the first Children's International Summer Village took place at St. Edmunds Conference Center in Glendale, Ohio. The effort was divided into four main components: the children's camp, the adult institute, the research program and contact with the community (Allen, 1976).

The children's camp included six children (three boys and three girls) from nine countries (, Denmark, England, France, , Mexico, Norway, Sweden and the United States). Two adults from each country accompanied the children in travel and delivered them to the camp. The adults then participated in the adult institute. Ted Wuerfel was chosen to direct the camp. Mr. Wuerfel was, at that time, the Headmaster of the Lotspeich School in Cincinnati. He also had a very strong reputation and long history of coordinating and operating children's camps. He and his staff created opportunities for the children to interact through games, drama and craft activities. There were also opportunities to experience the cultures that were represented through the sharing of international foods, songs, dances and costumes (Matthews, 1991).

6 The adult institute took place at the home of Dr. Allen. The adult institute focused upon the development of CISV including a mission and vision for the future. The adults would meet during the day and then join the camp participants for dinner and the evening program. The evening program typically consisted of a national night by one of the participating countries to share their culture through songs, stories and dances. The participants had been sent three questions before their arrival in order to prepare for the experience. What do you see as the purpose of the adult institute? What procedures to achieve these goals do you suggest? What specific topics would you suggest for the agenda? (Matthews, 1991 p. 48). The participants in the adult institute spent the first meeting establishing an agenda for the month together. Matthews (1991) reports that the following eight points were agreed upon as the focus of the adult institute:

1. Studies of individuals. Comparisons. 2. The superego—an examination of ideals of different countries. Value systems. 3. Adults have failed children's fresher outlook important to cultivate and consult. Let the children know that the adults were hoping to learn from them since the adults have been incapable of solving the peace problem.

4. Make a report.

5. Study the common factor in domestic life - music, religion, games etc. 6. Education (common games etc.).

7. Future villages - finances, locations etc.

8. The Extension Program - spreading CISV as an idea and a product. The research program involved eleven social scientists observing and investigating various aspects of the children's camp before, during and after camp activities. The purpose of the proposed research was defined in two statements:

1. To be able at the end of the month's experience to make some basic recommendation for similar camps to be held in different countries in subsequent years.

2. To be able to formulate some principles regarding the "learning" in international understanding which could aid other organizations in planning exchanges of persons at the high school and college level as well as the child level. (Matthews, 1991 p. 92)

7 This research was directed toward answering six questions: 1. What forces and factors are identified as facilitating international understanding? 2. What obstacles to international understanding are discovered? 3. Are lags in international understanding in certain individuals correlated more with nationality, or with the degree of rigidity of the individual personalities? Are prejudices to be found to a greater degree from the war-torn and occupied countries (France, Denmark, Norway, Germany, England and Austria) than in children of the other countries (e.g. Mexico, Sweden and the U.S.)? 4. In the presence of a variety of languages, how is communication affected by the children? 5. To what degree and by what steps can the heterogeneous group of six children from say each of ten countries (sixty youngsters approximately) become unified into one group with common goals? (Matthews, 1991 p. 93) The organizers also saw this incredible international experience as a unique opportunity for educational contacts with the community (local, national and global). Participants engaged in a number of excursions throughout Cincinnati to explore the local culture through sightseeing, shopping and eating. The youth also spent the middle weekend with local to have a more personal experience with the local culture. The camp created an open day in which community members were invited to visit the camp, meet the participants, experience some of the daily activities and learn some of the international songs and dances. The CISV experience was shared nationally and internationally as a result of the reports that were written by the camp staff, the adult institute and the research participants. Each participant shared their CISV experience personally through stories, journals, pictures and diaries upon their return home. Ingvild Schartum Hansen, one of the camp participants from Oslo, Norway, kept a very detailed and complete diary of her experiences. It was published by a New York firm and is still sold internationally under the title Ingvild’s Diary (Matthews, 1991 p. 105). One of the stories that has lived on in CISV history took place at breakfast before one of the adult institute meetings. Doris Allen often told the story of one of the participants of the adult institute, Kathleen Milne, arriving at the breakfast table and sharing a poem. She had written the poem during the previous night after being inspired by the activities of the camp. While Kathleen read the poem, another leader, Bjarne Kirk, from Norway, began to tap out a

8 rhythm. After breakfast, he went to the piano and created a melody for the inspired poem. This inspiration has become known as the CISV Song. It has been sung daily at camps and other CISV activities for the past 60 years (Matthews, 1991 p. 45). Here in this village you may see Children living happily Different race and different land Here we come to understand One another's point of view Learning through the things we do How alike am I to you

Here we live and eat and sleep Talk and laugh and sometimes weep Here we share our hopes and fears Build a bridge across the years Sow a seed and plant a tree Beneath whose branches there may be All the nations gathered free

That our children so may grow In a world we did not know Sharing all they have to give Learning how to love and live In our hands the future lies Seize the moment 'ere it flies Stamp the present with an act Dare to make our dream a fact The first CISV experience was deemed a success. Data were analyzed, reports written, presentations delivered, structures formalized, committees formed and plans were made. The second camp was held during the summer of 1952 in France. The third experience was held in Sweden in 1953. It was decided that this international connection of people must be

9 incorporated in order to proceed further as a recognized organization. They further understood that local incorporation in Cincinnati, Ohio would not be sufficient. Therefore, it was decided that CISV would be incorporated as an international organization. Following a number of discussions related to travel, finances and international relations by the newly identified board of directors, the decision was made to move the organization out of the Allen home to the first international office in Stockholm, Sweden in 1956. The international office remained there until 1966 when the main headquarters was moved to Newcastle upon Tyne, England (Matthews, 1991). The foundational structures that were created during the adult institute and subsequent committee meetings are still in existence today. During the past 60 years, the organization has grown. It has grown in participating national associations and it has grown in the types of programs offered. The original camp experience is now referred to as a CISV village. The Annual Review 2009 (p. 19) reported that 56 villages were held during the year hosting 3,641 participants at the camps for eleven year olds. In addition, other programs have evolved. A CISV Summer Camp is similar to a village except that the 14-15 year old participants are more actively involved in the leadership of the daily activities. An Interchange encourages a deeper encounter between two cultures by pairing young people within families while complimenting the experience with group activities. A Seminar Camp offers a more intensive intercultural experience for 17-18 year olds to explore issues and form opinions about individual, international and multicultural matters. The International Peoples Project offers delegates over 19 years old an opportunity to participate in a hands-on service project in partnership with local and national organizations. Throughout 2009, 283 international programs were hosted involving a total of 7,884 participants. Since the first village in 1951, volunteers have organized 5,851 international programs for 222,298 participants (CISV Annual Review 2009, p. 19). CISV is a volunteer organization. Volunteers run the programs and manage chapter business. Each chapter and national association has an organizational structure that is similar to the international organization. There are presidents, vice presidents, trustees, program chairs and committee leaders at the chapter, national and international level. Chapter leaders typically meet monthly. National officers meet quarterly, and the national board of trustees meets annually. International officers meet two to three times per year in addition to the ongoing web-based meetings. The Annual International Meeting (AIM) takes place once a year. The location rotates

10 between three regions (Europe, Asia Pacific, the Americas) and also rotates between the nations within the regions (AIM Executive Review, 2009). Training Leaders – Leadership Training In the beginning, there were two programs: the village and the adult institute. As the organization grew, the number of programs grew and the number of participants grew. From the very beginning, the organization saw the need for an orientation process for program participants. This orientation provided basic information regarding expectations and logistics (Kangaslahti, 1986, p. 13). While it was generally believed that every chapter offered an orientation, Kangaslahti (1986, p.13) proposed that an attempt to coordinate a worldwide scheme and network to respond to the different local orientation and training approaches was justifiable. The plan was made for national representatives to attend a special meeting at the International Board Meeting (IBM) in 1986. This meeting was called SOLAR. The original label for SOLAR was Stimulative Orientation Towards Life-Long Learning on Altruistic Ways of Responding. It was later referred to as the Seminar on Leadership and Training. The purpose of SOLAR was an effort to unify the approach to leadership and training at all levels of CISV (SOLAR Report, 1986). Forty-six participants took part in the first SOLAR gathering in Jacksonville in 1986. They identified the four principle goals to include: providing an overview and understanding of the connections among all CISV programs; providing a program to develop skills for the successful training of all those in a CISV leadership role; making possible a network for the cooperative exchange of ideas, opinions, and strategies with regard to the training of trainers; and to increase the participants’ awareness of cultural and cross-cultural issues (SOLAR Report, 1986). Leadership training received more and more attention following the SOLAR gathering in 1986. Workshops at the regional and national levels were organized, expanding and enlarging the group of people with experience in the field of leadership training. During the 1988 International Board Meeting (IBM) in , the idea of establishing a Leadership Training Circle was formed. This idea would bring together a circle of people from all CISV countries responsible for different CISV programs who are interested in further developing leadership training. The objectives of the circle are to exchange experiences or ideas in the field of leadership training and help each other in organizing workshops (Leadership Training Circle report 1989). Leadership Training Circles were organized over a two-year period that then led to the creation of the International Leadership Training Committee (ILTC) in 1990. ILTC would

11 then be an officially recognized committee of the international expanded executive committee. The ILTC spent the next few years gathering information from national associations and making recommendations for a formalized national leadership training process. Early on, it was apparent to the ILTC, through observation and feedback from program leaders, that the delegation leaders needed tools to prepare them for the experience. Initially, preparing the leader was an informal process guided by the local chapter. An analysis of evaluations following programs demonstrated that different chapters in different countries provided varying levels of support in leader preparation. This variance was considerable in terms of the amount of time involved in leader preparation, as well as the content and the materials used for leader preparation. Once the International Leadership Training Committee (ILTC) was developed, the ILTC members created a list of necessary areas of content that should be a part of every leader training program. This committee developed a guide for National Leadership Training that would occur within each National Association (NA). The first guide was distributed in 1994. This guide acted as a curriculum for national trainers to train program leaders from the various chapters within the NA. National Leadership Training was content specific relating to the international program that one would be representing (i.e. village, interchange etc.) (CISV ILTC Long Term Vision, 2008). CISV – Strategic Directions CISV has impacted the lives of thousands of people throughout the past 60 years. Many of those people remain actively involved at the local, national and international level following an experience with an international program. Many of those people have taken on leadership roles and left their mark within the world of CISV. Throughout the sixty-year history of CISV, there has been dynamic growth. CISV has had a long history of making a difference in many people's lives through peace education activities. While the foundations upon which CISV was created have not changed, the world definitely has. To keep up with the changes in the world, a long-range plan was undertaken in order to fully utilize the potential that lies within the organization. The trustees of AIM 2002 decided that strategic planning was the direction to take in order to begin the journey toward the next sixty years of CISV. In order to make this strategic plan meaningful and useful, it was developed based upon the input of CISV volunteers at all levels of the organization. Individuals from all chapters

12 around the world were asked to reflect on their experiences in CISV and to use these experiences to identify the core values of CISV and the dreams or wishes for where it could go in the future. The results of this work are reported in two documents. The values and wishes were summarized and can be found on the CISV resources website. A storybook was also compiled that includes the peak experiences from every chapter. This storybook can also be found on the resources website (CISV Strategic Planning Report, 2004). Values and Wishes were identified from chapters all around the world. Although the specific wording of the values and wishes changed, there were a number of themes that clearly emerged. The themes are divided into two categories: the first category identifies themes that were not only identified as values, but were identified as core values to the organization, and the second identifies the themes that emerged as values, but not necessarily core values (CISV Strategic Planning Report, 2004). Themes identified as core values included the following terms: friendship, communication, peace, personal development, teamwork, cultural understanding and the CISV organization. Other themes that were identified include: respect/tolerance, education, play, openness, people, taking action, handling conflict, agelessness, longevity, training/evaluation, identity and recognition (CISV Strategic Planning Report, 2004). Based on the above values and wishes, the team at AIM spent a day and a half processing the information and developing a set of statements that described CISV when it is working at its highest potential. These statements are the vision for CISV. The following statements were developed to effectively express the overall essence of CISV (CISV Strategic Planning Report, 2004):

• CISV creates an atmosphere that stimulates teamwork and thereby provides an opportunity for its members to experience support, cooperation, trust and personal growth.

• CISV works towards a peaceful world through educating people to understand and respect each other.

• CISV is a well-known organization that has more than 100 NAs with members of all ages and all socioeconomic classes involving local committees in their educational

activities.

• CISV provides the arena and the tools to meet in an open environment, which enables mutual cultural appreciation and respect.

13 • By creating a unique content-rich environment, CISV fosters dynamic in-depth learning experiences while having fun.

• CISV is an independent, non-partisan, voluntary NGO that provides intercultural education, which develops global citizens for a better tomorrow.

• CISV fosters a spirit and love of friendship by connecting people and creating local and international bonds that motivate its members to actively work with the organization to

achieve its goals.

• CISV provides a cozy environment that encourages inner growth and the development of leadership skills and positive attitudes in order to help individuals become global

citizens.

• CISV is an NGO that is a preferred partner for conflict management through intercultural understanding striving for world peace.

• CISV empowers you to move on from the safety of exciting CISV activities and to transform your beliefs into actions in the real world.

• CISV provides an environment that concentrates on the common aspects of a group rather than the differences. By stimulating feelings of hope, compassion and happiness it

provides a desire to last forever.

• CISV's organizational structure fosters inspiration, creativity, trust and commitment among volunteers. (p. 5) The next stage of the process was to begin to think about implementation of the vision. The first step in this process was to identify a concrete goal that the whole organization could work towards. This goal is: By 2009, to become a larger network of successful NAs in every continent. This goal formed the bridge between the vision and the actions that were needed to implement the vision. Based on achieving this goal, all members at AIM identified areas that they felt needed to be addressed. In total, 48 areas were identified. All of the participants at AIM were asked to work together on ideas regarding how to accomplish the goal. Forty-eight discussions were held to clarify the issue and specify the direction of need. Each small group presented a report of their findings. These reports were then categorized as to which vision statement they supported. The reports were prioritized and the top five priorities were selected for implementation.

14 1. Leadership is the key to success. 2. Increasing quality through cooperation with other organizations. 3. How to find a fair way to financially support the organization in every continent. 4. How to strengthen existing chapters. 5. How to avoid excluding people, groups, NAs, and Nations from CISV. (p.7)

Finally, the people who nominated each topic were asked whether they would work with the International Executive Committee and related committees to help develop a set of concrete deliverables and work plan. All participants were then asked to sign up for any topic they were interested in supporting. Plans were made and goals were set at the international level. Strategies were discussed to extend participation in working toward these priorities at the national and local chapter levels (CISV Strategic Planning Report, 2004). Discussion groups formed to identify the needs within each strategic area. The discussion for strategic priority #1 identified two main issues. First, leadership training was considerably different across National Associations, even though there was a common curriculum. Second, leaders within CISV were viewed not only as those attending international programs, but all those in leadership positions within CISV. As a result, it was determined that all leaders needed access to knowledge and skill development in the area of leadership. The process of training all program leaders, chapter officers, chapter committee chairs, national officers, national committee chairs, international officers and international committee chairs for both the adult and youth structures seemed a colossal task. Following AIM 2004, participants returned to their national associations (NA) and held brainstorming sessions to identify how these priorities currently existed and what each NA could do to work toward these goals. This process then also occurred at the local chapter level. By the winter of 2004, there were specific goals established for each priority at the local, national and international level (CISV Strategic Planning Report, 2004). Priority one, "leadership is the key to success," seemed to fit naturally within the realm of the International Leadership Training Committee (ILTC). This team organized a discussion in order to define and clarify international perspectives on this task. One vital factor resulting from this conversation and the information gathered from chapters, nations and international, related to the meaning of the term leader. The term leader is often used in CISV to refer to the adult who accompanies the youth to international programs (i.e. e village, summer camp, interchange).

15 The conversations held regarding priority one very clearly designate the term leader to be understood in much broader terms. A leader in CISV is anyone who takes on a leadership function. In this regard, the term leader includes those in leadership roles in CISV at the local, national and international level. It also includes those who lead activities at the local, national and international level. It includes those who demonstrate leadership within programs, meetings and conversations. Leadership is everywhere. The ILTC also investigated the current functions of international leadership training and the capacity of leadership training at the national as well as local level. The investigation discovered that there were some National Associations (NAs) that were very strong in the leadership training that they provided to the program leaders. There were some NAs that were not very strong. There were no examples of NAs providing training for anyone other than the program leaders. The ILTC in cooperation with the International Office (IO) made the decision to create a new charge for ILTC. First, it was important to expand training to include all those identified as leaders (not just program leaders). Second, it was important to provide resources and support so that all NAs could facilitate a similarly high level of leadership training for their members. A proposal was presented at the Annual International Meeting 2005 and was voted upon by the board of trustees. CISV had just created a direction for a new model of training for representatives of each NA. A Train the Trainer (TTT) format was identified. It was recommended that a search begin for an international pool of trainers (IPT) (G. Mandell, personal communication, January13, 2010). The Train the Trainer Model for CISV In November 2006 plans were made for the newly selected international pool of trainers (IPT) to meet and design the new training. A meeting designated as "The Writeshop" was organized in order to create, design, develop and plan the implementation of the new TTT. The committee knew the importance of input, feedback and buy-in from other areas of the organization. Therefore, others invited included individuals from the International Office (I0), the International Executive Committee (IEC) and selected leaders from the Organizational Development Committee (ODC) and the Education and Research Committee (EDR). In addition, a guest from The Society for Intercultural Education, Training and Research (SIETAR) would visit for some of the sessions. These individuals, along with the six newly

16 selected international trainers, set out to create a new dynamic model for training and supporting the volunteers of CISV (Writeshop Report 2008). In November 2007, the International Office (IO) hosted the first ever TTT Writeshop in order to create the TTT process and design a plan for implementation. The Writeshop brought together individuals from different cultures, experiences and philosophies. The first few hours of the meeting were spent laying a foundation for the workshop by creating plans for communicating effectively across languages, cultures and perspectives. The experience modeled several of the identified areas of need in terms of intercultural communication, interpersonal interaction and conflict management (Writeshop Report, 2008). They identified that the purpose of the TTT would be to directly respond to priorities one and four of the CISV strategic plan: leadership is the key to success and strengthening existing chapters. Unique to this new TTT Workshop was the notion that peace education would be the main thread of all discussions and activities. Since all of the facilitators have been CISV leaders in various capacities, they would be able to relay a deeper understanding of CISV's educational philosophy to all participants. The workshop would give the participants a new perspective on training using an experiential learning approach in training methodologies. The structure would have multiple levels in design since this workshop was not only training the trainees in these concepts but also training them to train. This workshop would focus upon enhancing training by developing objectives that center upon attitudes, skills and knowledge. Practicum sessions were integrated. Participants would be able to apply the learning gained from the different sessions, particularly in organizing sessions by using a session guide or an activity template (Writeshop Report, 2008). There are four (4) guiding principles for every Train the Trainer (TTT) workshop. The TTT workshop was designed to: • Enhance the training competencies of participants • Strengthen the ability of participants to design and deliver effective training programs • Empower participants to develop a training strategy • Encourage participants to promote the educational principles of CISV through their training programs. (Writeshop Report 2008) CISV-ILTC rolled out the new design for the Train the Trainers (TTT) workshop in three separate locations in March, 2008 -- Asia Pacific (Thailand), South America (Brazil), and Europe

17 (Portugal). In addition, a TTT was requested in Canada during the fall of 2008. It was run at the same site, simultaneously, with the Canadian National Board of Trustees meeting. More than eighty participants attended the new TTT workshop from 24 countries including: Brazil, Colombia, Canada, Costa Rica, Czech Republic, Denmark, , France, Germany, Indonesia , , Japan, Lebanon, Mexico, , Norway, Peru, Philippines, Portugal, , Thailand, United States and Vietnam (ILTC Report to AIM 2008). The result of the first rollout of the new TTT model generated highly positive responses from the workshop participants. The information was passed on to other NAs who would like to attend a future TTT workshop either at the regional or at the NA level. Members of the International Pool of Trainers (IPT) attended the AIM 2008 in in order to evaluate the TTT experience and come up with recommendations based on the feedback by the previous participants. This feedback included the following strategies: • Creation of an enhanced TTT design for the future implementation • Selection of additional international trainers to respond to the increasing demand • Creation of a multi-year schedule of future TTTs, either at the regional or NA level • Plan for an evaluation of the TTT to measure its effectiveness and impact to the organization. (ILTC report addendum, 2008) During AIM 2008, the International Pool of Trainers (IPT) requested a follow-up team meeting in order to: regroup; make modifications and adjustments based upon the experiences of the 2008 trainings; and structure plans for the IPT selection process. Due to budgetary restrictions, funds for a face-to-face meeting were not available. The IPT then made plans to regroup through online meetings and email. A call for applications to join the IPT was posted. A review process was implemented and plans were made for 2009 training. TTT 2009 courses were held in France, Germany, Colombia and Thailand during the spring of 2009. Following AIM 2008, members of the organization began to analyze data regarding the strategic plan. The strategic plan developed in 2004 was designed to be implemented over a five-year period. Reports were analyzed from national associations, program committees, the international executive committee and the international office. These data provided valuable information and feedback regarding the progress toward the goals of the 2004 strategic plan. AIM 2009 was held in Guatemala during the summer of 2009. The participants of AIM 2009 took to task to create a new strategic direction. The International Executive Committee

18 (IEC) led the development of the new strategic plan. They ensured broad consultation and consideration of internal and external factors including: • A review of what has been achieved in the current Strategic Plan –what needs more work • A survey of National Associations on their progress under the existing Strategic Plan and what they considered to be their major priorities and challenges in the coming years • Input at Regional Meetings and Training Forums, from International Committees and the International Office • Consideration of National Report Forms, Program Evaluations, Global Hosting Plan, External influences, such as other organizations, quality assurance issues and opportunities offered by technology. (Strategic Plan Overview, 2009) The Strategic Plan document clarifies that through this process there emerged patterns, best practices and concerns. Globally, the organization identified the need to focus on the health of chapters and build capacity in order to ensure that more high quality programs are hosted by increasing: understanding and access to quality CISV educational materials; leadership training on all aspects of leadership; and the ability to bring in more members, participants, supporters. Though the details of these needs were viewed differently in different parts of the organization, the themes and focus were generally quite consistent, from Chapter, to NA to Regions, to International. This analysis led very naturally and directly to the identification of three priority areas that clearly reflected the needs of the organization. The three priority areas are: Education, Training and Evaluation; Chapters; and Raising Profile. The period from AIM 2009 through the end of the calendar year would see final work on the existing priorities along with preliminary steps to launch of the new plan in January of 2010. It was decided that it would be much more manageable to monitor the new strategic plan over a three-year timeframe rather than the five-year process of the previous plan (IEC Post AIM 2009 Minutes). The guiding vision for this three-year strategic plan is that CISV is one organization with one plan working together to do the educational programs better and at higher frequency (Strategic Plan Overview, 2009). There are three priority areas. Each area has one overall direction with specific goals. Each goal has strategies identified and each goal has roles for each level of the organization (Strategic Plan Overview, 2009). Three goals were identified within priority area one: Education, Training and Evaluation.

19 1. To have high quality educational materials accessible to everyone at all levels of the organization 2. To have a sustainable and flexible train the trainer system 3. To have a common approach for leadership training (program and organizational) at all levels of the organization. (p. 3) While CISV had been developing high quality materials for the past few years, rollout and usage had been inconsistent. In response to goal one, an internal action research project was proposed in order to: evaluate the implementation and impact of the educational materials at all levels of the organization; and to identify effective strategies for distribution and use of materials for all levels of the organization (Strategic Plan Overview, 2009). The Train the Trainer process was just completing its second year of implementation. The initial feedback during the first two years of TTT implementation was very positive. Goal two specifically related to continued implementation of the TTT courses. Members of the IPT requested funds for the training team to meet face-to-face in order to efficiently finalize the plans and projects that were created during the Writeshop 2007 and subsequent online gatherings. Due to budgetary restrictions, funds for a face-to-face meeting were not available. One of the strategies identified, however, was an internal research project to evaluate the implementation and impact of the TTT courses and networks at all levels of the organization (Strategic Plan Overview, 2009). As a member of the International Pool of Trainers, I was approached to facilitate this internal program evaluation. Coincidentally, during this time, I was working with the Education Officer to identify a topic for my dissertation that would be connected with CISV. After discussions with advisors from Miami University, it was agreed that this program evaluation would be an appropriate topic for my dissertation. CISV has evolved as an organization during its sixty-year history. New programs have been developed and existing programs have adapted as needed. The vision of ‘one organization with one plan’ prompted those at the international program level to look at how each of the programs had evolved independently and in their own unique direction. As a result, goal three was developed and a meeting was proposed in order for international program chairs to come together to identify common ground (Strategic Plan Overview, 2009). The proposal for the international program chairs meeting was accepted and Writeshop 2010 was funded.

20 In January 2010, members of the international executive committee and the international program chairs met with members of the international office at the international headquarters in Newcastle, England. Members of the IPT would be proactive participants of this workshop since the outcome would be strongly connected to the process utilized within the Train the Trainer courses. In addition, all of those in attendance would be identified as stakeholders in the TTT research evaluation process. My attendance at this meeting would be the kickoff for data collection for the TTT program evaluation. Goals and objectives were identified for this meeting and each attendee was responsible for pre-work prior to attending that involved an analysis of CISV program goals, indicators and training-related issues specific to their program. The goals for this Writeshop were: Goal 1: To identify the training content for all leadership roles (programme and organisational). Goal 2: To have training content specific for each leadership role. Goal 3: To agree on a training structure for all leadership training in CISV. Goal 4: Design an evaluation mechanism for all leadership training. (Writeshop 2010 minutes) Participants commented that upon arrival, it was obvious that with the growth of programs within CISV, each had evolved in its own unique way (K. Hingorani, personal communication, January 12, 2010). Each had a program specific identity and program specific training needs. In addition, those involved in the leadership of each program were passionate and protective toward their program. The purpose of finding common ground would be more challenging than previously imagined. After five days of intense discussion, brainstorming and problem solving a creative team of passionate individuals created a common plan while maintaining the unique nature of each CISV program. The common goals for all training within CISV were: Goal 1: Prepare trainees to use CISV`s approach to active global citizenship Goal 2: Prepare trainees to effectively undertake practical responsibilities within CISV Goal 3: Prepare trainees to contribute to the development and growth of CISV Goal 4: Prepare trainees to interact effectively and appropriately in diverse environments.

The identified indicators for each goal are listed in Table 4 (appendix G).

21 The trainings that were held during the spring of 2010 utilized the newly aligned goals while also teaching the participants to implement these goals within their future trainings. Following Writeshop 2010, an online survey was created for the purpose gathering quantitative and qualitative data for the research project. The survey was electronically distributed to all TTT participants to date. During the summer of 2010, I was invited to present the preliminary data at the Annual International Meeting (AIM) in Berlin, Germany. In addition to the Board of Trustees meeting, the executive committee and expanded executive committee were also in attendance. During this meeting, a proposal was made to fund a face-to-face gathering of the international pool of trainers (IPT). The trainers had not been together in a face-to-face meeting since initially organizing the TTT in 2007. In addition, new members of the IPT have been added to the pool throughout the past two years. Funding was approved and the IPT meeting was scheduled for January 2011 in Copenhagen. This meeting had several target goals: continue to respond to feedback for improvement, clarify and align the original mandate within the parameters of the newly identified goals and indicators, create an updated trainers guide and participants workbook and set plans for long range objectives. While Writeshop 2010 was seen as the kickoff for the data collection process, the IPT meeting 2011 was seen as the culmination of the data collection process for this program evaluation. My Position as the Researcher Throughout the time of my involvement in the doctoral program, I had been talking with people at the university and with CISV in regard to conducting a research project that would somehow bring my world of education together with my world of CISV. Many ideas were discussed. Several projects were informally proposed each with a unique combination of interactions within these two facets of my world. In August 2009, during the Annual International Meeting (AIM) of Children’s International Summer Villages, two research projects were discussed by the board of trustees, as a result of the review of the strategic plan. Knowing of my interests, the education officer called asking if I would be interested in performing the program evaluation of the Train the Trainer process. The idea was discussed with my dissertation chair. It was agreed that this would be a very appropriate project. In addition, facilitating a real-life project that fulfilled the needs of an international organization was an added bonus. I was initially introduced to Children’s International Summer Villages (CISV) in 1986.

22 A teacher friend told me a little about the organization and provided the leader application form. I applied, was interviewed and found myself travelling to Stockholm, Sweden with four eleven year-old children from Cincinnati for a month long international camp during the summer of 1987. As part of the preparation process prior to departure, I participated in national leadership training. The camp was an incredible experience. Delegations from twelve countries gathered in one place living, playing and learning together. In all, there were five staff members, five junior counselors (age 16-17), twelve delegation leaders and forty-eight eleven year-old children. The month was enlightening, enriching, emotional and very fun. We now had friends from places we had only heard of previously. On the plane ride home, one of my kids had the realization that watching the news would never be the same. These places would no longer be just a location on the map. They were now homes to our friends. Upon my return, I became active in the local chapter. Throughout the years, I have served the chapter in various roles including: youth and adult selection, leader preparation, and chapter president. A couple years after joining CISV, I received a phone call from my national trainers. The two of them were moving on to different leadership roles on the national board and asked if I would chair the National Leadership Training committee. I served in that capacity for two terms. In addition, I have served the national organization in other roles both including: Chapter Development Committee chair and Presidential Appointee. In 2007, I was selected as one of six trainers chosen to design and implement the newly created International Train the Trainer process. In 2009, I was asked to facilitate an internal program evaluation of the TTT process. During this time period, I was also developing my professional philosophy as an educational psychologist. In my experiences in rural, urban and suburban schools from preschool to twelfth grade, I have come to understand the role of culture, experience and personality in the process of learning. Theorists such as Piaget, Dewey, Lewin, Kolb, Montessori, Jung, Rogers and Gardner have influenced my professional practice. The creation and implementation of the TTT and my involvement in this doctoral program were occurring simultaneously. My decision to complete a doctorate in Educational Leadership was made in order to expand my knowledge base in the field of education. I believe my involvement in this doctoral program has helped to inform my involvement with the TTT. I

23 also believe that my involvement with the TTT helped to frame my direction within this doctoral program. My experiences in education and as a volunteer with Children’s International Summer Villages (CISV) have provided opportunities to develop an understanding of the need for experience in human learning and the significance of human interaction in creating meaning. Leadership style has as much of an impact upon the process as does the unique differences of each individual. It is the collaborative effort between leader and follower that makes all the difference in the progress toward a mutual goal. Summary This chapter provided a chronicle of the development of CISV. This historical perspective demonstrates the importance of research within the organization as well as the decisions made as a result of an organizational self-study. Specifically, this chapter illustrated the evolution of leadership training and the development of the strategic plan that led to the creation of the TTT process. The next chapter will serve to further provide a context for this program evaluation study, by reviewing relevant scholarly literature on leadership, CISV’s educational foundations, and program evaluation design

24 Chapter 3!! Review of Literature This research project is an evaluation of the Train the Trainer process that has been implemented by CISV. This training process is aimed at providing tools to the leadership of CISV across all organizational areas and programs. This review of literature will, therefore, review concepts of leadership and the educational foundations for training within CISV. In addition, there will be a discussion of the issues related to program evaluation, especially the evaluation of a training program. A review of literature helps situate the study within the broader realm of scholarship on leadership and how CISV as an organization works to be an example of transformative leadership. It provides the reader with an understanding of the connections of the CISV philosophy to theory, in particular the work of Kurt Lewin. This review provides clarification for the reader as to how I have conceptualized core elements of the study including: transformational leadership, constructivist perspectives, intercultural competency and training for adult learning. In addition, it provides a discussion of models of program evaluation, especially the evaluation of training programs, in order to situate my choices of features for this program evaluation design. Leadership CISV has had a rich history of volunteerism and service over the past 60 years. CISV relies upon its volunteers to maintain programs and create new directions for positive growth. Throughout those years, a multitude of volunteers have stepped up to take leadership positions within the organization locally, nationally and internationally. The leadership of CISV is vital to the growth and continuation of this international peace education organization. The leadership of CISV has also maintained the vision of Doris Twitchell Allen that CISV is an educational organization that studies itself through research and evaluation in order to continually improve (Allen, 1986, p.5). Heifetz (2002, p. 51) discussed the importance of "getting on the balcony" in order to view the issues from a different perspective. The “balcony” metaphor has been an important concept for me personally and within CISV organizationally. There is an emphasis in CISV toward engagement and participation in order to meet the needs of all involved while also ensuring that all have a voice in current happenings and future changes. The “balcony” provides a means of getting a snapshot of the present in order to make necessary adjustments.

25 Harvey and Cambron-McCabe (2005) illustrate the ideas of Heifetz by stating that "leaders are expected to provide: direction, so that the group knows where it is headed; protection, so that the group can protect itself from attacks; orientation, because people do not like to be confused about what their role is; conflict resolution, so that equilibrium can be maintained; and norms because people have to understand what is involved with being a member of a particular community” (p. 31). In addition, the balcony metaphor is pertinent because involvement in an international CISV program impacts the way one views and interacts with the world. Bolman and Deal (1991, p. 403) argue that leadership has been crucial to every culture and historical period. Despite a large number of writings and research studies, the concept of leadership remains elusive and social science research provides very few reliable generalizations. An historical look at leadership reveals a wide variety of theories and perspectives. Bass (1990, p 49) illustrates that in Egyptian hieroglyphics symbols for the terms leadership, leader and follower were discovered that were written 5000 years ago. Chinese are filled with examples of advice regarding the responsibility of leaders. Confucius urged leaders to be a moral example and use rewards and punishment to teach right and wrong. Lao Tsu emphasized the importance of the relationship between leader and follower: • A leader is best when people barely know he exists, when his work is done, his aim fulfilled, they will say: we did it ourselves. • To lead the people, walk behind them. • Leadership is an opportunity to serve; it is not a trumpet call to self-importance. • Enlightened leadership is service not selfishness. The leader grows more and lasts longer by placing the well being of all above the well being of self alone. Paradox. By being selfless, the leader enhances self. (Tao Te Ching) Greek philosophers wrote of the need for character in those with leadership positions. , in the Republic, investigated the traits needed for the ideal leader of the ideal state. Bass (1990, p. 51) explains that for Plato, “the leader was the most important element of good government, educated to rule with order and reason.” , in Politics, was bothered by the lack of virtue of the people who wanted to be leaders. He addressed the need to educate youth for virtuous leadership (Bass, 1990, p.51).

26 There are many different definitions of leadership, many of them ambiguous. Stogdill (1974, p.7) demonstrates that the meaning of leadership depends upon the perspective and the situation of those involved in the institution. In his review of leadership, he describes a wide variety of definitions within a number of different categories including: leadership as a focus of group process, leadership as personality, leadership as the art of inducing compliance, leadership as the exercise of influence, leadership as act or behavior, leadership as a form of persuasion, leadership as an instrument of goal achievement, leadership as an effect of interaction, leadership as a differentiated role, and leadership as the initiation of structure. In reviewing these definitions of leadership, Bass (1990, p. 43) explains that even with this extensive variety of perspectives in viewing leadership, there are some commonalities. Bass offers a comprehensive way of defining leadership as: “the process of influencing an organized group toward accomplishing its goals. Leadership is a social influence process shared among all members of a group. Leadership is not restricted to the influence exerted by someone in a particular position or role; followers are part of the leadership process, too.” Bolman and Deal (1991, p.405) discuss leadership in terms of the relationship to power and authority. Leadership is distinct from authority. While many leaders do possess authority, one can have authority without being a leader. They assert that a leader cannot lead without legitimacy. Leadership involves a relationship between leaders and followers in the organization. However, any obedience shown must be voluntary not forced. Leadership is not static. It is interactional and goals are achieved through the relationship between the leader and followers. Two approaches that have attracted considerable attention are transactional leadership and transformational leadership. Transactional leadership identifies the relationship between leader and follower that emphasizes rewards and incentives to complete work (Yukl, 1994. p. 464). The transactional model does not align with the philosophy of CISV, as CISV has emphasized shared leadership throughout all levels of the organization in order for the organization to continually grow and improve. Transformational leadership has been defined as a more dynamic form of leadership that transforms and motivates followers to work at a higher level of activity that goes beyond self- interest (Yukl, 1994, p. 464). Burns (1978) explains that transformational leadership “is a relationship of mutual stimulation and elevation that converts followers into leaders and leaders

27 into moral agents.” He goes on to emphasize that transforming leadership “occurs when one or more persons engage with others in such a way that leaders and followers raise one another to higher levels of motivation and morality” (p. 100). Tichy and Devanna (1990, p. 281) describe the development of transformational leadership in large organizations. They claim that there are similar characteristics that arise within organizations that model transformational leadership. They state that transformational leaders 1) behave as change agents; 2) display courage; 3) believe in people; 4) live by a set of core values; 5) become lifelong learners; 6) have the ability to deal with complexity, uncertainty and ambiguity and 7) possess vision. Leadership development in CISV encourages transformational leadership as identified by Tichy and Devanna. Senge (1990, p. 357) describes leaders in a learning organization as designers, stewards and teachers. According to Senge, Cambron-McCabe, Lucas, Smith, Dutton and Kleiner, (2000, p. 5), “a learning organization involves everyone in the system expressing their aspirations, building their awareness and developing their capabilities together.” Leaders can attempt to influence change, but without important elements in creating and understanding learning organizations, the changes might never be implemented or continued. Research has demonstrated that leadership has been a vital facet of organizations throughout time. Leadership is key in moving organizations to the next level. Common characteristics of effective leaders include the need for lifelong learning. Senge has argued that “it is no longer sufficient to have one person learning for the organization … The organizations that will truly excel in the future will be the organizations that discover how to tap people’s commitment and capacity to learn at all levels in an organization” (Senge 1990, p.4). Lambert (1995) aligns Senge’s perspectives of learning organizations with a constructivist theory of learning. Lambert states, “the development of collective meanings is an essential characteristic of a learning organization. Adults learn through the processes of meaning making and construction of knowledge, participation and reflection” (p. 29). Lambert (1995 p. 29) goes on to state, “leadership is based upon constructivist learning and uses the term constructivist leadership. Constructivist leadership encourages a reciprocal process for participants to construct meanings that lead toward a common purpose.” Leadership within CISV demonstrates meaningful examples of the Five Learning Disciplines developed by Peter Senge (2000). Personal Mastery: Leaders are encouraged to

28 articulate their personal vision in order to gather others together to move the organization to the next level. Shared Vision: Leaders are encouraged to develop the skill to work with others to create a common purpose so as to build a guide map toward the future. Mental Models: The leaders know of the importance of dialogue and openness in creating the structural and perceptual framework. Team Learning: The leaders are trained to emphasize the importance of gathering many others to share ideas and perspectives in order to collaborate on decision- making. System Thinking: leaders are well aware of the interdependency and interconnectedness of all of the parts that make up the "whole" of the organization. The Educational Foundations of Learning Within CISV From its inception, Children’s International Summer Villages utilized the writings of Kurt Lewin as the core of its educational foundations; thus it has always been a program steeped in theory. Lewin’s work is based in an interpretivist and experiential paradigm. According to Denzin and Lincoln (2005), some studies in research theory for the social sciences distinguish two major philosophical traditions: positivist and interpretivist. These two traditions stand apart in their assumptions about the nature of reality. Positivists come from the perspective that knowledge is based solely upon what can be observed and experienced (Williamson, 2002. p. 285). Research for the positivist is typically grounded in quantitative data since measurement and objectivity are key components. The interpretivists, on the other hand are concerned with the meanings and experiences of human beings (Williamson, 2002. p. 286). For interpretivists, people are continuously involved in interpreting the changing world. The central component is in the belief that the social world is constructed by people in their search for meaning. The constructivist paradigm fits within this interpretivist philosophy. For the interpretivist, the world is constructed by each person according to both subjective and intersubjective dialogical exchanges (Putnam 1989). Interpretivists are concerned with symbolic meaning and forms of representation that help to better understand the situation (Bruner 1986). Interpretivists describe meaning as socially constructed through human interaction. Dewey (1959) describes education as a reconstructed experience. Education is a continuous reconstruction of past experiences as a result of present experiences. For example, participants come to a CISV TTT with an understanding of the world based upon their past experiences and interactions. During TTT, individuals have new experiences through interactions with other individuals from many different places and spaces. Their understanding

29 of the world is reconstructed through these interactions with these individuals also within this new “place.” Since the location of this international training is often a new experience for each individual, the place also plays an important role in the reconstruction of understanding. Schram (2003 p. 34) explains, “Generating and synthesizing these multi-voiced and varied constructions requires that you engage at some level in the lives of those around whom your inquiry is focused; it is through direct interaction with their perspectives and behaviors that you focus and refine your interpretations.” Since the goal of an interpretive approach is to gain an understanding of experiences and the way that the participants construct knowledge, it is important, as a researcher, to interact and engage with the participants in order to discover the issues that represent them most appropriately. Schram (2003, p. 34) explains that interpretivist researchers attempt to understand this complex and constructed reality from the point of view of those who live it. In this way the researcher is focused on particular people, in particular places, at particular times. Many of the processes that are used within CISV are based upon the teachings of Kurt Lewin. Lewin is recognized as the “founder of social psychology” and is renowned for his study of group dynamics and organizational development. Lewin’s research provided early understanding of change processes in social situations (Allen, 1982). For Lewin, studying human interaction within the organization was vital in working toward improvement in organizational effectiveness. Kurt Lewin is at the foundation of ideas utilized by CISV in terms of field theory and experiential learning. Doris Allen (1982,) explains the connection between CISV and Lewin’s ideas: Lewinian concepts were in the fabric of my thinking during the organizing years of CISV, 1946-1951, and thereafter. Part of this time I had lived in Lewins’ home with his family to tutor him in English in his preparation to travel to the United States for a quest professorship in psychology at Stanford University. And I had been a member of the group of former students whom Lewin, after arriving in the United States in 1933, had collected once a year for several days of meetings. (p. 11) Doris Allen (1982, p.10) described the need for a theoretical framework for research within CISV to create a common way of representing human relations and a common vocabulary. She realized that human behavior is too complex to deal with without some stable

30 system of analysis to which issues could be referred. The field theory of Kurt Lewin provided that structure. Field theory is a tool for analyzing human behavior. While many approaches offer an analysis of the person or the environment, field theory emphasizes the interaction of the person and the environment. Field theory deals with the behavior of a person and the psychological environment in which the interaction takes place (Lewin, 1951, p. 201). The central ideal of field theory is that “behavior has to be derived from a totality of coexisting facts” (Lewin, 1951, p. 187). This totality is not only an analysis of the forces at work in a particular individual at a particular time but also an analysis of the particular environment at that time (Lewin, 1951, p. 213). The general equation [B = f(PE)] states that the behavior (B) is a function (f) of the person (P) and his or her environment (E) (Lewin, 1951, p. 337). Kurt Lewin is also readily associated with experiential learning. CISV facilitates an experiential learning model within the leadership training process and within its educational programs. One of the most quoted connections to the importance of experience is attributed to Confucius. “I hear and I forget. I see and I remember. I do and I understand.” Others cite a quote from in Trachinae written in 400 B.C. “Knowledge must come through action. You will never be sure unless you put it to the test” (Torrence, 1966 line 592.). A number of different terms have been used to label the process of learning through experience. John Dewey (Dewey, 1916, p. 104) discussed “learning by doing.” Other terms include experience-based learning, trial and error learning, real world learning and authentic learning. Hoover and Whitehead (1975) clarify that traditional learning tends to focus within the cognitive domain while experiential learning goes beyond. They offer the following definition: “Experiential learning exists when a personally responsible participant cognitively, affectively and behaviorally processes knowledge, skills and/or attitudes in a learning situation characterized by a high level of active involvement” (Hoover &Whitehead, 1975, p. 25). In addition to Lewin and Dewey, many other well renowned theorists are associated with the concept of experiential learning including: Jean Piaget, William James, Carl Jung, Paulo Freire, Carl Rogers and others (Kolb & Kolb, 2005). David Kolb (1984) identified what he calls an experiential learning theory (ELT). According to Kolb (1984, p. 41), “one of the reasons the theory is called “experiential” is its intellectual origins in the experiential works of Dewey, Lewin, and Piaget. Taken together, Dewey’s philosophical pragmatism, Lewin’s social psychology, and Piaget’s cognitive- developmental genetic epistemology form a unique

31 perspective on learning and development.” Kolb (2005. p. 2) explains that ELT is a holistic model of the experiential learning process and a multi-linear model of adult development. ELT is the process in which knowledge is developed through the transformation of an experience. Kolb identified six characteristics of experiential learning: a) experiential learning is a process, not an outcome, b) it derives from experience, c) it requires an individual to resolve dialectically opposed modes of adaptation, d) it is a holistic integrative process, e) it requires the interplay between a person and the environment, and f) it results in knowledge creation (Kolb & Kolb, 2005, p. 194). The Train the Trainer (TTT) course as used within CISV utilizes an experiential learning approach introduced by Lewin and modeled after the concepts identified by Kolb in experiential learning theory. Program Evaluation Documentation of evaluative techniques dates back thousands of years. Formal evaluation was evident in the public sector as early as 2000 B.C. Chinese officials measured the proficiency of applicants for governmental positions through civil service examinations. Verbally mediated evaluations were inherent in the learning process used by . Some centuries later formal evaluations became a driving force behind social and educational decisions (Fitzpatrick et al. 2004, p. 31). Various reasons are provided for conducting program evaluations. Some simply state that it is a requirement of the program or grant. Some use program evaluation to judge the merit of sustaining programs. Hasenfeld et al. (2002, p.2) explain “program evaluation is important not only because it informs decision makers about the success of a program, but also because it is a valuable tool for learning. Evaluation can identify ways to improve the quality of a program, structure it more effectively and make it more responsive to the needs of the users. As such, it can be a catalyst for program innovation.” Evaluation can provide insight into effectiveness and efficiency and make a program more responsive to stakeholders needs. Research in the field of program evaluation has identified a variety of methods and approaches. Choosing a direction depends upon many factors including the proposed purpose of the evaluation and the needs of the organization. Generally, there are three purposes for performing training evaluations: formative evaluation, summative evaluation and confirmative evaluation (Morrison, Ross & Kemp, 2003). The purpose of formative evaluation is to enhance learning; it is usually performed in the early stages of training program development in order to

32 gain feedback to enhance the training program. Summative evaluation is used to measure training effectiveness after the training has been completed. Confirmative evaluation is performed to assess whether training was successful over time; it is thought to be a continuation of summative evaluation some time after the training was facilitated. There are many methods, models, processes and orientations available for evaluating training. Cousins and Earl (1992, p. 410) describe a participatory model that creates a partnership between the evaluators and stakeholders. In this approach, stakeholders actively engage in all aspects of the evaluation from the development to implementation. Participatory evaluation emphasizes sharing knowledge and building evaluation skills across the organization in order to develop a culture of continuous improvement and learning. Zukoski and Luluquisen (2002, p. 4) explain, “The process seeks to honor the perspectives, voices, preferences and decisions of the most affected stakeholders and program beneficiaries. Ideally, through this process, participants determine the evaluations focus, design and outcomes within their own socioeconomic, cultural and political environments.” They further identify the following six benefits to the implementation of a participatory program evaluation process: • Identify locally relevant evaluation questions: Participatory evaluation ensures that the evaluation focuses on locally relevant questions that meet the needs of program planners and beneficiaries. • Improve program performance: Participatory evaluation is reflective and action- oriented. It provides stakeholders, including beneficiaries, with the opportunity to reflect on project progress and generate knowledge that results in being able to apply the lessons learned. • Empower participants: A participatory approach is empowering because it claims the right for local people to control and own the process of making evaluation decisions and implementing them. • Build capacity: Conducting a participatory evaluation promotes participant learning and is an opportunity to introduce and strengthen evaluation skills. It can provide participants with tools to transform their environments. • Develop leaders and build teams: Participatory evaluation builds teams and participant commitment through collaborative inquiry. Inviting a broad range of stakeholders to participate in different parts of the process can develop and celebrate local leadership

33 skills. It can lead to stronger, more organized groups, strengthening the community’s resources and networks. • Sustain organizational learning and growth: Finally, a participatory evaluation is not just interested in findings; it is focused on creating a learning process. It creates a knowledge base among local people and organizations, which can be applied to other programs and projects. The techniques and skills acquired can lead to self-sustained action. (p. 4) According to Hasenfeld et al. (2002, p. 3) a different vision of the purpose and conduct of program evaluation is present within the participatory model. As with other forms of program evaluation, participatory evaluators are committed to accurate, objective, and impartial reporting of operations within the organization. In addition, these findings can be used to stimulate program changes, innovation and organizational learning. The participatory model depends upon collaborative agreements between the evaluators and the stakeholders. The evaluators need to accept the goals and objectives of the program and the stakeholders must accept the principles guiding the evaluation. Participatory evaluation has been identified as an appropriate approach for studying the Train the Trainer process since CISV clearly fulfills the organizational requirements delineated by Cousins and Earls (1992) as organizational realities. The first organizational reality identified is that evaluation must be valued by the organization. CISV is an educational organization that has actively engaged in research and evaluation since its inception in 1951. CISV identifies research and evaluation as a priority as seen in past and current strategic plans. The second reality states that the organization must be willing to provide the time and resources required to complete the evaluation (Cousins & Earls, 1992). CISV has been quite generous with access to materials and space. They have created opportunities for investigation and data gathering. The organization has encouraged volunteers and paid staff to fully participate in this active learning process. The third reality states that the organization must be committed to organizational learning as a means of continuous improvement. CISV has demonstrated the commitment to continuous improvement through the strategic planning process. The means by which the organization monitors progress toward strategic goals encourages action. Entities within the organization demonstrate proactive responses to progress monitoring and implement new directions.

34 The fourth reality states that those participating in the evaluation should be motivated to participate. CISV volunteers have a reputation for active involvement and participation. The international board of trustees, the international executive committee and the officers of the international office identified this research project as a priority within CISV as an organization. A review of research elicits numerous methods for performing program evaluation. Some of the more common methods include: Kirkpatrick’s Four Levels, Gap Analysis, Context, Input, Process, Product (CIPP), Training Validation System, Return On Investment and Success Case Method (Eseryl, 2002). McNamara (2000) identifies the following three approaches: Goal-based evaluations -- evaluating the extent to which programs are meeting predetermined goals or objectives; Process-based evaluations -- understanding how a program works, how it produces the results that it does; Outcomes-based evaluation -- asking if your organization is really doing the right program activities to bring about the outcomes you believe are needed by your participants. Kirkpatrick’s (1994, p. 21) model follows the goal-based approach and is based upon four questions that elicit the four levels that he defines in program evaluation. These four levels are commonly known as: reaction, learning, behavior and results. Kirkpatrick explains that reaction is simply a measure of how those who have participated in training actually react to it. This is often seen in traditional post training feedback forms. Learning is seen as a measure of how participants change attitudes, improve knowledge and increase skill as a result of the training. Behavior is shown in the manner in which the training has influenced a change in the way participants go about doing things. Results are defined as the final outcome that occurred as a result of attending the training. Kirkpatrick (1994, p. 25) explains that results can include increased production, higher return on investment, and increased quality. While all four levels of evaluation can provide unique and valuable information, not all are utilized uniformly. The American Society for Training and Development found that 45% of the organizations surveyed only evaluated the participant’s reactions to the training (Bassi & van Buren, 1999). They further stated that reaction was evaluated by 93% of the organizations; learning was evaluated by 52%, behavior by 31%, and 28% of the organizations evaluated results. Mixed Methods Evaluation Designs The physical sciences and the current national funding agencies for education research advocate the use of the scientific method, following a positivist methodology and utilizing a

35 quantitative experimental design, as the only valid and acceptable form of evaluation (St. Pierre, 2006). Over a century of scientific study has advocated this notion as truth. In search of validation and acceptance, social science researchers also promoted this research tradition. The beginnings of evaluation research predominantly used a deductive approach with quantitative measures and experimental designs (Fitzpatrick et al., 2004). In opposition and drawing upon the diverse philosophical assumptions regarding ontological and epistemological issues, educational researchers began acknowledging qualitative methodologies as a legitimate means of evaluation. Creswell (2003, p. 19) states, “the procedures of qualitative research, or its methodology, are characterized as inductive, emerging, and shaped by the researcher’s experience in collecting and analyzing data.” Researchers have varying opinions regarding the use of mixed methods in evaluation research. The issue tends to concentrate in the realm of a philosophical debate. According to Creswell (2003), there are three stances on the issue. Purists argue that qualitative and quantitative studies share different paradigms and are fundamentally incompatible. Situationalists believe that certain approaches are appropriate for specific situations. Pragmatists believe that integrating both approaches within a single study is often necessary. Creswell (2003) describes three models for combining methods. The two-phase model uses both qualitative and quantitative methods but at different times within the study. The dominant-less dominant model identifies one approach as primary and the other as complimentary. The mixed model approach uses both approaches in concert throughout the study. There are some aspects of a program evaluation of CISV that are measurable in a quantitative manner. The use of a quantitative, quasi-experimental design could evaluate the structural components of the organization. Currently, the annual report provides statistics describing the number of programs facilitated, the number of participants, the number of nations participating etc. Data could be gathered that report activities serving as indicators of the goals and objectives for each program. This information would fulfill accountability requirements. There are, however, a number of concepts within the goals and objectives of CISV that are difficult to measure such as: developing positive attitudes towards others, developing intercultural competence, and developing a sense of cooperation. Questionnaires, rating scales and surveys are often used to capture data related to these abstract concepts. However, these quantitative measures often leave gaps in the depth of understanding the personal response.

36 Given these constraints, I recommended a mixed method evaluation be conducted. The rationale behind a mixed method is in the interconnectedness of the information gathered. Quantitative data can inform the qualitative, and the qualitative data can inform the quantitative in order to construct a deeper understanding and more complete picture of TTT. Gaining a thorough understanding of social interactions in an international program is complex. Using a mixed method approach assists in constructing a clearer picture of the “whole” of the experience. It is appropriate to gather both qualitative and quantitative data within an interpretivist study. In evaluation research, the question of qualitative versus quantitative is less about choosing a methodology than about utilizing the best tools to respond to the purpose of the evaluation. Fitzpatrick (2004) states that most experts in the field of program evaluation involve all stakeholders in determining the means for assessing outcomes. The current thought promotes the use of diverse means (both qualitative and quantitative) with multiple measures of performance. Problems in utilizing only quantitative measures for program evaluation include an over-reliance on a single outcome measure, lack of validation of tests, an undue emphasis on quantitative measures and failure to educate the public concerning the meaning of labels such as proficient. Creswell (2003) proposes that evaluators advocate for not only the possibility, but also the necessity, of utilizing both qualitative and quantitative approaches in order to explain the complexity of social phenomenon.

37 Chapter 4 Method Overview of the Program Evaluation Process Throughout the time that I had been enrolled in the doctoral program, I had been talking with people at the University and with CISV about conducting a research project that would somehow bring my world of education together with my world of CISV. Many ideas were discussed. Several projects were informally proposed each with a unique combination of interactions within these two facets of my world. In August 2009, during the Annual International Meeting (AIM) of Children’s International Summer Villages, two research projects were discussed by the board of trustees as a result of the review of the strategic plan (K. Hingorani, personal communication, January 12, 2010). Knowing of my interests in pursuit of a research topic, the education officer called me to ask if I would be interested in performing the evaluation of the Train the Trainer process. This topic was discussed with my dissertation chair and it was agreed that this would be a very appropriate project. In addition, the idea that this was a real-life project that would be fulfilling the needs of an international organization was an added bonus. During the fall of 2009, the proposal for this research project was developed in collaboration with my dissertation chair, the international education officer of CISV and the chair of the research and development committee (who was also leading the second research project identified by the board of trustees). The proposal was developed and approved by the dissertation committee in December 2009. Approval was also granted from the Institutional Review Board for the Use of Human Subjects in Research (IRB). In addition to proposing the two research projects, the trustees of the Annual International Meeting 2009 also granted funds for a meeting of the expanded executive committee in order to address organizational and structural issues that were evolving between CISV programs. This meeting was to be known as the Writeshop 2010. The main goal of this meeting was to clarify the scope and direction of each of the programs within CISV while aligning goals and indicators that address the mission and principles of CISV. The Writeshop 2010 was held in January 2010 in Newcastle, England. It was the formal jumpstart of the TTT program evaluation process. Those involved were invited purposefully due to their representation on the international expanded executive committee. The participants included: members of the International office,

38 members of the International Pool of Trainers, and International Program Chairs. These representatives are referred to as the administrative participants. These administrative participants were given the opportunity to volunteer to be interviewed as part of this research project. These interviews would assist in the development of the online survey and provide information relating to their perception of the connection between their specific program and the goals and implementation of the TTT. These volunteers were provided detailed information regarding participation and informed consent was obtained. In addition to participation in the Writeshop 2010 and the interview process, my time at the International Office was also a time to review archival documents related to the history and procedures of training within CISV. I was given access to the archival documents, office space within the International Office and access to the copy machine in order to reproduce selected documents. Following the Writeshop 2010, the online survey was developed with ongoing feedback from my dissertation chair and the education officer of CISV and the chair of the International Pool of Trainers. This survey was developed based upon the objectives of the TTT, the goals and indicators of CISV, as well as, information from administrative participants regarding the types of feedback they would find helpful regarding the impact of training. The final draft of the online survey was posted May 2010. Preliminary quantitative data were analyzed and reviewed with my dissertation chair and the educational director of CISV. In August 2010, I was invited to present these preliminary data to the Board of Trustees at the Annual International Meeting (AIM). In addition to providing this information, it was also an opportunity to gather additional qualitative data from training participants and administrative participants who were present at this AIM. Those involved were given forms to provide informed consent and given the opportunity to opt out of participation in the interview. In January 2011, the International Pool of Trainers (IPT) met to review progress, develop new strategies, and respond to the feedback from various sources regarding the first three years of TTT implementation. Members of the IPT volunteered to have this process serve as an active part of the data gathering process for the program evaluation of the TTT. Informed consent was obtained from all the participants of this meeting. This organizational process was observed and participants actively engaged in documenting the actions taken throughout the meeting.

39 Interviews with Administrative Participants Participants. All those involved in Writeshop 2010 were invited to participate in the interviews. This group included fourteen members of the expanded executive committee and the education officer, who was facilitating the meeting. In addition, interview opportunities were made available to members of the international office who have direct responsibilities with these programs. In total, sixteen participants volunteered to take part in the interviews. These included the ten members of the expanded executive committee and six members of the international office. Of these participants, nine were female and seven were male. They represented eight countries across five continents including: , Colombia, Denmark, Egypt, Germany, Great Britain, Spain and the United States. The ten members of the expanded executive committee represent extensive involvement in CISV at the local chapter level, the national level and now serving as chairs of the international committees. Members of the international office represented a wide variability of CISV experience. Two members had been hired within the year and had no prior CISV experience. One member had been working in the international office for three years with no prior CISV experience. Three members had been working in the international office more than five years prior with extensive CISV experience. Of these sixteen participants, thirteen had participated in a TTT prior to interviews. Interview Procedures. Prior to the arrival at the Writeshop, all were informed of my presence and purpose in attending the event. Upon arrival, a brief overview was made of the purpose of gathering information. The interest in gathering interview data was clarified. Each participant was given an informational letter with an opportunity to provide informed consent (appendix A). A voluntary sign-up sheet was then made available for days and times during open times within the Writeshop schedule. The following questions (appendix C) were used during the interviews: • What is your current role in CISV? How long have you been involved and in what other capacities? • What were your initial opinions regarding the proposal for a TTT? • Do you see TTT connected to your role? If yes, in what way? • What information would you like to gain from this TTT program evaluation? (prompt if necessary regarding content, implementation, organizational logistics, utilization)

40 • What would you like to see for the future of the TTT? The majority of the interviews took place within office space at the international office. Two of the interviews took place in the homes of participants who lived locally. Interviews took place individually and in a face-to-face discussion format. A blank sign-up sheet was distributed to participants listing the available days and times for interviews. I maintained the master sheet in order to ensure confidentiality. Participants volunteered involvement by scheduling a time with me on the master sign-up sheet. Informed consent forms had previously been distributed in order to provide participants sufficient time to review and decide upon voluntary participation. Prior to beginning the interview, the informed consent form was reviewed especially in regards to confidentiality and the security utilized with the tape recordings. The consent form was signed and the tape-recorded interview commenced. Interviews typically lasted between twenty and sixty minutes. Interviews were discussion based using the general discussion questions listed above with probing questions for clarification. In addition to the tape recordings, I also took written notes in order to ensure responses for each area of questioning. I transcribed tape recordings. Written notes, tape recordings and transcriptions have been solely maintained by me personally and no one has had access to these data. Review of Archival Documents. The review of existing documents involved three main categories of information. Following each of the fourteen TTTs, traditional training evaluations were conducted. These evaluations were opportunities for the training participants to provided feedback to the trainers regarding the overall training in general, specifics of the course and reactions to new information and materials. These evaluations were collected and summarized by the trainers prior to submission to the education office. After ensuring that the evaluations contained no names of the trainees, the chair of the International Pool of Trainers sent these documents to me to be used as a part of this program evaluation. Documentation of official meetings in CISV are posted in the resources area of the CISV website. Minutes of the Annual International Meetings, details of the original and current strategic plan, actions of international committees and proposals and decisions of the board of trustees are available online. These documents provided detailed accounts of the procedures leading up to the decision to create a Train the Trainers model, the formation of the International

41 Pool of Trainers, and the decision to host Writeshop 2007 to create, develop and implement the TTT plan. While in Newcastle, England in January 2010 for the Program Chairs Writeshop, I was provided access to the archived documents stored in the International Office. It is here that I was able to collect copies of documents that provide a history of CISV (including Doris Allen’s proposal to the United Nations), a history of leadership training within CISV (including minutes from SOLAR, the first formal international meeting to structure national training), past copies of the CISV annual reports, and past copies of Interspectives, the research journal published by CISV International. Interviews with the International Pool of Trainers Participants. There were seven active members (including me) of the International Pool of Trainers (IPT) present at the time of this data gathering process. Of these seven members, there were three female and four male members representing the three defined regions of CISV training (Asia, Europe and the America’s). Individuals were from the United States, Canada, Mexico, the Dominican Republic, Denmark, the Netherlands and the Philippines. Interview procedures. The other six members of the IPT were initially contacted through email regarding the gathering of data from the trainers themselves. Trainers were provided an opportunity to opt out of this process. An informed consent statement was included in the email questionnaire and response to the questionnaire served as acknowledgement of informed consent. Of the six members contacted, two interviews were held via Skype and four responded through email. The emails were stored on a secure computer used solely by this researcher and deleted at the completion of this study. Written notes were taken during the Skype discussion and maintained in secure files by this researcher. The following information (appendix D) was requested from the members of the IPT. Four questions requested feedback regarding TTT in general, as well as, feedback regarding specific TTTs that the participant had facilitated: • Comment on the process and outcome of Writeshop 2007 • Comment on the TTT curriculum, schedule and logistics of the TTT • Comment on the IPT organizational process • Please provide feedback about the TTTs that you have facilitated.

42 Three questions requested feedback regarding issues that have been proposed or suggested for future planning of the TTT: • What is your opinion about an advanced level of TTT? How should it be structured? • What is your opinion of an online format? • What would you like to see happen with TTT regarding IPT responsibilities?

The Online Survey Participants. During the first three years of implementation, fourteen training courses were facilitated throughout the regions of CISV. The three regions for this study are defined as Asia- Pacific, Europe and the America’s. Training courses were facilitated in the following locations: • 2008: Brazil, Canada, Indonesia, Portugal and Sweden • 2009: Thailand, Colombia, Germany and France • 2010: Vietnam, Sweden, England, Mexico and Germany A total of 250 participants attended TTTs across the three regions during the first three years of implementation including 153 women and 97 men. These training participants represented leadership from National associations (NA) within the three regions. Training participants hailed from 52 countries. There were 89 training participants in five locations during 2008. There were 89 training participants in four locations during 2009. There were 72 training participants in four locations during 2010 (however the TTT held in Germany that year was held too late to be included in the survey). Three TTT courses were held in the Asia-pacific region with a total of 54 training participants. Six TTT courses were held in the European region (not counting Germany) with a total of 113 training participants. Four TTT courses were held in the Americas with a total of 83 training participants. Of the 250 training participants, 165 responded to the online survey. This represents a 66% response rate. Survey instrument. The draft online survey was created in English. It was pre-tested for content, structure and language-based issues for participants for whom English is not the primary language. The online survey was developed with ongoing feedback from my dissertation chair, the education officer of CISV and the chair of the International Pool of Trainers. This survey was developed based upon the objectives of the TTT, the goals and indicators of CISV, as well as information from administrative participants regarding the types of feedback they would find helpful regarding the impact of training.

43 The online survey consisted of twenty items (appendix E). Many items contained multiple response prompts. Items one, two and three contained multiple prompts relating to: personal demographics such as gender, age and country of origin; CISV demographics such as role in CISV, years of involvement and chapter affiliation; and TTT demographics such as TTT location and year, and the number of trainings they have facilitated since attending a TTT. In CISV, there is an emphasis on the development of attitudes, skills and knowledge (ASK) not only in training, but also in terms of program planning and implementation. Items four through nine are focused upon attitudes, skills and knowledge in relation to the TTT. One of the issues discussed when this training evaluation was initially proposed was in regard to the lack of pre-test data. For this reason items four, six and eight asked respondents to rate the items with their perspective as they remember before attending the TTT, as well as their perspective after attending the TTT. Item four included three prompts to rate on a one to five scale in relation to attitudes. Item five was an open-ended response to include comments relating to attitude. Item six included three prompts to rate on a one to five scale in relation to skills. Item seven was an open-ended response to include comments relating to skills. Item eight included three prompts to rate on a one to five scale in relation to knowledge. Item nine was an open- ended response to include comments relating to knowledge. Item ten included four prompts asking the respondents to rate the overall effect of TTT upon their role as a trainer. Item eleven was an open-ended response relating to overall effects. Items twelve and thirteen were open- ended responses relating to the expectations of the national association or national committee as a result of their attending the TTT. One of the initiatives of CISV during this time period was to encourage consistency across chapters, national associations and programs while continuing to maintain the unique aspects of each. Item fifteen included four prompts asking the respondents to rate their perspective on the use of the TTT to promote consistency. Items sixteen through twenty were open-ended questions regarding issues identified that relate to future planning. Procedures. Pre-notification of the survey was sent from the International Office to training participants informing them that they would be receiving this survey request and that it was part of a CISV approved research project. The final draft of the online survey was posted through the Miami University Checkbox survey tool in May 2010. An email was sent to all training participants that included the link to the online survey and an online version of informed

44 consent (appendix B). All participants were provided the option of opting out of participation in the survey. The online survey tool was set to automatically send reminders to those who had not responded one week and two weeks following the initial posting. Observations of the International Pool of Trainers Meeting Six members of the International Pool of trainers were present at this meeting held in Copenhagen January 2011. The International Education officer was present in order to facilitate some sessions and assist with action planning. In addition, the liaison from the Executive Committee was able to attend parts of the meeting. She provided insight and feedback from the Executive Committee perspective relating to analysis of past activities and ideas for future planning. The structure and curriculum for the TTT was developed at the Writeshop in Newcastle, England in November 2007. Those involved with the creation of this plan thought it important to view the first year of implementation during the spring of 2008 as a pilot year and regroup following the first round of courses in order to make any necessary adjustments based upon trainer and trainee feedback. Due to constraints with budget and resources, proposals for a follow-up face-to-face meeting were rejected during the Annual International Meeting in 2008 and 2009. The IPT interacted via email and Skype during the implementations of TTT for 2009 and 2010. The proposal was accepted at the AIM in 2010 and funds were granted for the January 2011 IPT meeting in Copenhagen. The purpose of this meeting was to review the first three years of implementation, align procedures of the TTT with the newly identified common goals and objectives, as well as create future plans. These plans included an updated trainer guide, materials for trainees, procedures for certification and recertification and the possibility of advanced courses. Preliminary discussions in planning the IPT meeting clarified the need to use the meeting not only as a means of continuous improvement but also to document how the IPT adapts and adjusts the TTT based upon feedback, experiences and changes within the organization for the purposes of this program evaluation. Written consent was obtained from those present. Members of the IPT alternated in taking continuous notes regarding procedures and decisions made throughout the four days of the meeting. In addition, I took observational notes to clarify actions specific to the program evaluation.

45 Chapter 5 Results and Discussion The preparation of leaders in CISV has evolved from an informal process that included a list of roles and responsibilities to a more formalized process including multi-level training procedures. Leadership preparation began with an emphasis on preparing individuals who would be involved with the youth programs. The perspective has changed to now include training not only for those attending programs, but also for anyone in a leadership position within the organization locally, nationally and internationally. The current directive for the International Pool of Trainers (IPT) is to provide a Train the Trainer (TTT) course to all within the organization who have a mandate to train. This chapter is organized first to address the quantitative and qualitative data gathered from the online survey. The data are analyzed initially by subject matter as it was presented in the survey, and then in terms of how the data align to the newly identified common training goals and indicators. Secondly, there is a discussion of the data obtained through interviews. These interviews provide data regarding how individuals view the TTT as it relates to CISV in general and specifically to their roles as leaders in the organization. Finally, there is a discussion of the actions of the International Pool of Trainers at the IPT meeting in January 2011. These actions show the response of the IPT to feedback from past training course participants, feedback from IPT members and feedback from leadership within the organization. Online Survey Results During the first three years of implementation, fourteen training courses were facilitated throughout the regions of CISV with 250 training participants attending. A total of 165 responses to the online survey were completed. This demonstrates a 66% response rate. A number of statistical analyses were run to compare the quantitative data. One-way ANOVA was used to compare responses between the regions (Table 2, appendix G) across the three years, as well as a comparison of the responses across the three years of implementation (Table 3, appendix G) across the regions. The questionnaire was designed to reflect components of training identified by the administrative participants. Since the objectives of the training are structured in terms of attitudes, skills and knowledge, the questionnaire contained sections focusing upon attitudes, skills, and knowledge, as well as, overall effect of the TTT training. Each area contained

46 questions in which the respondents were asked to provide a rating of their experience followed by an open-ended question for participants to provide additional qualitative data. Mean responses to the survey items on these attitudes, knowledge and skills are presented in Table 2 (appendix G). This first part of the analysis provides a general overview of participant ratings of their attitudes, skills, knowledge and overall effects after completing the TTT training. It also incorporates examination of whether there were regional or time differences. Attitude People are attracted to CISV because of its unique approach to intercultural experiences through peace education. Those who continue to participate in CISV and eventually take on leadership roles at the local, national or international levels are attracted by the mission and purpose of the organization. A number of responses to the survey spoke to the idea of having very strong positive attitudes towards CISV before and after participating in the TTT. Many participants did acknowledge that the TTT emphasized that attitude was an important part of developing skills and knowledge: “I have attended many training sessions before the TTT, therefore I already had many of these attitudes. However doing the TTT has helped me understand that training has to work with not only skill and knowledge but attitude too.” “TTT is a great inspiration, I have always been willing to change my attitude. But TTT actually changed it.” “It gave me motivation and an opportunity to sit down with a fellow trainee from my NA to plan.” Because the intention was to look simultaneously across years of program operation, and regions of the world when testing for differences between the before and after ratings of participant attitudes, skills and knowledge (these appear in the section on the goals), tests were first run to examine whether there were differences by region (Table 2, appendix G), or differences between the three years of the TTT program (Table 3, appendix G). The first attitude question related to adopting a proactive and responsible attitude. No significant difference was found between the three regions (F(2,162) = .045, p > .05), or between the three years of implementation (F(2,162) = .149, p > .05).

47 The second attitude question related to the development of an action plan for personal growth. No significant difference was found between the three regions (F(2,162) = 1.583, p > .05) or between the three years of implementation (F(2,162) = 2.628, p > .05). The third attitude question related to acting inclusively and with an open mind. No significant difference was found between the three regions (F(2,162) = 2.117, p > .05). A significant difference was found between the three years of implementation (F(2,162) = 6.763, p > .05). Tukey’s HSD was used to determine the nature of the differences between years of implementation. This analysis revealed that respondents from year 1 rated this area lower (mean = 4.27) than the respondents from year 3 (mean = 4.63). Respondents from year 2 were not significantly different from either of the other two groups. In summary, this analysis of the quantitative data demonstrates that there were no significant differences between regions or across years of implementation for attitude except for one area of comparison. There was a significant difference between year one and year three for the third attitude question relating to acting inclusively and with an open mind. When looking at the organization over the past three years this makes sense for several reasons. Many changes were occurring simultaneously with the roll out of the TTT. New educational materials and evaluation tools were being piloted. CISV was in the process of re-branding. In this process, the statement of purpose and the wording of the educational principles were aligned. In addition, the first year of the study was prior to the roll out of the new strategic plan. Inclusion is a major component of one of the four educational principles and all aspects of the organization were working toward the visibility of inclusion within their committee or program. In addition, CISV scheduled a year to focus on each of the four educational principles and 2009 was the year of Diversity. Skills The TTT was created to teach the “how” of training and not necessarily the “what” of training. The focus had been built upon effective strategies for adult learning, group dynamics and facilitation. However, one advantage of having a TTT process that brought together representatives from all areas of CISV was in terms of teaching the use of the new materials and new evaluation tools. Comments in the skills area showed differences between those who have been involved in CISV for an extensive period of time and those who have recently begun in leadership roles.

48 “Having been involved with National Trainings for several years many of the areas covered at the TTT were not new to me. However, the TTT brought the current emphasis of educational evaluation to my attention which has and will continue to affect how I think and deliver trainings.” “I feel much more competent to deliver training but I also feel more confident with the skills I received. I am much more aware of the group and the group needs after the TTT.” The questionnaire was formatted in such a way that participants were given the freedom to answer or opt out of answering questions if they so desired. One of the demographic questions that participants tended to opt out for was the question regarding length of time in CISV or in a training role. For this reason, it was not possible to run statistical data comparing long time CISV trainers with those who were new to their role. The first skill question related to planning and delivering training. No significant difference was found between the three regions (F(2,162) = 1.028, p > .05) or between the three years of implementation (F(2,162) = .779, p > .05). The second skill question related to facilitating reflection and discussion. A significant difference was found between the regions (F(2,162) = 5.704, p > .05). Tukey’s HSD was used to determine the nature of the differences between the regions. This analysis revealed that participants from region 2 rated this area lower (mean = 4.13) than participants from region 3 (mean = 4.44). Participants from region 1 were not significantly different from either of the other two regions. No significant difference was found between the three years of implementation (F(2,162) = 2.768, p > .05). The third skill question related to collecting evidence through goal-based evaluation. No significant difference was found between the three regions (F(2,162) = 1.534, p > .05) or between the three years of implementation (F(2,162) = 1.668, p > .05). An analysis of the quantitative data demonstrates that there were no significant differences between regions or across years of implementation for skills except for one area of comparison. There was a significant difference between region two (Asia) and region three (Europe) for the second skill question relating to facilitating reflection and discussion. A review of the qualitative data did not reveal information that might clarify this difference. One could speculate that the difference in training styles between the different IPT members who were

49 facilitating each of the TTTs could account for this difference. However, these training differences would also have appeared in other areas of training. One could speculate that cultural differences might play a role in the variance between regions. There may be differences in the way reflection and discussion in training are viewed from region to region. Knowledge The knowledge area focused upon the participants’ knowledge of their training roles and responsibilities, their knowledge of the CISV educational approach and the significance of educational evaluation. Much of this emphasis was due to the fact that CISV was rolling out new educational materials and new evaluation tools. Comments within the knowledge section were very positive regarding the thorough understanding of models, methods and approaches used within CISV. “I am impressed by the TTT when it came to educational evaluation and goals! I learned a lot about the importance of that. Also ASK really came alive again at the TTT and now really is a useful tool!” “The explanation of ASK and educational evaluation helped me to organize my knowledge of CISV education and I think this has been fundamental to improve the effectiveness of the training sessions I delivered after.” “The TTT gave me a better understanding of the current state of educational evaluation within CISV.” “I think that these are very much unique for CISV, and even though I know about facilitation, planning etc. this specific knowledge about CISV made it all fit together in my context much better.” The first knowledge question related to training responsibilities. No significant difference was found between the three regions (F(2,162) = .058, p > .05) or between the three years of implementation (F(2,162) = .402, p > .05). The second knowledge question related knowledge of the ASK model for education and training. No significant difference was found between the three regions (F(2,162) = 1.898, p > .05) or between the three years of implementation (F(2,162) = 1.497, p > .05). The third knowledge question related to the knowledge of the theory, practice and purpose of educational evaluation. No significant difference was found between the three

50 regions (F(2,162) = 1.204, p > .05) or between the three years of implementation (F(2,162) = .592, p > .05). An analysis of the quantitative data demonstrates that there were no significant differences between regions or across years of implementation for any of the comparisons regarding knowledge. Overall Effect Questions within the section relating to the overall effect of the TTT related to the participant’s perspective on how the TTT impacted their ability to fulfill their training role, their training competencies and their ability to design and deliver training. Comments within the overall effects section were very positive regarding the impact on the participant’s ability to provide future training. “I think the TTT strength is on the incorporation of educational, leadership, and facilitation theory and practice. It is important to the professionalization and sustainability of our organization to have volunteers in leadership positions understand the theory and have the tools to lead an educational organization. CISVers do not need to learn much more about CISV but more how what we do fits into the greater context of education, leadership and organizational development.” “TTT empowers trainers and provides them with a strong curriculum that can be easily replicated to any situation and training environment, even outside of CISV.” “I believe that the TTT is an excellent space for intensive, professional level training of CISV facilitators. It challenges CISVers to think about training in an objective and professional way, detached from the organizational-cultural norms. It has the potential to breed new creative practices and enrich the organization tremendously.” “We insure a high level of training across the globe. Trainers get a full pack of training tools; it is very good for people who have none or little training. By attending the TTT the trainers can got back and train in their NAs knowing all about how to train in CISV, the content areas, educational principles etc.” The first overall effect question related to enhancing the ability to fulfill their training responsibilities. No significant difference was found between the three regions (F(2,162) = .112, p > .05) or between the three years of implementation (F(2,162) = 2.224, p > .05).

51 The second overall effect question related to enhancing training competencies. No significant difference was found between the three regions (F(2,162) = 1.043, p > .05) or between the three years of implementation (F(2,162) = 3.317, p > .05). The third overall effect question related to strengthening the ability to design and deliver training programs. No significant difference was found between the three regions (F(2,162) = .330, p > .05) or between the three years of implementation (F(2,162) = .163, p > .05). An analysis of the quantitative data demonstrates that there were no significant differences between regions or across years of implementation for any of the comparisons regarding overall effects (see table 2 & table 3 appendix IG). An ANOVA was run to compare the means for any differences across the three years of implementation and across the three regions for each of the questions within the attitudes, skills, knowledge, overall effects and promoting consistency categories. In all there were 102 possible comparisons. Of these comparisons, there were two areas that showed statistically significant differences. In any statistical analysis, there are opportunities for error or bias. The participants were all active members of CISV who hold or will hold leadership positions. The participants are passionate about the organization and the role they play in it. The possibility of positive response bias exists. In addition, the possibility of type one error also exists. Type one error is an example the possibility of a false positive. A type one error exists when one rejects the null hypothesis when in fact it is true; type one errors have a greater likelihood of occurring when a large number of statistical tests are being conducted. TTT Goals and Indicators During the process of the first three years of implementation, a new strategic plan was created and a process was set in place to align the goals and indicators for all of the programs across the CISV organization. In order to provide meaningful data to the organization that has requested this program evaluation, the following data are an analysis of the information gathered through this research project as aligned with the newly identified goals and indicators. These goals and indicators are listed in Table 4 (see appendix G). The request for this program evaluation research project occurred at the 2009 Annual International Meeting. At that point in time, two rounds of TTT had been facilitated throughout the spring and fall of 2008 and spring 2009. It would have been ideal to have had a pretest assessment and then use a posttest assessment as comparison after attending the TTT. Since

52 there was not an opportunity for a pretest, questions within the survey asked the participants to provide a ranking of their attitudes, skills and knowledge as they remember before attending training and then as it is now after attending the training. A comparison of before and after responses are listed in Table 5 (appendix G). Goal 1: Active global citizenship. CISV’s approach to active global citizenship begins with the focus upon peace education. CISV’s Passport document clarifies that peace education provides the attitudes, skills and knowledge needed to become change agents locally and globally. It focuses upon issues that are relevant to all countries and recognizes that peace is much more than the absence of war. “Peace education encourages us to look at a wide range of issues and helps us gain a better understanding of: our own identity within the local and global community; basic human rights, as well as, forms of exploitation and injustice; conflicts and how they can be caused, prevented and resolved; and sustainable solutions for environmental and development issues” (CISV Passport, p. 6). It is for this reason that CISV has identified diversity, human rights, conflict and resolution and sustainable development as the four main content areas of peace education within the organization. These four concepts also define the educational principles upon which programs are developed. Educational methodology within CISV utilizes an experiential approach in order to develop attitudes, skills and knowledge toward active global citizenship. While the educational principles have been in place for quite some time, the new direction of the strategic plan clarified expectations and assisted in the development of new program evaluation materials. The TTT was deemed a useful mechanism for rolling out new initiatives and training trainers on the purpose and use of data driven educational evaluation. The questionnaire sent to TTT participants asked for both qualitative and quantitative information on Goal 1. Questions that related to Goal 1 were directly linked to the indicators within this goal. They included the alignment of training with the CISV educational principles, the approaches and methods used in connection with peace education and experiential learning and the theory, practice and purpose of educational evaluation. The following are a sample of training participant’s open-ended comments relating to Goal 1: “The TTT helped me acquire new skills and knowledge on mapping out action plans and these have come in very handy when re-establishing the leadership training committee of CISV Egypt.”

53 “TTT really contributed to put 'order in my thoughts', define my skills and weaknesses, aspects to improve, what to do in order to improve” “Action plan for personal growth has been a very good discovery for me and above all, I feel I need to pay attention more to the evidence from the evaluation.” “Without a doubt it gave me a lot of confidence to be a trainer, to facilitate sessions and discussions and to look more closely at the training goals I had.” “The thing that really helped me improve was giving and getting feedback.” “Planning on a goal-based system has made the trainings more efficient and effective” “Opened my eyes to new techniques.” “We came, we saw, we conquered.” “It took me from a complete beginner to a novice trainer.” “After TTT I understood what are the ingredients for an effective training session. TTT really inspired me. I learned a process and now I always follow it. I learned a lot about the use of PDPEF, the meaning of goals and indicators and now I can use them to evaluate my training but also I can explain their function to the trainees. Regarding facilitation, it's a skill I lacked a bit.” “How to deliver a session is important, but it is equally important to facilitate reflection and discussion.” “I am impressed by the TTT when it came to educational evaluation and goals! I learned a lot about the importance of that. Also ASK really came alive again at the TTT and now really is a useful tool.” Survey participants were asked to rate their attitudes, skills and knowledge before and after attending the TTT for concepts related to Goal 1. The rating was based upon a five-point scale from poor to excellent. The following data compare the very good and excellent ratings before attending the TTT to those after attending the TTT (see Table 5). • 95.2% responded that their ability to plan and deliver training was very good or excellent after attending the TTT compared to 17% who rated it very good or excellent prior to attending the TTT. • 98.2% responded that their understanding of CISV educational methods and the relevance to peace education was very good or excellent after attending the TTT compared to 16.9% who rated it very good or excellent prior to attending the TTT.

54 • 94.5% stated that their understanding of the theory and practice of educational evaluation was very good or excellent after attending the TTT compared to 15.1% who rated it very good or excellent prior to attending the TTT. • 89.1% stated that their ability to collect and use goal-based evaluation was very good or excellent after attending the TTT (100% if including the good category) compared to 20.6% who rated it very good or excellent prior to attending the TTT. • 97.6% of respondents agreed or strongly agreed that the TTT enhanced training competencies. • 95.2% agreed or strongly agreed that TTT strengthened their ability to design and deliver effective training programs. T-test results were significant at the .001 level in all areas that related to Goal 1. Paired sample t-tests were calculated to compare the mean of ratings before attending the TTT to those ratings after attending the TTT. A significant difference was detected between the before and after ratings on the following survey questions relating to Goal 1: • Relevance to peace education and training (t(164) = 23.40, p<.001) • Theory, practice and purpose of educational evaluation (t(164) = 22.64, p. <.001) • Collection and use of goal-based evaluation (t(164) = 19.07, p. <.001) Participants were able to see the connections between the principles of CISV and the educational approaches and methods. They not only formed a deeper understanding of the concepts but also felt confident in using the related skills and knowledge, as well as the ability to teach them to others. Significant differences were apparent between ratings before attending and after attending a TTT. Participants rated their attitudes, skills and knowledge after participation as significantly higher. Goal 2: Prepare trainees for practical responsibilities. Trainees attending a TTT are CISV volunteers who serve a training role either within their National Association or for their International Committee at the organizational level or the program level. Definitions of roles vary from country to country, committee to committee and program to program. It is for this reason that the curriculum and structure of the TTT has some generic components and some that are very role specific. There are aspects that are knowledge-based and those that are activity- based. The process evolves from theory to practice ending with practical application of specific action plans that have been developed during the TTT.

55 The questionnaire sent to TTT participants asked for both qualitative and quantitative information on Goal 2. Questions that related to Goal 2 were directly linked to the indicators within this goal. They included ratings of attitudes, skills and knowledge related to training competencies, planning and delivering training, roles and responsibilities of training and facilitation. The following are a sample of training participant’s open-ended comments relating to Goal 2: “Gained more confidence and openness” “I think the TTT made me more confident about my training skills, but it also showed me I still had a lot of experience to catch up on.” “Encouragement and inspiration that helps” “I now have a better understanding of what is expected of trainers in CISV and can therefore help my NA develop better quality training and training materials.” Survey participants were asked to rate their attitudes, skills and knowledge before and after attending the TTT for concepts related to Goal 2. The rating was based upon a five-point scale from poor to excellent. The following data compare the very good and excellent ratings before attending the TTT to those after attending the TTT. • 93.3% of participants reported that their proactive and responsible attitude toward training was very good or excellent after attending the TTT and 84.8% reported that it was very good or excellent prior to attending the TTT. Typically, for individuals to take part in a TTT they have been involved in CISV in some capacity for some time and are actively engaged in the philosophy and principles of the organization. • 97.0% reported that their knowledge of their roles and responsibilities were very good or excellent after attending the TTT compared to 23% who rated it very good or excellent prior to attending the TTT. • 92.1% agreed or strongly agreed that the TTT enhanced their ability to fulfill the training responsibilities in their role as trainer • 95.8% agreed or strongly agreed that TTT is a useful way to introduce new initiatives and materials within CISV Paired sample t-tests were calculated to compare the mean of ratings before attending the TTT to those ratings after attending the TTT. A significant difference was observed between before and after ratings for the following survey questions relating to Goal 2:

56 • Plan and deliver effective training (t(164) = 20.65, p<.001 • Knowledge of specific role and responsibilities as trainer (t(164) = 19.00, p. <.001) • Ability to facilitate reflection and discussion (t(164) = 25.14, p. <.001) No significant difference was found comparing the means for before and after attending the TTT for the following item: • Adopt a proactive and responsible attitude toward training (t(164) = 1.90, p. >.05) Participants responses demonstrated that while they may have had a general understanding of the role in their committee position or as a trainer, the TTT provided perspective on the connection to the broader picture in reference to the organization and the direction it is going with the educational principles and strategic plan. The TTT provided tools for further developing their current projects, as well as tools for action planning and evaluation. Significant differences were apparent between ratings before attending and those after attending a TTT. Participants rated their attitudes, skills and knowledge after participating in a TTT significantly higher in regard to preparation for their practical responsibilities. Participants comments suggest that they are in a leadership role within CISV because they have a highly positive attitude about CISV. Goal 3: Growth and development of CISV. One of the areas of focus within the strategic plan focuses upon strengthening chapters and programs while expanding to areas that are not yet developed. There has been an emphasis on continuous improvement within the organization and encouragement for collaboration between programs, chapters and national associations. In addition, there has been a strong push for the development of relationships with like-minded organizations outside of the organization. The questionnaire sent to TTT participants asked for both qualitative and quantitative information on Goal 3. Questions that related to Goal 3 were directly linked to the indicators within this goal. They included ratings of attitudes, skills and knowledge related to promoting common training practices, creating networks across the organization and collaborating on planning by sharing ideas and materials. The following are a sample of training participant’s open-ended comments relating to Goal 3: “It gave me motivation and an opportunity to sit down with a fellow trainee from my NA to plan.”

57 “I think that the TTT gave me a chance to share with others regarding ideas for training sessions.” “No doubt better quality, more commitment of the leaders that enter in the organization, better quality of programmes.” “This all helped in establishing a more solid & quality based training committee in my NA.” “I have already found that the TTT has heightened my awareness about CISV values and how to share and communicate them to leaders.” Survey participants were asked to rate their perception on the impact and purpose of the TTT concepts related to Goal 3. The rating was based upon a five-point scale from strongly disagree to strongly agree. The following data illustrate the percentage of ratings within the agree and strongly agree categories. • 98.2% agreed that the TTT is a useful way to promote common training practices across the organization. • 93.3% believe that the TTT is a useful means to share ideas and materials, as well as, collaborate on planning. • 92.8% believe that the TTT is a useful way to create networks across the organization. • 29.2% feel a part of a network of trainers. Participants agreed that the TTT was beneficial in bringing people from across the organization together in order to share perspective, ideas and materials. It was also an opportunity for people across chapters, national associations and committees to collaborate. They commented that it was good to hear the commitment to collaborating with like-minded organizations within another venue and through another source. Goal 4: Effective interaction in diverse environments. The statement of educational purpose from the CISV Passport, “CISV educates and inspires action for a more just and peaceful world,” comes to life in the structure of the educational programs. CISV programs bring diverse groups of individuals together for an extended period of time in order to share themselves and learn about others. In order for large groups of unique individuals to come together with this purpose and within this structure, the organizers must have many things in place to create a climate for positive growth and respond to issues that might arise. The TTT

58 provides tools that address the attitudes, skills and knowledge necessary for facilitating dynamic programs. The questionnaire sent to TTT participants asked for both qualitative and quantitative information on Goal 4. Questions that related to Goal 4 were directly linked to the indicators within this goal. They included ratings of attitudes, skills and knowledge related to acting inclusively and with an open mind, and mapping out personal action plans for future growth. The following are a sample of training participant’s open-ended comments relating to Goal 4: “I got a new insight in to leadership styles and how to use them.” “liked the part about personal growth” “think more activity can be included to help trainers develop their ability to be inclusive and open-minded” “I understood the importance of different leadership styles in a group and this contributed to better the relations with my co-trainers and open my mind, trust the others and empowering them instead of doing thing by myself all the time.” “I got another persons perspective on all aspects of training which helped me improvise and understand different case scenarios. For ex: the cultural difference and the economic difference can make a lot of difference in attitudes of the participants.” “It seems that young CISV Vietnam makes me sometimes feel I am the best. Being at TTT, listening so many other points of view from international participants who have more than 40 years experience and still so open-minded and humble in the way they listen and share. It reminds me that no matter what how much skills, knowledge you have, CISV need your attitude and your willingness for the better community but not only for yourself or the Controllers. I always aware of that ONLY YOU can control, manage and lead your attitude. Skills and knowledge you can learn soon or later” Survey participants were asked to rate their attitudes, skills and knowledge before and after attending the TTT for concepts related to Goal 4. The rating was based upon a five-point scale from poor to excellent. The following data compare the very good and excellent ratings before attending the TTT to those after attending the TTT. • 98.8% reported that their ability to facilitate reflection and discussion was very good or excellent after attending the TTT compared to 10.9% who rated it very good or excellent prior to attending the TTT.

59 • 94% reported that their willingness to map out a personal action plan for personal growth was very good or excellent after attending the TTT compared to 75.1% who rated it very good or excellent prior to attending the TTT. • 98.2% reported that their willingness to act inclusively and with an open mind was very good or excellent after attending the TTT compared to 58.8% who rated it very good or excellent prior to attending the TTT. Paired sample t-tests were calculated to compare the mean of ratings before attending the TTT to those ratings after attending the TTT. A significant increase was from before to after was found for the following survey questions relating to Goal 4: • Act inclusively and with an open mind (t(164) = 12.70, p<.001 • Map out personal action plan for future growth (t(164) = 4.06, p. <.001) Participants acknowledged that while there has been content related to self-awareness, group dynamics and conflict resolution for quite some time, it was beneficial to see the process of creating an effective climate that addresses these concepts in a transparent manner. Participants were able to practice skills in real life situations and share ideas, materials and perspective with individuals from across the organization. Significant differences were apparent between ratings before attending and those after attending a TTT. Participants’ rated their attitudes, skills and knowledge after participating in a TTT significantly higher in regard interacting effectively and appropriately in diverse environments. Interviews Leader preparation has evolved throughout the past sixty years from a general orientation describing the roles and responsibilities to a multi-level leadership training process. The focus upon who receives this preparation has changed as well. Initially, CISV was preparing the adults who would be chaperones to the international programs. Currently, the perspective is that anyone in a leadership role within CISV should have training. Those individuals who accompany youth to programs continue to be a part of this process. The definition is expanded to include others in leadership roles. Their title of delegation leader goes far beyond chaperone. These individuals are temporary legal guardians. These individuals also are part of the leadership of the program once the program begins. In addition, there are many in leadership roles who keep CISV alive through daily operations, organizational development and program planning at the chapter, national and international levels.

60 As the organization continued to grow in the number of chapters and also in types of programs, the need for more structure in training became apparent. SOLAR was the first attempt at providing that structure. SOLAR is an acronym for Stimulative Orientation Towards Life- Long Learning On Altruistic Ways of Responding. SOLAR was a gathering of individuals from around the globe who were responsible for leader preparation within their national association. It formalized national leadership training to provide specific content that would prepare leaders to effectively participate before, during and after selected programs. The content included subjects such as: cultural awareness, conflict resolution and communication strategies. In addition, there were workshops devoted to delegation preparation and management of the ever- changing legal documents necessary for international travel with youth. An analysis of training performed by the International Leadership Training Committee (ILTC) showed that a lot of training was occurring throughout the organization. There was program specific training for program leaders. There was committee specific training for committee leaders and members. There was chapter development training for chapter leadership. These trainings were occurring at the chapter level, national level and international level. The issue was that there was neither means of quality control nor an understanding of what was actually occurring at these trainings. The need for accountability was apparent. Also apparent was the need to somehow share the expectations of the organization to ensure that all the trainings that were occurring all over the world were aligned with the principles, approaches and methods which lay at the foundation of CISV. The idea of a multi-level approach to training was formed, and so began the TTT. To learn more about how CISV leaders view the effectiveness of the TTT training process, I conducted interviews with members of the international office, members of the expanded executive committee and members of the international pool of trainers. In total, twenty-two people were interviewed over the course of twelve months. Information received ranged from historical information to first impressions of the TTT to hopes for future implementation. These interviews were audiotaped, and then transcribed. I coded the responses through a process of reading the transcripts, writing notes in the margins on the ideas and themes that were emerging, then re-reading and re-noting to link ideas together. This process continued until I felt that a coherent “story line” could be presented from the interviews.

61 The Need for TTT The need for a TTT was widely accepted throughout CISV by organizational members, program chairs, national and international trustees. Members from all committees at the international, national and local levels, as well as members of national associations, have participated in a TTT during the first three years of implementation. Interviewees saw the TTT as a means of not only clarifying the expectations of training, but also improving the quality of training across the organization. TTT was seen as a means of quality control and accountability. “The TTT is important to help professionalize those who are delivering training.” “It is really good evidence of the positive direction that CISV is heading and that high quality advances really can be achieved.” “In my experience, I have seen that many people know what they are talking about but don’t necessarily know how to teach it. TTT is very necessary.” I thought it would be too theoretical but it was a great balance of how we do things in CISV and why.” I am a big fan of more accountability and ensuring the quality of training within CISV. The TTT is a tool to make that happen.” “TTT is a means for sharing best practices and encouraging people to use best practices in CISV training.” “As CISV volunteers, we know a lot about CISV and our experiences in programs but TTT is good to give training in the training processes and methodologies in how to give effective training.” TTT Connections to Current Role in CISV Each of the interviewees saw a direct connection between the TTT and their current leadership role in CISV either as a committee chair or officer at the local, national or international level. One concern noted was that prior to attending a TTT they thought the content would be too theoretical and not for practical use. Another concern of the interviewees was that the TTT was going to force each of the trainings occurring across the organization to be exactly the same. Feedback from the interviews showed otherwise. Since the TTT was designed to teach more about the “how” of training and not so much the “what” of training, the interviewees noted that program trainers were able to teach their content with knowledge and skills regarding to “best practices” in leadership training. Interviewees also acknowledged that

62 trainers were strongly encouraged to adapt the concepts of the TTT within their own style of teaching and learning. TTT participants and interviewees also realized that the TTT was not only teaching the concepts relating to the educational approach and methods but also modeling those concepts. The facilitators of the TTT needed to constantly operate at two levels. They were teaching the concepts and also teaching how to teach those concepts. “The TTT gives more tools to develop an actual strategy that can be specific to my committee and the people they will then be working with.” “TTT is person-oriented. It helps me to develop as a more effective trainer. I then need to translate that to how I work with my committee and make it specific for my program.” “It clearly helps improve the quality of our training staff and that then assists in leading the chapter in growth and development.” “I needed an effective way to transmit my knowledge to others. The TTT did that.” “The better the TTT and the more people who are TTT trained, the better we will improve our programs.” “Juniors are constantly delivering training and running activities. TTT gives guidance and information on how to improve the quality of training and how to reach our goals positively that then advances our leadership.” Implementation Issues Related to the Certificate While the need for the TTT was widely accepted, the means by which it would be implemented was a matter of debate. The train the trainer process was given high priority to such a degree that it was mandated that all those with a training role in CISV were required to attend a TTT. This was to be monitored by a certificate system. Communication and miscommunication are issues in any large organization, especially multi-national organizations. The large number of languages and cultures opens the door for issues in between languages and issues of interpretation within cultures. “Perceptions and rumors about why we were doing things the way we were doing them often leads to misunderstandings. I guess there were not enough people consistently explaining the reasoning. Communication and roll-out are always important in an organization like ours and we are always trying to learn how to do it better.”

63 “It was firmly believed from the beginning that there should be some from of recognition some credential that a person was TTT trained. It is CISV’s way of providing personal and professional development.” “The certificate is a means of acknowledging individuals for attending training. It is also a way to see how many from each program or national association have participated.” Some of those who had been training in CISV for an extensive period of time took offense that they now had to be certified in order to provide the voluntary service that they had been providing for the past ten to fifteen years. “I liked the TTT. I didn’t like that it was mandated, that I was forced to go.” “I would have gone anyway and I don’t need a certificate to show that I went.” “I have been a trainer for CISV for ten years and I am a trainer in my professional life. I spend a lot of time giving my volunteer service. This takes time away.” It is not clear as to whether the attendance at the TTT had an impact on these perspectives but interviewees were impressed by how quickly the TTT has permeated through the organization. “TTT grew so quickly, I don’t think anyone expected it to happen so fast. Despite all the initial murmurings, people love it and want it.” Some of those who had been training for an extensive period of time saw this certificate process as a means of continued professional development. “I have been a trainer for CISV for a long time and I am a trainer in my professional life. I realize the need for professional development and I appreciate that CISV has this to offer. The certificate is a nice way to show an appreciation of my attendance.” Responses from the IPT indicated that they feel the curriculum is comprehensive and well designed. It is different than anything that has been introduced through CISV to this point. The structure of the TTT provided the flexibility for the IPT to differentiate sufficiently for those who were veteran trainers as well as those who were new to their role. “The Writeshop 2007 was a wonderful and productive experience. We managed to put together a comprehensive curriculum that not only created dynamic introductions to the training topics but also aligned with the content of the programs and committees.” “The TTT is thorough and provides a balance of theoretical and practical information. It is a great model of building upon attitudes, skills and knowledge through experiential learning.”

64 “The flow of the TTT works to meet the objectives and meet the needs of all participants.” The original plan stated that TTT participants would receive a three-year certificate as a result of completion of the TTT. An issue that arose during the first years of implementation was in regard to the type of certificate to be offered. Should this certificate system be a certificate of attendance or a certificate of achievement? Some believe that the certificate of attendance is sufficient. If those attending the TTT have been selected to participate and if the training is of high quality, then it is providing the tools necessary to accomplish the task in a volunteer organization. Others feel that if CISV is aiming at higher quality, merely attending training does not mean that you have the skills to train. Therefore, trainees should demonstrate their training competencies while attending the TTT and a pass/fail system should be implemented. According to the online survey, 40.6% were in favor of a certificate of completion while 59.4% favored a certificate of achievement. The following is a sample of opinions expressed by those in favor of a certificate of attendance: “I believe it is very difficult to judge the performance and achievement of trainers. The problem in CISV is that we work with volunteers, so who is going to say that people are not good. It's a super difficult question, but I think for now you can focus on attendance and completing. Also because it means taking part of an international network of trainer and having the support.” “I think if you attend and complete the TTT, you should be certified unless your participation was lacking or unacceptable.” “The TTT is packed with a lot of information in very few days. For those who have not done a lot of training before, it can be helpful to let the information sink in some more. Therefore, a passing grade during the TTT could be too early.” “We are volunteers and I understand that people might have different means or styles in delivering trainings and evaluation, however basing the awarding of a TTT certificate on performance and achievement of a passing grade, seems a bit too strict.” The following is a sample of opinions expressed by those in favor of a certificate of achievement:

65 “Attending the program does not mean that a person is qualified to conduct training. A trainer must be a 'role model' for the participants. The trainers must demonstrate how to handle participant questions and answers, motivate participants, how to handle class issues and conflicts that may arise, etc. A facilitator/trainer should not only have the appropriate knowledge and skill, they also need to have and show a positive attitude and outlook.” “I am not sure there should be a test, but perhaps some kind of something to be made of the feedback. Solely attending the TTT doesn't automatically make you a good trainer, even though it might help you improve.” “I believe we as a organization need this to further make us believable and trustworthy. Important to have pass/fail at least, but not grades, it’s not necessary.” “I think that someone attending TTT have to prove that he learned how to be a good trainer and learning does not come automatically with attendance. To ensure quality we have to train trainers but also monitor their performance.” Responses from the IPT were also mixed depending upon the perspective from which they were viewing the situation. From a logistical perspective, most agreed providing a thorough and meaningful evaluation of each participant would be very difficult within the current structure. From an educational perspective, providing a certificate based upon achievement and merit would be good practice. “It would be very hard and time consuming to create a certificate of merit since it would require an in-depth evaluation of trainees. There really isn’t even enough time to make personal evaluations with constructive feedback.” “I believe it should be a certificate of merit. Attendance is just part of training. We need to know that our participants are fully prepared (consider the ASK) to handle the training responsibilities after the TTT.” “I don’t think we need a test at the end of the TTT but maybe using the objectives and indicators to honest self and peer evaluations could be a good way to measure if participants are really leaving as certified trainers.” Implementation Issues Related to Logistics Other issues relating to the implementation of the TTT were in regard to logistics. CISV already had a regional system in place called the Regional Training Forums (RTF). These were

66 opportunities for a variety of trainings to be held in regional sites and open to anyone from that region. Regional Training Forums rotated between chapters within regions from year to year. It made sense on paper that the TTT could be easily incorporated into the schedule of the RTF. The coordinators of the RTF worked with local chapters to find a training site with food, lodging and materials. The Organizational Development Committee (ODC) organized the RTFs. The TTT was organized by the Education Department. Communication and collaboration were vital. The application process. The application system for the TTT was structured in a way that applicants were to be sent by a chapter, national association or committee due to the training mandate that they would have with that entity. Paperwork included typical registration materials and also paperwork specific to the position that they held. These applications were meant to be reviewed by the Education Department. Registration for the RTF had always been a central system that flowed through the ODC. During the first year of implementation, individuals sent all of the registration and application materials to ODC. This created delays in processing applications since the paperwork needed to be processed through ODC for registration and then sent to the Education Department for application. “I know the intention was for the trainers to communicate with the participants prior to the TTT. This often didn’t happen especially when the trainers didn’t get a roster until training began.” “For those of us who were sending people to TTT each year, it was confusing to see a different application process each year.” “It took the first year of implementation to work through the kinks of several logistical issues.” Responses from the IPT indicated that there was much confusion during year one of the application process. While the plan was clearly delineated for the education department, the process was not clearly articulated to the ODC. Those coordinating the Regional Training Forums handled the TTT registration/application in the same manner that they handled each of the other groups attending the RTF. “We had a plan and then adjusted the plan. The process was clarified and streamlined for year two and three. I think that by really defining the target group, the selection will take place within the committee and the process will be much more smooth.”

67 The daily schedule. The TTT was designed to be four full days. The RTF was commonly three and a half. The RTF had several different types of program specific training sessions occurring simultaneously. Oftentimes common sessions were scheduled each day for all of the participants to come together. The TTT was designed to be an independent stand-alone process. CISV is personal and interpersonal, and it was obvious during year one that attendance at the common sessions was expected. As a result, some interviewees felt that the four full days of training for TTT were being compromised. “It was interesting to watch the dynamic of the groups. When the common sessions were about to begin, all those in other trainings automatically expected the TTT to attend and those attending the TTT looked like they either felt excluded or like they were being rude for not attending.” “During the first year, the RTF planners made arrangements for travel to and from the site, airport pick-ups, meals and lodging for a large group of people, it was just assumed that TTT would be included.” Responses from the IPT indicated that there was frustration and confusion during year one for the organizers. However, they also indicated that conversations with participants afterwards shared that any of these issues were not apparent to the trainees. “It is always difficult to match the schedule of the RTF and most of the time the TTT had to be cut by at least half a day.” “The RTF coordinators do an amazing job in making all of the arrangements for the venue. Once it was clear the TTT needed to arrive earlier and stay later, these issues seemed to be resolved.” Opportunities for Networking One other outcome of the TTT was an opportunity for individuals across the organization to gather together and collaborate. Individuals from different committees, programs and national associations could come together and share the ideas, activities and materials that they have used within their corner of the organization. Individuals could plan common activities that could be used across programs. Interviewees commented that the collaboration and networking while at the TTT were extremely beneficial. However, the participants did not feel like there was an avenue to maintain this networking contact. The online survey demonstrated that only 27.2% of the responders agreed or strongly agreed as to feeling a part of a network of certified trainers.

68 “The TTT is a great way to bring people together to collaborate. We need to find a way to continue that communication.” “I would very much like to see the TTT or the trained trainers become a closer network and at the moment it is a little difficult because of the scale and the lack of a networking opportunity.” “In order to build a network, people attending the TTT should cover all the roles of trainers and there should be networks between trainers of the same programs in different national associations. TTT has a network potential that hasn’t been exploited yet.” Interviewees demonstrated the power of ongoing dialogue through networking. This networking, along with many other conversations going on within the organization, highlighted how unique training within each program had become. The training and preparation had evolved to match the growth and evolution of each program. This awareness brought forth the idea that while it was important to maintain the unique flavor of each program, it was as important to ensure that they were all still aligned with the educational principles and approaches of CISV in general. This realization brought about the idea to hold the Writeshop 2010. The purpose of this meeting would be to gather representatives from all aspects of CISV in order to create common goals and indicators that would be specific enough to align with the educational principles yet general enough for each program to maintain its individuality. Perceived Limitations of TTT Data collection for both the online survey and the interviews requested constructive criticism regarding the implementation and operations of the TTT. Prompts requested both positive and negative feedback from these participants. The following are comments in response to perceived limitations of the TTT. Comments include issues related to application, levels of experience, training materials and certification. “It would be nice if the trainers sent things to read beforehand so we can get to the training with a big picture of what we will be seeing. For example, ‘we recommend to read the CISV passport before arriving’ or something like that so we are all on the same page.” “While the TTT was brilliant, I think it lacked sufficient materials to take home and the materials that I received were different from those received by other participants at other trainings even in the same year.”

69 “The material could be better. Maybe with a binder or notebook format.” “The range of experiences at my TTT was very large. Those with a lot of training experience need something different than those who are just beginning. There needs to be different levels for different audiences.” “The one I attended had many people who will not be doing training. I think that selecting the target audience is important so the focus is the same for everyone. I think it is important for everyone involved in CISV to know the tools but there are other places to get that information for people who are very new to CISV.” “I was hoping that once someone attended a TTT and was certified, they could then go and facilitate a TTT in their NA or committee. While we can share the ideas and information, we are not allowed to certify others. I was hoping we could” “I believe that for people who have been training for CISV for some time part of it was a bit too repetitive. However, as I usually say and think, we always learn something in any training.” “This is a volunteer organization. We should not get too academic and theoretical. I think it is too much.” Perceived Strengths The following are comments received in response to prompts requesting perceived strengths of the TTT. Strength areas identified include the curriculum, the focus, and the dynamics of the experience and the strength of the facilitators. “TTT empowers trainers and provides them with a strong curriculum that can easily be replicated to any situation and training environment even outside of CISV.” “We had great trainers who were really inspiring and professional. Good combination of group and team work on one hand and self-reflection on the other hand.” TTT is a great opportunity to met other trainers from far-flung parts of the organization and learn from their best practices as well as give and receive support and suggestions. We are developing trainer competencies. TTT extracts you from your situation and gets you to specifically think about your role and how you can better your own training.” “ I loved that it was a learning by doing approach kind of training. We had to actually apply what we were being trained on.” “It was informative, motivational, inspirational and fun.”

70 “I think it is a great place where trainers can meet face-to face, exchange plans and methods and give constructive feedback. It is a good way to get to know the new tools of CISV.” “I think the TTT strength is in the incorporation of educational, leadership and facilitation theory and practice. It is important to the professionalization and sustainability of our organization to have volunteers in leadership positions understand the theory and have the tools to lead an international educational organization.” Future Directions of CISV When asked about the future directions of CISV, responses often fell into several categories including: creating more opportunities to attend the TTT, creating a longer course, creating an advanced level TTT. A number of comments addressed the need to offer more TTTs during the year. The current system provided TTTs to be held at each of the RTFs once per year. This meant that there would be three or four TTTs held each year. While this structure worked well for participants from chapters close to the chosen location, it was difficult for participants from distant chapters. “Travelling to an RTF is difficult for participants from Australia. Everything is far and travel is very expensive. The rotation between national associations and chapters will hopefully address this issue. We can look ahead through the three-year cycle and wait for a location that is most reasonable.” Numerous comments related to having TTT on request. This would specifically benefit areas of a region where there were a large number of chapters, such as Scandinavia. It would also be beneficial for very large national associations such as: Canada, the United States and Sweden, who would be able to fill a course completely with members of their country. “Limiting to just RTF is a problem. It is very expensive to send people to TTT across the continent. Not allowing TTT to be hosted by national associations is a major concern.” The TTT was designed to take place over four full days, granted that it took some time to work out the scheduling issues with the RTF. The days of the course were structured with workshops and practicum experiences. A large number of respondents requested that the TTT be held for more than four days. Some requested a five-day course and some even suggested up to ten days.

71 “The experience was very beneficial and needed. This is not enough time for participants to show what they learned. It is not possible for the trainers to ascertain each participant’s knowledge and skill in such a short time. It would be beneficial if each participant was given ample time to practice in order to showcase their performance and abilities.” “There was so much to get through in a four day course. It was not until I arrived home that I could reflect on what I had learned and still needed to know.” I think it should be at least five days. Other training might just share knowledge. This training really needs time for practice.” “Maybe a little bit longer (one half day or one day more) to have the time to do all the sessions, share our experiences, practice the skills and assimilate all the new things the trainers bring to training. It is a lot in such a short time.” In addition to requesting a longer course, many also requested an advanced course. “It would be great that once you participated in a TTT you can then go to an advanced course where you can also share your experiences with other trainers and maybe focus on developing sessions which can then be practiced and discussed.” “Maybe everything has to start from a general and common TTT and then we could think about new TTT for experienced trainers, for those with certificates etc. more specifics, going deeper through the educational purpose and basically developing better tools to advance CISV as an educational organization. We could even offer those trainings to other organizations in the future.” Feedback from the TTT showed that there was an impact upon attitudes, skills and knowledge within the training of the organization. Interviewees have received feedback from their colleagues that they gained immensely by attending the TTT and left wanting even more. “The TTT has grown so quickly I don’t think anyone expected it to grow so fast. It is not only requested at every RTF but there are more and more requests for TTT outside the RTF structure.” “I believe the TTT is an excellent space for intensive, professional level training of CISV facilitators. It challenges CISV to think about training in an objective and professional way. It has the potential to breed new creative practices and enrich the organization tremendously.”

72 “I believe we have an excellent foundation in place. We have the concept and two years of experience. We have a curriculum, a pool of expert trainers a delivery forum in the RTF. We have a library, an intranet and other communication tools to enable effective preparation, follow-up and sharing of best practices. TTT has very quickly gained a good profile and judging from requests, people see the need and benefits of it in terms of training quality and consistency.” Feedback from the International Pool of Trainers Research has been an important aspect of CISV since its inception. CISV is an educational organization and as such realizes that research and data collection are only meaningful if the organization responds or reacts in some way to the information gained. For this reason, this section will demonstrate the means by which the IPT has reacted and responded to a variety of data collected throughout the three years of implementation. Some of the responses came through conversations following an analysis of the feedback from the TTT participants. Much of the formal response to structures within the TTT occurred at the IPT meeting in Copenhagen in January 2011. Five members of the IPT and the education officer were present throughout this meeting. In addition, the liaison from the executive committee to the education department was present for one of the days of the meeting. Two IPT members joined briefly through a Skype discussion and two members interacted through email and the Google document that was generated throughout the meeting. Background on the 2011 IPT Meeting Before presenting the feedback from the IPT meeting in January 2011, I present some background on the events that led up to this meeting. Writeshop 2007 was the kickoff to the Train the Trainers course. This is the meeting that created the curriculum and structure for the implementation of the TTT. The original intention was to use the first round of implementation during Spring 2008 as a pilot for this new endeavor. The IPT would then gather together to analyze feedback and observations in order to make any necessary changes and complete some of the tasks that were left incomplete. Due to budgetary constraints, a face-to-face meeting was not possible. Information was shared and conversations were held via email and Skype. The international composition of the IPT posed some scheduling complications since there were trainers from Asia, Europe, Canada, Mexico and the eastern part of the United States. Work schedules, availability and time zones complicated finding common planning times.

73 Conversations occurred between different members of the IPT at different times but it was quite difficult for all members to participate at the same time. Email and Skype conversations were the means of interaction throughout 2008, 2009 and 2010. Except for the opportunities for partner IPT members to train together at a TTT, there were not opportunities for face-to-face meetings. A variety of ongoing tasks were required throughout the year. Following the facilitation of the TTT, trainers put together a report based upon the feedback of participants and observations of the trainers. The IPT would then discuss any needed adjustments. Following the Writeshop 2010, IPT worked together to align the TTT with the newly identified common goals and indicators. In addition, the IPT created a system to increase the number of IPT members. The six original members were asked to commit to a three year plan and then given the option to continue with IPT as a full member, continue on an as- needed basis, or become a member of the IPT alumni. The plan for IPT sustainability was to add on new members each year for the first three years of implementation and then as needed thereafter. Some of the contact between the IPT during 2008 related to reviewing applications and creating a plan for observation. Finalists were invited to co-train during Spring 2009. Following this application process, two applicants were invited as members of the IPT. Unfortunately, one of the original members had to step down. This brought the current number of active IPT members to seven. Applications were also taken during Fall 2009 and some were invited to co-train during Spring 2010. Following this process four additional members were added to the IPT. The TTT was in the spotlight of the Annual International Meeting in Berlin August 2010 for a number of reasons. IPT members assisted in the presentation of the Education Department report. The IPT presented its annual report and since this research project was requested during the AIM one year earlier, a preliminary report of the results was presented. In addition, IPT members assisted in facilitating presentations throughout the conference. Toward the end of the AIM, the members of the IPT put forth a strong case for the need of a face-to-face meeting of the IPT members. The proposal was accepted and funds were allocated from the new budget. The IPT meeting was scheduled for January 2011. As a means of maintaining a frugal budget, the meeting was held in Copenhagen. This location was chosen since there would be easy access for members travelling from many points. The national office of Denmark CISV was available for the meetings and sufficient home stays were offered eliminating the need for

74 hotel and meeting room costs. I attended this meeting as a member of the international pool of trainers and as the researcher conducting the TTT program evaluation. Participants were fully aware that the information from this meeting would be used to inform the research project. Participants provided consent. A Google-doc was used throughout this meeting. It served two purposes: an on-going documentation of actions taken by this team, and a means by which IPT members not present could be updated and provide input. As active members of this participatory evaluation, members alternated documenting the minutes within this Google-doc. The agenda included a variety of items related to curriculum, certification and future planning. An ongoing and continuous review of the curriculum especially in light of the newly identified common goals and indicators was a high priority. A discussion regarding a clarification of the roles and responsibilities of members of the IPT as well as the newly formed IPT alumni was in order. Another high priority was creating a finalized plan for recertification since those certified in the first year of implementation would need recertification by the end of 2011. Some of the items for discussion in future planning included requests for an advanced TTT course, the possibility of IPT members facilitating other meetings within CISV, and offering the TTT course outside the boundaries of CISV. The TTT was created within a constructivist experiential perspective. The training was designed to be specific enough to provide a comprehensive educational experience aligned with the CISV educational principles, yet flexible so that each IPT member could utilize his or her strengths and teaching style. The original plan was to regroup after the first year of implementation to analyze the feedback and observations in order to make any necessary adjustments. This did not happen after year one or after year two of implementation. It was apparent after year three that each facilitator had taken sufficient license to incorporate his/her own style of facilitation and teaching to such a degree that each TTT evolved in different ways depending upon the facilitator. The IPT meeting provided an opportunity to analyze the impact of these different styles and create a plan for the future. In addition, it was necessary to finalize the IPT interpretation of the common goals and indicators that were created at the Writeshop 2010. Summary of the Discussions on TTT During IPT 2011 Extensive discussion and brainstorming occurred during day one and two of the IPT 2011 Copenhagen meeting in relation to the TTT curriculum. A mapping technique was used to

75 demonstrate the alignment of the TTT curriculum with the newly identified common goals and indicators. This mapping activity led the IPT members to then create an organized dynamic curriculum document to be used as a guide for facilitating future TTTs. This document included an orientation for new IPT members discussing the history and on-going processes of the IPT. A detailed analysis of the goals and indicators aligned with each specific content area provided a foundation for facilitating future TTTs. Examples of activities and activity documentation were provided for each session of the TTT schedule. A comprehensive plan for a unified booklet for TTT participants was drafted during IPT 2011. This had been part of the original plan designed at the Writeshop 2007. This item had been tabled in 2007 with an understanding that it would be completed when the IPT met to review the pilot year. Participants had been receiving materials at the discretion of the facilitators. Some of the materials were common to the trainings but other materials were unique to the facilitator. This booklet would ensure that all participants received common materials that then could be enhanced at the discretion of the facilitator. The booklet was designed to be specific to the content of each element of the training yet flexible for each trainer to adapt to his or her own style. It was also designed so that the participants could make use of the booklet in their own way. The booklet has an introductory section for each element. It is then followed by the theoretical foundation that connects the element to the educational principles. An extensive discussion related to the certification system took place. Information was reviewed from feedback from a variety of venues. The online survey for this research project identified that 59.4% of respondents preferred a certificate of achievement while 40.6% preferred a certificate of attendance. Additional information was received from members of the international office and the expanded executive committee. It was decided by the group that while there is merit to using a certificate of achievement, it was not feasible for immediate incorporation. The discussion included issues related to volunteerism, timing, expertise and quality. How would a certificate of achievement impact the voluntary aspect of the role of trainer? Is it possible to assess achievement during a four-day training or is there a need for an evaluation sometime after the completion of the TTT? If the assessment were at a later time, how would that be organized? And finally, what measures would be used to evaluate successful achievement of the goals of the TTT? It was determined that a certificate of attendance would suffice at this time with quality control measures in place. Attendance at a TTT is in part as a

76 result of a selection and application process. Facilitators of the IPT would use informal assessment strategies to identify strengths and areas of need in order to provide feedback to the participants. If any serious issues arose, IPT members would follow CISV protocol. Recertification was an area that needed to be finalized. Those participants who attended the TTT in 2008 would need to be recertified by the end of 2011. The procedures for recertification would need to be presented to the board of trustees at AIM 2011. After extensive discussion and breakouts into small work groups, an online recertification process was created. This process would include: a training log documenting past training, sample activities from training, a reflection on the current training in relation to attending the TTT, an updated action plan and several options for documenting new learning. The discussions in the area of future planning were topic starters intended for ongoing collaboration. These topics included: the vision of what an advanced TTT might include; the possibilities that offering TTT outside CISV might hold; and, the need to define roles for IPT alumni and active members who might take on administrative responsibilities.

77 Chapter 6 Key Findings, Recommendations, and Conclusions This research project is a program evaluation of the Train the Trainer (TTT) process as utilized by Children’s International Summer Villages (CISV) for the purpose of leadership training. The intent of this research was to study the impact of the Train the Trainer process by gaining a deeper understanding of the experiences and perspectives of those who have participated in the TTT process within Children’s International Summer Villages. In addition, the study investigated the scope of the impact of the TTT process, not only on the direct participants, but also upon the organization as a whole. The specific purpose of this study was two-fold: to gain an understanding of the degree to which the TTT has met its goals of implementation to date; and to study the impact of the TTT process. The beginning of this chapter reviews the key findings of the study. These key findings are based upon stakeholders’ perceptions of the defined purpose of the TTT, the common goals and indicators, as well as, the needs of the organization. Second, there will be a section on recommendations. These recommendations are based upon an analysis of quantitative and qualitative data from a variety of sources within CISV. Third, there will be a discussion of possible limitations to this evaluation design. And finally, there will be a discussion regarding future steps in the continuing evaluation of the TTT. Summary of Key Findings Data gathered through a variety of quantitative and qualitative sources provided information related to numerous categories identified within the needs assessment for this research project. Sixkey categories are discussed, including issues regarding implementation, communication, face-to face interaction, impact, as well as connections to the four guiding principles and the common goals and indicators. Issues Related to Implementation The TTT process was proposed following an extensive strategic planning process and an analysis of the overall operations of the organization. The idea for the TTT was born from the need to share expectations of best practices, high quality and flexibility while maintaining this philosophical connection. The first research question searches to discover the stakeholder’s perceptions of the implementation of the TTT. Kirkpatrick (1994) identifies ten factors that are considered for creating effective training programs including: determining needs, setting

78 objectives, determining subject content, selecting participants, determining schedule, selecting facilities, selecting instructors, selecting materials, coordinating the program and evaluating the program. Several of these factors are associated with the development and design of the program, where as others are associated with logistics, and with running and evaluating the program. Participants who contributed to this program evaluation study gave high marks in the design, scope and flow of the course noting that the variety of instructional methods, practical application and engagement were beneficial to meeting the needs and expectations of the participants. They acknowledged the extent taken to ensure a high quality, well-represented and thorough course of study. Feedback from TTT participants and IPT members praised the work of the RTF coordinators not only for their work in planning the event and making all logistical arrangements but also in their helpfulness during the training as well. Issues arose in regard to registration materials and daily schedules. While the interviewees noted concerns about daily schedules in the early years of the TTT, they did note that over time the TTT organizers had addressed these issues adequately. Interview and TTT participants commented on the need for a cohesive packet of materials. Members of the IPT explained the plan for this packet to be completed after a review of the first pilot year of the TTT. They further explained that while members of the IPT shared materials electronically, each then continued to use individualized materials until a comprehensive format was agreed upon. The IPT established that comprehensive and cohesive form during the IPT meeting at the January 2011 meeting. The Importance of Communication Communication has been an area of focus within CISV since its inception since CISV is an international organization that brings people from different cultures and different parts of the world together. Participants gave the highest marks in the area of communication related to: opportunities to collaborate with others, TTT as a means of introducing new initiatives and materials and opportunities to create networks across the organization. Participants noted that these concepts add greatly to their ability to train effectively within the organization. Participants also noted that while the opportunity for networking is great, the extent to which it is possible has not yet been realized. Participants praised the communication process used in the planning and design of the TTT at the Writeshop 2007. They noted that these aspects became an

79 active part of the design process for the TTT and lessons would strongly emphasize the need to learn about communication needs during training. While the interviewees noted concerns about communication in the registration and planning process in the early years of the TTT, they did note that over time the TTT organizers had addressed these issues adequately The Importance of Face-to-Face Interaction CISV has invested significantly in the implementation of the TTT. Feedback from the online survey and the interviews indicate that while it is expensive to send people to training nothing compares to the learning that takes place during a face-to face interpersonal interaction together in the same place at the same time. As mentioned earlier, participants rated the opportunities for face-to-face collaboration, sharing ideas and materials and networking with the highest marks noting the benefit to them personally and professionally was significant. Participants indicated that three meetings significantly impacted the design, development and implementation of the TTT. These included: the Writeshop 2007 to create the TTT; the Writeshop 2010 to create the common goals; and the IPT meeting 2011 to reorganize and respond to feedback. Participants noted that the process and outcomes of these meetings brought about an extremely high degree of quality work in a relatively short period of time. The process was described as exhausting yet inspiring. Even with the significant improvements in electronic media and the development of online learning mechanisms, this dynamic could not have been replicated. IPT members acknowledged that they did accomplish some work through email and Skype conversations but it did not match the intensity and productivity of a small group of people meeting together for four days. The Impact of the TTT Program on the Participants The TTT is designed to continuously operate on two levels. The members of the IPT are training the participants in the curriculum, and are training the participants to train in these areas as well. This section will discuss the perceptions of the participants relating to the first level of training. The second research question searches to discover the impact of TTT implementation had upon training leaders in the organization. Responses of the training participants and administrative participants were very positive. Participants perceived the course as strong, worthwhile, motivational and inspiring. Program evaluators often refer to Kirkpatrick’s (1994) four levels that represent ways to evaluate programs: reaction, learning, behavior and results. Reaction is often measured

80 immediately following a program and represents the participant’s perception of the overall program. Comments from course evaluations and feedback from committee chairs demonstrate that the reaction to the TTT was relatively strong. There were overwhelmingly positive comments about the trainers, the structure of the course, the experiential approach and the practical application of lessons. Common critiques centered on the need for additional time and the interest in a formal booklet or notebook. While the issue of time is still up for debate, the development of a high quality consistent yet flexible booklet was developed during the IPT meeting in January 2011. Learning relates to the extent to which the training has an impact upon attitudes, skills and knowledge. Qualitative and quantitative data demonstrated significant positive results. Participants stated that while they already do have strong attitudes as demonstrated in their passion for CISV, attending the TTT highlighted the importance of the connection between attitudes, skills and knowledge in training. Participants felt the approach to training is motivational and inspiring. In addition, participants reported a significant impact upon knowledge. While most of the participants have had experience with training in CISV, the TTT enhanced their understanding of the theoretical foundations of what they do and modeled an aligned approach for how to effectively deliver training. Practical application of new learning was woven through the learning process assisting in developing and evaluating skill levels. Information from the online survey demonstrated an increase in skill as rated before and after attending a TTT. Behavior refers to any change in behavior that might have occurred as a result in participating in the training. As mentioned above, the significant skill enhancement reported through both qualitative and quantitative data demonstrated this change in behavior. This skill enhancement was observed within the practicum experiences and reported anecdotally within the online survey and through interviews. Further data can be collected in this area through reports related to the training that they are facilitating following participation in a TTT course. Results refer to any changes in outcomes like production or quality as a result of the training. There are some anecdotal reports stating that attendance at the TTT has improved the quality of their facilitation skills, goal-based planning and evaluation, as well as, development and design of training. This is another area where a second level of evaluation could be used

81 based upon reports of the training that participants are facilitating following attendance in a TTT course. Results from both quantitative and qualitative data demonstrated highly positive responses for enhancing attitudes, skills and knowledge. Responses also indicated that the two levels of training that occur within the TTT model a program design that incorporates attitudes, skills and knowledge across the curriculum. Connections to the Four Guiding Principles The TTT approach was implemented in order to enhance the attitudes, skills and knowledge of the leaders personally and to enrich their abilities to train in these areas. This section will focus on the second level of training – training to train. There are four guiding principles for every Train the Trainer (TTT) workshop. The TTT workshop was designed to: enhance the training competencies of participants; strengthen the ability of participants to design and deliver effective training programs; empower participants to develop a training strategy; and encourage participants to promote the educational principles of CISV through their training programs (Writeshop Report 2008). Responses by participants were highly positive. They indicated that the practical application woven throughout the course was a very appreciated approach to developing training competencies. The focus upon giving and receiving feedback encouraged a climate of engagement and interconnectedness. The participants viewed the structure and design of the course as a model for developing a training strategy in order to design and deliver effective programs. The awareness of the interrelationship between attitudes, skills and knowledge made a large impact upon many participants. The focus upon goal-based planning and evaluation has taken CISV training to the next level in the eyes of many participants. Finally, the continued awareness that everything done in CISV, formal or informal, from program planning to committee work is related back to the educational principles, was reaffirmed for many of the participants. Alignment with the Common Goals and Indicators As a result of many conversations across the organization and through the analysis of feedback from a variety of sources (including the TTT), CISV created a vision of one organization with one plan. While CISV encourages each program to grow in its own unique fashion, members of the organization still need to maintain the connection to the foundational

82 educational principles that guide the organization. For this reason, CISV created common goals and indicators for program and organizational training throughout the organization. The creation of these common goals occurred after the first two years of implementation of the TTT. Data collection for this research project began after the third year of implementation. It was of interest to see whether and how the perceptions of TTT related to the newly developed common goals and indicators. Goal one relates to preparing trainees to use CISV’s approach to active global citizenship. Participants were able to see the connections between the principles of CISV and the educational approaches and methods. They not only formed a deeper understanding of the concepts but also felt confident in using the related skills and knowledge, as well as the ability to teach them to others. Goal two relates to preparing trainees to actively undertake their roles and responsibilities within CISV. Participant’s responses demonstrated that while they may have had a general understanding of the role in their committee position or as a trainer, the TTT provided perspective on the connection to the broader picture in reference to the organization and the direction it is going with the educational principles and strategic plan. The TTT provided tools for further developing their current projects, as well as tools for action planning and evaluation. Goal three relates to preparing trainees to contribute to the growth and development of CISV. Participants agreed that the TTT was beneficial in bringing people from across the organization together in order to share perspective, ideas and materials. It was also an opportunity for people across chapters, national associations and committees to collaborate. The commitment to collaborating with like-minded organizations also encouraged the growth and development of each organization. Goal four relates to preparing trainees to interact effectively in diverse environments. Participants acknowledged that the ability for practical application within an international venue provided opportunities to interact one-on-one, in small or large groups with individuals from diverse cultures, languages, ideas and beliefs. Participants acknowledged that while there has been content related to self-awareness, group dynamics and conflict resolution for quite some time, it was beneficial to see the process of creating an effective climate that addresses these concepts in a transparent manner.

83 Recommendations to CISV Data collection for this research project began following the third year of implementation of the TTT. While some may have been skeptical in the beginning, interest in the TTT has grown. Participants in this program evaluation study noted that TTT has permeated the organization, and that it is connected to all facets of CISV. The acronym TTT is widely known and readily recognized across the organization. The following recommendations are a result of analysis of feedback received through both quantitative and qualitative data. Recommendation 1: Continue TTT The responses from the online survey and from interviews are extremely positive in regard to the design, implementation and the learning that occurs. Participants used words like motivational, inspirational and energizing. It is highly recommended that CISV continue to offer the TTT course. Recommendation 2: Continuous Improvement The TTT is a work in progress and will continue to be improved over time. Each year thus far, the IPT responded to feedback by making the necessary adjustments in order to meet the needs of the participants. New members of IPT will bring new perspectives while current members and IPT alumni continue to stay grounded in the foundation of the TTT purpose. Together they can continue to update with best practice in the pursuit of excellence. Recommendation 3: Mindful Communication Continue to be mindful of effective communication practices. Be aware that miscommunications and misunderstandings can happen in the most unlikely places. As CISV is aware, positive, open interpersonal interaction can resolve most issues. Recommendation 4: Creative New Streams of Financial Support Budget issues are often difficult for non-profit volunteer organizations with fixed resources. The TTT is a different entity within the organization. It would be beneficial to find alternative sources of funding so that the TTT might be self-sustaining and is not competing with other parts of the organization for limited resources. Alternative sources might include grants, partnerships or associations with like-minded organizations. Several comments were made during data gathering about offering the TTT outside the organization. This could be a fee for service opportunity.

84 Recommendation 5: The Certificate The current designation for the certificate is one of attendance. However, there is an observational system in place and to some degree, the trainers would follow CISV procedures for critical concerns. The online survey provided significant data to show that 59.4% of respondents are in favor of a certificate of merit. Initial reactions against the certificate of merit were related to the logistics of management. It is possible to create a system that is reasonable and effective without creating a test. Evaluation tools such as rubrics would provide a guide for the trainers and the trainees. A rubric type system would delineate expectations of the organization and provide a structure for the trainer and trainee to collaboratively evaluate the identified areas in a manner that would not be time intrusive. Recommendation 6: Additional Courses The interest in TTT is great. There is a high interest in additional TTT courses. There will always be a need for the core TTT course due to the continuous turnover of volunteers. As more and more volunteers are certified, there will be a stronger need for continued professional development. While the recertification process can be completed online, some may choose to pursue additional coursework. In addition, there is a strong interest in creating something for the special population of trainers who have significant experience in training both inside and outside of CISV. These additional courses would also provide a stronger marketing strategy for advertising outside CISV. Limitations of the Evaluation Design Although this research project was carefully designed, potential limitations are unavoidable. This section addresses issues related to the limitations within this study. This discussion examines issues related to the design of the study, as well as potential bias related to the researcher or the research participants. The idea for this research project was proposed and accepted following two years of TTT implementation. As such, sufficient data regarding situations prior to the implementation of the TTT were not available. While the organization did have data regarding quantity of training and other demographic information, qualitative data were not yet available. At this point in time there was not an opportunity for a pre-test/post-test design. Bamberger et al. (2006, p. 92) discuss a number of viable strategies in reconstructing baseline data. Using recall is one of those options. This research used the recall strategy for both the online survey and the interview

85 questions in order to gain an understanding of perspectives before and after attendance at a TTT. In the future, it would be beneficial to utilize pretest strategies in order to identify reliable baseline information. This research project was designed as a participatory model and as such attempted to gain as much information from as many participants from a variety of perspectives as possible. As such, a probability sampling technique, such as random sampling, was not utilized. All TTT participants were invited to take part in the online survey. Participation was voluntary and the possibility of selection bias exists. Bamberger et al. (2006, p. 103) states, “Nonparticipants may be different in potentially important ways for those who did participate.” It was the opinion of this researcher and consultants from the university and CISV that using a technique such as random sampling across years of implementation and across regions would significantly reduce the sample size. This could limit the possibility of statistical analysis due to low response rate from that smaller number of respondents. It is often reported that responses to online surveys are typically quite low. This online survey recorded a 66% response rate, which is significantly higher than most often reported. In addition, a triangulation technique was used to confirm interpretations by using multiple data sources across regions and years of implementation through survey and interview data. The design of this study is also open to response bias. Bamberger et al. (2006, p. 335) explain that response bias is a type of research bias where the subject, consciously or subconsciously, gives response that they think that the interviewer wants to hear. Possible reasons for positive response bias relate first to the culture of the organization and then to the relationship of this evaluator to the organization. CISV volunteers typically stay involved because they become passionate about the purpose and vision of the organization. This passion has the potential to impact the responses. According to Mandell (personal communication, January 13, 2010) people in CISV are very concerned about the need to be kind to each other. This happens to such a degree that while it avoids tensions in the personal relationship it might hurt the organization later. Hingorani (personal communication, January 12, 2010) commented that the people in CISV are really nice people. They work hard not to hurt each others’ feelings. Positive response bias may be a result of this view. The ANOVA comparisons across the three years of implementation and across the three regions revealed significant differences in only two areas of the many pairings at hand. This may be a sign of positive response bias as well, as

86 questionnaire ratings were quite positive overall. On the other hand, the passion for CISV has also been associated with the strong desire to see the organization grow and improve. In this way, an honest critical opinion of both positive and negative comments was hopefully expressed. One other concern relating to the online survey participants relates to language. Multiple steps were used in carefully wording the questions. Procedures were used to ensure clarity including checking with individuals from the university and international office and piloting the survey with a variety of individuals. However, the online survey was created and distributed in English, and for a majority of participants English is not their primary language. While most participants would consider themselves fluent in English, there are nuances to the English language that complicate even basic communication, and that likely affected the survey and interview responses. This research project is an insider participatory study. I am not only part of the international pool of trainers (IPT) but also have been an active volunteer in many other capacities for quite a long time. This connection to CISV may have had an impact upon response bias as well as researcher bias. There is the potential that the responses to the online survey and the interview questions were influenced by my relationship with the individuals and the organization. Confidentiality was clarified in the introductory documents of the online survey. Names were not associated with entries. Participants were encouraged to provide thorough constructive feedback whether positive or negative. Interviewees held international positions within the organization. These participants were also encouraged to provide thorough constructive feedback whether positive or negative. It is the opinion of this researcher that these interviewees were interested in productive and constructive data gathering for future growth. Still, the fact that I am strongly connected to CISV likely had an impact on the study participants. There are benefits and limitations for an internal researcher performing a program evaluation. One advantage relates to already knowing and understanding the organization and the issues related to the project of study. Fitzpatrick et al. (2004, p.44) report that internal evaluations are increasing in priority as a research method of choice. They go on to state that the advantages of using an internal evaluator include: knowledge of the program model, familiarity with the stakeholders and knowledge of the history of the organization and the program. A disadvantage to internal evaluation is in the potential for researcher bias. Fitzpatrick et al. (2004

87 p. 416) explain that it is inevitable that an evaluator’s philosophy or personal feelings, ideas and beliefs can bias the evaluation. I have attempted to remain objective throughout the data collection. I have collected a large amount of quantitative and qualitative data. I have maintained contact with consultants from the university and from CISV as part of the participatory nature of this evaluation. I am serious in maintaining personal and professional ethics in my role as program evaluator. Still, my viewpoint is embedded in this evaluation, and I acknowledge that as the evaluator, I made many decisions about how to present the results, and discuss my findings, that were influenced by my insider knowledge of CISV and TTT, as well as my positive experiences within this organization. Future Steps in the Continuing Evaluation of TTT The Train the Trainer course is living project. As with any educational program striving toward continuous improvement, continual adjustments and adaptations are a natural process. It is for this reason that ongoing program evaluation is recommended. In the overall picture, the current program evaluation could be used as a baseline for future comparison. In addition, further data could be obtained for presentation at the conclusion of this current strategic plan. Additional information could be obtained from TTT participants regarding the means to which they have implemented concepts taken from the TTT. The proposal for recertification includes documentation that may be valuable in facilitating this form of data collection. This could be in the form of a collection of stories or actual samples of training sessions. Future evaluation could also implement pre-test/post-test strategies. There may be established measures (in such areas as leadership, training competencies or intercultural competence) that may have reliability and validity information readily available. These measures could be used at the primary level with TTT participants and even at the secondary level with their training audience. Conclusions This study is a program evaluation of the Train the Trainer process as used by Children’s International Summer Villages (CISV). CISV is a multi-national non-profit organization that promotes peace education and cross-cultural friendship. CISV has identified its purpose “to educate and inspire action for a more just and peaceful world (CISV Passport 2009, p. 8). CISV created the TTT process in 2007 in order to: share best practices in leadership training, promote an emphasis on the alignment with CISV’s educational principles and encourage consistency in

88 the training that occurs throughout the organization. The specific purposes of this study were: to gain an understanding of the degree to which the TTT has met its goals of implementation; and to study the impact of the TTT process. This program evaluation was situated within an interpretivist perspective and used a participatory, mixed-method program evaluation format in order to gain a deeper understanding of the experiences of those individuals who have participated in the Train the Trainer model as utilized within CISV. Data gathered through both quantitative and qualitative strategies produced highly positive results. Responses were consistent across regions and throughout years of implementation. Responses indicate that the general perception showed the TTT to be a dynamic process especially in the incorporation of goal-based design and evaluation, as well as the interconnection between attitudes, skills and knowledge. While there may have been some logistical concerns during initial implementation, it appears that many issues were resolved through ongoing, open communication. There was evidence to show that the TTT is fulfilling the guiding principles. There was evidence to show that the TTT is meeting the common goals and indicators. There was evidence to show that the TTT is making a significant impact upon participants in terms of attitudes, skills and knowledge. Respondents have reported the TTT to be informational, motivational, inspirational and energizing. Leadership is personal. Through the TTT, participants learn to better understand themselves as learners and trainers. The TTT enhances training competencies and one’s ability to design and deliver effective training that is goal-based and aligned with CISV’s educational principles. The TTT encourages self-reflection and the development of a personal action plan for future growth. Leadership is interpersonal. Through the TTT, participants learn to better understand their audience as learners in order to adapt and modify to accommodate the needs of learners. The TTT enhances their ability to facilitate reflection and discussion. It encourages the use of multiple methods toward a goal-based evaluation of all aspects of the training. Leadership is transformational. Through the TTT, participants learn to better understand their role within their program or committee, as well as the interconnections to the organization as a whole. The TTT encourages collaboration between participants who represent all aspects of the organization. This collaboration between programs, chapters and national associations

89 creates a network to share ideas, materials and processes and ultimately strengthen those programs, chapters and national associations. The TTT encourages collaboration with like- minded organizations as a means for continued improvement and growth. This collaboration within the organization and with outside groups with then in turn contributes to the growth and development of CISV. Leadership is vital to the growth and continuation of this international peace education organization. Transformational leadership in CISV encourages the development of attitudes, skills and knowledge toward the concept of one organization – one goal. The Train the Trainer process as used by CISV has an impact on leadership and leadership training to educate and inspire action for a more just and peaceful world.

90

Appendices Page

A. Interview Consent ………………………………………………………………………92

B. Online Survey Consent ………………………………………………………………….93

C. Interview Questions. …………………………………………………………………….94

D. IPT Questions ……………………………………………………………………………95

E. Online Survey …………………………………………………………………………...96

F. Charts………………………………………………………………………………..…..99

a. Attitude b. Skills c. Knowledge d. Overall Effect e. Promote Consistency

G. Tables…………………………………………………………………………………...104

91 Appendix A

Department of Educational Leadership 304 McGuffey Hall , Ohio 45056 513-529-6825

My name is Chuck Catania and I am a doctoral candidate in the Department of Educational Leadership at Miami University (located in Oxford Ohio, USA). Children’s International Summer Villages (CISV) has requested an evaluation of the Train the Trainer (TTT) process that has been implemented as a response to the first strategic plan priority “leadership is the key to success.” You are invited to participate in a preliminary investigation for the program evaluation of the TTT. Your participation would involve interaction in a focus group as well as an individual interview. Responses shared during the focus group will be kept confidential by the researchers. However, since responses will be heard by others, the principal investigator cannot guarantee that other participants will also keep answers confidential. Identifying information will be removed and the data will be treated as confidential information. The audiotapes and transcripts will be labeled with random identifying codes. All tapes, transcripts and data will be stored in a secure location for the duration of the project, accessed only by myself or my faculty advisor Dr. Lloyd. The tapes will be destroyed when the transcripts are completed, and the transcripts and data will be destroyed after the data have been analyzed. Signed consent forms will be kept in a separate locked file, and will be separated from the audiotapes/transcripts at all times; identifying information will never be placed into any electronic files at all. Your participation is voluntary and you may withdraw from the session at any time or decline to answer any questions that make you uncomfortable. You will not be asked to do anything that exposes you to risks beyond those of everyday life. The benefit of the study, scientifically, is to inform CISV about the impact of implementing the TTT process. At the end of the study, the researcher will make the general results available to you. The specific results of this study will be written up in my doctoral dissertation. The generalized results may be presented at professional conferences or published in articles describing the results of the research (after review by the CISV director). If you have further questions about the study, please feel free to contact me by e-mail at [email protected]. Questions or concerns may also be directed to the faculty advisor for this dissertation project. (Dr. Sally Lloyd, [email protected]) If you have questions about your rights as a research participant, please call the Office of Advancement of Research and Scholarship at 529-3600 or email: [email protected]. Thank you for your participation. I am very grateful for your assistance and hope that this will be an interesting session for you. You may keep this portion of the page.

Cut at the line, keep the top section and return the bottom section. I agree to participate in the preliminary study for the evaluation of the TTT. I understand my participation is voluntary and that my name will not be associated with my responses. By signing below, I acknowledge my consent and that I am 18 years or older. Participant’s signature ______Date: ______Investigator signature ______Date: ______

92 Appendix B

Department of Educational Leadership 304 McGuffey Hall Oxford, Ohio 45056 513-529-6825

My name is Chuck Catania and I am a doctoral candidate in the Department of Educational Leadership at Miami University (located in Oxford Ohio, USA). Children’s International Summer Villages (CISV) has requested an evaluation of the Train the Trainer (TTT) process that has been implemented as a response to the first strategic plan priority “leadership is the key to success.” You are invited to participate in a research study as a part of the program evaluation of the TTT. Your participation would involve the completion of this online survey. Participating in this study is completely voluntary. Even if you decide to participate now, you may change your mind and stop at any time. You may choose to not answer an individual question or you may skip any section of the survey. . You will not be asked to do anything that exposes you to risks beyond those of everyday life. Identifying information will be removed and the data will be treated as confidential information. The technology department at this University ensures that the computers are updated with the most current software available for virus and spyware protection.

The benefit of the study, scientifically, is to inform CISV about the impact of implementing the TTT process. At the end of the study, the researcher will make the general results available to you. The specific results of this study will be written up in my doctoral dissertation. The generalized results may be presented at professional conferences or published in articles describing the results of the research (after review by the CISV director). If you have further questions about the study, please feel free to contact me by e-mail at [email protected]. Questions or concerns may also be directed to the faculty advisor for this dissertation project. (Dr. Sally Lloyd, [email protected]) If you have questions about your rights as a research participant, please call the Office of Advancement of Research and Scholarship at 529-3600 or email: [email protected]. Thank you for your participation. I am very grateful for your assistance and hope that this will be an interesting session for you. You may keep this portion of the page. By beginning the survey, you acknowledge that you have read this information and agree to participate in this research, with the knowledge that you are free to withdraw your participation at any time without penalty.

93

Appendix C

Interview Questions

• What is your current role in CISV? How long have you been involved and in what other capacities?

• What were your initial opinions regarding the proposal for a TTT?

• Do you see TTT connected to your role? If yes, in what way?

• What information would you like to gain from this TTT program evaluation? (prompt if necessary regarding content, implementation, organizational logistics, utilization)

• What would you like to see for the future of the TTT?

94

Appendix D

IPT Interview Questions

• Comment on the process and outcome of Writeshop 2007

• Comment on the TTT curriculum, schedule and logistics of the TTT

• Comment on the IPT organizational process

• Please provide feedback about the TTT that you have facilitated.

• What is your opinion about an advanced level of TTT? How should it be structured?

• What is your opinion of an online format?

• What would you like to see happen with TTT regarding IPT responsibilities?

95 Appendix E

Online Survey

National Association Years in CISV CISV position at the time of your Chapter Years as a trainer TTT TTT attendance location Years as a trainer prior to Years in that position Age TTT Are you still in that position Gender Profession/Occupation

About you:

1. How many trainings have you given since you attended your TTT?

2. Please rate your willingness regarding the TTT attitudes

1. Poor 2. Fair 3. Good 4. Very Good 5. Excellent

My willingness to: Before TTT After TTT 1 2 3 4 5 1 2 3 4 5 Adopt a proactive and responsible attitude to training Map out a personal action plan for future personal growth Act inclusively and with an open mind Have you any comments regarding any of the above?

3. Please rate your abilities related to the TTT skills

1. Poor 2. Fair 3. Good 4. Very Good 5. Excellent My ability to: Before TTT After TTT 1 2 3 4 5 1 2 3 4 5 Plan and deliver effective training sessions Facilitate reflection and discussion Collect and use evidence for goals-based evaluation of training Have you any comments regarding any of the above?

96 4. Please rate your understanding of the TTT knowledge

1. Poor 2. Fair 3. Good 4. Very Good 5. Excellent

My understanding of: Before TTT After TTT 1 2 3 4 5 1 2 3 4 5 The specific responsibilities to fulfill the role as a trainer The concept of ASK (Attitude, Skills, Knowledge) and how it is relevant to education and training The theory, practice and purpose of educational evaluation Have you any Comments regarding any of the above?

5. What impact has TTT had on you as a trainer? Please rate following statements.

1. Don’t Know 2. Strongly Disagree 3. Disagree 4. Agree 5. Strongly Agree

1 2 3 4 5 TTT enhanced my ability to fulfill the training responsibility for my role in CISV TTT enhanced my training competencies TTT strengthened my ability to design and deliver effective training programs I feel a part of a network of certified trainers?

6. Who sponsored you to attend TTT training?

7. What expectation do they have of you now that you have completed the training?

8. How will the training in your NA or for your committee benefit as a result of your TTT training?

97 9. An additional purpose of TTT is to promote consistency of training across the organization Please rate following statements.

1. Poor 2. Fair 3. Good 4. Very Good 5. Excellent

1 2 3 4 5 TTT is a useful way to promote common training practices across the organization TTT is a useful way to introduce new initiatives within CISV TTT is a useful way to create networks across the organization TTT is a useful way to share ideas/materials and collaborate on planning

10. a) In your opinion: should the certificate at the end of the TTT be awarded for: • attendance and completion of the TTT • achievement of a pass grade?

b) Why do you think this? (open question)

11) How would you like to see the TTT develop over the next three years? (changes to curriculum, duration of TTT, development of handbooks and resources, creating an advanced level, etc)

Any other thoughts or feedback?

If you would be willing to be contacted for further information, please give your email address/contact details below.

Many thanks

98 Appendix F: Charts

Attitudes

Adopt a proactive and responsible attitude

345166178!

01*2!.))/!

.))/! 9:81*! ;1:)*1! +,-*!

())*!

"! #"! $"! %"! &"! '""!

Map out a personal action plan for future personal growth

345166178!

01*2!.))/!

.))/! 9:81*! ;1:)*1! +,-*!

())*!

"! #"! $"! %"! &"! '""!

Act inclusively and with an open mind

345166178!

01*2!.))/!

.))/! 9:81*! ;1:)*1! +,-*!

())*!

"! #"! $"! %"! &"! '""!

99

Skills

Plan and deliver effective training sessions

345166178!

01*2!.))/!

.))/! 9:81*! ;1:)*1! +,-*!

())*!

"! #"! $"! %"! &"! '""!

Facilitate reflection and discussion

345166178!

01*2!.))/!

.))/! 9:81*! ;1:)*1! +,-*!

())*!

"! #"! $"! %"! &"! '""!

Collect evidence for goal-based evaluation

345166178!

01*2!.))/!

.))/! 9:81*! ;1:)*1! +,-*!

())*!

"! #"! $"! %"! &"! '""!

100

Knowledge

Responsibilities to fulfill role as trainer

345166178!

01*2!.))/!

.))/! 9:81*!

+,-*! ;1:)*1!

())*!

"! #"! $"! %"! &"! '""!

Relevance of Theories and Methods

345166178!

01*2!.))/!

.))/! 9:81*! ;1:)*1! +,-*!

())*!

"! #"! $"! %"! &"! '""!

Theory, practice and purpose of goal-based evaluation

345166178!

01*2!.))/!

.))/! 9:81*! ;1:)*1! +,-*!

())*!

"! #"! $"! %"! &"! '""!

101

Overall Impact

Enhanced ability to fulfill role

@8*)7A62!9A*11! 9A*11! <-B,A*11! @8*)7A62!<-B,A*11! <)7=8!>7)?!

"! #"! $"! %"! &"! '""!

Enhanced training competencies

@8*)7A62!9A*11!

<-B,A*11!

<)7=8!>7)?! "! #"! $"! %"! &"! '""!

Strengthened ability to design and deliver training

@8*)7A62!9A*11! 9A*11! <-B,A*11! @8*)7A62!<-B,A*11! <)7=8!>7)?! "! #"! $"! %"! &"! '""!

Feel part of a network of trainers

@8*)7A62!9A*11! 9A*11! <-B,A*11! @8*)7A62!<-B,A*11! <)7=8!>7)?! "! #"! $"! %"! &"! '""!

102

Promote Consistency

Effective means of promoting common training practices

@8*)7A62!9A*11! 9A*11! <-B,A*11! @8*)7A62!<-B,A*11! <)7=8!>7)?! "! #"! $"! %"! &"! '""!

Effective means of introducing new initiatives

@8*)7A62!9A*11!

<-B,A*11!

<)7=8!>7)?! "! #"! $"! %"! &"! '""!

Effective means of creating networks

@8*)7A62!9A*11!

<-B,A*11!

<)7=8!>7)?! "! #"! $"! %"! &"! '""!

Effective means of sharing ideas and materials

@8*)7A62!9A*11! 9A*11! <-B,A*11! @8*)7A62!<-B,A*11! <)7=8!>7)?! "! #"! $"! %"! &"! '""!

103

Appendix G

Tables

104 Table 1

Mean Responses to Online Survey Questions, Post-Training Assessment of Attitudes, Skills and Knowledge

TTT Goal and Survey Item Meana Stand. Dev

Attitudes: My willingness to . . . Adopt a proactive and responsible attitude to training 4.07 0.51 Map out a personal action plan for future personal growth 4.19 0.55 Act inclusively and with an open mind 4.45 0.54

Skills: My ability to . . . Plan and deliver effective training sessions 4.13 0.46 Facilitate reflection and discussion 4.32 0.49 Collect and use evidence for goals -based evaluation 4.10 0.56

Knowledge: My understanding of . . . The specific responsibilities to fulfill the role as a trainer 4.25 0.50 Relevance to peace education and training 4.26 0.52 The theory, practice and purpose of educational evaluation 4.15 0.49

Impact of TTT on you as a Trainer:

Enhanced my ability to fulfill the training responsibility 4.58 0.64 Enhanced my training competencies 4.57 0.54 Strengthened ability to design and deliver effective training 4.65 0.57 I feel a part of a network of certified trainers 3.27 0.87

TTT goal of promoting consistency: TTT is a useful way to Promote common training practices across the organization 4.76 0.47 Introduce new initiatives within CISV 4.73 0.55 Create networks across the organization 4.50 0.66 Share ideas/materials and collaborate on planning 4.59 0.61

105 Table 2

ANOVAs by Region on Post Training Assessment of Attitudes, Skills and Knowledge

Means

TTT Goal and Survey Item Region 1 Region 2 Region 3 F

Attitudes: My willingness to . . . Adopt a proactive and responsible attitude to training 4.08 4.05 4.08 0.045 Map out a personal action plan for future personal growth 4.10 4.15 4.27 1.583 Act inclusively and with an open mind 4.33 4.54 4.49 2.117

Skills: My ability to . . . Plan and deliver effective training sessions 4.08 4.08 4.18 1.028 Facilitate reflection and discussion 4.29 4.13a 4.44a 5.704* Collect and use evidence for goals-based evaluation of training 4.10 4.23 4.04 1.534

Knowledge: My understanding of . . . The specific responsibilities to fulfill the role as a trainer 4.27 4.23 4.26 .058 Relevance to peace education and training 4.24 4.44 4.40 1.898 The theory, practice and purpose of educational evaluation 4.06 4.18 4.19 1.204

Impact of TTT on you as a Trainer: Enhanced my ability to fulfill the training responsibility 4.59 4.62 4.56 0.112 Enhanced my training competencies 4.65 4.49 4.56 1.043 Strengthened ability to design and deliver effective training 4.67 4.69 4.61 0.33 I feel a part of a network of certified trainers 3.16 3.41 3.27 0.87

TTT goal of promoting consistency: TTT is a useful way to Promote common training practices across the organization 4.75 4.67 4.82 1.38 Introduce new initiatives within CISV 4.82 4.64 4.73 1.099 Create networks across the organization 4.69 4.49 4.39 3.295 Share ideas/materials and collaborate on planning 4.71 4.69 4.45 3.514

* p < .05 a. means are significantly different

106 Table 3

ANOVAs by Year on Post Training Assessment of Attitudes, Skills and Knowledge

Means

TTT Goal and Survey Item Year 1 Year 2 Year 3 F

Attitudes: My willingness to . . . Adopt a proactive and responsible attitude to training 4.10 4.05 4.06 0.149 Map out a personal action plan for future personal growth 4.08 4.19 4.33 2.628 Act inclusively and with an open mind 4.27a 4.49 4.63a 6.763*

Skills: My ability to . . . Plan and deliver effective training sessions 4.19 4.09 4.10 0.779 Facilitate reflection and discussion 4.44 4.25 4.27 2.768 Collect and use evidence for goals-based evaluation of training 4.19 4.00 4.12 1.668

Knowledge: My understanding of . . . The specific responsibilities to fulfill the role as a trainer 4.22 4.25 4.31 0.402 Relevance to peace education and training 4.27 4.40 4.43 1.497 The theory, practice and purpose of educational evaluation 4.20 4.11 4.14 0.592

Impact of TTT on you as a Trainer: Enhanced my ability to fulfill the training responsibility 4.59 4.46 4.71 2.224 Enhanced my training competencies 4.51 4.49 4.73 3.317 Strengthened ability to design and deliver effective training 4.66 4.61 4.67 0.163 I feel a part of a network of certified trainers 3.34 3.14 3.35 1.004

TTT goal of promoting consistency: TTT is a useful way to Promote common training practices across the organization 4.71 4.82 4.75 0.854 Introduce new initiatives within CISV 4.78 4.7 4.71 0.326 Create networks across the organization 4.49 4.56 4.45 0.394 Share ideas/materials and collaborate on planning 4.46 4.61 4.59 0.891

* p < .05 a. means are significantly different

107 Table 4

CISV Common Goals and Indicators

Goal 1: Prepare trainees to use CISV`s approach to active global citizenship

Indicator A: Understand how Peace Education is relevant to Active Global Citizenship (K) Indicator B: Understand CISV’s educational principles and methods (K) Indicator C: Be able to plan, deliver and evaluate educational experiences (S) Indicator D: Be able and willing to support and promote CISV educational principles (S) (A)

Goal 2: Prepare trainees to effectively undertake practical responsibilities within CISV

Indicator A: Understand specific responsibilities and resources to fulfill a particular role (K) Indicator B: Understand how individual responsibilities are important in a wider context (K) Indicator C: Be able to meet the practical needs necessary to fulfill a particular role (S) Indicator D: Be willing to adopt a proactive and responsible attitude (A)

Goal 3: Prepare trainees to contribute to the development and growth of CISV

Indicator A: Will to seek opportunities for cooperation inside and outside the organization (A) Indicator B: Be able to apply best practices and lessons learned (S) Indicator C: Be willing to actively contribute to continuous improvement (A) Indicator D: Understand how the strategic direction of CISV relates to individuals (K)

Goal 4: Prepare trainees to interact effectively and appropriately in diverse environments

Indicator A: Develop an understanding of self and others (K) Indicator B: Be able to make a positive contribution to a group environment (S) Indicator C: Be able to respond appropriately to change and conflict (S) Indicator D: Be willing to act inclusively and with an open mind (A)

108 Table 5 Comparison of Before and After Train the Trainers Leadership Program Responses to Attitudes, Skills, Knowledge and Effects Before TTT After TTT % Very % Very Good or Good or TTT Domain and Survey Item Excellent Meana Excellent Meana t Attitudes: My willingness to . . . Adopt a proactive and responsible attitude to training G2 84.8 3.97 93.3 4.07 1.90 Map out a personal action plan for future personal growth G4 75.1 3.95 94.0 4.19 4.06*** Act inclusively and with an open mind G4 58.8 3.53 98.2 4.45 12.70***

Skills: My ability to . . . Plan/deliver effective training sessions G1 17.0 3.02 95.2 4.13 20.65*** Facilitate reflection and discussion G2 10.9 2.90 98.8 4.32 25.14*** Collect and use evidence for goals-based evaluation G1 20.6 3.16 89.1 4.10 19.07***

Knowledge: My understanding of . . . The specific responsibilities to fulfill the role as a trainer G2 33.6 3.15 97.0 4.25 19.00*** Relevance to peace education and training G1 16.9 3.08 98.2 4.36 23.40*** Theory, practice and purpose of educational evaluation G1 15.1 2.79 94.5 4.15 22.64***

* p < .05 **p < .01 *** p < .001

109

Table 5 Continued % Agree or Strongly TTT Domain and Survey Item Agree Meana Impact of TTT on you as a Trainer: Enhanced my ability to fulfill the training responsibility 92.1 4.58 Enhanced my training competencies G1 97.8 4.57 Strengthened ability to design and deliver effective training G1 95.2 4.70 I feel a part of a network of certified trainers G3 27.2 3.27

TTT goal of promoting consistency: TTT is a useful way to Promote common training practices across the organization G3 98.2 4.76 Introduce new initiatives within CISV 95.8 4.73 Create networks across the organization G3 92.8 4.48 Share ideas/materials and collaborate on planning G3 93.3 4.58 a. Scale ranged from 1 to 5

110 References Allen, D., & McKay, P. (1976). History of CISV 1951/1976. Newcastle upon Tyne, England: Children’s International Summer Villages Ldt. Allen, D. (1982). A theoretical background for CISV research. Research and Development: A Journal on Transcultural Education, 1, p. 10-12. Allen, D. (1986). The basics of CISV. Research and Development: A Journal on Transcultural Education, 5, p. 5-8. Association of American Colleges and Universities, (1995). The dream of diversity and democracy: Higher education and American commitments. Washington DC: Author. Bamberger, M., Rugh, J., & Mabry, L. (2006). Real world evaluation. Thousand Oaks, CA.: Sage Publications. Bass, B. M., Stogdill, R. M., & Stogdill, R. M. (1990). Bass & Stogdill's handbook of leadership: Theory, research, and managerial applications. New York, NY: Free Press. Bassi, L. J., & van Buren, M. E. (1999). The 1999 ASTD state of the industry report. , VA: ASTD. Bolman, L., & Deal, T. (1991). Reframing organizations. San Francisco, CA: Jossey-Bass. Burns, J. M. G. (1978). Leadership. In Wren, T. J. (Ed.), The leader’s companion: Insights on leadership through the ages. New York, NY: The Free Press. CISV International Ltd. (1986). SOLAR report. Newcastle upon Tyne, England: Author. CISV International Ltd. (1989). Leadership training circle report. Newcastle upon Tyne, England: Author. CISV International Ltd. (2004). CISV international strategic planning report. Newcastle upon Tyne, England: Author. CISV International Ltd. (2008). ILTC report to AIM. Newcastle upon Tyne, England: Author. CISV International Ltd. (2008). ILTC long term vision. Newcastle upon Tyne, England: Author. CISV International Ltd. (2008). CISV international writeshop 2007 report. Newcastle upon Tyne, England: Author. CISV International Ltd. (2009). CISV international post-AIM minutes. Newcastle upon Tyne, England: Author.

111 CISV International Ltd. (2009). CISV annual report 2009. Newcastle upon Tyne, England: Author. CISV International Ltd. (2009). CISV international strategic planning report. Newcastle upon Tyne, England: Author. CISV International Ltd. (2009). CISV passport. Newcastle upon Tyne, England: Author. CISV International Ltd. (2010). CISV international writeshop 2010 report. Newcastle upon Tyne, England: Author. Cousins, J. B., & Earl, L. M. (1992). The case for participatory evaluation. Educational Evaluation and Policy Analysis, 14, 397-418. Creswell, J.W. (2003). Research design: Qualitative, quantitative, and mixed-method approaches. Thousand Oaks, CA: Sage Publications. Denzin, N. K., & Lincoln, Y. S., (2005). The sage handbook of qualitative research (3rd edition). Thousand Oaks, CA: Sage Publications. Dewey, J. (1916). Democracy and education. New York, NY: Macmillan Company. Dewey, J. (1959). Experience and Education. New York, NY: Macmillan Company. Eseryl, D. (2002). Approaches to evaluation of training: Theory & practice. Educational Technology and Society, 5, 93-98. Fitzpatrick, J.L., Sanders, J. R., & Worthen, B.R. (2004). Program evaluation: Alternative approaches and practical guidelines. Boston, MA: Allyn and Bacon. Gardner, H. (1995). Leading minds: An anatomy of leadership. New York, NY: BasicBooks. Harvey, J., & Cambron-McCabe, N. (2005). Principles of adaptive leadership. In Cambron- McCabe, N. Cunningham, L. L. Harvey, J. & Koff, R. H. (Eds.), The superintendent’s fieldbook (pp. 30-36). Thousand Oaks, CA: Corwin Press. Hasenfeld, Y., Hill, K., & Weaver, D. (2002). A participatory model for evaluating social programs. Los Angeles, CA: The James Irvine Foundation. Heifetz, R. A., & Linsky, M. (2002). Leadership on the line: Staying alive through the dangers of leading. Boston, MA: The Harvard Business School Press. Kangaslahti, J. (1986). Reflections on SOLAR. Research and Development: A Journal on Transcultural Education, 5, 13-17. Kirkpatrick, D. (1994). Evaluating training programs – The four levels. San Francisco, CA: Berrett-Koehler.

112 Kolb, D. A. (1984). Experiential learning: Experience as the source of learning and development. Upper Saddle River, NJ: Prentice-Hall. Kolb, A.Y., & Kolb, D. A. (2005). Learning styles and learning spaces: Enhancing experiential learning in higher education. Academy of Management Learning and Education, 4, 193- 212. Lambert, L., Walker, D., Zimmerman, D. P., Cooper, J. C., Lambert, M. D., Gardner, M. E., & Szabo, M. (2002). The constructivist leader. New York, NY: Teachers College Press. Lewin, K. (1951). Resolving social conflicts and field theory in social science. Washington, DC: American Psychological Association. Matthews, W.P. (1991). Here in this village. Newcastle upon Tyne, England. CISV International Ltd. McNamara, C. (2000). Field guide to nonprofit program design, marketing and evaluation. Minneapolis, MN: Authenticity Consulting, LLC. Morrison, G.R., Ross, S.M., & Kemp, J. (2003). Designing Effective Instruction (4th Edition) Hoboken, NJ: Wiley International. Murphy, J. (1990). Helping teachers prepare to work in restructured schools. Journal of Teacher Education, 41(4), pp. 50-56. Putnam, H. (1989). Representation and reality. Cambridge, MA: MIT Press. Schram, T.H. (2003). Conceptualizing qualitative inquiry: Mindwork for fieldwork in education and the social sciences. Upper Saddle River, NJ: Merrill-Prentice Hall. Senge, P.M. (1990). The fifth discipline: The art and practice of the learning organization. New York, NY: Currency and Doubleday. Senge, P.M., Cambron-McCabe, N., Lucas, T., Smith, B., Dutton, J., & Kleiner, A. (2000). Schools that learn: A fifth discipline fieldbook for educators, parents and everyone who cares about education. New York, NY: Currency and Doubleday. Smith, D.G. (1997). Diversity works: The emerging picture of how students benefit. Washington DC: Association of American Colleges and Universities. Stogdill, R.M. (1974). Handbook of leadership: A survey of theory and research. New York, NY: Free Press. St. Pierre, E. A. (2006). Scientifically based research in education: Epistemology and ethics. Adult Education Quarterly, 56, 239-266.

113 Tichy, Noel M., & Devanna, M. A. (1990). The transformational leader. New York, NY: John Wiley & Sons Inc. Torrance, R. (1966). Sophocles: The women of Trachis and Philoctetes; A new translation in verse (line 592). Boston, MA: Houghton Mifflin. Williamson, K., & Bow, A. (2002). Analysis of quantitative and qualitative data. In K. Williamson (Ed.), Research methods for students and professionals: Information management and systems (2nd), (pp. 285–303). Wagga Wagga, NSW: Centre for Information Studies. Wren, T. J. (1995). The leader’s companion: Insights on leadership through the ages. New York, NY: The Free Press. Yukl, G. (1994). Leadership in organizations. Englewood Cliffs, NJ: Prentice Hall. Zukoski, A., & Luluquisen, M. (2002). Participatory Evaluation: What is it? Why do it? What are the Challenges? Community-Based Public Health Policy Practice, (5), 1-6.

114