Evaluation Report

EVALUATION REPORT

Company AY /Norge1

The report was written by: Bernd Hofmaier Division of Economics and Technology, Halmstad University Halmstad, June 2005

Company AY / Norway

With contribution from Arne Roar Lier and Ronny Sannerud

1 This evaluation report is based on the evaluation workshop that was conducted jointly by STX Florø and Akershus University College on 29/04, 2009, from 12 a.m. to 4 p.m.

1 Ronny Sannerud, Arne Roar Lier, Icara Holmesland Akershus University College, HIAK Kjeller, 15 09 2009

Summary report on the evaluation workshop: Self-evaluation of the educational cooperation between Company AY and Akershus University College

1. Introduction

This report provides a brief description of the evaluation workshop which took place April 29, 2009 at Company AY. Training at Company AY is an example of a form of training that an EU-funded project "Euronet-PBL: Education, Practice and Research" describes and analyzes. The project focuses on the implementation of practice by educational programmes offered at universities and poly-technical colleges in the areas of business administration and building and construction. The educational institutions that participate in the project are: Akershus University College (Norway), Sabanci University (Turkey), ITB, University of Bremen (Germany), University of Limerick (Ireland) and University of Maribor (Slovenia).

All educational institutions have different forms of practice for students. Akershus University College offers an educational programme for instructors at Company AY which has been included in the project and, therefore, is also being evaluated. This assessment - which is a self-evaluation by the participants in the educational programme – was conducted in the form of a workshop and is here reported. The report contains a brief statement and explanations about the background of the programme evaluated, about the application of the evaluation instrument and reflections about the evaluation.

Participants

The following persons participated in the workshop:

Manager 1, Company AY Manager 2, Company AY Instructor 1, Company AY Instructor 2, Company AY Instructor 3, Company AY ArneRoar Lier, Høgskolen i Akershus Ronny Sannerud, Høgskolen i Akershus

The evaluation was carried out by: Bernd Hofmaier, Högskolan i Halmstad

2 Background to the educational cooperation between Company AY and Akershus University College

Cooperation between Company AY and Akershus University College is consequence of the increasing competition which requires the company to: - Carry out production in a qualitative and efficient manner; - Invest in production, which requires high skills; - Recruit staff and trainees locally and keeps them. Such demands require that the company changes its focus from being a producer of finished vessels (primarily sheet metal) to being an equipment yard whose main task is to install technical equipment. This means that the company today has a greater need for pipe fitter and mechanics, and less need for welders. As part of this change an educational programme aimed at the instructors was organized and carried out with the purpose of promoting a more effective and relevant training of apprentices and employees.

The programme has consisted of six two-day gatherings. Each meeting lasted for about 5 hours, from 12.00 p.m. to 17.00 p.m. in the first day and from 09:00 a.m. to 12.30 p.m. in the second day. In addition, teachers in the University College provided guidance to student groups between sessions. The program has had a scope of 15 credits.

The aim of the course is to improve the educational abilities of instructors who are in charge of training/preparation of welders into becoming mechanics or plumbers. Participants in the instructors’ educational programme prepared local teaching plans in collaboration with the shipyards’s employees and teachers from Akershus University College.

The study programme focussed on the following themes: o Collaboration o Meetings management o Motivation o Sharing experience o Understanding Value Chain o Company AY 's curricula o Planning, implementation and evaluation of training

The study was conducted during 2008, starting in April and ending in November. The participants prepared project papers that were presented to master degree students in technical and vocational teacher education at Akershus University College’s. This presentation was part of their learning assessment.

3 2. The evaluation instrument

Evaluation, i.e., assessment of the activity, is based on a participative approach, which means that project stakeholders eventually with the project owner, along with process management, perform part of the evaluation process. This is done mainly through reflection and discussion of results regarding certain criteria.

The evaluation process generally consists of three general steps: (1) an evaluation workshop - organized by an outside team of specialists who are independent of the project (ideally the team should consist of a moderator, a secretary and an observer), (2) analysis of results - evaluation workshop evaluated by an external team, and (3) a feedback session - designed by an external team to present the evaluation results and to develop the project's future work.

The workshop lasts for at least four to five hours. Ideally, all participants in the partnership ought to be represented. For example, in innovation networks all actors and stakeholders are encouraged to discuss and reflect on how they regard the network's current status, its goals, and future developments in an open and critical way. Within the education context, participants are expected to discuss and reflect about the education’s form and content. The evaluation itself is only valuable and effective when all the players understand that they are "self-evaluators." It is not a traditional evaluation in which an external assessor collects data.

The evaluation workshop at Company AY was carried out on April 29th 2009. This report is the result of step one and two in the process above. The report can serve as a basis for further discussions and possible changes in the educational effort.

2.1 The weighting of criteria

In the first moment of the workshop, each of the participants assess the importance of various elements in their own cooperation. First, four main criteria that give a broad picture of the cooperation are weighted. The four main criteria (Preparation 'input', Praktikfas 'throughput', Post-placement 'output' and further impact 'outcomes') are important components in any educational project. The difference in Company AY case, however, is not about the students receiving training but that, instead, Akershus Høgskolen provides education to the staff of Company AY.

Then, the sub-criteria that clarify and specify the respective main criterion are weighted. The purpose of the weighting process is to clarify the significance that the individual participants attach to key areas of the activity. Afterwards, the individual weightings and every one’s arguments for these weightings are discussed by the whole group. This process includes the group definition of how the different sub- criteria must be understood in the context of the current network. To define the importance of the criteria is important, since these can be designed differently for different activities.

4 It should be emphasized that the weighting - and the subsequent assessment - applies to the current situation. Experience shows that the importance of the criteria may vary during the lifetime of the project. This means that the individual percentages and the relationship between the criteria are likely to change if this evaluation process is carried out at a later date. The figures in this report should not be regarded as a representation of the "status quo" but, instead, that the focus of this report is on the differences that exist between different weightings and assessments of the individuals.

2.1.1 The Weighting of Main Criteria

The weightings of the main criteria are presented in Table 1 below. The participants’ weightings are presented on the right side. The columns on the right side of the names show the highest and the lowest values of each criterion for each of the main dimensions. Discussion Consensus, i.e., the weightings reached through group discussions and agreement are shown on the left of the table. Some of the cells have been selected and show the weightings discussed in detail. It can, for example, be the areas where the largest differences were observed.2

Table 1: Main criteria (weighting) l y i d e e r e e

Discussion Standard t r j n r t n o n r

Medel Högst Lägst å n

Huvuddimensioner r e o n e d konsensus avvikelse j o I B T A T A K 5 R

1. Förberedelse (input) 30,0% 28,6% 8,3% 40% 20% 20% 20% 20% 40% 30% 30% 40%

2. Praktikfas (throughput) 40,0% 11,8% 60% 20% 30% 40% 60% 30% 30% 30% 20%

3. Uppföljning efter praktik 20,0% 24,3% 7,3% 30% 10% 30% 20% 10% 20% 30% 30% 30% (output) 4. Ytterligare effekter 10,0% 12,9% 4,5% 20% 10% 20% 20% 10% 10% 10% 10% 10% (outcomes) Translation of table: Huvuddimensioner: Main dimensions Standard avvikelse: standard deviation Högst: Highest Lägst: Lowest

As expected the practice phase, namely the educational phase, is the most important. Thereafter comes the preparation phase, which is meant to be central and vital, being thus emphasised in the discussion later. The differences found in the weighting are related to different perceptions between the teaching staff and the company’s employees participating in the educational programme. The differences found in the weightings refer to the different perceptions of teachers and the company’s influence on the educational programme. Above all, difficulties in combining study activities with the ongoing production at the shipyard were not mentioned. The low weighting of the criterion’s further effects (outcomes) was probably due to the fact that they were not important for the case, although in other circumstances it might be possible

2 Some of the individual weightings changed in connection with the discussions. The highlights of the tables, however, point out the major differences that existed prior to any changes that occurred as a result of group discussions.

5 to provide examples of the importance of this form of advertising. This criterion was related to the visibility of the education and, perhaps, was also highlighted by either the company or Akershus University College.

Sub criteria After the discussions about the main criteria, the discussions about several of the sub criteria add nuances to the main criteria. Here is just one example of the placement phase, illustrating that the students, namely the participants' skills and abilities seen in practice, are very important for the representative of the company.

Table 2: Practical learning phase (weighting)

2. Praktikfas (throughput) 40%

2.1. Stöd genom universitetet 20,0% 20,0% 9,4% 40% 10% 25% 40% 10% 20% 20% 20% 10%

2.2. Stöd genom företaget 20,0% 10,0% 7,8% 30% 10% 25% 30% 10% 10% 20% 20% 30%

2.3. Studentens kunskaper och 30,7% 14,7% 60% 10% 25% 10% 60% 30% 20% 30% 40% förmåga visad i praktiken 30,0% 2.4 Tid/plats för studentens 17,9% 3,6% 20% 10% 20% 15% 10% 20% 20% 20% 20% reflektion 15,0%

2.5 Samarbete mellan partner 15,0% 10,0% 7,1% 20% 0% 5% 5% 10% 20% 20% 10% 0%

Translation of table: Stöd genomuniversitetet: Support by the university Stöd genomföretaget: Support by the company Studentens kunskaper och förmåga visad i praktiken: Students’ skills and abilities seen in practice Tid / plats för studentens reflektion: Time / Place for the students’ reflection Samarbete mellan partner: Cooperation between partners

Assessment of criteria Through the assessment of criteria, one’s own work "is rated" on the basis of how far one considers to have come in the process, or how much of the target has been achieved. After the individual assessment results are discussed in the group. The assessment, which goes from 1 (completely disagree) to 10 (agree), is made by individual participants based on their roles and positions, being the differences discussed subsequently. There is a phase in the workshop that requires time. This is the moment when the discussion of differences takes place and participants must explain themselves. In many cases it appears that a consensus can be achieved but also differences in assessment may occur. The important thing is that the argument for why one assesses the activity or criteria in a certain way, be expressed. The 3rd and the 4th columns show the maximum or minimum assessment scores. The equal scores mean that a consensus has been reached after discussions.

6 Table 3: Preparation (Assessment)

Diskussio Diskussio l y i e e e r d e t n n nskonsens j t r r n a r n

Viktning Medel Högst Lägst n e å r o n e d j

Konsensu us (el. o I T B A T A K s (eller lägsta R 1. Förberedelse (input) 30% 1.1. Förberedelser i utbildningen 30% 8,9 10 8 9 8 9 8 10 10 8 (Universitetet) 8 8

1.2. Föreberedelser i företaget 25% 6 6 6,1 8 5 5 8 5 5 7 5 8

1.3. Studentens kunskaper och 20% 6,6 8 5 7 8 7 5 7 7 5 förmåga för praktikuppgifter 8 5 1.4. Tillräcklig förberedelsetid och 15% 5,0 7 3 5 5 7 3 5 4 6 plats (gemensam utrymme) 4 3

1.5 Integration av förberedelser 10% 8 3 5,3 8 3 8 5 3 5 4 7 Translation of table 1. Preparation 1.1 Preparation for education (University) 1.2 Preparation at the company 1.3 Students’ knowledge and ability for practical tasks 1.4 Adequate time and place for preparation (shared space) 1.5 Integration of preparation

As shown in the table, there are differences in the assessment. The participants in the educational activities expressed concerns about having enough time for preparation, etc. In criterion 1.5, Integration of preparations, there were relatively large differences in the assessments. Further comments are made in the Summary part of this report.

Table 4: Practical learning phase (assessment) l y i e e e r d e t n j t r r n a n r n e å r o n e d j o I T B A T A K R 2. Praktikfas (throughput) 40%

2.1. Stöd genom universitetet 20% 9 6 7,7 9 6 9 6 9 7 8 8 7

2.2.Stöd genom företaget 20% 8 5 6,6 8 5 7 6 7 5 5 8 8

2.3. Studentens kunskaper och 30% 7,6 10 6 7 8 6 6 7 9 10 förmåga visad i praktiken 10 6 2.4 Tid/plats för studentens 15% 5,0 8 2 3 4 6 2 6 6 8 reflektion 8 2

2.5 Samarbete mellan partner 15% 9 5 7,3 9 5 9 8 8 8 7 5 6 Translation of table 2. Internship phase 2.1 Support by the university 2.2 Support through the company 2.3 Students’ knowledge and abilities shown in practice 2.4 Time and place for the students’ reflection 2.5 Cooperation between partners

Although there were some differences here that were discussed, no consensus was reached during the limited time that was available. A further comment on these issues is made in the summary of Annex 1.

7 Table 5: Follow-up after training (assessment) l y i e e e e r d t n j t r r n a n r n e å r o n e d j o I T B A T A K R 3. Uppföljning efter praktik 20% (output) 3.1.Universitetet använder resultat 20% 8 6 6,1 8 3 3 6 7 6 5 8 8

3.2.Företaget använder resultat 30% 10 5 7,4 10 5 7 7 7 6 5 10 10

3.3. Studentens kunskaper och 30% 7,3 10 5 5 8 8 7 8 5 10 förmåga har växt 10 5 3.4 Tid/plats för studentens 10% 4,0 6 1 1 4 6 2 6 4 5 reflektion (rapporter etc) 6 1 3.5 Kunskapen delad mellan 10% 6,0 8 4 7 8 6 7 5 4 5 partner 8 4

Translation of table: 3. Follow-up after the practice 3.1 The university uses the results 3.2 The company uses the results 3.3 Students’ knowledge and ability has grown 3.4 Time / place for students’ reflections (reports, etc.) 3.5 Knowledge shared by the partners

These were relatively high assessment. However, the participants experienced problems related to time / space for reflection, which was considered a result of circumstances in the workplace.

Summary

The focus in applying the evaluation instrument is on the discussions that take place. Therefore, what is important is not the individual values and figures, but the differences or similarities, and the discussions surrounding them. The following diagram is thus the outcome of these discussions. It shows both how important we believe that different criteria are, and secondly the assessment of these criteria. Here you can have more or less different assessments or achieve consensus. It is important in this context to point out that the differences found are not to be regarded as negative. Instead, what is important is that the differences come to the surface and may be discussed. If agreement can be reached depends, of course, also on the time available and the workshop leader's behavior.

8 Figure 1: Summary image (weighting and analysis)

45% 10,0 g n i g

40% n n i 9,0 9,0 9,0 m

n 40% t ö k

8,6 d i e V 8,0 B 35%

30%6,9 7,0 30% 6,2 6,0 25% 5,7 5,6 Weighting 5,1 5,0 20%4,7 Judging low 20% 4,0 Judging high

15% Konsensus 3,0 10% 10% 2,0

5% 1,0

0% 0% 0,0 1. Förberedelse (input) 2. Praktikfas 3. Uppföljning efter 4. Ytterligare effekter 5. xxxxxxxxxxxxxxxxx (throughput) praktik (output) (outcomes) Translation: Viktning. weighting Bedömning: assessment

Figure 1 shows that the preparatory phase and the training phase (i.e., the educational program) are considered most important. There was also a relative agreement about the assessment beyond the education itself, where, for instance it was emphasised the difficulty of combining studies with work because it varied according to the company’s planning. It must be emphasized that differences in the assessment, to some extent, were also related to the limited time available for discussions. The important thing here is that the participants presented their views and arguments to each other.

References Deitmer, L. Davoine, E. Hofmaier, B. James, C. Manske, F. Ursic, D (2003) FINAL REPORT for the COVOSECO thematic network project within the STRATA programme. ITB, Bremen

Hofmaier, B. Eriksson, H (2008) Learning in Innovation Networks. A tool for support and evaluation of innovation networks Conference Insightful Encounters - Regional development and practice-based learning, Borgå/Poorvo.

9 Annex 1

A brief report on the educational cooperation between Company AY and Akershus University College

General The programme had approximately 30 participants, among which there were four instructors with special arrangements adapted for certain parts of the studies.

The instructors felt that the programme had been based on the students' preconditions and that the programme’s goals were realistic. Thus, the programme had a high relevance to the instructor's teaching responsibilities in the practical life. They used what was taught in conjunction with the apprenticeship training and the restructuring taking place, which required that welders would become mechanics and plumbers. The curricula developed during the study, which was aimed at the mechanics and plumbers, was used actively in the re-training.

The challenge for the students attending the programme was to have time for working afterwards with the content and time for preparation. This was not possible during the working hours, thus preparations had to be done in the spare time at home. The management had "pressed" the programme into a pressured life, but there was satisfaction with the programme from the company's side. The programme has been very important for the business, “because we depend on competence!” was said.

The managers experienced that knowledge-sharing between the college and the company provided a good professional development because the college did not come with "The answer", but insights were developed along with the company.

The company managers indicated to be pleased with the cooperation between the partners, i.e., students, the university college and the company. The instructors used what was learned in the programme in the activities related to the retraining of employees, i.e., the welders who became mechanics and plumbers.

The weakest part of the project is that it is not very visible in public spaces, which prevents the surroundings to become familiar with what has been done. From the university college’s perspective, it was emphasized the importance of the company’s trust in the sense that the teachers had the possibility to speak with people in the company and had the necessary time to "understand" the company's working methods and professional standpoint.

All students attending the university college were assessed to have good knowledge and academic qualifications to carry out the study, which is also shown in the results. The geographical separation meant that teachers were not followed up enough. This could be compensated with more frequent meetings. However, this was difficult due to high production demands. Better and more systematic arrangements related to students' reflection on practical experience would improve the quality.

10 Summary of the comments:

Preparation: • Difficult to adapt production demands in the company with study demands for the students. • college plans were very good and were well adapted to the students' qualifications and were highly relevant to the instructors. • The company gave the university college’s teachers freedom to seek the people they wanted, and plenty of time to prepare the study.

Implementation: • The meeting was interesting for the students and the combination of theory and work with practical exercises was good. • There was insufficient time for students in relation to the preparation and follow up of the programme. There was no time off to reflect about the meetings. • The topics included in the workshops were highly relevant for the instructor's tasks.

Results • Company AY has increased the instructors’ competence in relation to the training of apprentices and retraining of welders into mechanics and plumbers. • In general, the company uses the expertise that was acquired by the students through the programme. • The company, the students and the university college maintain contact and continue exchanging experiences after the study is completed. • The results of the programme are hardly used for promoting the image of the companies and students. The study and its results were presented to colleagues and students internally in the university college and in international meetings. • The company, the students and the university college were pleased with the planning, implementation and results attained.

11 Annex 2 EURONET- PBL Company AY/Norge

Weighting Weighting of of main Assessment of sub-criteria from 1 Main Criteria Sub-criteria Sub-criteria criteria (bad) to 10 (excellent) (%) (%) 1. Preparation ('Input') 1.1 Preparation for education (University) 1.2 Preparation at the company 1.3. Students’ knowledge and ability of practical tasks 1.4 Adequate time and place for prepartion (shared space) 1.5 Integration of preparation  100 % 2. Practical learning phase ('Throughput') 2.1 Support by the University 2.2 Support by the company 2.3 Students’ knowledge and abilities seen in practice 2.4 Time and place for students’ reflection 2.5 Cooperation between partners  100 % 3. Follow up after practice ('Output') 3.1 The university uses the results 3.2 The company uses the results 3.3 Students’ knowledge and ability has grown 3.4 Time/place for students reflections (repports, etc.) 3.5 Knowledge shared by the partners  100 % 4. Further effects ('Outcomes')

12 4.1 The results of cooperation are visible for the public 4.2 Partners' 'image' 100 %  100 %

13 Yildiz Arikan, Oguz Baburoglu TR_Eval-WS_EN.doc Faculty of Management, Sabanci University Istanbul, 15.05.2009

Summary report on the evaluation workshop discussion: Self-evaluation of the students’ CAP projects of the MBA Program at the Faculty of Management, by representatives from company, university and students

Introduction

This document is a report on the evaluation workshop that was arranged in the context of the European project Euronet-PBl to assess the functioning and effects of the Company Action Projects(CAP) of the MBA Program on student learning. The workshop was organised at the Sabanci University Campus as an afternoon session between 13.00 and 16.30.

The workshop was carried out by the following experts:  An external moderator (Lars Heinemann, code name LH),  Two academics who teach and run the CAP related courses and activities( Oguz Baburoglu and Yildiz Arikan, code names OB and YA),  Project supervisor from the CAP partner company ETI (code name MA) and  Project supervisor from the CAP partner company Autoliv-Turkey(code name BD) and  Eight students who had carried out CAP during the 2008-2009 academic year (code names YD, EK, MK, ZK, AK, MN, AT, OK) These students showed their willingness to participate in the workshop and represent 2008-2009 academic year.  One reporter (A student from the same MBA class, code name ES)

Below, the different phases of the workshop are described briefly: 1) The Introductory phase and the presentation of the evaluation instrument, 2) Weighting of the criteria and the discussion on the relative importance of diverse criteria, 3) Judgement of the functioning of CAP based on the criteria and related discussion. 4) Concluding reflections.

1. The Introductory Phase

Firstly the university lecturer OB welcomed the participants on behalf of the university that had initiated the workshop and thanked the CAP company sponsors who had agreed to participate in the workshop and to share their experiences. He then handed the coordination of the session to the invited moderator LH and to the reporter ES.

LH gave an introductory presentation on the empowerment evaluation - approach that had been chosen for this workshop and referred to prior projects in Sweden, Germany and at the level of European cooperation. He referred to the following key features of the approach and of the related instrument (the evaluation tool):  The main aim of the workshop is to help the participants to analyse their own activities and to draw conclusions for further development.

14  The role of the tool is not to collect comparable data but to help the participants to organize the evaluative discussion with reference to different weightings and judgements.  The instrument could be used as a means to support consensus-finding but since there were three different stakeholder-groups (company sponsors, students, CAP instructors) it was not appropriate to try to reach a consensus on judgements but perhaps on weightings.  Workshop participants were provided with a brief description of the evaluation tool before they attended the meeting so that they had a chance to review it.

After the introductory remarks of LH the participants presented themselves and the activities in which they had been involved regarding the CAP projects:

 The students were all from the graduating class of the 2-year MBA Program,  The students represented the 8 different projects that was undertaken in 8 different companies  The students had undergraduate degrees on different major subjects from different universities: o The student YD (Business Administration) o The student EK (Economics) o The student MK (Economics) o The student ZK (Computer Engineering) o The student AK (Mechatronics) o The student ML (Mechanical Engineering) o The student AT (Management) o The student OK (Economics)  CAP Project supervisors MA and BD presented their roles in receiving and supervising the students that were making their CAP projects at their related companies.  The university lecturers OB and YA explained their roles as the CAP course instructors and coordinators at the university and with companies participating to the University’s CAP Program.

2. The weighting phase 2a) Weighting process (data and comments)

Main Criteria Weighting Results 1. Preparatory Arrangements ('Input') 22,5% 2. Practical CAP Phase ('Throughput') 28,3% 3. Follow-up Phase after the CAP 26,3% ('Output') 4. Further Effects ('Outcomes') 22,9%

In the first working session the participants got the questionnaire that was presented in the introductory phase (see annex 1a and 1b). Each participant was asked to consider the relative importance of the criteria. Firstly the four main criteria were weighted and then the related sub-criteria. When all participants had completed the weighting, the results were collected and transferred to an excel-sheet. Then, the overview of the weightings was presented to the participants. The moderator had coloured the cells in which very high or very low weightings were given.

Below presented are some general trends and specific remarks reflecting the discussions made during the workshop:

15  Averages of the weights assigned to Preparatory Arrangements and Further Effects criteria are similar but smaller than the weights given to Practice and Follow up phases. This means that participants on the average consider the Practical CAP phase and the Follow-up phase to be more critical to CAP process than the first and last phases.  Concerning the preparatory phase of the CAP project, the participants’ assessment resulted in a weight of 22,5%. Related to this criteria company sponsor MA expressed his view that setting the expectations on both sides- University + the students and the Company is very important in the preparatory phase. Some students expressed their belief that the potential Academic Advisors should also be involved in this phase, especially when the projects are presented to the students in class. The university lecturer OB explained that Sabanci University does a very good job before the projects are selected and told that the process can also be improved (customized) after the projects are selected.  Concerning the implementation phase of the CAP project, both company representatives and the University lecturers gave it a relatively high weight, (50% by OB and 35% by MA) resulting in an overall weight of 35 %. Students’ weights were in the range of 15-35. This demonstrated how different the perceptions of the parties were with respect to project implementation phase requirements. Discussions on the criteria drew attention to academic advisor’s role and responsibilities in this phase. Students agreed that the advisors must be involved in the project with their guidance but they should also give the teams flexibility so that they can try alternative means to conduct their projects. OB mentioned that faculty advisors should however be doing more than today.  The other expectation of the students was mentioned as to bring together the company sponsors and academic advisors more often then employed currently. It was also mentioned that it is very important and must be the teams’ responsibility to establish a good relationship with the company during this phase.  Additionally it was mentioned by the student MN that real learning occurs when all the teams share their experience in class. Another student YD stated that he also learnt much from watching the daily company operations.  Concerning the immediate reflection and reprocessing phase weights given by the students averaged as 12,3 % which is very low. For the university lecturer OB this phase has stronger importance (his weight is 40%). Company sponsor BD assigned the highest weight to this criterion (25 %)  When the long-term effects of the CAP projects criteria is considered the two company sponsors MA and BD assigned very different weights (10% and 25 % respectively. For students weighting in this case was tricky because their CAP relationship with the company lasts only 6 months and it is difficult to judge the future from this limited relationship. However, some of the participants expressed that they could already see some long-term perspectives emerging from the projects(40% weight given)

3. The judgment phase

In this phase the participants received the questionnaire sheets back and got the task to give their judgments on the functioning of the CAP in the light of the previously discussed criteria. The scale that was used was between 1 (lowest rating) and 10 (highest rating). It was emphasized that the participants should give their ratings with reference to experienced practice (not vis-à-vis their expectations on how they things should work).

16 On the basis of this instruction the participants gave their ratings and the results were collected and transferred to the respective excel sheets. Once again, exceptionally high and low ratings were marked with colour. This provided the basis for the further discussion and exchange of views. These are summarized below as discussion points under the respective main criteria: a) Criterion 1: The preparatory measures

Here the ratings differed considerably depending on the actor-position and on the individual experiences:  It was difficult for the company representatives to rate the preparatory measures of the university because they were not part of the activities. For this reason the ratings by MA and BD are 6, and 7 respectively. The students’ ratings all were relatively higher (7 or 8). Average rating for this criteria is 7,5 which means parties are satisfied with the preparations by the university.  Regarding the preparatory measures in the company, the ratings of the students varied a lot ranging from 2 to 10. This means that depending upon the project company; the preparatory measures undertaken were between unsatisfactory to highly satisfactory. This is not an uncommon phenomenon in the CAP experience over the years. Company culture, level of institualisation and the power of the project sponsor are among the factors which shapes the preparations. One of the company sponsors made self criticism that he could not provide enough support for the team at the unset of the project. He explained that although he was the sponsor of the project, the project belonged to another department in the company and he could not push the limits very much. One student who undertook the project in this company said that the company did not show enough dedication to the project.  Students knowledge and capabilities and students’ preparations were assessed by the company sponsors were high. Sabanci lecturer did not share the same view with the sponsors. His judgment rate is at the minimum of all raters. This is due to the fact that OB knows the students very well and he expects them to take more responsibility.  Concerning the coordination between different parties involved in CAP process, the participants pointed out that more contact between the university and the companies may improve the conduct of the projects. Here the Open Space Technology workshops which are held once every CAP period were mentioned to be serving this purpose. In these workshops students, advisors and sponsors come together one afternoon and discuss CAP issues within self determined task group and come up with recommendations. It was agreed that to hold one workshop each semester would be very useful. A B D K K N

K T D K A K O M B

Sub criteria E M Z A M A O Y Y 0 1 2 1 2 3 4 5 6 7 8 9 1 1 1 1.1. pre-arrangements at 7 7 8 7 8 8 8 8 9 7 6 7 university 1.2. pre-arrangements at 10 9 2 5 9 8 6 6 7 8 5 8 company 1.3. student knowledge 10 6 5 7 9 9 8 8 8 5 7 8 and capabilities for Praktikum tasks 1.4. students' 8 7 5 6 7 8 7 8 8 5 8 9 preparation

17 1.5 integration of 8 10 6 7 8 8 7 7 8 8 7 9 arrangements b) Criterion 2: Practical implementation

Here the ratings are slightly higher in general but there is a variation in a range from 5 to 10 depending on the rater’s position and probably his/her experiences:  Company sponsors rated the prearrangements by the university as well as by their own companies’ quite low. However students knowledge for conducting the project was rated as highly satisfactory(7 and 9 )  There was consensus among the sponsors and the lecturers that students’ reflection and learning has been good. However the same group thinks that there should be increased collaboration between the parties, namely the university and the company.  Lecturer OB expressed his opinion that Autoliv project was one of the best practices based on learning experience students had. Moderator LH mentioned that creation of learning requires company involvement and what counts is how committed and motivated the people are. In this respect reasons behind the success with Autoliv project were mainly the high degree of support and feedback given to the students by the company and the sponsor. Moderator LH also stated that for better output with the project, there should be an ongoing iterative support process.  Concerning the cooperation between different stakeholders, some of the participants were critical or self-critical. Lecturer OB and Sponsor MA for instance thinks that the collaboration has not been to the required level. Except for the student MK, students in general were satisfied with the level of collaboration. However, they also emphasized the need for better means of exchanging information and having joint meetings. A B D K

K N Sub- criteria

K T D K A K O M B

E M Z A M A O Y Y 0 1 2 1 2 3 4 5 6 7 8 9 1 1 1 2.1. support by 10 6 7 8 7 9 9 9 9 8 6 6 university 10 10 5 6 9 8 6 6 7 6 5 9 2.2. support by company 2.3. students' 10 8 7 7 8 9 7 9 8 6 7 9 capabilities displayed at executing tasks 8 10 8 8 7 9 6 9 8 10 7 9 2.4 students' learning 2.5 collaboration 10 10 4 7 8 7 7 8 7 5 6 8 between actors

18 c) Criterion 3: The immediate reflection and re-processing of results

For this criteria the participants made only few remarks because students graduate and go away and don’t know what benefits it brings to the university. Company sponsors on the other hand could only guess what might happen after the project  Average rating of Criteria 3. 1 is very low compared with the other criteria. University lecturer OB’s rating is the lowest because he thinks that the design of CAP is such that the University’s benefits are limited, CAP can help increasing its reputation and developing a wider company network and by increasing the potential to offer jobs to our graduates. CAP may also result in writing Turkish cases.  The company sponsor BD rated the experience with Criteria 3.2 with 10. This is expected because the Autoliv project was one among the best project experiences up to now. Other project sponsor MA rated the same criteria with 4 because of the reasons explained before (little support and lack of involvement on the company side). Lecturer YA think that CAP companies in the past expressed the high value added with the specific projects they undertook. Even some sponsors when asked in the past, expressed the value of the project’s output in terms very high fees that would have been paid to consulting terms if a solution to the issue was sought there.  Ratings show that both sponsors are satisfied with the students’ reports and reflections. A B D K

K N Sub- criteria

K T D K A K O M B

E M Z A M A O Y Y 0 1 2 1 2 3 4 5 6 7 8 9 1 1 1 3.1. university 7 4 8 6 7 6 8 5 2 4 8 making use out of results 3.2. company 10 10 10 8 8 7 6 7 8 4 10 making use out of results 3.3. students' 10 10 9 7 8 9 7 8 8 10 7 9 knowledge and capabilities have grown 3.4 students' 5 6 9 7 7 9 6 8 8 5 7 9 reflection (reports etc.) 3.5. central 8 8 10 7 7 7 6 8 8 7 8 government d) Criterion 4: The long-term effects

This criterion received similar ratings to other criteria.  First of all there is a great difference between sponsors’ ratings. This must be due to the project experience which are opposites. This affected all their ratings. While sponsor MA is very pessimistic, company sponsor BD is quite hopeful and generous in his ratings.  What is promising about the CAP design is that the consensus about the high level of contribution of CAP to students’ future career. The average is 8,5 and it reflects that students and other parties all recognize the potential benefits.

19 the four main criteria are provided provided below.the are four main criteria to spider in evaluationrelation Bar/Linediagramand a summaryof asA the results either. difficulty could his job same much performwithout program the a of difficulty.anyThe reporterbeing withoutgraduate contributed therefore andexpect the from workshop wereto participantswellwhat informed about ofrelatively intime.The aHoweverworkshop short period was implemented remarks 4. concluding The 10 10 8 6 1Y D

1 9 8 5 2 EK

5 7 8 4 3 MK

5 7 8 7 5 AK

7 9 7 8 6 MN

7 8 6 6 7 AT

9 9 7 7 8 O K

8 9 7 8 9 YA 10

2 8 4 10 OB

1 7 6 2 11 MA

9 9 9 8 12 BD wider wider public achievementsvisible for 4.4 co-operation career students' further to 4.3 contribution development general company's to 4.2 contribution development university's general to 4.1.contribution Sub- criteria Sub- 20  The spider chart clearly shows that the “Preparatory Arrangements” and “Practical” phases of CAP needs improvements as the median of ratings for these two phases are quite distanced from best ratings. These two phases are actually the most critical phases for CAP process. It is in these two phases that the project development and conduct is achieved.

The concluding remarks from the workshop may be summarized as below:

 Company sponsors ratings and remarks indicated that they find CAP partnership worth undertaking. 1) Company sponsors were able to make self critics. This was beneficial to CAP lecturers to better understand the CAP success determinants. 2) All participants verbally expressed the need for better arrangements for the project development and conduct periods. 3) All participants agreed that high level of company involvement and the support and commitment of the sponsor, together with the match between the project and the academic advisor are crucial to successful CAP results. 4) Increased level of contact between the parties, (e.g. joint meetings for feedback giving) and resolution of socialization issues of the teams at the company sites were suggested as the means for improving the CAP Process. 5) Both the students and the company sponsors assessed the functioning of the CAP process depending upon their current experiences. Should there have been more participants from previous CAP projects, the results might have been different but perhaps the variation in the ratings would have been reduced and the overall assessment would be more reliable.