Military Communication Skills
Total Page：16
File Type：pdf, Size：1020Kb
The Military Communication Skills Project Evaluation an evaluative consultancy
Figure 1 - trainer and students, SLMA
Report commissioner: the Military Communication Skills Project Steering Committee Liaison and project management: Marion Hughes, MCSP Project Manager, British Council Sri Lanka Hamzi Haniff, Assistant Manager, Programme Delivery, British Council Sri Lanka Report preparation: Dr Christopher Tribble, King's College, London University www.ctribble.co.uk
Colombo / London 26 October – 17 December 2007
MCSP Evaluation Page 1 Dr C Tribble / www.ctribble.co.uk Executive Summary This evaluation was commissioned by the Steering Committee of the Military Communication Skills Project and, following a desk review of project documents, was carried out between 28 October and 7 November 2007. The evaluation responded to a very detailed terms of reference which include the major question,: Was the project and the subsequent activities based on a well thought out and credible Theory of Change? The Desk Review of project documents revealed inconsistencies in the Terms of Reference and the original project design, and these are commented on in Appendix 13 (pps 34 - 37). However, it was also clear from the review that the project's theory of change was sufficient and appropriate, and the evaluation itself has been able to fully respond to the initial ToR. Following an outline of the research approach and research methods, (pps 5), the Findings section of the report discusses three broad areas. In the first part (pps 6 - 10), Progress to Date is reviewed, and it is concluded that the project has delivered outputs that are appropriate to the needs of major stakeholders and which contribute to the project purpose, but that these outputs are, as yet, not sustainable. Section 6: Findings: output delivery to date (pps 11 - 22) discusses the outputs in detail and concludes that considerable progress has been made in updating the methodology for teaching English, and there there is considerable potential for improving the teaching of Tamil for the armed forces, but that without the Joint Services 3 month English Course and the Joint Services 50 hour Basic Tamil materials, the project will not bring about the institutional changes that are needed if it is to have sustainable outcomes. Section 7: Findings: three narratives (pps 22 - 25) provides a view of the project's impact from the perspective of the instructors and trainees in military training institutions, and reviews the impact (actual and potential) of the project in the area of peacebuilding. It concludes: "Each of these changes is leading to the achievement of the project's stated purpose: To improve the quality of language & communication skills for military forces in Sri Lanka If one accepts the project's theory of change, then the MCSP is making a contribution to peace building." (page 25) The report concludes with a set of recommendations, the most important of which are: Recommendation 3: funding To ensure continuity in planning to meet these major tasks, it will greatly assist the project team to a) implement the final stages of the project and b) put in place a coherent closure plan if: funding for the project is maintained until at least August 2009. Without such funding the sustainable outcomes of the project will be in serious jeopardy. and, if possible, funding can be provided for a final year so that the new materials can be fully embedded in the system. As there will be little need for further investment in resources by this stage, it should be possible for the final year to operate on a lower budget compared with 2008- 09. Recommendation 5: future training capacity Given the long term needs of the Sri Lankan military for communication and language skills training, there is a very strong argument for the development of a BA programme in either Language Teaching (for both English and Tamil), or, perhaps more interestingly, the establishment of a BA or BsC programme in Education with opportunities for specialisation in Languages or in other aspects of training (including technical and military subjects). Such a programme could provide the basis for a future Sri Lankan Education Corps a great lack at the moment. By developing this capacity in KDU, the long term training needs of the Sri Lankan military could be greatly enhanced. Although it goes beyond the remit of this evaluation to make a formal recommendation in this area, I would like to suggest that the Steering Committee actively investigates possibilities in this area as a means of ensuring the long term sustainability of the benefits that have arisen through the MCSP. MCSP Evaluation Page 2 Dr C Tribble / www.ctribble.co.uk Table of contents Executive Summary...... 2 1. List of acronyms used in this report...... 4 2. Report overview...... 4 PREPARATORY...... 5 3. Evaluation approach and research design...... 5 3.1 Overall approach...... 5 3.2 Visual documentation...... 5 3.3 Evaluation schedule...... 6 FINDINGS...... 6 4. Findings: Desk Review...... 6 4.1 Revision of the research questions used in the evaluation...... 6 4.2 Interim conclusion: how adequate is the project's theory of change?...... 7 5. Findings: The Military Communication Skills Project progress to date...... 8 5.1 The relevance of the project to local context...... 8 5.2 Project time line and major milestones...... 8 5.3 Interim conclusions progress to date...... 10 6. Findings: output delivery to date...... 11 6.1 Research questions...... 11 6.2 Appropriate English and Tamil language training materials...... 11 6.3 English...... 11 6.4 Stage 2...... 12 6.4.1 The English language programme...... 12 6.4.2 Human rights and peacebuilding...... 12 6.4.3 Tamil...... 13 6.4.4 Interim conclusion: output delivery appropriate English and Tamil language training materials...... 14 6.5 International standards of teaching and learning of both Tamil and English...... 14 6.5.1 English...... 14 6.5.2 Tamil...... 15 6.6 Methodology enhancement...... 15 6.6.1 English...... 16 6.6.2 Tamil...... 20 6.7 Sustainable and effective language training systems...... 20 6.7.1 English...... 20 6.7.2 Tamil...... 21 6.8 Answers to research questions 5, 6, 7, 8...... 21 6.9 Interim conclusions output delivery to date...... 22 7. Findings: three narratives...... 22 7.1 Narrative one the instructors...... 23 7.1.1 professional development...... 23 7.1.2 resources...... 23 7.1.3 future roles...... 23 7.2 Narrative two the trainees...... 24 7.2.1 English...... 24 7.2.2 Tamil...... 24 7.3 Narrative three communication skills and peace building in Sri Lanka...... 25 7.4 Answering the research questions...... 25 8. A final comment the question of project funding...... 27 ACKNOWLEDGEMENTS AND RECOMMENDATIONS...... 27 9. Acknowledgements...... 27 10. Recommendations...... 28 11. References...... 29 MCSP Evaluation Page 3 Dr C Tribble / www.ctribble.co.uk APPENDICES...... 31 1. Appendix: Original terms of reference for the MCSP project...... 31 2. Appendix: Comment on the terms of reference...... 34 2.1 Initial Research Questions...... 34 2.2 The problem with projects: language issues and project theories...... 34 2.2.1 Clarifying terms objectives...... 35 2.2.2 Clarifying terms goal, purpose, output...... 35 2.3 Interim conclusion: how adequate is the project's theory of change?...... 37 3. Appendix: Military Communication Skills Project Stakeholders...... 39 4. Appendix: Learning Support Centres current status...... 40 5. Appendix: Learning Support Centres (LSC) current status...... 43 6. Appendix: Tamil syllabus...... 44 7. Appendices: Print Cost Estimate (2007) – add 20% inflation for 2009...... 45 8. Appendices: Interview notes summary...... 45 9. List of meeting notes...... 46 10. List of observations...... 46
1. List of acronyms used in this report ELT English language teaching GCPP Global Conflict Prevention Pool ICELT In-service Certificate in English Language Teaching ICT Information and Communication Technology KDA Kotelawela Defence Academy (now University) KDU Kotelawela Defence University MCSP Military Communication Skills Project SCOPP Secretariat for Coordinating the Peace Process SLAF Sri Lankan Air Force SLMA Sri Lanka Military Academy TKT Teaching Knowledge Test ToR Terms of Reference
2. Report overview This report has the following structure: In the first section: PREPARATORY, I outline the research approach adopted for this evaluation. This section is followed by: FINDINGS (page 6). Here readers will find three main sets of findings: - Findings: Desk Review (page 6), - Findings: output delivery to date (page 11), and - Findings: three narratives (page 22). In this last section, impact is considered from the perspectives of instructors and trainees, and in terms of long term peacebuilding.
A final section Acknowledgements and Recommendations (page 27) summarises the main conclusions and recommendations which arise from the evaluation. This is followed be a set of: Appendices (page 30 and following) which includes a detailed comment on the terms of reference, and notes from all meetings and observations carried out during the evaluation.
MCSP Evaluation Page 4 Dr C Tribble / www.ctribble.co.uk PREPARATORY
3. Evaluation approach and research design 3.1 Overall approach Given the short time available for the study, and the logistical challenges involved in visiting the different establishments, I agreed with the project manager that we would focus on the limited set of stakeholders described above, and use three research strategies: document review to assess the relevance of the project design to project proposals and reports, documentation (including curricula, materials, and resources). semi-structured interviews to identify the extent to which different stakeholder interests have been met, and to build narrative accounts of what has worked for whom, in what circumstances, the three core questions in a realistic evaluation approach. observation to gain an insight into the extent to which those who have been trained through the project have been able to change behaviour as a result of their participation in the project, and the extent to which students perceive that the way they are being trained in their current courses differs from their earlier language learning experience. In the process of building the evaluation at least three potentially valuable narratives have emerged: Narrative 1: the instructors Narrative 2: the participants Narrative 3: communication skills and peace building in Sri Lanka We will draw on these narratives in looking at the impact the project has achieved so far. 3.2 Visual documentation
MCSP Evaluation Page 5 Dr C Tribble / www.ctribble.co.uk Figure 2 - trainees, KDU Alongside the interview and observation strategy outlined above, it was also agreed that I would provide a visual documentation of teaching and learning in the military training establishments. This documentation provides a further insight into the conditions in these institutions and of the achievements of the project. It is also a unique account of how things are, and can be, in language training for the Sri Lankan armed forces and will constitute valuable baseline date against which future progress can be assessed. Examples from the image archive in this documentation have been used to illustrate this report. 3.3 Evaluation schedule Following a document review undertaken in the UK, an evaluation schedule was agreed which has made it possible to meet with all major stakeholders. During the process we have conducted 24 interviews and to carried out 5 lesson observations in the main training centres for the Navy, Air Force, and Army, and in Kotelawela Defence University. The schedule for the evaluation is given below: 27 October Arrive Briefing meeting with British Council Director 28 October Travel to Habarana 29 October Naval and Maritime College SLAF China Bay 30 October Return to Colombo British High Commission 31 October SL Army Headquarters SL Navy Headquarters SL Air Force Headquarters I November SL Air Force Ekala National Languages' Commission 2 November Kotelawala Defence University Consortium of Humanitarian Agencies 3 November Tamil Instructors Tamil language consultant 4 November Leave for Bandarawela 5 November SL Military Academy SL Air Force Diyatalawa 6 November Return from Bandarawela 7 November Asia Foundation SCOPP
FINDINGS
4. Findings: Desk Review The main tasks which had to be carried out in the desk review were the clarification of some inconsistencies in the language of the Terms of Reference and in the project documentation that had been provided for the desk review process, and the development of a response to the first ToR research question: How adequate is the project's theory of change? 4.1 Revision of the research questions used in the evaluation An extensive review of the project documentation was carried out before the field visit (the full discussion can be found in Appendix 13). The first result of the desk review was a revision of the research questions set out in the ToR. The questions I have responded to in this review now read: Criteria Research questions Relevance 1) Was the project and the subsequent activities based on a well thought out and credible Theory of Change? 2) Are the project outputs relevant to the project’s stated goal and purpose? 3) How has the project adapted over time to changes in the external context? To what extent have internal planning, monitoring and evaluation processes contributed to this? 4) How has it contributed to meeting GCPP Sri Lanka Strategy targets? MCSP Evaluation Page 6 Dr C Tribble / www.ctribble.co.uk Effectiveness 5) Were the project main outputs produced? 6) Given the project goal and purpose, how better might project outputs have been formulated? 7) What hidden agendas if any may have driven project activities? Efficiency 8) How well have resources and activities been converted into outputs? Impact 9) What positive or negative impacts which are attributable to project activities can be detected at this stage? 10) How conflict-sensitive has British Council’s engagement been? 11) How is British Council perceived in Sri Lanka? Sustainability 12) How probable is it that long term benefits produced by the project will continue into the future?
4.2 Interim conclusion: how adequate is the project's theory of change? It was necessary to review critically all project documents in order to respond to this question (again, see Appendix 13). The original project document1 makes a strong case for a project to improve communication skills in English for the Sri Lankan military. It argues that at a higher level, the project - … will contribute (my emphasis) to the UK government's policy in relation to Sri Lanka “to strengthen incentives for lasting reconciliation, including through creating an environment conducive to reconciliation” - that key elements in this reform process are Defence, Judicial and Access to Justice - that English language skills are central to this process as they are: the medium of instruction in military training needed to improve professionalism of the forces a potential link between different communities in the long term and essential for interoperability in peace support operations
This original position is strengthened when the current priorities of the UK Government's Global Conflict Prevention Pool for Sri Lanka are taken into account2. Steve Ainsworth, Global Conflict Prevention Pool Advisor at the British High Commission summarised these as being: 1. A greater commitment to negotiated peace; 2. Improved safety and security in communities and adherence to human rights; 3. A governance reform agenda in key institutions, particularly the security services, to strengthen underlying conditions for a lasting peace; and 4. A strengthened civil society more effective in peace building. (my emphasis)
The original project focus on English has a clear relation to Priority Area 3, and as the evaluation has developed (see sections 6.2 and following) it has also become clear that improving the curriculum, testing and instruction in basic Tamil has the potential to make a major contribution to Priority Area 2. In the same way that The Language Exchange Programme in Guatemala and Belize is enhancing the capacity of opposing factions to communicate with one another, the MCSP project is now beginning to have an impact on the capacity of the Sri Lankan armed forces (and in the future, the police) to communicate in critical military / civilian interactions. The principle that communication skills are a necessary component in increasing the capacity of the armed forces seems to me to be reasonable. The way in which the integration of former Warsaw Pact countries into NATO has been facilitated through the Peace Keeping English Project is well documented3. The weakness in the current statement of the project's theory of change is that it appears to be far too ambitious. Documentation for the 2007-2010 project states that the project's goal is: To develop an environment within which long term peace and reconciliation can be effectively pursued and contribute to peace building in Sri Lanka However, a more reasonable aspiration would be:
1 Proposal for a Military Communication Skills Project in Sri Lanka (SL MilComSkill proposal 12Jun04.doc) 2 see: http://www.fco.gov.uk/servlet/Front? pagename=OpenMarket/Xcelerate/ShowPage&c=Page&cid=1062157895923 (accessed 9 November 2007) 3 see: http://www.britishcouncil.org/learning-elt-pep.htm (accessed 9 November 2007) MCSP Evaluation Page 7 Dr C Tribble / www.ctribble.co.uk To contribute to the development of an environment within which long term peace and reconciliation can be effectively pursued and contribute to peace building in Sri Lanka My recommendation is that such a wording is used in future project documentation, and future evaluations. The research questions which can then be asked can include one which asks: to what extent the project has enhanced the communication training skills capacity for the armed forces, and to what extent this has: - improved the effectiveness of the armed forces as modern agencies for peace-building and peace-keeping - improved military / civilian communication with the Tamil speaking minority in all areas of the island
5. Findings: The Military Communication Skills Project progress to date In this section I respond to the ToR research questions: Relevance 2) Are the project outputs relevant to the project’s stated goal and purpose? 3) How has the project adapted over time to changes in the external context? To what extent have internal planning, monitoring and evaluation processes contributed to this?
5.1 The relevance of the project to local context Although Sri Lanka now has two official languages, Sinhala and Tamil, with English as an official "link" language, most members of the Sinhala military have very low levels of knowledge of both English and Tamil. In the area of English language communication the key issues for the military are: all armed forces depend on English as the primary language for officer training the Air Force and Navy also depend on English for command and control all armed forces (again, especially the Air Force and the Navy) depend on English for advanced technical training, international liaison (including in the air and on the sea) and for international cooperation. the armed forces – and in particular the army have to compete for the best school leavers, and many new officer recruits have much lower levels of command of English than before as the parents of 18 year olds with good "O" and "A" levels prefer their children to take other options. (see appendices: 20071029b_SLAF_ChinaBay.doc, 20071029a_GI_Navy.doc and 20071031_Army_HQ.doc) The lack of capacity in the Tamil language has had a major negative impact on military / civilian communication in Tamil majority areas and throughout Sri Lanka as military staff from all services are responsible for checkpoints and security across the island for example, the Sri Lankan Air Force is responsible for security in the Colombo area (see appendices: 20071102_CHA.doc). The Military Communication Skills Project (MCSP) began as a response to this need amongst members of the armed forces for enhanced English and basic Tamil skills. To date, the project has focused on developing the English language skills of officer recruits in all armed services, and on the communication skills of recruits from other ranks in the Sri Lankan Air Force (SLAF), and latterly, the Sri Lankan Navy. This early emphasis was essential, as without establishing credibility in an area for which the UK has an established reputation for expertise (English language teaching) it would have been impossible to engage with the Sri Lankan military training authorities on the matter of Tamil instruction. In the third year of the project it was possible to begin discussions on the military's needs in Tamil instruction, and a new curriculum and training materials are now under development, with a Tamil trainer training programme planned for a new and final phase in the project. 5.2 Project time line and major milestones Major project achievements in the first two phases are summarised below. It is clear that the most of the work done so far has been focused on building English Language teaching capacity in the four military training establishments that are responsible for initial training of officers and technical recruits MCSP Evaluation Page 8 Dr C Tribble / www.ctribble.co.uk entering the armed forces. It has been considered to be neither feasible nor appropriate to extend the project's reach significantly beyond these institutions. Preliminary work on instruction in Tamil began in 2006 and is on-going. 2004 2005 2006 2007 September 2004 - Educational Technology - TKT courses completed - First workshops with PHASE 1 resources provided (28 successful trainees) Tamil Instructors - English is taught to - Preliminary work - SLMA Learning - On-going teacher officer cadets in all completed on Learning Support Centre opens training (ICELT) three services Resource Centre (LSC) / Trinco / China September 2007 - Basic levels of establishment Bay / KDA - Syllabus development equipment in place, but - Further teacher training - Preliminary research for the new 3 month inappropriate materials - Further materials in with Asia Foundation re Military English Course and lack of educational place Tamil language training book technologies and September 2005 September 2006 - 3 ICELT candidates knowledge in using - Teacher training PHASE 2 complete Module 1 (with these. ongoing - Research in Tamil 2 distinctions and Merit), - Project starts 1 - Teaching Knowledge training provision and 9 candidates now September Test (TKT) courses needs undertaken in completing - Visits to main started Military Training - book development for a institutions - Syllabus revision and Establishment basic Tamil course for - Initial Teacher training extension materials - In-Service Certificate in the military (with workshops for development English Language potential to extend to instructors from all 4 - ICT training Teaching (ICELT) starts the police) training centres with 22 participants - New teaching materials selected as interim resources for use during Phase 1 of the project
It is also clear that the project has changed its emphasis over time and has been able to take on a broader scope as trust has been established between the project team and senior staff in the three armed forces. The project documentation shows the transition as the detailed outputs have been extended to include Tamil and to increase the reach of the project. They also demonstrate how monitoring and evaluation processes have made it possible to review the unrealistic 2006-07 outputs and to transform these into the much more realistic 2007-2010 set. In setting out the outputs for the first two phases of the project, I include the original set, and an interpretation of these in terms of the tighter goods and services definition discussed above 2004-2006 Outputs Original Adapted - a bank of quality language learning - a bank of quality language learning materials for English: materials for English: - a network of trained teachers - a network of trained teachers - a learning support centre in a strategic - a learning support centre in a strategic location location - Security forces staff competent in use of English for access to education and training and for linking to the international community, with awareness of skills required for peace building issues.
2006-07 Outputs Original Adapted - An overall language and communication - An overall language and communication training plan for the military from training plan for the military from recruitment to retirement recruitment to retirement - Two more Learning Support Centres in - Two more Learning Support Centres in the Sir John Kotalawala Defence the Sir John Kotalawala Defence Academy (KDA) Ratmalana, and the Academy (KDA) Ratmalana, and the Army ( due to become Joint in 2007) Army (due to become Joint in 2007) Staff Staff College at Batalanda College at Batalanda MCSP Evaluation Page 9 Dr C Tribble / www.ctribble.co.uk - A team of Tamil language instructors in - A team of Tamil language instructors in the police and the military provided with the police and the military provided with suitable support suitable support - A bank of modern course materials - A bank of modern course materials including material on human rights and including material on human rights and peace building, including materials to peace building, including materials to support the learning of Tamil support the learning of Tamil - a CD Rom of ten modules of English - a CD Rom of ten modules of English materials for intensive courses materials for intensive courses developed with KDA for use in all training developed with KDA for use in all training establishments establishments - Advice for other agencies on how to use - A team of course instructors in the their resources effectively to build the military trained in English and trained in capacity of the police (and other forces) using modern teaching techniques and to develop communication training in equipment Tamil, train instructors, introduce contemporary materials (including those related to conflict resolution and human rights) and methods (including independent learning approaches). - Course instructors in the military trained in English and trained in using modern teaching techniques and equipment - Military personnel trained in use of English for access to education & training and for linking to the international community, with an awareness of the skills required for peace building issues
2007-10 Outputs - Beginner level package of Tamil students’ book, self-study workbook, teachers’ book and listening CD for a 45 hour course - A team of teacher trainers in the military to train Tamil instructors - Group of highly trained English instructors capable of taking on trainer training, teacher training and the provision of English courses for specific groups beyond the life of the current project - An English course book suitable for intensive courses in the military to be produced for Sri Lanka. The package would include a Students’ book of 150 hours, a Teachers’ book, CD and workbook. This would go with the material already produced at KDA to form a complete English course for the military. - Language Training Centre at KDA to cover the English training needs of all three services, including teacher training for new English instructors
5.3 Interim conclusions progress to date 2) Are the project outputs relevant to the project’s stated goal and purpose? Yes. Given the nature of the project's theory of change, the need to ensure the acceptability of a UK intervention in a sensitive area by the Sri Lankan authorities, and the expertise of the UK agency tasked with delivering the project, the initial emphasis on English was the right one, and good progress has been made in delivering relevant outputs. Although there has not been an explicit human rights / peacebuilding content to the programme to date, there is significant potential for a focus on these areas in the new Tamil language programme. As this course will deal explicitly with military civilian interaction, the emphasis on human rights and the military's peacebuilding role will be entirely appropriate. 3) How has the project adapted over time to changes in the external context? To what extent have internal planning, monitoring and evaluation processes contributed to this? The project has been successfully adapted over time. Internal monitoring and reporting has ensured that the team has been aware of where progress has been achieved, and where delays have been MCSP Evaluation Page 10 Dr C Tribble / www.ctribble.co.uk experienced, and the Project Board has been fully informed of changes in the schedule and new directions. The new emphasis on Tamil is a sensible extension from the initial project design, and the decision to create sustainable teaching materials is the correct one to make. 6. Findings: output delivery to date 6.1 Research questions In the following sections the research questions given below will be answered: Effectiveness 5) Were the project main outputs produced? 6) Given the project goal and purpose, how better might project outputs have been formulated? 7) What hidden agendas if any may have driven project activities? Efficiency 8) How well have resources and activities been converted into outputs?
Findings are separated into two major sections. In the first, I shall comment on the extent to which the major outputs to which the project is committed have been delivered. In the second, I will elaborate the four narratives discussed above: those of the students, the instructors, the training agencies and how all of this work might contribute to peace building. 6.2 Appropriate English and Tamil language training materials
Figure 3 - training materials, SLMA 6.3 English Stage 1 A decision was made at the beginning of the project to use published materials as the starting point for curriculum reform and teacher education. The core materials chosen (following extensive discussion with the Sri Lankan military instructors are Campaign4 and Cutting Edge5. The Campaign course has been developed through the UK supported Peace Keeping English Project in which a very large number of troops in former Warsaw Pact and other countries have been provided with English language training for military purposes. Cutting Edge is a general adult programme which instructors and the project manager felt would be an appropriate extension programme for Sri Lankan military trainees. Alongside these published materials, the project has provided a set of English language learning computer programs for use in the Learning Support Centres that have been developed as part of the project. With these materials in place it has been possible to: - introduce instructors to new teaching methods - enhance the learning experience of instructors - lay the foundation for development of language learning materials which cater specifically for the needs of the Sri Lankan armed forces.
4 see http://www.macmillanenglish.com/Course.aspx?id=28298 (accessed 9 November 2007) 5 http://www.pearsonlongman.com/cuttingedge/ (accessed 9 November 2007) MCSP Evaluation Page 11 Dr C Tribble / www.ctribble.co.uk It was recognised from the outset that these materials would not constitute a sustainable resource. However, without their use, the project could not have introduced immediate change, and could not have provided prospective course writers with clear examples of good practice which they could draw on when developing new instructional materials. The introduction of these new materials was warmly welcomed by the Air Force, Navy and Joint Services training institutions. The Sri Lankan Military Academy (SLMA) was not able to follow the same line as its sister organisations as there was already an agreement with the University of Saburagama to follow an English Language curriculum set by Professor Rajiva Wijesinghe, and which was based on books written by Prof. Wijesinghe and by a colleague in the university. After considerable discussion it was agreed that these teaching materials would be retained and that the SLMA would allow 50% of the curriculum to be based on the new, international materials. 6.4 Stage 2 6.4.1 The English language programme A new Joint Forces curriculum for intermediate to upper intermediate learners of English has now been agreed with instructors from the different training establishments and staff in the Kotelawala Defence University. This curriculum will have to be realised as a set of teaching materials for use across all officer training institutions in the 3 month Intensive English Course (although SLMA will continue to use its existing materials and curriculum until there is a programme review). It is reasonable to say that without these new training materials, the sustainability of the investment to date will be seriously compromised. With the materials, it will be possible for the current cohort of trained instructors to maintain standards, and to induct new colleagues into an agreed and feasible teaching programme. It is very important to note that these materials are not yet written. A great deal of planning has, however, been put in place and there is now: - A team of authors with capacity to develop the book - An experienced Sri Lankan production manager - An experienced text book illustrator - Editorial advice and copy editing support from Cambridge University Press, India - A production schedule - A distribution policy
Once initial sections of the book have been authored there will be a piloting phase which will also be used for initial teacher development, and as more materials become available for use they will provide the basis for trainer training so that the project team is in a position to ensure the successful dissemination and use of the books beyond the end of the project. The book itself will be made available to the Armed Forces on DVD for local printing, with an estimated print cost of around Rs 300. This low unit cost and the ability to have further print runs means that training participants can be asked to buy their own copy of the book, thereby ensuring long term sustainability. The Teachers' CD will have the full text of the book, the teachers' guide and the audio files needed for listening activities. Writing has now started on the book and there is every reason to expect that the final version will available by August 2009, with piloting and teacher training taking place during 2008 / 09. 6.4.2 Human rights and peacebuilding Within the new course it will be possible (through liaison with instructors in International Humanitarian Law in the military training institutions, and with INGO's such as the Asia Foundation (see Appendices, 20071107_Asia_Foundation.doc) to include materials which will focus on miltary / civilian interactions and thereby support peacebuilding agendas
MCSP Evaluation Page 12 Dr C Tribble / www.ctribble.co.uk 6.4.3 Tamil
Figure 4 - Tamil teacher & text book author Current situation Tamil language is presently delivered through an obligatory 50 hour programme. The general assessment of current teaching is that it is inappropriate as it focuses largely on the Tamil alphabet and grammar and offers no realistic basis for developing a competence in spoken Tamil. All major stakeholders recognise the importance of spoken Tamil in military / civilian interaction (a full analysis of meeting reports is available as an electronic document - see Meeting notes summary.xls). Typical comments include: - We have to understand the Tamil language in order to understand Tamil people and get information we need from them in military situations. (20071029a_GI_Navy.doc) - Tamil - need to do it in a more organised manner… It is as important as English - many of the problems could have been avoided if there had been more knowledge of Tamil … (20071031_Airforce_HQ.doc) - Priorities for a Tamil language programme from a military perspective. / checkpoint / searching / apprehending a suspect / getting information / asking directions (20071031_Army_HQ.doc)
As an example of why change is needed, the current approved Tamil curriculum for the Naval and Maritime Academy is given below: - writing vowel sound letters - writing consonant letters - using diacritics - writing mixed letters - pronunciation of the letters - writing Tamil letters to Sinhala letters - reading – our house - reading – our village - reading public notice boards - days - basic numbers (1-100) - personal pronouns - introducing basic spoken words - questions and answers
I make no apology for including the full text of the document here as it clearly indicates the need for curriculum renewal in Tamil instruction. This situation is compounded by a lack of appropriate, qualified instructors, the need to use Sinhalese instructors for Tamil in sensitive areas where security requirements preclude the admission of Tamils to military premises, and the lack of any effective instructor training. Next stages A revised Tamil curriculum has now been agreed with a group of instructors from military establishments and a consultant (Mr Yogarajah) recommended by the National Languages Commission. The curriculum that this group has prepared (see Appendix 16) addresses the areas MCSP Evaluation Page 13 Dr C Tribble / www.ctribble.co.uk which concern military staff i.e. patrol, checkpoint, search, transit etc, as well as ensuring that trainees have a minimum grasp of Tamil letters and (through transliteration) are better able to speak and understand in critical military/civilian interactions. Course book writing has now begun and sample units are already available. Although the team of Tamil teachers who have been brought together to undertake this task were initially sceptical about this (for them) radically new approach, thanks to the excellent advice from Mr Yogarajah and Marion Hughes, they are now fully committed to the approach. In a group interview (see appendices: 20071103_Tamil_authors.doc) they commented: - For the first time we are teaching the spoken language rather than beginning with the alphabet and grammar - Students find it easier to grasp ideas working this way and are enjoying our classes more - Using Sinhala transliteration is helping students to get the pronunciation right.
Book production will be managed by the same team outlined above, and it is realistic to expect that the Tamil book will be piloted and available for use by August 2008. 6.4.4 Interim conclusion: output delivery appropriate English and Tamil language training materials The project team has made excellent progress in preparing the ground for the production of locally authored, topically relevant and affordable teaching materials for both English and Tamil. However, none of these materials are fully developed, and a great deal of critical teacher and trainer training will have to be done before the new books will constitute a fully sustainable resource. With sufficient cooperation from the Armed Forces, the programme to train Tamil teachers could be satisfactorily completed by August 2009. However, for the English Language course book to be fully embedded in the military English Language system, it will be advisable for a period of 12 months beyond the publication date to be available for teacher and trainer training. I would therefore recommend that: funding for the project is maintained until at least August 2009. Without such funding the sustainable outcomes of the project will be in serious jeopardy. If possible, funding should also be provided for a final year of assured funding (to 2010) so that the new materials can be fully embedded in the system. As there will be little need for further investment in resources by this stage, it should be possible for the final year to operate on a slightly reduced budget compared with 2008-09. 6.5 International standards of teaching and learning of both Tamil and English
Figure 5 - trainer, NMA, Trincomalee A central strand of the project has been to ensure that teaching of English and Tamil more fully meet the needs of trainees and their organisations. 6.5.1 English To this end, an early priority has been the professional development of the English language instructors in military training establishments. This has been achieved through three processes: a series of teacher development workshops focused on the use of Campaign, Cutting Edge, and the software suite installed on the Learning Development Centre computers the preparation of all instructors for the Cambridge Teaching Knowledge Test (TKT), an internationally recognised professional updating qualification
MCSP Evaluation Page 14 Dr C Tribble / www.ctribble.co.uk the preparation of a smaller number of experienced instructors for the Cambridge In-service Certificate in English Language Teaching (ICELT), an advanced language and methodology course delivered through two modules. Module 1 (now completed by 10 instructors) focuses on language knowledge. Module 2, so be completed by 6 of the instructor cohort, focuses on methodology. Those who have completed Module 2 will be in a very strong position to take a leading role in text book authoring and teacher training. Given the various starting points of the instructors in the different training institutions (some with no degree, others with BA, others with post-graduate qualifications in ELT), I am in agreement with the approach taken in the project. Staff trained through the project are much better placed to provide appropriate levels of teaching and learning in English and are able to meet international standards. 6.5.2 Tamil In the case of Tamil, this has been more difficult to achieve. There is no internationally validated programme for Tamil language instructors, and the project, to date, has had much less contact with this group. The current plan is to use the new teaching materials for Tamil as a catalyst for change in Tamil as a second language instruction, and to base future Tamil teacher development programmes on the introduction and application of the new Tamil Language programme. Although this trainer training programme will not have the same level of impact as the professional development offered to English language instructors, it will, nevertheless, constitute a sustainable training and teaching resource for basic spoken Tamil for the armed forces. 6.6 Methodology enhancement
Figure 6 – Trainees, KDU Under this section, delivery of the following outputs is reviewed: - interactive and democratic classroom approaches - up-to-date materials which are effective for communication training and which raise awareness of peace building issues; - upgraded teaching & training skills;
The MCSP project design specifies a number of outputs all of which can be seen as contributing to changes in classroom methodology. For the purposes of this report they will be reported on under the one heading. Project impact to date will be reviewed for both English and Tamil
MCSP Evaluation Page 15 Dr C Tribble / www.ctribble.co.uk 6.6.1 English
Figure 7 - trainees, SLAF China Bay The teacher training programme put in place through the project has had a dramatic effect on the instructors capacity to use pair and group work, individualised task based training, and integrate ICT into language teaching. This has significantly enhanced the English Language instructors' ability to motivate and support trainees. Four kinds of evidence are available to support this assessment: comments from the instructors, comments from participants, classroom observation results and the visual documentation: Instructor comments The comments from instructors given below are typical of the reaction of all those met with during the evaluation. Detailed summaries of these discussions are available in the meeting notes: - Group work had been part of the process, but now syllabus has encouraged greater emphasis on interaction between the participants (20071105_SLMA_CO.doc) - Project improved the trainers - we were neglected before because it was assumed that the instructors had qualifications - but there were no upgrade and training opportunities before. Now we have had real professional development (20071101_SLAF_Instructors_GI.doc) - Instructors feel that they can give students opportunity to learn so we feel that we are no longer dictators in the classroom…(20071105_SLMA_GI_ELT.doc)
Trainee comments Importantly, trainees' comments confirm the positive accounts of their instructors. In a series of group interviews (carried out without the presence of instructors) trainees reported that they found the English language training they received in the military training establishments contrasted strongly with their experiences at school, and was highly valued. Typical comments include: - Here we can study well on our own - but at school teachers just teaching (20071105_SLMA_GI.doc) - At school we just studied the subject - here we can use the language practically (20071105_SLMA_GI.doc) - English at school the teachers had good knowledge but it was better at Ekala (20071105_SLMA_GI_SS_2.doc) - We can extend beyond classroom work with supplementary materials (20071029a_GI_Navy_T_S.doc)
Visual documentation The images below demonstrate some of the key features of this transformation.
MCSP Evaluation Page 16 Dr C Tribble / www.ctribble.co.uk Effective group work
Effective use of information technology
MCSP Evaluation Page 17 Dr C Tribble / www.ctribble.co.uk The teacher as facilitator
MCSP Evaluation Page 18 Dr C Tribble / www.ctribble.co.uk Independent study
I fully recognise that the examples given here cannot be taken as typical of every English lesson taught in military training institutions. They are, however, a very strong indication of what teachers consider to be good practice and offer a firm basis for the conclusion that the instructors observed are fully aware of contemporary approaches to language learning and teaching, and are fully able to implement these approaches. Observation During the five observations I was able to carry out during the evaluation (see Appendices) I was looking for evidence of the following indicators of good practice: 1. more than one skill is addressed during the lesson 2. pair work is used appropriately 3. group work is used appropriately 4. the teacher uses English most of the time. 5. the teacher has adapted the textbook (appropriately). 6. the teacher uses a variety of focus and types of activities. 7. the trainees are given an opportunity to “think” rather than “parrot”. 8. the teacher shows creativity in the use of limited resources. 9. the trainees are challenged and there is clear learning. 10. the teacher has planned an effective lesson – evident through actual lesson plan and effective delivery of the lesson.
Although the classes observed varied in terms of the materials used, the levels of participants, and the focus of the lesson, there was, without exception, always evidence of good to excellent use of at least 7 out the 10 indicators listed above. From a wide experience of teacher development processes world-wide, I take this as a strong indicator of the overall impact of the MCSP to date on the way in which English is taught in military training institutions.
MCSP Evaluation Page 19 Dr C Tribble / www.ctribble.co.uk 6.6.2 Tamil
It is too early to comment on the extent to which the project has had an impact on methodology for teaching Tamil as a second language. In this area, there will be considerable challenges: - the project will have to act through intermediaries, many of whom are not English speakers - many potential Tamil instructors will not have teaching backgrounds or qualifications - many Tamil instructors are not mother tongue speakers for reasons of military security, and also to ensure that people are not put in harms way (Tamils or Muslims working for the military can be targeted as collaborators by the LTTE).
Despite these challenges, these are grounds for optimism: - The Basic Tamil programme will have a clear, simple teachers guide - the structured programme in spoken communication and the availability of listening materials will reduce the burden on teachers with limited command of Tamil - the methodological design of the materials will encourage good teaching practices
It will, however, require a fuller evaluation at a later date to assess the extent to which Tamil instruction in the military has been strengthened through the MCSP. 6.7 Sustainable and effective language training systems. 6.7.1 English Sustainability is being worked towards in a number of ways: training materials, the Learning Support Centres, teacher development, institutionalisation training materials The most significant sustainability mechanisms will be the sets of locally authored, copyright free Joint Services English Language and Tamil Language training programmes. These will be available on CD ROM for local printing, with unit costs in the range of Rs 150 to Rs 300 depending on whether they are reproduced in colour (see Appendix 18). By selling copies of the book at cost price to trainees, the MCSP will provide a sustainable mechanism to support the long term delivery of quality English teaching in military training institutions. the Learning Support Centres Alongside the materials, the Learning Support Centres will continue to provide an essential resource during the remainder of the project, and beyond this if the military are able to maintain a basic level of funding. At the moment, (see Appendix: Learning Support Centres page 40) the Learning Centres offer a unique resource to learners and teachers in Sri Lankan military training institutions. The main limitations to the Centres' capacity to support training are summarised in Appendix: Learning Support Centres (LSC) current status (page 43). Key issues are:
MCSP Evaluation Page 20 Dr C Tribble / www.ctribble.co.uk - There are not enough computers in SLMA and Ekala LSCs. They have eight in each place with double headphones but the class sizes are such that at least four more computers in each place would be better. - Students should be encouraged to use the LSCs in the evening and in any free time available. For the most part they do not have much free time so the LSCs are not being used as Self Access Centres except in the evenings.
Although these limitations have restricted the impact of the LSCs, nevertheless, they continue to give the project important profile with local and national management, and provide language learners and teachers with resources that have never been available before in military training institutions. teacher education In the long term there will be a need for the creation of a training resource in English language teaching for military purposes. The English staff in KDU have the potential to be such a resource and the project manager is now working with this group and the management at KDU to investigate the possibility of: - in-service professional development courses for English language teachers entering the military - a BA in English language teaching which could form the basis for long term teacher supply into military training institutions (and also support the training of police and other members of the security forces)
6.7.2 Tamil The project team has had to take a slightly different approach to ensuring the sustainability of the Tamil programme, as the teachers who will be involved will, typically, have a lower level of academic qualification (or no qualification) as teachers, and the level of the Tamil language course will be very basic. The approach taken so far is based on ensuring the institutional acceptability of the basic Tamil for Military / Civilian interaction course through cooperation with the National Languages Commission (NLC) and its Director Mr Senerath Gunasena. In addition, by including Tamil instructors from the armed forces as materials writers has been possible to establish a strong sense of ownership of the course book and the instructional approach. This cooperation includes: - the use of a specialist consultant (Mr Yogarajah) recommended by NLC to lead the course writing team - the use of NLC consultancy in the development of the test for the Basic Tamil programme - the publication of the Tamil course on CD ROM for local, low cost printing.
6.8 Answers to research questions 5, 6, 7, 8 Before moving to the next interim conclusion, it will be helpful to give brief answers to the research questions which headed this section 5) Were the project main outputs produced? Not yet. An appropriate but unsustainable set of resources has been put in place and has been an essential catalyst for change. The Learning Support Centres are in place and are highly appreciated, but they are not a sustainable resource in the long term unless the Sri Lankan military makes a commitment to their long term funding. Good work has been done on the new Tamil materials introduced from 2006-2007 and the foundations for the new English 3 month intensive course are in place, but more time is needed for these to be completed. 6) Given the project goal and purpose, how better might project outputs have been formulated? I have already discussed the problems caused by the confusion of categories in early project documents. However, I feel that the overall statement of outputs is appropriate, and if they are fully delivered, they have an excellent chance of ensuring that the project's purpose is achieved in a sustainable manner. The only where there may be a question mark is the issue of the extent to which explicit peace-building content could have been built into the project thus far. From discussions with project stakeholders, my conclusion is that it would have been entirely inappropriate to start off teaching conflict resolution and similar topics in the early stages of the project. As it is, the project is now well placed through the development of the new 3 month intensive English course and the Basic Tamil course to ensure that, where relevant, conflict resolution and peace building topics can be MCSP Evaluation Page 21 Dr C Tribble / www.ctribble.co.uk included in the programme. It is my opinion that this has been the correct way of moving forward in this area. 7) What hidden agendas if any may have driven project activities? None. The only possible hidden agenda has been the need to focus on English before it was possible to focus on Tamil, but this has not been done in way that causes any risk to the project (quite the reverse) and has not diverted the project from its core purpose. 8) How well have resources and activities been converted into outputs? Very well. The British Council has dramatically added value to the project through its imaginative use of re-conditioned computers (the project deserves a green star for this!), the use of surplus furniture from British Council and High Commission sources, and through the negotiation of excellent terms with the project's software supplier. In all of these areas the project team deserves significant praise. 6.9 Interim conclusions output delivery to date Overall, the project has succeeded in delivering appropriate outputs in a timely and well managed way. I can also add, from my own experience of seeing the team at work with key stakeholders, that they have: a) managed the project in a remarkably sensitive and effective way, with the Project Manager and her assistant being able to negotiate the best possible conditions for the instructors and trainees, thereby ensuring the enthusiasm and commitment of the whole team, and b) added significant value to the project by being able to draw on the prestige of the British Council in Sri Lanka, and the resources of the British Council and the British High Commission when it has come to obtaining surplus IT resources, furniture, and even paint!
7. Findings: three narratives In the following section I shall attempt to answer the following research questions: Impact 9) What positive or negative impacts which are attributable to project activities can be detected at this stage? 10) How conflict-sensitive has British Council’s engagement been? 11) How is British Council perceived in Sri Lanka? Sustainability 12) How probable is it that long term benefits produced by the project will continue into the future?
In Narratives in social science research (Czarniawska 2004) an approach to dealing with interview and other qualitative data is outlined which both legitimates and problematises the 'stories' which each researcher necessarily constructs as they make sense of the data he or she has to confront. In the following sections, I shall outline three narratives that are drawn from the interviews and observations which have been carried out during this study. I fully accept that these are subjective interpretations of the experience that I have had during this evaluation, but I also hold that – because it is evidence based this account makes a valid and useful contribution to the evaluation of the Military Communication Skills Project. Indeed, at this point in the project's development, it is perhaps the best way of bringing together the different strands of a story which is not yet complete.
MCSP Evaluation Page 22 Dr C Tribble / www.ctribble.co.uk 7.1 Narrative one the instructors
7.1.1 professional development The English and Tamil instructors constitute a group which is in many way marginalised within the military structures which employ them. Some of them have volunteer status, some are civilians, none are career fighting officers, yet they have a pivotal role in the professional development of the officer cadets and recruits with whom they work. As English instructors they are appointed for their expertise in English language teaching but most have only college qualifications in English language, and the only methodology training that they have had has been for primary or secondary level. As Tamil instructors, many have only had a 3 month intensive training in Tamil followed by a short and very restricted general training in instruction (of little value from a language education perspective). Within MCSP many of the instructors are receiving the first professional development that they have experienced since qualifying, and some are gaining the opportunity to be materials developers and trainer trainers. They see this as a significant opportunity. 7.1.2 resources An additional benefit for the instructors in MCSP is that the English Instructors have been able to start working with relevant contemporary materials and to learn new skills in the use of Information and Communication Technologies (ICT) in Language education. These new resources have enhanced both the instructors' and the learners' motivation, and have given the instructors a new status within their institutions. There are some risks inherent in this process, as the instructors can also be the focus of jealousy within organisations that are starved of resources, but overall, the benefits outweigh the risks. The Tamil instructors as yet have not been able to benefit from new resources, but once the new Basic Tamil programme is in place, they will be able to provide a more appropriate service to their students and their professionalism will be more fully recognised. 7.1.3 future roles If the MCSP stopped in August 2008, the English instructors would be left as guardians of a slowly depleting asset. Books would deteriorate, ICT resources cease to function, and after a small number of years, many of the benefits that have been introduced through the project would have been lost. If the project is able to deliver the sustainable benefits of the new 3 month intensive course they will, in contrast, be empowered to deliver a coherent, locally relevant and affordable programme for many years to come. After time (5 to 6 years?) there may be a need to renew the curriculum and up-date the content of the programme, but the group will have the skills to do this, and the electronic template of the existing book will have sufficient flexibility to allow updating and re-printing. The same applies to the Tamil instructors. Leave them as they are, and the majority will be providing an inappropriate training programme with ineffective methodology to learners who gain little or nothing from the process. Deliver the project's outputs, and there is a much better chance that learners will be equipped with the basic skills to begin to establish positive relationships with their Tamil speaking fellow citizens.
MCSP Evaluation Page 23 Dr C Tribble / www.ctribble.co.uk 7.2 Narrative two the trainees
7.2.1 English The trainees have very little time in which to achieve a great deal. In English, they are supposed to gain sufficient grasp of the language to be able to engage in training, international cooperation, and command and control. In Tamil, after 50 hours of classes, they are supposed to be able to man road- blocks, conduct searches, pass through Tamil speaking areas, arrest and question suspects, and ensure the security of their own bases. Before MCSP, the trainees were being taught with outdated materials which had little or no relevance to their military careers. They had high levels of need, and low levels of satisfaction with the instruction they received. MCSP has revolutionised this experience, but it is not enough. Exit performance at the end of the 3 month English training period is still too dependent on the entry levels of the students you can make only so much progress in this short period, and students who start from a low base cannot be expected to gain adequate competence in the target language. Similarly, writing skills, presentation skills, and other advanced communication cannot be fully developed in the scope of the 3 month course. In the long term, there will be a need for additional specialist courses to support the growing communication needs of students in the military. This implies the need for continuing professional development for trainers, the development of a core trainer training resource (most probably at KDU), and a qualification pathway to encourage instructors to take part in this training. 7.2.2 Tamil The story of English students resonates with that of those attempting to learn Tamil but is amplified by the urgency of the need, and the inadequacy of the current resources. If your primary need for Tamil is to be able to survive in an environment which is often hostile but which is also part of your home country, deal with fellow citizens who might also be your enemy, and ensure the security of your unit and your colleagues in the face of suicide bombings and conventional attack, you don't want to be spending the 50 hours that are available for Tamil instruction in learning the alphabet and grammar of the written language. Yet this has been the core of the Tamil language programme to date. MCSP has the potential to change this, but it has, as yet, not been able to deliver change as it too early in the project process.
MCSP Evaluation Page 24 Dr C Tribble / www.ctribble.co.uk 7.3 Narrative three communication skills and peace building in Sri Lanka
The Military Communication Skills Project has, at this moment, made little or no direct contribution to peace building in Sri Lanka and that is entirely as it should be. The project's theory of change does not expect to make sudden and dramatic transformations. Instead, the project is beginning to produce: - instructors with the capacity and the resources to teach varieties of English which are of professional relevance to their students - instructional materials and resources which are motivating and engaging for learners - cadres of materials developers who will be able to offer alternative learning platforms to future students - officer cadets with the language skills needed to engage in future studies in areas which will make a direct contribution to peace building on the island
Each of these changes is leading to the achievement of the project's stated purpose: To improve the quality of language & communication skills for military forces in Sri Lanka If one accepts the project's theory of change, then the MCSP is making a contribution to peace building. The project has the same potential to do this for basic Tamil, an area where there is even greater potential for an immediate peace building return on investment, as military / civilian communication is of such critical importance. But why the emphasis on English? Although it is possible to accept the long-term potential of enhanced English language skills in increasing the international engagement of the Sri Lankan military, and enhancing the awareness of officers of international humanitarian law, wouldn't it have been better to start off with Tamil? The answer here has to be that this is a British High Commission initiative, which is implemented by the British Council. Senior staff in the Sri Lankan armed forces have a very high regard for UK expertise in English language teaching, and a great respect for the British Council and its teaching centres in Colombo and Kandy. In English language teaching, the British Council has no equal on the island. However, this is not the case for Tamil, and it would have been completely inappropriate to have begun an initiative to enhance communication skills training for the armed forces by attempting to reform the teaching of Tamil in the first instance. In a one sense, this answers the question of how conflict sensitive the British Council has been in developing this project. Had it attempted to jump directly to Tamil, alliances would have been damaged, reputation harmed, and the UK would have been seen as a heavy handed post-colonial force, attempting to impose solutions on a sovereign state. As it is, having demonstrated competence and commitment in the area of English language communication skills, the project has been welcomed as a partner in the urgent task of improving training in Tamil.
7.4 Answering the research questions The three narratives briefly outlined above, provide clear answers to the research questions posed at the outset of this review process.
MCSP Evaluation Page 25 Dr C Tribble / www.ctribble.co.uk 9) What positive or negative impacts which are attributable to project activities can be detected at this stage? Positive impacts are currently detectable in the areas of methodology, resource upgrade, and the management of educational processes, as well as in learning outcomes for students. It will take some time before sustainable transformations are put in place at an institutional level, but the foundations have been firmly laid. 10) How conflict-sensitive has British Council’s engagement been? Very. By offering key stakeholders what they feel they need, the British Council has begun the process of change which all involved in this project hope can be carried through to completion. Perhaps the most significant long term change that MCSP will bring about is through the Tamil programme. This new programme has the potential to make a far-reaching impact on Military / Civilian interaction, and to do this in a way which meets the conflict resolution imperative of do no harm. By providing teaching resources and training to Sinhala mother tongue teachers of Tamil, it will begin to be possible to provide appropriate language training for military personnel in highly sensitive areas of the country without compromising the personal security of Tamil citizens. I feel that this is a point which should be stressed when considering the extention of the funding of the MCSP through to 2010. 11) How is British Council perceived in Sri Lanka? Very positively. The project has confirmed the Council's professionalism in language education and its commitment to the long-term strengthening of Sri Lankan institutions. 12) How probable is it that long term benefits produced by the project will continue into the future? At the moment, the benefits that the project has produced are vulnerable. If consolidated during the next two to three years, it is likely that the benefits will be institutionalised, long term and renewable. The main benefits and challenges associated with MCSP from the perspective of those interviewed during the review process are summarised below. This quantification was drawn from the interview data which is given in the appendices (an Excel Spreadsheet with this analysis is available as an electronic appendix). To date, the main benefits which interested stakeholders identified were: professional development 27 improved student performance 16 teacher confidence 4 reaching more students 1
The 27 comments relating to professional development reflect the emphasis of the project so far, and underscore how appropriate this strategy has been. Stakeholders also reported that they already feel that this is having an impact on student performance. The issues which they identified included: lack of appropriate resources 8 group size 5 entry level 4 recruitment / retention 4 tamil curriculum 4 conflicting responsibilities 3 scheduling 3 administrative barriers 2
Here we can see some of the areas where the impact and sustainability of the project may be reduced. The first two areas mentioned are likely to be critical over the next two years. With increased intake into the armed forces, staff on the 3 month training courses in the Airforce and Navy (where all recruits received English language training) are going to face significant challenges. While it is well beyond the scope of the project to attempt to fix this problem, it is important that managers are aware that a mix of increased class size and the increased load on instructors who also have to carry out military duties, could jeopardise the mid term success of the project.
MCSP Evaluation Page 26 Dr C Tribble / www.ctribble.co.uk An additional challenge will also arise when project funding ends to ensure that the Learning Support Centres continue to receive support from the Sri Lankan side to maintain technological capacity and for for learning resources to be renewed. 8. A final comment the question of project funding I have already discussed at length the language issues identified in the project documents which have framed the MCS Project. In these final comments, I also wish to comment on the way in which a short funding cycle (basically no assurance of funding beyond one year from the project start) has restricted the scope of the project manager to work systematically towards long term sustainability. In the first two years it was possible to train staff and to introduce good practice in language teaching. However, as there was only a guarantee of funding year-on-year, it was impossible to work on the development of sustainable resources in this phase. In 2006-07, the one year of assured funding once again did not provide enough time to create a new 3 month intensive English courses and develop the Tamil programme. Fortunately the funding has been provided through to August 2008, but the reality is that, if the overall investment in MCSP is to produce long term institutional change, the project needs to run until at least 2009. It will be impossible to write, pilot and produce the essential training materials for Tamil and English in less time, and, even more important, time will still be needed to train instructors to work with the new materials. Similarly, if trainer training capacity is to be developed at KDU, there will be a need for funding through until at the very least the end of 2009, and preferably the end of 2010.
ACKNOWLEDGEMENTS AND RECOMMENDATIONS
9. Acknowledgements I would like to express my very sincere thanks to the MCSP Project Manager, Marion Hughes, and to Hamzi Haniff, British Council Assistant Manager, Programme Delivery, for their exceptional professional and personal support throughout this evaluation process. Their profound knowledge of the stakeholders and the context have been invaluable, and their attention to detail in managing the complex processes of this review have been exemplary. I would also like to thank Marion Hughes (and her husband Jim) for their kind hospitality during the evaluation, and finally, to thank British Council Director Gill Westaway for her professional interest in the evaluation, and the kind hospitality she and her partner Nina provided at the end of an exhaustive and, at times, exhausting evaluation process. Finally, I would like to thank the Steering Committee for its commitment to the evaluation process, and the Directors of Training in the institutions that we visited during the review for their cooperation and kind hospitality at a very busy time for the armed forces of Sri Lanka.
MCSP Evaluation Page 27 Dr C Tribble / www.ctribble.co.uk 10. Recommendations
Recommendation 1: project design It will be helpful if any final project document for the MCS Project is worded more consistently than has been the case in the past (see the extensive discussion in Appendix 13 Appendix: Comment on the terms of reference). I would recommend the following: GOAL To contribute to the development of an environment within which long term peace and reconciliation can be effectively pursued, and contribute to peace building in Sri Lanka. PURPOSE: A sustainable and effective language and communication training system for Sri Lankan military forces OUTPUTS - appropriate and sustainable English and Tamil language training materials which address the communication needs of the Sri Lankan armed forces in their role of ensuring peace and human rights for all citizens - pre- and in-service training programmes and resources to ensure: that instructors are able to make effective use of the new training materials in Military Institutions the induction of new staff
Recommendation 2: project evaluation If this project structure is put in place, an evaluation plan should be put in place which responds to the following research questions: To what extent has the project enhanced capacity in communication training skills for the armed forces? To what extent has the project:
MCSP Evaluation Page 28 Dr C Tribble / www.ctribble.co.uk - improved the effectiveness of the armed forces as modern agencies for peace-building and peace-keeping and - improved the capacity of the military to interact with the Tamil speaking minority in all areas of the island?
Recommendation 3: funding To ensure continuity in planning to meet these major tasks, it will greatly assist the project team to a) implement the final stages of the project and b) put in place a coherent closure plan if: funding for the project is maintained until at least August 2009. Without such funding the sustainable outcomes of the project will be in serious jeopardy. and, if possible, funding can be provided for a final year so that the new materials can be fully embedded in the system. As there will be little need for further investment in resources by this stage, it should be possible for the final year to operate on a lower budget compared with 2008- 09. Recommendation 4: management Given the growing involvement of MCSP in Tamil Language training, it would be helpful if there could be a Ministry of Defence representative on the Steering Committee to make possible the participation of the Kotmalle Tamil Language Training in the development of the Tamil communication skills programme Recommendation 5: future training capacity Given the long term needs of the Sri Lankan military for communication and language skills training, there is a very strong argument for the development of a BA programme in either Language Teaching (for both English and Tamil), or, perhaps more interestingly, the establishment of a BA or BsC programme in Education with opportunities for specialisation in Languages or in other aspects of training (including technical and military subjects). Such a programme could provide the basis for a future Sri Lankan Education Corps a great lack at the moment. By developing this capacity in KDU, the long term training needs of the Sri Lankan military could be greatly enhanced. Although it goes beyond the remit of this evaluation to make a formal recommendation in this area, I would like to suggest that the Steering Committee actively investigates possibilities in this area as a means of ensuring the long term sustainability of the benefits that have arisen through the MCSP.
Dr Christopher Tribble London, 11 November 2007
11. References Pawson, R. and N.Tilley, (1997) Realistic Evaluation, London: Sage Czarniawska, B. (2002) Narratives in Social Science Research (Introducing Qualitative Methods series) London: Sage
MCSP Evaluation Page 29 Dr C Tribble / www.ctribble.co.uk MCSP Evaluation Page 30 Dr C Tribble / www.ctribble.co.uk APPENDICES 12. Appendix: Original terms of reference for the MCSP project
Evaluation of The Military Communication Skills Project (supported by The British High Commission)
Background
In June 2004, the British High Commission in Sri Lanka committed to fund a proposed one to two year Military Communication Skills Project requested by the military and designed by the British Council (the B.C.). The Military Communication Skills project was planned “to support the defence reform and peace building components of the security sector transformation programme in Sri Lanka”. As outlined in the initial Project Proposal, the Military Communication Skills Project was designed as a one to two-year activity. Consistent with funding requirements under the Global Conflict Prevention Pool, the British High Commission provided assistance covering two years of the Program. In May 2006, the B.C. submitted a proposal seeking assistance for a third year of the project, consistent with the initial design. Following detailed negotiations with the British High Commission for support for a third year of the Military Communication Skills Project was agreed to in August 2006 and funding was provided in September 2006. The Military Communication Skills Project
The Goal of the project is: to assist in the achievement of the UK’s Peace Building Strategy which is to “ support conditions necessary for a sustainable peace in Sri Lanka”. The purpose of the project is: To improve the quality of language and communication skills for military forces in Sri Lanka through - design & development of appropriate English and Tamil language training materials; - promotion of international standards of teaching and learning of both Tamil and English - interactive and democratic classroom approaches; - use of up-to-date materials which are effective for communication training and which raise awareness of peace building issues; - upgrading of teaching & training skills; - development of sustainable and effective language training systems
The main outputs of the project are: 1 a bank of quality language learning materials: appropriate for specified target groups of security forces; designed to contemporary standards using modern approaches; making appropriate use of a range of ICT & education technologies; using content which raises awareness and develops understanding of key issues such as human rights and respect for international law; including materials for follow-up self study.
2 Six learning support centres in military training establishments acting as: - focus for training activity, - resource for development of locally appropriate teaching materials - learning resource for students
MCSP Evaluation Page 31 Dr C Tribble / www.ctribble.co.uk 3 Military officers and recruits competent in use of English for access to education & training and for linking to the international community, with an awareness of the skills required for peace building issues.
4 A network of trained teachers: - with good levels of language and training skills; - able to use a communicative, facilitative and democratic methodology; - equipped with a “survival pack” of teaching material; - able to disseminate awareness of key issues such as human rights and respect for international law; - with appropriate certification and recognised status.
Project Activities
The project was developed through extensive consultations with military training establishments and the military Directors of Training. The military training establishments focused on were:
Sir John Kotelawala Defense Academy Defence Services Staff College, Batalanda Sri Lanka Military Academy, Diyatalawa SLAF Base, Diyatalawa SLAF Base, China Bay, (Junior Command & Staff College, Combat School, and NCO Management School) Naval and Maritime Academy, Trincomalee SLAF Base, Ekala, (Advanced Trade Training School), SLNS Nipuna, Boossa SLNS Gemunu, Welisara SLNS Sheksha, Poonewa
Overall objectives of the evaluation
To evaluate British Council’s project according to the criteria and research questions below6.
Criteria Research questions Relevance Was the project and the subsequent activities based on a well thought out and credible Theory of Change? Are the project objectives relevant to the project’s stated goal and purpose? How has the project adapted over time to changes in the external context? To what extent have internal planning, monitoring and evaluation processes contributed to this? How has it contributed to meeting GCPP Sri Lanka Strategy targets? Effectiveness Were the project objectives achieved? Given the project goal and purpose, how better might project objectives have been formulated? What hidden objective(s) if any may have driven project activities? Efficiency How well have resources been converted into activities/outputs? Impact What positive or negative impacts which are attributable to project activities can be detected at this stage? How conflict-sensitive has British Council’s engagement been? How is British Council perceived in Sri Lanka? Sustainability How probable is it that long term benefits produced by the project will continue into the future?
6 These criteria are based upon those of the OECD-DAC Guidelines for Evaluations but have been elaborated MCSP Evaluation Page 32 Dr C Tribble / www.ctribble.co.uk Scope of the work Initial project briefing and agree Terms of Reference (2hrs) Familiarisation with project content and background, its goals, aims and objectives from project documents. Briefing from BC, BHC and steering committee teams plus additional reading (list of reports attached. (2 days) Development of more detailed methodology for the evaluation (2 days) Travel to Colombo for interviews with British Council, BHC, military training staff, project partners, direct beneficiaries and officials (6 days including travel) Production of draft report and verbal briefing for to steering committee (2 days), Distributed to steering committee for comments Production of final report (2 days) Total: 14 days
Expected Outcome and Deliverables At the end of the review the consultants will provide an insightful report which clearly shows the progress that has been made by the project over the duration of the project. It should highlight the project’s strengths and successes and also areas of weakness and recommend where greater energy should be focused in the future. It should provide an objective overview of the successes (or failures) of the project. The project steering committee is the client for this evaluation and the consultant reports to them. Further dissemination of the report is the decision of the Steering Group.
Conduct of the Work British Council will provide the consultants with relevant guidance, project information and documentation (including making available all relevant project documentation), and assist in organising travel and interviews with staff, project partners etc.
Person specification Essential - Proven experience in evaluation of similar projects
Desirable - Experience knowledge and understanding of Sri Lanka, - English language teaching - Understanding and experience of the military context
Timeline - Agree consultants – 30 June 07 - First draft available: 30 September 07 - Final draft available: 10 October 07
Project management responsibilities For British Council / Steering Committee, the contact person for issues relating to this ToR will be Marion Hughes; [email protected].
Financing The consultants will be paid subsistence at British Council rates for each day worked up to a total of 12 days. All evaluation related expenses will be paid for directly or reimbursed by British Council.
MCSP Evaluation Page 33 Dr C Tribble / www.ctribble.co.uk 13. Appendix: Comment on the terms of reference 13.1 Initial Research Questions The initial Terms of Reference (ToR) for this review set out twelve research questions to which the evaluation should respond (Figure 8). In this evaluation, I propose to do this within a broader framework of enquiry, drawing on two main evaluation approaches: scientific realism (Pawson & Tilley, 1997), and narrative in social research (Czarniawska, 2002). Before commencing this evaluation report, however, I need to comment on some problems in the wording of the the ToR, as issues that come up here are also going to be important on later in this evaluation report. Given below are the twelve research questions: Criteria Research questions Relevance 1) Was the project and the subsequent activities based on a well thought out and credible Theory of Change? 2) Are the project objectives relevant to the project’s stated goal and purpose? 3) How has the project adapted over time to changes in the external context? To what extent have internal planning, monitoring and evaluation processes contributed to this? 4) How has it contributed to meeting GCPP Sri Lanka Strategy targets? Effectiveness 5) Were the project objectives achieved? 6) Given the project goal and purpose, how better might project objectives have been formulated? 7) What hidden objective(s) if any may have driven project activities? Efficiency 8) How well have resources been converted into activities/outputs? Impact 9) What positive or negative impacts which are attributable to project activities can be detected at this stage? 10) How conflict-sensitive has British Council’s engagement been? 11) How is British Council perceived in Sri Lanka? Sustainability 12) How probable is it that long term benefits produced by the project will continue into the future? Figure 8 - research questions I will deal with the language issues which I consider these questions raise in the following section. 13.2 The problem with projects: language issues and project theories The first research question that is posed is: Was the project and the subsequent activities based on a well thought out and credible Theory of Change? I am fully in sympathy with this opening question as, along with a significant number of evaluation practitioners, I view projects as being driven by theories of change, and see evaluation as a theory testing process7. The problem I have faced in responding to the ToR for this evaluation and in dealing with the key project documents is that there is a confusion of terms at the heart of many of the questions in the Terms of Reference and, also, in the project documentation I have had to deal with. The MCSP is not alone; this confusion is found in many project designs. The term which causes the my problem with the ToR is objective. In the MCSP Project design (and this is a common problem when the Logical framework is used) the issues lie in the ways in which Goal, Purpose, and Outputs have been used. Simply put, when these words are used almost interchangeably, as has happened here, you problems arise. I will, therefore, comment briefly on the issues that have arisen as a result of the ways in which words have been used inconsistently in the ToR and the Project Proposal, and, having, I hope, established a clearer understanding of the project's theory of change, and clarified the remaining research questions, will then begin the report on the MCSP.
7 See Pawson, R (2003) "Nothing as Practical as a Good Theory", Evaluation Vol 9(4):471-490 for an entertaining account of this position. MCSP Evaluation Page 34 Dr C Tribble / www.ctribble.co.uk 13.2.1 Clarifying terms objectives The ToR proposes questions such as: Are the project objectives relevant to the project’s stated goal and purpose? What hidden objective(s) if any may have driven project activities? This is a problem as the word objectrives is not used in any of the project's own documents and it is also ambiguous as it can refer to: - long term strategic plans (which can be mapped on to Goal) - the specific goods or services that are required to achieve a project's purpose (mapping on to output) - a range of different (and confusing) terms such as project milestones, targets etc.
For the purpose of this review, therefore, I propose to dispense with the word objective. This will require the re-wording of a number of the research questions in the ToR, but I feel that this will significantly simplify the evaluation process, and make it easier for decisions to be made regarding any further funding. The revised research questions are, therefore: Criteria Research questions Relevance 1) Was the project and the subsequent activities based on a well thought out and credible Theory of Change? 2) Are the project outputs relevant to the project’s stated goal and purpose? 3) How has the project adapted over time to changes in the external context? To what extent have internal planning, monitoring and evaluation processes contributed to this? 4) How has it contributed to meeting GCPP Sri Lanka Strategy targets? Effectiveness 5) Were the project main outputs produced? 6) Given the project goal and purpose, how better might project outputs have been formulated? 7) What hidden agendas if any may have driven project activities? Efficiency 8) How well have resources and activities been converted into outputs? Impact 9) What positive or negative impacts which are attributable to project activities can be detected at this stage? 10) How conflict-sensitive has British Council’s engagement been? 11) How is British Council perceived in Sri Lanka? Sustainability 12) How probable is it that long term benefits produced by the project will continue into the future?
13.2.2 Clarifying terms goal, purpose, output Goal When the TeamUp consultancy group first developed the Logical Framework for World Bank projects, goal was used to identify the long term social benefits that a project would bring about. In many project disciplines, the term outcomes is used 8. In the project documentation for MSCP the stated goal for the project is: To develop an environment within which long term peace and reconciliation can be effectively pursued, and contribute to peace building in Sri Lanka. This is indeed a long term ambition - and the Objective Verifiable Indicator (I prefer this term to the highly problematic notion of measurability) given in the original project LogFrame is one that cannot as yet be systematically considered in this review as it stands: Military perceived as having the communication skills to promote peace building In reality, this present evaluation can only be an output to purpose review. It will require a much longer elapse of time before a meaningful review of the extent to which the MCSP's goal has been
8 see for example the Association for Project Management (www.apm.org.uk) the leading professional association for project managers in the UK. MCSP Evaluation Page 35 Dr C Tribble / www.ctribble.co.uk achieved. This has implications for the evaluation, as it some of the research questions in the ToR are simply not answerable at this stage. Purpose In the original TeamUp documentation, in social projects the Purpose is the institutional change which a project brings about as a result of the delivery of outputs. TeamUp always recommended a single purpose which could be stated simply and briefly. The MSCP project design which has been approved by the funding agency, and lived with by all concerned for the last three years includes the following purpose statement: To improve the quality of language & communication skills for military forces in Sri Lanka through :
- design & development of appropriate English language training materials - promotion of international standards of teaching and learning; - interactive and democratic classroom approaches; - use of up-to-date materials which are effective for communication training and which raise awareness of peace building issues; - upgrading of teaching & training skills; - development of sustainable and effective language training systems.
This is confusing and circular as it is, in fact, a statement of the outputs and quality standards necessary for the deliver of a purpose which could, more usefully, be summarised as follows: An enhanced and sustainable capacity in Sri Lankan military training organisations to provide communication skills training The original project documentation in fact states the foundations of the project's Theory of Change, the theory being an If > Then theory of causation:
IF the project delivers - appropriate English language training materials - international standards of teaching and learning; - interactive and democratic classroom approaches; - up-to-date materials which are effective for communication training and which raise awareness of peace building issues; - teacher & trainer training;
THEN it will have created sustainable and effective language training systems.
This is the Output to Purpose theory of change, and it is well within the scope of this evaluation to identify and assess the extent to which the project has been successful to date. Outputs An industry standard definition of outputs is "the goods and services required to deliver planned benefits".9 We have already seen that the MCSP project summary has stated a set of major outputs in its purpose statement. In the outputs section we find a restatement of these in the outputs 1 to 3: Output 1 Bank of quality language learning materials Output 2 Network of trained teachers: Output 3 Learning support centre in a strategic location
… though each of these is glossed by what are, in reality, the quality standards which should have been included in the OVI section of the LogFrame. Output 4: Security forces staff competent in use of English for access to education and training and for linking to the international community, with awareness of skills required for peace building issues.
9See APM Body of Knowledge: http://www.apm.org.uk/page.asp?categoryID=5&subCategoryID=169&pageID=0 MCSP Evaluation Page 36 Dr C Tribble / www.ctribble.co.uk … unfortunately takes us all the way back up the snakes and ladders of the LogFrame to the Goal level, and I do not feel it is appropriate to include this output statement in a review of the extent to which the project has been efficient. Instead, I will focus on the goods and services delivered through the project's implementation, and the extent to which these are fit for purpose, and have been delivered in a cost effective, timely and professionally adequate manner. The full project theory can now be seen as having the following elements: IF the project delivers - appropriate English language training materials - international standards of teaching and learning; - interactive and democratic classroom approaches; - up-to-date materials which are effective for communication training and which raise awareness of peace building issues; - teacher & trainer training;
THEN it will have created sustainable and effective language training systems. IF adequate systems are in place for providing for delivering effective communication training for the military THEN there will be an environment within which long term peace and reconciliation can be effectively pursued, and contribute to peace building in Sri Lanka.
13.3 Interim conclusion: how adequate is the project's theory of change? The original project document10 makes a strong case for a project to improve communication skills in English for the Sri Lankan military. It argues that at a higher level, the project - … will contribute (my emphasis) to the UK government's policy in relation to Sri Lanka “to strengthen incentives for lasting reconciliation, including through creating an environment conducive to reconciliation” - that key elements in this reform process are Defence, Judicial and Access to Justice - that English language skills are central to this process as they are: the medium of instruction in military training needed to improve professionalism of the forces a potential link between different communities in the long term and essential for interoperability in peace support operations
This original position is strengthened when the current priorities of the UK Government's Global Conflict Prevention Pool for Sri Lanka are taken into account11. Steve Ainsworth, Global Conflict Prevention Pool Advisor at the British High Commission summarised these as being: 1. A greater commitment to negotiated peace; 2. Improved safety and security in communities and adherence to human rights; 3. A governance reform agenda in key institutions, particularly the security services, to strengthen underlying conditions for a lasting peace; and 4. A strengthened civil society more effective in peace building.
The original project focus on English has a clear relation to Priority Area 3, and as the evaluation has developed (see sections 6.2 and following) it has also become clear that improving the curriculum, testing and instruction in basic Tamil has the potential to make a major contribution to Priority Area 2. In the same way that The Language Exchange Programme in Guatemala and Belize is enhancing the capacity of opposing factions to communicate with one another, the MCSP project is now beginning to have an impact on the capacity of the Sri Lankan armed forces (and in the future, the police) to communicate in critical military / civilian interactions. The principle that communication skills are a necessary component in increasing the capacity of the armed forces seems to me to be reasonable. The way in which the integration of former Warsaw Pact
10 Proposal for a Military Communication Skills Project in Sri Lanka (SL MilComSkill proposal 12Jun04.doc) 11 see: http://www.fco.gov.uk/servlet/Front? pagename=OpenMarket/Xcelerate/ShowPage&c=Page&cid=1062157895923 (accessed 9 November 2007) MCSP Evaluation Page 37 Dr C Tribble / www.ctribble.co.uk countries into NATO has been facilitated through the Peace Keeping English Project is well documented12. The weakness in the current statement of the project's theory of change is that it far too sweeping. Documentation for the 2007-2010 project states that the project's goal is: To develop an environment within which long term peace and reconciliation can be effectively pursued and contribute to peace building in Sri Lanka A more reasonable aspiration would be: To contribute to the development of an environment within which long term peace and reconciliation can be effectively pursued and contribute to peace building in Sri Lanka My recommendation is that such a wording is used in future project documentation, and future evaluations. The research questions which can then be asked can include one which asks: to what extent the project has enhanced the communication training skills capacity for the armed forces, and to what extent this has: - improved the effectiveness of the armed forces as modern agencies for peace-building and peace-keeping - improved military / civilian communication with the Tamil speaking minority in all areas of the island
12 see: http://www.britishcouncil.org/learning-elt-pep.htm (accessed 9 November 2007) MCSP Evaluation Page 38 Dr C Tribble / www.ctribble.co.uk 14. Appendix: Military Communication Skills Project Stakeholders PRIMARY - the direct beneficiaries - the people for whom the project is designed (end users of products and services) - Officer cadets in Army, Navy and Air Force Academies - Recruits in Air Force and Navy - Junior Staff and Command College (a three month course for officers with 10 years experience) - Defence Services Command College ( a one year course like a Masters for about 60 officers with 15 years experience) - Kotelawala Defence University cadets - Military Course instructors in the Army, Navy and Air Force - Military English Instructors; presently 36 - Military Tamil Instructors; in contact with 10 but there are more - Commandants and staff of the Military Academies - Kotelawala Defence University, Ratmalana, near Colombo - Defence Services Command College, Batalanda, near Colombo - Sri Lanka Military Academy, Diyatalawa - Naval and Maritime Academy, Trincomalee - Air Force Academy, China Bay, Trincomalee - Commanding Officers and staff of training establishments - SLAF Ekala - SLAF Diyatalawa - SL Navy Base Gemunu, Welisara - SL Navy Base Nipuna, Boossa - SL Navy Base Shikasha, Poonewa
SECONDARY - those who are either Funding, Authorising, Doing or Supplying the project - The Steering Group Committee - Funding and Authorising: The British High Commission Defence Advisor: Col. Anton Gash Steve Ainsworth Rohan Salgadoe - Authorising: Directors of Training Army: Brigadier Kumudu Perera Air Force Air Vice Marshal R. A. Dayapala Navy Commodore Kithsiri Weerakoon - All based in Colombo in the various Headquarters - The British Council - Doing: Marion Hughes, Project Manager - Hamzi Hannif, Assistant Manager, Programme Delivery - Deputy Director, British Council, Sri Lanka, Duncan Wilson (my line-manager) - Director, British Council, Sri Lanka, Gill Westaway - Commandant of the Sri Lanka Military Academy - Brigadier Janaka Walgama - Kotelawala Defence University: - Director, Defence Services Command College
EXTERNAL - those who neither benefit from or use project products and services, nor are involved in FADS activities, yet who have an interest in the project. - Sri Lanka Ministry of Defence - Peacekeeping English Project Managers
MCSP Evaluation Page 39 Dr C Tribble / www.ctribble.co.uk 15. Appendix: Learning Support Centres current status
Sri Lanka Military Kotelawala Defence Naval and Maritime SLAF China Bay SLAF Ekala Academy University Academy, Trincomalee Where is it? In the library In a classroom In a designated room In the library In a designated room
How many computers? 8 10 10 10 8 Software? Tensebuster, Grammar Tensebuster, Grammar Tensebuster, Grammar Tensebuster, Grammar Tensebuster, Grammar in use, Sky in use, Sky in use, Sky in use, Sky in use, Sky pronunciation, pronunciation, pronunciation, pronunciation, pronunciation, reactions, Study Skills reactions, Study Skills reactions, Study Skills reactions, Study Skills reactions, Study Skills for Success, Road to for Success, Road to for Success, Road to for Success, Road to for Success, Road to IELTS IELTS IELTS IELTS IELTS Furniture? 25 chairs, 8 computer 20 chairs, 10 computer 25 chairs, a very large 25 chairs, 10 computer 25 chairs, 10 computer tables tables table, 10 computer desks. desks, lockable A very large table desks, book shelves cupboards, two large tables Books: Packs of: Packs of: Packs of: Packs of: Packs of: A pack consists of the Intermediate Cutting Intermediate Cutting Intermediate Cutting Intermediate Cutting Intermediate Cutting Students book, Work Edge Edge Edge Edge Edge book, and listening Intermediate English Intermediate English Intermediate English Intermediate English Intermediate English cassettes File File File File File Breakthrough Breakthrough Breakthrough Breakthrough Breakthrough *see additional book list Campaign 1 Campaign 1 Campaign 1 Campaign 1 Campaign 1 below Campaign 2 Campaign 2 Campaign 2 Campaign 2 Campaign 2 Workcards Grammar and Grammar and Grammar and Grammar and Grammar and vocabulary work cards vocabulary work cards vocabulary work cards vocabulary work cards vocabulary work cards with keys with keys with keys with keys with keys Equipment Two cassette players Two cassette players Two cassette players Two cassette players Data Projector Two cassette players In charge? Librarian also checked Head of the English Technician and Librarian and Technician and checked
MCSP Evaluation Page 40 Dr C Tribble / www.ctribble.co.uk by IT Department Department checked by IT Technician and by IT Department Captain Thushara (or Captain Wimansha Department checked by IT Flight Lt Hamza Gonaduwa Abeywickrama) Lt Cdr Nirosh Ananda Department Flight Lt Sanjeewa Software and work Yes Yes Yes Yes Yes cards integrated into syllabus? Attendance Book? Yes Yes
Used for? Mainly used as part of Constantly used as part Mainly used as part of Mainly used as part of Mainly used as part of the Intensive English of the Intensive English the Intensive English the Intensive English the Intensive English course class but also course class but also course class course class but also course class available for self access available for self access available for self access Internet access? No Yes, on some No No Yes, on one computer computers Opening times 08.00 to 14.00 for 08.00 to 14.00 for 08.00 to 14.00 for 08.00 to 14.00 for 08.00 to 16.00 for regular classes regular classes regular classes regular classes regular classes 15.30 to 17.30 special Self Access 17.30 to Self Access 17.30 to classes Self Access 17.30 to 20.00 20.00 Self Access 17.30 to 21.00 21.00 Date started on May 2006 September 2006 December 2007 April 2006 April 2006
MCSP Evaluation Page 41 Dr C Tribble / www.ctribble.co.uk Course books Campaign 1 Student book 2 copies Test Your Listening 2 copies Campaign 1 Work book 2 copies Campaign 1 Class CD 2 copies Recycling Intermediate English 1 copy Campaign 2 Student book 2 copies Campaign 2 Work book 2 copies Recycling Elementary English 1 copy Campaign 2 Class CD 1 copy Listen Carefully 2 copies Cutting Edge Intermediate Student Books 2 copies New English File Pre Intermediate Student’s Book 2 copies Cutting Edge Intermediate Work Book 2 copies New English File Pre Intermediate Work Book 2 copies Cutting Edge Intermediate Class CD 1 copy New English File Pre Intermediate Class CD 1 copy Cutting Edge Intermediate Student CD 1 copy
English File Intermediate Student’s Book 2 copies Campaign Dictionary of Military Terms 5 copies English File Intermediate Work Book 2 copies English File Intermediate Class CD 1 copy Just Listening and Speaking 3 copies Just Vocabulary 3 copies English File 2 Work Book 2 copies Just Reading and Writing 3 copies English File 2 Student Book 2 copies English File 2 Class CD 1 copy English File 2 Student CD 1 copy New Cutting Edge Intermediate Student Books 2 copies New Cutting Edge Intermediate Work Books 2 copies New Cutting Edge Intermediate Class CD 1 copy New Cutting Edge Intermediate Student CD 1 copy
MCSP Evaluation Page 42 Dr C Tribble / www.ctribble.co.uk 16. Appendix: Learning Support Centres (LSC) current status
There are not enough computers in SLMA and Ekala LSCs. They have eight in each place with double headphones but the class sizes are such that at least four more computers in each place would be better. The computers are now on the maintenance rota of the IT Departments in each training establishment and the person in charge has received basic training on maintenance, basic cleaning of the hardware and installation of the software. There is a problem with viruses being introduced onto the computers by people using USB keys. The computers are not linked to the internet so that the virus protection cannot be upgraded. It would be better to network the computers in each LSC so that they can be protected more efficiently. The use of Word and USB keys will be banned in LSC to try and protect the computers. The LSC has been integrated into class time successfully in all the training establishments. Registers of use are being kept in SLMA and SLAF China Bay. They should be introduced in the other LSCs. Students should be encouraged to use the LSCs in the evening and in any free time available. For the most part they do not have much free time so the LSCs are not being used as Self Access Centres except in the evenings. More work cards are needed in the LSCs.
MCSP Evaluation Page 43 Dr C Tribble / www.ctribble.co.uk 17. Appendix: Tamil syllabus a 60 x 45 minutes lesson (45 hour) course for beginner level military personnel
Each Unit would have three lessons:1.
Tamil alphabet – provide wall chart of 11. Descriptions alphabet man, woman, child Adjectives: tall, short, fat, thin, old, young 2. Greetings What did he/she look like? What’s your name? My name’s … 12,Jobs 3. Numbers 1 – 9 What do you do? Where do you work? ID card please (also; unemployed, refugee, retired) What’s your phone number? 13.Family members 4. Class room vocabulary (pen, book, clock, Are you married? Children? table, window, ) What’s this? It’s a … 14. Rooms in a House What’s that? It’s a Furniture Who’s this? It’s my mother / father / daughter / 5. Days of the week (planets) son morning, afternoon, evening, night breakfast, lunch, dinner 15. At a Check Point What’s your name? 6. Numbers 1-12 Where are you from? Time: What’s the time? Where are you going? What time do you have breakfast? 16. Directing people 7. Provinces in Sri Lanka – Provide Tamil Come here / Go there / Wait here / Sit down / map of Sri Lanka Stand here / Open this / Run / Walk / Get off / Places in Sri Lanka Get on / Get up / Get down Where are you from? Checking baggage Where are you going? What’s this?
8. Places in a town: hospital, school, post 17. Months of the year office, market, bus stop, bank, police station, Numbers 1-31 Divisional secretary’s office Date of birth Age How old are you? Where’s the …..? Numbers 1-100 Directions: north, south, east, west, right, left, go straight 18. Parts of the body What’s the matter? I’ve got a sore …. 9. Clothes I’m + feeling Colours He’s deaf, blind, dumb, mentally ill ( include What was he wearing? hand sign for deaf / dumb in the book) He can’t hear, walk, see, talk 10. Vehicles ( three-wheeler, lorry, van, car, Help! Emergency! He’s dead / He’s bleeding motor bike, bike, bullock cart, ship, fishing boat, plane, bus, train, police car, ambulance) 19. Food guns, bombs Fruit and vegetables What did you see? sweet, tasty, hot, spicy Where did they go? In a shop How much is a …. Can I have a …
20. Filling in a simple Form Name ands address Examples of common forms in Tamil
MCSP Evaluation Page 44 Dr C Tribble / www.ctribble.co.uk 18. Appendices: Print Cost Estimate (2007) – add 20% inflation for 2009 100pages Colour (CMYK) 1500nos 3000nos 100GSM Art paper 442,750.00 620,125.00 Per book Rs. 295.17 206.71 80GSM Bank paper 418,625.00 573,750.00 Per book Rs. 279.08 191.25 B/W 1500nos 3000nos 100GSM Art paper 229,875.00 387,250.00 Per book Rs. 153.25 129.08 80 GSM Bank paper 206,875.00 341,750.00 Per book Rs. 137.92 113.92
19. Appendices: Interview notes summary Benefits Sustainability professional development 27 institutionalisation 11 ss performance 16 buy-in 8 teacher confidence 4 materials 6 reaching more students 1 contract 3 staff 1 Issues Next Steps lack of appropriate resources 8 group size 5 materials 5 entry level 4 INSETT 4 recruitment / retention 4 institutionalisation 3 tamil curriculum 4 extend training 2 conflicting responsibilities 3 KDU BA ELT 2 scheduling 3 additional resources 1 additional programmes 2 logistics 1 administrative barriers 2 peacekeeping 1 lack of professional development 1 recruit trainers 1 post course 1 speed of change 1 turnover of SL managers 1 UK commitment 1
Relevance of English international comms 7 internal comms 6 training 4 IHR 3 career 2 military/civilian 2 governance 1
Relevance of Tamil military/civilian 13 community relations 2
MCSP Evaluation Page 45 Dr C Tribble / www.ctribble.co.uk 20. List of meeting notes 20071027_Gill_Westaway.doc 20071029a_GI_Navy.doc 20071029a_GI_Navy_T_S.doc 20071029b_SLAF_ChinaBay.doc 20071030_BHC_Steve_Ainsworth.doc 20071031_Airforce_HQ.doc 20071031_Army_HQ.doc 20071031_Navy_HQ.doc 20071101_NLC.doc 20071101_SLAF_C_Instructor.doc 20071101_SLAF_Instructors_GI.doc 20071101_SLAF_Trainees_GI.doc 20071102_CHA.doc 20071102_KDU_GI.doc 20071102_KDU_managers.doc 20071103_BC_designer.doc 20071103_Tamil_authors.doc 20071105_SLAF.doc 20071105_SLMA_CO.doc 20071105_SLMA_GI.doc 20071105_SLMA_GI_ELT.doc 20071105_SLMA_GI_SS_2.doc 20071107_Asia_Foundation.doc 20071107_Peace_Foundation.doc
21. List of observations 20071029_Navy_Observation record.doc 20071101_SLAF_Observation record.doc 20071102_KDU_observation.doc 20071105_SLMA_01.doc 20071105_SLMA_02.doc
MCSP Evaluation Page 46 Dr C Tribble / www.ctribble.co.uk