Challenges and opportunities in implementing Computer Based Test (CBT)... B. Suryadi, Y. Rahmawati Challenges and opportunities in implementing Computer Based Test (CBT) of national assessment for non-formal education B. Suryadi Universitas Islam Negeri Syarif Hidayatullah, Jakarta, Indonesia Y. Rahmawati Universitas Negeri Jakarta, Jakarta, Indonesia ABSTRACT: Computer Based Test (CBT) in National Assessment for Non-formal Education as new policy of the Ministry of Education and Culture has been implemented in 2017. The purpose of this study was to analysis the implementation of CBT for national assessment in terms of policy and technical implementation for non-formal education. This study used a qualitative method. Interviews, observation and document analysis were carried out as data collection technique. The results of this study showed that the policy of CBT implementation in national assessment for non-formal education faced challenges in facilities in relation to technical problems, students’ competences in computer skills, and stakeholders’ engagement. However, the policy has provided opportunities in non-formal education quality in improvement of assessment system, students and teachers competences, and non-formal education database system. Therefore it needs improvement for further implementation for all non-formal education institutions throughout the nation.. 1. INTRODUCTION education is conducted in the form of As a consequence of standard driven Computer Based Test or CBT (BSNP, reform, the government of Indonesia has 2017). According to American implemented standard based education Educational Research Association (2014) since 2003 for improving the education computer-based test or computer quality, including non-formal education. administered test is a test administered by The national education standards consist computer; that indicates test takers of 8 standards which are graduates competency, content, process, respond by using a keyboard, mouse, or assessment, facilities and infrastructure, other response devises. In broader educators and education personnel, concept, the implementation of CBT in management, and finance standards. In national assessment is also seen as the the context of non-formal education, the means to revolutionize education and national assessment as part of standard build human character (Noor, 2008). based education has been implemented In addition to the influence of along with formal education in 2005 and assessment in students’ learning and in 2015, the use of computer based test teaching approaches has been recognised (CBT) for national assessment has been in research and different literature started for formal education. (Carrillo-de-la-Pen˜a and Pe´rez, 2012). Since 2017 onward, Board for Therefore, the different ways of National Standard in Education (BSNP) conducting assessment could provide and the Ministry of Education and possible negative effects of testing on Culture stated that the implementation of students: anxiety, categorizing and national assessment for non-formal labelling the students, damaging students’ Publishing : Faculty of Education, Universitas Negeri Surabaya 184 Proceeding the 1st International Conference on Education Innovation (ICEI) Page 184-194 ISBN : 978-602-50898-0-0 self-esteem, and creating self-fulfilling example, the title on the envelope does prophecies (Linn & Miller, 2005). Wass, not match the contents of the envelope. In Van der Vleuten, Shatzer, and Jones terms of assessment items, PBT items can (2001), it is well known that students only be used once, while CBT items can adjust their learning processes according be used more than one because all the to the particular type of assessment used. questions are stored inside the computer Therefore, assessment should be and are encrypted. Thus, the meaningfull for encourage students’ confidentiality of the assessment items in learning, including non-formal education CBT becomes assured. With regard to in Indonesia. item variation or package, item packages Prior to 2017, the national for PBT are very limited. So far, the item assessment for non-formal education was variation consisted of 20 item packages conducted in the form of paper and pencil in accordance with the number of based test (PBT). There are certain examinees in one room. Conversely, for consideration factors of implementing CBT the item package is unlimited, CBT for non-formal education. because it is a computer generated items. According to BSNP report on monitoring The implication is, with the availability and evaluation of national assessment, of various assessment items, CBT can be CBT is considered more effective and implemented at any time, in accordance efficient than PBT in terms of time, cost, with the readiness of learners. energy, and human resources (BSNP, In terms of scoring, technically, the 2016). In terms of CBT application used results of CBT can be obtained after the in the national assessment, CBT participants completed the questions. application is user friendly and simple. In However, considering there is also PBT the globalisation, young generations have mode of assessment and students’ been familiar of using computer, psycological condition, therefore the therefore they shouldn’t have difficulty in CBT results are released along with the taking CBT as the assessment mode. PBT results. Finally, in terms of integrity, From the participants' perspective, the it was able to reduce number of students’ students prefer to take national cheating, since each students work on assessment with CBT rather than PBT, different questions (BSNP, 2015). since its technically easy to choose and Despite numerous advantages of change the answers without using CBT over PBT as mentioned above, there pencil.Thus, students can focus more on are several challenges from different answering the national assessment. stakeholders, such as teachers or tutors, In addition, from the perspective of parents, students, and school principals. implementing committee duties and They were resistant and reluctant to responsibilities, CBT is more efficient in conduct CBT in national assessment. terms of time, material preparation, Infrastructure and facilities limitation, scanning (scorring?), and delivery of test computer illiteracy, shortage of electricity results. It is reduce the long chain of power supply and access to schools, were process including auction and distribution considered as the challenges. which involving different parties, This study was carried out to including the police. For the school analysis the implementation of CBT for principals as the head of the assessment national assessment in terms of policy committee, they do not need to distribute and technical challenges for non-formal the assessment material from the centre education. Therefore this study addressed storage to the District Education Office. the following research questions: (a) From the perspective of technical What are the policies of national process, its minimise lack of assessment assessment for non-formal education?, number , there is no shortage of (b) What are the challenges faced in assessment questions or manuscript implementing CBT for non-formal errors as often happens in PBT, for education?, and (c) What are the Surabaya, 14 October 2017 185 Challenges and opportunities in implementing Computer Based Test (CBT)... B. Suryadi, Y. Rahmawati opportunities faced in implementing CBT education services as replacement, for non-formal education? improvement or complementary of formal education in order to promote life 1. METHODS long education. Non-formal education serves to develop the potential of learners This study employed a qualitative with an emphasis on mastery of method. Document analysis, interviews, knowledge and skills and the observation and were carried out as data development of professional attitude and collection technique. The documents of personality. Non-formal education national education system law, includes life skills education, early government regulations, ministry childhood education, youth education, regulations, and other documents related women's empowerment education, to national standard of education and literacy education, vocational education standard operating procedure of national and job training, equality education, and assessment issued by BSNP were other education aimed at developing analysed. The techiques of comparing, learners' abilities. Non-formal education contrasting, criticizing, and synthesizing units consist of courses, training were applied in analyzing the data. institutions, study groups, community Interviews involved examinees, centers, and community religious studies, teachers, and other stakeholders of non- as well as similar educational units. The formal education. They were asked about results of non-formal education can be their opinions on issues related to the considered as equal to the outcome of a national assessment. The student formal education program after going interviewees selected in this study were through an equivalency assessment based on the subject interest, namely process by an agency designated by the sciences and social sciences. For Government or local government with interview purpose, guideline and reference to the national standard of structured questions were prepared and education. about 30 minutes were taken for each In this study, taking into respondent. consideration the above mentioned Observation was conducted during definition of non-formal education, the the national