Challenges and opportunities in implementing Computer Based Test (CBT)... B. Suryadi, Y. Rahmawati Challenges and opportunities in implementing Computer Based Test (CBT) of national assessment for non-formal education

B. Suryadi Universitas Islam Negeri Syarif Hidayatullah, ,

Y. Rahmawati Universitas Negeri Jakarta, Jakarta, Indonesia

ABSTRACT: Computer Based Test (CBT) in National Assessment for Non-formal Education as new policy of the Ministry of Education and Culture has been implemented in 2017. The purpose of this study was to analysis the implementation of CBT for national assessment in terms of policy and technical implementation for non-formal education. This study used a qualitative method. Interviews, observation and document analysis were carried out as data collection technique. The results of this study showed that the policy of CBT implementation in national assessment for non-formal education faced challenges in facilities in relation to technical problems, students’ competences in computer skills, and stakeholders’ engagement. However, the policy has provided opportunities in non-formal education quality in improvement of assessment system, students and teachers competences, and non-formal education database system. Therefore it needs improvement for further implementation for all non-formal education institutions throughout the nation..

1. INTRODUCTION education is conducted in the form of As a consequence of standard driven Computer Based Test or CBT (BSNP, reform, the government of Indonesia has 2017). According to American implemented standard based education Educational Research Association (2014) since 2003 for improving the education computer-based test or computer quality, including non-formal education. administered test is a test administered by The national education standards consist computer; that indicates test takers of 8 standards which are graduates competency, content, process, respond by using a keyboard, mouse, or assessment, facilities and infrastructure, other response devises. In broader educators and education personnel, concept, the implementation of CBT in management, and finance standards. In national assessment is also seen as the the context of non-formal education, the means to revolutionize education and national assessment as part of standard build human character (Noor, 2008). based education has been implemented In addition to the influence of along with formal education in 2005 and assessment in students’ learning and in 2015, the use of computer based test teaching approaches has been recognised (CBT) for national assessment has been in research and different literature started for formal education. (Carrillo-de-la-Pen˜a and Pe´rez, 2012). Since 2017 onward, Board for Therefore, the different ways of National Standard in Education (BSNP) conducting assessment could provide and the Ministry of Education and possible negative effects of testing on Culture stated that the implementation of students: anxiety, categorizing and national assessment for non-formal labelling the students, damaging students’

Publishing : Faculty of Education, Universitas Negeri Surabaya 184

Proceeding the 1st International Conference on Education Innovation (ICEI) Page 184-194 ISBN : 978-602-50898-0-0 self-esteem, and creating self-fulfilling example, the title on the envelope does prophecies (Linn & Miller, 2005). Wass, not match the contents of the envelope. In Van der Vleuten, Shatzer, and Jones terms of assessment items, PBT items can (2001), it is well known that students only be used once, while CBT items can adjust their learning processes according be used more than one because all the to the particular type of assessment used. questions are stored inside the computer Therefore, assessment should be and are encrypted. Thus, the meaningfull for encourage students’ confidentiality of the assessment items in learning, including non-formal education CBT becomes assured. With regard to in Indonesia. item variation or package, item packages Prior to 2017, the national for PBT are very limited. So far, the item assessment for non-formal education was variation consisted of 20 item packages conducted in the form of paper and pencil in accordance with the number of based test (PBT). There are certain examinees in one room. Conversely, for consideration factors of implementing CBT the item package is unlimited, CBT for non-formal education. because it is a computer generated items. According to BSNP report on monitoring The implication is, with the availability and evaluation of national assessment, of various assessment items, CBT can be CBT is considered more effective and implemented at any time, in accordance efficient than PBT in terms of time, cost, with the readiness of learners. energy, and human resources (BSNP, In terms of scoring, technically, the 2016). In terms of CBT application used results of CBT can be obtained after the in the national assessment, CBT participants completed the questions. application is user friendly and simple. In However, considering there is also PBT the globalisation, young generations have mode of assessment and students’ been familiar of using computer, psycological condition, therefore the therefore they shouldn’t have difficulty in CBT results are released along with the taking CBT as the assessment mode. PBT results. Finally, in terms of integrity, From the participants' perspective, the it was able to reduce number of students’ students prefer to take national cheating, since each students work on assessment with CBT rather than PBT, different questions (BSNP, 2015). since its technically easy to choose and Despite numerous advantages of change the answers without using CBT over PBT as mentioned above, there pencil.Thus, students can focus more on are several challenges from different answering the national assessment. stakeholders, such as teachers or tutors, In addition, from the perspective of parents, students, and school principals. implementing committee duties and They were resistant and reluctant to responsibilities, CBT is more efficient in conduct CBT in national assessment. terms of time, material preparation, Infrastructure and facilities limitation, scanning (scorring?), and delivery of test computer illiteracy, shortage of electricity results. It is reduce the long chain of power supply and access to schools, were process including auction and distribution considered as the challenges. which involving different parties, This study was carried out to including the police. For the school analysis the implementation of CBT for principals as the head of the assessment national assessment in terms of policy committee, they do not need to distribute and technical challenges for non-formal the assessment material from the centre education. Therefore this study addressed storage to the District Education Office. the following research questions: (a) From the perspective of technical What are the policies of national process, its minimise lack of assessment assessment for non-formal education?, number , there is no shortage of (b) What are the challenges faced in assessment questions or manuscript implementing CBT for non-formal errors as often happens in PBT, for education?, and (c) What are the

Surabaya, 14 October 2017 185

Challenges and opportunities in implementing Computer Based Test (CBT)... B. Suryadi, Y. Rahmawati opportunities faced in implementing CBT education services as replacement, for non-formal education? improvement or complementary of formal education in order to promote life 1. METHODS long education. Non-formal education serves to develop the potential of learners This study employed a qualitative with an emphasis on mastery of method. Document analysis, interviews, knowledge and skills and the observation and were carried out as data development of professional attitude and collection technique. The documents of personality. Non-formal education national education system law, includes life skills education, early government regulations, ministry childhood education, youth education, regulations, and other documents related women's empowerment education, to national standard of education and literacy education, vocational education standard operating procedure of national and job training, equality education, and assessment issued by BSNP were other education aimed at developing analysed. The techiques of comparing, learners' abilities. Non-formal education contrasting, criticizing, and synthesizing units consist of courses, training were applied in analyzing the data. institutions, study groups, community Interviews involved examinees, centers, and community religious studies, teachers, and other stakeholders of non- as well as similar educational units. The formal education. They were asked about results of non-formal education can be their opinions on issues related to the considered as equal to the outcome of a national assessment. The student formal education program after going interviewees selected in this study were through an equivalency assessment based on the subject interest, namely process by an agency designated by the sciences and social sciences. For Government or local government with interview purpose, guideline and reference to the national standard of structured questions were prepared and education. about 30 minutes were taken for each In this study, taking into respondent. consideration the above mentioned Observation was conducted during definition of non-formal education, the the national assessment period in selected term of non-formal education is schools in certain provinces, such as specifically defined as equity education Jakarta, , , Bali, and and non-formal education units in this South Kalimantan. The purpose of this study are limited to the Community observation was to gain information on Learning Center (PKBM). Data from students’ behavior during the Center for Educational Asessment show implementation of national assessment. It that there are 5.476 PKBM across the was also aimed to find out the obstacles nation (Puspendik, 2017). faced during the implementation of Non-formal education is provided national assessment. for educational equity. According to Nizam (2017) characteristics of equity education can be categorized into two 2. FINDINGS AND DISCUSSION things. First, the nature of equity In this section, findings of the study education, in general, its learning process are presented according to the research is done by andragogy approach, although questions in challenging and in reality there are learners who are of opportunities of CBT in national school age. Second, the learning process assessment for non-formal education. is implemented in part-time basis (self- According to Article 26, Law No. paced learning), although there are 20 year 2003 about National Education education units which implement full System, non-formal education is provided time schooling. Third, learners have the for Indonesian citizens who need motivation to learn. In terms of service,

Publishing : Faculty of Education, Universitas Negeri Surabaya 186

Proceeding the 1st International Conference on Education Innovation (ICEI) Page 184-194 ISBN : 978-602-50898-0-0 equity education has flexible, adaptive, detailed and practical implementation, and ubiquitous learning. BSNP issued regulation Number 0043/P/ In addition, The Law No. 20 year BSNP/I/2017 about Standard Operating 2003 revealed in the context of education Procedures of National Assesment Year quality assurance, evaluation, 2016/2017. With this regulation and accreditation and certification are carried policy, national assessment is expected to out. Evaluation is conducted in the be more credible, acceptable, and framework of national quality control of accountable. education as a form of public educational The policies issued by BSNP and the accountability. Evaluation is conducted Ministry of Education and Culture related on students, institutions, and education to the national assessement for equity programs for both formal and non-formal education are as follows. In term of the education including all levels, units, and registration requirements of national types of education. Evaluation of learning examinees, since 2015, national outcomes is conducted by the education assessment participants must be enrolled units to monitor the process, progress, in Data of National Education and improvement of student learning (DAPODIK) under the Ministry of outcomes in long terms basis. The Education and Culture. For students in evaluation of students, education units, order to be enrolled in DAPODIK, are and educational programs is carried out required to have a National Student by independent institutions on a regular, Identity Number (NISN), whereas the comprehensive, transparent and systemic education units must have an Education basis to assess the achievement of Unit Number (NPSN). The enforcement national education standards. of these requirements is intended to ensure that national assessment 2.1. Policies of national assessment made participants from non-formal education by BSNP and Ministry of Education follow the learning process conducted by and Culture for non-formal education the education units. Then from the number of subjects The national assessment policy its assesed in the national assessment, there related to the task of BSNP as the are seven subjects for the C program government board which collaborate with majoring in social sciences, namely Puspendik. According to Government , Geography, Regulation No. 19/2005, one of the tasks Mathematics, Sociology, English, of BSNP as a self-reliant, professional Economics, and Civic Education. While and independent agency is to organize the for students majoring in natural science, national assessment. The regulation the subjects include Indonesian language, further stated that national assessment is Chemistry, Mathematics, Biology, held to measure the achievement of English, Physics, and Civic Education. graduate competence at the level of The national assessment subjects for the elementary and secondary level. National B package program consists of five assessment results are used for mapping subjects namely, Indonesian language, the quality of education, selection at Civic Education, Social Sciences, higher levels of education, as well as the English, and Natural Sciences. basis for delivering assistance and In addition to assessment mode, in educational funding. 2017, the CBT was carriet out for the The implementation of national national assessement of non-formal assessment in 2017 refers to the education. This policy was taken BSNP Regulation of the Minister of Education through meetings together with Center and Culture No. 3 of 2017 about the for National Assessment (Puspendik), assessment of learning outcomes by the Agent for Research and Development Government and assessment of learning (Balitbang), and Directorate Jenderal for outcomes by the education units. In its Early Education and Non-formal

Surabaya, 14 October 2017 187

Challenges and opportunities in implementing Computer Based Test (CBT)... B. Suryadi, Y. Rahmawati

Education. The policy aims to improve Table 1: Policies for National Assessment the quality of national assessment of non-formal education implementation for non-formal education. Considering the condition of Aspect 2016 2017 Remarks PKBM which still need further National Registered Registered Same development on infrastructure and Assessment in in computer facilities, the implementation of Requirement DAPODI Dapodik CBT for non-formal education was K implemented by resource sharing. Assessment Paper and Computer Different mode Pencil Based PKBMs which do not have computer based test Test facilities can join formal education units, (PBT) (CBT) such as junior secondary schools, senior secondary schools, and vocational Time table Weekdays, Weekend, Different secondary schools. Monday to Saturday In terms of time table, national Friday and assessment for non-formal educaion was Sunday conducted on weekend, Saturday and Invigilation Not cross- cross-over Different over PKBM Sunday. This is intended to provide PKBM opportunities for those students who Different work during the weekdays. Before 2017, Assessment District Accredited when the assessment was held on organizer Education Education weekdays, as a consequency, the students Office Unit cant attend because they did not get (PKBM) permission from employers. For assessment timeline, in 2017, The policy of CBT implementation unlike formal education, for non-formal for non-formal education were not educaiton there was no subsequent difficult to be conducted by PKBM. For national assessment for those unable to PKBM that have at least one server and take the assessment on given stipulated computer devices more than 20 pieces time table,which is normaly done a week can be set to be the implementing agency after the main assessment. However, non- UNBK. This policy is clearly stipulated formal national assessment was done in in the national assessment standard two waves, ie in April and May for operating procedure (SOP) as well as Packages C and B, as well as in October. clearly mentioned in Circular Letter (SE) Thus, those who were unable to take the Minister of Education and Culture No. 1 national assessment in April or May can of 2017 on Implementation of National take the assessment in October. Examination Year 2016/2017 that sent to In terms of invigilation, assessment Governor and Regent / Mayor. invigilators were assigned cross-over In a circular letter issued on January PKBM, involving teachers and tutors. 10, 2017 the Minister also empashized a Finally, from the perspective of national step to optimize the use of computers for assessment organizer, unlike the previous national assessment through resource years, from 2017 onward, the authority of sharing schemes. The scheme encourages execution of the assessment is returned to local education offices to allow students each educational unit (PKBM). Prior to from schools with limited insfratructure this, considering many PKBMs that have to sit for national assessment usisng CBT not been accredited, the authority of in other schools within a radius of a national assessment implementation is maximum of five kilometers. given to the District Education Office. In In addition, there are different roles short, policies for national assessment of of educational units, school committees, non-formal education are summarized in parents, education offices, and colleges. Table 1 (BSNP, 2017). School committees and parents collaborate in provide the laptops and

Publishing : Faculty of Education, Universitas Negeri Surabaya 188

Proceeding the 1st International Conference on Education Innovation (ICEI) Page 184-194 ISBN : 978-602-50898-0-0 computers for the CBT. If a school does To address the second research not have sufficient server and computer question: What are the challenges faced facilities for CBT, the principal may in implementing CBT in Indonesia apply to a local education office to national assessment, interviewed with conduct CBT using resource sharing teachers, students, school principal, and scheme. Schools and school committees examination authority was done. The should not charge students or parents for challenges were classified into four computer procurement to support CBT categories, namely national standards of implementation. Basically, the use of education for non-formal education, computers is intended to support human resources, infrastructure, and computer-based learning process. In resistant attitudes among learners and addition, the local governments allocate PKBM managers. The national standards Regional Expenditure Budget (APBD) of education are considered as a reference for education for computer procurement in developing curriculum structure of for schools that have not been able to non-formal education. However, the implement computer-based learning national standards of education for non- process which can then be used in the formal education has not been developed implementation of CBT. The local yet. Therefore, graduate competence government also mapped schools that standards and content standards for non- were included in resource sharing in the formal education still apply the standards implementation of CBT (Kemdikbud, for formal education. This policy is 2017). considered unfair, since non-formal According to national asssessment education has different characteristics SOP issued by BSNP there are three from formal education. paths, namely the school applying for the Then, the majority of PKBM has not implementation of CBT to the District / been accredited, whereas one of the basic City Education Office or Provincial requirements for educational units to be Education Office. Second, the District / able to conduct national assessment is Municipal Education Office or Provincial that PKBM should be accredited. As a Education Office verifies and establishes consequence, non-accredited PKBM have the CBT organizing school. Third, the to join other PKBM or schools in order to school conducts the exam simulation of organize national assessment. However, the examinees and then conducts the in appointing school units where non- exam with CBT (BSNP, 2017). accredited PKBM join the national The procedure of CBT assessment, also created another problem. implementation (semi online) consisted The problem is that accredited school of four paths. First, the exam questions units applied certain charges based on are sent online from the central server to the number of students taking national the school server via the internet assessment. Due to these extra expences, (synchronization). Second, exam tokens many PKBM widrawed from were sent online from the central server emplementing CBT to PBTm becaure to the school server. Third, the test was they could not afford to pay the executed offline through the school additional charges. server and student computer (client). In the challenge of human Fourth, the results of the exam are sent resources. Most PKBM are owned and online from the school server to the run by private sectors or community central server (uploaded results). member with various financial and infrastructure constraints. Surprisingly, 2.2. The challenges faced in implementing these limitations are sometimes used by CBT in Indonesia national PKBM managers to benefit from learners assessment in different ways. For example, learners who do not follow the learning process regularly are allowed to take national

Surabaya, 14 October 2017 189

Challenges and opportunities in implementing Computer Based Test (CBT)... B. Suryadi, Y. Rahmawati assessment as long as they pay certain Finally, the biggest challenge is the amount of money to PKBM managers. In attitude of the managers and learners who short, there is educational are still reluctant to change from PBT to commercialization, while learners come CBT. For them CBT is considered to from low-class economic societies. cause anxiety and fear because their ICT In terms of learners, majority of literacy level is low, so they are afraid to them were from low-income society. This fail the exam. Therefore, their acceptance social economic status affected their of CBT is vely low. This finding is ability to pay tuition fees. Many of them consistent with a previous study done by were aunable to pay tuition fees on time Boeve et al. (2015) who found that and even some of them who paid with students preferred PBT because the test staple food from the rice field or garden. taking experience is substantially Another characteristic of non-formal different from CBT. A study by students is the fact that they were already Hochlehnert et al. (2011) in the German working or dropping out of schools. This higher education context showed that conditions made them unable to take part only 37% of studetns voluntarily chose to learning process effectively. take a high-stakes exam via the computer, In addition to infrastructure and and that test-taking strategies were a learning facilities, the majority of PKBM reason why students opted for the paper- did not have sufficient computer units based exam. as required by BSNP in the Indonesia is well known as the implementation of CBT. This problem world's largest archipelago which can be solved by resource sharing. representing more than 17,000 island However, another problem amerged, that from Sabang in northern Sumatra to is the distance between the place where Merauke in Irian Jaya. Indonesia is leaners stay and PKBM location is far connected by sea with total area of away. The place of national assessment is 1.910.931.32 km2 and total population of located in the city, whereas the leaners 248.818.100 and has 34 provinces (BPS, are in the village as so they have to make 2014). Besides the geographical extra time, efforst, and cost. This finding conditions, Indonesia has around 300 supported a previous study by Boeve, ethics groups which have different values Meijer, Albers, Beetsma, and Bosker and beliefs. Data from Center for (2015) who found that certain barriers in Educational Assessment show that there implementing CBT such as the additional are about 5.476 PKBM for both Package need for adequate facilities, test-security, B and C programs across the country back-up procedures in case of with a total of 383.850 students as technological failure, and time for staff presented in Table 2 (Puspendik, 2017). and students to get acquainted with new technology.

Table 2: Number of PKBM and Students Taking National Assessment in PBT and CBT Students Taking National Level PKBM Assessment

PBT % CBT % TOTAL PBT % CBT % TOTAL

Package 3,83 83.3 16.6 4,600 115,63 85.0 20,28 14.9 B 765 135,913 5 7 3 1 8 2 2

Package 3,54 80.1 19.8 4,427 207,54 83.7 40,38 16.2 C 879 247,937 8 4 6 9 1 8 9

Total 4,56 83.4 16.6 323,18 84.1 60,67 15.8 909 5,476 383,850 7 0 0 0 9 0 1

Publishing : Faculty of Education, Universitas Negeri Surabaya 190

Proceeding the 1st International Conference on Education Innovation (ICEI) Page 184-194 ISBN : 978-602-50898-0-0 Data on Table 2 show that out of the DIY implemented CBT. Similarly, total number of PKBM in Indonesia supports from non governmental sectors (5.476), the number of PKBM such as telecommunication providers was implementing PBT is 4.567 with a great apportunity for implementing percentage of 83.40 percent and the CBT in national assessment. Their number of PKBM implementing CBT is support is ensuring that the internet 909 with percentage of 21.37 percent. access is sufficient enough during This indicates that most PKBM still downloading the assessment items from implement PBT in national assessment in the central server. academic year 2017. Tha data also show The characteristic of CBT that the number of students taking PBT is application which is called as user- much higher than the number of students friendly was also seen as a great taking CBT in national assessment with opportunity. For this reason, students percentage of 84.19 percent and 15.81 who use mobile phones will have no percent respectively. difficulties in taking CBT during the The low number of PKBM national assessment. In addtion, the CBT implementing CBT could be due to the business process was found to be simple lack of facilities. In this context, BSNP and easy because it was not conducted (2017) has issued a regulation saying that fully online, rather it was conducted in for PKBM to organize CBT in national term of semi-online way. This means that assessment, it must have at least 20 first online conductance was done about computer units and one server. However, three days before the assessment if PKBM did not fulfill this requirement, implementation to download question BSNP allowed it to join either formal or items from the central server to local non-formal education school units server. At the time of CBT equipped with such facilities. Another implementation, it was done offline opportunity is the policy related to non- because it used the local server. The formal education management authority. second online conductance was when the According to Law No. 23 Year 2014 proctors sent student’s responses from the about Local Governance, the local server to the central server. Given management of non-formal education, this fact, there is no worry about the including equity education is under the internet connection for CBT authority of District Education Offices implementation. for both Package B and C programs. This In terms of national assessment time offers opportunities to have better table, BSNP (2017) made a policy to coordination and arragemant in conduct CBT for non-formal education implementing CBT for non-formal on Saturday and Sunday. This policy education. differed from that of formal education In addition, there is effective where the national assessment is done cooperation among stakeholders, such as during the weekdays. The main reason government sectors especially state for this is to give more opprotunities to electricity company (PLN) which ensured leaners in taking national assessment. that there is no electricity shortage during Based on an evaluation done by BSNP the implementation of CBT. PLN also (2016), the presence of national provided generator as an anticipating assessment takers from non-formal action if the power supply encountered eduation was low because they did not troubles. Policy made by Mayor or get permission from their employers to Governor in each region to apply CBT in leave the work place. Having this policy, ational assessment was considered as it is most likely the idea of resource another opportunity. In 2016, Surabaya sharing for CBT national assessment in city was the only city that carried out non-formal education can be successfully CBT in all schools. By 2017, all schools implemented. in the Province of DKI and Province of

Surabaya, 14 October 2017 191

Challenges and opportunities in implementing Computer Based Test (CBT)... B. Suryadi, Y. Rahmawati

The above mentioned advanteges of of Education and Culture also provided CBT supported the previous studies public services by establishing a helpdesk (Hosseini, Zainol Abidin, and team. The main duties and tasks of this Baghdarnia, 2014; Boeve et al., 2015; team were to provide assisstance for any Piaw, 2012; and Retnawati, 2015). complains and difficulties occurred in the Hosseini et al. (2014), for example, found field during the implementation of CBT that among the advantages of CBT are for non-formal education. The helpdesk cost-effective administration, ease of team also created social media groups to administration, more accuracy, coordinate and share informations on immediacy of scoring and reporting, and technical knowledge, problem solving, flexible test scheduling and location. and policies related to CBT Similarly, Retnawati (2015) pointed that implementation for non-formal CBT offers the richness of the interface, education. The use of social media, such for example, the use of graphics allows a as facebook, whatapps, twitter and so dynamic presentation of the test content. forth in disseminating information is an In addition to above mentioned innovation that deserved appreciations. opportunities, there were several effective Another innovation done by the practices found during the implementaion helpdesk team under the Ministry of of CBT for non-formal education. These Educatioin and Culture was the notion of practices were considered as giving tips to leaners who wanted to take improvement and event innovation in the CBT for the first time. The messages organizing national assessment for non- were clearly articulated in simple formal education in Indonesia and it language as so it is easy for students of might be adapted in other situation where non-formal education to understand. the CBT is conducted for non-formal Below were examples of the socialization education. materials in order to make students feel Report by Puspendik (2017) showed comfortable with CBT (Kemdikbud, that there was cooperation between the 2017). Basically, this tips were meant to Ministry of Education and Culture assist students during the implementaion (MoEC) and the Ministry of Religious of CBT. For instance, it mentioned that Affairs, Ministry of Internal Affairs, students should stay calm and relaxed Ministry of Communication and when they enter assessment room and Information, Higher Education whacted the computer on the table. They Institutions, Provincial and District have start by reciting prayer to God and Education Offices, Center for Education then to read carefullly the instruction Quality Assurance (LPMP), State given on the computer screen. They were Electricity Company (PLN), internet also advised to raise up their hands when network providers, school principals, they encounter technical problems as to teachers, and parents. Those involved in proctors could help them. the implementaion of CBT for non- The tips also mentioned about the formal education had the same ambitions importance of time management. Each to make CBT a success. Therefore, they subject is allocated with 120 minutes for dedicated their times, efforts, and 40 to 50 assessment items. In this thougths in order to make CBT run context, it is advised that students to do smoothly as this year was the first time the easiest questions first, instead of the CBT conducted for non-formal most difficult ones. Therefore, time educatioin. A success story of CBT management is important for national implementation this year will have a asessment takers. Most importantly, great impact on the implementation of students were advised to observe honesty CBT for non-formal education in future. during the test. This is because honesty is During the implementaion of CBT the best policy. If they were unable to for non-formal education, the central observe honesty in the national organizing committee under the Ministry

Publishing : Faculty of Education, Universitas Negeri Surabaya 192

Proceeding the 1st International Conference on Education Innovation (ICEI) Page 184-194 ISBN : 978-602-50898-0-0 assessment, their personal integrity is education institutions throughout the questionable. nation. Considering the above mentioned Despite the challenges found in this opportunities and challenges as well as study, the findings of this study have effective practices in implementing CBT several implications in improving CBT for non-formal education, the following implementation for non-formal education assessment takers’ testimony from a test in future.The resource sharing between taker in Banjarmarin South Borneo, gives formal and non-formal education units, more evidence on the success story of integrated database of national CBT implementation for non-formal assessment participants, and effectives education. practices found in certain districts during “As a national assessment the implementation of national taker, I felt that the use of CBT assessment, should be taken into in national assessment for consideration in future. Most importantly, non-formal education was road map and milestones for improving more effective than PBT. It the acceptability and credibility of was very simple and user national assessment for non-formal friendly. I have more time to education should be clearly designed and check and recheck our well articulated.

answers. In case I did mistake and want to change the REFERENCES answer, I just move the cursor American Educational Research Association. to the destinated answer, (2014). Standards for educational and without rubbing the answer as psychological testing. Washington DC: it is done in PBT”. American Educational Research Student’s testimony as mentioned Association. above clearly indicates that students Badan Pusat Statistik. (2014). Penduduk Indonesia enjoyed taking CBT in national Hasil Sensus Penduduk 2010. Available in: assessment for non-formal education. https://www.bps.go.id/index.php/publikasi/ This could be due to the fact that students 14 [retrieved on 25 June 2017] were called as Y-Generation as Suryadi Badan Standar Nasional Pendidikan/BSNP. (2015) said that one of their (2015). Laporan Hasil Monitoring Ujian characteristics is curiosity to explore new Nasional. Jakarta: BSNP. [In Indonesian] things, including new technology. Thus, students who are familiar with compputer Badan Standar Nasional Pendidikan/BSNP. feel more comfortable taking CBt in (2016). Laporan Hasil Monitoring Ujian national assessment. Nasional. Jakarta: BSNP. [In Indonesian] Badan Standar Nasional Pendidikan/BSNP. 3. CONCLUSIONS (2017). Prosedur Operasional Standar The results of this study showed that Penyelenggaraan Ujian Nasional Tahun the policy of CBT implementation in Pelajaran 2016/2017. Jakarta: BSNP. [In national assessment for non-formal Indonesian] education faced numerous challenges, Boevé, A. J., Meijer, R. R., Albers, C. J., Beetsma, including resistance from the students Y., & Bosker, R. J. (2015). Introducing and stakeholders, technical problems, computer-based testing in high-stakes students’ competences in computer skills, exams in higher education: Results of a and public engagement. However, the field experiment. PloS one, 10(12), policy was regarded as an innovation for e0143616. improving the quality of non-formal education and assessment system. Carrillo-de-la-Peña, M. T., & Pérez, J. (2012). Continuous assessment improved academic Therefore it needs improvement for achievement and satisfaction of psychology further implementation for all non-formal

Surabaya, 14 October 2017 193

Challenges and opportunities in implementing Computer Based Test (CBT)... B. Suryadi, Y. Rahmawati

students in Spain. Teaching of Psychology, Retnawati, H. (2015). The Comparison of 39(1), 45-47. Accuracy Scores on the Paper and Pencil Testing vs. Computer-Based Testing. Hosseini, M., Abidin, M. J. Z., & Baghdarnia, M. TOJET: The Turkish Online Journal of (2014). Comparability of Test Results of Educational Technology, 14(4). 135-142. Computer based Tests (CBT) and Paper and Pencil Tests (PPT) among English Suryadi, B. (2015). Generasi y: karakteristik, Language Learners in Iran. Procedia-Social masalah, dan peran konselor. Seminar dan and Behavioral Sciences, 98, 659-667. Workshop Internasional MALINDO 4 di Bali, 22-23 Mei 2015. Diselenggarakan Kementerian Pendidikan dan Kebudayaan. (2003). oleh Asosiasi Bimbingan dan Konseling Undang-Undang Republik Indonesia Indonesia (ABKIN). Nomor 20 Tahun 2003 Tentang Sistem Pendidikan Nasional. Jakarta: Kemdikbud. Wass, V., Van der Vleuten, C., Shatzer, J., & [In Indonesian] Jones, R. (2001). Assessment of clinical competence. The Lancet, 357(9260), 945- Kementerian Pendidikan dan Kebudayaan. (2005). 949. Peraturan Pemerintah Nomor 19 Tahun 2005 Tentang Standar Nasional Pendidikan. Jakarta: Kemdikbud. [In Indonesian]

Kementerian Pendidikan dan Kebudayaan. (2017). Peraturan Menteri Pendidikan dan Kebudayaan Nomor 3 Tahun 2017 tentang Penilaian Hasil Belajar oleh Pemerintah dan Penilaian Hasil Belajaar oleh Satuan Pendidikan. Jakarta: Kemdikbud. [In Indonesian]

Kementerian Pendidikan dan Kebudayaan.(2017). UNBK, Kenapa Tidak? Jendela Pendidikan dan Kebudayaan: Media Komunikasi dan Inspirasi. Vol. X. April 2017. Hal. 18-19.

Linn, R. L., & Miller, M. D.(2005). Measurement and assessment in teaching. New Jersey: Prentice Hall.

Nizam (2017). Karakteristik pembelajaran pendidikan kesetaraan. Bahan paparan dalam workshop kurikulum 2013. Jakarta: Pusat Kurikulum dan Perbukuan. [In Indonesian]

Noor, Mohd, M. Z. (2008). The role of national examination in maintaining national education quality. Book Chapter in Laporan Seminar Sehari: Examination Authority. Jakarta: Pusat Penilaian Pendidikan, Kemdikbud.

Piaw, C. Y. (2012). Replacing paper-based testing with computer-based testing in assessment: Are we doing wrong?. Procedia-Social and Behavioral Sciences, 64, 655-664.

Pusat Penilaian Pendidikan/Puspendik. (2017). Data peserta Ujian Nasional Pendidikan Non-formal tahun pelajaran 2016/2017. Jakarta: Puspendik.

Publishing : Faculty of Education, Universitas Negeri Surabaya 194