International Journal of Learning, Teaching and Educational Research

IJLTER.ORG

International Journal of Learning, Teaching And Educational Research

2493

2116

ISSN:1694

ISSN:1694 -

- Vol.16 No.1 e

PUBLISHER International Journal of Learning, Teaching and London Consulting Ltd Educational Research District of Flacq Republic of Mauritius www.ijlter.org The International Journal of Learning, Teaching and Educational Research is an open-access Chief Editor journal which has been established for the dis- Dr. Antonio Silva Sprock, Universidad Central de semination of state-of-the-art knowledge in the Venezuela, Venezuela, Bolivarian Republic of field of education, learning and teaching. IJLTER welcomes research articles from academics, ed- Editorial Board Prof. Cecilia Junio Sabio ucators, teachers, trainers and other practition- Prof. Judith Serah K. Achoka ers on all aspects of education to publish high Prof. Mojeed Kolawole Akinsola quality peer-reviewed papers. Papers for publi- Dr Jonathan Glazzard cation in the International Journal of Learning, Dr Marius Costel Esi Teaching and Educational Research are selected Dr Katarzyna Peoples through precise peer-review to ensure quality, Dr Christopher David Thompson originality, appropriateness, significance and Dr Arif Sikander Dr Jelena Zascerinska readability. Authors are solicited to contribute Dr Gabor Kiss to this journal by submitting articles that illus- Dr Trish Julie Rooney trate research results, projects, original surveys Dr Esteban Vázquez-Cano and case studies that describe significant ad- Dr Barry Chametzky vances in the fields of education, training, e- Dr Giorgio Poletti learning, etc. Authors are invited to submit pa- Dr Chi Man Tsui pers to this journal through the ONLINE submis- Dr Alexander Franco Dr Habil Beata Stachowiak sion system. Submissions must be original and Dr Afsaneh Sharif should not have been published previously or Dr Ronel Callaghan be under consideration for publication while Dr Haim Shaked being evaluated by IJLTER. Dr Edith Uzoma Umeh Dr Amel Thafer Alshehry Dr Gail Dianna Caruth Dr Menelaos Emmanouel Sarris Dr Anabelie Villa Valdez Dr Özcan Özyurt Assistant Professor Dr Selma Kara Associate Professor Dr Habila Elisha Zuya

VOLUME 16 NUMBER 1 January 2017

Table of Contents

Item Consistency Index: An Item-Fit Index for Cognitive Diagnostic Assessment ...... 1 Hollis Lai, Mark J. Gierl, Ying Cui and Oksana Babenko

Factors That Determine Accounting Anxiety Among Users of English as a Second Language Within an International MBA Program ...... 22 Alexander Franco and Scott S. Roach

(Mis)Reading the Classroom: A Two-Act “Play” on the Conflicting Roles in Student Teaching ...... 38 Christi Edge

Coping Strategies of Greek 6th Grade Students: Their Relationship with Anxiety and Trait Emotional Intelligence ...... 57 Alexander- Stamatios Antoniou and Nikos Drosos

Active Learning Across Three Dimensions: Integrating Classic Learning Theory with Modern Instructional Technology ...... 72 Thaddeus R. Crews, Jr.

The Effects of Cram Schooling on the Ethnic Learning Achievement Gap: Evidence from Elementary School Students in Taiwan ...... 84 Yu-Chia Liu, Chunn-Ying Lin, Hui-Hua Chen and He Huang

Teachers’ Self-Efficacy atMaintaining Order and Discipline in Technology-Rich Classrooms with Relation to Strain Factors ...... 103 Eyvind Elstad and Knut-Andreas Christophersen

Using Reflective Journaling to Promote Achievement in Graduate Statistics Coursework...... 120 J. E. Thropp

Competence and/or Performance - Assessment and Entrepreneurial Teaching and Learning in Two Swedish Lower Secondary Schools ...... 135 Monika Diehl and Tord Göran Olovsson

Review in Form of a Game: Practical Remarks for a Language Course ...... 161 Snejina Sonina 1

International Journal of Learning, Teaching and Educational Research Vol. 16, No. 1, pp. 1-21, January 2017

Item Consistency Index: An Item-Fit Index for Cognitive Diagnostic Assessment

Hollis Lai,1 Mark J. Gierl,2 Ying Cui,2 Oksana Babenko3

1 School of Dentistry, Faculty of Medicine & Dentistry 2Centre for Research in Applied Measurement and Evaluation 3Department of Family Medicine, Faculty of Medicine & Dentistry University of Alberta, Canada

Abstract. An item-fit index is a measure of how accurately a set of item responses can be predicted using the test design model. In a diagnostic assessment where items are used to evaluate student mastery on a set of cognitive skills, this index helps determine the alignment between the item responses and skills that each item is designed to measure. In this study, we introduce the Item Consistency Index (ICI), a modification of an existing person- model fit index, for diagnostic assessments. The ICI can be used to evaluate item-model fit on assessments designed with a Q-matrix. Results from both a simulation and real data study are presented. In the simulation study, the ICI identified poor-fitting items under three manipulated conditions: sample size, test length, and proportion of poor-fitting items. In the real-data study, the ICI detected three poor-fitting items for an operational diagnostic assessment in Grade 3 mathematics. Practical implications and future research directions for the ICI are also discussed. Keywords: Item Consistency Index; cognitive diagnostic assessment; test development

Introduction In educational testing, items are developed to elicit a correct response when examinees demonstrate adequate knowledge or understanding on the required tasks and skills within a specified domain. The methods of specifying knowledge, the conceptualization of content domains, and the design of how an item elicits responses are currently undergoing significant change with the evolution of our test designs. But one outcome that remains the same is that an item must assess the tasks and skills as intended, and the quality of each item must be judged to be high if it is to be included on the test. In most test designs, item discrimination power is a statistical criterion that is synonymous with describing item quality.

Item discrimination helps describe how well an item can differentiate examinees at different performance levels. Depending on the test design and how the scale of examinee performance is realized, different measures of item discrimination may be used. Additional information about item discrimination can also be garnered from measures of item- model fit. An item-model fit index describes the overall difference between real responses on a given item with a corresponding set of expected responses predicted by the test design. Item-model fit indices can be summarized, in general, as a ratio between the expected and actual correct responses on each item to compare the proportion of correct responses across examinees of different abilities with an expected correct proportion from the test design model. Different criterions that represent the examinee overall performance such as total score, estimated ability, or pseudo-scores have been used to group the responses of examinees' with similar ability to produce variations of item-model fit (Bock, 1972; Yen, 1981; Rost & von Davier, 1994; Orlando & Thissen, 2003). Application of item-model fit indices include the identification of poor performing items, cheating, or test administration anomalies, along with addressing issues related to dimensionality, item construction, calibration, and model selection (Reise, 1990).

Cognitive Diagnostic Assessment and Model Fit Demand for more assessment feedback to better guide instruction and learning has led to the development of more complex test designs. Cognitive diagnostic assessment (CDA) is an example of a test design that yields enhanced assessment feedback by providing test takers with specific information about their problem-solving mastery on a given domain (Gierl, Leighton, & Hunka, 2007). The cornerstone of a CDA is the use of a cognitive model to guide test development. The use of a cognitive model allows CDA to provide enhanced feedback because cognitive information can be extracted from the examinees’ item responses which, in turn, provide more detailed and instructionally relevant results to test takers. Compared to traditional tests where an item response is linked to a single outcome scale, the cognitive inferences made in CDA allow each item to measure multiple skills related to student learning. Due to the complexity of interpreting and modeling different aspects of cognitive skills, many approaches to modeling and scoring examinee responses are available. Sinharay, Puhan, and Haberman (2009) summarized three common features among different methods of CDA: (1) tests assess student mastery based on a cognitive model of skills; (2) items probe student mastery on a pattern of skills expressed in a Q- matrix; and (3) items probing the same pattern of skill mastery should elicit a similar pattern of student responses.

An essential part of CDA development relies on the definition of a Q- matrix. The Q-matrix is an item-by-attribute matrix used to describe the skills probed by each item. For example, if a CDA is designed to determine examinee mastery on four skills, and an item was designed to elicit a correct response if the examinee has mastered the first and the fourth skill, then the row corresponded to that item in the Q-matrix would be expressed as {1,0,0,1}. The Q-matrix and the student response patterns are used to calibrate the model parameters and provide students with diagnostic results related to their cognitive problem-solving strengths and weaknesses.

To ensure that CDA results provide the most accurate information to examinees about their cognitive skills, the quality of CDA items must be scrutinized. The evaluations of the claim that items are to probe a specified set of skills have varied by the scope of how item-skill relations are represented. Model-data fit has traditionally been used to evaluate how items are aligned with construct of the skills based upon item responses. Few studies have investigated the relations of item-skill alignment. Wang, Shu, Shagn, and Xu (2015) have developed a measure which allows the evaluation of skill-to-item fit based on the DINA model that assumes a probabilistically scaled skill representation. To evaluate item-model fit in CDA, items need to be evaluated beyond the relationship of the correct responses on a particular item and single outcome scores. Because each item is designed to provide student mastery information on multiple skills, an item-model fit index is needed to ensure item responses are aligned with the intended cognitive skills.

Evaluating Model-Fit for CDA The rationale evaluating model fit in CDA can be considered in two approaches, evaluating the fit with the expected psychometric properties of the test items or evaluating the fit of responses with the blueprint of skills. Existing developments tend to focus on the former approach. For example, Jang (2005) compared total raw score distributions between observed and predicted responses using the mean absolute difference (MAD). Jang’s approach to evaluating model-fit is akin to IRT model fit approaches, where the level of fit is determined by total score differences between the expected and examinee results. But with each correct response of a CDA item linked to mastery on a vector of skills, evaluating item-model fit for CDA need to consider the fit of an item with the prerequisite skills rather than a single test-level outcome.

Sinharay and Almond (2007) also developed an approach for evaluating item fit for CDA by assuming that examinees categorized with the same skill pattern should also have the same diagnostic outcome. With their

approach, the proportion correct response for examinees with the same skill pattern is compared with the expected proportion predicted by the cognitive model. Differences between the expected and observed correct proportions are then summed across all skill patterns and weighted proportionally by sample size. That is, model-fit for item j was defined as:

2 2 . 푁푘 (푂푘푗 −퐸푘푗 ) 푋푗 = 푘 , 퐸푘푗 (푁푘 −퐸푘푗 ) where 푁푘 is the number of examinees with skill pattern k, 푂푘푗 is the number of examinees with skill pattern k that responded correctly to item j, and 퐸푘푗 is the product of the expected proportion of correct response for pattern k multiplied by 푁푘 . Although this approach can be applied to account for fit among multiple sets of skills, results rely on an expected correct response rate of a given item for each skill pattern. As the expected correct response for a given set of skill pattern is not readily available, application of this method for determining model fit may be problematic. Moreover, a poor sample representation of a skill pattern or psychometrically indistinguishable skill patterns will also misestimate item-model fit. One way to avoid the influence of misclassification on an item-model fit measure for CDA is to comparatively evaluate items that measure the same skills. That is, items measuring the same skills are expected to elicit similar response patterns with one other. Hierarchy Consistency Index (HCI) One statistic developed specifically for CDA to evaluate person-model fit is the Hierarchy Consistency Index (HCI; Cui & Leighton, 2009; Cui & Li, 2014; Cui & Mousavi, 2015). The HCI is a statistic for evaluating the fit of the observed responses from an examinee with the expected responses from a CDA model based on a comparison between the observed and expected response vectors. The main assumption for the HCI is that if an examinee gives a correct response to an item requiring a set of skills, then the examinee is assumed to have mastered that set of skills and therefore should also respond correctly to items that designed to measure those skills. For example, if an examinee gives a correct response to an item that requires the first and third skill in a CDA that assess four skills (or an item with a skill pattern of [1,0,1,0] in the Q matrix), then the examinee is also expected to respond correctly to items that probe the same set of skills [1,0,1,0], or a subordinate or prerequisite of those skills (e.g., [1,0,0,0] , [0,0,1,0]), which require skills should have been acquired. In this manner, the number of misfitting responses across all items with their corresponding subsets of skills is calculated for each examinee to determine an index of person-fit.

Given I examinees were administered with J items, the HCI for examinee i is calculated as: 퐽 . 2 푗=1 𝑔휖 푠 푋푖푗 (1−푋푖𝑔 ) 퐻퐶퐼 = 1 − 푗 , (1) 푖 푁 where Xj is the examinee’s scored response for item j, sj is an index set that includes items requiring the subset of attributes measured by item j, and Xg is the examinee’s scored response for item g. For example, if item j is answered correctly, then all items that measure the attributes or a subset of attributes probed by item j is represented by index set sj , where g is an item index within sj. N is the number of comparisons made across all sj. The HCI has a maximum of 1 and minimum of -1, where a high positive HCI value represents good person-fit with the expected response model.

The HCI is a useful index for analyzing person-fit across different types of CDAs, as it requires only the use of the Q-matrix and examinee responses. In this study, we modify the HCI to create an index for analyzing item- model fit. Thus, the purpose of this study is twofold. First, we introduce and define an item-model fit index called the item consistency index (ICI). The ICI is used to evaluate the fit of an item related to the underlying cognitive model used to make diagnostic inferences with that item. Second, we present results from two studies to demonstrate both the simulated and practical performance of the ICI across of host of testing conditions typically found in diagnostic assessments.

Item Consistency Index (ICI) As elaborated earlier, the HCI measures the proportion of misfitting observed examinee responses relative to the expected examinee responses on a diagnostic assessment. This principle can also be extended to evaluate item-fit. With the HCI, the misfitting responses related to each item is summed across all items for each examinee. As described in (1), misfit for examinee i (mi) can be written as:

퐽 . 푚푖 = 푗 𝑔휖푠푗 푋푖푗 (1 − 푋푖𝑔 ) . (2)

Alternatively, to evaluate the misfit for item j, the number of misfitting responses from the subset of item j can be summed across all examinees. This modification can be written as:

. 푚푗 = 푖 𝑔∈푆푗 푋푖푗 1 − 푋푖𝑔. , (3)

where 푋푖 is student 푖’s score (1 or 0) to item 푗, and 푋푖𝑔 is student 푖’s score (1 or 0) to item 𝑔. Item g belongs to 푆푗 , a subset of items that require the

subset of skills measured by item j. In this manner, for a correct response to item j for examinee i (푋푖푗 = 1), one can consider any incorrect responses in 푆푗 to be a misfit for examinee i. The number of misfits is then summed across all examinees.

It should be noted that the HCI only considers students’ correct responses for analyzing misfit of a given item (푋푗 = 1). That is, misfit is calculated against the required skills only when students have provided the correct response. While this was adequate for analyzing misfit for person-fit, analyzing item-fit against a cognitive model also requires comparisons to be made when students respond to an item incorrectly (푋푗 = 0). As such, an evaluation of item-fit needs to account for this alternative comparison. For example, suppose an incorrect response was given on our exemplar item that required the skill pattern of [1,0,1,0]. From this item response, we could infer that the examinee does not possess all the necessary skills required to solve this item and, therefore, should respond incorrectly to all items that require the same skill pattern of [1,0,1,0]. Furthermore, the examinee should also respond incorrectly to items that require more skills than the current item (i.e., [1,1,1,0], [1,0,1,1], [1,1,1,1]). These items that require the same skill or a more complex skill pattern can be ∗ conceptualized as an alternative subset of item j (푆푗 ), and a correct ∗ response in any of the items belonging to 푆푗 can be conceptualized as a misfit. This outcome can be expressed as:

∗ . 푚 = ∗ 푋 (1 − 푋 ) 푗 푖 ℎ∈푆푗 푖ℎ 푖푗 . (4)

The set of alternative comparisons combined with comparisons from correct responses form the numerator of the ICI. To maintain the same scale of comparison with HCI, the numerator is then divided by the total number of comparisons, which effectively transforms the outcome to a proportion of misfit responses for item j. The proportion is then rescaled to a maximum of 1 and a minimum of -1. The ICI for item 푗 is then given as:

2 푋 (1−푋 )+ ∗ 푋 (1−푋 ) 푖 𝑔∈푆푗 푖푗 푖𝑔 ℎ∈푆 푖ℎ 푖푗 퐼퐶퐼 = 1 − 푗 , (5) 푗 푁 푐푗

where 푋푖푗 is student 푖’s score (1 or 0) to item 푗, 푆푗 is an index set that includes items requiring the subset of attributes measured by item 푗, 푋푖𝑔 is ∗ student 푖’s score (1 or 0) to item 𝑔 where item 𝑔 belongs to 푆푗 , 푆푗 is an index set that includes items requiring all, but not limited to, the attributes measured by item 푗, 푋푖ℎ is student 푖’s score (1 or 0) to item ℎ ∗ where item ℎ belongs to 푆푗 , and 푁푐푗 is the total number of comparisons for

item 푗 across all students.

To illustrate the calculation of the ICI, consider a hypothetical administration of a CDA with 15 items and a Q-matrix presented in (6).

1 0 0 0 0 1 0 0 1 1 0 0

0 0 1 0

1 0 1 0 0 1 1 0 1 1 1 0 0 0 0 1 . (6) 1 0 0 1 0 1 0 1 1 1 0 1 0 0 1 1 1 0 1 1 0 1 1 1 1 1 1 1

Suppose this CDA of four skills was administered to an examinee who produced the item response vector (0,0,0,0,0,1,1,0,0,0,0,0,0,0,0). That is, the examinee responded correctly to items 6 and 7 only. To calculate the ICI for item 6, we first consider that the examinee has responded to the item correctly, therefore comparisons should be made with items that require skills that are prerequisites to or same with the original item. In this case, items 2 and 4 belong in 푆6. Since both item responses were incorrect, two comparisons were made (푁푐6 = 2) and two unexpected responses were found (푚6 = 2) for this examinee. In addition, suppose we wanted to ∗ calculate the ICI of item 2 for this examinee. The alternative subset (푆푗 ) will be needed since the examinee responded to the item incorrectly. For ∗ this instance, seven items form the alternative subset for item 2 (푆2 = {3,6,7,10,11,14,15}). Since the examinee responded correctly to item 6 and 7, there were two unexpected responses (푚2 = 2) from a total of seven comparisons (푁푐2 = 7). In this manner, the number of unexpected responses and comparisons are summed across all examinees and rescaled to form the ICI.

To demonstrate the performance of this item-model fit index across a variety of different testing situations, a simulation study was conducted to determine the performance of ICI for detecting poor-fitting items. Then, a real data study was conducted to demonstrate how the ICI can be applied in operational testing situations a CDA in Mathematics.

Methods and Results

Study 1: Simulation Study To evaluate how well the ICI can identify items that fit poorly relative to their underlying cognitive model, a Monte-Carlo study was conducted by simulating responses from a diagnostic test designed to measure seven skills. To determine the performance of the ICI using simulated CDA data, examinee responses were generated under the Bernoulli distribution. In addition to generating examinee responses, different testing conditions were manipulated to probe conditions that may occur in a real CDA administration. Finally, to classify poor-fitting items using ICI, a common evaluation criterion was used to determine which items were fit poorly with the given cognitive model.

The simulation process is similar to the actual steps used in developing CDAs (Gierl, Leighton, & Hunka, 2007), where cognitive model, items, and responses were developed in a sequential manner. First, an existing cognitive model from Cui and Leighton (2009) was used to guide the simulation process. The cognitive model consists of seven skills, with 15 patterns of skill mastery identified as permissible. The patterns of required skills for each item are expressed in the Q-matrix presented in Table A1 in the Appendix. To generate examinee responses, examinees were first assigned to an expected pattern of skill mastery from one of the 15 skill patterns. In addition to the 15 skill patterns, a null pattern [0,0,0,0,0,0,0] was also used to represent examinees who did not master any skills. In total, sixteen expected skill patterns are distributed equally among the sample examinees. To simulate response for an examinee on a given item, the examinee’s assigned skill pattern is compared with the skills required by that item as indicated by the Q matrix. A probability of correct response is assigned based on whether the examinee has all the prerequisite skills of the item. Based on this assigned probability, the examinee’s response to each item was generated using a Bernoulli function.

To simulate the effectiveness of ICI under different testing conditions, three factors were manipulated. First, the number of items representing each skill pattern in the CDA was varied by three levels. If a CDA is lengthened by including multiple items probing the same set of skills, then the reliability of each corresponding skill measured is expected to increase (Gierl, Cui, & Zhou, 2009). In our study, the number of items in the CDA varied by one, two, or three items representing each possible skill pattern. These three levels of variation on a total of 15 skill patterns resulted in test lengths of 15, 30, and 45 items, respectively.

Second, unlike the related person-fit HCI which is independent of sample size, the ICI is based on the proportion of misfit responses from all examinees. Therefore, different sample sizes may affect the outcome of the ICI. Three levels of sample sizes were manipulated: 800, 1600, and 2400. Since the 15 skill patterns and a null pattern are distributed equally among the examinees, the numbers of examinees representing each skill pattern are 50, 100, and 150, respectively.

Third, an important feature for an item-model fit index is to detect items that fit poorly with the expected response determined by cognitive model. This concept is contaminated when the ICI is influenced by misfitting items related to the skills of the original item. To investigate whether the proportion of poor-fitting items have an effect on the ICI, the proportion of poor-fitting items were manipulated at three levels proportional to the test length: 5%, 10% and 25%. In Cui and Leighton (2009), a well-fitting item was deemed to have a 10% chance for slips, where an examinee without mastery of the necessary skills will have a 10% chance of responding correctly while an examinee who has mastered the necessary skills will have a 90% chance of responding correctly. While there can be many reasons for an item to fit poorly with the underlying cognitive model (e.g., model misspecification, item quality, option availability), generally a poor-fitting item yields a response that is aberrant from the cognitive model. To simulate a poor-fitting item, items responses were generated close to random. Table 1 contains the probabilities of correct response given the level of item fit (good or poor fit) and whether the examinee possesses the required set of skills. Taken together, three manipulated factors with three levels each yielded a total of 27 conditions as shown in Table A2 of the Appendix.

Table 1. Correct response probability given the level of item fit and whether the examinee possesses the required set of skills

Item Fit Required skills Good Poor Present 0.9 0.6 Not present 0.1 0.4

To evaluate the effectiveness of the ICI for detecting poor-fitting items, a criterion is needed for the ICI to differentiate between poor- and well- fitting items. A classification approach was used to measure the precision of the ICI in this study. A cut-score criterion, set at an ICI value of 0.5, was used to illustrate the classification characteristics for poor-fitting items. For example, if an item was calculated to have an ICI value of less than 0.5, then that item was deemed to fit poorly with the expected responses from the cognitive model. This preliminary criterion for dichotomizing

item fit was needed because no point of comparison currently exists in determining an appropriate level of fit with an existing cognitive model. Further, an ICI value of 0.5 for any item translates to roughly 75% of the responses on a given item fitting with the expected skill pattern as defined by the cognitive model. Using this initial cut-score, we could then classify items as poor- or well-fitting.

To ensure the classification results were consistently produced, each of the 27 testing conditions was replicated 100 times. The dependent variables for the simulation study included the average proportion of correctly identified poor-fitting items and misclassification of well-fitting items across all conditions. The simulation environment, the implementation of the ICI, and the replication of results were programmed in R (R Core Development Team, 2011), and are available from the first author.

Table 2 contains a summary of the mean ICIs for each condition. The mean ICIs were calculated separately for the poor- and well-fitting items. The overall mean for poor-fitting items was 0.30 whereas the mean ICI for well-fitting items was 0.53. Three observations must be noted from the results in Table 2. First, test length tended to have a positive impact on the values of ICI. For example, CDAs with only one item measuring each skill pattern (i.e., test length=15) had consistently lower ICIs compared to CDAs with two or three items measuring each skill (i.e., test length=30 or 45). Second, as expected, the magnitude of the mean ICI differences between poor and well-fitting items tended to decrease when an increase in poor-fitting items included in the ICI. Third, the means of ICI were relatively stable across different sample sizes for each condition.

Table 2. Summary of the mean ICIs across the three variables manipulated in the simulation study

Proportion of Mean ICI Sample Test Poor Fitting Poor Fitting Well-Fitting Size Length Items Items Items 800 5% 15 0.24 0.49 5% 30 0.22 0.57 5% 45 0.30 0.59 10% 15 0.31 0.48 10% 30 0.29 0.56 10% 45 0.38 0.58 25% 15 0.37 0.43 25% 30 0.29 0.56 25% 45 0.32 0.51

1600 5% 15 0.21 0.41 5% 30 0.22 0.56 5% 45 0.29 0.59 10% 15 0.27 0.44 10% 30 0.29 0.57 10% 45 0.38 0.58 25% 15 0.36 0.41 25% 30 0.29 0.56 25% 45 0.32 0.51 2400 5% 15 0.24 0.55 5% 30 0.23 0.58 5% 45 0.30 0.59 10% 15 0.32 0.53 10% 30 0.30 0.57 10% 45 0.38 0.58 25% 15 0.32 0.53 25% 30 0.29 0.56 25% 45 0.32 0.51

Items were also classified based on the cut-score criterion. This simulation process was repeated 100 times, with the correct classification rate, or power, being the likelihood of correctly identifying a poor-fitting item using the ICI across the conditions in the simulation study. The power values for the 27 conditions are shown in Table 3. The conditions with the highest power were found in CDAs with the longest test-length (45), specifically with conditions that had the largest proportion of poor-fitting items (25%). Under those conditions, the highest power was 0.99, meaning that for the ICI criterion of 0.50, 99% of all poor-fitting items were correctly classified across 100 replications. The lowest power values were found in conditions with the smallest sample size (800), where a power of 0.67 was found for a 30-item CDA with 5% of poor-fitting items and 1600 examinees.

Table 3. Power of ICI for identifying poor-fitting items

Test Sample Proportion of Poor-Fitting Items Length Size 5% 10% 25% 15 800 0.68 0.76 0.92 1600 0.93 0.89 0.95

2400 0.79 0.79 0.92

30 800 0.67 0.73 0.79 1600 0.77 0.74 0.81

2400 0.73 0.72 0.79

45 800 0.76 0.80 0.99 1600 0.77 0.83 0.99

2400 0.76 0.81 0.99

Table 4 summarizes the likelihood of a well-fitting item being misclassified by the ICI as a poor-fitting item in each condition. The lowest misclassification rates were associated with CDAs that have the longest test-length (45) and the smallest proportion of poor-fitting items (5%). Under those conditions, the lowest misclassification rate was 15%. The highest error rates were observed with the shortest test length (15), where misclassification was 78%.

Taken together, the simulation study results highlight important trends and outcomes that can be used to interpret how accurately the ICI identifies poor-fitting items. The power values of ICI were erratic when the number of items probing each skill pattern was small, but stabilized as the number of items representing each skill pattern increased. For example, each increase in test length resulted in a decrease in the variation of power values among the same proportion of poor-fitting items and between different sample sizes. This finding suggests that the reliability of using the ICI to classify poor-fitting items is related to the reliability of the CDA as a whole. Moreover, the proportions of misclassification were approximately 2.5 times higher in CDAs with a single item representing each test skills as compared to the other two levels. This outcome further supports the conclusion that as skills are measured more accurately, the ICI better distinguishes poor- from well- fitting items.

Table 4. Misclassification rate of ICI in identifying well-fitting items

Test Sample Proportion of Well-Fitting Items Length Size 5% 10% 25% 15 2400 0.28 0.35 0.66 1600 0.78 0.65 0.72 800 0.50 0.50 0.66 30 2400 0.16 0.20 0.22 1600 0.28 0.20 0.27 800 0.27 0.22 0.24 45 2400 0.15 0.18 0.33 1600 0.17 0.19 0.34 800 0.15 0.19 0.33

There were no obvious trends that the sample size manipulated across the three levels yielded important differences among the power or misclassification of well-fitting items. This finding suggests that the sample sizes used in this study do not yield important ICI differences across our study conditions. This outcome could also suggest that the representation of approximately 50 examinees per skill pattern may be sufficient for evaluation of the ICI.

When the proportion of poor-fitting items was manipulated, the power increased with the proportion of poor-fitting items in the CDA, where the overall power rose as the proportion of poor fitting item increased. An increase of poor-fitting items also yielded more misclassification of well- fitting items. This finding suggests that poor-fitting item responses contribute to an overall decrease in the magnitude of ICI, where the resulting errors are reflected using the classification criterion of 0.50.

Study 2: Use Case Application The purpose of the second study is to demonstrate how the ICI can be used to identify poor-fitting items in an operational CDA. The ICI was used to evaluate item-model fit for a CDA program designed to assess students’ knowledge and skills in Grade 3 mathematics. From this CDA program, 324 students responded to an 18-item CDA (see Gierl, Alves, & Taylor-Majeau, 2010).

The CDA we used was designed to evaluate student mastery for subtraction skills. Each item was designed to yield specific diagnostic information in a hierarchy of cognitive skills were the first skill was the easiest (Subtraction of two consecutive 2 digit numbers) and the last skill was most difficult (Subtraction of two 2 digit numbers using the digits 1

to 9 with regrouping). The CDA was developed as follows. First, a cognitive model of task performance was created by specifying the cognitive skills necessary to master subtraction in Grade 3. The domain of subtraction was further specified into a set of six attributes related in a linearly hierarchical manner by a group of subject matter experts. The attributes produced a total of seven unique patterns of skill mastery (six plus null). Three items were created by content experts to probe student mastery on each attribute to ensure adequate representativeness of each skill pattern resulting in eighteen items for this CDA. The test was administered to students in 17 Grade 3 classrooms. A list of the attributes and the Q-matrix for the 18-item CDA are shown in Table A3 and Table A4 of the Appendix, respectively.

Three hundred and twenty four student responses were collected, which would yield approximately 45 students per skill pattern if the patterns were distributed equally across the skills. Participating teachers would first instruct on the topics relevant to subtraction within their classrooms, and then administer the CDA to students at a convenient time within two-week of instruction. The CDA was delivered using an online computer-based testing system. Students were presented with CDA items that contain both an item stem to prompt for a typed-response and an interactive multimedia component that provided additional information for students to understand the item. From this administration process, responses were collected, formatted and scored dichotomously. As the participation of this CDA was voluntary, students with greater than two missing responses were removed from the analysis to minimize unmotivated responses (as the completion of the CDA was not mandatory). For the purposes of demonstrating the ICI, only the scored student responses were used.

The results are summarized first at the test level and then at the item level. Overall, the results were ideal at the test level. The median HCI, which is used to quantify the fit of the responses to the expected model of response on a CDA, was 0.81. With a cut-off of 0.70 as the quality criterion for CDA designs (Gierl, Alves, & Taylor-Marjeau, 2010), this result suggests that the student responses fit with the expect model of response for subtraction. As the purpose of this CDA is to identify non-mastery students in order to refine and enhance instruction, the majority of students were expected to master the CDA.

At the item level, Table 5 provides a summary of the results from the subtraction CDA. The p-values of each item and the discrimination value (i.e., point-biserial correlation) are presented along with the ICI values. Three findings should be noted from these results. First, the ICI was not

correlated with either the difficulty or discrimination values. This result supports the idea that item-model fit is summarizing a different outcome from the classically defined notion of difficulty and discrimination. Second, with items created in a principled manner, with three items representing each skill pattern, the real data results support the results of the simulation study. Further, as p-values decrease, ICI values increase because the items change from measuring simple to more complex skills. Third, using the cut-score criterion of 0.50 from the simulation study, only three items were deemed to have poor item fit (Items 1, 2, 3). The poor ICI values for these items may suggest a problem at the attribute level (see Table A3 in the Appendix for the description of the skills assessed). It is important to note that without the ICI conventional scoring and psychometric approaches would not have identified issues of misfit at the attribute level, where items one through three are performing nominally at the item level. Although subject matter experts did not evaluate the cognitive model in the light of the student results, a follow-up study may find that a reorganization of the attributes may yield better fitting responses.

Table 5. Summary of the results from the subtraction CDA

Attribute Item Number P-Value Discrimination ICI 1 1 0.76 0.58 0.22 2 0.78 0.87 0.39 3 0.80 0.96 0.46 2 4 0.84 0.89 0.64 5 0.87 1.11 0.72 6 0.85 0.94 0.65 3 7 0.86 1.06 0.76 8 0.80 0.68 0.65 9 0.84 1.01 0.75 4 10 0.77 0.79 0.73 11 0.72 0.78 0.72 12 0.75 0.82 0.73 5 13 0.74 0.82 0.78 14 0.77 0.92 0.79 15 0.79 0.98 0.80 6 16 0.35 0.56 0.81 17 0.34 0.57 0.81 18 0.33 0.53 0.80

Discussion The purpose of this study is to introduce a statistic for determining item- model fit with CDA. The item consistency index (ICI), an extension of a person-fit index for CDA called the Hierarchy Consistency Index (HCI), is a standardized outcome that measures the ratio of misfitting responses relative to the total number of response across all examinees on a given item. Similar to the HCI, the requirements for evaluating item-model fit using the ICI is an item-by-attribute definition of skill mastery called the Q-matrix in addition to the student response vectors. The ICI has a maximum value of 1, which suggests all students responded identically to an expected skill pattern, and a minimum value of -1, which suggests item responses were the exact opposite to what the expected skill patterns suggest. We present two use cases to demonstrate the properties of the ICI under simulation. In addition, we demonstrate the applicability of the ICI through the use of real data to highlight how the ICI can be applied to identify poor-fitting items on a CDA. These two proof-of-concept applications demonstrate how the ICI can be applied in the real world and call for future studies to establish better evaluation criterion for the ICI.

Results from the simulation study provided some general insights on how the ICI performs as a method for detecting item misfit in CDA across a range of testing conditions. Using a cut-score classification method to determine poor-fitting items, the ICI was able to identify the majority of the poor fitting items across different simulated conditions. Although the item-model fit is described in a range by the ICI, the use of a cut-score to classify poor fitting items provided a simple outcome to interpret for evaluating how the ICI will perform in a given testing scenario. In addition, results from the simulation study demonstrated a few assumptions that must be met for the ICI to detect item misfit accurately. The number of items used for each skill pattern and the total number of poor fitting items were two features that affected ICI performance. The implication from these findings demonstrate that although CDA demands a different paradigm of scoring and statistical approaches, traditional issues such as consistency of the responses for a given set of skill can still be problematic in estimating item-model fit. From our simulation results, we suggest the use of three items per attribute or more per skill pattern to ensure adequate ICI detection. This finding is consistent with the research in establishing an adequate reliability in measuring attributes of skills (Gierl, Cui, & Zhou, 2009), where the authors stated that the idea of a short yet diagnostic test will not likely yield results with sufficient reliability.

Sinharay and Almond (2007) noted that tests with many poor-ﬁtting items indicate a problem with the overall model, whereas tests with few poor- fitting items indicate problems lie in the items themselves. In our simulation, we demonstrated that the ICI will produce similar results, where an increase of poor-fitting items in a CDA will lower the precision of the ICI. This finding may be linked to the fact that as more poor-fitting items are introduced, these items affect the fit of items requiring the same set of skills leading to an overall decrease in magnitude of ICIs. Table A5 in the Appendix illustrates this effect, where the mean ICI for well- and poor-fitting items under the 45-item simulation decreases as the proportion of ICI increases. In sum, a rigorous and principled test development process is needed for CDA to ensure all test items are created with minimal deviation from the expected set of skills they were designed to probe. Otherwise, poor model-fit results will lead to poor diagnostic outcomes.

The second study provided a snapshot on the utility of the ICI when applied to an operational CDA. Using a set of carefully designed CDA items, the ICI detected three consecutive poor-fitting items at the beginning of the assessment. This finding suggests that the ICI can not only be used for evaluating item-model fit, but can also be used for evaluating the consequences of test design at the item, attribute, or the cognitive model level. In our example, the three items flagged as poor fitting measure the same attribute revealing that the attribute may be mis- specified in the cognitive model. In addition, the independence of ICI from the difficulty and discrimination values suggest that item model-fit for CDA provides a unique measure of how an item is able to accurately predict performance. Hence, the definition of a good item for CDA may not only be how well an item is able to distinguish poor-performers from good-performers, but also how consistently an item can elicit responses that match the expected response patterns specified in the cognitive model (i.e., Q-matrix).

Item-model fit is challenging to measure, especially when cognitive inferences are involved in the test design. Items have to be aligned with the cognitive skills in the Q-matrix, skills have to be defined and organized in a systematic manner, and examinee responses have to match the expected skill patterns. The ICI can provide a source of evidence for identifying poor-fitting items or poor models for Q-matrix based CDA.

Implications for Future Research By introducing and demonstrating an item-model fit index for CDA, our study provides two practical implications for the development of diagnostic assessments in addition to a new measure of item-fit. The ICI

has the benefit of applicability, meaning that it can be used with a Q- matrix based CDA for determining the relationship between items and skills. Using the Q-matrix, item and examinee responses can be compared to provide a measure of item model-fit. While research on CDA has prompted a plethora of diagnostic scoring methods, one common starting point is the use of the Q-matrix in defining the skills and item requirements. Because item development, validation, and administration all depend on the veracity of the Q-matrix, evidence for validating the cognitive model is paramount. The ICI offers some initial evidence that can be used for validating the definition of skills through item response patterns to determine the relative fit between an item and its set of required skills defined in the Q-matrix.

While the ICI provides a new statistical method for scrutinizing CDA development, the second study highlighted the fact that the most crucial part of a well-designed CDA remains with item development. The importance of item development is, sometimes, neglected in CDA. Although CDA scoring methods can account for different levels of skill contributions, the link between how a skill is measured with how the skill is presented in the form of an item remains largely a subjective interpretation of the test developer and content specialist who create the CDA. To reliably measure a set of skills, multiple items are needed. Yet creating parallel items is often time consuming and expensive. Ensuring that each item is uniformly developed with the same set of skills is one critical activity in test development for CDA that ensures examinees receive useful diagnostic feedback.The ICI is co-dependent with all items requiring a related set of skills. Therefore, to ensure adequate item model- fit, every item in the CDA must adhere to a high level of quality and alignment relative to the expected skill the item is designed to measure.

Through introducing an item model-fit index for CDA, we have demonstrated how such measure can be applied to identify problematic items that are aberrant from the expected response model. This initial study provides directions of future research as further investigation is needed to apply and validate the use of this index. We also suggest three directions of future research. First, more research is needed to ensure different structures of knowledge represented by the Q-matrix can be evaluated with the ICI to identify misfitting items. The number of possible skill pattern representation increases exponentially as the number of evaluated skills increases, therefore more research is needed to ensure ICI provides an appropriate measure for different organization of skills. Second, guidelines to interpret ICIs are needed so we can accurately identify and distinguish adequate and problematic items. As the ICI provides a scaled measure of item model-fit, interpretations of the

index has not yet been established and is required to determine the adequacy threshold of item model-fit. Third, as the reliability of CDA measures is highly dependent on the defined skills, more research is needed to determine which model structure is ideal in the application of the ICI. Our analysis relies on non-compensatory attributes, meaning skills are independently defined, acquired and cannot be moderated by existence of other skills. This will likely limit the ICI in measuring item fit for testing complex skills but not for general skills such as elementary mathematics. More research is needed to evaluate appropriate use cases of the ICI.

References

Bock, R. (1972). Estimating item parameters and latent ability when responses are scored in two or more nominal categories. Psychometrika, 37, 29-51. Cui, Y., & Leighton, J. (2009). The hierarchy consistency index: Evaluating person fit for cognitive diagnostic assessment. Journal of Educational Measurement, 46(4), 429-449. Cui, Y, & Li, J. C.-H. (2014). Evaluating person fit for cognitive diagnostic assessment. Applied Psychological Measurement, 39, 223-238. Cui, Y, & Mousavi, A. (2015). Explore the usefulness of person-fit analysis on large scale assessment. International Journal of Testing, 15, 23-49. Gierl, M., Leighton, J., & Hunka, S. (2007). Using the attribute hierarchy method to make diagnostic inferences about examinees’ cognitive skills. In J. Leighton & M. Gierl (Eds.), Cognitive diagnostic assessment for education: Theory and applications (pp. 242-274). Cambridge, MA: Cambridge University Press. Gierl, M., Cui, Y., & Zhou, J. (2009). Reliability and attribute-based scoring in cognitive diagnostic assessment. Journal of Educational Measurement, 46(3), 293-313. Gierl, M., Alves, C., & Taylor-Majeau, R. (2010). Using the Attribute Hierarchy Method to Make Diagnostic Inferences about Examinees’ Knowledge and Skills in Mathematics: An Operational Implementation of Cognitive Diagnostic Assessment. International Journal of Testing, 10(4), 318-341. Jang, E. (2005). A validity narrative: Effects of reading skills diagnosis on teaching and learning in the context of NG TOEFL (Doctoral dissertation). University of Illinois at Urbana-Champaign, IL, USA. Orlando, M., & Thissen, D. (2003). Further investigation of the performance of S-X2: An item fit index for use with dichotomous item response theory models. Applied Psychological Measurement, 27(4), 289-298. R Development Core Team (2011). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. . Reise, S. (1990). A Comparison of item- and person-fit methods of assessing model-data fit in IRT. Applied Psychological Measurement, 14(2), 127-137. Rost, J., & von Davier, M. (1994). A conditional item-fit index for Rasch models. Applied Psychological Measurement, 18(2), 171-182.

Sinharay, S., Puhan, G., & Haberman, S. (2009, April). Reporting diagnostic scores: Temptations, pitfalls, and some solutions. Paper presented at the National Council on Measurement in Education, San Diego, CA, USA. Sinharay, S., & Almond, R. (2007). Assessing fit of cognitive diagnostic models a case study. Educational and Psychological Measurement. 67(2), 239-257. Wang, C., Shu, Z., Shagn, Z., & Xu, G. (2015). Assessing Item-Level Fit for the DINA Model. Applied Psychological Measurement, 1-14. Yen, W. (1981). Using simulation results to choose a latent trait model. Applied Psychological Measurement, 5, 245-262.

APPENDIX A

Table A1. The Q-matrix and skill patterns used for the simulation of CDA responses

Skill Pattern 1 2 3 4 5 6 7 1 1 0 0 0 0 0 0 2 1 1 0 0 0 0 0 3 1 1 1 0 0 0 0 4 1 1 0 1 0 0 0 5 1 1 1 1 0 0 0 6 1 1 0 1 1 0 0 7 1 1 1 1 1 0 0 8 1 1 0 1 0 1 0 9 1 1 1 1 0 1 0 10 1 1 0 1 1 1 0 11 1 1 1 1 1 1 0 12 1 1 0 1 0 1 1 13 1 1 1 1 0 1 1 14 1 1 0 1 1 1 1 15 1 1 1 1 1 1 1

Table A2. Variables manipulated in the simulation

Level Conditions 1 2 3 Test length 15 30 45 Sample size 800 1600 2400 Proportion of poor-fitting items 5% 10% 25%

Table A3. Description of the skills assessed in the CDA for subtraction in Grade 3

Cognitive Skill Descriptor: Apply a mental mathematics strategy to subtract Attribute # 6 Two 2 digit numbers using the digits 1 to 9 with regrouping 5 Two 2 digit doubles (e.g., 24, 36, 48, 12) 4 Two 2 digit numbers where only the subtrahend is a multiple of 10 3 Ten from a 2 digit number 2 Two 2 digit numbers where the minuend and subtrahend are multiples of 10 1 Two consecutive 2 digit numbers (e.g., 11, 22, 33)

Table A4. Q-matrix of the CDA for subtraction in Grade 3

Skill Pattern 1 2 3 4 5 6 1 1 0 0 0 0 0 2 1 0 0 0 0 0 3 1 0 0 0 0 0 4 1 1 0 0 0 0 5 1 1 0 0 0 0 6 1 1 0 0 0 0 7 1 1 1 0 0 0 8 1 1 1 0 0 0 9 1 1 1 0 0 0 10 1 1 1 1 0 0 11 1 1 1 1 0 0 12 1 1 1 1 0 0 13 1 1 1 1 1 0 14 1 1 1 1 1 0 15 1 1 1 1 1 0 16 1 1 1 1 1 1 17 1 1 1 1 1 1 18 1 1 1 1 1 1

Table A5. Summary of the mean ICI in extreme situations when n=2400

Proportion of Poor-Fitting Items Item Quality 0% 25% 50% 100% Well-Fitting Items 0.61 0.49 0.39 n/a Poor-Fitting Items n/a 0.33 0.28 0.15

International Journal of Learning, Teaching and Educational Research Vol. 16, No. 1, pp. 22-37, January 2017

Factors That Determine Accounting Anxiety Among Users of English as a Second Language Within an International MBA Program

Alexander Franco and Scott S. Roach Stamford International University, Graduate School of Business Bangkok, Thailand

Abstract. The primary goal of this study was to determine the factors related to accounting anxiety among MBA students who utilize English as a second language (ESL). The analysis included components within the learning environment and also differentiations as to demographic variables such as gender, age, ethnicity, and any prior undergraduate exposure to the study of accounting. A secondary goal of the study was to determine perception of anxiety among ESL students in an MBA program regarding quantitative courses as opposed to qualitative courses. Finally, the study examined different strategies used by ESL students to deal with accounting anxiety. The study found that there were significant differences in accounting anxiety based on gender, ethnicity, and exposure to undergraduate accounting. However, age was not a factor. In addition, the study supported the hypothesis that there is a negative relationship between levels of English proficiency and accounting anxiety. It also supported the hypothesis that there is a positive relationship between the levels of anxiety with classes involving quantitative subject matter. Finally, the study rejected significant differences in coping strategies by levels of accounting anxiety.

Keywords: accounting; accounting anxiety; English as second language (ESL); language anxiety, strategies regarding accounting anxiety

Introduction

Within the context of globalization, English has become the lingua franca of the business world, a transnational instrument vital in both a local and a global context (Buripakdi, 2014; Easthope, 1999). The study of language anxiety among students using English as foreign language has been steadily growing for the past three decades (Horwitz, 1991; Kao & Craigie, 2013; Kondo & Yang, 2004; Mahmoodzadeh 2012, Marwan, 2007; Ozturk & Gurbuz, 2014; Semmar, 2010; Wang, 2010). During this period, a body of work has also been developed that focused on anxiety suffered by students while studying accounting, although

none of the studies specifically examined a student body primarily consisting of ESL students (Ameen, Guffey, & Jackson, 2002; Borja, 2003; Buckhaults & Fisher, 2011; Chen, Hsu, & Chen, 2013; Clark & Schwartz, 1989; Dull, Schleifer, & McMillan, 2015; Duman, Apak, Yucenursen, & Peker, 2014; Ghaderi & Salehi, 2011; Malgwi, 2004; Uyar & Gungormus, 2011).

This study sought to investigate those factors that are related to varying anxiety levels among students of accounting who are challenged with learning this quantitative subject and its nomenclature while utilizing English as a second language. The first section of this paper presents a review of related material on accounting anxiety and proposes the hypotheses to be tested. The second part of this paper provides a discussion of the research methodology and analysis of the data collected. The final part presents utilitarian suggestions for minimizing anxiety by ESL students as they learn accounting, as well as recommendations for future research.

1. Literature Review Academic anxiety, within a pedagogical context, can best be seen as emotional state that is not inherent, but which is situational and can be “treated” by creating an effective association between teaching and receiving apprehension (Chu & Spires, 1991; Malgwi, 2004). Anxiety as to the learning of accounting at a level of higher education has been based on students’ perceptions that the nomenclature of the subject is akin to learning a new language (Borjas, 2003). Further, the knowledge base for this subject is perceived as being extensive and usually there is a corresponding apprehension that the period of time necessary to properly comprehend the principles and application of accounting is inadequate (Malgwi, 2004).

Previous studies suggest that differences in anxiety levels regarding the study of technical material may related to variables such as gender (Todman, 2000), age, background experience or exposure to the subject being studied (Chu & Spires, 1991; McIlroy, Bunting, Tierney, & Gordon, 2001; Towell & Lauer, 2001) or nationality/ethnicity (Burkett, Compton, & Burkett, 2001; Rosen & Weil, 1995). Based on this, the following hypotheses were examined:

H1: There will be differences in accounting anxiety levels of ESL students in an international MBA program across different demographic groups.

H1a: There will be differences in accounting anxiety levels of ESL students in an international MBA program across age groups. H1b: There will be differences in accounting anxiety levels of ESL students in an international MBA program across genders. H1c: There will be differences in accounting anxiety levels of ESL students in an international MBA program across different ethnic groups.

H2: There will be differences in accounting anxiety levels of ESL students in an international MBA program for those students who took an undergraduate accounting course as opposed to those who did not.

Among ESL students, the level of anxiety in learning technical subjects and in communication apprehension has been tied to the degree of their proficiency in the use of the English language (Casado & Dereshiswsky, 2004; Horwitz, Horwitz, & Cope, 1986; Marwan, 2007; Onwuegbuzie, Bailey, & Daley, 1999; Pappamihiel, 2002). Therefore, H3 was proposed:

H3: There will be a negative relationship between level of English proficiency and accounting anxiety for ESL students enrolled in an international MBA program.

The degree of quantification in a course of study impacts on the level of anxiety experienced by students (Kao & Craigie, 2013; Kondo & Yang, 2004; Rosen & Weil, 1995; Todman, 2000). Kondo & Yang (2004) devised a typology of strategies (5 strategy categories from 70 basic tactics) that ESL students use to cope with language anxiety. The strategies include peer seeking, positive thinking, preparation, and resignation. From this, the following hypotheses were proposed for testing:

H4: There will be a positive relationship between level of anxiety with classes involving quantitative subject matter and accounting anxiety for ESL students enrolled in an international MBA program.

H5: There will be differences in the accounting anxiety associated with the coping strategy selected by ESL students enrolled in an international MBA program.

2. Research Methodology and Findings 2.1 Sample The population studied was an international university in Thailand with an MBA student body consisting of 380 ESL students which were 57% female, 43% male; 64% were Thai and 36% were non-Thai. As per Krejcie and Morgan’s (1970) table of sample size determination, a sample population of 190 was calculated for this study. The sample consisted of 107 females (56% of the sample population), and 83 males (44%). Within the sample, 105 (55.3%) were Thais, 16 (8.4%) were Thai of Chinese lineage (1st and 2nd generations) and 69 (36.3%) were non-Thai.

2.2 Instrument A self-administered questionnaire was used with 15 accounting-focused, Likert scale questions, many which were modifications from the Horowitz et al. (1986) Foreign Language Classroom Anxiety Scale (FLCAS), a survey that has been used in several studies (Argaman & Abu-Rabia, 2002; Casado & Dershiwsky, 2004; Marwan, 2007; Matsuda & Gobel, 2004; Semmar, 2010; Yashima, 2002). All scales had a Cronbach alpha internal reliability score of over .80, indicating consistency (Hair, Black, Babin, & Anderson, 2010; Sekaran, 2000; Tavakol &

Dennick, 2011). The questionnaire also tested coping strategies by incorporating the Foreign Language Anxiety Coping Scale, which was designed by Kondo and Wang (2004). This scale was assessed to have an alpha coefficient of .91 (Marwan, 2007), demonstrating high internal reliability.

The questionnaire consisted of a forced, 4-point Likert scale from “strongly agree” to “strongly disagree.” A neutral option (e.g., “not sure”) was deliberately avoided because of cultural traits within Thai society that inhibit the motivation to express personal opinion: a strong hierarchical system with high power-distance and kreng jai –the culturally operationalized practice of avoiding the display of emotion or asserting one’s opinion (Holmes, Tangtongtavy, & Tomizawa, 2003; Johnson & Morgan, 2016; Suntaree, 1990). The questionnaire was translated into Thai for Thai students (and translated back into English to assure accuracy) in order to maximize effective feedback (Behling & Law, 2000; Harkness, Van de Vijer, & Mohler, 2002; Domyei & Taguchi, 2009). An English language version was distributed to non-Thai ESL students. The questionnaire was administered during a six-month period by the same lecturer who taught the only accounting course (a core course) required by the university’s MBA program. The actual day in which the questionnaire was administered was the first day of each starting class during that period.

2.3 Findings The first hypothesis proposed that there would be differences in accounting anxiety levels across groups defined by the demographic variables of age, gender and ethnicity. Descriptives for the first of these three demographic factors are presented below in Table 1. As shown in the table, the mean accounting anxiety rating declines consistently across the four age groups.

Table 1: Descriptive Analysis of Accounting Anxiety Ratings by Age Group*

Age Group N Min Max M SD

18-22 58 1 4 3.17 .920 23-25 48 1 4 2.94 .836 26-30 46 1 4 2.91 .784 30 + 38 1 4 2.74 .724

Total 190 2.96 .838

*Where 1 = Strongly Disagree and 4 = Strongly Agree with the statement: Taking an accounting class gives me high anxiety (i.e., feeling of stress, fear).

In order to test whether this decline was statistically significant, a one-way ANOVA was performed to analyze differences in accounting anxiety ratings across the age groups. The results are displayed in Table 2 below. Results indicate no significant difference across the four age groups for accounting anxiety, F (3, 186) = 2.242, p = .085. Therefore, Hypothesis 1a is rejected.

Table 2: One-Way Analysis of Variance of Accounting Anxiety Scores by Age Group

Source df SS MS F p

Between Groups 3 4.633 1.544 2.242 .085

Within Groups 186 128.109 .689

Total 189 132.742

The second part of this hypothesis proposed differences in accounting anxiety across gender groups. Descriptive statistics by gender are presented below in Table 3. As shown in the Table, the mean female accounting anxiety rating is slightly higher than the mean rating for males.

Table 3: Descriptive Analysis of Accounting Anxiety Ratings by Gender*

Gender N Min Max M SD

Male 83 1 4 2.77 .860

Female 107 1 4 3.11 .793

Total 190 2.96 .838

*Where 1 = Strongly Disagree and 4 = Strongly Agree with the statement: Taking an accounting class gives me high anxiety (feeling of stress, fear).

In order to test whether this difference was significant, a t-test was conducted. Results of that test are provided in Table 4, below. The results indicate a significant difference in scores with women reporting significantly higher levels of accounting anxiety (M=3.11, SD= .793) as compared to males (M=2.77, SD= .860), t (188) = -2.834, p = .005. Therefore, Hypothesis 1b is supported.

Table 4: Comparison of Anxiety Ratings by Gender*

Gender N Mean SD t df p 95% Confidence Interval

Male 83 2.77 .860 Female 107 3.11 .793

Total 190 2.96 .838 -2.834 188 .005 -.578 – -.101

*Where 1 = Strongly Disagree and 4 = Strongly Agree with the statement: Taking an accounting class gives me high anxiety (i.e., feeling of stress, fear).

The third part of Hypothesis 1 proposed that there would be differences in accounting anxiety ratings across different ethnic groups. Table 5 provides the descriptive statistics associated with the three ethnic groups that were analyzed.

Table 5: Descriptive Analysis of Accounting Anxiety Ratings by Ethnic Group*

Ethnic Group N Min Max M SD

Thai of 18 2 4 3.13 .619 Chinese Thai 106 1 4 3.09 .810 Not Thai 69 1 4 2.74 .885

Total 190 2.96 .838

*Where 1 = Strongly Disagree and 4 = Strongly Agree with the statement: Taking an accounting class gives me high anxiety (i.e., feeling of stress, fear).

Testing for significant differences in accounting anxiety ratings across the three ethnic groups was conducted with a one-way ANOVA. Findings of this analysis are presented in Table 6 below. As depicted in the table, there was a statistically significant difference between the ethnic groups as determined by the one-way ANOVA F (2, 187) = 4.010, p = .020. Therefore, Hypothesis 1c is supported. A Tukey post hoc test was then performed revealing that the Thai group had statistically significant higher ratings of accounting anxiety as compared with the Other Than Thai group (3.09 + .810, p = .020).

In sum, Hypothesis 1 proposed that there would be differences across the demographic groups of age, gender and ethnicity. Upon testing, the age portion of Hypothesis 1 was rejected, the gender differences hypothesis was supported and differences in accounting anxiety were found to exist between “Thai” and the “Other Than Thai” groups.

Table 6: One-Way Analysis of Variance of Accounting Anxiety Scores by Ethnic Group

Source df SS MS F p

Between Groups 2 5.459 2.730 4.010 .020 Within Groups 187 127.283 .681 Total 189 132.742

Hypothesis 2 proposed that there would be differences in accounting anxiety levels for those ESL students that had taken an undergraduate accounting course

and those who had not. Descriptive statistics for these two groups are presented in Table 7.

Table 7: Descriptive Analysis of Accounting Anxiety Ratings by Whether or Not Student Had an Undergraduate Accounting Class*

Undergrad N Min Max M SD Class

Yes 96 1 4 2.79 .882 No 94 1 4 3.14 .756

Total 190 2.96 .838

*Where 1 = Strongly Disagree and 4 = Strongly Agree with the statement: “Taking an accounting class gives me high anxiety” (i.e., feeling of stress, fear).

As shown in the table, those students who reported having had an undergraduate class in accounting had lower mean accounting anxiety ratings. To test to see if this difference was significant, a t-test was run on the accounting anxiety ratings between the two groups. The results of this test are below reported in Table 8. The results indicate a significant difference in scores with ESL students in the group that did have an undergraduate accounting course reporting significantly lower levels of accounting anxiety (M=2.79, SD= .882) as compared to those students who had not had an undergraduate accounting course (M=3.14, SD= .7.56), t (188) = -2.271, p = .004. Therefore, Hypothesis 2 is supported.

Table 8: Comparison of Anxiety Ratings by Whether or Not Student Had Taken an Undergraduate Accounting Class*

Undergrad N Mean SD t df p 95% Confidence Class Interval

Yes 96 2.79 .882 No 94 3.14 .756

Total 190 2.96 .838 -2.834 188 .004 -.582 – -.111

*Where 1 = Strongly Disagree and 4 = Strongly Agree with the statement: “Taking an accounting class gives me high anxiety” (i.e., feeling of stress, fear).

The third hypothesis proposed that there is a significant negative relationship between English proficiency and accounting anxiety for ESL students. Self- reported English proficiency levels ranged from 1, “Bad” to 5, “Excellent” (N =

190; M = 3.54; SD = .801). Ratings of accounting anxiety ranged from 1, “Strongly Disagree” to 4, “Strongly Agree” with the statement “Taking an accounting class gives me high anxiety” (i.e., feeling of stress, fear) (N = 190; M = 2.96; SD = .838). A simple regression analysis showed that the level of English proficiency significantly affected ratings of accounting anxiety. Results of the analysis are presented in Table 9, below. The higher the English proficiency ratings, the lower the accounting anxiety ratings (t = -2.899; p < .001). Therefore, Hypothesis 3 is supported. However, the R2 =.043, so the predictive power of the model is quite low.

Table 9: Summary of the Simple Regression Analysis for English Proficiency and Accounting Anxiety

Variable B SE(B) β t p

English -.216 .075 -.207 -2.899 .004 Proficiency

R2 =.043

Hypothesis 4 suggests a positive relationship between classes involving quantitative subject matter and accounting anxiety ratings. This was based on self-reported anxiety with classes that are quantitatively based and which was ranged from 1, “Strongly Disagree” to 4, “Strongly Agree” with the statement, “I get anxiety from an accounting class because of the numbers involved” (N = 190; M = 2.75; SD = .913). A simple regression analysis was used to test this relationship. The results of this analysis are presented below in Table 10. These results indicate that as a person’s anxiety with quantitatively based classes increases so does their ratings of accounting anxiety ratings (t = 10.386; p < .001). Therefore, Hypothesis 4 is supported. The R2 =.365 so the independent variable (anxiety with quantitative based classes) explains 36.5% of the variance in the dependent variable, accounting anxiety.

Table 10: Summary of the Simple Regression Analysis for Quantitative Class Anxiety and Accounting Anxiety

Variable B SE(B) β t p

English .555 .053 .604 10.386 < .001 Proficiency

R2 =.365

The final hypothesis suggests that differences in accounting anxiety will be associated with the coping strategy employed by ESL students. As displayed in Table 11, the means do differ across the various strategies employed by the students. This is particularly true for “Positive Thinking” and for “Peer Seeking” which fall at the lowest and highest levels of accounting anxiety, respectively. In order to determine whether these differences were significant, a one-way

ANOVA was performed to examine group differences in accounting anxiety scores. The results of this analysis are reported in Table 12.

Table 11: Descriptive Analysis of Accounting Anxiety Ratings by Coping Strategy*

Coping Strategy N Min Max M SD

Preparation 100 1 4 2.97 .758 Relaxation 22 2 4 2.91 .921 Positive 47 1 4 2.79 .977 Thinking Peer Seeking 21 2 4 3.38 .669

Total 190 2.96 .838

*Where 1 = Strongly Disagree and 4 = Strongly Agree with the statement: “Taking an accounting class gives me high anxiety” (i.e., feeling of stress, fear).

Table 12: One-Way Analysis of Variance of Accounting Anxiety Scores by Coping Strategy

Source df SS MS F p

Between Groups 3 5.189 1.730 2.522 .059 Within Groups 186 127.553 .686 Total 189 132.742

As shown in Table 12, the results indicate no significant difference across the four coping strategy groups for accounting anxiety, F (3, 186) = 2.522, p = .059. Therefore, Hypothesis 5 is rejected.

A summary of the findings of this study is provided below in Table 13. There was support for two of the demographic factors and varied levels of accounting anxiety (gender and ethnicity) but differences by age was rejected. Having taken an undergraduate course in accounting significantly reduced accounting anxiety. In addition, English proficiency was shown to be negatively related to higher levels of accounting anxiety. Anxiety toward courses with quantitative content was positively related to accounting anxiety. Coping strategies employed by students did not vary significantly by level of accounting anxiety.

Table 13: Summary of Study Findings

Hypothesis SS

H1a Differences in Accounting Anxiety by Age Rejected

H1b Differences in Accounting Anxiety by Gender Supported

H1c Differences in Accounting Anxiety by Ethnicity Supported

H2 Differences in Accounting Anxiety by Undergraduate Supported Accounting

H3 Negative Relationship between English Proficiency Supported and Accounting Anxiety

H4 Positive Relationship between Anxiety for Supported Quantitative Courses and Accounting Anxiety

H5 Differences in Coping Strategy by Level of Rejected Accounting Anxiety

As a part of this study, the ESL students were requested to rate the various core subjects and work on their thesis in terms of difficulty of learning the subject in English. Table 14 presents results of these questions. As shown in the table, those subjects that are based on a primarily quantitative content (accounting M = 2.09 SD = .733; and finance M = 2.22 SD = .751) were rated as more difficult than those subjects that are more theoretical in nature (marketing M = 2.97 SD = .6.29 and management M = 2.94 SD = .6.72). The two subject areas that employ both quantitative analysis and theory (research methods M = 2.55 SD = .780 and thesis M = 2.25 SD = .8.77) were rated in the middle in terms of difficulty with thesis being closer to the quantitative subjects.

Table 14: Difficulty of Studying Subjects in English Ratings by Percentage*

Subject Very Somewhat Somewhat Very Mean Standard Difficult Difficult Easy Easy Deviation

Accounting 16.8 63.2 14.2 5.8 2.09 .733 Finance 12.6 59.5 21.1 6.8 2.22 .751 Marketing 1.1 17.9 63.7 17.4 2.97 .629 Research 6.8 42.1 40.0 11.0 2.55 .780 Methods Management 1.6 21.1 59.5 17.9 2.94 .672 Thesis 19.5 45.8 25.3 9.5 2.25 .877

*Where 1 = Very Difficult and 4 = Very Easy

3. Conclusion and Recommendations Though the findings did not support a significant statistical difference in accounting anxiety by age, it did reveal significant differences for the factors of gender, ethnicity, and exposure to undergraduate accounting. The findings also

supported a negative relationship between levels of English proficiency and accounting anxiety as well as a positive relationship between the levels of accounting anxiety and the quantitative nature of business courses. Finally, the study did not find significance difference between levels of accounting anxiety and the selecting of coping strategies for such anxiety. These mixed results conform within the disparity within the studies discussed in the literature review. However, it is important to emphasize that this study differs from most of the literature review studies in that it examines anxiety within the context of learning the subject of accounting by using English as a second language. Within that context, Franco (2016) suggested the following eight tactical components for lowering anxiety in general, and accounting anxiety in particular, within an ESL environment:

1. Initial assessment of students. This can be done in two ways: On the first day of class, he student fills out a simple one-page form that requests information on the student’s knowledge of the subject matter but also asks the student to evaluate himself/herself as to English proficiency by way of a Likert scale. The form should also include questions like, “Who do you admire most?” Each student is then asked to introduce himself/herself to the class and verbally answer some of the questions on the form. This allows the teacher make initial assessments of each student (written and oral presentations) as well as obtain a general assessment of the level of English proficiency of the group in order to adapt the course accordingly. Secondarily, the assessment form would allow the instructor to determine any previous knowledge of accounting by the students as a result of undergraduate courses and or work-related experience. This allows making a better initial determination as to the speed the accounting course should take.

2. Vocabulary Buildup and Word “Dissection.” Absorption of the nomenclature of accounting is difficult enough for those tackling the subject in their native language. In an ESL environment, it is vital that students be introduced to key words and phrases even if this requires a discussion of such vocabulary before beginning the lecture. The lecturer should reinforce the meaning of key terms/phrases and provide a context within which they have meaning. Without a focus on building up the vocabulary for a particular lecture, there is a stronger likelihood that some students will not be able to follow the narrative. Frustration will set in as key terms, not properly absorbed by the student, will become obstacles in comprehending the narrative and context of the discussion.

The lecturer should write key words and phrases on the board, along with their definitions, and require the students to them write down. This creates a mental imprimatur since students are more likely to remember a word if they physically see it and work with it. Grammatical analysis of a word can be performed by “dissecting” it and presenting its grammatical variations. For example, a word like “accountability” – defined as being held responsible for something – can be broken up from

its noun form to its adjective – “accountable” – and the verb phrase “to account” for. This dissection, along with the lecturer’s use of the word within a context and the solicited use of the word from students in a sentence or two, allows the students to “chew” on the word or phrase and obtain an adequate comfort level of understanding.

3. Concept Checking. Concept checking involves asking questions to students to test the depth of their knowledge of newly accumulated information. These questions are sometime difficult to construct and some see their creation as more of an art form than a skill. The checking of concepts is developed in part, by anticipating, beforehand, concept checking questions you might use. However, it is primarily developed through practice and experience – “thinking on your feet.” Concept checking should be used throughout the lecture. In some situations, you can repeat a concept checking question that was successfully used in the same lecture in the past. However, the teacher will have to be conscious of coming up with new and pertinent concept checking questions within the serendipitous dynamics of the classroom discussion. This is art form more than anything else and the interaction of concept checking allows for a good balance between teacher talking time and student talking time.

Concept checking is not open questioning. Avoid questions such as, “Do you understand? that can merely be answered with “yes” or “no.” If your narrative flow causes you to create a question that can be answered in that way, follow up with “why?” “Marry” students in the class to come up with financial solutions to a marriage or business partnership problem. This personalizes the class analysis and gets students to interact with each other. The teacher should avoid adding unfamiliar vocabulary when working through concept checking. This is part of a self-imposed discipline that is always conscious of the ESL experience and the appropriate implementation of knowledge within that setting.

4. Eliciting. Eliciting can be simply defined as asking for answers (information) instead of just giving out the information. In a learner- centered classroom this provides for constant interaction. Eliciting should be performed by choosing students – not by depending on volunteers (i.e., the “alpha” few that will dominate classroom discussions if the teachers allows for this). Choosing students also keeps all students alert (“on their toes”) and avoids the awkward situation where a question asked to the entire class is met with silence. Even if the student chosen by the lecturer does not have an answer, he/she will usually provide some response that the teacher can build on. Letting everyone know that they can and will be called on helps to identify students who are falling behind (“stragglers”).

Pace yourself in your elicitations. Avoid repetition, condescension, and the need to turn everything into a question. Avoid asking questions

about material that has already been covered unless you are conducting a review for an examination.

5. Pacing. Even while abiding by the institution’s guidelines, rules, and expectations, the lecturer remains the “master of his domain” within his/her classroom. Lectures, homework, assignments, projects, and examinations are all the creations of the teacher. Especially in the ESL environment, the teacher must recognize the need to alter the pace of a lecture and even the pace of the entire course. Slow down when red flags and bells are going off. This is particularly true regarding subject matter that is built in layers (like accounting) where the next layer requires that you fundamentally understand the prior layer(s) of knowledge. If the lecturer keeps moving just to follow a schedule of his own design (e.g., a stated calendar on the syllabus), the results will be poor performances on the midterm exam. At that point the lecturer will have to go “back to basics” or risk moving forward and witnessing poor performances again, this time on the final exam. Almost nothing is more nonsensical for a lecturer than shackling himself/herself to rigid or impractical time restraints that were self-created and self-imposed.

6. Monitoring. In an ESL, learner-center environment the interaction should not only be verbal but also physical. The lecturer should not hide behind a podium or desk. Instead, the lecturer should be moving around to keep the students alert, away from their phones, or Facebook on their laptops. Moving amongst the students also allows for better eliciting, “marrying students,” and concept checking. When students are performing an in-class exercise (e.g., accounting), the teacher should move from one student to the next to see if the student is stuck on a word or a concept. Sometimes they are stuck on a verb or some other word within an explanatory or instructional text. An explanation or clarification at that moment is crucial. Otherwise, the student gets stuck and needlessly frustrated at the very start and gives up on solving the problem or resorts to looking to the student next to him/her for the answer. Sometimes a student who is stuck asks another student for an explanation. When a teacher sees this, he/she should step in, do the explanation, and provide further guidance.

7. Use of Paper. ESL students need to see physical words, not just hear them. They need a physical imprimatur. Power points have limited impact, unless the students have the physical text of the power point slides in front of them. If the lecturer gives handouts of core material (material that will be tested) the student has the pertinent text and can make notes including the meaning of the word in their native language. For test preparation, ESL students tend to rely on paper since they are not only looking at concepts but also the specific words that constitute the definition or explanation of that concept.

8. Feedback. It is nonsensical to wait until the student evaluations to obtain feedback on how well ESL students were coping with their English comprehension in a business course. Feedback is best solicited from the first day of the course, on an individual basis when the student feels he/she can be more candid or less embarrassed (i.e., no disclosure in public). Feedback can be attained before and after class, during breaks, by email, and at office hours. The teacher can also specifically approach students that he/she feels are having trouble. Individual feedback, in the aggregate, can help the teacher determine the overall situation in the class and who the “stragglers” are.

The continuation of globalization guarantees the internalization of higher education business studies using English as the commercial lingua franca. This study focused specifically on accounting anxiety experienced by ESL students. A body of literature needs to be created to specifically address accounting anxiety within the context of ESL education.

References Ameen, E.C., Guffey, D. M., & Jackson, C. (2002). Evidence of teaching anxiety among educators. Journal of Education for Business, September/October, 16-22. Argaman, O., & Abu-Rabia, S. (2002). The influence of language anxiety on English teading and writing tasks among Hebrew speakers. Language, Culture, and Curriculum, 15(2), 143-160. Behling, O., & Law. K. S. (2000). Translating questionnaires and other research instruments: Problems and solutions. Thousand Oaks, CA: SAGE Publications, Inc. Borja, P. M. (2003). So you’ve been asked to teach principles of accounting. Business Education Forum, 58(2), 30-32. Buckhaults, J., & Fisher, D. (2011). Trends in accounting education: Decreasing accounting anxiety and promoting new methods. Journal of Education for Business, 86, 31-35. Buripakdi, A. (2014). Hegemonic English, standard Thai, and narratives of the subaltern in Thailand. In P. Liamputtong (Ed), Contemporary Socio-cultural and Political Perspectives In Thailand (pp. 95-109). Dordrecht, Netherlands: Springer. Burkett, W. H., Compton, D.M., & Burkett, G.G. (2001). An examination of computer attitudes, anxieties, and aversions among diverse college populations: Issues central to understanding information sciences in the new millennium. Informing Science 4(3), 77- 85. Casado, M. A., & Dereshiswsky, M. I. (2004). Effect of educational strategies on anxiety in the second language. College Student Journal, 38(1), 23-35. Chen, B. H., Hsu, M., & Chen, M. (2013). The relationship between learning attitude and anxiety in accounting classes: The case of hospitality management university students in Taiwan. Qual Quant 47, 2815-2827. Chu, P. C., & Spires, E. E. (1991). Validating the computer anxiety rating scale: Effects of cognitive style and computer courses on computer anxiety. Computers in Human Behavior 7(1/2), 7-21. Clark, C. E., & Schwartz, B. N. (1989). Accounting anxiety: An experiment to determine the effects of an intervention on anxiety levels and achievement of introductory accounting students. Journal of Accounting Education 7, 149-169. Domyei, Z., & Taguchi, T. (2009). Questionnaires in second language research: Construction, administration, and processing (2nd ed.). London: Routledge.

Dull, R. B., Schleifer, L. F., & McMillan, J. J. (2015). Achievement goal theory: The relationship of accounting students’ goal orientations with self-efficacy, anxiety, and achievement. Accounting Education: An International Journal 24(2), 152-174. Duman, H., Apak, I., Yucenursen, M, & Peker, A. A. (2015). Determining the anxieties of accounting education students: A sample of Aksaray University. Procedia – Social and Behavioral Sciences 174, 1834-1840. Eastrope, A. (1999). Englishness and national culture. London: Routledge. Franco, A. (2016). MBA instructor’s guide for teaching business to ESL students. Unpublished manuscript, Bangkok, Thailand. Ghaderi, A. R., & Salehi, M. (2011). A study of the level of self-efficacy, depression and anxiety between accounting and management students: Iranian evidence. World Applied Sciences Journal 12(8), 1299-1306. Hair, J. F. Jr., Black, W. C., Babin, B.J., & Anderson, R. E. (2010). Multivariate data analysis: a global perspective (7th ed.). Saddle River, NJ: Prentice-Hall International. Harkness, J. A., van de Vijver, F. J. R., & Mohler, P. P. (2002). Cross-cultural survey methods. Hoboken, NJ: Wiley-Interscience. Holmes, H., Tangtongtawy, S., & Tomizawa, R. (2003). Working with the Thais: A guide to managing in Thailand (2nd ed.). Bangkok: White Lotus Press. Horwitz, E. (1991). Preliminary evidence for the reliability and validity of a foreign language anxiety scale. In E. K. Horwitz & D. J. Young (Eds.) Language anxiety: From theory and research to classroom implications. Englewood Cliffs, NJ: Prentice Hall. Horwitz, M. B., Horwitz, E. K., & Cope, J. A. (1986). Foreign language classroom anxiety. The Modern Language Journal, 70(2), 125-132. Johnson, R. L., & Morgan, G. B. (2016). Survey scales: A guide to development, analysis, and reporting. New York: The Guilford Press. Kao, P., & Craigie, P. (2013). Coping strategies of Taiwanese university students as predictors of English language learning anxiety. Social Behavior and Personality 41(3), 411-420. Kondo, D. S., & Yang, Y-L. (2004). Strategies for coping with language anxiety: The case of students of English in Japan. ELT Journal 58(3), 258-265. Krejcie, R. V., & Morgan, D. (1970). Determination of sample size for research activities. Educational and Psychological Measurement 30, 607-610. Mahmoodzadeh, M. (2012). Investigating foreign language speaking anxiety within the EFL Learner’s inter-language system: The case of Iranian learners. Journal of Language Teaching and Research 3(3), 466-476. Malgwi, C. A. (2004). Determinants of accounting anxiety in business students. Journal of College Teaching and Learning 1(2), 81-94. Marwan, A. (2007). Investigating students’ foreign language anxiety. Malaysian Journal of ELT Research, 3, 37-55. Matsuda, S., & Gobel, P. (2004). Anxiety and predictors of performance in the foreign language classroom. System 32, 21-36. McIlroy, D., Bunting, B., Tierney, K., & Gordon, M. (2001). The relation of gender and background experience to self-reported computing anxieties and cognitions. Computers in Human Behavior 17, 21-33. Onwuegbuzie, A., Bailey, P., & Daley, C. E. (1999). Factors associated with foreign language anxiety. Applied Socio Linguistics 20(2), 218-239. Ozturk, G. & Gurbuz, N. (2014). Speaking anxiety among Turkish EFL learners: The case at a state university. Journal of Language and Linguistic Studies 10(1), 1-17. Pappamihiel, N. E. (2002). English as a second language student and English language anxiety issues in the mainstream classroom. Proquest Education Journal 36(3), 327- 355.

Rosen, L. D., & Weil, M. M. (1995). Computer anxiety: A cross-cultural comparison of university students in ten countries. Computers in Human Behavior 11(1), 45-64. Sekaran, U. (2000). Research methods for business: A skill building approach (4th ed.). NY: John Wiley & Sons, Inc. Semmar, Y. (2010). First year university students and language anxiety: Insights into the English version of the foreign language classroom anxiety scale. The International Journal of Learning, 17(1), 81-93. Suntaree, K. (1990). Psychology of the Thai people: Values and behavioral patterns. Bangkok: Research institute of Development Administration. Tavakol, M., & Dennick, R. (2011). Making sense of Cronbach’s alpha. International Journal of Medical Education 2, 53-55. Todman, J. (2000). Gender differences in computer anxiety among university entrants since 1992. Computers & Education 34, 27-35. Towell, E. R., & Lauer, J. (2001). Personality differences and computer related stress in business students. Mid-American Journal of Business 16(1), 69-75. Uyar, A., & Gungormus, A. H. (2011). Factors associated with student performance in financial accounting course. European Journal of Economic and Political Studies 2, 139-154. Wang, S. (2010). An experimental study of Chinese English major students’ listening anxiety of classroom learning activity at the university level. Journal of Language Teaching and Research, 1(5), 562-568. Yashima, T. (2002). Willingness to communicate in a second language. The Japanese EFL context. Modern Language Journal 86(1), 54-66.

International Journal of Learning, Teaching and Educational Research Vol. 16, No. 1, pp. 38-56, January 2017

(Mis)Reading the Classroom: A Two-Act ―Play‖ on the Conflicting Roles in Student Teaching

Christi Edge, Ph.D. Northern Michigan University Marquette, Michigan, United States of America

Abstract. This case study examined concentric and reciprocal notions of reading—that of high school students, a pre-service teacher, and a teacher educator. An intern charged with teaching students to read, interact with, and compose texts in an English/language arts classroom constructed her role in the classroom based on her reading the ―text‖ of her internship experiences, relationships, and responsibilities. Using interviews and observations, a teacher educator read and interpreted the classroom ―text‖ the pre-service teacher ―composed‖ during her internship and then constructed a two-act ―play‖ which details the conflict in the intern‘s enacting the dual role of student-teacher and her subsequent reading of the classroom ―text‖ from her stance as student- teacher. Concepts of classroom literacy for teachers and teacher educators are considered.

Keywords: teacher education; reading classroom text; classroom literacy; student teaching internship; stance

Introduction In light of growing pedagogical, professional, and public awareness that twenty- first century literacy involves more than just printed words on a page and that specific literacies are acquired throughout the duration of an individual‘s education (Barton, 2000; Biancarosa & Snow, 2006; Buehl, 2014; Clark & Flores, 2007; Draper, 2011; Gee, 2012; International Reading Association, 2012; Langer, 1987; Lankshear & Knobel, 2007; Maclellan, 2008; National Council Teachers of English, 2007, 2008; National Center for Education Statistics [NCES], 2006, 2007; Rogers, 2000), it is time to consider the professional literacy needs of the very individuals to whom we look to educate our children and our adolescents (International Literacy Association, 2015).

Review of the Literature Lad Tobin (2004) implies a connection between the disciplinary focus of studying texts and the pedagogical importance of studying classrooms as text by asserting that ―teaching is a way of reading and writing. Students learn to teach through, first, learning to read the classroom and, second, learning to write themselves within that classroom‖ (p. 129). A teacher is simultaneously a reader and a writer of her classroom. Like readers whose meaning making is framed by

the stances they take along an aesthetic-efferent continuum (Rosenblatt, 1978, 2005), the stance or ―horizon of expectations‖ (Popper, 1962, p. 47) with which a teacher approaches a classroom conceivably guides his or her meaning-making process of that classroom text (Edge, 2008; Edge, 2011). How a teacher constructs and enacts her perception of her role as ―teacher‖ is an important focus for investigation; this perception guides the teacher‘s stance toward the classroom and potentially shapes how she makes sense of what happens in the classroom. A teacher‘s definition of her role in a classroom shapes her stance as a reader and a writer of the classroom text; it influences how she plans her instruction, creates learning opportunities, gauges her students‘ learning, and reflects on or assess her own teaching. While some scholars have alluded to the similarities between reading and writing pieces of text to the reading and composing of a classroom text (e.g. Burke, 2008; Edge, 2008; Tobin, 1991; Tobin, 2004; Perl, 1994), none has yet to explore how stance shapes that reading, what kinds of cues teachers use to read the classroom text, or what implications the reading of the classroom text might hold for prospective teachers or for teacher educators. Given the relative importance of literacy instruction, the gap in our knowledge base between the development of teachers‘ professional classroom literacy and the literacy instruction they provide for their students, one must wonder, as Tobin (2004) does, what we are to make of colleagues ―who read everything as a text—except their own classrooms and pedagogical methods? It seems a comically studding act of denial, like Freud asserting that everything is a phallic symbol—except his own cigar‖ (p. 129, original emphasis). In teacher education curriculum and reform, clinical field experiences and the practice of teaching are topics of perennial interest in the United States (e.g., Edge, 2015; Zeichner, 2012; Zeichner & Beir, 2012). For students in a College of Education, completing their full-time student teaching internship is a culmination of their undergraduate education (Cherian, 2003; Cruickshank & Armaline, 1986; McIntyre, Byrd, & Foxx, 1996) and an influential time (Veal & Rikard, 1998; Wilson, 2006) of transition between being a student and becoming a teacher (Burke, 2008; Graham, 2006). It was the goal of this study to investigate how stance, as revealed in three interns‘ expressed perception of their roles in the classroom, guided their teaching experiences. Understandings gained from studying the relationship between the role a teacher enacts and her stance toward reading and interpreting the classroom, could potentially contribute insight into prospective teachers‘ classroom literacy (Edge, 2011)development as these teachers move along the teaching-learning continuum, or what Jim Burke (2008) refers to as the ―continuum of performance‖ (p. 22), in transition from being expert students to becoming novice teachers. Specifically, understanding how interns construct their roles during internship offers insight into: a) how interns construct their interpretation of their role in the classroom ; b) how their perception of their role is shaped by their internship experiences and relationships ; c) how the perceived roles of the other members of the internship triad—the university supervisor and the cooperating teacher—influence the stance an intern takes toward her internship; and d) how a teacher educator might learn from reading the text of her university students‘ classrooms during their internship.

Theoretical Framework Understanding how pre-service teachers read and interpret the classroom text is a complex issue. In a transactional paradigm (Dewey & Bentley 1949; Rosenblatt 1978, 2005), reading and teaching share a common theory of meaning making. Like Louise Rosenblatt‘s transactional theory of reading, teaching too is an experience, an event in the moment of a classroom —a dynamic interaction between a particular person‘s repertoire of experience and the ―text‖ of the classroom itself. To be classroom literate, a teacher must read and make meaning of specific classroom situations, including reading classroom discourse, making meaning of teachable moments, connecting theory and practice, constructing scaffolds to aid students‘ understandings, questioning and evaluating student progress, ―reading between the lines‖ of students‘ verbal and nonverbal language, and thinking critically and metacognitively about the process of teaching (Edge, 2011). Couched in the concept of the classroom as a type of living, dynamic ―text‖ (Edge, 2008; Edge, 2011; Tobin, 2004; Witte, 1992) with which the teacher must transact in order to effectively teach, this study is guided by a framework which acknowledges the transactional and communicative nature of teaching and learning (Allen, 1995; Cooper & Simonds, 2007; Dewey & Bently, 1949; Greene, 1983; Hurt, Scott, & McCroskey, 1987; Langer, 1987; Rosenblatt, 1978, 2005; Smagorinsky & Whiting, 1995) and relies on interpretivisim to focus on meanings and the interactive communication of meanings through story (Bochner, 2005).

Method Interpretation and the gathering of interpretations are central to case study research (Erickson, 1986; Stake, 1995). Since this study was guided by a framework which acknowledges the transactional and communicative nature of teaching and learning (Allen, 1995; Cooper & Simonds, 2007; Dewey & Bently, 1949; Greene, 1983; Hurt, Scott, & McCroskey, 1987; Langer, 1987; Rosenblatt, 1978, 2005; Smagorinsky & Whiting, 1995), case study methodology is appropriate for interpreting how a teacher‘s stance shapes meaning-making within the context of classroom teaching and learning.

Sample. This qualitative case study was originally part of a multiple-case study of three pre-service teachers who served as interns to high schools in a large metropolitan school district in the southeast United States. Each participant was a female, senior, English education major in a College of Education housed in a large research university in the same district that the internships took place.

Participants. The participants were selected using theoretical sampling according to participants‘ potential ability to contribute to a developing theory (Creswell, 2018; Onwuegbuzie & Collins, 2007; Onwuegbuzie & Leech, 2007) for purposes of refining ideas and determining relevant contexts and conditions of a developing theory (Charmaz, 2000) of classroom literacy (Edge, 2011). Prior to this study, each pre-service teacher demonstrated diverse abilities, personality, and internship placement that marked her as a potential participant. As former students in two university courses I taught, Methods of Teaching English and

Methods of Teaching Reading in the English Classroom, I had a context for knowing if these individuals could contribute to an emerging understanding of stance in relation to reading the ―classroom as a text,‖ in general, and stance as a lens through which to understand pre-service teachers‘ roles during internship, specifically. Therefore, I invited these three former students to participate, and each agreed. This manuscript focuses on one of the three participants in order to offer ―thick description‖ (Stake, 1995, p. 43) of a central theme that emerged from all three case studies.

Data Collection. Stake (1995) writes that the interview is the path toward discovering multiple realities. To investigate the intern‘s reading of the classroom and to identify the stance(s) that she use to read it, interviews were the primary source of data collection. Data also included classroom observations and an interview with the intern‘s cooperating teacher to triangulate data obtained through interviews. The intern participated in an interview at the beginning of her internship, approximately three weeks into her semester-long assignment and in a second interview during the final two weeks of her internship. The second interview took into consideration the themes emerging from data collected during the first round and then formulated questions to further explore these themes and to gather additional details in light of extended experiences in the field. The first interview took place on the internship site and was accompanied by observations of each preparation the intern taught: freshman English and junior honors English. To better accommodate the intern‘s schedule, the second-round interview was conducted off campus after school hours and included an email follow-up component. Interviews and observations were digitally recorded. Observations were also videotaped. Copious field notes were taken during the observations and interpretive anecdotal notes were written immediately following observations and interviews.

Data Analysis. The primary data analysis used to investigate expected and unexpected patterns was interpretation of observation and interpretation of interviews (Stake, 1995). Interviews, field notes, and anecdotal notes and interpretations were read and analyzed within the context of each case. Secondary data analysis included coding data into emerging themes and then rereading and analyzing data to refine themes and chart examples of themes (see tables 1 and 2 in Appendix A for examples). Analysis through a search for patterns, called ―correspondence‖ by Stake (1995, p. 78) allowed for construction of meaningful understanding. Although finding similarities across cases was not the goal of this study, a cross-case chart of themes was made in order to aggregate themes across cases and to consider contexts and conditions under which a developing theory (Charmaz, 2000) of stance in teaching and reading a classroom text might exist.

Credibility. For consensual validation (Eisner, 1991 as cited in Creswell, 2018), two other English education teacher educators, who had no prior knowledge of this study or participants, reviewed both ongoing interpretations of classroom observations and the findings. Coded data and interpretations from

interviews were also member checked with participants (Creswell, 2018). Findings from multiple data sources were composed with attention to structural corroboration, (Creswell, 2018; Eisner, 2005), thick description (Stake, 1995), and researcher reflexivity.

Methodological Concerns. A few limitations of this study need be acknowledged. First, as previously mentioned, the participant was my former student in two university methods courses. While it appears that familiarity allowed her freedom to speak without feeling as if I were assessing her, it is possible that traditional student/teacher power dynamics could still have limited both the scope of what she revealed as well as her ability to be candid about ways in which her university experience did or did not prepare them to be successful in her classroom. Second, although I did observe the intern teach in conjunction with her first interview, it is plausible that this limited exposure did not provide context enough to fully understand the subtleties of the feelings, ideas, and experiences she shared with me in interviews. Hillocks (1999) has suggested that classroom observations are necessary to understand the more subtle nuances of teachers‘ thinking. While I did get to know the intern over the span of an academic year prior to her internship, that knowledge was limited to her being a student and a preservice teacher in simulated teaching experiences to their peers. Prolonged observations might offer additional insight and serve to further crystallize (Richardson, 2000) my understanding of interview data. Finally, interpretation of data is very much shaped by my own conceptual framework. Although findings were analyzed in light of literature on field experiences and triad relationships, I openly acknowledge that in any given reading, there exists a text, a reader, a context, and an intertext that greatly influence the meaning made. To account for this, I have tried to make my own reading transparent when possible.

Findings To offer ―thick description‖ and ―experiential understanding‖ (Stake, 1995, p. 43), the findings of this study are organized around a central theme that emerged in the case of Teri Thompson (the participant‘s name and school site have been changed to protect and respect her anonymity) who approached her internship from her stance or mindset as a student. I later refer to this as a student-teacher stance to underscore the divided nature of reading the classroom from the perspective of a student who is attempting to be the teacher. The blurred roles of student and teacher appeared to make the intern feel like she was doing neither as well as she wished. Furthermore, this student-teacher stance shaped the manner in which she evaluated her success in the internship placement, her ability to take on the role of ―teacher,‖ and her thoughts about her future as a high school English teacher. Although case studies are not ―stories‖ in the traditional sense (Stake, 1995), detailed portions of this case are heuristically presented as a two-act play to allow others ―to develop vicarious experiences‖ (Stake, 1995, p. 63) through the detailed ―snapshots‖ into the text of this case. The telling of Teri‘s story is organized around classroom moments when her student-teacher stance revealed

tensions in her role as teacher, and it is set within Teri‘s physically, emotionally, and educationally bifurcated classroom, where I observed her teaching. Findings are reported and framed by my own reading of the participant‘s classroom. This reading took place during my observations of her teaching and interviews, and is read in light of both the specific context I observed and, at times, in light of my long-term reading of the participant as a university student.

Teri: A Two-Act Play on the Tensions of Teaching

Act One: The Conflict of Playing Two Roles As I slip into a student desk in Teri‘s classroom and into a lesson that is now well underway, I quickly gather that students are reading a play. The majority of the students sit in desks facing forward, listening and watching or following along in their books, but a few are seated at the apex of the room as if on stage, reading parts from their textbooks. Teri stands front and center, reading the narrator‘s part with enthusiasm and inflection, pausing from time to time to comment or to ask a question from the students; she seems quite pleased and comfortable to be the focal point on this classroom ―stage.‖ Over the last year, I have come to know Teri to be confident, passionate, and outspoken, a noticeable presence in any room, so what I see doesn‘t surprise me. What did emerge as a bit of a surprise is that this one lesson captured the crux of the actual ―plot‖ and ―conflicts‖ in the story of Teri‘s internship. The text of Teri‘s classroom is written around a series of related tensions: the tension between simultaneously enacting student and teacherly roles in the context of her internship and the tension created in (mis)reading students from her own vantage point and history of experiences as a student. In trying to simultaneously enact a dual role—that of participant (student) and director (teacher) in the living ―dramatic‖ text of her internship— Teri blurred the bounds of classroom student and teacher in a way that created not only tension, frustration, and uncertainty in her perception of herself as teacher, but also left her feeling as if her role was written, and her part must be carried out until the end of the ―show.‖ As she later told me,

In my [honors] classes, and I know this isn‘t right, but unfortunately, this is the foot I started out on, and now it‘s going to be difficult to correct it, I am ―cool Mrs. Thompson‖ who tries to do fun activities. They kind of take me seriously sometimes, and they will work with me. My age, I think is a bit of an aid, and um, at the same time, it prohibits the full range of my teaching abilities. They look at me like I‘m their peer. ―Who is this girl telling me what to do?‖ Well, this girl has an almost bachelor‘s degree, a house, a marriage, another job—this girl has a lot more life experience than you do, so just shut up and listen to this girl!

Even though Teri laughs off the seriousness of her perceived inability to be seen as ―the teacher,‖ this tension is quite apparent in her relationship with her students. In one of her eleventh grade honors classes, a student asks,

―Where‘s Dr. B [the cooperating teacher]?‖

―I’m Dr. B. now,‖ replies Teri.

―No, you‘re Tommy,‖ concludes a second student.

The student‘s tone is playful but serious, as if to remind Mrs. Thompson of her place. Teri responds by playfully poking her tongue out a little. Her lighthearted attempt to save face and brush off the students‘ remarks only position her further into the role of a ―student-teacher‖ simultaneously both student and teacher and yet fully neither. In her ninth-grade class, similar patterns of interaction emerge. After a student answers a question correctly during the introduction to her lesson, Teri excitedly slaps the student a high-five.

―Ouch!‖ he yelps. ―Your ring hurt me!‖

―Well, don‘t give me a high-five on my ring hand!‖ she retorts through a grimaced face and then playfully sticks her tongue out at him in an exaggeratedly childish manner. As I watch, I surmise that she seems embarrassed and struggles to regain the professional role of the teacher and still be liked by her students. The students don‘t seem to know how to respond either. Later into her lesson—a lesson designed to help the ninth-grade students prepare for the standardized writing exam that they would be taking the following day—Teri writes the words ―And,‖ ―But,‖ and ―Because‖ in big letters on the board to emphasize the ways students should not begin their sentences when they write. One student frowns at his graded practice essay, glances back up at the words written on the white board, and then responds,

―I‘m retarded. Damn, I‘m so retarded!‖

His comments indicate that he hears her as an authority figure who evaluates and finds writing such as his to be poor. Nevertheless, his choice of language is a subtle statement, heard by the entire class, that her authority does not extend to what is acceptable language in the composition of the classroom. Another student openly challenges Teri in a way that implies she is not his authority figure,

―You began a sentence with ‗Because‘,‖ he snidely accuses.

―That wasn‘t a sentence, it was a fragment,‖ she quickly rationalizes in her rebuttal. Despite her quick wit, a brief but awkward silence hangs like a thick curtain of tension in the classroom. Teri appears unable to ―rewrite‖ or redirect the flow of the lesson. The act of classroom learning ends without dénouement. Nevertheless, these scenes illumine Teri‘s tension in not being fully ―teacher.‖ Although she is a full-time university student, she is not among peer

students here either. Teri seems to have no clearly defined space in which to write her role into the text of this classroom.

During a second interview conducted through email in the final week of her internship, Teri reveals with what she describes as ―brutal honesty‖ that this tension remains unresolved:

I have found establishing a relationship with my students to be challenging. Part of me enjoys the same music and movies that my kids do, and find common grounds in that aspect. Part of me looks down on my kids for having such superficial worldviews, and I have made that clear to them (oops) in the past. I‘m torn between two worlds myself – I‘m 23, and find some juvenile things interesting and humorous (for instance, I quote ―Family Guy‖ with my kids sometimes) but I‘m also married and a homeowner, and these experiences have broadened my perception and made me wonder why my kids, who aren‘t that much younger than I am, still only see value in superficial nonsense. I think I have a lot of self-discovery to do, and I don‘t know if I‘m going to be at a healthy mentality by fall. Regardless, I have to teach next year – I can‘t wait tables forever. But I‘m wondering if I need to consider middle school or (YELP) elementary school, where such a dramatic age difference exists between me and my kids that some of my neuroses aren‘t exposed. I‘m wondering if all new teachers feel that way, or if I have some baggage I need to unload before I can be a successful mentor.

The tension in trying to take on the role of a teacher while still trying to relate to students from her stance as a student leads her to question her emotional maturity and thus, her ability and future plans to teach high school students. In her interviews, Teri‘s perception of other‘s roles and her definition of a successful internship underscore how her liminal role as student-teacher in the high school classroom is related to her overall place in the internship triad. For instance, as much of Teri‘s language connotes, and nearly all of her comments combine to reveal, her primary stance toward her internship experience is that of an advanced student or a teaching apprentice—someone who is charged with mastering discrete components of teaching. The following excerpts from her interviews underscore this primary stance:

Researcher: How would you define a successful and a failed internship?

Teri: Successful? Fulfilling the requirements set by [the university], establishing a good rapport with students and colleagues, learning about the system from within… Failed? Not meeting any of the aforementioned benchmarks.

Researcher: How would you define your role within your internship?

Teri: I feel like a young Jedi knight…These other teachers [gesturing to the teachers she usually shares lunch with] took me under their wing…I‘m not just Dr. B‘s intern, I‘m some woman who‘s going to be teaching, maybe teaching their kids if they have little ones. They want to make sure I do it right and represent the field well… I‘m a little Anakin Skywalker…but I won‘t go to the dark side...‖

The first criterion Teri used to define success is meeting the university‘s objectives. To her, she is foremost a student, working to complete her coursework and earn a good grade. Her second criterion, establishing a good rapport, also resonates with the kind of student—well-liked by students and teachers—she was prior to her internship. Furthermore, her description of herself as a young Jedi knight is a clear statement of her position or stance as a student apprentice. In the movie Star Wars, a Jedi Knight is an apprentice who is taught the ways of ―the force‖ by masters. Jedi Knights are also socially well- received, even ―popular‖ in the Star Wars galaxy. Teri also described her specific relationships with her cooperating teacher (CT) and university supervisor (US) in light of her being a student whose goal it is to learn from masters and to demonstrate her learning to her supervisors. For instance, when I asked her to describe her relationship with her CT, Teri referred to her as ―Dr. Grandma.‖ Someone who ―has taken a very maternal role‖ with her, yet someone ―who expects certain things‖ and shows her what to do with ―a voice of experience.‖ Someone who ―sees where [she] is weak, and pulls [her] up a little.‖ Teri also said that Dr. B. was someone who might possibly evolve into the role of a mentor by the time internship concluded. However, when I asked if her US could be a mentor, she corrected, ―No, no, he‘s an evaluator‖ who grades her, and ―he‘s a supervisor and a source of information.‖ Teri‘s relationships with the other two members of her triad communicate that she sees herself as a student who is evaluated and an apprentice who is to be taught and nurtured by experienced teachers in the profession. Her professed focus and goals for the semester also reveal her student stance in that she focuses on what she has not yet mastered. Teri eschews her own assessment of herself as ―an excellent planner‖ and chooses to center her comments on talk of classroom management and the need to be able to ―recognize how kids learn.‖ Her focus is not necessarily on what students are or are not learning, although she does communicate that she is concerned with their learning; rather, her focus is on herself as someone who is not yet able to master a skill of teaching. This student-apprentice focus fuels a second, related conflict in her classroom: her struggle to comprehend students who are unlike her.

Act Two: Conflict Created from Misreading the “Text” of the Classroom ―Acts One and Two‖ in Teri‘s teaching schedule are separated by an intermission of lunch. When the bell rings, Teri takes me ―back stage‖ to the teacher‘s cafeteria where I conduct her first interview. Initially, Teri had only

invited me to observe her eleventh-grade honors English class, but before lunch ends, she extends her invitation to ask me to watch her ninth-grade ―regular‖ English class—the ―class from Hell,‖ as she put it, that Teri initially did not want me to observe. Unlike the eleventh-grade honors classes where she feels she must be ―Cool Mrs. Thompson,‖ in her ninth grade classes Teri describes her role as someone who has to be the locus of control and ―the gatekeeper of knowledge‖ for students.

…I‘m a drill sergeant, a jail warden, prison guard, wicked witch, and, uh, behavior corrector, manners enforcer, ―No, we don‘t pull on girls‘ hair,‖ ―Sit back down in your seat,‖ ―Don‘t use ‗aint‘,‖ ―Don‘t use the F word in my classroom,‖ and so on and so forth.

Within minutes, maybe even seconds of the beginning of the next class period, I see the shift in roles that Teri articulated during her interview. She seems to step into character according to how she perceives her role as the classroom teacher. The juxtaposition of the two types of classes she teaches further depicts tensions created by Teri‘s teaching from her stance as a student- teacher. Teri‘s fifth period freshman English class has 17 students present, 11 males, six females; five students are students with special needs. Just before class, Teri warned me that she would be ―laying down the law‖ with this class today, and true to her word, class begins with a speech that seems designed to remind student who is boss, who is not, and that referrals await any who don‘t want to play the ―right‖ role. As I watch and listen, I come to contextualize the interview responses she shared with me, and I begin to see her room as a metaphorical stage and a means of understanding her bifurcated teaching roles, as she explained them to me. Separated by a moveable wall best described as a heavy curtain, Teri‘s room is really two rooms, one a tiny classroom and the other a book closet. In the center of the room, the curtain is pulled back, but it doesn‘t recede all the way to the walls, causing it to jut out a little and create an odd configuration and a one-room-two-room sensation. Some students sit on the bookroom end of the partition, but the majority are in the classroom section where the whiteboard is, and where Teri writes as she teaches. The room itself seems to echo the tensions Teri communicates in her interview and in her stance as a student-teacher. On the one side of her internship experience, Teri teaches students who are like herself—bright, self- motivated, mostly middle-class, traditional honors students. On the other side, she teaches students who are very different from herself, and she has trouble ―reading‖ them from her stance as a traditional, successful, well-liked student. Their knowledge is sophisticated but worldly rather than academic, and they struggle to follow Teri‘s vision for the lesson as they fidget through a traditional listen-and-take-notes kind of class. It appears to me that because Teri is interpreting the happenings of the classroom from her own student stance, she does not know how to read her students and connect with them. In fact, she seems to repeatedly misread their comments as deliberate attempts to thwart her lesson and rebel against her as the teacher. For instance, Teri attempts to think-

aloud some ideas for answering an essay prompt, ―After 2:52 [the end of the school day], you‘re not learning—―

―Yes we are,‖ a student interrupts. ―I‘m learning football plays.‖

―I‘m learnin‘ tricks on my bike,‖ shouts out another student.

Suddenly the room is bustling with students talking over one another about everything they learn after school is over. Teri is frustrated at the direction the conversation is taking and appears to interpret their comments as attempts to annoy her. ―Hands, hands! Simmer down, guys, simmer down.‖ The majority of the students appear really willing to participate by offering their opinions and personal experiences at a point in the lesson that they can connect to. This is frustrating to Teri who doesn‘t quite know how to read their responses or channel their energy, and so she collectively chides them for being ―silly like middle schoolers.‖ She doesn‘t hear them making connections between their own personal, less academic, perceptions of learning; she heard talk that seemed irrelevant to her conception of learning and interpreted their actions as disruptive behavior that interrupted the lesson she had prepared. The lesson on this day is geared to prepare students for the standardized writing test that the students will take the following day. Teri‘s plan is to first use students‘ practice essays as a vehicle for talking about what to do and what not to do when they write, and then to think through an essay prompt with students. These students have a hard time following her oral construction of an essay intermitted with asides on the proper use of conventions. As a result, Teri ends up doing the majority of the mental work by telling students what to think and do from her perspective as a successful writer. ―When I got a [highest score possible] on my [state writing assessment] , I …‖

The learning objectives appear distant from students‘ background knowledge and interests, but several students try to please her by responding when they can. When they do respond, Teri tends to misinterpret what they say and then responds with the lion-tamer mentality that she feels defines her role in these classes. For instance, Teri begins the lesson by holding up a student‘s practice essay.

―This person is using vivid details from her life experiences. This is a very good example of what students should do.‖

And then a moment later,

―Do not use ―you‖ unless you are addressing a specific audience. I want to see everyone writing this down in their notes—―

―How do you spell audience?‖ calls out one student.

―What?‖ Teri seems shocked out of her train of thought.

―Repeat that, Miss,‖ requests a second student who has been fumbling with his paper.

―What‘s vivid?‖ a boy in the front of the room asks.

―Don‘t interrupt me and call yourself stupid,‖ Teri chides and then sighs, exasperated with what she interprets as misbehavior.

A few minutes later, a girl who has been silent the entire lesson offers an idea for the essay they are constructing out loud as a class, ―Teachers hate us. That‘s what we should write about.‖ When Teri responds with disbelief, the class quickly becomes loud as students shout out (vivid) examples of how teachers demonstrate their negative feelings toward them.

―Talk to them,‖ Teri ironically replies.

―They don‘t listen!‖

An exasperated Teri stops to hush the class and proceed with her lesson. She misses the fact that students are speaking from their personal experiences (the writing trait she praised in the beginning of the lesson), and that by listening to them she could use their experiences as a bridge to help them conceptualize ways to write a persuasive essay. Teri talks in terms of things to do and not to do from her personal knowledge of her own successful, academic writing experiences while the students respond from their knowledge base of experiences outside of school and to their negative experiences with teachers in school. As a result, students and teacher talk at and over one another, never really connecting. Much like the layout of the classroom itself, a wall seems to separate their ability to accurately read and understand one another.

Discussion: An Epilogue for Educators Positioning theory has been used to explain how triad members take on roles or parts to play within the unfolding classroom story. For instance, Bullough and Draper (2004) write, that ―[w]ithin storylines, actors take different parts, and the parts they play and how they play them reveal the meaning of events as well as present definitions of self and other‖ (p. 408). They also assert that ―context…personal characteristics and individual biographies…profoundly affect the kind and quality or relationships that form between beginning teachers and those who would assist them in learning to teach‖ (Bullough & Draper, 2004, p.409). In light of Teri Thompson‘s conflicts, I might add that such factors potentially profoundly influence the quality of teaching and the relationships that form between beginning teachers and the students they assist. Considering that student teachers come to the classroom after spending a lifetime in an apprenticeship of observation (Lortie, 1975) from the vantage point of the student desk, transitioning from playing the part of a student studying to teach, to actually being the classroom teacher to other students requires a challenging and complex shift in thinking. This transition is complicated by the

regular role shifting (Bullough and Draper 2004; Veal & Rikard, 1998; Smagorinsky, Cook, Moore, Jackson, & Fry, 2004) student teachers must do in order take on competing roles as student and as teacher during the semester of their internship. The relationships of the roles played by each member of internship triad is one which often results in power shifts, depending on who is present (Bullough & Draper, 2004; Veal & Rikard, 1998). When a student teacher is alone in the classroom, she is expected to be the teacher; when the university supervisor comes to observe and brings an assessor stance, she must shift to student; and when the cooperating teacher is present, the intern could be expected to assume a stance anywhere along the student-teacher continuum. When we consider what it is that we ask interns to do—walk out of the world they know and step into one they‘ve only studied and then straddle both worlds while the line distinguishing the two continues to move, it is a wonder pre- service teachers don‘t feel more academically and professionally schizophrenic during their internship. It‘s not a surprise then, that each participant studied during this investigation expressed degrees of failure in terms role ambiguity and developing ―a sense of self-as-teacher‖ (Sudzina & Knowles, 1993, p. 255) through questioning her emotional maturity and whether she would actually pursue a teaching position after internship. The fact of the matter is that prospective teachers are students, and stepping into a classroom does not a teacher make (as Yoda might phrase it). Simply trading the college campus for a secondary school campus does not instantly transform a student into a teacher any more than stepping into an operating room makes a medical student a surgeon or stepping into a courtroom makes a student of the law an attorney. Can we, as teacher educators and mentor teachers expect interns to bring to their internship a stance other than student or student-teacher? After all, aren‘t teachers like Teri supposed to be learning from their internship experiences? Well, yes. And yet, potential exists for the scenes from Teri‘s classroom to be re- envisioned, revised, and re-composed.

Re-Envisioning Alternate Tales from the Cut Scenes of Teri’s Two-Act Play. The Holmes Group (1986) reported in Tomorrow’s Teachers, ―[p]aradoxically, teachers are the butt of most criticism, yet singled out as the one best hope for reform.‖ Before I proceed further, I think it should be said that despite Teri‘s misreading her classroom text and her struggle to construct a teacherly role within the already composed lines of the parts she perceived she needed to play during internship, Teri was a pre-service teacher who communicated both a desire to be and the knowledge to do the kind of teaching which resonates with best-practice research and literature. That she repeatedly acknowledged the dissonance between what she knew she should do and what she realized, with dismay, she was actually doing demonstrates a level of awareness that holds hope for her ability to see and perhaps to re-compose her. This alternate version of Teri‘s tale can be pieced together from fragments of data that didn‘t make it into the ―two-act play‖ production of her classroom teaching. In Teri‘s first interview, she shared that her understanding of and reasons for teaching had changed significantly in the last year. When asked to explain, she suddenly spoke in a high pitched and excessively jovial voice that

sounded like a cartoon character‘s, ―You know, I, I—like everybody—I wanted to guide others to learn.‖ I interpreted this tin voice as her way of expressing the naiveté of her initial perception of teaching. Speaking once again in her adult voice, Teri listed two factors which helped her to gain a more three-dimensional conceptualization of teaching: a methods course in teaching reading and her present, ―real-life‖ experiences. What Teri‘s two turning points have in common is that each required her to work closely with students. In the methods of teaching reading course, Teri wrote a case study after working with a struggling reader. From this experience Teri realized, ―I never knew there were people who didn‘t have the capacity to do certain things and had to be shown and given skills in order to help them learn. Before, I just thought I would give them information.‖ Prior to her experiences in the methods course, Teri looked at readers through her own stance as a successful reader who never even gave much thought to the complex moves required to make sense of written texts. However, in working with a struggling reader, she took a stance that differed slightly from that of ―student stance‖ to what I‘d like to call a learnership stance. In both stances, Teri was learning; however, when she had guided opportunities to reflect on and make meaning of her working closely with a student who was very different from her, she was able to take on a transactional learnership stance that enabled her to read her interactions with this student in a way that taught her from his experiences. Thus, the experience was transactional—each perspective informing the other. Terri read and made meaning from her experiences working with students

Reading Classrooms as Texts. Students potentially teach the teacher when a learnership stance guides the teacher‘s reading of the classroom. In the same way that the transactional nature of a written text can lead a reader toward exploration (Rosenblatt, 1938), meaningful understanding, and new knowledge, so too can the transactional nature of communication (Cooper & Simonds, 2007) lead teachers to make meaning of and to generate new knowledge about their profession. Said another way, because of the transactionally communicative nature reading and writing share with teaching (Cooper & Simonds, 2007; Dewey & Bently, 1949; Greene, 1983; Hurt, Scott, & McCroskey, 1987; Langer, 1987; Rosenblatt, 1978, 2005; Smagorinsky & Whiting, 1995), what is known about readers‘ and writers‘ processes can also offer frameworks for understanding how teachers develop their ability to read and to communicate with their classrooms. In a transactional paradigm (Dewey & Bentley 1949; Rosenblatt 1978, 2005), reading and teaching share a common theory of meaning making. Like Rosenblatt‘s transactional theory of reading, teaching too is an experience, an event in the moment of a classroom —a dynamic interaction between a particular person‘s repertoire of experience and the ―text‖ of the classroom itself (Edge, 2011). To become classroom literate, a teacher must read and make meaning of specific classroom situations, including reading classroom discourse, making meaning of teachable moments, connecting theory and practice, constructing scaffolds to aid students‘ understandings, questioning and evaluating student progress, ―reading between the lines‖ of students‘ verbal and nonverbal language, and thinking critically and metacognitively about the

process of teaching. Couched in the concept of the classroom as a type of living, dynamic ―text‖ (Edge, 2008; Edge, 2011; Tobin, 2004; Witte, 1992) with which the teacher must transact in order to effectively teach, pre-service teachers can learn to construct their perception of teaching using their prior knowledge of their pedagogical training and learning to attune to the cues of a classroom as text. Through a learnership stance, Teri could conceivably learn to read the classroom as text and to compose her role in it by transacting with her students even as they transact with the texts they learn to read and write. Learning to read, and learning from one‘s misreading of the text of the classroom is a first step toward developing teacher literacy. For interns like Teri, shifting from a student-teacher stance where meanings are fixed based on previous readings of the classroom from her perspective as a student, to a learnership stance where the transactional nature of teaching and learning is valued and cultivated through a classroom community, places emphasis on the process of making meaning in the classroom context. It invites students to participate in the construction of their learning. It shifts power dynamics. It empowers teachers to create and revise teaching roles that are not scripts set in type.

References

Allen, J. (1995). It’s never too late: Leading adolescents to lifelong literacy. Portsmouth, NH Heinemann. Barton, P. E. (2000). What jobs require: Literacy, education, and training, 1940-2006. Washington, DC: Educational Testing Service. Retrieved December 10, 2007 from http://www.ets.org/Media/Research/pdf/PICJOBS.pdf. Biancarosa, C., & Snow, C. E. (2006). Reading next—A vision for action and research in middle and high school literacy: A report to Carnegie Corporation of New York (2nd ed.). Washington, DC: Alliance for Excellent Education. Burke, J. (2008). Thinking about teaching and learning. The English teacher’s companion: A complete guide to classroom, curriculum, and the profession (3rd ed., pp. 3-27). Portsmouth, NH: Heinemann Bullough, R. V., Jr., & Draper, R. J. (2004). Making sense of a failed triad: Mentors, university supervisors, and positioning theory. Journal of Teacher Education, 55(5), 407-420. Charmaz, K. (2000). Grounded theory: Objectivist and Constructivist methods. In N. K. Denzin and Y. S. Lincolm (Eds.), Handbook of qualitative research (2nd ed., pp. 509- 535). Thousand Oaks, CA: Sage. Cherian, F. V. (2003). Student teachers and their journey: A reflexive ethnography. Ph.D. dissertation, University of Toronto (Canada), Canada. Retrieved November 14, 2008, from Dissertations & Theses: Full Text database. (Publication No. AAT NQ78608). 439 pages. Clark, E. R. & Flores, B. B. (2007). Cultural literacy: Negotiating language, culture, and thought. Voices in the Middle (15)2, 8-14. Cruickshank, D. R., & Armaline, W. D. (1986). Field experiences in teacher education: Considerations and recommendations. Journal of Teacher Education, 37(3), 34-40. Cooper, P. J., & Simonds, C. J. (2007). Communication for the classroom teacher (8th ed.). Boston: Pearson, Allyn and Bacon. Creswell, J. W. (2018). Qualitative inquiry & research design: Choosing among five approaches (4th ed.). Thousand Oaks: CA, Sage.

Dewey, J., & Bently, A. F. (1949). Knowing and the known. Boston: Beacon. Edge, C. (2008). Classroom literacy: Toward a framework for reading and writing the classroom text.‖ Unpublished manuscript. University of South Florida, Tampa. Edge, C. (2011). Making meaning with ―readers‖ and ―texts‖: A narrative inquiry into two beginning English teachers‘ meaning-making from classroom events. Graduate School Theses and Dissertations. http://scholarcommons.usf.edu/etd/3722/ Edge, C. (2015). On the nature of experience in the education of prospective teachers: A philosophical problem. International Journal of Learning, Teaching and Educational Research, 13(1), 29-41. Eisner, E. W. (2005). Reimagining schools: The selected works of Elliot W. Eisner. New York: Routledge. Eisner, E. W. (1991). The enlightened eye: Qualitative inquiry and the enhancement of educational practice. New York, NY: Macmillan. Erickson, F. (1986). Qualitative methods in research on teaching. In M. Wittrock (Ed.), Handbook of research on teaching (pp. 119-161). New York: Macmillan. Gee, J. P. (2012). Social linguistics and literacies: Ideology in discourses (4th ed.). New York: Routledge. Graham, B. (2006). Conditions for successful field experiences: Perceptions of cooperating teachers. Teaching and Teacher Education, 22, 1118-1129. Greene, J. (1983). Teaching and learning: A linguistic perspective. Elementary School Journal (33), 353-391. Guyton, E., & Byrd, D. (Eds.), (2000). Standards: For field experiences in teacher education.. Reston, VA: Association of Teacher Educators. Hillocks, G. Jr. (1999). Ways of thinking, ways of teaching. New York: Teachers College Press. Holmes Group Inc. (1986). Tomorrow’s Teachers: A Report of the Holmes Group. Retrieved February 10, 2008 from http://www.holmespartnership.org/Tomorrows_Teachers.pdf. Hurt, H.T., Scott, M. D., & McCroskey, J. C. (1987). Communication in the classroom. Reading, MA: Addison-Wesley. International Reading Association. (2012). Adolescent literacy (Position statement, Rev. 2012 ed.) (pp. 1-18). Newark, DE. Langer, J. A. (1987). A sociological perspective on literacy. In J. A. Langer (Ed.), Language, literacy, and culture: Issues of society and schooling (pp. 1-20). Norwood, NJ: Ablex Publishing Corporation. Lankshear, C., & Knobel, M. (2007). Sampling ―the new‖ in new literacies. In M. Knobel, & C.Lankshear (Eds.), A new literacies sampler (pp. 1-24). New York: Peter Lang. Lortie, D. C. (1975). Schoolteacher: A sociological study. Chicago: University of Chicago Press. McIntyre, D. J., Byrd, D. M. & Foxx, S. M. (1996). Field and laboratory experiences. In J. Silkula, T. J. Buttery, & E. Guyton (Eds.), Handbook of research on teacher education (2nd ed., pp. 171-193). New York: Macmillan. Maclellan, E. (2008). Pedagogical literacy: What it means and what it allows. Teaching and Teacher Education, 24, 1986-1992. National Center for Education Statistics. (2006). Nation’s report card: Reading 2005. Washington, DC: U.S. Department of Education. Retrieved December 10, 2007 from http://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2006451. National Center for Education Statistics. (2007). Nation’s report card: Reading 2007. Washington, DC: U.S. Department of Education. Retrieved December 11, 2007 from http://nces.ed.gov/nationsreportcard/pdf/main2007/2007496.pdf National Council Teachers of English. (2007). Mapping diverse literacies for the twenty- first century: Opportunities, challenges, promising new directions. New York, National Council Teachers of English.

National Council Teachers of English (NCTE) (2008). The NCTE definition of 21st century literacies. NCTE Position Statement. Urbana, IL: National Council Teachers of English. Last retrieved April 27, 2009 from http://ww.ncte.org/positions/statements/21stcentdefinition. Onwuegbuzie, A. J., & Collins, K. M. T. (2007). A typology of mixed methods sampling designs in social science research. The Qualitative Report, 12(2), 281-316. Onwuegbuzie, A. J., & Leech, N. J. (2007). Sampling designs in qualitative research: Making the sampling process more public. The Qualitative Report, 12(2), 238-254. Retrieved March 20, 2008 from http://www.nova.edu/ssss/QR/QR12- 2/onwuegbuzie1.pdf. Perl, S. (1994). Composing texts, composing lives. Harvard Educational Review, 64(4), 427- 449. Popper, K. (1962). Conjectures and refutations. New York: Harper and Row. Richardson, L. (2000). Writing: A method of inquiry. In N. K. Denzin and Y. S. Lincoln (Eds.), Handbook of qualitative research, 512-29. Thousand Oaks, CA: Sage. Rogers, A. (2000). Literacy in the information age: Final report of the International Adult Literacy Survey. International Review of Education, (46)5, 467-473. Rosenblatt, L. (1938/1995). Literature as exploration. New York: MLA. Rosenblatt, L. (1978). The reader, the text, the poem: The transactional theory of the literacy work. Carbondale, IL: Southern Illinois University Press. Rosenblatt, L. (2005). Making meaning with texts: Selected essays. Portsmouth, Heinemann. Smagorinsky, P., Cook, L. S., Moore, C., Jackson, A. Y., & Fry, P. G. (2004). Tensions in learning to teach: Accommodation and the development of a teaching identity. Journal of Teacher Education, 55(1), 8-24. Smagorinsky, P. & Whiting, M. E. (1995). How English teachers get taught: Methods of teaching the methods class. Urbana, Illinois: National Council Teachers of English. Stake, R. E. (1995). The art of case study research. Thousand Oaks, CA: Sage. Sudzina, M. R. & Knowles, G. J. (1993). Personal, professional and contextual circumstances of student teachers who ―fail‖: Setting a course for understanding failure in teacher education. Journal of Teacher Education, 44(4), 254-262. Tobin, L. (1991). Reading students, reading ourselves: Revising the teacher‘s role in the writing class. College English, 53(3), 333-347. Tobin, L. (2004). Reading student writing: Confessions, meditations, and rants. Portsmouth, NH: Heinemann. Veal, M. L. & Rikard, L. (1998). Cooperating teachers perspectives on student teaching triad. Journal of Teacher Education, 49(2), 108-119). Wilson, E. K. (2006). The impact of an alternative model of student teacher supervision: Views of the participants. Teaching and Teacher Education, 22, 22-31. Witte, S. P. (1992). Context, text, and intertext: Toward a constructivist semiotic of writing. Written Communication, 9, 237-308. Zeichner, K. (2012). The turn once again toward practice-based teacher education. Journal of Teacher Education, 63(5), 376-382. Zeichner, K., & Beir, M. (2012). The turn toward practice and clinical experience in U.S. teacher education. Beiträge zur Lehrerbildung, 30(2), 153-170.

Appendix A: Themes Emerging During Data Analysis

Table 1 Teri: Interview Themes

Theme Examples from Teri’s Case Guiding Image or Sees herself as a Jedi Knight who is to learn the ways of the Metaphor for force Internship Relationship to Concludes, ―They hate me‖; feels like she must be the cool students one because she started off that way; thinks students view her as a ―peer‖; in another class she must be the ―prison warden‖; speaks of students as both a collective class unit and as individuals Relationship to Perceives her situation to be ―lucky‖ in comparison to Peers other intern‘s; adopts struggling peer interns; seeks council of experienced, practicing teachers as professional peers Relationship with ―Dr. Grandma‖ is both an experienced, credentialed, Cooperating professional, who expects results, and a maternal figure to Teacher turn to for professional support and guidance Teacher Role Characterized by tension between who she wants to be and the role she is currently in; Ideally a teacher is a mentor who facilitates and provides opportunities and choices for students to engage in personally meaningful learning; a teacher is someone who provides students with tools to become successful and to ―be something big‖; a teacher is an agent of change Primary Stance Student apprentice stance; inclination to master learning Toward Internship opportunities and objectives; sees herself as ―a young Jedi knight‖ who ―won‘t go to the dark side‖ Primary Foci when 1. Classroom management Teaching 2. Relating to students 3. Knowing when students are learning Personal Learning Internship has helped her ―grow up a bit,‖ but she still ―questions [her] own maturity‖ and struggles to establish a balanced relationship with students Definition of ―[F]ulfilling the requirements set by [the university], Successful establishing a good rapport with students and colleagues, Internship learning about the system from within‖ Self-Evaluation of Successful with ―mini failures scattered intermittently‖ Internship Most Influential How students respond to her Aspect of Internship

Table 2 Teri: Synthesized Themes

Theme Examples Tensions Between Sees her early notions of teaching as naïve; feels conflicted Ideal and Real by what she knows to do and what she actually does. ―I know I shouldn‘t raise my voice‖; ―Proximity control is supposed to work, but students keep talking even though I stand right there‖; ―I know you told us not to threaten students with writing, but I do…‖ Focus on what She Asserts that she is an excellent planner, yet her focus is on Struggles to her inability to manage student behavior and manage Accomplish or student questions in fast-paced honors courses with Needs to Master multiple projects going on at once Role of Teacher Defines and differentiates her role in terms of how Defined by different students respond to her, especially negative Different Classes‘ responses; in some classes she is the ―cool‖ ―peer,‖ the Responses to Her ―facilitator‖ or the ―mentor,‖ and in others she is the ―gatekeeper of knowledge‖ or the ―prison warden‖ Relationship to Evaluates success of her own conception of teaching in Others light of how others respond to her Perception of the Her role is to learn as much as possible; her university Triad supervisor is an evaluator and a source of information; her cooperating teacher is a professional guide who is to make sure specific things are accomplished, who provides scaffolding for Teri‘s learning, and who acts as a nurturing source of support. Stance as An overall synthesis of Teri‘s responses reveals her focus Student/Apprentice to be on mastering what she is unable to do; yet she is in a Teaching frustrated by her not knowing how to use what she does Situation know and how to learn what she does not know; she is a ―Jedi Knight‖ and a ―mentor‖ to her peers simultaneously; the possibility of teaching every grade level by the end of the semester thrills her; revises her prior understanding of teaching to assimilate new experiences; her perception of a successful internship is defined in terms of meeting the university‘s objectives.

International Journal of Learning, Teaching and Educational Research Vol. 16, No. 1, pp. 57-71, January 2017

Coping Strategies of Greek 6th Grade Students: Their Relationship with Anxiety and Trait Emotional Intelligence

Alexander- Stamatios Antoniou Department of Special Education and Psychology, Faculty of Primary Education, National and Kapodistrian University of Athens, Athens, Greece

Nikos Drosos Psychology Department, Faculty of Philosophy, Pedagogy, and Psychology National and Kapodistrian University of Athens, Athens, Greece

Abstract. The aim of this study was to investigate children’s coping strategies and explore their relationship with anxiety and trait emotional intelligence using a sample of 245 Greek 6th Grade students. Coping strategies are estimated with “Kidcope - Children version” (Spirito, Stark, & Williams, 1988). Trait Emotional Intelligence is measured with the “Trait Emotional Intelligence Questionnaire-Adolescent Short Form” (TEIQue-ASF) (Petrides, Sangareeau, Furnhum, & Frederickson, 2006) and anxiety is assessed with “State-Trait Anxiety Inventory for Children” (STAIC; Spielberger, 1973). Students reported “wishful thinking” and “positive coping” as their most frequently used coping strategies, while “blame and anger” is the least used strategy. The results, also, show that trait emotional intelligence is positively correlated with “positive coping” and negatively correlated with “social withdrawal”, “blame and anger”, “passive acceptance/ distraction”, and “wishful thinking”. On the other hand, both trait and state anxiety were positively correlated with “social withdrawal”, “wishful thinking”, and “blame and anger”, and positively correlated with “positive coping”. The results are discussed in terms of their implications for future research and early adolescents’ counselling.

Keywords: Children’s coping strategies, Trait Emotional Intelligence, State and Trait Anxiety

Introduction Late childhood and early adolescence are associated with changes in almost all domains of life: social, physical, cognitive and educational. These changes are often the cause of heightened stress. There are also other issues that contribute to children’s everyday stress, for example, negative events in their

environment and exposure by the media (economic crisis, refugee crisis, crimes, wars, etc.), family issues (financial problems, marital problems, divorce, illness, etc.), and school issues (peer interactions, academic achievements, bullying, etc.). Transition to Junior High school is a significant source of potential stress (Elias et al., 1992; Munsch & Wampler, 1993). This transition is accompanied with major changes in both social and academic contexts. Junior High school is much more demanding and complex than Primary Education School, while parents’ demands for academic success increase (Eccles et al. 1993). At the same time children will have to enter a new learning and social environment, start developing a new peer network and adjust to the demands of the new teachers. In Greece, the economic crisis has affected all major sectors of Greek social and economic life. According to Eurostat (10/2016) unemployment has reached an extremely high rate of over 23% while in the same period of 2008 it was reported as being under 8%, with an estimated 44% of the population living below the poverty line. Apart from the decrease of employment rates, the quality of employment has also deteriorated, as many individuals who remain employed are often forced to accept reduced working hours, and/ or lower salaries and benefits. An up to 40% reduction in salaries and pensions has forced Greeks to change their life style and adjust to a new reality within a very short period of time. These changes reflect an increase in psychological problems (Economou et al, 2011; Giotakos, 2010; Giotakos, Karampelas, & Kafkas, 2011); and have various negative effects on family life. As stressed by Takeuchi, Williams, & Adair (1991) long-term financial difficulties have often major effects on parents. They often exhibit symptoms of stress such as deterioration in physical health, and marital problems, while their parental behavior declines. The aforementioned issues are often felt by the children, thus increasing their own feelings of anxiety and stress (Wagner et al., 2015).

Anxiety and stress The various stressful situations that children face may lead to feelings of anxiety. Anxiety can be defined as feelings of unease, worry, tension, and stress (Nolen-Hoeksema, 2011). Anxiety is not directly caused by stressors, but it is rather the reaction to the perception of stressors. Cattell and Scheier (1958, 1961) proposed a distinction between state and trait anxiety, based on factor analytic studies. Trait anxiety refers to a relatively stable and permanent characteristic of an individual, while state anxiety refers to a transient condition that may vary from day to day. In the same concept of ideas Spielberger and his colleagues (1970) conceptualized trait anxiety as an acquired behavioral disposition that reflects stable individual differences in anxiety proneness; and state anxiety as a temporary emotional state. Anxiety and stress may manifest with both physical symptoms and emotional and behavioral changes. Physical symptoms may

include headaches, fast heartbeat, muscle tension, upset stomach, back pain, cold or sweaty hands and feet, chest pain, dry mouth, upset stomach, digestive difficulties, and sweating (Frank, 2003). Emotional and behavioral changes may include: changes in appetite, trouble sleeping, feeling of frustration, loss of interest in previously interesting activities, poor self-esteem and self-efficacy, concentration problems, excessive worry, fatigue, reduced school productivity, self-medicating (abuse of drug or alcohol), hyperactivity, difficulty concentrating, disturbing dreams/ nightmares, irrational anger, and outbursts.

Coping mechanisms Coping refers to thoughts and/or behaviors that an individual takes in order to minimize the negative effect that stressful events cause (Lazarus, & Folkman, 1984). The aforementioned definition of coping includes both changes that someone can impose on his/herself (e.g. cognitive restructuring) and changes that someone can impose on his/ her environment in order to minimize the negative emotional effects that he/she is experiencing. Attempting to categorize the various coping strategies, most authors recognize two broad types of coping: “approach” and “avoidant” strategies (e.g. Dempsey, Overstreet, & Moley, 2000; Rosario, Salzinger, Feldman, & Ng-Mak, 2003). Whilst approach strategies try to address the problem by managing the stressor, avoidant strategies attempt to assuage the negative emotions by evading the stressor. Exploring coping strategies used by children in Greece is of major importance in order to design interventions for empowering them against the most serious stressful events, such as the effects of the economic crisis. Such an understanding necessitates an adequate measure of coping among children. Given the relatively small number of available measures for assessing children’s coping mechanisms, the present study will investigate the psychometric properties of Kidcope (Spirito, Stark, & Williams, 1988), which is an instrument which is utilized worldwide, for measuring coping. . To our knowledge, this is the first time that this measure has been used among a Greek population.

Trait Emotional Intelligence According to Coleman (2008) emotional intelligence (EI) is a complex ability that includes monitoring one’s own and other people’s emotions, differentiating the various emotions and identifying them, and exhibiting the appropriate thinking and behavior. People with high EI are able to link intelligence, empathy and emotions; and subsequently to better understand the dynamics of interpersonal relationships (Mayer, 2008). Nevertheless, there are still substantial disagreements for the operational definition of EI. Two different EI constructs were proposed by Petrides and Furnham (2000, 2001, 2003): (a) trait

EI (or “trait emotional self-efficacy”) and (b) ability EI (or “cognitive-emotional stability”). Trait EI can be defined as a constellation of emotion-related self- perceived abilities and dispositions and can be measured with the use of self- report techniques. On the other hand, Ability EI can be defined as a constellation of emotion-related cognitive abilities and can be measured with the use of maximum performance tests. Emotional Intelligence is considered to be closely linked with effective coping (Matthew, & Zeidner, 2001; Snyder & Dinoff, 1999). As Furnham, Petrides, and Spencer-Bowdage (2002) have found, EI seems to be associated with healthy social coping styles. According to Gohm, Corse and Dalsky (2005) there is a negative correlation between EI and “avoidant” coping strategies, such as alcohol-drug engagement and behavioral disengagement, while there is a positive correlation between EI and “approach” coping strategies, such as seeking social support and religious coping styles. Siegling and his colleagues (2015) had similar findings, although their study was focusing on the psychometric characteristics of Trait Emotional Intelligence Questionnaire – Adolescent Short Form rather than exploring Trait EI relationship with coping styles. Nevertheless, there have been some recent studies that showed no correlation between ability EI and coping styles (Zeidner, Matthews, & Shemesh, 2016), and further research is needed.

The present study

The present study examines coping strategies that Greek 6th grade students adopt in order to reduce anxiety. Furthermore, the present study aims to investigate the level of student’s anxiety and trait emotional intelligence and to explore the possible correlations between coping strategies, anxiety and trait emotional intelligence.

Methodology Participants The sample included 245 6th Grade students, 107 (43.7%) of whom were boys and 138 (56.3%) were girls. All students were enrolled in Greek public schools. Schools that were selected to take part in the study were derived from various regions of Athens to ensure that families of diverse socioeconomic status were included in the sample. We chose to conduct this research among 6th grade students because within the Greek educational system, sixth grade is the final grade of Primary Education, whereby students need to prepare for their

transition to Junior High School where they will have to face new challenges. This upcoming transition might function as an extra stressor for children.

Instruments State-Trait Anxiety Inventory for Children (STAIC; Spielberger, 1973) was used to measure children’s distress. STAIC is a self-report psychometric tool which assesses state and trait anxiety of children aged between 8-18 years. STAIC provides two different scores: state anxiety and trait anxiety. Each scale consists of 20 items. Children are invited to report how they feel (e.g., calm, tense) at a particular moment in time (state scale) and how they feel in general (trait scale) rated on a 4-point Likert-type scale. STAIC was translated and adapted to Greek by Psychountakis (1995). Both STAIC-State and Trait scales have shown very good psychometric properties, and several studies with children support their reliability and validity. Evidence for the construct validity of the STAIC-State subscale has been presented (Spielberger, 1973). Additionally, STAIC-Trait scale exhibits relatively high correlations with other psychometric tools that assess similar constructs, which ensures its’ concurrent validity. Internal consistency reliability coefficients in the present study were found to range from 0.82 (STAIC- trait scale) to 0.91 (STAIC- state scale). Trait Emotional Intelligence Questionnaire – Adolescent Short Form (TEIQue-ASF; Petrides, Sangareeau, Furnhum, & Frederickson, 2006) was used to assess children’s trait emotional intelligence. TEIQue is a self-report inventory that assesses emotion-related self-perceived abilities and dispositions. The adult full version of TEIQue encompasses four factors (“well-being”, “self-control”, “emotionality”, and “sociability”) that can be further divided into 15 subscales. It also provides a score for global trait EI (Petrides, 2009). The Adolescent Short Form is a more simplified version of the same questionnaire. It includes 30 short statements that assess the 15 trait EI facets, and can also provide an overall score for global trait EI. Although scores for the 15 subscales can be calculated, Petrides recommends the use of the global trait EI only for this version of the questionnaire. In our study internal consistency reliability coefficient is 0.86. According to its authors this form of the questionnaire can be addressed to children as young as 11 years old. It was translated and adapted to Greek by Petrides, Pita and Kokkinaki (2007). Several studies provided evidence regarding it incremental validity (e.g. Davis & Humphrey, 2012; Siegling et al., 2015). Kidcope – Children Version (Spirito, Stark, & Williams, 1988) was used to assess children’s coping strategies. KIDCOPE is a brief checklist that assesses both cognitive and behavioral coping strategies. There are two different versions of Kidcope: one for adolescents (aged 13 to 16 years) and one for younger children

(aged 5 to 13 years). The child version that was used in the present study is comprised of 15 items that measure the use of 10 coping mechanisms (“distraction”, “social withdrawal”, “cognitive restructuring”, “self-criticism”, “blaming others”, “problem solving”, “emotional regulation”, “wishful thinking”, “social support”, and “resignation”). Although Kidcope is designed to assess both frequency (i.e. “How often did you do this?”) and efficacy (i.e. “How helpful was it?”) of the use of each strategy, in the present study only the frequency was measured. A four-point Likert-type scale ranging from “not at all” (1) to “almost all the time” (4) is used. As there are few available instruments that assess children’s coping strategies, Kidcope is widely utilised, although there are varying results regarding its psychometric properties and factor structure. Several different factor structures have been proposed. Among them, there are: a five-item single factor (Spirito, 1996), two factors (Cheng, & Chan, 2003; Spirito, 1996), three factors (Spirito, Stark, & Tyc, 1994; Vigna, Hernandez, Kelley, & Gresham, 2010), and four factors (Vernberg et al, 1996). It should be noted that even the studies with the same number of factors did not find the same factor structure. The aforementioned findings suggest that Kidcope’s factor structure is not stable and may vary across diverse samples. Although the Kidcope measure has been used to measure coping strategies in a variety of populations, to our knowledge it is the first time that it has been used in Greece.

Procedure Special permission to conduct the research in Greek public Primary Education schools was granted by the Greek Ministry of Education. Subsequently, members of the research team explained the aim of the study to the school principals. The children that took part in the study were instructed to answer all questions with sincerity stressing that the questionnaires are anonymous, participation is voluntary, and the results will only be used for research purposes. Questionnaires were answered exclusively in class, while there were no time constraints.

Results As Kidcope’s factor structure is not well established, factor analyses were performed in order to examine it. We chose to perform Exploratory factor analysis (instead of Confirmatory), as previous studies had presented a plethora of different factor structures in various populations.

EFA was performed to investigate Kidcope’s internal structure. Factorability of the 15 items was checked with the use of Kaiser-Meyer-Olkin measure of sampling adequacy, and Bartlett’s test of sphericity. Both tests suggested reasonable factorability [ΚΜΟ=0.62, χ² for the Bartlett’s test of sphericity = 456,076 (df=105, p<.001)]. Factor analysis included 15 items for the 245 6th grade students. Multiple solutions with varying number of factors were examined using varimax and oblimin rotations as extraction methods. The final solution (using varimax rotation) revealed six (6) factors, to which 61.67% of the variance can be attributed; and was preferred because it provided the best interpretation of the factors. Internal reliability coefficients were rather low (Table 1), but it was expected due to the small number of items in each factor.

Table 1. Factor loadings based on PCA with varimax rotation for the 15 items of the Kidcope

Factors Items 1 2 3 4 5 6

“Try to sort the problem out by doing something .75 -.27 -.05 -.11 -.01 -.04 or talking to someone”

“Try to sort out the problem by thinking of .71 -.19 .16 .26 .04 .02 answers”

“Try to see the good side of things” .55 .31 -.34 .14 -.13 -.18

“Do nothing because the problem cannot be -.06 .79 .05 -.10 -.02 .02 sorted anyway”

“Keep quiet about the problem” -.26 .57 .18 -.29 -.19 -.21

“Try to forget the problem” .00 .47 -.16 .21 -.01 .12

“Do something like watch TV or play to forget it” -.13 .44 .17 -.32 .42 .04

“Wish I could make things different” .05 -.16 .78 -.15 -.11 .09

“Wish the problem never happened” -.03 .17 .75 .14 .01 -.09

“Blaming someone else for causing the problem” -.03 -.01 -.02 -.78 -.05 .02

“Shout, scream and get angry” -.07 .06 .03 -.75 .02 -.07

“Stay on my own” -.11 .21 .36 -.05 -.65 -.06

“Try to feel better by spending time with others” .53 .15 .04 .03 .60 .09

“Try to calm down” -.13 .01 .08 .12 .12 .85

“Blaming myself for causing the problem” -.26 -.14 .25 .19 .48 -.56

Percentage of Variance explained (total: 61.67%) 12.84 11.35 10.87 10.70 8.12 7.79

Internal Consistency Coefficients .52 .47 .55 .49 .40 .39

Based on the content of the items that constitute each component, they were correspondingly labeled: Component 1, labeled Positive Coping, consisted of items that refer to behaviors focusing on solving the problem and on cognitive restructuring (e.g., “try to sort the problem by thinking of answers”, or “try to see the good side of things”. Component 2, labeled Passive acceptance/ Distraction, included items that refer to behaviors of avoidance of the problem such as distraction and resignation (e.g., “Try to forget the problem”, “do nothing because the problem cannot be sorted anyway”). The third component, labeled Wishful Thinking, included items pertaining to wishful thinking (e.g. “wish the problem never happened”). The fourth component, labeled Blame and anger, consisted of items pertaining to behaviors like blaming others and to emotional outburst (e.g. “Blaming someone else for causing the problem”, “Shout, scream and get angry”). Component 5, labeled Social withdrawal, consisted of items pertaining to behaviors such as seeking loneliness, and not spending time with others (e.g. “Stay on my own”, “try to feel better by spending time with others”). One item that had a reversed factor loading was reverse scored. Component 6, labeled Emotional Regulation, included items pertaining to behaviors of remaining calm and not blaming one-self (e.g. “try to calm down”, “blaming myself for causing the problem”. One item that had a reversed factor loading was reverse scored. Descriptive statistics were calculated for all variables (trait Emotional Intelligence, trait Anxiety, state Anxiety and the 6 coping factors). Means and standard deviations for each gender are presented in Table 2.

Table 2. Means and standard deviations for each gender for all measures

Males Females Total Variables M S.D. M S.D. M S.D. Min Max

Trait EI 144.86 27.23 143.14 25.98 143.81 26.43 63.00 203.00

Trait Anxiety 32.35 6.81 35.51 8.23 34.14 7.79 20.00 55.00

State Anxiety 30.71 6.47 32.37 7.90 31.63 7.33 20.00 54.00

Coping factors

Positive Coping 2.98 0.64 2.91 0.70 2.94 0.67 1.00 4.00

Passive 2.50 0.96 2.18 0.65 2.32 0.82 1.00 4.00 acceptance/Distraction

Wishful Thinking 2.82 0.83 3.18 0.81 3.02 0.84 1.00 4.00

Blame and Anger 1.85 0.86 1.72 0.70 1.77 0.77 1.00 4.00

Social Withdrawal 2.15 0.80 2.18 0.85 2.17 0.83 1.00 4.00

Emotional Regulation 2.60 2.22 2.43 0.85 2.50 1.60 1.00 4.00

Note. “Anxiety (State)”: Min possible: 20, Max possible: 60. “Anxiety (Trait)”: Min possible: 20, Max possible: 60, “Trait Emotional Intelligence”: Min possible: 30, Max possible: 210, “Coping factors”: Min possible: 1, Max possible: 4.

One-way MANOVA was conducted with gender as the independent factor and all measurements as depended variables. There was a statistically significant difference in measurements’ scores based on students’ gender, F (9, 193) = 3.08, p = .002; Wilk's Λ = 0.874, partial η2 = .15. Gender has a statistically significant effect on trait anxiety (F (1, 201) = 7.46; p < .01; partial η2 = .04), on passive acceptance/ distraction (F (1, 201) = 7.34; p < .01; partial η2 = .04), and on wishful thinking (F (1, 201) = 9.45; p < .005; partial η2 = .05). With regard to trait anxiety, boys presented lower scores (M=32.35, S.D.=6.81) than girls (M=35.51, S.D.=8,23). Regarding coping strategies, boys had higher scores for passive acceptance/ distraction (M=2.50, S.D.=0.96) than their female classmates (M=2.18, S.D.=0.65), while girls had higher scores for wishful thinking (M= 3.18, S.D.=0.81) than boys (M=2.82, S.D.=0.83). Subsequently, correlation coefficients (Pearson r) among the 6 coping factors and Trait EI, Trait Anxiety and State Anxiety were calculated (Table 3). Trait EI has a moderate positive correlation with Positive Coping (r=.47), a moderate negative correlation with Social Withdrawal (r=-.41), and low negative correlations with Blame and Anger (r=-.36), Passive acceptance/ Distraction (r=- .31), and Wishful Thinking (r=-.14). Trait Anxiety has a moderate positive correlation with Social Withdrawal (r=.45), a low negative correlation with Positive Coping (r=-.25), and low positive correlations with Wishful Thinking (r=.26), and Blame and Anger (r=.15). State Anxiety has a moderate positive correlation with Social Withdrawal (r=.44), a low negative correlation with Positive Coping (r=-.31), and low positive correlations with Wishful Thinking (r=.23), and Blame and Anger (r=.20).

Table 3. Correlation Coefficients (Pearson r) among the 6 coping factors and Trait EI, Trait Anxiety and State Anxiety

Coping factors Trait EI Trait Anxiety State Anxiety

Positive Coping .47** -.25** -.31**

Passive acceptance/Distraction -.31** .07 .11

Wishful Thinking -.14* .26** .23**

Blame and Anger -.36** .15* .20**

Social Withdrawal -.41** .45** .44**

Emotional Regulation .02 -.02 .02

Note. *p<0,05, **=p<0,01,

Discussion The purpose of the present study was two-fold: to explore coping strategies of 6th grade Greek students, and to examine the relationship of coping strategies’ with anxiety and trait emotional intelligence. Sixth grade students are preparing for the transition to Junior High School, and at the same time they are about to enter adolescence, both of which could function as major stressors. In order to assess children’s coping strategies, we used Kidcope, a widely accepted measurement of coping, and we examined its factor structure. Previous studies had suggested that Kidcope’s factor structure is not stable and may vary across diverse samples (Cheng, & Chan, 2003; Spirito, 1996; Spirito, Stark, & Tyc, 1994; Vigna, Hernandez, Kelley, & Gresham, 2010; Vernberg et al, 1996). The EFA of Kidcope yielded 6 factors: Positive Coping, Passive Acceptance/ Distraction, Wishful Thinking, Blame and Anger, Social Withdrawal, and Emotional Regulation. The presence of 6 coping strategies might indicate that 6th grade students’ coping strategies are quite differentiated. Although there were six clear factors, reliability coefficients were relatively poor, which might be attributed to the limited items that constitute each factor, and the children’s young age (Altshuler, & Rumble, 1989). The role of coping in child psychosocial adjustment has become the epicenter of several studies over the last decades. As various studies have shown, individuals who adopt active coping strategies, such as problem solving, are better functioning, in comparison to individuals who prefer coping strategies that are less active, such as social withdrawal (Endler & Parker, 1990; Peterson, 1989). According to our findings, Greek 6th grade students reported wishful thinking and positive coping as their most frequently used coping strategies. Emotional Regulation, Passive acceptance/ Distraction and Social Withdrawal are used less often, while Blame ad Anger is the least used strategy. These

findings suggest that Greek students use mainly functioning strategies that can result in reducing anxiety. Perhaps it would be interesting in future studies to examine whether different stressors (e.g. in the family, in school, etc.) result in the use of the same coping mechanisms. Our hypothesis that children would demonstrate quite high levels of anxiety because of the effects of the economic crisis and the transition to high school and to adolescence was not supported. Students reported rather low levels of both State and Trait Anxiety. Although the role of parental attitudes in children’s anxiety is undisputed (e.g. Bogels, & van Melick, 2004; Rapee, 1997; Sarason et al., 1960; Siqueland, Kendall, & Steinberg, 1996), further research is needed in order to explore the relationship between economic crisis and parental attitudes. It should be noted that the stressors adolescents experience may vary as they originate from different socio-economic backgrounds. Furthermore, 6th grade students are preparing for the transition to high school, but the transition has not yet taken place, and they are still in a familiar environment. Perhaps the transition related stress will manifest within the next year, when the children actually begin attending high school. One of the possible explanations for the low levels of anxiety could be associated to the functioning coping strategies that are used. There was no sex difference identified for State Anxiety, while for Trait Anxiety females reported higher levels than their male counterparts. There are several studies reporting preponderance of females in showing anxiety (e.g. Cole, Martin, Peeke, Seroczynski, & Fier, 1999; Lewinsohn, Gotlib, Lewinsohn, Seeley, & Allen, 1998). Multiple explanations have been proposed to explain this difference. McCauley Ohannessian and her colleagues (1999) associated anxiety with self-competence, and found evidence that the correlation between sex and trait anxiety decreased when the variance explained by self-competence was taken into account. Thus, these authors hypothesized that self-competence is a partial cause for the observed sex differences in depression and anxiety in early adolescents. Other studies (e.g. McLean, & Anderson, 2009) highlight the complexity of factors (biological influences, temperamental factors, cognitive factors, environmental factors) that might contribute to female predominance in anxiety. The current study suggests that trait Emotional Intelligence levels were moderately high. This finding emerges in accordance to the low levels of anxiety, and the high frequency of using functioning coping strategies. As hypothesized, trait Emotional Intelligence was positively correlated with positive coping, and negatively correlated with social withdrawal, blame and anger, and Passive acceptance/ distraction. These findings are consistent with previous studies (e.g. (Matthew, & Zeidner, 2001; Snyder & Dinoff, 1999; Furnham, Petrides, and Spencer-Bowdage, 2002; Gohm, Corse and Dalsky, 2005)

that have suggested a positive relationship with “healthy” coping styles. The negative correlation of Emotional Intelligence with anxiety was expected because it is natural that trait emotional intelligence is not consistent with feelings of unease, worry, tension, and stress. On the other hand, anxiety was expected to be positively correlated mainly to social withdrawal, which was the least frequently used coping strategy. This finding is consistent to previous studies that have shown that poor adjustment at the end of 6th grade was related to higher levels of school stress and less social support from family and teachers. Anxiety was also positively correlated to wishful thinking, blame and anger and negatively correlated to positive coping. Future research Our study highlights the necessity for further research in order to understand the coping mechanisms that Greek children use. Although, students reported the use of mainly functioning mechanisms, further investigation is needed to examine whether they exhibit the same mechanisms in different environments and stressors (e.g. school, family etc.). Students did not appear to exhibit high levels of stress as we hypothesized (due to their soon-to-be transition from Primary Education School to Junior High School). Future research should investigate whether increased stress appears in the first grades of Junior High School and whether students continue to have functioning coping mechanisms. Finally, further investigation should focus in the effects of Greece’s current economic situation in parental attitudes and subsequently in the effects of parental attitudes in their children’s anxiety levels and coping mechanisms.

References

Altshuler, J. L., & Ruble, D. N. (1989). Developmental changes in children’s awareness of strategies for coping with uncontrollable stress. Child Development, 60, 1337-1349. Bögels, S.M., & van Melick, M. (2004). The relationship between child-report, parent self- report, and partner report of perceived parental rearing behaviors and anxiety in children and parents. Personality and Individual Differences, 37, 1583-1596. Cattell, R. B., & Scheier, I. H. (1958). The nature of anxiety: A review of thirteen multivariate analyses composing 814 variables. Psychological Reports, 4 (Monograph Supplement 5), 351-388. Cattell, R. B., & Scheier, I. H. (1961). The Meaning and Measurement of Neuroticism and Anxiety. New York: Ronald Press. Cheng, S. T., & Chan, A. C. (2003). Factorial structure of the Kidcope in Hong Kong adolescents. Journal of Genetic Psychology, 164, 261-266. Cole, D. A., Martin, J. M., Peeke, L. A., Seroczynski, A. D., & Fier, J. (1999). Children's Over- and Underestimation of Academic Competence: A Longitudinal Study of Gender Differences, Depression, and Anxiety. Child Development, 70(2), 459-473. Coleman, A. (2008). A Dictionary of Psychology (3rd ed.). Oxford: Oxford University Press.

Davis, S. K., & Humphrey, N. (2012). The influence of emotional intelligence (EI) on coping and mental health in adolescence: Divergent roles for trait and ability EI. Journal of Adolescence, 35, 1369–1379. doi: 10.1016/ j.adolescence.2012.05.007 Dempsey, M., Overstreet, S., & Moely, B. (2000). “Approach” and “avoidance” coping and PTSD symptoms in inner-city youth. Current Psychology, 19, 28-45. Eccles, J. S., Midgley, C., Wigfield, A., Buchanan, C. M., Reuman, D., Flanagan, C., Mac Iver, D. (1993). Development during adolescence: The impact of stage- environment fit on young adolescents' experiences in schools and in families. American Psychologist, 48(2), 90-101. Economou, M., Madianos, M., Theleritis, C., Peppou, L. E., & Stefanis, C. N. (2011). Increased suicidality amid economic crisis in Greece. The Lancet, 378, 1459. Elias, M. J., Urbiaco, M., Reese, A. M., Gara, M., Rothbaum, P. A., & Haviland, M. (1992). A measure of adaptation to problematic academic and interpersonal tasks of middle school. Journal of School Psychology, 30, 41- 57. Endler, N. S., & Parker, J. D. A. (1990). Multidimensional assessment of coping: A critical evaluation. Journal of Personality Processes and Individual Differences, 58, 844-854. Frank, K. (2003). The handbook for helping kids with anxiety and stress. Chapin, SC: Youth Light Inc. Furnham, A., Petrides, K., & Spencer-Bowdage, S. (2002). The effects of different types of social desirability on the identification of repressors. Personality and Individual Differences, 33, 119-130. Giotakos, O. (2010). Oikonomiki krisi kai psichiki ygeia [Economic crisis and mental health]. Psichiatriki [Psychiatry], 21, 195-204. Giotakos, O., Karampelas, D., & Kafkas, A. (2011). Epiptosi tis oikonomikis krisis stin psichiki ygeia [Effects of economic crisis in mental health in Greece]. Psichiatriki [Psychiatry], 22: 109-119. Gohm, C. L., Corser, G. C., & Dalsky, D. J. (2005). Emotional intelligence under stress: Useful, unnecessary, or irrelevant? Personality and Individual Differences, 39, 1017- 1028. Lazarus, R. S., & Folkman, S. (1984). Stress appraisal and coping. New York: Springer. Lewinsohn, P. M., Gotlib, I. H., Lewinsohn, M., Seeley, J. R., & Allen, N. B. (1998). Gender differences in anxiety disorders and anxiety symptoms in adolescents. Journal of Abnormal Psychology, 107(1), 109-117. Matthew, G., & Zeidner, M. (2001) Emotional intelligence, adaptation to stressful encounters & health outcomes. In R. Bar-On & J. D. A. Parker (Eds.), The handbook of emotional intelligence. San Francisco, CA: Jossey-Bass/Pfeiffer. Mayer, J. D. (2008). Emotional intelligence: New ability or eclectic traits? American Psychologist, 63, 503-517. McCullough, M. E., & Snyder, C. R. (2000). Classical sources of human strength: Revisiting an old home and building a new one. Journal of Social and Clinical Psychology, 19, 1-10. McLean, C. P., & Anderson, E. R. (2009). Brave men and timid women? A review of the gender differences in fear and anxiety. Clinical Psychology Review, 29(6), 496-505. Munsch, J., & Wampler, R. S. (1993). Ethnic differences in early adolescents' coping with school stress. American Journal of Orthopsychiatry, 63, 633-646. Nolen-Hoeksema, S. (2011). Abnormal Psychology (5th ed.). New York, NY: McGraw-Hill. Peterson, L. (1989). Coping by children undergoing stressful medical procedures: Some conceptual, methodological, and therapeutic issues. Journal of Consulting and Clinical Psychology, 57, 380-397. Petrides, K. V. (2009). Technical Manual for the Trait Emotional Intelligence Questionnaires (TEIQue). London, England: London Psychometric Laboratory. Petrides, K. V., & Furnham, A. (2000). On the dimensional structure of emotional intelligence. Personality and Individual Differences, 29, 313-320.

Petrides, K. V., & Furnham, A. (2001). Trait Emotional intelligence: Psychometric investigation with reference to established trait taxonomies. European Journal of Personality, 15, 425-448. Petrides, K., & Furnham, A. (2003). Trait emotional intelligence: Behavioural validation in two studies of emotion recognition and reactivity to mood induction. European Journal of Personality, 17, 39-57. Petrides, K. V., Pita, R., & Kokkinaki, F. (2007). The location of trait emotional intelligence in personality factor space. British Journal of Psychology, 98, 273-289. Petrides, K. V., Sangareau, Y., Furnham, A., & Frederickson, N. (2006). Trait emotional intelligence and children’s peer relations at school. Social Development, 15, 537- 547. Psychountakis, M. (1995). To Erotimiatologio Agchous Katastasis kai Prodiathesis gia paidia (STAIC): Prosarmogi se Elliniko plithismo [The State and Trait Anxiety Inventory for Children (STAIC): Adaptation in Greek population]. Thesis. Ekpaidefsi Psychologikon Aksiologiseon [Training in Psychological Assessment], Amoikto Psichotherapftiko Kentro [Open Center of Psychotherapy]. Rapee, R. M. (1997). Potential role of childrearing practices in the development of anxiety and depression. Clinical Psychology Review, 17(1), 47-67. Rosario, M., Salzinger, S., Feldman, R. S., & Ng-Mak, D. S. (2003). Community violence exposure and delinquent behaviors among youth: The moderating role of coping. Journal of Community Psychology, 31, 489-512. Sarason, S. B., Davidson, K. S., Lighthall, F. F., Waite, R. R., & Ruebush, B. K. (1960). Parental attitudes. In S. B. Sarason, K. S. Davidson, F. F. Lighthall, R. R. Waite, & B. K. Ruebush (Eds) Anxiety in elementary school children: A report of research, (pp. 189-233). Hoboken, NJ, US: John Wiley & Sons Inc Siegling, A. B., Vesely, A. K., Saklofske, D. H., Frederickson, N., & Petrides, K. V. (2015, June 26). Incremental Validity of the Trait Emotional Intelligence Questionnaire- Adolescent Short Form (TEIQue-ASF). European Journal of Psychological Assessment. Advance online publication. http://dx.doi.org/10.1027/1015- 5759/a000267 Siqueland, L., Kendall, P. C., & Steinberg, L. (1996). Anxiety in children: Perceived family environments and observed family interaction. Journal of Clinical Child Psychology, 25(2), 225-237. Snyder, C. R., & Dinoff, B. L. (1999). Coping: Where have you been? In C. R. Snyder (Ed.), Coping: The psychology of what works (pp. 3- 19). New York: Oxford University Press. Spielberger, C. (1973). STAIC preliminary manual. Palo Alto, CA: Consulting Psychologists Press. Spielberger, C. D., Gorsuch, R. L., & Lushene, R. E. (1970). Manual for the State-Trait Anxiety Inventory. Palo Alto, CA: Consulting Psychologists Press. Spirito, A. (1996). Commentary: Pitfalls in the use of brief screening measures of coping. Journal of Pediatric Psychology, 21, 573-575. Spirito, A., Stark, L. J., & Tyc, V. L. (1994). Stressors and coping strategies described during hospitalization by chronically ill children. Journal of Clinical Child Psychology, 23, 314-322. Spirito, A., Stark, L. J., & Williams, C. (1988). Development of a brief coping checklist for use with pediatric populations. Journal of Pediatric Psychology, 13, 555-574. Takeuchi, D. T., Williams, D. R., & Adair, R. K. (1991). Economic stress in the family and children’s emotional and behavioral problems. Journal of Marriage and the Family, 53, 1031-1041. Vernberg, E. M., La Greca, A. M., Silverman, W. K., & Prinstein, M. J. (1996). Prediction of posttraumatic stress symptoms in children after Hurricane Andrew. Journal of Abnormal Psychology, 105, 237-248.

Vigna, J. F., Hernandez, B. C., Kelley, M. L., & Gresham, F. M. (2010). Coping Behavior in Hurricane-Affected African American Youth: Psychometric Properties of the Kidcope. Journal of Black Psychology, 36(1), 98-12. Wagner, S. L., Cepeda, I., Krieger, D., Maggi, S., D’Angiulli, A., Weinberg, J., & Grunau, R. E. (2015). Higher cortisol is associated with poorer executive functioning in preschool children: The role of parenting stress, parent coping and quality of daycare, Child Neuropsychology, DOI: 10.1080/09297049.2015.1080232 Zeidner, M., Matthews, G. & Shemesh, D.O. (2016). Cognitive-Social Sources of Wellbeing: Differentiating the Roles of Coping Style, Social Support and Emotional Intelligence. Journal of Happiness Studies, 17: 2481-2501. doi:10.1007/s10902-015-9703-z

International Journal of Learning, Teaching and Educational Research Vol. 16, No. 1, pp. 72-83, January 2017

Active Learning Across Three Dimensions: Integrating Classic Learning Theory with Modern Instructional Technology

Thaddeus R. Crews, Jr. Western Kentucky University Bowling Green, KY 42101

Abstract. Many educational researchers have proposed moving away from the traditional pedagogical model of long lectures with students as passive learners, listening and taking notes. There is over a century of pedagogical and cognition research that strongly suggests learning occurs best when students take an active role in the construction of their own knowledge. The call for change has only been accelerated by the increasing availability of computer-based educational technology and the exponential growth of digitally available information. This paper combines the long history of educational research with the rapid advancements of instructional technologies to build an Active Learning Model with three active dimensions (teachers, students, and technology). This model can help improve authentic learning within multiple disciplines and pedagogies. A Case Study is presented illustrating how the model helps improve student satisfaction as reported using standardized teaching evaluation metrics.

Keywords: Constructivism; pedagogy; active learning; problem based learning; educational technology

Introduction A century ago John Dewey said, “Give the pupils something to do, not something to learn; and the doing is of such a nature as to demand thinking, or the intentional noting of connections; learning naturally results” (Dewey, 1916, p. 181). Many educators agree with Dewey that students become more engaged in the learning process when the learning activity is dynamic, significant, and relevant to their lives. Cognitive scientists have an explanation for this, reporting that neural pathways in the brain are rewarded by the release of neurochemicals (such as dopamine) when student learning is active, meaningful, and authentic (Doyle, 2012). In this way, neuroscience is independently confirming the findings of multiple generations of educational researchers in the areas of effective teaching and learning.

Technology is doing more than just confirming classic research. Modern educational technologies are providing new opportunities to support the efforts of improving teaching and learning. This report extends a classical model for

engaged active learning by adding a third dimension that represent the increasingly important role of technology.

Classic Educational Research: The Roles of Teacher and Learner Educational research has a rich history of investigation into instructional designs for improving teaching effectiveness. Reflections on education go back at least to the Greek philosopher Socrates who is often recognized as the first great teacher (Nelson, & Brown, 1949). The Socratic method is a form of inquiry and debate based on asking and answering questions to stimulate critical thinking and to illuminate ideas. The Socratic method placed emphasis on the status of the individual to challenge the polity of the state, and the consensus of society. It was a thus a precursor of modern western intellectual individualism (Anthony, 2006). The skills of critical thinking and of challenging assumptions are just as useful today (if not more so) as they were 2500 years ago when Socrates developed his students in these areas.

Another important cognitive theory that impacts education is constructivism, based on the idea that learning improves when students construct their own understanding through experiencing things and reflecting on those experiences. According to the theory, when a person encounters something new, it has to be reconciled with the person’s previous ideas and experiences. This reconciliation might cause the learner to change what they previously believed, to discard the new information based on previous experience, or to refine their knowledge if the new experience is compatible with prior experience and understanding (Anzai & Simon, 1979). In the classroom, the constructivist view of learning leads to a variety of teaching practices that encourage active techniques for creating new knowledge and reflection to explain how one’s understanding is changing. Similar to a Socratic method, the teacher may guide the learner through the activity by asking foundational questions and encouraging the student to then build on them.

One advantage of active learning is that it avoids the problem of inert knowledge, which is information the learner possesses but cannot apply. To understand inert knowledge, consider a student who learns Distance/Rate/Time formulas (e.g., D = R * T and T = D / R and R = D / T) during math class through drill-and-practice on a worksheet of word problems, pulling numbers from the paragraph and plugging them in as the (hopefully) appropriate variable in the (hopefully) appropriate formula. But later in the day when learning about Charles Lindbergh and the Spirit of St. Louis in history class, the student is asked to figure out how long it took to complete the first solo non-stop transatlantic flight. When the student says she does not know, the teacher asks “What information would you need to determine how long the flight lasted?” The student does not realize to ask for the distance travelled or the rate of the plane because the Distance/Rate/Time formulas were learned as memorized inert information rather than active knowledge (Whitehead, 1929; Gick & Holyoak, 1980; Bransford, Franks, Vye, & Sherwood, 1989).

Active Learning The best way to avoid inert knowledge is to acquire knowledge as part of an active learning process. “In the process of learning, the learner's dynamic cooperation is required” (Gragg, 1940). Such cooperation from students does not arise automatically, however. It has to be provided for and continually encouraged. (Duncker, 1945). Active learning goes beyond memorization of facts and builds on the students' preexisting conceptions (Bonwell & Eison, 1991). Active learning activities involve opportunities for students to view information as means to important ends, which in turn helps students learn about the conditions under which knowledge is useful (Simon, 1980). When knowledge is learned actively, it increases the chances of spontaneously using that knowledge to solve new problems that are confronted later on (Bransford et al., 1990). Modern neurological science further supports the idea of active learning: “Fifteen years of neuroscience, biology, and cognitive psychology research finding on how humans learn offer this powerful and singular conclusion: It is the one who does the work who does the learning.” (Doyle, 2012, p. 7)

One example of active learning is problem based learning which often involves a process similar to the four problem solving stages as stated by Polya: (1) understanding the problem, (2) making a plan, (3) executing the plan and (4) reviewing the solution (Polya, 1945). Problem-Based Learning (PBL), as a general model, was refined in medical education in the early 1970's and since that time it has become common practice in most medical schools where PBL is used in the first two years of medical science curricula, replacing the traditional lecture based approach to anatomy, pharmacology, physiology, etc. (Savery and Duffy, 1995). The model has been adopted in an increasing number of other areas including Business Schools (Milter & Stinson, 1995), Schools of Education (Bridges, 1992; Duffy, 1994); Architecture, Law, Engineering, Social Work (Boud & Feletti 1991); and high school (Barrows & Myers, 1993). As Bransford et al. state, “Problem-oriented acquisition helps students appreciate the value of information” (Branford et al, 1990, p.121).

Great Teachers in 1966 The educational concepts of active learning and problem based learning are old ideas that have been successfully applied in a wide variety of classrooms over the last century. On May 6, 1966, Time Magazine did a cover story on 10 highly effective college professors from ten different universities and who taught in ten different academic disciplines (Figure 1). These professors may not have been aware of the active learning insights of James (1899), Thorndike (1913), and Dewey (1916), and they certainly were unaware of the future insights from neuroscience regarding active learning from the likes of Goswami (2006), Willis (2010) and Sigman et al. (2014). Nonetheless, these Time magazine professors were successful as teachers because they knew the critical importance of engaging their students with the material.

Figure 1. Time Magazine’s “Great Teachers” Cover, 1966

These “Best Teachers” of 1966 were each unique in their situations, teaching different subject disciplines to different student populations at different universities. What they had in common was intentional efforts to create a space where students would be engaged in highly interactive and authentic learning. This space can be captured visually in two dimensions where one axis represents the activity level of the teacher and the second axis represents the activity level of the learner (see Figure 2). In this model, the lower-left quadrant represents the Traditional Classroom where the student is a passive learning (listening and taking notes) and the teacher is also a passive participant (non-stop lecturing). This is not meant to be viewed as an indictment of lectures or the traditional classroom model. Rather it is a visual summary of the opportunities to make students more active in their education process, and also to make the instructor more active as well.

Figure 2: Active Learning Model for “Great Teachers” (circa 1966)

It is important to note that Figure 2 does not suggest any specific approach to active or authentic learning. Indeed, there is no “one size fits all” recommendation because effective learning depends on many factors, including the content being learned, the knowledge level of the learners, the class size, and the learning objective just to name a few. Specific techniques might work better in some situations than others. The good news is that there are many techniques designed to help participants (students and teachers alike) become more active in the learning process. If a teacher were to plot a specific instance of a course onto the Figure 2 chart, the location of that class would not be static, but would make many small moves over time based on the dozens of choices made in the design and refinement of the course, each decision moving the class slightly toward the upper-right quadrant or slightly toward the lower-left quadrant, or somewhere in-between.

Computing Technology in Education Research efforts to explore and understand the impact of computing technology in education has a longer history than one might think. Before a man walked on the moon, Atkinson and Wilson edited a book of research efforts in the area of computer-assisted instruction (Atkinson & Wilson, 1969). Over a decade before the introduction of the IBM personal computer, Seymour Papert and others were developing tools to allow children to develop hands-on computer programming skills at a time when computers were the size of refrigerators and cost tens of thousands of dollars (Papert & Solomon, 1971). In the early 1980s, Lepper analyzed four sets of research issues raised by the rapid intrusion of microcomputers into the lives of children, including the use computers as a vehicle for intrinsic motivation, the study of the instructional effectiveness in educational software, contrasting philosophies of instruction in different designs

of computer-based educational programs, and the effects of the computer on the goals of formal education (Lepper, 1985). These and similar investigations at the time are particularly impressive as they occurred before it was clear that computing technology was certain to play some type of role in education, and well before the World Wide Web came into existence in 1991 and the resulting exponential growth of information in the two-plus decades that have followed.

Today’s computing technology is becoming ubiquitous, and as such we cannot leave it out of our discussion about Active Learning and Great Teachers. An interesting thing happens when technology is added as a third dimension in the Active Learning model (see Figure 3). As with the teacher and student dimensions, emphasizing interaction with technology moves the class away from the passive traditional classroom and toward a more authentic and personalized learning experience.

Figure 3. The 3D Active Learning Model in Three Dimensions

This model has implications for a variety of high profile educational technology issues. For example, enrollments in online classes have grown significantly (Clark & Mayer, 2007) and many traditional institutions are expanding their capacity for online courses (Allen & Seaman, 2013). Likewise, hybrid courses that blend face-to-face and online instruction are one the rise (Bonk & Graham, 2006; Keengwe, Onchawari, & Oigara, 2014). Too often faculty are not provided sufficient guidance on how to integrate technology into these new online or hybrid courses.

It is possible to replace face-to-face lecture with recorded lecture, but that does not move the learning experience out of the lower-left quadrant of the Active Learning Model. Using technology to replicate the traditional classroom might be cost effective, but it does not improve learning. When technology is

introduced with the goal of increasing the interactivity of the learning experience, educational technology begins to fulfill its promise. This is consistent with a study by Crews and Butterfield where students identified interaction through class discussions and other types of active learning activities as the single greatest predictor of success for a face-to-face course (Crews, Butterfield 2014).

CASE STUDY: Applying the 3D Active Learning Model to a Computer Literacy Course The 3D Active Learning Model has been implemented and revised over multiple semesters in a college of business service course on computer literacy. The class covers traditional literacy content on computer hardware, computer software, and computer networks. The course also requires students to complete numerous hands-on projects with Microsoft Word, Microsoft PowerPoint and Microsoft Excel. This is significant amount of material for a single class, with some schools having required textbooks that total over 700 pages of reading material. Furthermore, students enter the class with diverse backgrounds regarding computer knowledge and skills, which makes it even more challenging to teach in a way that the majority of the students are challenged and engaged.

For this case study, technology has been incorporated into this course according to the 3D Active Learning model with the goal of moving the course toward the highly interactive and authentic learning space. The first active technology introduced in this course was SAM Projects, an expert system for grading student projects in Microsoft Word, PowerPoint and Excel (Cengage, 2016). This technology is a type of expert system that allows students to get detailed feedback on their hands-on Microsoft Office projects. SAM Project grades the students work and provides detailed feedback almost immediately regardless of the time of day. When using SAM, a student may submit an assignment at 2:00 am, and the student will receive immediate detailed expert feedback while the project is still fresh in the student’s mind. Furthermore, students are able to use the feedback to resubmit their work multiple times (as determined by the faculty member) and to improve their work based on the expert feedback.

A second technology element used in this revised course is recorded lectures, which are increasingly common in a variety of online and flipped classrooms (Thompson, 2011). Recorded lectures themselves are not more interactive than a classroom lecture, but they lecture delivery to occur outside of class, which allows face-to-face classroom time to be repurposed to support more active learning activities. Using technology outside of the classroom to increase the human interaction within the classroom is consistent with the book “Teaching Naked: How Moving Technology out of your College Classroom Will Improve Student Learning” (Bowen, 2012) that won 2013 Best Book on Higher Education from the American Association of Colleges and Universities.

A third technology element incorporated in this Case Study was to increase interactivity through online practice quizzes. After each assigned chapter

reading, an online practice quiz was made available to the students containing a pool of questions first drafted by the publishers and edited and pruned by the faculty member. These questions are made available for a period of time (e.g., 1 week) and the students could take the practice quiz multiple times during that week.

Strategic technology adoptions are only part of this case study. Lectures became shorter and more interactive, emphasizing the major challenges and opportunities that students will be facing. Student participation was increased. Small group projects, discussions, debates, class-wide problem solving activities, and in-class demonstrations were all considered based on the main learning objectives for that material and the technique that seemed most intuitive to the faculty member.

The 3D Active Learning model is not a one-size-fits-all solution. It involves treating each learning objective uniquely with the goal of increasing interactivity by the student, the faculty member and the available technology. It involves a series of small steps that cumulatively help move the course toward the desired furthest quadrant where active learning is at its highest.

Results The course revisions over three semesters based on the 3D Active Learning Model have had a demonstrably positive effect. Students showed an increase in attendance, alertness, participation, and attitude. Class meetings were more engaged and pleasant.

In addition, students reported higher satisfaction with various elements of the course as captured by the university’s standardized Student Instructor Teaching Evaluation (SITE) scores. Students rated the professor of the 3D Active Learning course above college and departmental averages across all questions in the SITE evaluation over each of the past three semesters.

Furthermore, student evaluations show a positive increase over the past three semesters while college and departmental averages remain relatively flat (with spring 2015 included as a baseline). By increasing engagement, students felt the learning was more authentic and in line with the course objectives (Figure 4), feedback (Figure 5), teacher effectiveness (Figure 6), and student learning (Figure 7).

Figure 4. Improved SITE evaluations regarding learning objectives

Figure 5. Improved SITE evaluations regarding feedback

Figure 6. Improved SITE evaluations regarding teacher effectiveness

Figure 7. Improved SITE evaluations regarding student learning

Conclusion There is a long and rich history of educational research showing improved student learning through increased interaction and engagement. Each teacher, each student, each topic, and each setting is unique. But with each of these unique challenges, there are multiple opportunities to make small changes in course design and implementation with the goal of moving students toward the most authentic and interactive learning space possible.

The 3D Active Learning Model provides a visualization for faculty looking to increase the level of active learning in their course. The model is especially helpful for faculty incorporating technology in the form of online or flipped

classes. The 3D Active Learning Model does not recommend any specific activity or event or technology. Rather it is a model that encourages incremental change away from a traditional classroom toward an environment of more authentic learning involving increased interaction across three dimensions: teacher, learning, and technology. It is a journey over time that requires a series of pedagogical decisions where each choice is guided by the goal of increasing active learning. Collectively these small changes have the effect of creating a learning space that is well suited for better teaching and more authentic learning.

References

Allen, E., & Seaman, J. (2013). Changing Course: Ten Years of Tracking Online Education in the United States, Babson Survey Research Group. Retrieved from http://www.onlinelearningsurvey.com/reports/changingcourse.pdf Anthony, M. (2006). A genealogy of the western rationalist hegemony. Journal of Futures Studies, 10(4), 25-38. Anzai, Y. & Simon, H.A. (1979). The theory of learning by doing, Psychological Review, 86, 124–140. Atkinson, R. C., & Wilson, H. A. (1969). Computer-Assisted Instruction: A Book of Readings. New York: Academic. Barrows, H. S., & Myers, A. C. (1993). Problem-based learning in secondary schools. Unpublished monograph. Springfield, IL: Problem-Based Learning Institute, Lanphier High School and Southern Illinois University Medical School. Bates, A., & Poole, G. (2003). Effective Teaching with Technology in Higher Education. San Francisco: Jossey-Bass/John Wiley. Bloom, B. S. (1984). The 2 sigma problem: The search for methods of group instruction as effective as one-to-one tutoring. Educational researcher, 13(6), 4-16. Bonk, C., & Graham, C. (2006). The Handbook of Blended Learning Environments: Global Perspectives, Local Design. San Francisco: Jossey-Bass/Pfeiffer Bonwell, C.C. & Eison, J.A. (1991). Active learning: Creating excitement in the classroom. ASHE-ERIC Higher Education Report No. 1. Washington, DC: ERIC Clearinghouse on Higher Education, The George Washington University. Boud, D., & Feletti, G. (1997). The challenge of problem-based learning. Psychology Press. Bowen, J. (2012). Teaching naked: How moving technology out of your classroom will improve student learning. Hoboken. Bridges, E. M. (1992). Problem Based Learning for Administrators. ERIC Clearinghouse on Educational Management, University of Oregon, 1787 Agate Street, Eugene, OR 97403-5207. Bransford, J. D., Franks J. J., Vye, N. J., Y Sherwood, R. D. (1989). New approaches to instruction: Because wisdom can’t be told. In S. Vosniadou & A. Ortony (Eds.), Similarity and analogical reasoning (pp. 470-497). New York: Cambridge University Press. Bransford, J. D., Sherwood, R. D., Hasselbring, T. S., Kinzer, C. K., & Williams, S. M. (1990). Anchored instruction: Why we need it and how technology can help. In D. Nix & R. J. Spiro (Eds.), Cognition, education, and multimedia: exploring ideas in high technology. Hillsdale N.J.: L. Erlbaum. Cengage Learning (2016). SAM Instructor User Manual. Retrieved from http://assets.cengage.com/pdf/gui_sam-inst-comp-user-guide.pdf Clark, R., & Mayer, R. (2007). eLearning and the Science of Instruction. San Francisco: Pfeiffer

Crews, T. & Butterfield, J. (2014). Data for Flipped Classroom Design: Using Student Feedback to Identify the Best Components from Online and Face-to-Face Classes. Higher Education Studies; 4, 3, 38-47. Dewey, J. (1916). Democracy and Education: An Introduction to Philosophy of Education. Macmillan. Doyle, T. (2012). Learner-centered teaching: Putting the research on learning into practice. Stylus Publishing, LLC.. Duncker, K. (1945). On problem-solving. Psychological monographs, 58(5). Duffy, T. M. (1994). Corporate and Community Education: Achieving success in the information society. Unpublished paper. Bloomington, IN: Indiana University. James, W. (1899). Talks to teachers on psychology: And to students on some of life's ideals. New York: Henry Holt and Company Keengwe, J., Onchawari, G., & Oigara, J. (2014). Promoting Active Learning through the Flipped Classroom Model. Hershey PA: IGI Global. Gick, M. L., & Holyoak, K. J. (1980). Analogical problem solving. Cognitive psychology, 12(3), 306-355. Goswami, U. (2006). Neuroscience and education: from research to practice?. Nature reviews neuroscience, 7(5), 406-413. Gragg, C. I. (1940). Because wisdom can't be told. Harvard Business School Publishing. Lepper, M. R. (1985). Microcomputers in education: Motivational and social issues. American Psychologist, 40(1), 1. Nelson, L., & Brown, T. K. (1949). Socratic Method and Critical Philosophy Selected Essays. Milter, R. G., & Stinson, J. E. (1995). Educating leaders for the new competitive environment. In Educational innovation in economics and business administration (pp. 30-38). Springer Netherlands. Papert, S., & Solomon, C. (1971). Twenty things to do with a computer. Polya, G. (1945). How to Solve It. Princeton University Press. Savery, J. R., & Duffy, T. M. (1995). Problem based learning: An instructional model and its constructivist framework. Educational technology, 35(5), 31-38. Sigman, M., Peña, M., Goldin, A. P., & Ribeiro, S. (2014). Neuroscience and education: prime time to build the bridge. Nature neuroscience, 17(4), 497-502. Simon, H. A. (1980). Problem Solving and education. In D. T. Tuma & R. Reif (Eds.), Problem solving and education: Issues in teaching and research (pp. 81-96). Hillsdale, JR: Lawrence Erlbaum Associates. Thompson, C. (2011). How Khan Academy is changing the rules of education. Wired Magazine, 126, 1-5. Thorndike, E. L. (1913). Educational psychology: the psychology of learning. New York: Teachers College Press. Willis, J. (2010). The current impact of neuroscience on teaching and learning. Mind, brain and education: Neuroscience implications for the classroom, 45-68. Whitehead, A. N. (1929). The aims of education. New York: MacMillan.

International Journal of Learning, Teaching and Educational Research Vol. 16, No. 1, pp. 84-102, January 2017

The Effects of Cram Schooling on the Ethnic Learning Achievement Gap: Evidence from Elementary School Students in Taiwan

Yu-Chia Liu Department of Hospitality & Tourism, Taiwan Hospitality & Tourism College, Taiwan R.O.C. and Department of Curriculum Design and Human Potential Development, National Dong Hwa University, Taiwan R.O.C.

Chunn-Ying Lin＊, Hui-Hua Chen Department of Early Childhood Education, National Dong Hwa University, Taiwan R.O.C.

He Huang Department of Clinical Psychology, Chang Gung University, Taiwan R.O.C.

Abstract. For the last three decades, many studies have found an obvious achievement gap between non-minority students and minority students, which was mainly associated with a lower socioeconomic status and a deficiency of family learning resources, such as learning after school, of minority students. In many countries, cram schooling is the most commonly extra learning activity which is believed to have positive effects on learning achievement. However, there are few empirical studies to explore the relationships between cram schooling and the learning achievement gap of different ethnic students. This study used 630 fifth-grade students in Taiwan as samples and carried out a hierarchical regression analysis to discover the effects of cram schooling on the ethnic learning achievement gap of Taiwan’s young children. The results showed that cram schooling participation has non-linear effects, first ascending and then descending, on students’ learning achievement. In addition, those students who participated in privately tutored classes with higher fees charged demonstrated better learning achievement, but students enrolling in after-school programs in cram schools might not show the same outcomes. Further analysis indicated that minority children had fewer and poorer cram schooling resources than non-minority children. In addition, most minority children have lower socioeconomic status, giving them less opportunity to participate in cram schooling activities and after-class programs in schools, so their learning achievement was significantly lower than non-minority children.

Keywords: cram schooling, ethnic learning achievement gap, elementary school students, hierarchical regression analysis

Introduction Over the past few years, a number of researchers have concentrated on analyzing the academic achievement of students in different ethnic groups, and discovered that the learning gap between minority students and their counterparts of non-minorities remains constant (Brown-Jeffy 2009; Byun and Park 2012; Rowley and Wright 2011). Some of them suggest that the minorities’ learning performance falls off because of the lack of extracurricular and learning activities after school. Among the researches focusing on the relationship between extracurricular activities and the ethnic learning achievement gap, the function of academic cram schooling has aroused sincere consideration. Cram school is believed to have a positive effect on academic achievement, and has become a very visible worldwide phenomenon (Bray 2013; Bray, Zhan, Lykins, Wang and Kwo 2014; Kim and Park 2010; Kuan 2011; Kenayathulla 2013; Zhan, Bray, Wang, Lykins and Kwo 2013 ).

Cram schools have also existed in East Asian countries for a long time (Bray et al. 2014; Fung 2003; Lee and Shouse 2011), particularly those countries influenced by Confucianism, such as Taiwan, mainland China, South Korea and Japan. Traditional education values may make cram schools more crediable as an fundamental and much needed social organization (Liu 2012). Moreover, cram schooling activities may be influenced by parental socioeconomic status, thus affecting plurality and deprived students to face greater learning achievement gaps when compared to those who attend cram schools (Kim and Park 2010; Lin, Hsieh and Chen 2015; Lee and Shouse 2011).

Aboriginal students are a minority group of the population in Taiwan. The learning weakness of aboriginal students has been proven and is under constant analysis (Kuan 2011; Liu 2012). In the past 20 years, the population composition in Taiwan had changed enormously. This is the result of Taiwanese males marrying females from mainland China and southeast Asia due to their low socioeconomic status or other reasons. Therefore, Taiwan has a considerable number of children of cross-national marriage in all levels of schools. These children are more likely to fall behind their classmates in learning performance, but still little research has considered such possibilities and the effects of cross-national marriages. As a result, the academic field has limited understanding of how the family composition of cross-ethnic or cross-national marriages may affect a child’s learning outcome.

Based on the above considerations, this study used 630 fifth-grade Taiwanese students as samples, and carried out a hierarchical regression analysis. The researchers incorporated ethnic cross-national marriages, family background, and participation in academic cram schooling (time spent, expenditures, and patterns) as the mediating variables. These factors were then used to clarify the influential mechanisms of ethnic and cross-national marriages differences on students’ academic achievement. Hopefully this investigation will fill in the gaps in current research on the education achievement of different ethnic groups so

that we can better understand both the theoretical and practical implications of ethnic achievement gaps.

Literature Review Cram schooling is similar to shadow education (Bray et al. 2014; Kuan 2011; Lee and Shouse 2011). This is due to its coexistence with the mainstream schooling and how it mimics the regular school system, i.e. it duplicates the curriculum, the regulations, even the purposes of formal schooling. Cram schooling offers supplementary instruction aiming to assist students to catch up, keep up, or make headway of their peers. Furthermore, it is a fee-paying service for extra learning activities after school or during summer vacation, rather than the unpaid tutoring provided by family members or teachers in after-class programs. Additionally, cram schooling mainly focused on how to enhances students’ performance in terms of academic subjects (Liu 2012; Zhan et al. 2013), rather than extra-curricular activities to foster their cultural capital, i.e., music, art, sports, and so on (Shih & Yi 2014). In this study, cram schooling includes languages (Chinese and English), mathematics, science, and other academic subjects that feature in mainstream school examinations. All of these subjects are important branches of learning in the senior high school and university entrance examinations. In this respect, cram schools consist of outside school instruction, provided by profit-oriented organizations (Liu 2012), aiming to help students master academic subjects of school curriculum and improve academic performance in school (Byun and Park 2012; Lin et al. 2015), earn admission to elite schools (Fung 2003) and benefit in their future occupations and the social status (Stevenson and Baker 1992).

Cram schools are also known as Buxiban in Taiwan, Juku in Japan, Hagwon in South Korea, and private tuition or a shadow education system in Western countries (Bray 2013; Byun and Parker 2012; Kim and Park 2010; Kuan 2011). Though cram schools have many different names, they share a common feature in that they serve as a remedial or enrichment strategy. In Taiwan, two major types of cram schools can be categorized: (a) the cram schools that help students do schoolwork and prepare for entrance exams; (b) the institutes that provide training in foreign languages (Liu 2012; Shih and Yi 2014). These two types of cram schools can be regarded as academic cram schools due to the primary focus on students’ educational advancement, such as languages, writing, and mathematics, with the primary objectives of improving students’ abilities for taking tests, helping them gain higher grades, and enter prestigious high schools and universities. Attending cram schooling can increase academic performance, thereby opening up more opportunities for higher education.

Some empirical studies reveal that cram schooling has a positively significant influence on students’ academic achievement (Dang 2007; Kuan 2011; Liu 2012). Rhy and Kang (2013) illustrated that the true effect of private tutoring on academic performance remains, at most, modest. Specially, Chen and Hwang (2011) found a non-linear influence, first increasing and then decreasing, on the hours spent in cram schools. They explained that students’ fatigue caused by long cramming hours may decrease the learning efficiency, and that the long

hours spent cramming may decrease the time expended on schoolwork. However, Zhang (2013) doubted the effectiveness of private tutoring. In his study, he found trivial association between private tutoring and students’ academic performance. Therefore, the relationship between academic cram schooling and achievement still needs to be investigated thoroughly.

It needs clarification that cram schools in Taiwan are usually private and authorized by the local government. Their objectives are not only to assist students with their learning, but also to earn a profit. These cram schools are consequently fee-based. Moreover, students in Taiwan need to go to different types of cram schools for better scores on tests, leading to a vast household investment in cram schooling. The more famous cram schools produce better student performance, attract more parents and their children, and charge higher tuition fees, causing a heavy economic burden on less wealthy families (Lin et al. 2015). Accordingly, whether children can participate in private cram schools and how many subjects they can take closely relate to the socioeconomic status of the families. For instance, several studies have found that the higher the parental education level, the more academic subjects their children take at cram schools (Bray et al. 2014; Jung and Lee 2010; Kenayathulla 2013). Moreover, children from high socioeconomic status families are more likely to take part in academic cram schooling activities. Their families are also more likely to spend money on private tutoring (Bray et al. 2014; Jung and Lee 2010; Kim and Park 2010; Kenayathulla 2013; Lin et al. 2015; Shih and Yi 2014; Stevenson and Baker 1992). It may be that if there were more cram schools, the opportunities for students from lower socioeconomic status to attend cram schools and take cram courses would be greater.

There are minority ethnic groups in Taiwan: aborigines and cross-national marriage families with the mothers from mainland China and several southeastern Asia countries. These cross-national marriage families are named “new inhabitants”. The population of these minorities is less than ten percent and generally possesses lower socioeconomic status compared to their counterparts. In particular, most of the aborigines inhabit mountains or remote areas, engage in manual labor jobs, earn less income, possess less education, and therefore can provide their children with less educational resources. Similar to Western research results of learning racial/ ethnical gap (Brown-Jeffy 2009; Rowley and Wright 2011; Whitley, Rawana and Brownlee 2014), aboriginal students’ learning discrepancies in Taiwan have been verified, including lower educational levels, lower learning achievement, and lower ratio of universities enrollment, etc. (Lin and Hwang 2009 ).

Accordingly, we assume that aboriginal parents-most from lower socioeconomic status families cannot afford extra academic cram schooling expenditures. Moreover, based on the consideration of profits, few, even none, private cram schools are located in aboriginal areas. Likewise, some Taiwanese males choose to marry females from developing Asian countries, such as Vietnam, Indonesia and so forth. The primary reason for such cross-national marriage families is that

these low socioeconomic status Taiwanese males might marry females with the same socioeconomic status by means of foreign marriage matchmaking. The economic status of these cross-national marriage families might be relatively low, similar to the aboriginal groups. In addition, these mothers from cross-national marriage families cannot get jobs without Taiwanese resident identification until several years after getting married, and they may have difficulties in cultural adaption and oral communication. These difficulties probably cause disadvantages to cross-national marriage children regarding learning performance, including not being able to afford extra academic cram schooling expenditures.

Because of these circumstances, the Ministry of Education in Taiwan promoted a new project beginning in 2008. The project is named the “Night Angel Illumination Program” (Ministry of Education 2009). Its purpose is to help underachieving students to decrease the learning performance gap between minority and non-minority students (Lin & Ou 2010). Two dimensions of this project are “After-School Care Program” and “Hand-in-Hand Program”. ”After-School Care Program” offers after-school services, aiming to look after elementary school and kindergarten children, help with homework assignments, and guide these students learning. This program provides academic guidance in classrooms after school or during vacations. Generally speaking, children and their families have to pay a fee, but for physically and mentally handicapped students, and students from low-income and aboriginal families, this program is free of charge. Regarding the “Hand-in-Hand Program”, the students are from aboriginal families, single-parent, grandparent, new-resident families, or those from low socioeconomic families. Such programs are non-profit in orientation and aim to enhance learning outcomes of disadvantaged students and decrease the learning gap. Additionally, some academic guidance is sponsored by charity groups, religious groups, or foundations established by private enterprises. Financially disadvantaged university and college students or people with teaching experience are hired to provide extra after-school academic guidance. Such programs have low tuition fees, or are even free. The students participating in these programs are primarily minority or underprivileged students. However, there is no existing relevant research to verify the effectiveness of such academic guidance services, compared to private academic cram schools.

Based on the above considerations, the cram schooling activities in Taiwan can be grouped into three categories: (a) private profit-oriented cram schools, regarded as Buxiban; (b) fee-based after-school programs provided by schools; and (c) free after-school programs sponsored by non-profit organizations or schools. According to relative viewpoints and empirical results (Brown-Jeffy 2009; Dang 2007; Kuan 2011; Lin et al. 2015; Liu 2012 ; Rowley and Wright 2011; Whitley et al. 2014), this study assumes that minority groups cannot afford private cram schooling expenditures due to their low socioeconomic status and hence, their children possess lower academic performance. However, with few relevant studies, some questions need to be addressed. First, few studies have been conducted to compare the learning environment and learning performance of cross-marriage children with aboriginal groups and non-minority groups.

Second, the effectiveness of free academic guidance services provided by schools or non-profit organizations, and their effect on the academic achievement of ethnic and cross-marriage children need more investigation.

Method Participants The analysis sample for this study includes children between age of 10 and 11 who had valid questionnaire data and their grade achievement assessment at the time of the survey in 2013. In total there were 630 pieces of valid student and parent data. Among the subjects, 49.4% were boys and 50.6% were girls. The average length of education for the fathers and mothers was 12.07 years (SD = 2.23) and 12.07 years (SD = 2.00), respectively, which was the equivalent of a senior high school graduation. The family economic status for the subjects were given 1-3 points according to the level of wealth, and the mean was 2.66 (SD = 0.50).

Measures Table 1 shows the design and scoring methods for each variable in this study. All variables used in this study are grounded into two types: (1) student individual characteristics, such as ethnicity, gender, and academic achievement, (2) family background and characteristics, including parent educational level, parent occupation, family economic conditions, family structure, sibling size, and children’s participation in cram schooling.

Table 1 Variable measurement Variable Description Metric Ethnicity The measures were divided Non-minorities = 0 (as into three categories: reference category) Non-minorities, aboriginals (minorities), and cross-national marriage (minorities).

Parent’s Convert the specified Elementary school = 6 years, educational parent’s educational level Junior high school = 9 years, level into education years. Senior high school = 12 years, Junior college = 14 years, College =16 years, Graduate school = 18 years. Parents’ What are fathers’ and professional and occupation mothers’ occupations? The semi-professional = 0 (as measures include four reference category) categories: professional and semi-professional, trading and service, as well as labor-related work and low-tech jobs.

Family The economic status of the Item ranged from 1 to 4. 4 = economic status families when the child Wealthy families, 1= poor studied at the fifth grade. families

Family structure Family structure is measured Single-parent and other with a set of binary household = 0 (as reference indicators reflecting whether category) the student lived in the two-parent, single-parent or other household. Student’s What is the student’s 1= boy, 0 = girl gender gender?

Sibling size How many siblings does the Item ranged from 1 to 6. 6 = child have? six or more siblings, 1= only one child in family Academic cram How many hours do you 0 = lowest hours; 20 = highest schooling spend each week hours Hours participating in cram schooling?

Expenditures How much do you spend on NT$ 0 = lowest amount; NT$ (NT$/thousand) participating in cram 20,000 = highest amount

schooling each week? After-school Whether have you taken part 1= yes 0 = no (as reference programs in in after-school programs in category) schools schools this semester?

After-school Whether have you taken part 1= yes 0 = no (as reference tutorial classes in after-school tutorial category) (private cram classes (private cram schools) schools) this semester?

After-school Whether have you taken part 1= yes 0 = no (as reference programs in after-school programs category) outside outside schools ( cost-free) Schools this semester ? (cost-free)

Academic The achievement estimation The achievements of each achievement values the average scores subject were based on from the first semester different scoring criteria in (including Chinese, math, each class. Therefore, social, science, and English) choosing a class as a group, in fifth grade students’ this study standardized survey. students’ learning achievements, then made linear transformation in scores with the mean value 85 and the standard deviation 10. The approximate distribution range for the achievements of each subject is as follows: Chinese: lowest score = 10.44 and highest score = 95.55; Math: lowest score = 39.74 and highest score = 92.51; Social: lowest score = 30.89 and highest score = 96.52; Science: lowest score = 25.08 and highest score = 95.01.

Analysis Strategy A series of hierarchical regression analysis was used with different groups of predictors in the model, first with ethnicity (including non-minorities, aboriginals, and cross-national marriages), then with ethnicity and controls for family background (such as parent educational level, parent occupation, family economic status, family structure, and sibling size) and children’s gender, and finally with children’s participation in cram schooling measures added. In this

study, we hypothesize that, when parental SES, family background, and children’s personal characteristics are included, the remained academic achievement gap between non-minorities and minorities would be reduced when cram schooling is held constant. Namely, the main purpose of this study is to examine the potential mediating pathways through which academic cram schooling mediates the ethnic gap of students’ academic achievement in Taiwan. Table 3 shows the effects of the predictor variables on the students’ grade achievement. Table 4 and table 5 are the analysis on mediating variables such as cram schooling hours, expenditures and participating types (as dependent variables). Since the dependent variables shown in Tables 3 and Table 4 were continuous variables, the researchers practiced multiple regression analysis. However, we conducted logistic regression analysis in Table 5 due to the dependent variables were binary scale.

Results Descriptive Analysis Table 2 displays the results of the descriptive analyses among the minority group students and non-minority group students in the respects of socioeconomic status, personal attributes and family background, participation in cram schooling, and their academic achievement. The results indicated that the parents from minority group families had a lower educational level than their counterparts, especially the cross-national couples (M = 10.82, 9.48; SD = 3.06, 2.97, respectively), which was equivalent to a junior high school level. Secondly, among the non-minority groups, many fathers were engaged in professional or semi-professional work (47.1%), whereas among the aboriginals and cross-national marriage, most fathers were engaged in labor-related work or low-tech jobs, occupying 66.3% and 64.3%, respectively. Among the non-minority group, the percentages of maternal jobs in the professional and semi-professional areas, trading and service sectors, and low-tech sectors were almost the same, over 30% each. However, in aboriginals and cross-national marriage, the mothers were mostly engaged in labor and low-tech jobs (61.1% and 74.1%, respectively).

An analysis of family economic status (based on a Likert 4-point scale) showed that most non-minority families had middle-class income level (M = 3.36, SD = 0.71), while the aboriginals families had a below-average living standard (M = 2.86, SD = 0.83). Also, among the non-minority groups and cross-national marriages, over 10% of the families were single-parent families or were headed by grandparents, whereas 27.3% of the aboriginal people had formed single parent families or families headed by grandparents. In all of the ethnic groups, girls tended to occupy a higher ratio. The average number of siblings was highest in the aboriginals families (M = 3.10, SD = 1.36).

Table 2 Descriptive statistics for family SES, family background, participation in cram schooling, and children’s test scores by ethnicity Non-minorities Minorities Minorities Ethnicity (aboriginal) (cross-national marriages) Variable N % Mean SD N % Mean SD N % Mean SD Parental educational level Father 502 13.82 2.78 100 11.23 2.95 28 10.82 3.06 Mother 500 13.50 2.52 100 10.69 2.78 27 9.48 2.97 Father’s occupation Professional and semi-professional 230 47.1 15 15.8 6 21.4 Trading and service 107 21.9 17 17.9 4 14.3 Labor-related work and low-tech jobs 151 30.9 63 66.3 18 64.3 Mother’s occupation Professional and semi-professional 159 32.1 12 12.6 2 7.4 Trading and service 141 28.5 25 26.3 5 18.5 Labor-related work and low-tech jobs 195 39.4 58 61.1 20 74.1 Family economic status 504 3.36 0.71 100 2.86 0.83 28 3.14 0.71 Family structure Two-parent household 421 84.7 72 72.7 24 88.9 Single-parent and other household 76 15.3 27 27.3 3 11.1 Student’s gender Boy 244 48.9 42 42.0 13 46.4 Girl 255 51.1 58 58.0 15 53.6 Sibling size 492 2.08 0.72 96 3.01 1.36 27 2.15 0.72 Cram schooling Hours 502 7.75 6.97 100 1.30 3.07 28 5.61 6.80 Expenditures (NT$/ thousand) 501 5.14 4.45 100 0.96 2.77 27 2.96 3.56 After-class programs in schools 499 21.4 99 64.6 28 39.3 After-class programs outside schools 499 69.9 99 15.2 28 53.6 (private cram schools) After-class programs outside schools 499 4.0 99 31.3 28 3.6 (cost-free ) Academic achievement (average) 490 85.86 7.47 99 79.88 8.77 28 84.59 7.78

In the aspect of cram schooling, the data showed that students from aboriginal families experienced the lowest number of hours and spent the least amount of money on cram schooling, followed by students from cross-national marriage, while students from non-minority families had the highest number of hours and largest expenditures spent on cram schooling. Furthermore, of the three main types of after-class schooling, nearly 70% of the students from non-minority families chose private cram schools that demand higher tuitions, whereas 64.6% of the aboriginal students chose to take extra hours in tutorial classes organized by schools. Meanwhile, 31.3% of the aboriginal students enrolled in tutorial classes organized by non-profit organizations, which were free of charge. Students from cross-national marriages, like their counterparts from non-minority families, tended to choose private cram schools (53.6%) and tutorial classes organized by schools (39.3%). Lastly, in the aspect of student academic performance, non-minority students scored the highest (M = 85.86, SD = 7.47), closely followed by students from cross-national marriage (M = 84.59, SD = 7.78). The aboriginals students, however, scored lower than their peers (M = 79.88, SD = 8.77).

Multivariate Analysis This study made ethnic group a predictive variable, as shown in Table 3, Model 1, and found that among the minority groups, aboriginal students scored far lower than non-minority students (B = -5.76, SE = 0.85, p < .05), yet there was no discernible difference between cross-national marriage students and their non-minority counterparts. In Model 2, students’ personal attributes and family backgrounds were included as control variables. Compared with the results showed in Model 1, the difference between aboriginal students and their non-minority peers became smaller (B = -1.98, SE = 1.57, p <.05). It was also discovered that students whose mothers worked as laborers or with low-tech jobs had poorer academic achievement. In Model 2, the explanatory power (R square) reached 0.20. In Model 3, hours of cram schooling and the quadratic term of cram schooling hours were included as mediators to observe the impact of cram schooling hours on academic achievement. The regression coefficient showed that longer hours of cram schooling resulted in better academic performance (B = 0.40, SE = 0.14, p <.05). However, the square of the hours spent on after-school cram schooling did not have a remarkable influence on academic achievement. By further taking hours in cram schooling into Model 3 for analysis, the disadvantage of aboriginal students grew smaller (B value dropped from -1.98 to -0.72), and the influence plummeted to 63.63%. This indicated that aboriginal students’ poorer performance might come from shorter hours in cram schooling. The explanatory power (R square) of Model 3 reached 0.22. Model 3 indicated that the cross-national marriage students, with hours in cram schooling excluded, had better academic achievement than the other two groups. When dealing with factors separately, such as personal attributes, socioeconomic status and hours in cram schooling taken as predict variables, the cross-national marriage students performed better than the non-minority students. This is worth of further discussion and scrutiny in future research. Model 4 and Model 5 displayed similar results.

Table 3 Estimates on the achievement score for 5th grade students Model 1 Model 2 Model 3 Model 4 Model5 B SE B SE B SE B SE B SE Ethnicity Non-minorities(reference categories) Minorities (aboriginal) -5.76 ** (0.85) -1.98 * (0.93) -0.72 (0.97) -.069 (0.96) -1.02 (1.02) Minorities(cross-national marriage) -1.05 (1.51) 2.97 (1.57) 3.32 * (1.55) 3.47 * (1.58) 3.59 * (1.55) Father educational level 0.53 ** (0.16) 0.50 ** (0.15) 0.46 ** (0.15) 0.47 ** (0.16) Mother educational level 0.16 (0.16) 0.16 (0.16) 0.10 (0.16) 0.21 (0.16) Father’s occupation Professional and semi-professional (reference categories) Trading and service 0.15 (0.87) 0.26 (0.86) 0.25 (0.85) 0.14 (0.86) Labor-related work and low-tech jobs -1.10 (0.79) -0.95 (0.79) -0.95 (0.78) -0.94 (0.79) Mother’s occupation Professional and semi-professional (reference categories) Trading and service -0.74 (0.82) -0.84 (0.82) -0.74 (0.81) -0.40 (0.82) Labor-related work and low-tech jobs -1.78 * (0.77) -1.50 (0.76) -1.51 * (0.76) -1.71 * (0.77) Family economic status 0.49 (0.46) 0.37 (0.46) 0.22 (0.46) 0.29 (0.47) Family structure Two-parent household 2.42 ** (0.86) 2.25 ** (0.86) 2.17 * (0.86) 2.06 * (0.86) Single-parent and other household (reference categories) Boy (reference categories: Girl) -0.48 (0.60) -0.47 (0.60) -0.40 (0.59) -0.41 (0.60) Sibling size -0.61 (0.37) -0.46 (0.36) -0.38 (0.36) -0.33 (0.37) Cram schooling Hours 0.40 ** (0.14) Hours × Hours -0.01 (0.01) Expenditures 0.71 ** (0.18) Expenditures × Expenditures -0.03 * (0.11) After-school tutorial classes -2.22 ** (0.76) After-school programs outside 1.23 # (0.72) school(private cram schools) After-school programs outside 0.74 (1.21) school(cost-free) N 617 582 579 578 575 R square 0.08 0.20 0.22 0.23 0.22 # p < .10 * p < .05 ** p < .01

This study included expenditures in cram schooling and the quadratic term of expenditures in Model 4 to explore the relationship between academic achievement and expenditures of cram schooling. The result indicated that higher expenditures of cram schooling resulted in better academic performance (B = 0.71, SE = 0.18, p < .05). In the meantime, the impact made by the quadratic term of expenditures on academic achievement showed an apparent negative value. This further indicated that the positive impact of expenditures on academic achievement showed diminishing marginal utility. In other words, spending too much tuition on cram schooling was negatively correlated with academic achievement. With expenditures as a factor, the disadvantage of aboriginal students became less obvious (the B value of Model 4 dropped from -1.98 to -0.69) by 65.15%. This finding showed that the poorer performance of aboriginal students might be caused by their families’ economic condition. The explanatory power (R square) of Model 4 reached 0.23, an increase of 15% over Model 2.

This study further analyzed how different cram schooling types affected student academic performance, as shown in Model 5. The tutorial classes organized by the schools had a negative effect on academic achievement. Although the government invested sufficient funds into the program that allowed students from lower-income families not to pay tuition, the effect was less than satisfactory. On the other hand, students who chose private cram schools often scored higher in their studies. The positive impact on such students reached a remarkable result of nearly .05 (B = 1.23, SE = 0.72, p = .06). Students who participated in tutorial classes set up by non-profit organizations did not show improvement in academic achievement. The explanatory power (R square) of Model 5 reached 0.22, an increase of 10% over that in Model 2.

Mediators Table 4 showed that the aboriginal students spent less tuition and had fewer hours spent on cram schooling than their non-minority counterparts (B = -4.94, SE = 0.83, B = -2.01, SE = 0.51). Aboriginal families provided apparently fewer resources for learning, which affected their children’s academic achievement. Meanwhile, the cross-national marriage couples provided a similar level of resources for their children’s after-class education compared with non-minority families. Table 5 showed that the aboriginal students had a higher percentage of choosing after-school tutorial classes (B = 1.44, odds ratio = 4.23, p < .05) or tutorial classes provided by non-profit organizations (B = 2.27, odds ratio = 9.67, p < .05) over private cram schools (B = -2.21, odds ratio = 0.13, p < .05). By compiling the analytical results of Tables 3 to 5, it was concluded that the less satisfactory academic achievement of the aboriginal students was caused by choosing to attend after-school tutorial classes organized by the schools rather than attending private cram schools. However, their choice of free tutorial classes offered by non-profit organizations seemed not to have affected their academic achievement (see Table 3).

Table 4 Estimates on the achievement score for 5th grade students Hours Expenditures B SE B SE Ethnicity Non-minorities (reference categories) Minorities (aboriginal) -4.94 ** (0.83) -2.01 ** (0.51) Minorities (cross-national -1.48 (1.40) -0.86 (0.88) marriage) Father educational level -0.01 (0.14) 0.21 * (0.08) Mother educational level -0.04 (0.14) 0.18 * (0.09) Father’s occupation Professional and semi-professional (reference categories) Trading and service -0.89 (0.76) -0.33 (0.47) Labor-related work and low-tech -1.27 (0.70) -0.47 (0.43) jobs Mother’s occupation Professional and semi-professional (reference categories) Trading and service 0.56 (0.73) 0.36 (0.45) Labor-related work and low-tech -1.49 * (0.68) -0.43 (0.42) jobs Family economic status 0.50 (0.41) 0.62 * (0.25) Family structure Two-parent household -0.03 (0.75) 0.47 (0.46) Single-parent and other household (reference categories) Boy (reference categories: Girl) -0.18 (0.53) -0.25 (0.33) Sibling size -0.78 * (0.32) -0.50 * (0.20) N 595 594 R square 0.16 0.25 # p < .10 * p < .05 ** p < .01

Table 5 Estimates on the achievement score for 5th grade students After-school After-class After-class tutorial classes programs programs outside school outside school (private cram (cost-free) schools) B Odds B Odds B Odds Ratio Ratio Ratio Ethnicity Non-minorities (Reference categories) Minorities (aboriginal) 1.44 ** (4.23) -2.21 ** (0.13) 2.27 ** (9.67) 0.88 (2.41) -0.48 (0.62) 0.22 (1.25) Minorities(Cross-natio nal marriage) Father educational level -0.13 * (0.88) 0.02 (1.02) -0.13 (0.88) Mother educational level 0.10 # (1.11) 0.00 (1.00) 0.15 (1.16) Father’s occupation Professional and Semi-professional (Reference categories) Trading and service -0.21 (0.81) 0.07 (1.07) -0.34 (0.71) Worker and other 0.29 (1.34) -0.15 (0.86) -0.16 (0.85) Mother’s occupation Professional and Semi-professional (Reference categories) Trading and service 0.53 # (1.70) -0.14 (0.87) 0.42 (1.52) Worker and other 0.18 (1.20) -0.48 # (0.62) 0.35 (1.42) Family finance situation -0.35 * (0.70) 0.54 ** (1.71) -0.48 * (0.62) Family structure Two-parent household -0.05 (0.95) 0.18 (1.19) -0.11 (0.89) Single-parent and other household (Reference categories) Boy (Reference 0.32 (1.37) -0.04 (0.96) 0.25 (1.29) categories: Girl) Sibling size 0.32 ** (1.37) -0.29 * (0.75) 0.23 (1.25) N 591 591 591 Cox & Snell R Square 0.16 0.22 0.11 Nagelkerke R Square 0.23 0.30 0.27 # p < .10 * p < .05 ** p < .01

Conclusion and Discussion Over the years, many researchers have made an effort to explore the relationships between after school learning activities (such as private tutoring, cram schooling and so forth) and students’ academic achievement (Bray et al. 2014; Kim and Park 2010; Shih and Yi 2014). Students in East Asia countries have long stressed cram schooling, and they consider it as an effective way to enhance their ability to pass national entry examinations and achieve the schooling successfully (Fung 2003; Kuan 2011; Liu 2012). However, the relationships between diverse cram schooling (including time, expenditures, and types of participation) and students’ academic achievement are still not clear and insufficient. Therefore, this topic has a highly referential value for educational policies, instructional affairs, and the education system, where credentials and educational background still have an immense effect on an individual’s career in society.

This study discovered that aboriginal students had less satisfactory academic achievement than non-minority students; yet the learning performance of children from cross-national marriage families was similar to that of the non-minority students. This indicated that the main division of achievement only existed between aboriginal students and non-minority students. Secondly, this study discovered that the number of hours spent on cram schooling was positively correlated with academic performance. On the other hand, although expenditures showed diminishing marginal utility, it still had a positive link with academic achievement. This indicated that, for Taiwanese students, cram schooling is a crucial factor in academic achievement, thereby supporting the previous research results (Chen and Hwang 2011; Dang 2007; Kuan 2011; Kim and Park 2010; Liu 2012). Moreover, the results showed that the less satisfactory academic achievement of aboriginal students was due to the scarcity of additional cram schooling provided by their families after school. This result reflected that gaps in learning outcomes between ethnic or racial groups are associated with the abundance of family learning resources and poor educational environment (Brown-Jeffy 2009; Byun and Park 2012; Lin and Hwang 2009; Rowley and Wright 2011).

There has been relatively little research in the link between after-class tutorial classes and academic performance, especially in regards to what type of cram schooling and organizations can achieve better academic performance, what kind of cram schools ethnic groups will choose, and what kind of division exists in student achievement. This study found that tutorial classes organized by schools had a negative impact on students, and that the students attending these classes were mostly from aboriginal groups. This contributed to the lagging behind of aboriginal students. The authorities concerned have been putting great effort and funding into after-school tutorial classes that held in school, as it is felt that they are a great help to minority students in their learning performance. However, this study found that, the government’s well-intended policy is not providing the desired effect (possibly due to a poorly-organized teaching plan and poorly trained teaching staff). This poses a concern for how the Taiwan

government allocates educational resources, and it is advised to take immediate action and revise its policies.

Meanwhile, this study found that private cram schools, which generally demand higher tuition fees than schools and non-profit tutorial classes, can be a great help to students’ academic achievement. Other than the remote areas, cram schools can be found in almost every city and town in Taiwan. The business of a cram school depends on how its students perform academically. Thus, many parents are eager to pay higher tuition fees as long as their children make improvements in their academic studies (Lin and Hwang 2009; Lin et al. 2015). To attract renowned teachers, cram schools have to be set up in urban areas. Aboriginal students from remote areas find it difficult to commute to these cram schools, and their families face the challenge of higher extra tuition fees. The only choice left to them is to choose after-school tutorial classes, yet such tutorial classes are of little help to diminish the learning gap.

In recent years, a large number of enterprises and non-profit organizations in Taiwan have tried to bridge the gap with less-privileged families by providing considerable funds and manpower to hire college students to teach aboriginal students in remote areas or through online classes. The results of these actions have also proved less satisfactory than expected. As Table 5 shows, aboriginal students have a higher percentage of choosing tutorial programs that are free of charge. Therefore, it is important to enhance the operations of such free tutorial classes so that aboriginal students can have the opportunity to compete with their non-minority peers.

The above findings also showed that children from cross-national marriage families have similar academic performance compared with other non-minority peers, and that the tuition and hours they spend on cram schooling are similar to their counterparts. They may be less likely to attend off-campus cram schools, and they tend to participate in on-campus after-school classes. Many mothers in cross-national marriage families come from mainland China and Southeast Asia, and they often face the problems of obtaining an ID card and finding a job, not to mention language difficulties and culture conflict. The above all pose issues for their children. This study used fifth graders as the research samples, and found that their learning was inhibited considerably.

Although our analytical approach provided new insights regarding the association between cram schooling participation and students’ academic achievement gap of ethnic and cross-national marriage, there are several limitations that need to be considered and possibly addressed in the future research. Firstly, multi indicators of cram schooling were selected as a mediator in this study, and it did not examine other mediators that could affect a student’s academic achievement or the indirect impact of cram schooling on different academic groups. This study also did not discuss the relationship between student age and academic performance in different educational stages.

Secondly, other questions left to discuss include: What are the effects of cram schooling on student’s learning adaptation in the long term? Will it affect a student’s career or income (Lin and Hwang 2009)? Does any difference exist about the relative findings between Eastern and Western societies? Additionally, the long hours spent on a cram schooling cause higher pressure and less time for leisure activities. This might lead to a lower sense of happiness or maladjustment at school (Chen & Lu 2009; Csikszentmihalyi and Hunter 2003). Hence it is important to pay attention to the negative effects brought forth by cram schooling.

Lastly, a striking discovery of this study was that, after excluding socioeconomic factors and cram schooling, students from cross-national marriage families often performed better than non-minority students. This study, however, did not investigate a deeper understanding of this phenomenon. The above limitations are all topics that are worth further exploration in future research to create better strategies and educational policies.

Acknowledgement This research was sponsored in part by the Ministry of Science and Technology, Taiwan, under Grant MOST 103-2410-H-259-031-SS2

References

Brown-Jeffy, S. (2009). School effects: Examining the race gap in mathematics achievement. Journal of Africa- American Study, 13, 388-405. Bray, M. (2013). Shadow education: Comparative perspectives on the expansion and implications of private supplementary. Procedia-Social and Behavioral Sciences, 77, 412-420. Bray, M., Zhan, S., Lykins, C., Wang, D., & Kwo, O. (2014). Differentiated demand for private supplementary tutoring: Patterns and implications in Hong Kong secondary education. Economics of Education Review, 38, 24-37. Byun, S. Y., & Park, H. (2012). The academic success of East Asian American youth: the role of shadow education. Sociology of Education, 85, 40-60. Chen, C. W., & Hwang, Y. J. (2011). A re-exploration of stratification and efficacy in cram schooling: An extension of the Wisconsin model. Bulletin of Educational Research, 57(1), 101-135 (In Chinese) Chen, S. Y., & Lu, L. (2009). After-school time use in Taiwan: Effects on educational achievement and well-being. Adolescence, 44, 891-909. Csikszentmihalyi, M., & Hunter, J. (2003). Happiness in everyday life: The uses of experience sampling. Journal of Happiness Studies, 4, 185-199. Dang, H. A. (2007). The determinants and impact of private tutoring classes in Vietnam. Economics of Education Review, 26, 683-698. Fung, A. (2003). Cram schooling in Hong Kong: The privatization of public education. Asian Anthropology, 2, 179-195. Jung, J. H., & Lee, K. H. (2010). The determinants of private tutoring participation and attendant expenditures in Korea. Asian Pacific Education Review, 11, 159-168. Kenayathulla, H. B. (2013). Household expenditures on private tutoring: Emerging evidence from Malaysia. Asia Pacific Education Review, 14, 629-644. Kim, J. H., & Park, D. (2010). The determinants of demand for private tutoring in South Korea. Asia Pacific Education Review, 11, 411-421. Kuan, P. Y. (2011). Effects of cram schooling on mathematics performance: Evidence

from junior high students in Taiwan. Comparative Education Review, 55, 342-369. Lee, S., & Shouse, R. C. (2011). The impact of prestige orientation on shadow education in South Korea. Sociology of Education, 84, 212-224. Lin, C. C., & Ou, I. C.(2010). The implement condition of Night Angel Illumination Program. School Administrators Research Association, 67, 122-149. (in Chinese) Lin, C. Y., Hsieh, Y. H., & Chen, C. H. (2015). Use of latent growth curve modeling for assessing the effects of summer and after-school learning on adolescent students’ achievement gap. Asia Pacific Education Review, 16, 49-61. Lin, H. M., & Hwang, Y. J. (2009). The study on relationship among the Aborigines, Hans, cram schooling and academic achievement. The example of the eighth graders in Taitung. Contemporary Educational Research Quarterly, 17(3), 41-81 (in Chinese). Liu, J. (2012). Does cram schooling matter? Who goes to cram schools? Evidence from Taiwan. International Journal of Educational Development, 32, 46-52. Ministry of Education (2009). Night Angel Illumination Program. Retrieved August 2, 2014, from http://english.moe.gov.tw/ct.asp?xItem=14748&ctNode=11435&mp=1. Rhy, D., & Kang, C. (2013). Do private tutoring expenditures raise academic performance? Evidence from middle school students in South Korea. Asian Economic Journal, 27, 59-83. Rowley, R., & Wright, D. W. (2011). No “White” child left behind: The academic achievement gap between black and white students. The Journal of Negro Education, 80, 93-107. Shih, Y. P., & Yi, C. C. (2014). Cultivating the difference: social class, parental values, cultural capital and children’s after-school activities in Taiwan. Journal of Comparative Family Studies, 45, 55-75. Stevenson, D. L., & Baker, D. P. (1992). Shadow education and allocation in formal schooling: transition to university in Japan. American Journal of Sociology, 97, 1639-1657. Whitley, J., Rawana, E., & Brownlee, K. (2014). A comparison of aboriginal and non-aboriginal students on the inter-related dimensions of self-concept, strengths and achievement. Brock Education, 23(2), 24-46. Zhang, Y. (2013). Does private tutoring improve students’ national college entrance exam performance? A case study from Jinan, China. Economics of Education Review, 32, 1-28.

International Journal of Learning, Teaching and Educational Research Vol. 16, No. 1, pp. 103-119, January 2017

Teachers‟ Self-Efficacy at Maintaining Order and Discipline in Technology-Rich Classrooms with Relation to Strain Factors

Eyvind Elstad University of Oslo, Norway

Knut-Andreas Christophersen University of Oslo, Norway

Abstract. Teachers‟ operational control is challenged by pupils‟ access to computers with internet in the classroom. The teacher‟s control of operations will be diminished in a technology-rich classroom. Professional growth is conditioned by teachers‟ experience of mastery in their work. Therefore, it is interesting to study strain factors that are associated with teacher efficacy at maintaining order and discipline in technology-rich classrooms. This study splices two theoretical lenses: theory on teacher efficacy and strain factors and theory of teachers‟ learning orientation. The purpose of this article is to explore the factors that are associated with teacher efficacy at maintaining order and discipline in technology-rich classrooms. The data set comes from a survey of 156 teachers in Norway. The analysis shows that the perception of conflict is negatively associated with teacher efficacy. Stress and perception of shortcomings in the ability of the teacher to influence the pupils is associated in a slightly weaker manner associated with teacher efficacy for classroom management. There is a strong, positive relationship between stress and conflict, and there are positive associations between teachers‟ sense of shortcoming and both stress and conflict. Teachers‟ emphasis on practical knowledge rather than expert recommendations is positively associated with shortcomings, stress and conflict. Implications for practice are discussed.

Keywords: self-efficacy; classroom management; strain; technology-rich classrooms; ICT; teachers‟ work.

Introduction The Norwegian educational authorities have defined “using digital tools” as “a core skill” that is to be stimulated by schools (Ministry of Education and Research, 2014). Computers are used in school as writing tools, for the gathering of information and for communication, and students are expected to make

academic use of the Internet. Norwegian sixth-form colleges have thereby seen an equipment revolution in terms of accessibility to computers in the classroom (Elstad, 2016a). Each pupil has his own computer with internet access and some school districts expect these machines to be used often.

Off-task activity on the part of pupils during lesson time is a recurrent problem (Elstad, 2006; Blikstad-Balas, 2012) and represents a frequent challenge to teachers, whose work is often in the form of the teacher going through new material on the blackboard or guiding pupils in their individual or group-based studies. The physical design of the classroom includes the classroom artefacts. In the traditional classroom, the arrangement of the desks is physically designed to allow monitoring of pupils‟ academic work. Desks are generally laid out in rows in order that each pupil can see the blackboard. This physical layout creates a challenge in terms of the teacher's ability to see whether the pupils are using their computers for the purpose of academic work or off-task behaviour. In the technology-rich classroom, classroom control of this kind is harder to maintain because it is more difficult to monitor the pupils‟ behaviour. Classroom management is “the actions teachers take to create an environment that supports and facilitates both academic and social-emotional learning” (Evertson & Weinstein, 2011: 4). It is interesting to explore the potential managerial challenges that teachers confront in technology-rich classrooms. Professional growth for teachers is an important ambition in Norwegian educational policy (White Paper no. 11, 2008) and teacher efficacy in terms of classroom management is important if the teacher is to increase skills and professional growth in their work. A teacher's first experience of teaching can be a challenge: situations are often complex and the teacher needs to take account of several, often unforeseen, things happening at once. This can easily create stress and working-memory overload among inexperienced teachers when decisions need to be taken (Leinhardt & Greeno, 1986). Through experience, teachers learn to tackle complex decision-taking situations in a better way: the skill becomes procedural (Anderson, 1982). In this way, the cognitive overload experienced by beginners becomes reduced. After the initial phase in which the goal is survival, teachers attempt to consolidate their pedagogic repertoire and experiment with ways of carrying out their roles in ways give them mastery over new challenges (Berliner, 1986; Huberman, 1989; Chi et al., 2014). Several studies document that professional growth among teachers – allowing for certain nuances – is related to improved pupil performance (Rivkin et al., 2005; Rockoff, 2004).

Teacher efficacy is conceptualized as a teacher‟s judgment of one‟s capabilities to plan and enact specific tasks (Bandura, 1994). The study of teacher efficacy in various teaching-related aspects shows a surprising curvilinear pattern: expectations of mastery increase with experience but can sink again after many years of teaching experience (Klassen & Chui, 2010). The same pattern is observed in studies of the teacher's contribution to pupil learning performance: performance has a positive association with the number of years of teaching experience; (Rockoff, 2004) but learning improvement reaches a plateau after a few years' experience or even sinks slightly after 5-6 years of practice (Rivkin et al., 2005).

A premise of our study is that a teacher's experience of mastery of the job is a condition for professional growth (Tschannen-Moran, Hoy & Woolfolk, 2007). Too low a feeling of mastery can be unfavourable for a future professional career. It is therefore interesting to study which factors we presume will have a connection with teacher efficacy. The purpose of this article is to investigate strain factors statistically associated with teacher efficacy in maintaining order and discipline in a situation in which the pupils each have a computer and internet access. The purpose is also to investigate the relationships between all the factors that form part of the theoretical framework. We will therefore investigate how a disparity in teacher efficacy for the maintenance of order and discipline corresponds with a disparity in how these teachers perceive (i) opportunities to influence pupils in the 15-19 age range (in the Norwegian school system, these pupils have a say in a number of significant matters), (ii) teachers‟ self-perception of experience as a starting point for their own professional development and their experience of (iii) stress and (iv) conflict when they are to lead learning work in their own classes.

Theoretical assumptions Two theoretical lenses are brought together in this study: (1) Bandura's self- efficacy theory (1994) and (2) the theory of “teacher orientation to learning” (Opfer, Pedder & Lavicza, 2011: 5). A self-efficacy belief is an assessment of a person‟s capabilities to attain a desired level of performance in a given endeavour. Bandura assumed that belief in one's abilities was a powerful driving mechanism influencing motivation to act, the effort put forth in the endeavour, and the persistence of coping mechanisms in the face of setbacks. The other theory concerns teacher orientation to learning which is integrated set of attitudes, beliefs and practices as well as the alignment of oneself and one‟s ideas to circumstances and context. These learning orientations are context dependent. A part of teachers‟ orientations to learning remain unchanged overtime, while the context, the phase of teacher experience and the pupils a teacher has influence the orientation to what, how and why they learn as professionals.

Zee and Koomen (2016) present a recent review of teacher efficacy studies. They integrated four decades of teacher efficacy research to explore the consequences of teacher efficacy for the quality of classroom processes, students‟ academic adjustment, and teachers‟ psychological well-being. Their results suggest that teacher efficacy shows positive links with students‟ academic adjustment, patterns of teacher behaviour and practices related to classroom quality, and factors underlying teachers‟ psychological well-being, including personal accomplishment, job satisfaction, and commitment. Negative associations were found between teacher efficacy and burnout factors. However, they do not explicitly focus on teacher efficacy at maintaining order and discipline in technology-rich classrooms.

A new kind of managerial challenge has appeared with the advent of free Internet access in the classroom: motivational conflicts between immediate rewards of electronic chatting, surfing and games and the long-term rewards of academic achievement (Elstad, 2008). Van Acker et al. (2013) explored the role of

teacher efficacy, subjective norm and attitude in technology-rich situations. They found that teachers‟ ICT skill was the strongest predictor of teacher efficacy. Subjective norm played only a limited role in the intention to use the educational use of digital learning materials. Basing on the outcome of this study, the authors regard persuasive communication focusing on positive outcomes and skills based training seem appropriate interventions to promote a positive attitude towards the educational use of digital learning materials and improve teacher efficacy in using digital learning materials. However, these scholars do not focus on teacher efficacy at maintaining order and discipline. Quite few researchers focus explicitly on teacher efficacy at maintaining order and discipline (Evertson & Weinstein, 2011:3). This is especially true for classroom management and technology: Bolick and Cooper (2011) and Bolick and Bartels (2015) address how little research exists on how 1:1 computers affects classroom management. In this article we have attempted to fill this gap in the literature.

We argue that educators and policymakers need to understand better teachers‟ challenges of classroom management in technology-rich classrooms. In this article we assume that factors such as teachers' general perception of the limitations of what can be achieved through teaching, as well as stress and conflict in connection with the high availability of technology in the classroom can determine their self-efficacy in highly technological learning environments. Self-efficacy is primarily influenced by experiences of mastery1 (Bandura, 1994). Skaalvik and Skaalvik (2007) identify strain factors statistically associated with self-efficacy among primary and lower secondary teachers. Burnout is among the factors they find to be strongly associated with teacher efficacy, while teachers' general perceptions of the limitations on what can be achieved through teaching are indirectly associated with teacher efficacy. In other words, the study provides confirmation that issues perceived as stressful help account for teacher efficacy. Teacher stress originates in a conflict between pupil interests and the teacher's need for control over pupils' academic activities in the classroom. We connect experiences both of stress and conflict to teacher efficacy in classroom management (hypothesis no. 1 and no. 2).

We anticipate a range of opinions regarding how teachers perceive, for instance, conflict with pupils and personal stress when enforcing the rules for acceptable use of technology. We will therefore also investigate the range of teacher efficacy in terms of classroom management. In addition, our assumptions suggest that a perception of inadequacy in this respect is negatively associated with teacher efficacy in maintaining order and discipline (hypothesis no. 3).

There is relatively limited theoretical knowledge about how teachers should tackle the non-academic use of PCs. However, it is interesting to contrast the significance of practical skills for teachers' perception of expert knowledge and to investigate connections between this and teacher efficacy in relation to maintaining order and discipline (hypothesis 4).

1 According to Bandura (1997), expectation of mastery is based on five types of information source, one of which is personal experience.

The high degree of access to technology in Norwegian schools provides opportunities both for the use of adequate technology for the purposes of learning (for instance investigating what takes place “within” an electrical circuit, Zacharia & de Jong, 2014), but at the same time, various kinds of challenges such as the non-academic use of technology in the classroom (Blikstad-Balas, 2012). We assume that some people will experience this as a source of stress that can influence teacher efficacy. Situations may also arise in which some pupils cast doubt on the rights of the teacher to limit, for instance, off-task pupil behaviour. In other words, teachers may experience both stress and conflict-laden situations when they are dealing with rules, but also in situations in which technology is used to promote learning: it is a stated political goal that teachers must be able to teach using digital tools and integrate digital skills as one of the five so-called core skills (Ministry of Education and Research, 2014). For this reason we assume that both skills and conflicts have a positive relation with an experience of shortcoming (named „con‟) in terms of influence over pupils (investigative assumption nos. 1 and 2). Most sixth-form pupils are in the 15-19 age range, in which pupils as they grow older will claim rights to self-determination and personal positioning (Koepke & Denissen, 2012). The rationale behind the teacher's work is to exercise a positive influence on the pupil's learning process (the teacher has in this respect a clearly-defined responsibility for pupil learning outcomes; White Paper no. 11, 2008), but pupil engagement in learning processes will nevertheless depend on their motivation and self-discipline when working in technology-rich learning environments. The result depends on input from several parties. In other words, the teacher's task of influencing pupils is dependent on their self-discipline.

Teachers are expected to be concerned with their own professional development. Since the 1980s, research has focussed on the significance of teachers' epistemic beliefs upon various aspects of teaching work (e.g. Fang, 1996; Chinn, Buckland, & Samarapungavan, 2011). Several studies have provided empirical support for suggesting a connection between teachers' personal epistemology and their actual actions in the classroom (e.g. Brown & Rose, 1995), but this research field is still immature (e.g. Greene & Seung, 2014). A theory developed by Opfer, Pedder and Lavicza (2011) indicates how teachers' approach to learning (including learning of practices) influences their skills development (“learning change”). This theory joins a succession of theoretical contributions emphasising practical skills alongside theoretical knowledge as fundamental for personal teaching development. All professional practitioners employ theory in the exercise of their profession. Theoretical development may be experience-based (and thus “weak”), but may also be based on, for instance, research (and thus “strong”). The emphasis can be contextual. When teachers need to master new classroom challenges they can look to both experience and theory as sources from which to learn how to tackle the situation. Norway has a deliberate policy of stimulating teacher networks and developing practical skills. It is therefore interesting to investigate how teachers use their own experiences and those of other teachers to develop their own classroom management in technology-rich classrooms, in other words, to develop practical skills.

Research about teachers' sense of their shortcomings and their development of practical skills has – so far as we can see – not come very far. If the development of practical skills is effective for professional development, it is reasonable to expect a negative association between practical skills and perceptions of personal shortcoming. If practical skills do not contribute to effective professional development in challenge solving, we can expect a positive association between shortcomings in terms of influence over pupils and practical skills (investigative assumption no. 3). We can make corresponding arguments in terms of the connection between practical skills and stress (investigative assumption no. 4).

IA4

Figure 1 Theoretical model of hypotheses (abbreviated H) and investigative assumptions (abbreviated IA).

Method In order to investigate our hypotheses and investigative assumptions, we carried out a survey among 156 teachers at 3 Norwegian sixth-form colleges. The data collection took place in the form of a paper-based questionnaire filled in by the teachers in connection with a planning day. The advantage of this method is that all the teachers were required to take part in the planning day. No teachers declined to take part in the survey. This means that apart from those who were off sick on the day in question, the selection we have analysed can be regarded as complete. In other words the selection has no self-selection issues.2 The three schools in question, however, cannot be regarded as representative of Norwegian sixth-form colleges and their teachers. All three, which offer the curriculum directed towards preparation for higher education, are regarded as

2 Seven sixth-form colleges were originally included in the selection but the participation percent was too low at four of these schools. In order to avoid self-selection we have removed these four schools from the material to be analysed.

fairly similar. Pupils' average grades on entry were 4.25 [on a scale of 1-6] and the value-added measures of the three schools are relatively similar (averaging - 0.35) (Falch et al., 2016). In other words, the schools can be regarded as fairly average in terms of pupil attainment on entry and pupil progress during their time at the school (pupil entry statistics and school contribution indicators based on year group 2008-2009). This aspect of the selection – as we will describe later – is of significance in relation to the conclusions that can be drawn.

The purpose of the investigation was described at the beginning of the session in which the questionnaires were distributed in plenum and the teachers were given sufficient time to consider the alternatives while filling out the questionnaire. We thus believe that the answers are relatively well considered. In all three schools, the pupils each have a PC with internet access available and it is up to the individual teachers to tackle the challenges involved in terms of classroom management. To remind the teachers about this challenge we introduced the questionnaire with the following text:

The government's strategy document The teacher promotion states that 'Teachers must be able to teach using digital tools and integrate digital skills as one of the five core skills‟. Consider that someone has decided that pupils are each to have their own computer and access to the internet during the lessons that you are teaching.

After this text, the teachers were asked to indicate on a seven-point scale (with named extremes) which alternative corresponded best to their own views. The teachers were asked to indicate values for the following questions or statements:

Teacher efficacy classroom management (secm)

How certain are you that in such a situation you would be able to:

 maintain discipline in classes of over 25 pupils.  persuade even the most games-addicted pupils to concentrate on academic tasks.  persuade pupils who often switch between different social media to follow classroom rules.

These questions on teacher efficacy are inspired by Skaalvik and Skaalvik's (2007) “Maintain discipline” construct, but here adapted to the situation of technology- rich learning environments. The extremes of the scales regarding teacher efficacy were: 1= Not sure at all, 7= Absolutely certain. After these questions regarding teacher efficacy, teachers were asked to consider the following statements (the extremes of the scales were: 1=completely disagree, 7=completely agree):

Shortcomings in terms of influence over pupils (con)3

 How much pupils learn at school through using ICT is primarily determined by their self-discipline.

3 Items are inspired by, but not identical to, those that Skaalvik & Skaalvik (2007) term their “External Control Scale”.

 It is difficult for a teacher to improve pupil performance by using ICT unless the pupil can be bothered working on the tasks.

Stress in technology-rich classrooms (str1)

 Allowing pupils free access to the internet during lessons creates an extremely stressful situation for me.  When pupils have the freedom to do as they wish on the internet, I lose control.

Where technology creates conflict situations (str2)4

 Many conflicts with pupils arise when pupils are allowed to use the internet during lessons.  Some pupils are unable to maintain focus on academic tasks when they use the internet during lessons.

Teachers' practical skills development as a source of improvement (sop)5

 In terms of developing my own teaching, I trust my own practice evaluations more than what the experts say.  In terms of what teaching techniques work well, I have greater trust in experienced teachers than in experts.

Analysis Structural equation modelling (SEM) and descriptive statistical analysis were used to analyse the relationships between the variables. Structural equation modelling is suitable for confirmatory factor analysis and path analysis. Assessments of fit between model and data are based on the following indices: root mean square error of approximation (RMSEA), Tucker-Lewis index (TLI), goodness-of-fit index (GFI) and comparative fit index (CFI). RMSEA < .05 and TLI, GFI and CFI > .95 indicate good fit and RMSEA < .08 and TLI, GFI and CFI > .90 indicate acceptable fit (Kline 2005). The measurement and structural models were estimated with IBM SPSS Amos 22. The values of RMSEA, TLI, GFI and CFI indicate that the structural model in Figure 2 has acceptable fit.

4 Items are inspired by, but not identical to, those that Skaalvik & Skaalvik (2007) term “Strain Factors”. 5 Items are inspired by, but not identical to, measurements of teachers' “internal” attitudes to learning, developed by (Opfer, Pedder & Lavicza, 2011).

Results

Tabel 1: Mean, standard deviation, skewness, kurtosis, and Cronbach’s alpha. N = 156 Indicators Mean SD Skewness Kurtosis Alpha w2 3.87 1.80 -0.08 -1.00 secm w3 3.12 1.52 0.22 -0.98 0.831 w4 3.44 1.57 0.10 -0.83 w33 5.43 1.28 -0.85 0.61 con 0.710 w34 5.53 1.33 -0.53 -0.69 w37 3.84 1.90 0.14 -1.06 str1 0.753 w39 3.60 1.78 0.25 -0.86 w41 3.50 1.68 0.28 -0.76 str2 0.876 w43 3.46 1.64 0.24 -0.77 w103 4.76 1.45 -0.35 -0.31 sop 0.801 w104 5.24 1.38 -0.76 0.28

Figure 2: Structural equation modelling by using AMOS 22. Constructs and abbreviations: Self-efficacy for classroom management (secm), external control (con), stress (str1), conflicts induced by ICT in classrooms (str2), and teachers’ practical knowledge epistemology as a source for improvement (sop).

Table 2

Hypothesis Formulation Result 1 Stress has a negative association with teacher efficacy for The association is negative classroom management. (b(str1→secm)= -.09), but relatively weak (not significant). 2 Conflict has a negative association with teacher efficacy The negative association for classroom management. (b(str2→secm)= -.23) is significant. Hypothesis supported. 3 Perception of shortcoming is negatively associated with The association is positive but teacher efficacy for classroom management not significant (b(str2→secm)= -.16). Hypothesis not supported. 4 Practical skills re positively associated with teacher Practical skills development is efficacy for classroom management. not associated with an expectation of mastery in classroom management: (b(sop→secm)=.00)

In respect of the investigative assumptions, we find the following:

Table 3

Formulation Result 1 We investigate the relationship between conflicts and The association (b(con→str2)= perceptions of shortcoming in terms of influence over -.04) is very weak. pupil motivation and self-discipline 2 We investigate the relationship between the significance The association of practical skills and shortcoming. (b(sop→con)=.13) is positive. 3 We investigate the relationship between the significance The association of practical skills and stress. (b(sop→str1)=.29) is positive. 4 We investigate the relationship between stress and The positive association perception of shortcoming in respect of influence over (b(str1→con)=.35) is pupil motivation and self-discipline significant.

Discussion The purpose of the article was to investigate strain factors and the significance of teachers' practical skills development as a source of improvement statistically associated with teachers' self-efficacy for succeeding in classroom management, as well as reciprocal associations between the factors. The analysis indicates that a perception of conflict (str2) is negatively associated with expectations of mastering classroom management (secm). Stress (str1) and a perception of shortcoming in terms of influencing pupils (con) have a rather weaker negative association with teacher efficacy for classroom management (secm). There is a strong, positive association between stress (str1) and conflict (str2), as well as positive associations between shortcoming (con) and both stress (str1) and

conflict (str2). Teachers' emphasis on practical skills in preference to expert advice (sop) is positively associated with shortcoming (con), stress (str1).

If these path coefficients reflect causal processes, the causal direction6 may be in several directions. Since this cross-section investigation is unable to prove causal direction, we must be cautious not to draw too strong conclusions about causality. Research incorporating sequential analysis or controlled experiments may help resolve questions about causal direction, but such research approaches are demanding to carry out. We also note the positive association between shortcoming in terms of influence on pupil motivation and self-discipline and perceptions of stress and conflict. Here too, the causal direction is uncertain even though it seems most plausible to assume that perceptions of stress and conflict cause the perception of shortcoming.

There are sometimes significant differences of opinion regarding the use of technology in schools, both amongst teachers, so-called experts, politicians and policymakers. This article focusses on teacher perceptions. If politicians and policymakers are to succeed with their leadership signals regarding improving pupil learning and using ICT, it is important to understand the opportunities and challenges and to respect the work done by teachers in present-day sixth- form colleges.

Relatively little research is to be found concerning the work situation of teachers in sixth-form colleges and teachers' perceptions, beliefs, preferences and managerial challenges. One reason may be that “classroom management is sometimes equated with a mechanistic, authoritarian orientation that minimizes the importance of positive interpersonal relationships and maximizes control and compliance” (Evertson & Weinstein, 2011:4). However, managerial challenge is a topic of enduring concern for teachers. Some opinions expressed by teachers may suggest that a gap exists between perceptions of reality among teachers, particularly in sixth-form colleges, and those who are in a position to shape educational policy (Kval, 2014) 7 plus educational researchers (Elstad, 2005). For example, a teacher in a sixth-form college writes the following:

As a teacher in a sixth-form college I experience every single day the problem of digital gadgets. PCs and a variety of apps may be very helpful to strongly-motivated pupils, but for a large proportion of pupils they are a distraction that removes focus from subject-based learning. Pretty much all research confirms this. As a county-council employee, however, I have to be loyal to the resolutions passed by county

6 Strictly speaking, the analytical design does not provide a basis on which to determine causal direction; only the strength of statistical associations. 7 This is one example. One of the top civil servants in the Ministry of Education and Research, Ole Briseid, condemned some years ago teacher-centered instruction and emphasized progressive, student-oriented and activity-oriented teaching methods: ”The Ministry has wanted to reform the teaching methods in schools (in Norway) for a long time. … He (Briseid) wants more project work, less traditional teaching using the blackboard as a visual aid and more problem-based teaching. To a larger extent, pupils will work independently and in groups with topics they develop themselves. The teachers will be more like supervisors than lecturers, and computer technology forces the development of new teaching methods (Kluge 2001, emphasised here).

councillors and by the Norwegian Directorate of Education and Training. And they have decided that not just PCs but also the use of portable applications on mobiles and tablets should play a greater and greater role in teaching, in all subjects and at all levels. I have to accept this. I do not like this development, but during my 15 years in the school no-one has asked me (or my colleagues) whether we want this tremendous focus on digital tools (Sætre, 2016).

A task for future research is to determine the extent to which this type of opinion is widespread and explore how contextual factors influence teacher efficacy and performance in technology-rich environments. If there is a widespread perception of a gap between the managers and the managed, those who are responsible for political and administrative decisions should be concerned about how teachers can succeed in their work. For the sake of balance: opinions are also expressed at the other end of the scale. Some teachers, for instance, claim that “in the digital classroom, access to and use of tablets is a fantastic opportunity” (Ramo, 2016).

There are several limitations in our study. This type of analysis has limitations from a conceptual perspective and in terms of its methodological (cross-sectional) approach. We acknowledge these limitations and argue that they can serve as point of departure for future research.

Conclusions and implications for future research

The strong emphasis on the accessibility of technology in the school is controversial among politicians, teachers and researchers (Ministry of Education and Research, 2014). For more than three decades, political quarters have been expressing great expectations for the use of technology in schools (Elstad, 2016b). This belief has been dented in the wake of an OECD report that provides an empirical basis for stating that those countries that have invested in high availability of technology display disappointing results. The internet navigating skills of Norwegian pupils are actually poorer than those in countries with limited availability to technology (but in which they actually train in adequate navigating). Nor do computers improve pupil learning results (OECD, 2015). Hattie uses the term distraction politics to describe the faith in technology as a “magic bullet” in educational policy (Hattie, 2015:30). It is likely that views on educational technology in Norway are on the change. For more than three decades, researchers, policy makers and school professionals have all harboured great expectations towards the use of technology in schools. This belief has received a hard knock. School professionals and policy makers are seeking answers to the question of how schools ought to relate to challenges created by the use of technology in the school based on a firmer grasp on reality.

Our study documents empirical associations between teachers' perceived shortcoming, stress, conflict, and teacher efficacy in respect of classroom management. We note that the average for teacher efficacy for classroom management (3.48) is lower than the neutral mid-point of the scale (which is 4), which provides an indication of the challenges encountered by teachers in these schools when they are to master class management in technology-rich

surroundings. This should generate humility regarding teachers' work challenges. There is a fairly high standard deviation to the various items used to measure teacher efficacy (see Table 1). This reveals a considerable disparity in how teachers view this question. As mentioned earlier, the selection is taken from schools with a score of roughly the national average in terms of school added value indicator and that admit pupils with average grades from secondary school. Since we have found that the challenges of leading pupil groups in classrooms in which each pupil has a computer with full internet access are fairly different in schools whose pupils have high entry grades, we would not like to claim that measurements which show how teachers at such schools perceive mastery of teaching are representative of Norwegian sixth-form colleges in general. More research is needed here based on a selection of schools at which pupils have a generally high motivation for schooling and schools in which pupils have a low educational motivation in technology-rich learning environments. Our measurements strictly speaking are only valid for the three schools included in the selection, but we believe that our estimates provide an indication of how teachers perceive self-efficacy and similar at sixth-form colleges that are located in the mid-stream of pupil performance. The three schools had no previous history of ICT-supported teaching before the system of one PC per pupil was introduced beyond the fact that this system was a consequence of national and local management signals.

Indications exist that schools with a strong focus on ICT-supported teaching attract teachers with a particular interest in technologically-supported teaching (Hauge, 2016). The selection effect in terms of teachers' choice of workplace has been documented (Bonesrønning, Falch & Strøm, 2005) and as such our estimates are not valid for with a predominance of this type of teacher. In addition, this research should be followed up by studies relating to courses other than the pre-academic study programme. For instance it seems plausible that pupils on vocational programmes display other types of unwillingness than pupils on the pre-academic study programme. This may not least depend on the subject that the teachers are teaching. Our analysis explains some, but not all, of the variance in the dependent variables. It is a matter for future research to develop larger analytical data sets that will provide opportunities to analyse the significance of subjects, programme and different types of pupil groups.

The challenges of the technology-rich classroom create pressure on teachers when leading educational politicians state that: “good classroom management and very clear rules are required regarding how PCs are used in the school. It is up to the teacher and the school to ensure that these exist” (Nilsen, 2011). A natural conclusion from such statements is that anyone who does not succeed is quite simply not good enough. There is a high degree of non-academic activity in the lessons (Hatlevik et al., 2013). That the teacher has a clearly-defined responsibility for what the pupils learn at an age range at which many pupils feel that they themselves should be deciding what they should do during lesson times, creates pressure on teachers. If teacher efficacy for maintaining order and discipline is contextual, it is unhelpful when responsible politicians express themselves in this manner. For this reason we need more research about the significance of context.

We also need more research to provide normative conclusions about how teachers can succeed better with classroom management. An emphasis on practical experience and skills has a negative association with self-efficacy for teaching and a positive association with stress, conflict and a sense of shortcoming. Future research should investigate whether an emphasis on experience-based knowledge dissemination through teacher networks (Larsen & Nilsen, 2012) is useful for promoting teachers' professional development in technology-rich environments. However, the averages for items measuring teachers' belief in experience rather than expert advice as a source of improvement are fairly high (Table 1). A plausible interpretation is a low level of faith in research providing information on pedagogic implications. A broader set of terms and items is needed here in order to draw clear conclusions. Our survey indicates no statistical association between teachers' belief in experience as a source of improvement and their teacher efficacy for classroom management.

In another article we have attempted to deduce the implications for classroom management of what we regard as research-based knowledge about teaching and learning in technology-rich environments (Arnesen et al., 2016). There are grounds for saying that even subtle differences in the physical setting for interaction between teacher and pupils and the terms of the teacher's authority (computers with internet access for all pupils, tightening up of the final assessment principle and the removal of the absence limit in Norway are examples from the period 2007-2014) can have significance in relation to the opportunities a teacher have to exert influence. If teachers are to succeed in exerting an influence to promote pupil learning in technology-rich classrooms, the teachers need adequate tools for classroom management (Ifenthaler and Schweinbenz, 2013). There has been too little awareness of this matter (Elstad, 2016c). We need more knowledge about the significance of the management tools in one-to-one laptop environments.

Acknowledgement This work was supported by the Norwegian Research Council under Grant [number 218245]. Sources of support for the work, including sponsor names along with explanations of the role of those sources if any in study design; collection, analysis, and interpretation of data; writing of the report; the decision to submit the report for publication; or a statement declaring that the supporting source had no such involvement.

References Anderson, J. R. (1982). Acquisition of cognitive skill. Psychological review, 89(4), 369. Arnesen, T., Elstad, E. & Christophersen, K.A. (2016). Antecedents of pupils‟ self- regulatory strength in technology-filled school environments. International Journal of Learning, Teaching and Educational Research. 15 (3), pp. 218-241, March 2016. Retrieved from: http://ijlter.org/index.php/ijlter/article/view/457/pdf Bandura, A. (1994). Self-efficacy. New York: John Wiley and Sons. Berliner, D. C. (1986). In pursuit of the expert pedagogue. Educational Researcher, 15(7), 5- 13.

Blikstad-Balas, M. (2012). Digital Literacy in Upper Secondary School–What Do Students Use Their Laptops for During Teacher Instruction?. Nordic Journal of Digital Literacy, 7(02), 81-96. Bolick, C.M. & Bartels, J.T. (2015). Classroom Management and Technology. In E. Emmer, & E. Sabornie (eds.), Handbook of Classroom Management, New York: Routledge. (pp. 479-495). Bolick, C.M. and Cooper, J.M. (2011) Classroom Management and Technology. In Evertson, C. M., & Weinstein, C. S. (Eds.), Handbook of classroom management: Research, practice, and contemporary issues. New York: Routledge. (pp. 541.557). Bonesrønning, H., Falch, T., & Strøm, B. (2005). Teacher sorting, teacher quality, and student composition. European Economic Review, 49(2), 457-483. Brown, D.F., & Rose, T.J., (1995). Self-reported classroom impact of teachers' theories about learning and obstacles to implementation. Action in Teacher Education, 17 (1): 20-29. Chi, M. T., Glaser, R., & Farr, M. J. (2014). The nature of expertise. New York: Psychology Press. Chinn, C. A., Buckland, L. A., & Samarapungavan, A. L. A. (2011). Expanding the dimensions of epistemic cognition: Arguments from philosophy and psychology. Educational Psychologist, 46(3), 141-167. Elstad, E. (2005). Om prosesstyring og resultatorientering i utdanningspolitikken [On process steering and result orientation in educational policies]. Norsk Pedagogisk Tidsskrift, 89 (2), 132-145. Elstad, E. (2006). Understanding the nature of accountability failures in the technology- filled classroom: disaffected students and teachers who give in. Journal of Curriculum Studies, 38 (4), 459-481. Elstad, E. (2008). Building self-discipline to promote learning: students‟ volitional strategies to navigate the demands of schooling. Learning Inquiry, 2 (1), 53-71. Elstad, E. (2016a). Educational technology in schools. Policymaking and policy enactment. In Elstad, E. (ed.). Digital expectations and experiences in education (pp. 48-57). Rotterdam/ Boston /Taipei: Sense Publishers. Elstad, E. (2016b). Educational technology: expectations and Experiences. An introductory overview. In Elstad, E. (2016). Digital expectations and experiences in education (pp. 3-28). Rotterdam/ Boston /Taipei: Sense Publishers. Elstad, E. (2016c). Why is there a wedge between the promise of educational technology and the experiences of a technology-rich pioneer school? In Elstad, E. (2016). Digital expectations and experiences in education (pp. 77-96). Rotterdam/ Boston /Taipei: Sense Publishers. Evertson, C. M., & Weinstein, C. S. (2011). Classroom Management as a Field of Inquiry. In Evertson, C. M., & Weinstein, C. S. (Eds.). (2011). Handbook of classroom management: Research, practice, and contemporary issues (pp. 3-15). New York: Routledge. Falch, T., Bensnes, S. & Strøm, B. (2016). Skolekvalitet i videregående opplæring. Utarbeidelse av skolebidragsindikatorer og mål på skolekvalitet. [School Quality in upper secondary schools. Value Added Estimation and measuring School Quality], Trondheim: Senter for økonomisk forskning rapport nr. 01/16. Fang, Z. (1996). A review of research on teacher beliefs and practices, Educational Research, 38 (1): 47-65

Fraillon, J., Ainley, J., Schulz, W., Friedman, T., & Gebhardt, E. (2014). Preparing for life in a digital age: The IEA International Computer and Information Literacy Study. International report. Cham: Springer. Greene, J. A., & Seung, B. Y. (2014). Modeling and measuring epistemic cognition: A qualitative re-investigation. Contemporary Educational Psychology, 39(1), 12-28. Hatlevik, O.E., Egeberg, B., Gudmundsdottir, G.B., Loftsgarden, M. & Loi, M. (2013). Monitor skole 2013. Om digital kompetanse og erfaringer med bruk av IKT i skolen [Monitor School 2013. On digital competence and experiences with use of ICT in schools], Oslo: Senter for IKT i utdanning. Hauge, T.E. (2016). On the life of ICT and school leadership in a large-scale reform movement: A case study. In Elstad, E. (2016). Digital expectations and experiences in education. SENSE Publishers, Rotterdam/ Boston /Taipei, (pp. 97-116). Huberman, M. (1989). The professional life cycle of teachers. The Teachers College Record, 91(1), 31–57. Ifenthaler, D., and Schweinbenz, V. (2013). The acceptance of Tablet-PCs in classroom instruction: The teachers‟ perspectives. Computers in Human Behavior, 29(3), 525- 534. Klassen, R. M., and Chiu, M. M. (2010). Effects on teachers‟ self-efficacy and job satisfaction: Teacher gender, years of experience, and job stress. Journal of Educational Psychology, 102(3), 741. Kline, R. B. (2005). Principle and practice of structural equation modeling. New York: The Guildford Press. Kluge, L. (2001). Reformforsøk i 700 skoler [Reform attempts in 700 schools], Aftenposten July, 5th. Koehler, M. J. & Mishra, P. (2009). What is technological pedagogical content knowledge? Con-temporary Issues in Technology and Teacher Education, 9, 60–70. Koepke, S., & Denissen, J. J. (2012). Dynamics of identity development and separation– individuation in parent–child relationships during adolescence and emerging adulthood–A conceptual integration. Developmental Review, 32(1), 67-88. Kval, K.E. (2014). Det store skolesviket. En diagnose og 18 forslag til forbedringer [The large school betrayal. A diagnosis and 18 suggestions for improvements], Oslo: Dreyer. Kvernbekk, Tone (2001). Erfaring, praksis og teori [Experience, praxis, and theory]. In T. Kvernbekk (ed): Pedagogikk og lærerprofesjonalitet. Gyldendal Akademisk. s 146 - 163 Larsen, S.K. & Nilsen, D. (2012). Mener PC-bruk i skolen er lærerens ansvar [Believe computer use in schools is the teacher's responsibility] VG 26 juni 2012. Retrieved from: http://www.vg.no/nyheter/innenriks/skole-og- utdanning/mener-pc-bruk-i-skolen-er-laererens-ansvar/a/10059849/ Leinhardt, G., & Greeno, J. G. (1986). The cognitive skill of teaching. Journal of educational psychology, 78(2), 75-95. Mulholland, J., & Wallace, J. (2001). Teacher induction and elementary science teaching: Enhancing self-efficacy. Teaching and Teacher Education, 17(2), 243–261. Nilsen, D. (2011). Lærer: - Skolen er blitt som en internettkafé [Teacher: - The school was as an internet cafe] VG 18.juli 2011. http://www.vg.no/nyheter/innenriks/skole-og-utdanning/laerer-skolen-er- blitt-som-en-internettkafe/a/10080543/

Ministry of Education (2014). Strategy: The teacher promotion. Pulling towards knowledge in school. Oslo: Ministry of Education. OECD (2015). Students, Computers and Learning. Making the Connection, Paris: OECD. Opfer, V. D., Pedder, D. G., & Lavicza, Z. (2011). The role of teachers‟ orientation to learning in professional development and change: A national study of teachers in England. Teaching and Teacher Education, 27(2), 443-453. Ottenbreit-Leftwich, A., Glazewski, K., Newby, T. & Ertmer, P. (2010). Teacher value beliefs as-sociated with using technology: Addressing professional and student needs. Computers and Education, 55, 1321–1335. Ramo, M.U. (2016) IKT i skolen, en fantastisk mulighet[ICT in schools: a fantastic opportunity]. http://www.minervanett.no/ikt-i-skolen-en-fantastisk-mulighet/ Rivkin, S. G., Hanushek, E. A., & Kain, J. F. (2005). Teachers, schools, and academic achievement. Econometrica, 73(2), 417–458. Rockoff, J. E. (2004). The impact of individual teachers on student achievement: Evidence from panel data. The American Economic Review, 94(2), 247–252. Scherer, R., Siddiq, F., & Teo, T. (2015). Becoming more specific: Measuring and modeling teachers‟ perceived usefulness of ICT in the context of teaching and learning. Computers and Education, 88, 202–214. Senter for IKT i utdanning (2015). Hensiktsmessig bruk av IKT i klasserommet – en veileder. [Appropriate use of ICT in the classroom - a manual] Oslo: Senter for IKT i utdanning. Retrieved from: https://iktsenteret.no/sites/iktsenteret.no/files/attachments/veileder_hensikts messig_bruk_bm_lav.pdf Skaalvik, E. M., & Skaalvik, S. (2007). Dimensions of teacher self-efficacy and relations with strain factors, perceived collective teacher efficacy, and teacher burnout. Journal of Educational Psychology, 99(3), 611. Strømsø, H. I., & Bråten, I. (2010). The role of personal epistemology in the self- regulation of internet-based learning. Metacognition and Learning, 5(1), 91-111. Sætre, B.O. (2016). Kostbar loking på Facebook [Pricey surfing on Facebook]. Retrieved from https://disqus.com/by/bjornsatre/ Tschannen-Moran, M. Hoy, M. A. & Woolfolk A.E. (2007). The differential antecedents of self-efficacy beliefs of novice and experienced teachers. Teaching and Teacher Education, 23(6), 944–956 Van Acker, F., van Buuren, H., Kreijns, K., & Vermeulen, M. (2013). Why teachers use digital learning materials: The role of self-efficacy, subjective norm and attitude. Education and Information Technologies, 18(3), 495-514. Woolfolk, A.E. & Hoy, W.K. (1990). Teachers‟ sense of efficacy and beliefs about control. Journal of Educational Psychology, 82(1), 81–91. Zacharia, Z.C., & de Jong, T. (2014). The effects on students‟ conceptual understanding of electric circuits of introducing virtual manipulatives within a physical manipulatives oriented curriculum. Cognition and Instruction, 32, 101-158. Zee, M., & Koomen, H. M. (2016). Teacher Self-Efficacy and Its Effects on Classroom Processes, Student Academic Adjustment, and Teacher Well-Being A Synthesis of 40 Years of Research. Review of Educational Research, doi: 0034654315626801.

International Journal of Learning, Teaching and Educational Research Vol. 16, No. 1, pp. 120-134, January 2017

Using Reflective Journaling to Promote Achievement in Graduate Statistics Coursework

J. E. Thropp Embry-Riddle Aeronautical University Daytona Beach, FL

Abstract. By encouraging students to reflect on their own learning processes and identify areas that they understand as well as areas that require further improvement, their academic performance can improve. This reflection promotes the development of metacognitive skills, and may be especially beneficial for students facing new and challenging material, notably statistics coursework. Students’ grades from two different sections of a graduate-level introductory statistics course were compared. In the experimental section, students were encouraged to complete a weekly reflective journal about their learning progress and related experiences in the course. The control section followed the same course structure as the experimental section, but did not complete the journals. Independent samples t-tests were conducted on assignment and test scores as well as the final course grade for the experimental and the control sections. The experimental section performed significantly better than the control section on one assignment and one test. The experimental section also had a marginally higher average than the control section on a second assignment. The results and analysis of journal content show support for the use of reflective journaling to improve the student learning experience.

Keywords: Journals; statistics education; learning management systems.

Background Instructors strive to encourage and support students to enhance their learning outcomes and to set them on a path to succeed not only in their coursework, but also in their objectives outside of the classroom. Motivation is undoubtedly a key component to college student success, yet can be hindered when the material to be learned is associated with negative affective components, evokes anxiety, and is perceived as difficult. Statistics is one such course that often fits the aforementioned description. This paper discusses how keeping weekly reflective metacognitive journals may have improved the learning experiences and academic performance of students in a graduate-level statistics course.

Factors Influencing Students’ Statistics Attitudes and Performance Coursework in statistics has been generally described by students as non- engaging and difficult (Pan & Tang, 2004), oftentimes resulting in generally adverse experiences (Murtonen & Lehtinen, 2003). A review of the literature

121

reveals a wide variety in potential factors underlying the reasons why many students harbor negativity towards statistics and exhibit difficulty in performing the coursework.

Seeing the value of statistics in one’s career and other applications beyond the classroom is a factor influencing attitudes towards statistics (Fullerton & Kendrick, 2013; Ramirez, Schau, & Emmioglu, 2012; Schau, Millar, & Petrocz, 2012). Interest level in statistics also plays a role in statistics-related anxiety. Macher et al. (2013) reported a negative correlation between the two variables, as individuals tend to devote greater time and effort to objectives that interest them. Anxiety and lack of intrinsic motivation have been linked with students’ procrastination of statistics tasks (Dunn, 2014). Indeed, task-engagement and seeing the value in a task are important motivational components that underpin learning (Ryan & Deci, 2000).

Academic self-concept (Marsh & Martin, 2011) involves students’ perceptions of their own academic skill level and knowledge in a specific domain. Macher et al. (2013) found a negative relationship between mathematics self-concept and anxiety about statistics. Low self-efficacy in mathematics and negative prior experiences with mathematics can be linked to anxiety specific to statistics coursework (Zeidner, 1991). Students who feel they’ve performed well in previous mathematics coursework also tend to have positive attitudes towards statistics (Hannigan, Hegarty, & McGrath, 2014), yet an important consideration is that mathematical and statistical abilities, while related in terms of numerical manipulation, also differ in that statistical ability draws upon additional domains, such as verbal reasoning and use of logic (Fullerton & Kendrick, 2013; Johnson & Kuennen, 2006).

Statistics anxiety is a ―situation-specific form of anxiety…that occurs when an individual is exposed to statistics content, problems, instructional situations, or evaluative contexts‖ and is ―an enduring, habitual type of anxiety‖ (Macher et al., 2013, p. 536). It can interfere with the process of learning course material, thereby resulting in poor grades (Cook, 2010; Onwuegbuzie & Wilson, 2003; Williams, Payne, Hodgkinson, & Poade, 2008; Zeidner, 1991).

In the development of the Statistics Anxiety Rating Scale, a six-factor model of statistics anxiety comprised of two dimensions emerged (see Cruise et al., 1985). The first dimension, which pertains to anxiety, has three factors: a) interpretation anxiety, b) test and class anxiety, and c) fear of asking for help. The second dimension, which pertains to attitudes, also has three factors: a) worth of statistics, b) computational self-concept, and c) fear of statistics teachers. This model has been supported using several diverse populations of statistics students over the years (e.g., Mji & Onwuegbuzie, 2008; Papousek et al., 2012; Teman, 2013), and recently by DeVaney (2016) who validated it with graduate– level statistics students.

Macher et al. (2011) indicated that students with statistics anxiety tend to exert less effort in studying and procrastinate in their coursework as well as in

122

studying for exams. Decremented intrinsic motivation has also been linked with students’ statistics anxiety and task procrastination (Dunn, 2014). At the graduate level, statistics anxiety can extend into other aspects of academic performance, such as comprehending research articles involving statistical results (Onwuegbuzie, 1997), and performing research-related activities that require statistical knowledge (Onwuegbuzie, 2003).

Various individual differences in personality and disposition have also been linked with performance in statistics coursework. Furnham and Chamorro- Premuzic (2004) documented three such influential factors: The Big Five personality traits of conscientiousness and extraversion (see Costa & McCrae, 1992), and spatial ability. They reported a positive relationship between conscientiousness and statistics exam scores at the college level, a negative relationship between extraversion and statistics performance, and a positive relationship between spatial ability and statistics performance. Individual differences in mathematical orientation have been evaluated with the preference for numerical information (PNI) construct. Viswanathan (1993) describes PIN as a disposition for the use (or disuse) of numerical information in general, and that it reflects students’ willingness to learn and use mathematics and statistics. Additionally, trait anxiety, a stable disposition toward feeling anxious in general (Spielberger, 1972) has been linked to statistics anxiety (Macher et al., 2013; Papousek et al., 2012). Anxious individuals also generally tend to be pessimistic about their future task performance (Eysenck & Derakshan, 1997), which could likely lead to negative attitudes regarding statistics coursework.

Metacognition and Reflective Journals According to Paris and Winograd (1990, p. 7), ―students can enhance their learning by becoming aware of their own thinking as they read, write, and solve problems.‖ This awareness of one’s own thinking process is referred to as metacognition, and is the basis for promoting self-directed learning, which encourages learners to become active participants in their own academic pursuits. Flavell (1976) further describes metacognition as: …one’s knowledge concerning one’s own cognitive processes and products or anything related to them, e.g., the learning-relevant properties of information or data. … Metacognition refers, among other things, to the active monitoring and consequent regulation and orchestration of these processes in relation to the cognitive objects or data on which they bear, usually in the service of some concrete goal or objective. (p. 232)

Cognitive self-appraisal is one component of metacognition. As observed by Flavell (1978), this self-appraisal focuses on students’ abilities to address what they know about a subject, how they process information about the subject, and when and why they should apply related knowledge. Further, Douglass and Morris (2014) suggest that an understanding of one’s own learning style promotes self-directed or more autonomous learning in which students develop their own comprehension of the topic at hand. This knowledge enables a learner to select and use appropriate metacognitive self-management strategies to

123

develop or modify plans for solving a problem in order to achieve his or her goal (Paris & Winograd, 1990). According to the theory of self-determination, intrinsic motivation to succeed academically is facilitated by a more autonomous style of learning (Ryan & Deci, 2000), the benefit of which is often improved performance in coursework (Giuffrida, Lynch, Wall, & Abel, 2013).

Understanding one’s personal learning style is a result of the self-reflection of ―intellectual and affective activities in which individuals engage to explore their experiences in order to lead to new understandings and appreciations‖ (Boud, Keogh, & Walker, 1985, p. 19). Students can use journals to document these experiences and synthesize them with their other prior knowledge to enhance learning (Boud, 2001). Moon (1999) describes several ways in which journal- keeping can promote proactive learning through reflection, such as developing critical thinking and greater inclination towards inquiry about a topic, increasing active participation and ownership in the learning process, promoting self- empowerment, and developing an alternative means of communication when self-expression is difficult. A journal can also aid the writer in identifying his or her intentions and goals as they relate to a situation, and is especially helpful when the situation is complex and unfamiliar. Further, through reflection, a writer can identify questions that need to be addressed in order to be productive in unfamiliar situations (Boud, 2001).

Journaling in Learning Mathematics Writing to Learn (WTL) is a movement in education that emphasizes the benefit of language as a key component in knowledge acquisition, regardless of curricula. WTL asserts that students develop an understanding of material through their own written thoughts and observations (Bazerman et al., 2005). Writing about course material is an active, engaging process that encourages students to reconstruct the meaning of the material in their own relatable terms (Borasi & Rose, 1999). Particularly supportive of learning is expressive writing, in which writers probe their own thoughts and ideas for reflection (Britton, Burgess, Martin, McLeod, & Rosen, 1975).

Writing to Learn Mathematics (WTLM) is a branch of WTL that focuses on writing activities to enhance understanding of mathematics. Examples of such activities include writing to explain topics in terms that are more relatable to the student and writing about attitudes regarding the topic. Barker et al. (2004, p. 16) assert that ―requiring students to write about mathematics helps them learn and gives instructors valuable insight into the nature of their understandings‖. Indeed, by reading students’ journals, instructors can gain an awareness of their students’ perceptions and in turn, this can give them opportunities to evaluate their own pedagogical styles (Kenny, Schoffner, & Norris, 2014). Further, Meel (1999) describes how writing about mathematics helps to personalize otherwise unfamiliar concepts to the students.

Borasi and Rose (1999) showed how WTLM is an expressive form of writing that can take the form of journals. In their study, undergraduate algebra students

124

used journals to document their feelings about mathematics, and reflecting on their own learning processes bolstered their knowledge about the course material. In a study by Bagley and Gallenberger (1992), high school mathematics students kept daily journals to help develop problem-solving strategies. They described this activity as ―a means of knowing what we think – a means of shaping, clarifying, and discovering our ideas‖ (p. 660). Loud (1999) found that undergraduate mathematics students who kept a weekly journal exhibited not only higher final exam scores, but also more positive attitudes regarding the course.

Writing activities have also been shown to aid in reducing statistics-related anxiety and improve statistics course performance (Chiou, Wang, & Lee, 2014). In the one-minute paper strategy, students anonymously document and submit to their instructors their responses to the following questions: What is the most important concept you learned in class today? and What questions remain unanswered? This exercise allows students to identify concepts that require further understanding and describe difficult concepts using their own words.

Research Questions and Hypotheses Given the current body of knowledge, the researcher sought to address whether keeping a reflective, online journal could improve the learning experiences of students enrolled in a statistics course, specifically at the graduate level. In doing so, an exploratory study was conducted in which students from one section of the course were offered an optional journaling task, and their average grades were compared to those from a section in which journaling was not offered. It was hypothesized that students who completed the journals would have higher mean scores on the evaluation items in the course (e.g., grades on assignments and tests) as well as the final average in the course as compared to students who did not complete the journals. If results indicated that keeping a journal was associated with higher average grades, then a more widespread implementation of journaling activities in future statistics courses might be supported. Additionally, the content of the journals was evaluated in the interest of investigating trends in student perceptions of the course and self- assessments of their own knowledge, skills, and affective states. The researcher was also interested in the nature of the questions and concerns students had throughout the course, as well as any evidence of motivational factors that could be driving academic behavior and performance.

Methodology Study design. This study used a mixed-methods quasi-experimental design to investigate the effect of reflective journaling on student performance in a statistics course. Students enrolled in two different sections of a graduate-level introductory statistical analysis course were recruited as participants. This course is a 16-week, in-person, core requirement in a Master of Science degree program in the university at which this study was conducted.

In the experimental section of the course, the students (n = 13) had the option of volunteering to keep online journals to document their learning experiences in

125

the course throughout the semester. The control section (n = 14) was conducted in the same manner as the experimental section, with the exception that the students were not offered the option to complete the journals. Participation was voluntary, and approval to conduct the study from the institution’s ethics review board was obtained.

Participant description. Students enrolled in both sections of the course were comprised of a variety of backgrounds; they were a mixture of traditional and nontraditional types, various nationalities, and differing levels of prior mathematics coursework. While all of the students had completed at least one introductory level mathematics course at the undergraduate level, only a portion of them had completed any undergraduate statistics coursework. All of the students were familiar with the Blackboard online learning management system (LMS) that was used in the course, but none had expressed previous use of the journal tool available on it.

Procedure. On the first day of class in the experimental section, the instructor announced the option of participating in the study to the students, explaining that they had the option of completing an online journal regarding their learning progress each week. The journals were made available through a journal tool embedded in the Blackboard LMS site for the course. Students in the experimental section were asked to document briefly their weekly learning experiences (not including weeks in which tests were administered) using the online journal regarding the following issues: • Concepts covered in class in the current week that they felt they had difficulty understanding and might necessitate further clarification • Questions they had regarding the concepts covered in class that week • Information that they felt might be helpful to explain or clarify the difficult concepts • How they planned to seek clarification of these concepts Although these specific journal topics were posed to them, they were not prohibited from writing about other topics pertaining to their experiences in the course.

Students were expected to spend not more than approximately five to ten minutes per week on the journaling task, and were advised to work on the journal when it was convenient for them. They were told that participation was strictly voluntary with no bearing on course grade and that the journals were to be used as a learning aid and would not be read by anyone except the instructor. They were also informed that the instructor only assessed the journals for completion and honest effort, not for content. Neither course credit nor any other compensation was offered in exchange for participation. The instructor described the journaling activity as an optional task to the students in the first week of the course, and listed it as an optional weekly task on the course syllabus. The instructor did not remind them to do the journals afterwards; considering the exploratory nature of this study, the instructor preferred to observe the natural tendency for students to use the journal without being prompted.

126

In both sections, students were evaluated on three assignments, two tests, and one final exam in fulfillment of course requirements. These evaluation items are described below.

Data analysis assignments. Students’ scores on three different data analysis assignments were collected. The course lectures, textbooks, and other instructional resources (e.g., checklists for proper formatting and requirements for what to report in results sections, templates to use for the assignments) instructed the students in how to successfully complete the assignments.

The Statistical Package for the Social Sciences (SPSS) was the statistical software package students learned to use in the course. For each of the three data analysis assignments using SPSS (i.e., SPSS Assignments 1, 2, and 3), students were provided with a dataset and asked to test several different hypotheses using the software. Each of the three SPSS assignments covered a different set of statistical operations taught in the course (e.g., one assignment covered t-tests, another covered correlations and regressions, etc.). To succeed in these assignments, the students were required to know the proper type of statistical test to execute, how to execute it using SPSS, how to interpret the SPSS output, and how to write the results and discussions in the American Psychological Association (APA) format.

Tests. There were two multiple-choice tests administered in the course. The questions covered statistical concepts that were taught in the course lectures, but did not cover the use of SPSS or APA formatting. Test 1 covered material taught in the first half of the course, while Test 2 covered material taught in the second half of the course.

Final exam. The final exam was cumulative, covering all statistical operations taught in the course. For the final exam, students received a dataset and were asked to test several hypotheses using SPSS. Successful performance on the final exam required students to be able to determine which statistical operation to execute for each hypothesis test and explain why they chose each particular operation, report the results of each hypothesis test, and briefly explain what the results of each hypothesis test meant.

For both the experimental and control sections, student grades were collected from the LMS at the end of the semester. Independent samples t-tests were conducted on grades for all evaluation items to compare the experimental and control sections. For the experimental section, the content from the online reflective journals was collected from the journal tool in the LMS.

127

Results Student performance. Grades on three SPSS assignments, two multiple-choice tests, one cumulative final exam, and the final course average were compared. The experimental section performed significantly better (p < .05) than the control section on SPSS Assignment 1 and Test 1. The experimental section had a marginally higher mean score (p = .07) than the control section on SPSS Assignment 2. Although not significantly different, the experimental section also had numerically higher mean scores than the control section on the remaining evaluation items. See Table 1 for the significant differences.

Table 1: Significant Differences between Experimental and Control Sections Evaluation Experimental Control t-statistic and Item Section Section Cohen’s d (where Mean Score Mean Score significant) (SD) (SD) SPSS Assignment 1 173.75 157.79 t(24) = 2.416, (200 points possible) (17.77) (15.14) p = .024* d = .93 (large effect) SPSS Assignment 2 183.46 173.79 t(25) = 1.884, (200 points possible) (10.86) (14.97) p = .071 Test 1 86.15 77.32 t(25) = 2.377, (100 points possible) (9.02) (9.88) p = .025* d = .93 (large effect) Note. * Indicates significant at the .05 level.

Although the mean scores of the experimental section were numerically higher than those of the control section in the remaining evaluations (SPSS Assignment 3, Test 2, and the final exam), the differences were not significant (all p > .05). There was also no significant difference in the final course averages between the two sections (p = .096).

A closer look was then taken at the students in the experimental section. Six of the 13 participants opted to complete a journal entry more than once. The performance of these six journal writers was compared to that of the students who opted to not use the journal more than one time. Performance was significantly higher among journal users on SPSS Assignments 2 and 3 (see Table 2). Although not significantly different, the journal users also had numerically higher mean scores on the remaining evaluation items than those who did not opt to use the journal.

128

Table 2: Significant Differences between Journal Users and Non-Journal Users within the Experimental Section Evaluation Journal Users Non-Journal t-statistic and Item Mean Score Users Cohen’s d (where (SD) Mean Score significant) (SD) SPSS Assignment 189.86 176.0 t(11) = 2.759, 2 (9.55) (8.56) p = .019* (200 points d = 1.53 (large possible) effect) SPSS Assignment 189.43 178.67 t(11) = 3.603, 3 (6.13) (4.27) p = .004* (200 points d = 2.04 (large possible) effect Note. * Indicates significant at the .05 level.

Analysis of students’ journal contents. The contents of the journals from the experimental section were analyzed. Of the 13 students enrolled in the semester in which the metacognition journals were introduced, seven students used the journal at least once (six of whom used the journal more than once), creating a total of 25 entries. Six of the students refrained from using the journal altogether. The number of journal entries was greatest in the first five weeks of class, and then declined thereafter (see Figure 1).

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

Figure 1. Number of journal posts by week.

Some journal entries documented multiple topics. The content was categorized according to whether it featured a question or a statement, and then by the topic. The content of the entries in terms of the specific topics posed to the students is as follows. Students made eleven statements about their learning of statistical concepts, four statements about using SPSS, and one statement about using APA format. Students also used the journal to ask the instructor a total of seven

129

questions regarding statistics material: four questions were about statistical concepts, two questions were about how to do an assignment, and one question was asked about how to use APA format.

In addition to the specific topics posed to them, students also used to the journal to express general motivational factors; there were ten instances of this type of content. One student (i.e., ―Student A‖) regularly used the journal to document her feelings about the course: how she was progressing, external pressures that added to her workload, her desire to perform well in the course, questioning her ability at times, and also expressing confidence in herself. Some of her selected quotes are as follows:

Student A: ―The hard work really paid off! I still can’t believe how well I did on this last paper. Now I just need to keep up the work and do well on the next paper and test.‖

―Alright, so I had a lot of things going on at work and in my life so I was not able to take as much time for the paper…then again, I didn’t realize how much time was really needed.‖

―I am happy to see the other students from our class tutor each other…it shows how dedicated we all are to try to do well in this class.‖

Some examples of the entries made by other students are as follows:

Student B: ―This week in class the professor went over the basics of SPSS. I understood most of the topics covered. However, I will likely review on my own with the SPSS software to become comfortable with the processes of re-coding data, analyzing it, and creating charts to represent the data visually.‖

Student C: ―Pretty much I understand everything covered in class, however this [sic] are the things I might bump into later:

 Forgetting specific definitions and concepts  Remembering the rules for the whole course

The plan? Keep going back to do quick readings of notes and search for material on the internet that reviews the topics... regarding the rules my idea is to keep in touch with the TA and read the syllabus...

Other thing for me was that many things were new to me since I haven't been in ERAU before or for that matter in a University in the U.S.A.‖

Student D: ―I knew the day would come when the material becomes more difficult. I still understand most things in class but when I started working on the paper, APA and the table made it so difficult. I will definitely seek help and try to see if the TA has time to help.‖

130

Instructor reply to Student D: ―Formatting tables is one of the most difficult parts of APA. Do your best on the tables for now and come see us for help. You will get it right with practice.‖

Discussion and Conclusion In general, the experimental section that used the journals executed better academic performance in the course assignments and tests than the control section. These differences were significant for SPSS Assignment 1 and Test 2, and marginal for SPSS Assignment 2. The differences were not significant in the other course evaluation items. There was also no significant difference in final course averages.

Though completing the weekly journals was entirely voluntary, some students participated somewhat regularly. At the beginning of the semester, participation was more frequent; however, participation declined as the semester continued, possibly due factors such as students forgetting to participate and reduced novelty of the task. To maintain a naturalistic environment, the instructor did not regularly remind students to complete their journals. This hands-off approach may have contributed to the decline in journal entries as a function of time. Similarly, Borasi and Rose (1989) reported that students’ journaling in a mathematics course waned when instructors did not prompt them to do so.

While it cannot be determined why some of the students refrained from using the journals, Boud (2001) describes some of the factors that can discourage individuals from keeping journals. Writing about one’s perceived deficiencies, sources of unease, and problems can be prohibitive, especially if the writer is concerned about the identity of the reader and perceived negative consequences that could result. Knowing that the instructor would be reading about students’ criticisms of not only themselves, but also possibly of the course itself, could have been deterrents to participation.

Students used the journals to document what they had learned that week mainly in statistical concepts; however, some also documented what they had learned about the SPSS software and APA formatting. Statements about what they had learned outnumbered questions they had about the course material. Questions about statistics were the most frequent, followed by questions about the assignments and one question about APA format. According to Douglass and Morris (2014, p. 16) ―talking with professors to identify gaps in learning‖ facilitates student self-directed learning. The content of students’ journals showed that the journals indeed served as a means of communicating with the instructor about where their perceived knowledge needed improvement.

Another positive use of the journal was students reflecting on their own learning experiences. In some cases, students had been anticipating having a difficult time with statistics, yet were surprised that they were learning effectively and pleased with their own progress in the course. These statements likely reflected their intrinsic motivation to succeed in the course. One student regularly used the journal as a means to express her motivation-related feelings; sometimes

131

they were positive and sometimes negative, but the instructor used her writings as an opportunity to encourage her perceived self-efficacy. As Herman (2012) states, students’ intrinsic motivation can improve when they feel their instructors believe in their abilities.

Aside from reflecting on their own learning, the journals may have also provided another means for students to communicate with the instructor. While students were encouraged to express themselves in class, and visit and email the instructor for assistance during the course, the journal provided an additional, perhaps less formal means of communication between the students and the instructor. The instructor noticed that some students who did not vocalize during class did write journal entries; in accordance with Moon (1999), perhaps the journal served as a more comfortable format in which communicate.

This study was exploratory in nature and thus it only investigated two different sections of a graduate-level statistics course. A larger sample size may have produced more powerful hypothesis testing. With only 14 participants in the control section and 13 in the experimental section, the study could have benefited from larger sample sizes, but may still provide a glimpse into the advantages of journaling in statistics coursework. Collecting more data from future sections could increase the ability to detect achievement benefits as well as trends in how the students are choosing to use the journals.

In conclusion, the practice of keeping a journal may have been a positive influence on student performance in graduate-level statistics coursework. It may have improved some aspects of student performance in terms of grades, and also served as an effective and comfortable venue for students to communicate with the instructor and receive positive feedback on their learning. Keeping reflective journals in statistics coursework is a practice that is worthy of continued exploration. Many online LMSs currently provide journal platforms that can be employed for documenting students’ reflections on their coursework in statistics and other quantitative methods. Future work can address the impact of reflective journaling using a larger sample size as well as possibly reminding students about participating in the journal multiple times throughout the semester in order to increase the number of entries made.

References Bagley, T. & Gallenberger, C. (1992). Assessing students’ dispositions: Using journals to improve student performance. Mathematics Teacher, 85(8), 660-663. Retrieved from http://webapp1.dlib.indiana.edu/virtual_disk_library/index.cgi/4273355/FID 1736/journals/enc2205/2205.htm Barker, W., Breesoud, D., Epp, S., Ganter, S., Haver, B., & Pollatsek, H. (2004). Undergraduate programs and courses in the mathematical sciences: CUPM curriculum guide 2004. Washington, DC: Mathematical Association of America. Bazerman, C., Little, J., Bethel, L., Chavkin, T., Fouquette, D., & Garufis, J. (2005). Reference guide to writing across the curriculum. West Lafayette, IN: Parlor Press. Retrieved from http://wac.colostate.edu/books/bazerman_wac/wac.pdf Borasi, R. & Rose, B.J. (1989). Journal writing and mathematics instruction. Educational Studies in Mathematics, 20(4), 347 – 365. doi: 10.1007/BF00315606

132

Britton, J.N., Burgess, T., Martin, N., McLeod, A., & Rosen, H. (1975). Development of writing abilities: Ages 11 – 18. New York: Macmillan. Boud, D. (2001). Using journal writing to enhance reflective practice. In L.M. English & M.A. Gillen (Eds.), Promoting journal writing in adult education (pp. 9-18). San Francisco: Jossey-Bass. Boud, D., Keogh, R., & Walker, D. (1985). Promoting reflection in learning: A model. In D. Boud, R. Keogh, & D. Walker (Eds.), Reflection: Turning experience into learning (pp. 18-40). New York: Nichols. Chiou, C.C., Wang, Y.M., & Lee, L.T. (2014). Reducing statistics anxiety and enhancing statistics learning achievement: Effectiveness of a one-minute strategy. Psychological Reports, 115(1), 297-310. doi:10.2466/11.04.PR0.115c12z3 Cook, A. (2010). Improving the success rate in statistics (No. 415). Brisbane: University of Queensland, School of Economics. Retrieved from http://www.uq.edu.au/economics/abstract/415.pdf Costa, P. T. & McCrae, R. R. (1992). NEO PI-R Professional Manual, Odessa, FL: Psychological Assessment Manuals. Cruise, R.J., Cash, W., & Bolton, D.L. (1985, August). Development and validation of an instrument to measure statistical anxiety. Paper presented at the Annual Meeting of the American Statistical Association, Las Vegas, NV. Devaney, T.A. (2016). Confirmatory factor analysis of the Statistical Anxiety Rating Scale with online graduate students. Psychological Reports 118(2), 565-586. doi: 10.1177/0033294116644093 Douglass, C. & Morris, S.R. (2014). Student perspectives on self-directed learning. Journal of the Scholarship of Teaching and Learning, 14(1), 13-25. doi: josotl.v14i1.3202 Dunn, K. (2014). Why wait? The influence of academic self-regulation, intrinsic motivation, and statistics anxiety on procrastination in online studies. Innovative Higher Education, 39, 33 – 44. doi:10.1007/s10755-013-9256-1 Eysenck, M. W. & Derakshan, N. (1997). Cognitive biases for future negative events as a function of trait anxiety and social desirability. Personality and Individual Differences, 22, 597-605. Flavell, J.H. (1978). Metacognitive development. In J.M. Scandura & C.J. Brainerd (Eds.), Structural/process theories of complex human behavior. The Netherlands: Sijthoff & Noordoff. Flavell, J.H. (1976). Metacognitive aspects of problem solving. In L.B. Resnick (Ed.), The nature of intelligence. Hillsdale NJ: Lawrence Erlbaum Associates. Fullerton, J.A., & Kendrick, A. (2013). The math problem: Advertising students’ attitudes towards statistics. Journal & Mass Communication Educator, 68(2), 134- 149. doi: 10.1177/1077695813478181 Furnham, A., & Chamorro-Premuzic, T. (2004). Personality and intelligence as predictors of statistics examination grades. Personality and Individual Differences, 37(5), 943-955. Guiffrida, D.A., Lynch, M.F., Wall, A.F., & Abel, D.S. (2013). Do reasons for attending college affect academic outcomes? A test of a motivational model from a self- determination theory perspective. Journal of College Student Development, 54, 121- 139. doi: 10.1353/csd.2013.0019 Hannigan, A., Hegarty, A.C., McGrath, D. (2014). Attitudes towards statistics of graduate entry medical students: The role of prior learning experiences. BioMed Central Medical Education. doi: 10.1186/1472-6920-14-70 Herman, G.L. (2012). Designing contributing student pedagogies to promote students’ intrinsic motivation to learn. Computer Science Education, 22, 369-388. doi: 10.1080/08993408.2012.727711

133

Johnson, M., & Kuennen, E. (2006). Basic math skills and performance in an introductory statistics course. Journal of Statistics Education, 14(2). Retrieved from ww2.amstat.org/publications/jse/v14n2/johnson.html Kenny, R., Shoffner, M., & Norris, D. (2014). Reflecting on the use of writing to promote mathematical learning: An examination of preservice mathematics teachers’ perspectives. The Teacher Educator, 49(1), 28 - 43. doi: 10.1080/08878730.2013.848002 Loud, B.J. (1999). Effects of journal writing on attitudes, beliefs, and achievement of students in a college mathematics course (Doctoral dissertation). Retrieved from ProQuest Dissertations Publishing. (Accession No. 9923962) Macher, D., Paechter, M., Papousek, I., & Ruggeri, K. (2011). Statistics anxiety, trait anxiety, learning behavior, and performance. European Journal of Psychology of Education, 27(4), 483-498. doi: 10.1007/s10212-011-0090-5 Macher, D., Paechter, M., Papousek, I., Ruggeri, K., Freudenthaler, H., & Arendasy, M. (2013). Statistics anxiety, state anxiety during an examination, and academic achievement. British Journal of Educational Psychology, 83, 535-549. doi: 10.1111/j.2044-8279.2012.02081.x Marsh, H.W., & Martin, A.J. (2011). Academic self-concept and academic achievement: Relations and causal ordering. British Journal of Educational Psychology, 81(1), 59- 77. doi: 10.1348/000709910X503501 Meel, D.E. (1999). Email dialogue journals in a college calculus classroom: A look at the implementation and benefits. The Journal of Computers in Mathematics and Science Teaching, 18(4), 387 – 413. Mji, A., & Onwuegbuzie, A.J. (2004). Evidence of score reliability and validity of the statistical anxiety rating scale among technikon students in South Africa. Measurement and Evaluation in Counseling and Development, 36, 238–251. Moon, J. (1999). Reflection in learning and professional development. London: Kogan. Murtonen, M. & Lehtinen, E. (2003). Difficulties experienced by education and sociology students in quantitative methods courses. Studies in Higher Education, 28, 171- 185. doi:10.1080/0307507032000058064 Onwuegbuzie, A.J., (1997). Writing a research proposal: The role of library anxiety, statistics anxiety, and composition anxiety. Library & Information Science Research, 19, 5 - 33. Onwuegbuzie, A.J. (2003). Modeling statistics achievement among graduate students. Educational and Psychological Measurement, 63, 1020-1038. Onwuegbuzie, A.J., & Wilson, V.A. (2003). Statistics anxiety: Nature, etiology, antecedents, effects, and treatments – a comprehensive review of the literature. Teaching in Higher Education, 8(2), 195-209. doi: 10.1080/1356251032000052447 Pan, W., & Tang, M. (2004). Examining the effectiveness of innovative instructional methods on reducing statistics anxiety for graduate students in the social sciences. Journal of Instructional Psychology, 31, 149-159. Retrieved from http://search.proquest.com.ezproxy.libproxy.db.erau.edu/docview/213906873 ?accountid=27203 Papousek, I., Ruggeri, K., Macher, D., Paechter, M., Heene, M., Weiss, E., & Freudenthaler, H. (2012). Psychometric evaluation and experimental validation of the statistics anxiety rating scale. Journal of Personality Assessment, 94(1), 82–91. doi:10.1080/00223891.2011.627959 Paris, S.G., & Winograd, P. (1990). Promoting metacognition and motivation of exceptional children. Remedial and Special Education, 11(6), 7-15. Ramirez, C., Schau, C., & Emmioglu, E. (2012). The importance of attitudes in statistics education. Statistics Education Research Journal, 11(2), 57-75. Ryan, R.M., & Deci, E.L. (2000). Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American Psychologist, 55, 68-78. doi:10.1037/0003-066X.55.1.68

134

Schau, C., Millar, M., & Petocz, P. (2012). Research on attitudes towards statistics. Statistics Education Research Journal, 11(2), 57-75. Spielberger, C.D. (1972). Anxiety as an emotional state. In C.D. Spielberger (Ed.), Anxiety: Current trends in theory and research. London: Academic Press. Teman, E.D. (2013). A rasch analysis of the statistical anxiety rating scale. Journal of Applied Measurement, 14, 414–434. Viswanathan, M. (1993). Measurement of individual differences in preference for numerical information. Journal of Applied Psychology, 78(5), 741-752. doi: 10.1037/0021-9010.78.5.741 Williams, M., Payne, G., Hodgkinson, L., & Poade, D. (2008). Does British sociology count? Sociology students’ attitudes toward quantitative methods. Sociolgy, 42, 1003 – 1021. Zeidner, M. (1991). Statistics and mathematics anxiety in social science students—some interesting parallels. British Journal of Educational Psychology, 61, 319–328.

135

International Journal of Learning, Teaching and Educational Research Vol. 16, No. 1, pp. 135-160, January 2017

Competence and/or Performance - Assessment and Entrepreneurial Teaching and Learning in Two Swedish Lower Secondary Schools

Monika Diehl and Tord Göran Olovsson Umeå University Umeå, Sweden

Abstract. Entrepreneurial teaching and learning, thus entrepreneurial education corresponds well with formative assessment/assessment for learning. Both are characterised by an approach to education, teaching, and learning, which puts pupils in the centre of their own learning. Learning aims to ―go deep‖ and generate ―real learning‖ where competencies rather than measurable results are the focus. Both entrepreneurial education, and assessment for learning are promoted by the Swedish National Agency for Education. Entrepreneurial education has been inscribed in the national curriculum for Swedish compulsory schools since 2011. The same curriculum and syllabuses also focus on several knowledge requirements, which form the basis for assessing pupils‘ performances. Thus, the Swedish national curriculum can be said to send two rather disparate messages. This research focuses on lower secondary school and the broad approach of entrepreneurial education and uses Basil Bernstein‘s theory of performance and competence models to elaborate on entrepreneurial teaching and learning in relation to assessment. Observations along with interviews with teachers and pupils in two Swedish lower secondary schools provide the empirical basis for the research. The results reveal some differences between the schools but indicate that both teachers and pupils are relating to the prevailing dominance of performance models and thus encounter difficulties when trying to adopt entrepreneurial education and assessment for learning.

Keywords: entrepreneurial education; assessment for learning; lower secondary school; performance model; competence model.

Introduction Entrepreneurial education (EE) aims to develop an entrepreneurial mind-set among pupils and, thus, the future labour force (European Commission, 2002; 2006; 2010; 2012; 2013; 2016; OECD, 1989; 1998; Swedish Government Office, 2009). This means that pupils‘ abilities such as creativity, curiosity, and problem- solving are to be encouraged. Aspects such as interdisciplinary collaboration, involvement with the surrounding society as well as pupils‘ participation, influence, activity, and experiences are advocated, thus entrepreneurial teaching and learning is to be perceived as an educational approach (Erkkilä, 2000; Kyrö,

2005; Lackéus, 2015; Røe Ødegård, 2014). The need for EE has been expressed in national and international policy text for decades; in 2011, it was inscribed in the Swedish national curriculum (SNAE, 2011a).

In Sweden, there has also been an increased focus on assessment for learning (AfL), (also referred to as formative assessment)1, where assessment is used as a tool in the ongoing learning process. The state has encouraged implementation of AfL in Swedish classrooms through some of its policies (e.g. SNAE, 2011b). The motive behind this is to support pupils' learning and development. Among others, Stobart (2008) claims that AfL is a process, but also an approach to learning and teaching. It is obvious that EE can be easily linked to AfL. In literature on entrepreneurial teaching and learning, AfL is advocated and seen as a part of its means (Falk-Lundqvist, Hallberg, Leffler, & Svedberg, 2011; 2014; Josefsson, Bostani, & Josefsson, 2009). In parallel with this, there is also an increased summative focus in Swedish compulsory schools (Wahlström & Sundberg, 2015). Summative results usually are considered as what counts concerning pupils‘ future possibilities, and, to a large extent, the focus on summative results controls the mediation of knowledge (Stobart, 2008). Thus, the message to teachers and pupils is somewhat contradictory as summative assessment can be perceived as requiring other types of classroom practices and ultimately other forms of knowledge rather than those promoted by EE and AfL.

Basil Bernstein (2000) uses the concepts of performance models and competence models to distinguish between the knowledge forms that different pedagogic practices generate. This study is a part of a continuing professional development (CPD) programme on EE, including 25 schools across Sweden. It took place between 2012 and 2015 and its goal was to combine school improvement and research. The aim was to provide a better understanding of EE, further develop it, and encourage its adoption. The present study involves two lower secondary schools taking part in the programme. The study uses the concepts of the performance and competence models to elaborate on teachers and pupils understanding of knowledge, teaching, and learning in relation to EE and AfL. There is a lack of empirical research on EE in relation to assessment in compulsory schools. Furthermore, in the current research on AfL, there seems to be a lack of thorough studies using direct examples from classroom observations (Black, 2015). Therefore, this study aims to fill this gap.

Aim and research questions The aim of this study is to elaborate on how different modes of teaching and learning are associated to different forms of knowledge and assessment, and

1 In some research, the concepts of ‖assessment for learning‖ and ‖formative assessment‖ are given somewhat different meanings (e.g., Bennett, 2011; Swaffield, 2011). In this article, the concepts are used synonymously, which is also common in other research (Swaffield, 2011). Both concepts are used in the article: In most sections assessment for learning is mainly used, but in the Results section, formative assessment is used.

how this relates to entrepreneurial education in Swedish lower secondary schools. The following research questions will guide the study:  How is teaching conducted and questions about knowledge and assessment expressed in practice?  How do teachers express themselves about teaching, knowledge, and assessment in relation to entrepreneurial learning?  How do pupils express themselves about and relate to teaching, learning, and assessment?

Background In this section, we place entrepreneurial education in an international and national (Swedish) context. The Swedish assessment system in compulsory school will be presented, followed by the means of summative assessment and assessment for learning. Views on entrepreneurial learning in relation to assessment will conclude this part of the article.

Entrepreneurial education International and national policy documents have expressed the need for EE for a long time (European Commission, 2002; 2006; 2010; 2012; 2013; 2016; OECD, 1989; 1998; Swedish Government Office, 2009). Globalisation and marketization are the main incentives for introducing EE from an early age. In Sweden, this is a thread throughout the educational system (Swedish Government Office, 2009). EE is often delineated by a narrow and a broad approach. The narrow approach is connected to teaching and learning about how to plan, start, and run businesses. The broad approach, most commonly called entrepreneurial learning, is about usurping an entrepreneurial mind-set (e.g. Erkkilä, 2000; Jones & Iredale, 2010; Leffler, 2006; Lindster & Norberg, 2016; Mahieu, 2006). In Sweden, the narrow approach is primarily a question for upper secondary school and higher education, whereas the broad approach permeates the entire educational system. Education should provide the conditions for pupils to develop an entrepreneurial mind-set, as the Swedish curriculum (SNAE, 2011a, Lgr 11) says:

An important task for the school is to provide a general but coherent view. The school should stimulate pupils‘ creativity, curiosity and self-confidence as well as their desire to explore their own ideas and solve problems. Pupils should have the opportunity to take initiative and responsibility, and to develop their ability to work both independently and together with others. The school in doing this should contribute to pupils developing attitudes that promote entrepreneurship. (p.11, italics added).

In Swedish national syllabuses, the pupils‘ abilities to analyse, reflect, communicate, argue, and evaluate can be connected to entrepreneurial teaching and learning.

Assessment in Swedish compulsory school In Swedish compulsory school, a criterion-referenced assessment system has been operating since the mid-1990s (Wikström, 2006). This system requires that student performance is assessed in relation to knowledge requirements (criteria) in each subject. Teachers assess their own pupils‘ performance also regarding summative assessment, which is the basis for selection for further studies. Several assessment reforms have been implemented in recent years with consequences for classroom assessment. Therefore, there is a greater emphasis on assessment, particularly since the revised national curriculum Lgr 11 (SNAE, 2011a) was introduced in 2011. The new curriculum included new syllabuses in all subjects, clearer requirements for what students should achieve, a new grading scale with more levels, end-of-term grading in Grades 6 and 7 (instead of starting in Grade 8, which was formerly the case), and national tests in more subjects starting at earlier grades. The implementation of these reforms brought about a greater focus on summative assessment (Wahlström & Sundberg, 2015) in classroom practice, with teachers more carefully using the national steering documents in teaching and assessment. The main purposes of the assessment reforms were to improve student performance and school effectiveness through better monitoring of pupils‘ progress (SOU, 2013: 30) and to detect pupils‘ knowledge gaps at an early stage (Samuelsson, 2012).

Summative assessment Summative assessment (assessment of learning) is usually described as a summarising of learning after teaching is completed. For a number of years, there has been an international trend towards more summative assessment for accountability, which involves external (political, administrative) requirements that are easily measured and provide clear results (Lundahl, Hultén, Klapp, & Mickwitz, 2015; Stobart, 2008; Torrance, 2011;). This may cause tensions between the kind of assessment that teachers perceive as meaningful and assessment concerning accountability (Black, 2015; Gardner, 2010; Suurtam & Koch, 2014). Public announcement of schools‘ results based on students‘ achievements in terms of test scores and grade levels are now common and may counteract the use of formative strategies in classroom practice (Brookhart, 2009; Natriello, 2009). Assessment methods commonly attributed to summative assessment are standardised tests (in Swedish compulsory school: national tests) and grades. The concept of ―high-stakes testing‖ (e.g. Au, 2007) is usually used to describe tests with major consequences for students. The national tests in Grade 9, in lower secondary school in Swedish compulsory school, is generally perceived by students as being high-stakes (Eklöf & Nyroos, 2013). The national test scores may have implications for final grades in Grade 9, which, in turn, give students access to further studies in upper secondary school. National tests in Swedish compulsory school are provided in Grades 3, 6, and 9.

Test scores and grades are often criticised for not contributing to real learning (Stobart, 2008; Jönsson, 2013; Lundahl et al.,2015). Previous research on summative assessment (e.g. Eklöf, 2017) shows that a focus on test scores and grades detracts from actual learning, which implies increased negative stress for both pupils and teachers. One downside of standardised tests (Håkansson &

Sundberg, 2012) is that they limit teaching content and drive teaching to include only what is coming on the test, a practice which is called ―teaching to the test‖ (Mehrens & Kaminski, 1989). However, summative assessments themselves do not necessarily prohibit good teaching, rather it is how the summative results are used. Good teaching and assessment involves assessment for learning and assessment of learning, in interaction (Black, 2015; Olovsson, 2017; Tveit, 2014)

Assessment for Learning Black and Wiliam (1998) argued that formative classroom assessment can improve student performance. Their argumentation was built upon their idea that the main purpose of assessment in education is to support pupils‘ learning. Wiliam and Thompson (2007) elaborated a framework for AfL using three key processes of teaching and learning. The processes involve determining goals for learners, what the learner already knows or can do, and what is required to get the learner to the goal. In this process, feedback is the key element for effective learning, and constructive, forward-looking feedback is what enhances learning most effectively (Hattie & Timperley, 2007). However, criticism of the positive impact of AfL has emerged. The criticism e.g. concerns a perceived instrumental approach to teaching (Bennett, 20011; Torrance, 2007).

If feedback and instructions from teachers to students is too detailed, there is a risk that students may become too dependent on others in their learning process and take a more passive role (Groome, 2005; Jönsson, Lundahl, & Holmgren, 2015; Torrance, 2007; Torrance, 2012). In this context, Torrance (2007) argues that criteria compliance replaces learning. If AfL becomes too teacher-centred and aimed at supporting pupils to achieve a certain grade, it might lose a significant part of its value: ―At the heart of it is a vision of active and self-regulating learners who work to make sense of what they are learning …. Peer- and self- assessment play a key role in this self-regulation, as does feedback‖ (Stobart, 2008, p. 169). In conjunction with this, Stobart discusses the spirit of AfL (see also Marshall & Drummond, 2006), which implies a clear understanding of why teaching is organised in certain ways. It is not just about formative techniques, used without reflection in the classroom, a practice referred to as being the “letter‖ of AfL (Marshall & Drummond, 2006; Stobart, 2008).

Entrepreneurial education and assessment Literature and research on the broad approach of EE advocates using AfL as it is and sees it as being a natural part of EE, and aligned with its means (Falk- Lundqvist, Hallberg, Leffler, & Svedberg, 2011; 2014; Josefsson, Bostani, & Josefsson, 2009; Lackéus & Moberg, 2013; Lundahl, 2011). The connections between EE and AfL are obvious in many ways. Primarily, they both concern the recognition that learning is not about being able to reproduce what is already known but is increasingly about creating new knowledge relevant to solve problems (e.g. Gibb, 2000; Johannisson, Amundsson, & Kivimäki, 2010; Lundahl, 2011; SNAE, 2015). EE is often linked to Dewey and progressivism, where the learning process rather than measurable knowledge outcomes is the focus (e.g. Røe Ødegård, 2014), and the same applies for AfL (Lundahl, 2011). EE and AfL are approaches based on pupils‘ knowledge level; they represent a different

view of teaching and learning than what is seen as a traditional approach (Erkkilä, 2000; Gibb, 1993; Kyrö, 2005; Lackéus, 2015; Lundahl, 2011; Stobart, 2008). Nonetheless, not much research on assessment of EE in compulsory school has been done. The problem of assessing complex abilities such as creativity, fantasy, innovation, critical thinking, and problem-solving are addressed by researchers within the fields of AfL and EE (Diehl, Lindgren & Leffler, 2015; Diehl, 2016a; Diehl 2016b; Lundahl, 2011). Lackéus, Lundqvist and Middleton (2015) and Lackéus (2016) offer suggestions based on a smartphone app where students are able to report and reflect on critical learning events as they happen. Other than that, Swedish literature on entrepreneurial learning suggests different ways for teachers to conduct AfL by providing examples of classroom situations and reflecting on them (Falk-Lundqvist, Hallberg, Leffler, & Svedberg, 2011; 2014) or by offering examples of different scoring rubrics (Josefsson Bostani & Josefsson, 2009).

Theoretical Framework This study aims to investigate what kind of knowledge is focused on in schools and classes taking part in a CPD programme on entrepreneurial learning. Contradictions in curriculum and syllabuses and their implications on teaching and learning have been described above. A way to explain the different knowledge forms that emerge is to use Bernstein‘s (2000) pedagogic models, performance and competence. These in turn have an impact on what kind of consciousness and identities are shaped among teachers and pupils. ―Our identities, or our sense of who we are, is realised or brought about through what Bernstein calls three ―message‖ systems: curriculum, pedagogy and evaluation‖ (South African Institute for Distance Education, SAID, 2010). Curriculum concerns what is defined as legitimate knowledge (selection), pedagogy concerns the form or mode used for transmission (how the selection is taught), and evaluation concerns how knowledge is tested or assessed. Evaluation, thus assessment, is dependent on curriculum and pedagogy (SAID, 2010).

Bernstein (1975) distinguishes between two different types of curriculum, one working according to a collection code and one according to an integrated code. In Sweden, as in many other countries, the national curriculum is a merging of both to some degree and thus has many contradictory aspects. The introduction of EE in many ways presupposes an integrated code, but is contradicted by very strict knowledge requirements and required learning outcomes, which to a high degree presupposes schoolwork according to a collection code. The codes in turn are characterised by strong or weak classification and framing. Strong classification means that each category (in this case subjects) has a unique identity, voice, and rules for internal relationships. Weak classification indicates less specialised discourses, voices, and identities. Framing concerns the regulation of communication and interaction in different categories. It is about the internal logic of a pedagogical practice and describes the nature of control over sequencing and pacing, selection of communication, expected assimilation, learning criteria, and the social context in a classroom. Framing, like classification, can be stronger or weaker. Strong classification and strong

framing denotes a pedagogical practice characterised by a collection code, weak classification and framing, thus an integrated code (Bernstein 1975). The different codes consecutively generate contrasting pedagogic models, which influence tests and evaluation/assessment.

The performance model The performance model is characterised by strong classification.

Briefly, a performance model of pedagogic practice and context places the emphasis upon a specific output of the acquirer, upon a specific text the acquirer is expected to construct and upon the specialised skills necessary to the production of this specific output, text or product. (Bernstein, 2000, p. 44)

The pedagogic discourse thus is based on the specialisation of subjects, the rules for legitimate texts are explicit, and pupils‘ performances are graded. An important part of teachers‘ professionalism is to be explicit and to know the grading procedures, and when pupils work is evaluated (assessed), the focus is on what is missing. ―Accountability is facilitated by the ‗objectivity‘ of the performance and thus outputs can be measured and optimised‖ (Bernstein 2000, p. 50). The grading system implies a potential repair service, and pupils are to blame if they fail. Teachers essentially have control over selection, sequencing, and pace, and the space for the pedagogic practice is clearly marked and explicitly regulated. The modes of instruction imply disciplining regulation where deviances are very visible and not much time is spent or needed on personalised modes of control. In such pedagogic practice, the acquirer (pupil) invisibly is positioned in the past. Thus, teaching is constructed on the past but it is made visible that they learn for the future. Bernstein (2000) distinguishes between introverted and extroverted modalities. ―In the case of introverted modalities the future is the exploration of a specialised discourse itself as an autonomous activity‖ (p. 49) which allows for more autonomy for teachers and learners within curriculum regulation. ―In the case of extroverted modalities the future is likely to become dependent on external regulation, for example, the economy or local markets‖ (p.49), resulting in much less autonomy. The performance model entails rather low costs due to ―the explicit structures of transmission and of its progression‖ (p. 50).

The competence model The competence model is characterised by weak classification and in ―competence theories there is an in-built procedural democracy, an in-built creativity, an in-built virtuous self-regulation. And if it is not in-built, the procedures arise out of, and contribute to a social practice, with a creative potential‖ (Bernstein, 2000, p. 43).

The pedagogic discourse bases its transmission on competencies that pupils already possess or are thought to possess. It is group based, and pupils appear to have a great deal of control over selection, sequence, and space. The pedagogy is largely based on projects, themes, and experiences. Pupils have access to, and

considerable control over, the construction of many different spaces. Different activities are not punctuated, and progression is implicit to the pupils. Pupils experience the pedagogic practice in the present tense, and teachers understand or ―read‖ the pupils competence development in their products to be able to facilitate further challenges. ―The consequence is that the meaning of an acquirer‘s sign is not available to the acquirer, only to the teacher‖ (Bernstein, 2000, p. 48). Teachers focus on and discuss pupils‘ products based on what is present, and criteria for evaluation (assessment) are implicit and diffuse. Even if positional and imperative control of pupils occurs, focus is on self-regulation and individual communication with each pupil about intentions, dispositions, and relationships, based on pupils‘ own reflexivity. Competence models require a wide range of autonomy, but there still needs to be a homogeneity in practice, which reduces each teacher autonomy. When it comes to teaching material, packages and textbooks are seldom used. Instead teachers construct their own, which requires autonomy. Pupils products are difficult to evaluate objectively, and the models are less amenable to public scrutiny and accountability. Competence models are not connected to specialised futures. The costs for competence models are rather high, as there are no time limits for pupils learning and hidden costs in terms of time needed for teachers‘ commitment, planning, and socialising with parents are usual. Other than that, feedback to pupils and constructing learning material is time consuming. The

(…) lack of recognition of hidden costs may lead to ineffective pedagogic practice because of the demands of the practice, or, if these are met, the lack of recognition may give rise to ineffectiveness because of the fatigue of the teachers. (Bernstein, 2000, p. 49)

There are different modes of performance models and different modes of competence models but also merges between them in pedagogic practices. Bernstein (2000) argues that performance modes are normal across all levels of education, and thus competency modes may imply an interruption ―or resistance to this normality or may be appropriated by official education for specific and local purposes‖ (p.51). When the state introduced the broad approach of entrepreneurial education into the Swedish national curriculum, this constituted what can be seen as an appropriation for specific purposes. Before looking into its realisation in practice and the assessment challenges involved, a description of this studies‘ methodology will be presented.

Methodology

Sample Six of the schools taking part in the CPD program were lower secondary schools. Out of these, two were chosen for data gathering due to their geographic similarities (both situated in suburbs of major Swedish cities) and differences in pupil demographics. Almost all the students in School 1 have a Swedish

background, and 1/3 of the students in School 2 have backgrounds other than Swedish.

School 1 is situated in a wealthy middle-class area where housing mainly consists of villas. It has about 150 pupils (grades 7–9, ages 13–16) at the time of data gathering. At the time of data gathering, some subjects where merged on occasion, but the school work was primarily conducted within subject-specific classes. School 2 is situated in a suburban community with mixed housing. It has around 300 pupils (grades 6–9, ages 12–16). The work was organised by subject.

Data collection The data collection was guided by Bernstein‘s (1975; 2000) theoretical frameworks. It included classroom observations and interviews with teachers and pupils and went on for about three weeks at each school. The aim was to interview the teachers whose lessons were being observed. The idea was to be able to link the interviews to what had been observed, thus the observations were conducted first.

All in all, 52 lessons were observed: 21 observations in School 1 and 31 in School 2. A structured observation schedule (Cohen, Manion, & Morrison, 2011) with fixed categories was created. The schedule was based on Bernsteinian categories to distinguish the degrees of teachers and pupils‘ control of communication, criteria, time, pace, sequencing, and order in the classroom. In addition, the observation schedule included Swedish (and international) policy documents (European Commission, 2002, 2006, 2010; Government Office of Sweden, 2009; OECD, 1989, 1998), and the Swedish national curriculum and syllabuses, which describe the required abilities for entrepreneurial education (SNAE, 2011a). Additional field notes were made to absorb the situation and to describe events and behaviours not covered by the structured part of the schedule. Situations, actions, and expressions connected to assessment made up a considerable amount of the field notes. Overall, the method can be described as a form of participant observation (Cohen et al., 2011; Kvale, 1997).

The interviews were semi-structured and used prepared questions that matched the observation protocol as well as questions on assessment, particularly formative assessment, and its relationship to entrepreneurial teaching and learning. The questions allowed scope for open-ended answers (Hannan, 2007). The interview guide was changed to some extent; questions were added and reformulated based on what had been observed during the time spent at the schools. Altogether, eight teachers were interviewed, three in School 1 and five in School 2, and the interviews lasted for 40–90 minutes. Pupils from all grades were interviewed in both schools, resulting in 14 group interviews and one single interview. Nine interviews were conducted in School 1, and six in School 2. Altogether, 38 pupils took part in the interviews. The interviews lasted 30–50 minutes. All interviews were recorded.

Data analysis Verbatim transcriptions were made to help the researchers remember the details in the context and situations in which the observations and interviews took place (Kvale, 1997). The field notes were transcribed and categorised. For purposes of this study, we focused on observations connected to assessment. The data was analysed by school in order to identify similarities and differences between them. The interviews were transcribed verbatim. The material was reviewed with the research question in mind, and general patterns, regardless of school, were searched for (generating natural units of meaning). The next step was to discern pupils‘ and teachers‘ statements about assessment practices. This meant organising the data into feasible and adequate categories (classifying, categorising, and ordering the units of meaning). A search for similarities and differences in the statements identified differences within and between the schools, allowing us to formulate new variables and recognise sub-categories and themes. Based on the concepts of entrepreneurial teaching and learning and assessment, a text was formed and organised, structuring narratives to describe the interview contents. Finally, the interviews were interpreted to create meaning, together with the field notes (Cohen et al., 2011; Watt & Boolsen, 2007).

Ethical considerations The research was part of a CPD programme, which the participants were aware of. Still, all respondents had to be willing to be interviewed, and the teachers had to be willing to be observed. On the occasion of the first observation in each class, the researcher presented the reason for being in the classroom. Before interviewing willing learners below 15 years of age, parental consent was obtained by having them sign a paper with information about the research. All respondents were informed that they could choose to end the interview at any time (Swedish Research Council, 2002). The names and locations of the schools and the names of the participants were anonymised. Due to the small amount of teacher interviews, they are not specifically presented after the citations in the Results section of the article. Gender and grade is presented regarding the pupils, as they were many more of them and the information is seen as important for understanding their statements.

Results At the time of data gathering, both schools were engaged in the implementation of the national tests. This meant that a lot of time was spent preparing and conducting the tests. The first part of the results will start by presenting a general picture of the teaching culture in the observed classrooms. This will be followed by a description of classroom work and assessment when the schools are preparing for the national tests. Along with this, other observations related to assessment will be presented. After that, the teachers‘ discussions of teaching, testing, formative assessment, and entrepreneurial learning will be presented. Finally, the pupils‘ opinions about learning, tests, and assessment will be heard.

Entrepreneurial teaching, learning, and assessment in practice School 1 On a general level, many of the observed lessons were characterised by a high degree of creativity. Pupils were given the opportunity to work together, find

their own solutions, and at times work in an interdisciplinary manner. Often, but not always, the pupils seemed to be aware of what they are supposed to learn. Although many lessons were characterised by teachers urging the pupils to reflect and think on their own, that is, teaching was directed towards more complex learning goals, sometimes the ―right‖ answers were given directly and the pupils were told to do things in a certain manner to get it right.

Preparations for the national tests were usually a part of the classroom lessons. One example is that part of lesson in Social Science was devoted to discussion and practice tests from a previous year. The teacher urged the pupils to pay special attention to the questions that required reasoning, reflection, and analysis and told the class that they would later look at examples of how these questions are assessed. On one occasion, one of the pupils said that he was nervous about the upcoming tests, and the teacher said that he was too.

Regarding other observations related to assessment practice, elements of formative assessment occurred quite frequently. One example is from a lesson in Physics and Technology (the subjects are merged). During the lesson, the teacher give support and feedback on what the pupils were working on at the moment and frequently walked around the classroom, talking to individual pupils about their results on former tasks and tests. The teacher told the pupils that they need to be more analytic and that they have to develop their reasoning to get higher grades. The teacher often used a scoring rubric, which helps her explain what areas the pupils need to develop. As she said, ―The more the pupils see the rubrics, the more used they will become to the way of thinking and can develop their work.‖

In Social Science, the teacher encouraged the pupils to develop their language and write complete sentences, to reason about advantages and disadvantages regarding the issue they are writing about. In Home Economics, the pupils were required to reflect on the processes they learned. In general, the teachers were very encouraging towards the pupils. One example from the field notes is from a Physics and Technology lesson when the teacher was facilitating a discussion with pupils about their thought and different solutions. Following are some of the teacher‘s comments: - ―This was really good, now you are close!‖ - ―What fun, then you‘ve got a challenge today also!‖ - ―Let‘s try it this way!‖

Grading and assessment was explicitly discussed on many occasions during everyday classroom work. The teachers often referred to upcoming tests when they were teaching or commenting on pupils‘ work. Pupils often asked how their work is going to be assessed. One example is that some classes were building a city in Mindcraft2 during a lesson in Geography, which was carried out in smaller groups. One pupil asked the teacher how he was going to grade their cities. He replied, ―How you have been thinking as a city planner, the

2 A computer programme that enables the user to build houses and cities.

thought behind it. If it looks good in Mindcraft or not does not give high grades. It‘s how you have been thinking.‖ Regarding behavioural matters, the teachers repeatedly urged the pupils to put away their mobile phones or prodded them to get started on their work. However, no teacher directly connected behaviour with assessments and grades.

School 2 On a general level, the lessons in this school followed the same pattern. They started out with an introduction, followed by the pupils working on tasks, often answering fixed questions from the textbooks, while the teacher walked around the classroom helping the ones asking for help. Often the end of the lesson seemed to come suddenly as the teachers realised time was running out. It was very seldom that pupils seemed to be aware of why they were working with tasks, and what they actually learned. In handicrafts, a more apparent entrepreneurial and formative approach in teaching could be observed: The handicraft teacher worked according to a process-driven routine where the pupils made a plan, decided how to execute the plan, and then evaluated their work.

During many lessons, much time was spent preparing the pupils for the upcoming national tests. In Mathematics for example, it was common for teachers to hand out previous tests, and to urge the pupils to work out the problems in them. The teachers checked whether the pupils were having difficulty with assignments and spent time explaining to them individually or for the whole class to hear. Teachers also spent a great deal of time explaining how the previous tests were assessed and what the pupils have to consider when taking the upcoming tests. The field notes reveal that ―The teacher urges the pupils to notice how many points the different assignments generate, as it gives them a hint about how much they need to answer and how advanced the question is.‖ The same teacher also informed the pupils that showing how they think, the process, is more important than the actual answer.

A great deal of concern about the national tests can be heard among both teachers and pupils. One pupil asked the teacher how the test results will affect the grades. Another pupil asked if it is a good idea, as a preparation, to read all the textbooks from lower secondary school in the subject being tested. One teacher expressed concerns about the lack of time for preparation and revealed that she has lately devoted more time to the subject that will be tested than to other subjects she is teaching. The time issue is also obvious. Teachers often tell the pupils to hurry up and to practice as much as possible during the lessons. Along with this, the teachers are supportive of pupils´ learning and positive towards the pupils: ―Are you doing fine? I think you are doing sooo fine, it‘s going to be fun to summarise your results later on‖.

On a general level, other observations of the assessment practice indicate a good relationship between the teachers and the pupils. However, many of the teachers reacted strongly when the pupils did not do as they were told, and the pupils were often rebuked. They were often told to stay in their places and to be silent

and focused. Behavioural reprimands were often connected to grades and assessment. One example noted when a boy is noisy: ―Stefan, now you have to change places, you get high scores on the national test, and then you waste your talent!‖

The teachers often referred to upcoming tests to motivate pupils to get started and to work. If the class worked diligently and behaved well, they were promised to be able to end the lesson a bit earlier than usual. Lesson attendance was of utter importance and was related to grades and to behaviour.

Teachers’ comments of entrepreneurial teaching, learning, and assessment School 1 Relations between entrepreneurial teaching, learning, and assessment One teacher said he was struggling to find a balance between the required subject knowledge outcomes, which require a more teacher-led style of teaching, and entrepreneurial teaching methods. He argued that pupils need a base of knowledge to be able to understand, reason, and analyse. Entrepreneurial education, on the other hand, he said, requires more interdisciplinary and investigatory working methods where the pupils are active and have influence over their learning. The challenges regarding these working methods are for example: finding time for planning and enacting interdisciplinary work, getting colleagues to join in on such working methods, and establishing fair assessment of individual pupils. The teacher seemed stressed to ensure that all pupils achieved the knowledge requirements. Even if he wanted to work more entrepreneurially and intended to let the pupils have greater influence, the national tests, the required learning outcomes, and fear of not being able to provide fair grades, had a repressive influence on his teaching. Another teacher confirmed that assessment controls the teaching to a great extent. She said that she needs tests to be able to show parents why their children got certain grades, and that grades are important to the pupils:

They are incentive for some. If one can show them, now you have developed these competences, but if you want to get better, you can do this and this. The grades clarify their learning, they get help and can choose to work for a better grade.

Even this teacher expressed that the national tests are something that stresses her, but thought that more interdisciplinary projects would provide an opportunity to assess the pupils together with colleagues.

Teachers’ understanding of formative assessment Two of the teachers said that formative assessment is about supporting pupils‘ learning. One said it is about the pupils‘ ability to self-assess and to be able to set their own goals. One teacher said:

I think it is difficult….I see formative assessment as sitting down with the pupil and talk about their work and I don‘t really think I

have that time, but I can see that many need more of it. …. You can also view formative assessment as working digitally, that I give comments on their work over the internet…but still I think that meeting them, to sit down and talk to them is better….but that is difficult with a whole class.

Even though this teacher saw supporting pupils during a lesson as a form of formative assessment, he did not find the classroom situation a good environment for this, because some pupils never ask for help. The teacher considered different ways to achieve good formative assessment, but thought it would be difficult to get it working.

Another teacher thought she developed her use of formative assessment in a good way; she uses rubrics and evaluates each lesson together with the pupils. That gives her good understanding about what the pupils have understood or not, which does not make the results of the tests so crucial.

School 2 Relations between entrepreneurial teaching, learning, and assessment One teacher argued that entrepreneurial learning is about motivating the pupils to learn in a safe environment, for example, that pupils would dare to talk in the classroom. She said that many pupils question the teaching content and therefore she intends to motivate them, to persuade them that the knowledge is important for them to be able to improve learning. The teacher suggested that one way to motivate pupils is to be an enthusiastic teacher; she claimed the enthusiasm would be reproduced in the classroom. The teacher gave an example of a situation when she talked about an issue and it later turned up as a question on the national test. She said, ―It‘s important how you mediate things. The pupils may think that the issue will turn up as a question on the national tests‖.

This teacher thought she was teaching entrepreneurially in Technology and said that the pupils are highly motivated during those lessons. The pupils were eager to get good grades and even stayed after the lessons to finish their work. The teacher tried having the pupils grade their own and others‘ work.

Another teacher related entrepreneurial teaching to paying attention to every pupil, engaging them in planning, and letting them have a say about when tests should be conducted. Having good relations with the pupils is essential, she said. She did not think that the national tests have a negative influence on entrepreneurial learning but thought that the national testing system is wrong in many aspects. First of all, they imply a lot of negative stress for the pupils. Secondly, she thinks the tests signal a mistrust of teachers, she says that carrying them out:

(…) underestimates us teachers. We already have grades (to assess the pupils‘ knowledge) … one must trust that we know what we do. I feel mistrusted, the tests are good but it feels like I am being tested. The pupils want to have as good grades as possible, but they don‘t even know what to practice.

. Yet another teacher admitted they have focused very much on Physics this year, as she thought it was going to be the science subject in the national test. Her way of working entrepreneurially has been to engage the pupils in correcting their own tests, which she found made the pupils committed and contributed to an interest in finding out the right answers on failed questions. She also wanted to improve in making pedagogic plans so that the pupils understand the knowledge requirements, the means for progressing, and the reason for learning a certain teaching content.

Teachers’ understanding of formative assessment Two teachers connected formative assessment solely to how tests are assessed and engaging pupils in correcting tests. One of these teachers said she always tries to assess pupils‘ working process and sometimes ―threatens‖ pupils if, for example, they talk too much, implying that they will be assessed for grading based on their behaviour. She admitted that she assesses behaviour too, even if she knows she should not. The handicraft teacher saw formative assessment as guiding the pupils continuously. She also thought a lot about how formative assessment could be enacted and its relationship to summative assessment. The teacher initiated a working process where the pupils first consider what they want to work with and how, then carry the project through and finally evaluate their work. She discussed assessment in relation to this process, which means that not only the final product could be assessed but also the idea and the considerations about the project—in other words, the actual work, as well as the pupils‘ reflections about their work. She said she finds it very demanding to grade and assess the implemented process. She regarded the described working process to align with entrepreneurial learning, and as both formative and summative assessment. The Social Science teacher meant that everything the pupils do is to be assessed, to their advantage. One should assess when they do good things and always look on their work positively. She said she tries to give the pupils positive feedback, lead them in the right direction, talk to them and be attentive to see if someone does not understand.

Pupils’ comments of teaching, learning, and assessment School 1 The pupils expressed rather negative thoughts about studying for tests, which they think has a negative influence on their learning:

I don‘t really like to read books and study for a test. Then I think it feels like one just studies, studies, studies … just crude study all the time. Then you do the test, and the day after you have forgotten it all. Sometimes it feels like you just do it because you have to, not to learn … sometimes it would be better to learn in other ways … like doing a project and a presentation or something like that. Then, you learn more when you work together. (girl, Grade 8)

Many pupils requested more practical work in school as they think they learn better in that way. One boy (Grade 9) said, ―The pupils would learn more if

there was more practical work, (the knowledge) would settle more due to that both the body and the brain work at the same time.

Regarding if the pupils were aware of the knowledge requirements, one pupil (boy, Grade 8) said: ―Yes, we often get to take part of the knowledge requirements. It‘s just that it‘s not only those that count. It‘s also how one behaves in lessons, how one hands in an assignment, how one has written the assignment‖.

Many pupils felt negative stress regarding grades. This was most obvious among 9th graders:

There is such a tremendous pressure now…..we get homework in all subjects, and then there are the national tests, and thus we have tests in other subjects too. Like, this week, we had a national test in Swedish and above that we had a test in Mathematics. (girl, Grade 9)

Overall, the amount of time required to keep up with other homework and study for tests was experienced by pupils as stressful.

Not many pupils thought they had experienced formative assessment in the sense of getting feedback during the semester about their learning progress. Some said they never got feedback, and others said they experienced it on occasion. Some pupils mention that the Science teacher has given them feedback on their work and has pointed out what and how they can improve. The Science teacher uses a scoring rubric, which makes the feedback clearer. Other teachers give written feedback on tests and require pupils to save the tests in a binder. The pupils can then review the comments before the next test to inform themselves about what they need to improve. Yet other teachers, the pupils say, just tell them that they should plod along but do not clarify what to do and how to do it. ―He kind of doesn‘t say what I need to practice … so I don‘t really know what I should do or what I need to study, so it kind of never works‖ (girl, Grade 8). Even if the pupils try to get higher grades, the feedback they receive is not seen as sufficient. Many would like to get more continuous feedback on their performance and not only at the end of the semester when they receive grades. Some think that the teachers do not think about their grades during the semester but do a summative calculation only at the end.

Based on the pupils‘ statements, it does not seem common for them to be part of the assessment process. The only subject the pupils refer to is Swedish, where they have given each other responses on written texts. That said, many considered it is difficult to judge someone else‘s work or to be able to find something to comment on.

School 2 In this school, some pupils prefer conventional tests rather than assignments that require a lot of writing, thinking, and formulating long coherent texts. Even if they are aware that this kind of studying has a negative effect on their learning,

they argue that it is easier just to read and memorise content and answer questions in a test than to elaborate on an issue in a broader sense. They study for the test and then forget everything, especially if they feel the subject is uninteresting:

I think some tests, when you are not interested … then you just study sentences and words…then when there is a question then I can answer it exactly, but at the same time I don‘t understand what it is about, but I write exactly as it should be … but … I myself I don‘t understand it …and the day after the test I know nothing about it … I have just learned the words. (girl, Grade 9)

The interviews revealed that pupils occasionally are aware of the knowledge requirements, as some of the teachers have shown them to the pupils. However, the requirements generally seem to be rather unclear to the pupils.

The pupils expressed different degrees of stress connected to grades and assessment. It is rather obvious that, on a general level, the feeling of stress is related to age and grade. Pupils in Grade 9 are more stressed than those in Grades 6 and 7. Pupils in Grade 9 had a lot of national tests during the time of data gathering:

Now when we have the national tests … it is very, very much in one week! Now we have three (national tests) this week, two in History and one in French … it is very much …. it becomes very stressful. (girl, grade 9)

The pupils were not only worried about the national tests, they also had other tests at the same time. Some of the pupils were aware that the purpose of the national tests is to identify their subject knowledge, and that they are not meant that to study a lot in beforehand. But they still study and seem to feel very frustrated:

I‘ve heard the test will be about History … there is endless much history and I think it is meaningless to study… think I won´t have time to learn everything! So I just choose a little … it‘s hard to know what the test will be about. (girl, Grade 9)

The pupils were also asked if they were informed about how they were performing and how to proceed to improve their grades. The most common answer was that they get feedback from the teachers at the end of each semester, when the grades were already set. Other than that, they also get feedback at the individual evaluation talk (together with parents) every semester. Some pupils said that it also is ones‘ own responsibility to ask the teacher about feedback. Some pupils experienced oral and written feedback on tests and other schoolwork, but the most common feedback regarding grades was given in ―bad situations‖. One girl, for example, was upset because the teacher contacted her mother instead of talking to her when she was about to fail in a subject. The interviews also revealed situations where teachers used grades to correct what

they saw as misbehaviour or laziness. Occasionally, the pupils experienced involvement in the assessment process. In some subjects, they corrected their own tests or commented on a classmate‘s test or work.

Analysis and discussion

Practice The observations revealed that the overall pedagogic model in both schools can be linked to what Bernstein (2000) calls the performance model. The pedagogic practices are characterised by isolation between the different subjects, thus strong classification, even though the framing within the subjects differs to some degree between the schools. In School 1, the teachers in the observed lessons can be said to have a more profound understanding and interpretation of EE (Backström-Widjeskog, 2010), and as a result, elements of the competence model can be found. This is illustrated in that pupils have more room for creativity, regulate their time and pace to a higher extent, cooperate and communicate, and thus experience more self-regulatory elements (Bernstein, 2003).

Common in both schools were the preparations that were undertaken before the national tests in connection with everyday teaching. David (2011) discussed the ―narrowing of the curriculum‖, which means that the subjects or the content included in tests has a more prominent role in regular everyday teaching than other subjects or content. This is also something that is apparent in both schools. The teachers, in different ways, expressed that the national tests control teaching content, which implies a ―teaching to the test‖ practice (Mehrens & Kaminski, 1989). This in turn entails a risk that scores and grades, instead of learning, become the main focus (Eklöf, 2017). In School 2, references to grades are not seldom used in connection with pupils‘ behaviour. It can be argued that grades and tests to a certain extent control both teaching content and pupils‘ behaviour (Bernstein, 1971; Stobart, 2008). In the same school, grades and national tests are used as tools to motivate pupils to work harder, and if they do so, they may quit the lesson earlier, which can be seen as corresponding badly with the means of entrepreneurial learning. When assessment is described in literature about entrepreneurial learning, it is linked to AfL (Josefsson & Josefsson, 2009; Falk- Lundqvist, Hallberg, Leffler, & Svedberg, 2011). Thus, formative or ―entrepreneurial‖ assessment can be especially found in the feedback to pupils in School 1. Other than that, such assessment often takes the form of general praise and is non-specific in relation to the progress of learning (c.f. Gamlem & Smith, 2013). More examples of AfL are found in School 1, even if they mainly seem to be used to meet summative requirements, and tests and grading criteria control the formative activities in everyday teaching (Torrance, 2007).

In conclusion, summative requirements point out the direction for teaching and assessment in the classrooms that were studied (c.f. Stobart, 2008). The observations indicate that the teaching is to some extent directed towards complex goals/knowledge requirements and that parts of ―entrepreneurial learning‖ (e.g. the ability of analysing and reasoning) are present, even if some

pupils have difficulty knowing what is expected of them regarding different qualities and content in their work (Diehl, 2016b; Olovsson, 2015).

Teachers’ expressions In School 1, it is obvious that teachers struggle to find a balance between the curriculum merge of the performance and competence model (Bernstein, 2000). The introduction of EE has exposed teachers to the contradiction between the two models and induced worries about how to ensure that pupils meet the knowledge requirements yet are given the opportunity to work entrepreneurially. The performance model requires teachers to be explicit and objective when grading pupils‘ abilities, which is more feasible in a practice characterised by strong classification and framing, thus a curriculum following a collection code (Bernstein, 1975; 2000). However, the prerequisites for EE include weak classification and framing, i.e. a competence model according to an integrated code. Yet, this entails individual-assessment communication about intentions, dispositions, and relations based on pupils‘ own reflexivity (Bernstein, 2000). One of the teachers has solved the problem by introducing score rubrics, which she views as enabling her to be specific and clear with pupils and herself regarding grades and at the same time allowing for what is labelled entrepreneurial teaching and learning (SNAE, 2011a). However, the use of score rubrics has occasionally been met with criticism, namely that it evokes surface learning, leads to a more passive role for pupils, and entails a type of AfL that can be seen as more ―teacher-centred‖ (Jönsson, Holmgren & Lundahl, 2014). Stobart (2008) argues that there is a risk that AfL will be reduced to an instrument operated by high-stakes accountability, simply offering preparation for summative assessment. The dominance of external accountability pressures implies that there is a risk that the letter rather than the spirit of AfL merges into the accountability systems, and that AfL only becomes a means for achieving accountability requirements (Black, 2015; Stobart, 2008). Most teachers in School 2 are not as concerned, for they link EE to assessment by letting the pupils form part of the assessment process through correcting their own and/or their classmates‘ tests. In this school, it is also clear that not only the pupils‘ knowledge concerning the knowledge requirements are assessed. Skills such as creativity, responsibility for ones‘ own work, and aspects such as concentration, behaviour, and diligence, form the basis for assessment (cf. Klapp, 2015). This is in line with the performance model where the mode of instruction implies disciplinary regulation (Bernstein, 2000).

Pupils’ expressions Pupils in both schools talk about the lack of ―real‖ and ―deep‖ learning when studying for tests; they prepare by crude studying, or even memorising, before a test and then forgetting everything after. The testing culture which they are experiencing can clearly be connected to the performance model (Bernstein, 2000). Even if national tests include questions aimed at abilities required by the competence model, such as analysing, reflecting, and evaluating, many pupils seem to perceive them as high-stakes (Eklöf & Nyroos, 2013) and prepare them in accordance with a performance model. Thus, ―The shadow of high-stakes summative assessments operates as an obstacle through students as well as

directly for teachers‖ (Black, 2015, p. 174). The pupils in School 1 seek more projects and practical work as well as other methods for showing their knowledge achievement. Such requests were not as common in School 2, where some pupils preferred conventional tests requiring only short answers easily identified as right or wrong. A possible explanation may be the lack of experience with pedagogic modes connected to entrepreneurial learning or learning in accordance with an integrated code, more of a competence model, which the pupils in School 1, to some extent, have experienced (Bernstein, 2000; Diehl, 2016b). The predominance of the performance model and its implications as a repair service is revealed in School 2 in a pupil‘s expression of disappointment with the teacher, who contacted the pupil‘s mother instead of the pupil regarding the potential failure in a subject.

The empirical material does not provide information about how the development of the complex competences and abilities are presented to the pupils, but, at least in School 2, the pupils argue that their awareness of the knowledge requirements is inadequate. This gives reason to suppose that discussions about different qualities on reasoning and analysing do not occur to any significant degree (c.f. Gyllander Thorkildsen & Erickson, 2016). Pupils think that their involvement in the assessment process can be increased, even if pupils in School 2 already have some relevant experience. More feedback from the teachers regarding pupils‘ progress in different subjects is asked for, as it is seen as developing learning (e.g. Hattie & Timperley, 2007; Wiliam, 2011). More detailed reflection of pupils‘ awareness of the knowledge requirements and what is assessed is found in pupils in School 1; these pupils find the information rather clear in some subjects, though not in all subjects. For the pupils in School 2, it is more common not to be aware of the knowledge requirements. In relation to the two knowledge forms (performance and competence) described by Bernstein (2000), it can be concluded that School 1, to a greater extent than School 2, merges the performance and the competence models. School 2 can be said to work based on the performance model in all subjects except for Handicraft. Hence, when it comes to the pupils‘ awareness of the knowledge requirements, they seem to be implicit and diffuse (as in the competence model) in both schools. Yet, this may depend on how pupils view ‗criteria‘. In School 1, the pupils seem to know what to do to achieve the knowledge requirements but not why, whereas the pupils in School 2 more often have a diffuse sense of why but do not know, and can, on occasion, feel uncertain about how to meet the knowledge requirements.

Conclusion With a few exceptions, either teaching or learning in the two schools seems largely aligned with whether AfL or EE. Teaching usually takes place within specific subjects and is based on the knowledge requirements, most likely to ease and facilitate assessment (Wahlström & Sundberg, 2015). Thus, there are clear indications that the summative assessment is central to both schools, as demonstrated by both observations and interviews with teachers and pupils. Summative assessment can be linked to the knowledge form represented by the performance model (Bernstein, 2000). This model is, in turn, associated with a

curriculum based on a collection code and a pedagogy that Bernstein refers to as ‗visible‘. A visible pedagogy is characterised by strong classification and framing and emphasises pupils‘ performance and their external products (Bernstein, 2000). According to Bernstein (2003), visible pedagogy is the standard European practice and the one that leads to professional occupational placement. This means that the message systems, curriculum, pedagogy, and evaluation (assessment) are mainly influenced by the performance model, which subsequently impacts teachers and pupils in terms of consciousness and identity. The consciousness and identity, or the mind-set, that EE pursues requires more of a competence model and can thus be seen slightly more in School 1 than in School 2, even if the performance model is predominantly featured in both schools. Through, among other aspects, the introduction of EE along with strong OECD recommendations (Nusche, Halász, Looney, Santiago & Shewbridge, 2011) and efforts to implement AfL (Jönsson, Lundahl & Holmgren, 2015), the Swedish curriculum can be said to fuse modes of both performance and competence models. Bernstein (2000) argues that modes of competence models can be seen as interrupts or resistance to the ―normal‖ performance model. This may explain the difficulties and dilemmas that both teachers and pupils encounter and experience in relation to EE as well as AfL. As the implications of EE and AfL may not ―fit‖ pupils and teachers understanding and socialisation of the means of schooling, teaching, and learning.

The question of teacher autonomy in relation to national tests was raised during the teacher interviews. The introduction of EE in educational policy and curricula is intimately linked, nationally and internationally, to neoliberalism and marketisation: Pupils are to be prepared for the demands of the global labour market (e.g. European Commission, 2010; 2013; 2016; Government Office of Sweden, 2009; OECD, 1998; SNAE, 2015). This, on one hand, implies a need for pupils bound for the labour market to leave school with an amount of complex skills, abilities, and competences that are advocated by EE and supported by AfL. On the other hand, neoliberalism and marketisation underlie competition and ―survival of the fittest‖, which may explain the increased focus on measurement, performance, and accountability. Thus, it reflects the means of summative assessment. Bernstein argues that ―Accountability is facilitated by the ‗objectivity‘ of the performance and thus outputs can be measured and optimised‖ (Bernstein 2000, p. 50). Among the educational research society, there is significant concern about what comes in the wake of the ideological shift and its focus on performance assessment, measurement, quality assurance (Apple, 2009; Englund, Forsberg & Sundberg, 2012; Lidman, 2011, Ozga, Dahler- Larsen, Segerholm & Simola, 2011; Ozga, 2017), tests, and test results (Ball 2003; Lindgren, Hult, Segerholm & Rönnberg, 2012; Lindgren, 2012 Power, 1999). For example, there are concerns about the focus on teachers‘ abilities in eliciting certain outcomes replaces a discussion about the desirability (or lack thereof) of the outcomes themselves (Biesta, 2009). In Bernsteinian (2003) terms, this development would be connected to extroverted modalities within the performance model, which result in less autonomy for teachers, instead of external regulation such as economy and local market regulation of their work.

As such, this means that entrepreneurial teaching and learning, as well as the means of AfL, includes an inherent policy and curriculum contradiction, for they strive to foment ―real‖ and ―deep‖ learning in an aim to educate and develop autonomous, confident, analytic, and reflexive citizens. Thus, it seems that the letter rather than the spirit of AfL (Marshall & Drummond, 2006; Stobart, 2008) is at use. This reasoning can be linked to Bernstein‘s (Bernstein, 1990) theory of sacred and profane knowledge forms. The sacred can be seen as the ―spirit‖ and connected to ‗bildung‘ and thus to ―know-why‖ knowledge, and the profane as the ―letter‖, thus more practical ―know-how‖ knowledge. In educational practice, for teachers and pupils, the fusion of the performance and competence models is quite a challenge!

Acknowledgements This study is a part of a PhD project financed by the Swedish independent research institution Innovation, Research, and Development in Schools (IFOUS) and the Umeå School of Education, Umeå University, Sweden.

References

Apple, M.W. (2009). Foreword. In S. Gewirz, P., Mahony, In. Hextall, & A. Cribb (Eds.), Changing Teacher Professionalism International trends, challenges and ways forward. London & New York: Routledge. Au. W. (2007). High-stakes testing and curricular control: A qualitative metasynthesis. Educational Researcher, 36(5), 258-267. http://journals.sagepub.com/doi/pdf/10.3102/0013189X07306523 Backström-Widjeskog, B. (2010). Teachers‘ thoughts on entrepreneurship education. In K. Skogen, & J. Sjøvoll, J. (Eds), Creativity and Innovation: Preconditions for Entrepreneurial Education. (pp. 107-120). Trondheim: Tapir Academic Press. Ball, S. (2003). The teacher‘s soul and terrors of performativity. Journal of Education Policy. 18(2), 215-228. http://www.tandfonline.com/doi/abs/10.1080/0268093022000043065 Bennett, R. E. (2011). Formative assessment: A critical review. Assessment in Education: Principles, Policy & Practice, 18, 5–25. http://dx.doi.org/10.1080/0969594X.2010.513678 Bernstein, B. (1971). Class, codes and control. Volume I. Theoretical studies towards a sociology of language. London: Routledge. Bernstein, B. (1975). Class, Codes and Control, Volume 3, Towards a Theory of Educational Transmission. London: Routledge & Kegan. Bernstein, B. (1990). The structuring of Pedagogic discourse. London: Routledge. Bernstein, B. (2000). Pedagogy Symbolic Control and Identity – Theory, Research, Critique. Bristol: Taylor & Francis. Bernstein, B. (2003). Class, Codes and Control, Volume IV, The structuring of Pedagogic Discourse. Oxon: Routledge. Biesta, G., (2009). Good education in an age of measurement: On the need to reconnect with the question of purpose in education. Educational Assessment, Evaluation and Accountability. 21(1), 33-46. http://dx.doi.org/10.1007/s11092-008-9064-9 Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education: Principles, Policy & Practice, 5, 7–74. http://dx.doi.org/10.1080/0969595980050102

Black, P. (2015). Formative assessment – an optimistic but incomplete vision, Assessment in Education: Principles, Policy & Practice, 22(1), 161-177. DOI: 10.1080/0969594X.2014.999643 Brookhart, S.M. (2009). Assessment and examination. In: L. J Saha & A.G Dworkin (Eds.). International handbook on teachers and teaching. P. 2. New York: Springer. Cohen, L., Manion, L. & Morrison, K. (2011). Research Methods in Education. London & New York: Routledge. David, J.L. (2011). Research says../ High-Stakes testing Narrows the Curriculum. Educational Leadership. 68(6), 78-80. http://www.ascd.org/publications/educational_leadership/mar11/vol68/nu m06/High-Stakes_Testing_Narrows_the_Curriculum.aspx Diehl, M., Lindgren, J., & Leffler, E. (2015), The impact of Classification and Framing in Entrepreneurial Education: Field observations in Two Lower Secondary Schools. Universal Journal of Educational Research. 3(8), 489-501. http://dx.doi.org/1013189/ujer.2015.030803 Diehl, M. (2016a). From Entrepreneurship to entrepreneurial Education in Lower Secondary School: Pedagogising by Means of Pedagogic Device. Journal of Educational Issues. 2 (1), 36-56. http://dx.doi.org/10.5296/jei.v2i1.8748 Diehl, M. (2016b). Teachers‘ and Learners‘ Experiences of Entrepreneurial Education: Practice and Challenges in Relation to Visible and Invisible Pedagogy. Problems of Education in the 21st Century, 73 (73), 27-44. http://www.scientiasocialis.lt/pec/?q=node/1029 Englund, T., Forsberg, E. & Sundberg, D. (2012). Vad räknas som kunskap – Läroplansteoretiska utsikter och inblickar i lärarutbildning och skola. Stockholm: Liber. Eklöf, H., & Nyroos, M. (2013). Pupil perceptions of national tests in science: Perceived importance, invested effort and test anxiety. European Journal of Psychology of Education, 28(2), 497–510. doi:10.1007/s10212-012-0125-6 Eklöf, H. (2017). Betygen i den svenska skolan. I A. Hult & A. Olofsson (Red.), Utvärdering och bedömning i skolan (s. 62-76). Stockholm: Natur och Kultur. Erkkilä, K. (2000). Entrepreneurial Education. New York & London: Garland Publishing Inc. European Commission (2002). Entrepreneurship education at school in Europe – National strategies, curricula and learning outcomes. Brussels: Eurydice European Commission. Europakommissionen (2006). Promoting entrepreneurship in schools and universities. Bryssel: Eurydice.

European Commission (2010). Europe 2020. A European strategy for smart, sustainable and inclusive growth. Brussels: European Commission. European Commission (2012). Entrepreneurship education at school in Europe – National strategies, curricula and learning outcomes. Brussels: Eurydice. European Commission (2013). Entrepreneurship Action Plan. Brussels: European Commission. European Commission (2016). Entrepreneurship Education at school in Europe. Eurydice Report. Luxembourg: Publications Office of the European Union. Falk-Lundqvist Å., Hallberg P-G., Leffler E. & Svedberg G. (2011). Entreprenöriell pedagogik i skolan – drivkrafter för elevers lärande. Stockholm: Liber. Falk-Lundqvist Å., Hallberg P-G., Leffler E., & Svedberg G. (2014). Entreprenöriellt lärande – i praktik och teori. Stockholm: Liber. Gardner, J. (Ed.) (2010). Assessment and learning. London: Sage.

Gamlem, S. M. & Smith, K. (2013). Student perceptions of classroom feedback. Assessment in Education: Principles, Policy & Practice, 20(2), 150-169. http://dx.doi.org/10.1080/0969594X.2012.749212 Gibb, AA. (1993). Enterprise culture and education: Understanding enterprise education and its links with small business, entrepreneurship and wider educational goals. International Small Business Journal, 11(3), 11-34. https://www.researchgate.net/publication/247738528_Enterprise_Culture_and _Education_Understanding_Enterprise_Education_and_Its_Links_with_Small_B usinessEntrepreneurship_and_Wider_Educational_Goals Gibb, AA. (2000). In pursuit of a new ―enterprise‖ and ―entrepreneurship‖ paradigm of learning: creative destruction, new values, new ways of doing things and new combinations of knowledge. International Journal of Management Reviews, 4(3), 233-269. http://onlinelibrary.wiley.com/doi/10.1111/1468-2370.00086/abstract Government Office of Sweden (2009). Strategi för entreprenörskap inom utbildningsområdet. Stockholm: Regeringskansliet. Groome, T. H. (2005). Educating for life. New York, NY: Crossroad. Gyllander Thorkildsen, L. & Erickson, G. (2016). ‗If they‘d written more…‘- On students‘ perceptions of assessment and assessment practices. Education Inquiry 7(2), 137- 157. http://dx.doi.org/10.3402/edui.v7.27416 Hannan, A. (2007). Interviews in Education Research. [Online] Received from: http://www.edu.plymouth.ac.uk/resined/interviews/inthome.htm 20161005 Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research,77, 81–112. http://journals.sagepub.com/doi/full/10.3102/003465430298487 Håkansson, J. & Sundberg, D. (2012). Utmärkt undervisning – Framgångsfaktorer i svensk och internationell belysning. Stockholm: Natur och Kultur. Johannisson, B., Amundsson, A. and Kivimäki, K. (2010) Training in entrepreneurship as a many-sided struggle for growing insight. In Skogen, K., &J. Sjøvoll (Eds.), Creativity and Innovation: Preconditions for Entrepreneurial Education (pp.171-188). Trondheim: Tapir Academic Press. Jones, B & Iredale, N. (2010) Enterprise education as pedagogy. Education + Training, 52(1), 7-19. http://dx.doi.org/10.1108/00400911011017654 Josefsson Bostani, S. & Josefsson, R. (2009). Formativ bedömning vid entreprenöriellt lärande så här gör du. Kristianstad: Me University AB. Jönsson, A. (2013). Lärande bedömning. Malmö: Gleerups.Jönsson, A., Lundahl, C & Holmgren, A. (2015). Evaluating a large-scale implementation of Assessment for Learning in Sweden, Assessment in Education: Principles, Policy & Practice, 22(1), 104-121. DOI: 10.1080/0969594X.2014.970612 Klapp, A. (2015). Bedömning, betyg och lärande. Lund: Studentlitteratur. Kvale, S. (1997). Den kvalitativa forskningsintervjun. Lund: Studentlitteratur. Kyrö, P. (2005). Entrepreneurial learning in a cross-cultural context challenges previous learning paradigms. In: P. Kyrö & C. Carrier (Eds.). The Dynamics of Learning Entrepreneurship in a Cross-Cultural University Context. Tampere: University of Tampere. Lackéus, M., & Moberg, K. (2013). Entreprenörskapsutbildning – Från ABC till Phd. Entreprenörskpsforum. Lackéus, M. (2015). Entrepreneurship in Education – What, why, when, how. Entrepreneurship 360. Background paper. Paris: OECD Publishing. Lackéus M., Lundqvist, M & Williams Middleton, K. (2015). Opening the Black Box of Entrepreneurial Eduction – outline of an app-based research project. Peer

reviewed conference contribution, 3E Conference; Lueneburg Germany, April 22-24 Lackéus, M. (2016). Value Creation as educational Practice – Towards a new Educational Philosophy grounded in Entrepreneurship. Doctoral Thesis, Chalmers University, Gothenburg. Leffler, E. (2006). Företagsamma elever – Diskurser kring entrepreneörskap och företagsamhet i skolan. Doctoral thesis, Umeå University. Liedman, Sven-Erik (2011). Hets! En bok om skolan. Stockholm: Bonniers Förlag. Lindgren, Joakim (2012). Judgment or evidence?: Knowledge in Swedish Schools inspection, paper presented at the 40th Annual Congress of the Nordic Educational Research Association, 8-10 March 2012, Copenhagen Lindgren, J., Hult, A., Segerholm, C. & Rönnberg, L. (2012). Mediating school inspection – Key dimensions and keywords in agency text production 2003-2010, Education Inquiry 3(4), 569-590. http://www.diva- portal.org/smash/get/diva2:572937/FULLTEXT01.pdf Lindster Norberg, E-L. (2016). Hur ska du bli när du blir stor? En studie i svensk gymnasieskola när entreprenörskap i skolan är i focus. Doctoral Thesis, Umeå: Umeå University. Lundahl, C. (2011), Bedömning för lärande. Stockholm: Norstedts. Lundahl, C., Hultén, M., Klapp, A., & Mickwitz, L. (2015). Betygens geografi – forskning om betyg och summativa bedömningar i Sverige och internationellt Stockholm: Vetenskapsrådet. Mahieu, R. (2006). Agents of Change and Policies of Scale. Doctoral thesis. Institutionen för svenska och samhällsvetenskapliga ämnen, Umeå University. Marshall, B., & Drummond, M.J. (2006). How teachers engage with assessment for learning: Lessons from the classroom. Research papers in Education, 21(2), 133- 149. http://dx.doi.org/10.1080/02671520600615638 Mehrens, W. A., & Kaminski, J. (1989). Methods for improving standardized test scores: Fruitful, fruitless or fraudulent? Educational Measurement: Issues and Practice, 8(1), 14-22. http://onlinelibrary.wiley.com/doi/10.1111/j.1745- 3992.1989.tb00304.x/pdf Natriello, G. (2009). High stakes testing and teaching to the test. In: L. J Saha & A.G Dworkin (Eds.), International handbook on teachers and teaching. P. 2. New York: Springer. Nusche, D., Halász, G., Looney, J., Santiago, P & Shewbridge, C. (2011). OECD Reviews of evaluation and assessment in education: Sweden. OECD. OECD (1989). Towards an “Enterprising” Culture: Challenge for Education and Training. Paris: OECD/CERI. OECD (1998). Fostering Entrepreneurship. The OECD Job Strategy. France: OECD. Olovsson, T.G. (2015). Det kontrollera(n)de klassrummet – Bedömningsprocessen i svensk grundskolepraktik i relation till införandet av nationella skolreformer. Doctoral thesis, Departement of Education, Umeå University. Olovsson, T.G. (2017). Betygens införande i årskurs 6. I A. Hult & A. Olofsson (Red.), Utvärdering och bedömning i skolan (s. 170-184). Stockholm: Natur och Kultur. Ozga, J., Dahler-Larsen, P., Segerholm, C. & Simola H. (Eds.) (2011). Fabricating Quality in Education: Data and governance in Europe. London and New York: Routledge. Ozga, J. (2017). Education policy should not be driven by performance data. Nature Human Behviour, 1(0014). http://www.nature.com/articles/s41562-016-0014 Power, M. (1999). The Audit society: rituals of verification. Oxford: Oxford University Press.

Røe Ødegård, I K. (2014). Pedagogiskt entreprenørskap i lӕrerutdanningen: en framtidsrettet lӕringsstrategi, Norway: Cappelen Damm. SAIDE (2010). Draft outline of Bernstein’s concepts. South African Institute for distance Education. http://www.saide.org.za Samuelsson, P. (2012). Betygshistorik. In K. Nordgren, C. Odenstad & J. Samuelsson (Eds). Betyg i teori och praktik. Malmö: Gleerups. SOU 2013: 30. SOU 2013:30. Det tar tid – om effekter av skolpolitiska reformer. Stockholm: Utbildningsdepartementet. Stobart, G. (2008). Testing times. New York: Routledge. Suurtamm C., & Koch M. J. (2014). Navigating dilemmas in transforming assessment practices: experiences of mathematics teachers in Ontario, Canada. Educational Assessment, Evaluation & Accountability, 26(3), 263-287. http://link.springer.com/article/10.1007/s11092-014-9195-0 Swaffield, S. (2011). Getting to the heart of authentic assessment for learning. Assessment in Education: Principles, Policy & Practice, 18, 433–449. http://dx.doi.org/10.1080/0969594X.2011.582838 Swedish Government Office (2009). Strategi för entreprenörskap inom utbildningsområdet. Stockholm: Regeringskansliet. Swedish National Agency for Education, (SNAE), (2011a). Läroplan för grundskolan, förskoleklassen och fritidshemmet. Stockholm: Skolverket. Swedish National Agency for Education, (SNAE), (2011b). Kunskapsbedömning i skolan: Praxis, begrepp, problem och möjligheter. Stockholm: Skolverket. Swedish National Agency for Education, (SNAE), (2015). Skapa och Våga Om entreprenörskap i skolan. Stockholm: Skolverket. Swedish Research Council, (2002). Forskningsetiska principer inom humanistisk- samhällsvetenskaplig forskning. Stockholm: Vetenskapsrådet. Available at http://codex.vr.se/texts/HSFR.pdf ; 2016-01-15 Torrance, H. (2007). Assessment as learning? How the use of explicit learning objectives, assessment criteria and feedback in post-secondary education and training can come to dominate learning. Assessment in Education: Principles. Policy & Practice, 14(3), 281-294. http://dx.doi.org/10.1080/09695940701591867 Torrance, H. (2011). Using assessment to drive the reform of schooling: Time to stop pursuing the chimera? British Journal of Educational Studies, 59 (4), 459- 485. http://dx.doi.org/10.1080/00071005.2011.620944 Tveit, S. (2014). Educational assessment in Norway, Assessment in Education: Principles, Policy & Practice, 21(2), 221-237. http://dx.doi.org/10.1080/0969594X.2013.830079 Wahlström, N., & Sundberg, D. (2015). Theory-based evaluation of the curriculum Lgr 11. Uppsala: IFAU. Watt Boolsen, M. (2007). Kvalitativa analyser, Malmö: Gleerups. Wikström, C. (2006). Education and assessment in Sweden. Assessment in Education: Principles, Policy & Practice, 13(1): 113-128. http://dx.doi.org/10.1080/09695940600563470 Wiliam, D., & Thompson, M. (2007). Integrating assessment with instruction: What will it take to make it work? In C. A. Dwyer (Ed.), The future of assessment: Shaping teaching and learning (pp. 53–82). Mahwah, NJ: Lawrence Erlbaum. Wiliam, D. (2011). What is assessment for learning? Studies in Educational Evaluation 37(1), 3–14. https://www.udir.no/globalassets/filer/vurdering/vfl/andre- dokumenter/felles/what-is-assessment-for-learning1.pdf

International Journal of Learning, Teaching and Educational Research Vol. 16, No. 1, pp. 161-175, January 2017

Review in Form of a Game: Practical Remarks for a Language Course

Snejina Sonina University of Toronto Scarborough Toronto, Canada

Abstract. The article summarizes my own experience of conducting reviews for exams in French language courses. Through more than ten years of teaching French language and linguistics, I have developed pre- exam review sessions based on game models to help my students repeat term lessons in an engaging and memorable atmosphere. The article is intended as a practical guide for creating and organising such a review game. It is constituted of seven parts with subtitles, which should make it easy to navigate. After a short introduction describing challenges encountered in end-of-year reviews, the reader will find a general description of the game in part 2 and examples of one complete round of it in part 4, while the interceding part 3 will provide practical tips on support materials, which themselves make up part 5. These supports deal with ways of formulating questions and displaying answers using animations in PowerPoint presentations. The final section, part 6, will offer advice on using a course website and classroom aids to increase student attendance and encourage more effective classroom participation. The brief conclusion enumerates the beneficial effects of the game, underlines the value of a well-prepared PowerPoint presentation, and gives examples of students’ positive feedback on this format of material review.

Keywords: language teaching; technology and PowerPoint; game-based approach; final review

Introduction Year after year, teachers and students face the same situation: as the end of the term nears, inevitably bringing with it final exams, there comes a need to review the entire term’s material in one or two hours. Review sessions, meant to help a nervous and busy student population in its preparations, turn into proto exams that sometimes bore, never manage to address all questions and concerns, and most often only heighten students’ anxiety. To counter this situation, I have throughout the past fifteen years developed review sessions based on game models to engage my students and help them repeat term lessons with a lighthearted but effective touch. Having noticed that games and game-like approaches are used rather minimally in university teaching practices, I turned to the didactic literature

available on the subject in our Ontario Institute for Studies in Education (OISE). Despite scarce sources on the matter, those few authors who do examine play activities provide ample confirmation of the effectiveness of games — not only for teaching language or for instructing young people but for teaching in general (though they do utter some words of caution too). So before introducing and offering up my own game for reader review, let me provide a brief overview of recent research on the didactic value of games in the following paragraphs. I was pleased to see that contemporary teachers and researchers still refer to Johan Huizinga and his Homo Ludens: The Playing Man or Man the Player, a work that influenced my own approach to education years ago. Justin Hodgson (2013, 46) uses Huizinga’s arguments to justify his design of a whole course based on a video game, while Robert J. Blake (2013, 163–64) insists on the importance of gaming environments for learning when he follows Huizinga’s argument that the ―deep-seated imperative to play‖ is embedded in our human nature. Since Huizinga outlined the crucial importance of game-like activities for humans from ancient civilizations to modern-day cultures, I have always considered playing an appropriate activity for the university classroom. Nonetheless, I still relegated it to a secondary role even in language-practice courses. This approach seems to reflect the general situation in our didactics: most authors writing about play activities deal either with younger learners or concentrate on computer games and computer-assisted language learning (CALL) as a supplement to a wider-ranging education. The researchers all agree that the games add to the pleasure of and motivation for learning. The authors of one of the most recent volumes dedicated to engaging activities in school language classes offer many variations of word games and brand-game formats as ―a motivating way to contextualize language learning‖ based on their association with ―fun, relaxation, and pleasure‖ (Watts and Phillips, 2014). These same reasons — fun and motivation — are often put forth in favour of educational video games. In this domain, James Paul Gee’s book What Video Games Have to Teach Us about Learning and Literacy, which proffers thirty-six learning principles used in digital games that, according to Gee, should exist in any good learning environment, became as influential as Huizinga’s Homo Ludens. Although Gee’s tome deals mainly with concepts specific to digital games, such as role models, false identities, or command obedience, all researchers promoting game use in teaching point to one characteristic of games that Gee deems to be of paramount importance: A game is always engaging, and thus it can ―engage in situated and embodied learning,‖1 therefore ―transform[ing] language learning experiences‖2 and contributing to ―deep, sustained, and transferable learning.‖3 It should not be left unsaid that a few voices do warn against too much play. For example, Jackson, Dempsey, and McNamara question the balance between the efforts necessary for mastering game skills and the learning outcomes,4 while Robert J. Blake urges educators implementing games into the

1 Hodgson (2013, 48) 2 Sykes and Reinhardt (2013, 3) 3 Filsecker and Bündgens-Kosten (2012, 50) 4 Jackson, Dempsey and McNamara (2012, 115)

curriculum to ask themselves a question: ―Are students learning to play or playing to learn?‖5 This question uses the title of an online article by H. C. Arnseth and follows the latter author’s argument. Although these concerns appear more relevant for digital games than for my in-class game-like activities, I, too, have to respond to this question. As I hope to demonstrate in the description of the game below, I give my answer with certainty in alignment with another influential book on games. Much like Prensky6, who, with his Games2train, teaches serious business content with the help of game-like speed and graphics, I place much greater importance on the content to be learned through the games than on the pleasure they provide. A report from the Summit on Educational Games by the Federation of American Scientists, quoted by Robert J. Blake, lends further evidence to the value and importance of engaging learning activities: ―Students only remember 10 percent of what they read; 20 percent of what they hear; 30 percent if they see visuals; 50 percent if they watch someone model something while explaining it; and 90 percent, if they engage in the job themselves, even if only as a simulation or game.‖7 In this article I will summarize my own experience of conducting reviews for exams in French language courses. The main goal of this undertaking is to encourage language teachers to use ludolinguistics8 at least in review sessions by showing that a little extra work is worth the effort. My secondary objective is to engage in the discussion about the use of PowerPoint in the classroom by forwarding a few arguments in its favour. Since the article is intended as a kind of practical guide for organising a review game, it seems sensitive to introduce the contents of its seven parts in form of an outline with keywords in bold to make it easier to navigate. The reader will find a general description of the game in part 2 and examples of one complete round of it in part 4, while the following questions will be addressed in each part in the following order: 1. What kind of challenges do we face in the reviews for final exams? 2. How can a game transform a tiresome review into an engaging activity? 3. How do different support materials influence the effectiveness of the game? 4. What is the best way to formulate game questions? 5. What is the best way to display answers and how can PowerPoint help? 6. What kind of practical tips can I offer based on my experience? 7. Why is the game an excellent way to conduct a review for a final exam?

1. Challenges of reviews for final exams In final review classes, we often encounter two main challenges: the immense quantity of material to cover and an overall nervousness about the future exam. To review the material for the final exam in a helpful manner usually means to

5 Robert J. Blake (2013, 175) 6 Prensky (2007) 7 Robert J. Blake (2013, 164) 8 This term describing play activities in language teaching was coined by Anthony Mollica. See Anthony Mollica, Ludolinguistica e Glottodidattica (Guerra-Soleil, 2010).

discuss the format of the exam, to give samples of questions, to offer strategies for dealing with them, and to remind students of possible mistakes or traps. It is difficult to fit all this into a one-hour class, especially at the very end of the semester, when we cannot count on students to practise on exams from previous years or to formulate pertinent questions based on their studies—most of them are too busy with end-of-term tests and assignments. The task of providing useful strategies for exam preparation is not an easy one either: all the ―shoulds‖ and ―musts‖ we feel we should offer (as in ―you should cope with limited time, you must decide how many hours you need to study, you should never give up, etc.‖9) clearly only add to students’ anxiety and, given in the form of a list, do nothing but overwhelm. Even if phrased as friendly advice or personal experience, these warnings act mainly on the surface of the mind and do not sink deeper without guided experience: students simply do not know how to fill these abstract recommendations with content. When, after a few conventional review classes, I recognized the difficulty of this task if I truly wanted to contribute to student success, I decided that a practical activity with some positive emotional value would be a logical solution for the problem.

2. Review as a game One possible way to provide real-life experience of time management, which can be an issue for students both during the exam and in preparation, is to create a kind of competition. To further create a positive atmosphere and increase students’ motivation in a competition, treats like chocolates offer themselves as a simple but useful solution. So when my conventional review sessions failed to produce the desired results, I began trying out a few different review games— from a simple question lottery to a team competition—all encouraged with various Belgian chocolates, which the class infallibly welcomed with a loud ―Wow!‖ Yet these initial activities did not work for all students: the games did not address well either the quantity of the material or the students’ varying levels of proficiency in French. Most often, while the best students would answer and win chocolates for themselves or their teams, the rest of the class could not follow. I could see it in their eyes but could not afford to dwell longer on each question. So while these games definitely added some fun to the reviews, they worked mainly for those who would get good marks anyway, leaving the students most in need of review with even more anxiety and despondency, expressed in sighs and desperate looks. A few years ago I finally developed a format for a final-review game that truly works for everybody. The format closely resembles that of Who Wants to Be a Millionaire?, whose French title, Qui veut gagner des millions?, can be easily transformed into Qui veut gagner des chocolats?. Just like the well-known television game, the review game requires a main player, who must answer a series of questions, and an audience, from which the player is selected and which watches the answer progress closely and

9 Cf. James L. Clare and G. D. Morison, Hints on Study and Exam Techniques, http://www.casact.org/admissions/syllabus/2016-new/index.cfm?fa=hints, December 07, 2016.

is ready to participate in questions through the ―ask-the-audience‖ lifeline. The game’s particular purpose in school of course also warrants some differences from the TV show. For instance, a rather frequent change of main players and activities keeps all students on their toes and interested. Let me briefly describe the individual steps of the review game: 1. The Rules. I explain the game’s purpose and its rules, reminding students that the questions are formulated as on the exam; I also show the chocolate prizes. 2. The Fastest-Finger Question. We run through one question to make sure that everybody understands how the game works, and then we play the first question for real and select the first main player: the one who answered it correctly first. 3. The Rounds. The main player has a chance to answer four questions, after which the round is over: the main player receives a chocolate prize, and everyone else in class gets ready to participate in the next round of the fastest-finger question that will provide the next main player. Within an hour it is usually possible to go through about six rounds of four questions each. 4. The End. I reward all students with chocolates earned for participation in the game and in the term, remind them that the game will be posted on the course website, and encourage them to practise one round of the game at a time before the exam. One further change to the rules of the original television game became necessary while playing the game in class for the first time. I had tried to be strict about correct and incorrect answers and to withdraw a first chocolate won in case of an incorrect answer to a following question, as it is done with money in the television game. This did not go well: the whole class almost sang ―Oh, no!‖ when I announced the rule for the first time. I realized immediately that since the point was to review grammar in an enjoyable environment it was better not only to leave all chocolates in the hands of the players but also to help them win more. So now I sometimes even suggest to call on a friend or to use one more ―ask-the-audience‖ lifeline under the pretext that the question is too tricky. The latest game rules thus look like this:10

10 The text translates as follows: ―The rules of the game: 1. Write your answers to the ―fastest-finger‖ question on your yellow paper – 2. Raise you hand AFTER having written them – 3. If you are the first, you’ll need to answer four questions – 4. If you answer two questions correctly, you’ll get a small chocolate – 5. If you answer all four questions correctly, you’ll get a big chocolate – 6. You can ask the class to help you.‖ NB: I will not translate question samples: they do not work for English.

Le règlement du jeu :  4. Si vous répondez bien à deux questions, vous gagnerez un petit 1. Écrivez vos réponses à la question chocolat. « qui est le premier » (= fastest-finger) sur votre papier jaune.  5. Si vous répondez à toutes les questions, vous gagnerez un grand chocolat. 2. Levez la main APRÈS les avoir écrites.

 6. Vous pouvez demander à la classe 3. Si vous êtes LE PREMIER, vous de vous aider. devrez répondre à quatre questions.

7 8

3. Different support materials: from paper to PowerPoint presentations It is possible to use different support materials to organize this game. Even in the years when not all classrooms were equipped with computers and projectors, the game worked relatively well. In this case, the rules were written on a poster and the questions printed out and distributed to each student. In the paper version, it is necessary to fit all questions for one round on a single page, because the main player should not see the questions in advance: once the main player for the round is selected, all the students can turn the page and work with the questions, but the main player gets the questions one by one from the instructor. It is also important to write the correct answer on the blackboard afterward, because there are always students who miss oral instructions. One of the advantages of paper is that it leaves a hard copy with examples in students’ hands, which is helpful if we can trust that they marked the answers correctly and added appropriate comments about incorrect answers and possible traps. The use of transparencies adds two more advantages: no time is wasted with writing on the blackboard, and it is also easier to comment on the wrong answers, because the difficulties of the question can be pointed at on the screen in front of everyone. Plus, if an erasable marker is used to indicate correct answers, the transparencies become recyclable. In classrooms equipped with computers and projectors, one can start the review session by playing a short video with the French opening of Qui veut gagner des millions? to set the mood and to facilitate the explanation of the rules.11 Once the students have recognized the prototype of the proposed game, one can proceed to a PowerPoint presentation that greatly eases the flow of the game in comparison with the use of paper and transparencies. Moreover, the use of PowerPoint facilitates the recycling of questions and the changing of content, which helps to constantly improve the quality of the review. Well-prepared PowerPoint slides can also enhance the quality of the presentation with colours and animations. I will give more examples of and explanations for such quality

11 See Qui veut gagner des millions générique TV France 2010, http://www.youtube.com/watch?v=K8LFAHkaO30&feature=related, December 07, 2016.

improvements in part 5. In the following part, I will provide examples of questions that tolerate any support: paper, transparencies, or PowerPoint slides.

4. Examples of questions The game questions provide the means to discuss the format of the exam and to review the material. There are two main types of questions: fastest-finger questions, used to choose the main player for each round, and actual game questions (most often concerning grammar), answered by the main player or by the class as a lifeline. In terms of preparation for the final exam, fastest-finger questions serve to review vocabulary learned during the semester. In a language course they can be of two main types: for verbs, reorganizing actions in a logical order and, for nouns, adjectives, and adverbs, rearranging a list of given words according to their meaning—size, value, position, or the like, starting from the biggest, the cheapest, the lowest, etc. Here are two examples, one for each type, for a game in a first-level French course (FREA01).  Réarrangez ces actions selon leur ordre logique!  a) se laver  b) s’en aller  c) se réveiller  d) se lever12  Réarrangez ces adverbes de quantité selon la quantité exprimée en commençant par la plus petite!  a) beaucoup  b) peu  c) trop  d) assez 13 When the students are writing their answers to the fastest-finger question, I usually wait until I see at least three hands, remember in what order they rose, and then collect the pieces of paper with answers from the three fastest students. This is necessary in case the first and the second students get it wrong. In the rare case that all three answers are incorrect, I offer a collective game to the class and make students vote for each question, that is, raise their hands in support of the answer they deem correct. In such a case, I distribute the small chocolates after this round, rather than at the end of class, as they represent a reward for everybody’s participation. The grammar questions are used as a tool to discuss the way in which questions on the final exam will be phrased, to review the main grammar points, and to warn about possible traps. Once the main player has been chosen, I explain the grammar task and the part of the exam to which it corresponds. For example, in the round for the review of pronouns, the player will be managing

12 The text translates as follows: ―Rearrange these actions according to their logical order! a) get up; b) go out; c) wake up; d) get washed/take a shower‖ 13 The text translates as follows: ―Rearrange these adverbs of quantity according to the quantities that they represent starting from the smallest! a) much/many; b) little/few; c) too much/many; d) enough‖

two types of tasks that correspond to the parts D and E of the final exam. First, I show and read the task in the exact wording of the exam:  D. Les pronoms personnels. Remplacez les mots soulignés par « y », « en » ou un pronom personnel. Mettez le pronom à la place appropriée. Faites tous les changements ou les accords nécessaires.14 (10 points) Then I display the first question with multiple-choice answers:  1. Est-ce que le singe a bouffé les bananes?15  A) Il les a bouffés  B) Il en a bouffé  C) Il les a bouffées Once the answer is given by the main player, I ask the class why the two other answers are incorrect and remind them of the dangers of forgetting about agreement (A and C) or not paying attention to the article (B and C). Then comes the second question:  2. Est-ce que je parle à mon chat de mes problèmes?16  A) J’en parle  B) Je lui parle des miens  C) Je lui en parle Once the correct answer is given, the class is again invited to explain why in this case the answer A is insufficient and B does not fit this exercise. After the last question that corresponds to one of the exam exercises, I ask whether students have any particular difficulties with this material. I usually look at the students who did not fare well on grammar tests and invite questions from them; then, depending on the feedback, I briefly review the most difficult points or the general guidelines. For example, in this instance, I could go back to the previous question or to the formulation of the task to remind students that direct object pronouns are identical to definite articles; that indirect pronouns are lui and leur; that en is used to replace nouns with the preposition de, the indefinite article, and quantities; and that y stands for places and objects with the preposition à. After that, we continue with the next task, phrased exactly as on the future exam:  E. Les autres pronoms. Remplissez les blancs. Utilisez les pronoms possessifs, démonstratifs, interrogatifs. 17(10 points) Then follow the question and its possible answers:  3. Je ne prends pas d’autres médicaments que ______qu’on m’a prescrits.18

14 The text translates as follows: ―D. Personal pronouns. Replace the underlined words by ―y‖, ―en‖ or a personal pronoun. Put the pronoun into a proper place. Make all necessary changes and agreements.‖ 15 The question translates as follows: ―Did the monkey devour the bananas?‖ – the correct answer translates ―He devoured them‖ 16 The question translates as follows: ―Do I talk to my cat about my problems?‖ – the correct answer translates ―I talk to him about them‖ 17 The text translates as follows: ―E. Other pronouns. Fill in the blanks. Use possessive, demonstrative, and interrogative pronouns.‖

18 The correct answer translates as follows: ―I do not take any other pills than those that was prescribed‖

 A) ce  B) ceux-ci  C) ceux This, too, is followed by an in-class discussion and a warning about the possible confusion of ce and ceux (A and C) and a brief reminder about the use of –ci and –là (B) with demonstratives. Finally comes the last question for this player:  4. – Est-ce que c’est ta valise? – Oui, c’est ______.19  A) celle-là  B) quelle  C) la mienne Once the main player gets the chocolate prize and applause, I go back to the original task phrase and talk with the class about particular difficulties they might have with these pronouns. If there are no burning questions, I will simply remind them of the agreement of gender and number with those of the antecedent using question 4 and replacing ta valise with ton sac and tes affaires. At the end I always remind my students that despite the multiple-choice format of the review game, the final exam does not offer various answer options, and that I am offering these choices here only to explain common errors, often using incorrect answers from students in previous years.

5. Examples of answer displays20: Custom animation The most important aspect of the game is to make sure that all students get the correct answer to each question and understand why the other answers are unacceptable. Unlike paper and transparencies, PowerPoint offers many opportunities for the effective presentation of answers. For example, for comments and warnings about incorrect answers I use call-outs that gradually appear on the PowerPoint slide21. In the case of the first question of the round described above, animations would appear in this order:

19 The correct answer translates as follows: ―Is it your suitcase?‖ – ―Yes, it is mine‖

20 The pictures below illustrate various techniques for displaying answers and other useful information, the translation of the French text, which contains mostly examples understandable for an English speaker, is therefore unnecessary.

21 In terms used by Sophie Dufour and Chantal Parpette in their article on the use of PowerPoint presentations in university courses (2014, 25), the slide itself represents ―the object of the explanations‖ while the call-outs correspond to ―the extracts of the explanations themselves.‖ I believe that such use of the call-outs solves the problem pointed out by Dufour and Parpette because they do not distract students’ attention from oral explanations, as slides filled with longer explanatory extracts would do (2014, 28), but on the contrary draw the attention to the main points.

Pronoms 1. smiley indicating the

 1. Le singe a bouffé les bananes! correct answer

= m.pl. 2. call-out for ―les‖ in C  A) Il les a bouffés  B) Il en a bouffé Remplace « des 3. call-out for ―en‖ in B  C) Il les a bouffées bananes » 4. call-out for ―-es‖ in C = f.pl.

Remplace « les 5. call-out for ―-s‖ in A bananes » 11

In the fastest-finger questions, the screen may not only display the correct order of the words but also some images that augment the visual salience22 of the material and thus reinforce the memorising of vocabulary. The pictures below show the same slide on which the answers appear in the correct order and with illustrations, right after the main player is selected. Qui est le premier? Qui est le premier?

 Réarrangez ces animaux selon leur  Réarrangez ces animaux selon leur taille en commençant par le plus taille en commençant par le plus petit! petit!

 a) une vache  a) une vache

 b) un lapin  b) un lapin

 c) un cochon  c) un cochon

 b c a

18 19

In actual game questions, several explanations can be added to the answers; they can explain the quantity of points lost or gained or even provide brief definitions of complex concepts, as in the following slides taken from a course on commercial French (FREB18):

Vocabulaire : Synonyme Qui est le premier? (transports)

Une valeur mobilière  Réarrangez ces formules selon les frais du fournisseur en commençant par la plus coûteuse ! 1 Un titre 1 point A 2 Une action 0,5 point a) port payé Le fournisseur paie tout C b) port dû Le client (acheteur) paie tout 3 Une sécurité 0 Franco bord (free on board) B c) FOB Franco bord (free on board)

23 20

22 The importance of the salience of the material to be learned is discussed by Maria Dakowska in In Search of Processes of Language Use in Foreign Language Didactics (2014, 293)

If the explanations are too bulky to fit the 6 x 6 PowerPoint rule,23 one can use another slide to illustrate the point. I usually indicate the correct answer with a smiley and change the background colour for the explanatory slide to show that it is not part of the game:

Question de compréhension p.109 + étymologie Quelle réduction est accordée 1 Escompte = quand on paie au comptant au client qui paie au comptant ? = ex (=out) + (argent) comptant 2 Ristourne = un remboursement 1 Escompte retourné au client (à la fin de l’année) 2 Ristourne 3 Remise = le prix est remis (reconsidéré) pour quantité, service etc. 3 Remise 4 Rabais = le prix est abaissé à cause 4 Rabais d’un défaut (ou hors-saison)

26 27

The most important part, however, is to choose the multiple-choice answers wisely. Each one should serve a particular purpose: to illustrate a range of grammatical or lexical phenomena,24 to point to major difficulties, or to reveal possible confusions. For this reason, I never offer simply wrong or impossible answers—they would not have any explanatory force and might even lead to an unconscious memorisation of mistakes.

6. Practical advice plus templates, clickers, and natural pauses To run the game smoothly, it is important to do a few things in advance: a) assign students vocabulary review as homework—it is a smaller task than having to review the entire course material, so there is more probability they will do it, especially if one mentions that it will be important to win chocolates; b) post the format of the final exam on the course website and announce that the final and VERY important review will take place in the last class; c) buy some Belgian or Swiss chocolates of different sizes—the smaller ones as encouragement prizes and the bigger ones as main prizes: so about six big chocolates and enough small chocolates for each student in the class, plus about ten more for intermediate prizes and other unexpected encouragement needs; d) cut enough pieces of colour paper, 2‖ x 2‖, to distribute to all students— they will be necessary to jot down the answers to the fastest-finger questions and to draw attention to the rules. They should be small to be

23 A well-designed and readable slide contains no more than six lines, with no more than six words in each line.

24 For example, the pronouns celle/quelle/mienne, offered as answers for the fourth question in 4, give examples of all types of pronouns— demonstrative/interrogative/possessive—required for the reviewed exercise; the lexical items in the slide above offer the whole range of synonyms for the term discount.

easy to collect and coloured to be easy to spot and to be referred to as ―the yellow paper,‖ for example. The Internet offers a lot of help for those who want to create classroom activities, including sites that provide templates for all sorts of games. Yet in my experience readymade templates have rarely helped. For example, the template for Who Wants to Be a Millionaire would force me to offer four possible answers for each question. This is undesirable in terms of time management, both in slide preparation and in classroom timing, and also unjustified in terms of possible mistakes worthy of discussion; usually two are enough to exemplify reasonable errors. My own simple PowerPoint slides offer more flexibility and thus suit the needs of the game better. After a few games, I also realised that there is no need to economise each minute ―wasted‖ on the technical needs of the game. At first, I had students help me collect pieces of paper with answers and bring chocolates to the winners to save a few seconds and to be able to do more rounds of the game: the idea was ill-advised. In fact, the natural pauses that occur when I walk toward the winner to present the chocolate, for example, actually allow students to breathe: to write down the correct answer still displayed on the screen, to clap, or to simply recover briefly from the quite intense tempo of the game. It appears necessary to give students some room for just breathing in any class, but during a final review, this is even more the case because of the quantity of the material covered: it is better to combine two exam exercises in one round to be able to go through all of them than to rush through more rounds without having enough time to dwell on the main points and to give students a few seconds to digest the information. Finally, to improve students’ participation, I always encourage as many ―ask-the-audience‖ situations as possible, because they invite the whole class to become actively involved in the game. For this purpose, clickers, which are becoming more and more popular in university classrooms, work very well. Vos réponses (clickers) Each student presses the letter • a) corresponding to the answer chosen as

• b) correct, and we can see the distribution • c) of choices on the screen. 30 If a class is equipped with clickers and it is essential with a particular question that the majority of the class understands it well, this same game can be conducted without the main player. In the same way, slides inspired by the game can prove very useful in everyday teaching practice, especially for classes that use clickers.

7. Conclusion The described game certainly offers a solution to the review challenges outlined in 1. It allows the discussion of all main grammar points in the very format of the future exam and shows many possible traps in practice. Thus the game solves the problem of an enormous quantity of material through the careful choice of questions and answers, which include typical errors from previous years. Moreover, this type of review allows including game activities into

university courses, which usually do not have any room for such activities because of a tight schedule. I believe that the time deficit is the main reason for the lack of attention to ludolinguistics in the higher education: for example, the most recent volumes regrouping publications concerning didactics in higher education contain only a few articles on game-like activities25 and even though there are many publications that prompt ―innovative teaching and approaches to student learning‖26, very few of them suggest games, and even fewer professors ever have a chance to try games in the classroom27. However, my review game has always been time efficient. Since each student tries to choose the right answer while the main player is taking the time to decide, all students become more aware of grammatical dangers after seeing themselves falling into well-designed traps during the game. The game also helps students maintain their highest level of attention during the review, not only because it offers a diversity of activities but also because everybody can keep up with the tempo thanks to natural pauses and well-displayed information. The game also solves the problem of typical exam anxiety by creating a positive atmosphere of mild competition, by offering challenging questions that provoke a desire to try and succeed, and by offering a sweet reward for participation. It also offers an encouragement for everybody when the chocolates are distributed at the end. It is a great pleasure to see the students positively reacting in class—―Yes!‖, ―I knew it!‖, ―Bien sûr!‖—and leaving class inspired and willing to try the game again at home. The data from surveys that I conduct during the final test in the commercial French course confirm the effectiveness of the review game: no one has ever answered ―no‖ to the question whether the final game helped remember the material on the test. From this point of view, the value of a well-prepared PowerPoint presentation is priceless. Of course, it also contributes to the smooth flow of the game and to better timing, as well as to the quantity and quality of the reviewed material. Most of all, however, it helps by encouraging every student to revisit the game posted on the course website. My parting words usually are: ―See, if you review only one round a day, and one round takes only five minutes—n’est- ce pas?—and then spend a little more time to review the corresponding

25 The collection of articles on Emerging Issues in Smart Learning has only two articles on game-based learning activities – one describing their benefits for listening and speaking, the other for working memory capacity. The Proceedings of the 11th European Conference on Technology Enhanced Learning in Lyon (Adaptive and Adaptable Learning, 2016) contain only three presentations dealing with games but only one of them deals with the game activities in the classroom, while two other treat game-based exercises.

26 Deborah Cartmell and Imelda Whelehan ―A Short History of Adaptation Studies in the Classroom‖ in Teaching Adaptations (edited by Deborah Cartmell) New York : Palgrave Macmillan, 2014, 10.

27 Most of the articles in the Emerging technologies for the classroom: a learning sciences perspective (2013) actually describe educational games in virtual worlds and not in the classroom.

grammar in the textbook, you’ll do great!‖ ―Oui, ça va! Merci! C’est promis!‖, comes the answer.

References Adaptive and Adaptable Learning : 11th European Conference on Technology Enhanced Learning, EC-TEL 2016, Lyon, France, September 13-16, 2016, Proceedings. (edited by Katrien Verbert, Mike Sharples, Tomaž Klobučar)Cham: Springer International Publishing, 2016.

Blake, Robert J. Brave new digital classroom: technology and foreign language learning. Washington, D.C. : Georgetown University Press, 2008.

Cartmell, Deborah and Imelda Whelehan ―A Short History of Adaptation Studies in the Classroom‖ in Teaching Adaptations (edited by Deborah Cartmell) New York: Palgrave Macmillan, 2014, 1-10.

Dakowska, Maria. In Search of Processes of Language Use in Foreign Language Didactics. Frankfurt am Main: Peter Lang GmbH, Internationaler Verlag der Wissenschaften, 2015.

Dufour, Sophie and Chantal Parpette. « Les powerpoints dans les cours universitaires : une méthodologie à stabiliser » in Langues, cultures et pratiques en contexte : interrogations didactiques. Paris : Riveneuve éditions, 2014. 23-45.

Emerging Issues in Smart Learning (edited by Guang Chen, Vive Kumar, Kinshuk, Ronghuai Huang, Siu Cheung Kong) Berlin, Heidelberg : Springer Berlin Heidelberg: Springer, 2015.

Emerging technologies for the classroom: a learning sciences perspective. (edited by Chrystalla Mouza & Nancy Lavigne) New York; London: Springer, 2013.

Filsecker, Michael and Judith Bündgens-Kosten. ―Behaviorism, Constructivism, and Communities of Practice: How Pedagogic Theories Help Us Understand Game- Based Language Learning.‖ In Digital Games in Language leaning and Teaching (edited by Hayo Reinders). New York: Palgrave Macmillan, 2012, 50-69.

Gee, James Paul. What Video Games Have to Teach us About Learning and Literacy. New York: Palgrave Macmillan, 2007.

Jackson, Tanner G., Kyle B. Dempsey and Danielle S. McNamara. ―Game-Based Practice in a Reading Strategy Tutoring System: Showdown in iSTART-ME.‖ In Digital Games in Language leaning and Teaching (edited by Hayo Reinders). New York: Palgrave Macmillan, 2012, 115-138.

Hodgson, Justin. Developing and Extending Gaming Pedagogy: Designing a Course as Game.‖ In Rhetoric/Composition/Play through Video Games: Reshaping Theory and Practice of Writing. (edited by Richard Colby, Matthew S.S. Johnson, and Rebekah Shultz Colby) New York : Palgrave Macmillan, 2013, 45-60.

Huizinga, Johan. Homo Ludens: a Study of the Play-Element in Culture. Boston: Beacon Press, [1972], 1950.

Mollica, Anthony. Mollica, Anthony. Ludolinguistica e Glottodidattica. Perugia: Guerra- Soleil, 2010.

Prensky, Marc. Digital Game-Based Learning. St. Paul, MN: Paragon House, 2007.

Sykes, Julie M. and Jonathon Reinhardt. Language at Play: Digital Games in Second and Foreign Language Teaching and Learning. Boston: Pearson Education, 2013

Watts, Catherine and Hilary Phillips. Jumpstart! French and German: Engaging Activities for Ages 7-12. New York: Routledge, 2014.