Please cite as: Fletcher, P.R., (1997) Master of Science Thesis - How Students Learn Quantum Mechanics (School of Physics, University of Sydney)

CHAPTER FOUR

METHODOLOGY

4.1 INTRODUCTION The prime aim of this investigation was to develop an instrument to explore the difficulties that students encounter when studying quantum mechanics, and to reveal the mental models students use when solving problems, their interpretations of physical models and their understanding of basic ‘technical’ terminology. The design of this instrument was strongly directed by the materials gathered during the analysis of the preliminary surveys described in Chapter 3. This information assisted in the development of questions and in the selection of appropriate tools for analysis.

4.2 DEVELOPMENT OF THE INSTRUMENT The development of the instrument involved several stages. (1) Selecting the content of the survey questions, (2) identifying the context in which each question could be presented, (3) designing the general layout of the instrument and a suite of tools for analysis of the responses and (4) taking account of a set of administrative constraints. The development process was iterative, in the sense that at each stage the prior steps would be re-evaluated to ensure the integrity of the instrument as a whole.

4.2.1 Content and Context This section builds upon the recommendations identified in Chapter 3. Each of the four content areas was researched with the aim of finding the most appropriate contexts in which to present the final questions.

A literature search of textbooks, research papers and quiz material was undertaken. This data was combined with ideas obtained informally through discussions with academic staff in order to explore novel ways in which to present each piece of content. These discussions provided a set of recommended contextual environments in which the questions could be set.

42 43

The significance of the photoelectric effect The photoelectric effect is of historical importance in establishing the particle nature of light and is presented in the majority of secondary school and introductory tertiary physics text books. The subject matter concerning the two experimental results were explicitly covered in the curriculum of the New South Wales Higher School Certificate and is a focus topic in the Physics 1 and Physics 1A streams at the University of Sydney. The two experimental results covered by secondary and introductory tertiary texts are ‘for monochromatic light above the threshold frequency, the number of electrons ejected from a metal surface increases as the intensity increases’ and the second being ‘that electrons are only ejected for frequencies of monochromatic light above a certain frequency; and for light below this frequency no electrons will be ejected’ (Young, 1992).

The key concepts associated with the results are involved in the way the particle and wave models of light are used to explain these experimental observations. Textbooks present these ideas in several formats, ranging from textual descriptions through to pictorial representations. The textual descriptions usually describe the wave and particle models of light, then describe the processes and predictions that each makes. The pictorial models, such as a “bird on a wire” use this visual scenario to describe the processes involved with each of the models.

From the inspection of examination papers and quizzes1 it is evident that those who wrote these examinations and quizzes expect that a student should be able to clearly describe the observed results of the Photoelectric Effect in terms of the wave and particle models of light.

The nature of the topic and the associated material lends itself to the development of a survey question being constructed utilising an analogy and/or visual model. This would allow exploration of both the students’ understanding of the photoelectric effect and their interpretation and handling of the particle and wave models of light.

1 Upon inspection it was noted that University of Sydney physics quiz and examination papers 1988-1994 contained questions referring to the photoelectric effect.

44

The meaning of Uncertainty The term “uncertainty” has a variety of meanings in different situations. When the word is used in everyday language it can mean doubtfulness, not being confident, an uncertain state or unpredicability. Even within the discipline of physics, uncertainty carries with it different meanings depending on the context in which it is used. For example, the term uncertainty in classical measurement refers to a range of values for which the measured result lies2, arising from unavoidable discrepancies from measurement to measurement.

In quantum mechanics the idea of “uncertainty” was proposed in response to the wave particle duality of microscopic objects. Knowing where an object will be at a particular time is a particle-like measurement. But nature tells us that quantum mechanical objects do not necessarily behave like particles in the sense that if you travelled back in time and repeated your measurement of the object's position at the same moment in time, you would not necessarily get the same answer. The results of such a measurement are said to be “uncertain”.

The preliminary Third Year quiz revealed that there was confusion between the meanings of “uncertainty” and “indeterminism” in the context of quantum mechanics. Therefore it would be appropriate to step back and just investigate the students’ understanding of the term itself, in a context where uncertainty is explicit.

To create the context for a question an imaginary world scenario, similar to that used by George Gamow (Gamow, 1993)3, where quantum effects existed on a bigger scale, would be both novel and create an explicit quantum mechanical environment. A question could then address the meaning of the term uncertainty in relation to making a measurement.

2 Uncertainty in classical experimental measurements is often expressed in the following formats (8 mm +/- 10%) or equivalently (8 +/- 0.8) mm. 3 The book entitled “Mr Tompkins in Wonderland” (first published 1940) written by George Gamow. A chapter of the book has Mr Tompkins exploring a world in which the value of Planck’s constant is many magnitudes larger than in the real world. In such a world the quantum effects that are relegated to the submicroscopic in our world are clearly evident in the Quantum Jungle that Mr Tompkins is exploring.

45

The nature of Waves A good understanding of the wave model and the set of properties associated with the measurement and description of waves is crucial and considered to be assumed knowledge after the completion of first year study at university. This is especially true in the study of quantum mechanics because much of the mathematics and associated concepts are dominated by the ideas contained in wave mechanics. The preliminary Third Year quizzes revealed the following result (see Chapter 2).

The students articulated their understanding of ‘What is a wave?’ using one of a small number of paths resulting in fifty percent of the sample not closing or including in their response the defining properties of interference and/or diffraction. Therefore it is appropriate to repeat the experiment especially with First Year students to see if these same two paths are evident earlier in their academic studies and across a broader cross-section of students. The preliminary study had revealed a series of distracters which could be utilised in a tick-a-box response section. It was expressly decided that the accepted ‘correct’ response of interference/diffraction would not be included in the distracters.

The nature of Energy Levels The idea of an energy level is introduced fairly early in Secondary Schools’ science curricula in topics associated with atomic structure. It happens during discussions concerning the Bohr model of the atom and is revisited during topics in line spectra and the photoelectric effect. Introductory university level physics and chemistry texts further expand the idea to types of orbitals, then to band structures associated with semi-conductor materials. The types of questions supplied in secondary and undergraduate textbook quizzes are mainly limited to simple calculations concerning the differences in energies between levels and questions that ask the student to draw pictures of orbits or energy level diagrams.

It would therefore be interesting to probe what ideas the students hold about these basic ideas and models. Two prominent questions that could be explored are, the very basic question, “What is an energy level?” and one which probes the connection with standing waves “What does it mean for a wavelength to fit into an atom?”. There had been no preliminary research into these questions and therefore

46 open-ended dialogue type questions would be appropriate to elicit a range of student ideas and conceptions.

4.2.2 Guidelines The development of good questions requires that certain guidelines should be followed to make sure that the final instrument will yield valid and analysable responses.

Design and Layout The guidelines presented in Cohen on good practice design (Cohen and Manion, 1994) were considered. The points given were adapted to the environment of the current study to assist in the development of the instrument. The final set of adopted recommendations follow.

The following recommended design and layout practises were adopted:

• The appearance of the survey must look attractive and interesting. • The question sheet will be well laid out on a light buff coloured paper. • The answer booklet will include the question and clear instructions with ample space for the student to write and/or draw their responses. • The placement of a tick in a box will be used where appropriate and questions with parts should be sub-lettered. • Instructions should be on the survey question sheet and in the answer booklet. • The first question should be interesting and stimulating. • The sequencing of the questions needs to be varied to ensure a consistent mix of responses.

Logistics The survey instrument is subject to a number of logistical and administrative constraints:

• The survey will be conducted in a lecture theatre. • The survey must be distributed to, and collected from, approximately 90 students per lecture. • 40 minutes will be given to complete the survey. • The analysis must quickly provide feedback to the students. • The questions require the students to construct carefully thought out answers and provide a rich response. • The survey must provide some mechanism of immediate feedback to the student and lecturer.

47

Form of the questions As reported in the preliminary analysis of the first year multiple-choice questions the responses to the questions give no real measure of a student’s understanding or conceptual development. It was noted that at best you could only be sure of those students who “got it wrong”.

Open ended questions tend to generate a lot of diverse responses that can result in coding and validity problems in the final analyses. The analysis of diverse responses is also very time consuming. As a prerequisite, the survey requires an initial formative feedback mechanism to lecturers and students, therefore a reasonably quick and easy to analyse format will need to be utilised by the researcher. Purely open ended questions are therefore inappropriate.

To strike a medium that allows initial quick analysis of responses it was decided that questions where good distracters are known, be developed as tick-a- box-and- explain format questions. Questions where no information is available to develop distracters should follow an open-ended question format with sufficient prompting to direct and contain the diversity of the responses. This type of open- ended question would hopefully elicit a set of distracters for future tick-a-box-and- explain format questions.

Guideline summary Addressing the constraints and aforementioned guidelines it was decided that one question would be devoted to each area/topic and that the overall size of the question sheet be confined to one page (double sided). The form and style of each question would be as follows: The “Nature of Waves”, “Photoelectric Effect” and “Uncertainty” would take the form of tick-a-box-and-explain because there where known distracters isolated in the preliminary analyses and the “Structure of the Atom” question would be open-ended.

4.2.3 Intended Analysis The data collected from the analysis would be recorded in a spreadsheet. Each row would contain all the information pertaining to an individual student. Each column of the spreadsheet would contain information pertaining to a single

48 item (either a tick response or agreed category). A one [1] in a cell would indicate that the student’s response met the criteria for that item, whereas a zero [0] would indicate they were not a member.

This method of data recording would provide a simple format for comparison between any combination of the items.

Tick-a-box Responses A simple correctness analysis would be performed by comparing the student’s response to the approved sample tick-a-box response. This analysis would in the first instance provide a fast feedback mechanism for the students (an essential requirement of the formative assessment process) and secondly provide timely statistical information for the lecturers.

Open-ended Responses Phenomenographic A phenomenographic analysis would be directed toward examining how the students think about the concepts presented in the responses, seeking to categorise the responses into qualitatively different groupings, based solely on the data contained in what the students say or write.

The responses to each question would be examined independently by each member of the research team in order to identify a provisional set of categories. A meeting would be held and an agreement would be made as to the suitability of the analysis. This decision would depend on whether the phenomenographic analysis of the responses yielded a set of identified codable categories.

Assuming the analysis was to proceed, a final set of categories and an associated set of shared meanings would be developed. A team of three researchers would then independently re-categorise all the responses in accordance with this set of agreed shared meanings. These analyses would then be compared and any differences resolved by discussion. The phenomenographic analysis would provide a preliminary mapping of the students’ perceptions for each of the concepts/models presented in the students’ response.

49

Context A contextual analysis of the responses is considered important because the context in which a student couches their response may be closely related to the mental models they have constructed. It may also be closely associated with the difficulties that students encounter in answering questions. It was expected from the preliminary analysis students would offer their responses in a variety of contexts. For example some would chose to mention properties of a particle (what it has); some would offer a metaphor or pictorial image (what it is like); and some would bring forward experimental evidence (what it does). The appropriate categories would be agreed upon and coded.

Content The use of terminology by a student is an important component of their response therefore the appropriateness of a content analysis would be assessed by the research team. A record of all the terms and ideas presented in each response would be recorded.

Correctness The correctness analysis of the written responses would not be considered as important as the preceding analyses. A simple “correct” or “incorrect” would be recorded against each written response. Therefore a marking scheme would not be developed for each question. Correctness would be gauged by comparing the student’s response to the sample answer. A response would be considered “correct” if it was equivalent to the sample answer; an additional contradictory statement or lack of detail would constitute an “incorrect” response.

Trends in Responses The initial data analysis will consist of the generation of simple matrix and graphical representations of a number of combinations of items. By examining the resulting diagrams it is hoped to reveal trends that can be investigated in more detail.

For example, the data for a question would be sorted firstly by the phenomenographic, context and content categories then a stacked histogram would be constructed against each item comprising of the numbers of tick-a-box responses.

50

This graph would show the types of evidence that the students give in support their tick-a-box choice.

4.2.4 Preliminary wording of the questions The draft set of questions was formulated within the guidelines and identified constraints. The initial draft is outlined below.

Significance of the Photoelectric Effect The question would provide the students with all the information about the photoelectric effect as presented in an undergraduate physics text book. A pictorial model of a “bird on a wire” would form the basis of the question. The particle model — represented by throwing stones at the bird to knock it off; and the wave model — where shaking the wire dislodges the bird. The students would then be asked to choose the most appropriate tick-a-box and explain the following key observations in terms of the “bird on a wire” models.

Part A) For light of sufficiently high frequency the number of electrons ejected per second increases as the intensity of the light increases.

Part B) Electrons are only ejected from the metal surface for frequencies of light above a certain frequency.

The tick-a-boxes would consist of the four possible combinations of the two models. The wave model not particle model; particle model not wave model; both the wave and particle models and neither the wave nor particle model. The student would then be asked to support their choice by explaining their answer in terms of the bird on a wire models.

Meaning of Uncertainty The student would imagine they are in a quantum mechanical world. They would be asked to consider a measurement involving catching a bus. Their timetable says the bus will arrive at 9:00am. The student would then be asked to explain what is meant by the bus having an associated uncertainty, in terms of being in a quantum mechanical world. The correct answer would be included in the tick- a-boxes, for which the distracters would be derived from the preliminary analyses.

51

Nature of Waves The student would be provided with information about what constitutes a particle and then be given several examples of waves. They would then be asked to tick the box that most clearly describes what is meant by “something is a wave”. The Tick boxes would not include the “correct” response superposition / diffraction / interference. Each of the tick options would utilise the distracters reported in the preliminary analyses. The final “None of the above” option would be therefore the “correct” response. They would then be asked to support and explain their answer.

Nature of Energy Levels The student would pretend they were listening to a conversation between fellow class members discussing concepts raised in the previous lectures. The conversation would include : Spectral lines are strong evidence for energy levels; Bohr’s model combined several earlier ideas; de Broglie proposed that electrons are also waves. The conversation would then turn to the student, who would be asked whether they know “What is meant by energy levels?” and “What is meant by wavelength fitting into an atom?”. The question would be open-ended and not include tick-a-box options.

4.2.5 Final Iteration The evolution of the final instrument involved many iterations as each separate question developed within the context of the instrument as a whole. A draft of the instrument was produced and prepared for final approval. The instrument was presented to a panel of physicists and educational researchers for comment.

Two meeting were held, one consisting of six senior physics lecturers and a second consisting of two educational researchers. The Physicists and the Educational researchers were pleased with the form and content of the final questions. Only one minor detail was raised by two of the physics lecturers in relation to the photoelectric effect. It was suggested that the additional statement “that the incident light was monochromatic” be added into the question’s description.

52

Interestingly, during the discussions several heated debates erupted between the physicists. The first was in regards to the question “What is a wave?”. Although they agreed with the sample answer they argued as to the importance and meaning of interference and to the importance of mathematics and the related importance of the suite of other properties associated with a wave. The second debate centred on the meaning of uncertainty, they agreed with the sample answer but argued over the finer points surrounding the interpretations of quantum mechanics.

The educational researchers were critical of the regime of analysis tools selected. The content and correctness analyses were seen as side issues to the research question. They argued that the content analysis was superfluous to the investigation and that the correctness analysis would be based upon criteria set by and seen to be correct by the physicists. The main point put forward was that these two analyses would not reveal the underlying difficulties encountered by the students. These points were noted but from the perspective of a physics educational researcher, the terminology and correctness aspects of the students’ responses are considered important. They were therefore left in.

4.3 THE INSTRUMENT The final instrument was prepared and trialed on five post graduate physics students and two physics academic staff members. The trials revealed no operational problems with the question or answer booklets. A copy of the question sheet component of the final instrument follows. The complete package — Question sheet, answer booklet and “official” answers are in Appendix 3.

53 PHYSICS 1/1A :: QUANTUM PHYSICS LECTURE 10 :: CONCEPT QUIZ :: 1995

.

QUESTION 1

Monochromatic light (light of a single colour) may be considered either as a continuous electromagnetic wave or as a stream of energy quanta (or photons). The photoelectric effect is the emission of electrons from a metal surface when light is shone onto the surface. The energy in the light is transferred to some of the electrons in the metal surface and they are ejected. Two key observations are as follows:

1. For monochromatic light of sufficiently high frequency, the rate of electrons ejected per second increases as the intensity of the light increases.

2. Electrons are ejected from the surface only when the frequency of the monochromatic light is above a certain frequency; and for monochromatic light below this frequency, no electrons are ejected no matter how great the intensity.

Picture in your mind the following model. Think of electrons in the metal surface as being like birds sitting on a telegraph wire. The effect of light could be pictured in two ways. If it is a classical electromagnetic wave then it would be like trying to dislodge the birds by shaking the wire. But the effect of the stream of photons (particles of light) would be like trying to knock the birds off the wire by throwing stones at them.

Wave Model Particle Model

Which of these models can account for the above observations?

PLEASE ANSWER THE QUESTIONS IN THE QUIZ ANSWER BOOKLET.

.

QUESTION 2

In 1927 Werner Heisenberg published his Uncertainty Principle, which included the statement that all measurements have an associated uncertainty.

Consider a "measurement" involving catching a bus in a quantum mechanical world. Your timetable says that the bus will arrive at 9:00 am. Heisenberg would say that the time of arrival must have an associated uncertainty.

What does he mean by uncertainty?

PLEASE ANSWER THE QUESTIONS IN THE QUIZ ANSWER BOOKLET.

.

54

.

QUESTION 3

In classical physics we say that something is a particle if it has well defined position, velocity, mass, momentum and energy.

We also talk about another entity, a wave.

We know about water waves, surfing waves, radio waves, microwaves, sound waves and light waves. In 1924 Louis de Broglie proposed that electrons and other microscopic entities were also waves.

What do you mean when you say "something is a wave"?

PLEASE ANSWER THE QUESTIONS IN THE QUIZ ANSWER BOOKLET.

.

QUESTION 4

You are listening to a discussion about the structure of atoms. You listen carefully to what Sue and Jim have to say and then they ask you for your opinion.

Sue: The lecturer said that the spectral lines observed in excited atoms is strong

evidence that energy levels exist in atoms.

Jim: In 1913, Niels Bohr proposed a model for the hydrogen atom that combined the ideas of Max Planck's quantum theory, Albert Einstein's photoelectric effect and Ernest Rutherford's atomic model.

Sue: And... In 1924 Louis de Broglie proposed that electrons were also waves. The lecturer said that it was then proposed that the missing piece of Bohr's theory was that the electrons wavelength had to fit into the atom. Do you know what

she meant Jim?

Jim: No. But I think it had something to do with the energy levels.

{ Sue and Jim turn to you and Sue asks }

Sue : I don't understand what she meant by energy levels and I don't understand what she meant by the wavelength of an electron fitting into the atom. Do you know?

You have been asked to join into the conversation.

Think carefully about the points Sue and Jim have been discussing, and

WRITE YOUR RESPONSES TO SUE'S QUESTIONS IN THE QUIZ ANSWER BOOKLET.

.

55

BIBLIOGRAPHY

Cohen, L. and Manion, L., (1994) Research Methods in Education 4th ed. (Routledge, London) pp83-105.

Gamow, G., (1993) Mr Tompkins in Paperback 8th ed., (Cambridge University Press, Cambridge)

Haliday, R., Resnick, R., and Krane, K.S., (1992) Physics 4th ed. (John Wiley and Sons inc.)

Haliday, D., Resnick, R., and Walker, J., (1993) Fundamentals of Physics 4th edition, (John Wiley and Sons Inc.)

Young, H.H., (1992) University Physics 8th ed. (Addison-Wesley Publishing Company), pp1104-1108, pp1141-1160, pp1162-1182

56

CHAPTER FOUR...... 42

METHODOLOGY...... 42 4.1 INTRODUCTION ...... 42 4.2 DEVELOPMENT OF THE INSTRUMENT...... 42 4.2.1 Content and Context ...... 42 4.2.2 Guidelines ...... 46 4.2.3 Intended Analysis ...... 47 4.2.4 Preliminary wording of the questions ...... 50 4.2.5 Final Iteration...... 51 4.3 THE INSTRUMENT...... 52