Master’s Thesis – First Draftdduncan/teachingseminar/Jor… · Web view1. Introduction. Astronomy 1010 is an introductory astronomy class designed for non-science majors at

Measuring Student Learning Gains from a Spectroscopy Laboratory ActivityMaster’s Project

Amy Jordan, May 25, 2005

Table of Contents

Abstract…………………………………………………….. 21. Introduction……………………………………………… 22. Theoretical Perspective…………………………………. 33. Methods…………………………………………………... 4

3.1 Participants………………………………………. 63.1.1 Faculty…………………………………. 63.1.2 Students………………………………... 6

3.2 Description of Lab Setting and Activity…………. 73.3 Assessment Techniques………………………….. 93.4 Assumptions/Limitations of the Study…………… 11

4. Results…………………………………………………….. 124.1 Faculty data………………………………………. 12

4.1.1 Content Objectives……………………... 134.1.2 Nature of Science Objectives…………... 154.1.3 Affective Objectives…………………… 17

4.2 Student data.……………………………………… 184.2.1 Quantitative…………………………….. 18

Pre-Test #1…………………………… 18Pre-Test #2 and Post-Test……………. 19Learning Gains……………………….. 21

4.2.2 Qualitative……………………………… 23Prior Knowledge……………………... 24Post-Test/Interview Data…………….. 32

5. Discussion………………………………………………… 466. Recommendations………………………………………... 48References…………………………………………………… 53

Appendix A: Astronomy 1010 Spectroscopy LabAppendix B: AssessmentsAppendix C: Interview transcript

AbstractSpectroscopy is taught in many introductory astronomy courses. It is a

fundamental concept for understanding how scientists in several different fields (astronomy, physics, chemistry, geology, environmental science, etc.) study the composition and physical states of gases and solids. As students generally have extremely little prior experience with the ideas and phenomena involved in spectroscopy, the concepts can be new and difficult. In this paper, I will examine the objectives of astronomy professors when teaching the spectroscopy lab at the University of Colorado at Boulder. I have studied three sections of an introductory astronomy class (n=53-55 students) to test in which of these objectives students make learning gains, as measured by pre- and post-lab written assessments and nine interviews. When spectroscopy is taught in a lab-based setting, there is evidence that students make large learning gains on certain concepts, but no statistically significant gains on others. The possible explanations of this dichotomy are discussed.

1. IntroductionAstronomy 1010 is an introductory astronomy class designed for non-science

majors at the University of Colorado at Boulder (UCB). In Spring 2005, the course was taught by Professor Nick Schneider of the Astrophysics and Planetary Sciences (APS) Department. The lecture component of the course consisted of three 50 minute classes per week, while five graduate student teaching assistants (TAs) supervised ten laboratory sections, each of which was one hour and fifty minutes long and held once a week. The TAs were aided by “learning assistants,” undergraduates with some training in science and education who helped make the labs more interactive.

Students in the College of Arts and Sciences must complete a core curriculum, similar to general education requirements at universities nationwide. The Astronomy 1010 course can be applied towards the Natural Sciences requirement by fitting into both the “sequence” and “lab experience” categories.1 As a result, many of the students in the course are taking it specifically for the purpose of satisfying a graduation requirement. (A different introductory astronomy sequence, Astronomy 1030/1040, exists for intended astronomy majors.) Most students in Astronomy 1010 have never taken a college-level science class before.

The laboratory section of the course is taught with the same lab manual from semester to semester, although the teachers have the option to arrange the labs in any order, to add labs of their design, and to remove labs they do not wish to include. In general, however, all the students who take Astronomy 1010 are essentially performing the same or very similar laboratory activities as one another.

Spectroscopy is a topic commonly covered in university-level introductory astronomy classes. A study by Zeilik and Morris-Dueer (2005) examined professor goals of introductory astronomy courses by presenting 18 astronomy instructors with a list of 200 concepts to rank from 1 (highly essential) to 5 (not at all essential) for their students to learn. The top concept was “the electromagnetic spectrum,” while the fifth highest concept was “spectra,” and the eighth was “the atom.” The importance of spectroscopy in introductory astronomy is also evidenced by its inclusion in a list of goals compiled during two American Astronomical Society (AAS) meetings in 2001. Thirteen primary

1 For more information, visit http://www.colorado.edu/ArtsSciences/students/undergraduate/core.html

2

objectives were identified, which were quite general (compared to the 200 specific concepts identified in the study above), and these goals covered all of what I am calling content, nature of science, and affective goals (Section 4.1, faculty objectives). Spectroscopy was mentioned, in particular, under the banner of “related subjects” from physics (Partridge and Greenstein 2003).

This study is intended to be relevant not only to the one Astronomy 1010 course taught in Spring 2005, but to all teachers of Astronomy 1010, and further, to others who teach the subject of spectroscopy to non-science majors in a lab-based setting. Approximately 250,000 U.S. college students take introductory astronomy every year (Fraknoi 2001), and some aspects of this study are even relevant to teaching spectroscopy in a geology, chemistry, or physics course.

The research questions I am seeking to answer with this study are the following:1. Among the professors who teach Astronomy 1010, what are their

objectives for the spectroscopy lab and the laboratory portion of the course in general, and how do these objectives differ among faculty members?

2. What learning objectives have students achieved as a result of doing the spectroscopy lab activity?

3. What are the common ideas, whether correct or incorrect, with which the students enter and leave the spectroscopy lab?

4. What inferences can be made about why some objectives are met and others are not?

5. How can the lab be modified to reflect faculty objectives and aim for increased learning gains?

This paper is organized as follows. First, Section 2 provides a theoretical framing and discussion of where this study fits in to the body of educational research that currently exists. Section 3 describes the methodology of the project, including the participants (Section 3.1), a description of the lab activity (Section 3.2), the assessments used (Section 3.3), and the assumptions and limitations of the study (Section 3.4).

The results are then presented in Section 4, including the faculty data (objectives of all types – Section 4.1) and student data (Section 4.2). Within the student data, the quantitative results are found in Section 4.2.1 (i.e. test scores, measured learning gains, and statistical analysis of the data) and the qualitative results are in Section 4.2.2 (i.e. common student ideas). A discussion of the results is found in Section 5. Because the lab manual for Astronomy 1010 at the UCB is about to undergo an overhaul by the APS department, Section 6 contains recommendations – which are data-driven – for the improvement of the spectroscopy lab.

2. Theoretical PerspectiveTo avoid re-inventing the wheel, or to avoid missing the wheel altogether, the

development of effective science curricula should be performed within the framework of modern science education theory. A vast amount of research has been done in the fields of cognitive studies and education, and a wide body of evidence has been collected, on the subject of how people learn scientific concepts and which teaching styles fail to promote significant learning gains among students. This project fits into the framework of education research by examining a particular topic, spectroscopy, that does not appear to have been previously subject to a comprehensive study. It does, however, involve

3

concepts (e.g. light and atomic structure) that have been studied by others in the context of physics or chemistry education research.

Professors, departments, and colleges are well aware that a laboratory experience is a critical part of science education. This is why UCB’s college of Arts and Sciences core curriculum includes not only physical sciences courses, but also a laboratory requirement. However, the presence of a laboratory environment and a lab manual full of activities does not by itself guarantee significant student learning, either for content or nature of science objectives. Development of a laboratory activity in a so-called “inquiry-oriented” (McDermott 1991) or “discovery-based” format has been shown to increase student learning on both those types of objectives. Such a format requires students to develop their own hypotheses, design the experiments necessary to test said hypotheses, conduct the experiment and collect/analyze the data, and write up their findings (Handelsman et al. 2004).

An understanding of the effects of students’ prior knowledge is another key component of developing effective curricula. The idea of students as a “blank slate” ready to be filled with new knowledge has long been abandoned. Students enter a course with a complicated array of mental models, based on a combination of real-world experience and prior classroom instruction, to describe the world around them. Mental models, as described by Redish (1994), may be incomplete, contradictory, and muddled in a person’s mind. The student may not know how to “run” the model. It is also difficult to dislodge or replace the naïve model with the correct model. Preconceptions can be so deeply entrenched or so off-base that “meaningful learning is precluded,” as McDermott (1991) writes.

An understanding of students’ prior knowledge and common misconceptions will help professors and teaching assistants tailor their instruction to try to guide students from a naïve mental model to an expert one. A study of teaching fifth graders the cause of the day/night cycle (Diakidoy and Kendeou 2001) showed that students taught with common incorrect models taken into account exhibited greater learning gains than the control group, who were taught with traditional textbook materials. However, it is not enough, as McDermott (1991) points out, simply to identify common naïve ideas and then tell the student not to believe this or that. She writes,

“Helping students develop a sound conceptual understanding is not simply a mater of making a list of misconceptions and explaining mistakes that they should avoid…such an approach is seldom effective…Conceptual and reasoning difficulties cannot be overcome through assertion by the instructor. Such changes in thinking require a significant intellectual engagement by the student. The only method likely to be effective in addressing serious difficulties is to design instruction to expose them and then to address them specifically, not just once but several times.” –p.314

The constructivist approach to teaching takes into consideration the importance of students’ prior knowledge and the difficulty in changing or amending their incoming mental models. The work of the pioneering cognitive scientist Jean Piaget suggested that students’ mental models could be changed through a process called “self-regulation” or “equilibration” (Lawson and Renner 1975), whereby a student is guided into disequilibrium by a teacher – i.e., the student discovers, through experimentation, an

4

outcome that contradicts their mental model – and then back into equilibrium by the resolution of the conflict. The cultivation of students as active learners is a key strategy for developing effective curricula.

The importance of data-driven curriculum reform cannot be overstressed. As McDermott (1991) notes, “There are often significant differences between what the instructor thinks students have learned in a physics course and what students may have actually learned.” The development of meaningful assessments to understand the latter is a part of letting the students teach the instructors about the effectiveness of their teaching. This study attempts to do that, with a focus on a spectroscopy lab activity in particular.

3. MethodsI began by interviewing the six professors who have taught Astronomy 1010 at

UCB most often in previous years. From these interviews, I assessed what the professors hope the students learn from the spectroscopy lab (their content goals), what they expect the students to know prior to this lab activity, and what they hope the students gain from the laboratory portion of the course in general. The results of these interviews are given in Section 4.2.2. The objectives, for the purpose of organization, were condensed in to seven main themes (codes in parentheses refer to objectives, prior knowledge, or other concepts, which are given in Tables 1-3 in Section 4.1.1 below):

Theme 1.- Atomic structure (Pr7)- Energy levels (Pr7, OB1)- How a line is excited (OB2)- Atoms are quantized (OB3)

Theme 2.- Each element has a distinctive signature (OB4)- Spectra of atoms vs. molecules vs. solids (OB5)

Theme 3.- Spectroscopy is a tool for exploring distant objects (OB6)- Look at spectrum in lab, direct comparison to astronomical object (OB7)- Use of spectroscopy to study planets (OB8)- Eye is not sufficient to determine color (Pr6)- Splitting of light – refraction through prism, diffraction grating (E3)

Theme 4.- Light verbs: emit, absorb, transmit, reflect/scatter (OB9)

Theme 5.- Color (E2)- Filters (OB10)

Theme 6.- E-M spectrum: wavelength ranges (Pr1, OB11, OB12)- Wavelength vs. energy vs. frequency (Pr2)- White light is a mixture of colors (Pr3)- Most of the E-M spectrum is beyond the visible wavelengths (Pr4)- Light as a particle/wave (Pr5)- Speed of light is constant (E1)

Theme 7.

5

- Blackbody spectrum (OB13)- Temperature/color/Wien’s Law (OB14)

Based on the topics identified by the faculty, I developed two written pre-assessments and one written post-assessment for the students. These assessments are described in Section 3.3. I also recruited nine students to participate in semi-structured post-lab interviews. There were three laboratory sections involved in this study, with a total number of 53–55 students. The lab setting and a description of the lab activity is given in Section 3.2.

After collecting the data, I transcribed the interviews, graded the written tests, and analyzed the results of the qualitative and quantitative data. The data are presented in Section 4.

3.1 Participants

3.1.1 Faculty: Prior to developing the student assessments, I interviewed six faculty members

who have either taught Astronomy 1010 in the past or will be teaching it in the near future. These six faculty were a diverse group in terms of field of research (i.e. astrophysics versus planetary), amount of teaching experience, and teaching style. The faculty interviews took place during the week of February 14-18, 2005, and were audiotaped.

3.1.2 Students:Demographics

The students in the three lab sections ranged in age from 18 to 49, with a strong majority in the 18-20 age range. Gender, major, and race data were not taken for the three sections, but the first two are known for the entire class, and the last is known for the entire campus. In the Spring 2005 Astronomy 1010 class, 131 students (63%) were male and 77 (37%) were female (compared to the campus-wide distribution of 52% male, 48% female). The students’ majors (or intended majors) were primarily not in the sciences: 62% nonscience, 28% science, math, or engineering, and 10% unsure (Figure 1). Of 36 enrolled students polled separately, 23 (64%) took chemistry in high school, and 22 (61%) took physics.

Although the racial demographics of the entire class or the sections are unknown, according to the university’s Office of Planning, Budget, and Analysis webpage, the undergraduate population of UCB for 2004-2005 was made up of 14% minorities (Asian American, American Indian, Black/African American, and Hispanic/Latino), and it is reasonable to assume that the Astronomy 1010 class reflected this distribution as well.

Recruitment MethodsThe nine interview subjects were chosen using different methods. The idea

behind the interviews was not to obtain a random sample but to recruit students who would be talkative, participatory, and articulate. The first four interviewees were chosen by me and ranged from one of my highest-scoring students (based on lab grades) to one of my lowest-scoring students. The other five were taken from the second and third

6

sections and were the first students to volunteer. Among the nine interview subjects, four were female and five were male. Their ages ranged from 18 to 49. They represented a wide diversity of classroom performance. The interviews were conducted in another room by the TA (myself) during the students’ normal lab periods, and ranged in length from fourteen to twenty-one minutes.

Major or Intended Major, Astronomy 1010 Students, n=114

Non-Science (62%)

Engineering or Math (16%)

Biological Sciences (8%)

Physical Sciences (4%)

Unsure (10%)

Figure 1: Astronomy 1010 students’ major or intended major. Most (62%) are non-science majors, with 10% unsure if they will enter a science major or not. The rest (28%) intend to major in or are majoring in engineering, math, and the physical or biological sciences.

The students were informed that the interview would be videotaped, that their participation in the interviews would be entirely voluntary, that they could opt out at any time, and that their participation would not affect their grade in any way. They were given a consent form to sign, approved by UCB’s Human Research Committee (Protocol Number 0305.24), that explained the study and its confidentiality. Individual privacy was promised to be protected (in this paper, participant names have been replaced with pseudonyms) and the interview tapes and transcripts are the property of the researcher.

All students present in a section were given the written pre- and post-tests during their class time.

3.2 Description of the Lab Setting and ActivityThe laboratory (“recitation”) section of the Astronomy 1010 course meets in a

large classroom at Sommers-Bausch Observatory on the UCB campus. The lab sessions are one hour and fifty minutes long, and each session is run by one TA, with an undergraduate learning assistant providing additional support and discussions with the students. The students sit at long tables, and work in groups of 3-4. In the case of the

7

spectroscopy lab, several “stations” were set up that the students had to visit for the various parts of the lab.

For this study, I have collected data from three of the ten lab sections. Two of the sections were normally taught by this author, but during the week of the spectroscopy lab, the third section was also taught by me. Therefore, the spectroscopy lab was presented to all three sections in the same way, with only minimal differences in instruction.

The spectroscopy lab is found in the Astronomy 1010 lab manual, and is attached as Appendix A (the entire Astronomy 1010 manual can also be found online2). It is a mixture of instructions, explanations, graphs, and questions for the students to answer. Six of the seven themes are mentioned in the manual: the only one not explicitly discussed is Theme 5, color and filters, which is a more “basic” topic than the spectroscopy lab covers.

The lab begins with a synopsis, a list of the equipment to be used, a warning about touching the gas tubes, and then launches into some introductory material about spectroscopy and atomic structure. The first sentence of this section sums up Theme 3 succinctly: “Most of what astronomers know about stars, galaxies, and nebulae – and much of what planetary scientists know about planetary atmospheres and surfaces – comes from spectroscopy, the study of the colors of light emitted or absorbed by different materials.”

The next paragraph explains the absorption and emission of photons by atoms (Theme 1: atomic structure, and Theme 4: the light verbs), with only the briefest description of what photons are or what makes up an atom. A diagram is provided (Figure 7 in Section 4.2.2) showing how different wavelengths of light are given off by transitions between different “Allowed Electron Orbits.” (As discussed later in Section 4.2.2, only some of the interviewed students ultimately understood what this drawing was attempting to portray: the link between the color of the emitted photon and the energy difference between levels.) The introductory material goes on to describe how emission and absorption lines are produced, and what a continuum spectrum is. Next it shows a diagram of the handheld spectroscope and explains the grating used to break up light into a spectrum.

In Part I of the lab, the students view glowing discharge tubes of gas with their spectroscopes and sketch the spectrum in their lab manual. In some cases, they are asked to make note of the color of the glowing tube, and compare it with the appearance of the spectrum. In this section, they see spectra of helium, neon, molecular nitrogen, an incandescent lamp, and the fluorescent ceiling lights. Here, the students receive proof of Theme 2: every element has its own distinct spectral signature, and molecules and solids (like the glowing filament in the incandescent light) have spectra that appear different from atoms.

The next section is an activity designed to help students come to understand Theme 3, which involves the idea that you can learn the composition of an object by observing its spectrum. The students observe a “mystery gas” and compare it to a table describing the wavelengths of spectral lines for hydrogen, mercury, and krypton. Next, they observe a tube of air, and are asked to compare it with the spectrum of nitrogen observed earlier.

2 http://lyra.colorado.edu/sbo/manuals/astr1010/astr1010.html

8

The third part of the lab is entitled “The Solar Continuum,” and it is about Theme 7 (blackbody radiation and temperature). Two paragraphs explain the blackbody spectrum, and two graphs are given (Figure 8 in Section 4.2.2) showing a few blackbody curves at different temperatures. Wien’s Law is given in the form of an equation relating temperature to the wavelength that corresponds to the peak intensity on the blackbody curve. The students then proceed to measure this peak using a photometer and the observatory’s heliostat spectroscope. The heliostat is a solar telescope on the roof of the Sommers-Bausch Observatory; the light of the sun is directed into the lab classroom, and the solar image can be directed through a slit and on to a diffraction grating. From there, the solar spectrum is projected onto a large (~1.3 meters) curved white board.

Using the photometer, the students measure the relative intensity of light across the spectrum. They should find that the sun has its peak at a wavelength that corresponds to the color green. They then use this information to calculate the temperature of the sun using Wien’s Law.

Part IV involves the solar absorption lines (called Fraunhofer lines), which are caused by the sun’s atmosphere. On a clear day, with the lights turned low in the classroom, these lines are quite visible using the heliostat spectroscope, which offers marked improvement over the handheld devices. The box containing the spectroscope slit also contains two tubes of gas, hydrogen and neon, that can be powered up to produce comparison lines above and below the solar spectrum. The students are asked to identify which spectrum belongs to which element, perhaps by noticing hydrogen’s fewer lines and thus simpler atomic structure (Theme 1) or by comparing the neon spectrum to their drawing of neon from Part I (Theme 3). Next, they are asked whether there is evidence that these two gases are found in the sun’s atmosphere. They should observe that the hydrogen emission spectrum from the comparison gases lines up perfectly with absorption lines in the sun, while neon does not.

Next, the students receive a vivid demonstration that the solar spectrum includes wavelengths beyond what our eyes can see (Theme 6, the electromagnetic spectrum). Bleached white paper is held up to the purple end of the solar spectrum, and the paper fluoresces from the ultraviolet light just beyond the violet. Several thick absorption lines are seen in the ultraviolet.

The remaining two pages of the lab include an activity that does not relate to any of the professors’ identified objectives. In this section, the students use a labeled chart of the solar absorption lines to find which line corresponds to which element. They are asked to observe the lines of molecular oxygen, hydrogen α and β, sodium, iron, and calcium H and K, and describe the appearance of each line, i.e. whether it is dark, broad, fuzzy, etc. A few paragraphs of background material precede a list of four possible explanations for the appearance of the lines. The students are asked to identify which lines match with which explanation, and answer two more questions involving these concepts. Based on the students’ questions during this part of the lab, I believe that this is a very difficult section for them to understand.

In a typical lab session, if there is time left over after the lab is completed (there was not, in the case of the spectroscopy lab sections I taught), a class discussion or small group discussions are held, moderated by the learning assistant and TA. The students’ homework is to write up the lab neatly, answering all the questions, including all the data and calculations. The lab write-up is due the following week.

9

Figure 2: A timeline showing when students performed the spectroscopy lab, took the various Pre- and Post-tests and interviews, and had their first lecture on light and the electromagnetic spectrum.

3.3 Assessment TechniquesThe first pre-test, which I will call Pre1, was given to two sections (n=36) of students one week before the date of their spectroscopy lab. The second pre-test (Pre2) was given to three sections (n=55) the week of the spectroscopy lab (before the lab was performed), and the post-test (Post), which included almost exactly the same questions as Pre2, was given to the same three sections (n=53, due to absences) the week following the lab. A timeline of key dates is shown in Figure 2.

Pre1 was designed to test the prior knowledge of the students on topics the professors expected them to know before the spectroscopy lab. Pre2, on the other hand, was designed in conjunction with Post to measure learning gains on topics that were main objectives of the professors for the spectroscopy lab. Pre2 and Post were identical with one small change: the substitution of “infrared” for “ultraviolet” in question 8. This change had unintended consequences, as discussed below.

The first pre-test was multiple choice, True/False, Yes/No, with two questions asking the students to draw something; the second pre-test and the post-test required write-in answers. The benefit to the former is forcing the student to make an equivocal choice, while with the latter, the students would sometimes evade the question or write simply “yes” or “no” without explanation. However, a bounty of information about common and naïve ideas came from the students who did take the time to complete thought-out, multi-sentence answers. The three tests are attached as Appendix B.

The tests were administered at the beginning of the period, and most students were given enough time to finish them to their satisfaction (~12 minutes of class time). The students were informed of the reason for these assessments and asked to try their best. The tests were taken anonymously and no grades were recorded.

I conducted interviews to compare their results with the written assessments. The interviews were capable of revealing more about the students’ thought processes than the constricting questions of an exam; if a student said something incorrect, I would follow up on it to explore the nature of their misunderstanding and gain a sense of the student’s

February 22Section 1 takes Pre-Test 1

Class Meeting Times:Section 1: Tuesday 9:00–10:50 a.m.Section 2: Tuesday 11:00 a.m.–1:50 p.m.Section 3: Friday 11:00 a.m.–1:50 p.m.Lecture: Mon, Wed, Fri 2:00–3:00 p.m.

February 25Section 3 takes Pre1 March 1

Sections 1 and 2 take Pre-Test 2 and do spectro. lab

March 4Section 3 takes Pre2 and does spectro. lab

March 8Sections 1 and 2 take Post-Test and 6 students are interviewed

March 11Section 3 takes Post-Test and 3 students are interviewed

March 9Students receive a lecture on light

Spectroscopy Lab

Mon Tues Wed Thurs Fri Mon Tues Wed Thurs Fri Mon Tues Wed Thurs Fri

10

mental model. It can be difficult to gauge a student’s conceptual understanding of a topic from a written assessment.

The development of optimal assessments is a tricky and iterative process. Sometimes, it takes a flawed first attempt to realize how to improve the questions on pre- and post-tests, and better tests will result in more accurate and relevant results. For example, I realized in retrospect that certain questions on Pre2 and Post allowed students to answer simply “yes” or “no,” as they frequently ignored the instruction to “explain.” I learned that certain multiple choice questions may have misleading results: if the question, for example, is #2 on Pre1 (“True or false: blue light travels faster than red light”), the students who answered “false” may have answered such because they know that light travels at the same speed regardless of wavelength, or they believe that red light travels faster than blue light. (Post-test results from interviews and the written responses indicated that a small number of students thought that longer wavelength light travels faster than shorter, a preconception that I did not expect, whereas most students who answered incorrectly believed that the shorter wavelength light travels faster.) A prior awareness of mistakes students are likely to make is crucial to developing a good assessment.

While I will not go into detail on the individual improvements that could be made to each question, I will point out one flaw in the pre/post assessments that had unintended consequences. The written post-test was identical to Pre2, except I made the mistake of making a small change on a question that I thought would not affect the students’ answers (“ultraviolet” to “infrared”) but instead I got responses that had to do with students’ awareness that infrared corresponds to heat. Because of the changed nature of the question, it no longer provided the straightforward before/after measurement that I hoped to obtain.

In the future, to encourage more thoughtful answers, I would choose to have the tests not be anonymous and count for class credit. This would also allow the tracking of individual student improvement.

The quantitative data from the written assessments – numerical learning gains – are given in Section 4.2, followed by qualitative results from the interviews and the tests.

3.4 Assumptions and Limitations of the StudyBecause all of the sections studied for this project were taught the spectroscopy

lab by the same instructor (myself), there is the danger that the results obtained here are entirely related to my effectiveness or ineffectiveness at teaching certain concepts. This is a true limitation to the study, because it is based on the assumption that I am a “representative” TA. While the spectroscopy labs were taught similarly amongst the Astronomy 1010 TAs (a pre-lab meeting of all the TAs, learning assistants, and the course instructor ensured a certain amount of consistency between sections), the TAs were given the freedom to lecture on background material and run the section as they pleased.

Nonetheless, all of the students used the same lab manual, performed the same activities, and answered the same lab questions. Differences among the sections may have included:

Total length of TA lectureo Amount of background material covered

11

o Different coverage of topics within the lab Quality of TA lecture Willingness of TA to supply direct answers to students’ questions Speed at which the class finished the lab Whole-class or small-group discussion facilitated by TA or learning assistant

(generally depends on amount of time left over after students finish the lab) Overall classroom environment of section (i.e. are students comfortable asking

questions?) Time of day/day of week The presence or absence of disruptive or distracting students

There are many more. If this study had been performed on sections with other TAs, the results may have turned out differently. While the absolute amount of learning gains measured in the students for the objectives may have varied from TA to TA, I believe that the learning gain results presented here are still of extreme value, if the assumption that I am a “representative” TA is accepted. Further, the universal results found by this study include the common ideas held by students prior to the lab, the effectiveness or ineffectiveness of the activities presented in the lab manual (and performed by the students), and the relation of faculty objectives to the lab activities.

When presenting results, I will make note of how and where a certain topic was presented (i.e. in the TA’s lecture, in the lab manual, etc.; see also Table 8). This will clarify whether a result can be linked to a certain style of presentation to the student. To verify the topics covered in my lecture, the lecture was videotaped and transcribed.

The students did the spectroscopy lab activity before their professor’s lecture on light and spectroscopy. Therefore, in this case, the students’ first contact with the material was their TA’s lecture at the beginning of the lab period (unless the student read the lab manual’s coverage of the topics prior to coming to lab, or read the relevant sections of the textbook). The first pre-test, which was given a week before the spectroscopy lab, was scheduled to avoid the discrepancy between those who had read the lab and those who had not. Regrettably, the second pre-test was given the day of the lab, prior to the TA’s lecture, so an unknown percentage of the students had recently been exposed to some of the material from their readings.

It is only the post-test data for one of the three sections that may have been affected by the professor’s lecture on March 9. Two of the sections had their post-tests and interviews on March 8, and the last on March 11.

4. Results

4.1 Faculty DataI interviewed six faculty members who commonly teach Astronomy 1010

(including the one teaching it while I conducted this project) and asked them three questions:

1. What do you expect students to learn about spectroscopy?2. What do you expect students to come to lab knowing about light and atomic

structure?3. What are your overall goals for the lab portion of this course?

12

Afterwards, I asked them if they had any other comments about the spectroscopy lab.

The professors had three main types of goals for their students: content objectives, nature of science, and affective goals. Content objectives involve student learning gains on specific canonical concepts. For example, a professor noted that one of his goals for the spectroscopy lab was that the students understand how a spectral line is formed in relation to atomic structure. The second type of goal dealt with imparting to the students the idea of the “nature of science” or scientific inquiry. Finally, the professors had goals that fall into a broad third category, which I will call affective objectives, and which include improving or changing the students’ attitudes and beliefs about science. Some had hopes that students would obtain, from the laboratory portion of the course, an improved attitude about science or a sense of empowerment that they, too, could perform scientific activities. Here, the importance of the lab as a hands-on experience was repeatedly stressed by the faculty.

In the course of instruction, the three categories overlap as a teacher directs a lesson. For example, if the students are expected to learn the Bohr model of the atom, they would also be expected to know that it is just a model, which leads to an excursion into scientific inquiry. The three types of goals – content, nature of science, and affective – are discussed further below.

4.1.1 Content ObjectivesThe topics on which professors hoped students would make learning gains are

listed in Tables 1-3, with the number of faculty members expressing the objective in parentheses. There was a great diversity of concepts represented in the faculty interviews; for example, some listed atomic structure as a main goal, while others felt that the scope of the course did not include this topic. The idea of whether an objective is “valid” for the course will not be discussed here, as it is a topic for departments, professors, and others to decide.

The fourteen objectives, seven prior knowledge concepts, and three extra concepts were condensed into seven main themes. These themes were listed in Section 3 above: atomic structure, the uniqueness of spectra, uses of spectroscopy, the light verbs, color and filters, the electromagnetic spectrum, and blackbody radiation/temperature.

The goals of the professors tended to follow certain trends depending on whether the faculty member’s research was primarily in the realm of planetary science or astrophysics. There were deviations from these trends, and there were also some concepts that many of the faculty had in common as main objectives, regardless of field of research. However, two main disparities arose between the goals of the three faculty I considered as “planetary” and the three I considered to be “astrophysics.”

First, the “astrophysics” faculty tended to emphasize atomic structure as being an important objective (one “planetary” faculty member also identified atomic structure as an objective, while another stated that he did not feel it should be an emphasized concept in Astronomy 1010.) In general, however, the “astrophysics” faculty brought up objectives under the banner of Theme 1 (atomic structure) much more frequently and placed its importance higher.

13

Second, the “planetary” faculty consistently brought up the spectroscopy of planetary bodies to be a main goal of the spectroscopy lab, as Astronomy 1010 at the University of Colorado is a “solar system course,” as one professor put it. (The university’s catalog reads: “Introduces principles of modern astronomy for nonscience majors, summarizing our present knowledge about the Earth, Sun, moon, planets, and origin of life.” The next semester of the sequence for non-majors, Astronomy 1020, hasno laboratory section, and includes the Sun and stars among its many topics.) None of the astrophysics faculty mentioned planetary spectroscopy as an objective. Again, as noted above, it is not the aim of this paper to suggest whether an objective is “valid” for the course, but instead to discover which objectives have been “met” (as evidenced by increased learning gains by the students).

Table 1: Content Objectives Objective Theme Pre-Test

QuestionPost-Test Question

OB1 Atomic structure, energy levels, nature of atom, matter, and light (3)

1 Pre1 #5, 13 Pre2 #2a

Post #2a, Interview

OB2 Understand how a line is excited (1) 1 Pre1 #13 Pre2 #2a

Post #2a, Interview

OB3 Spectra prove that atoms are quantized (2)

1 Pre1 #12 Pre2 #2b

Post #2b, interview

OB4 Each element has a distinctive signature – a “fingerprint” (4)

2 Pre2 #3 Post #3

OB5 Spectra of atoms vs. molecules vs. solids (1)

2 Pre2 #4 Post #4

OB6 Spectroscopy is a tool for exploring different objects – can determine composition without going there (3)

3 Pre2 #1 Post #1, Interview

OB7 Look at spectrum in lab, make direct comparison to astronomical object (1)

3 Pre2 #7 Post #7

OB8 Link to planetary spectroscopy (3) 3 Pre2 #6 Post #6, Interview

OB9 “Light verbs:” Emit, absorb, transmit, reflect/scatter (3)

4 Pre1 #11 Pre2 #9

Post #9, Interview

OB10 Composition information can come from filters (1)

5 N/A N/A

OB11 Light is made up of different wavelengths, can be broken up into these wavelengths (1)

6 Pre1 #8 Pre2 #8

Post #8

OB12 Light and radio waves are the same thing (1)

6 Pre2 #5 Post #5

OB13 Blackbody radiation (1) 7 N/A InterviewOB14 Temperature/Wien’s Law (1) 7 Pre1 #7 Interview

14

Table 2: Prior Knowledge ExpectationsConcept Theme Pre-Test


Pr1 E-M spectrum: wavelength ranges 6 Pre1 #4 InterviewPr2 Wavelength vs. energy vs. frequency 6 Pre1 #1 InterviewPr3 White light is a mixture of colors 6 Pre1 #6, 8 InterviewPr4 Most of the E-M spectrum is beyond the

visible range6 Pre1 #3 N/A

Pr5 Light as a particle/wave 6 N/A N/APr6 Eye is not sufficient to determine color 3 Pre2 #10 Post #10Pr7 Atomic structure 1 Pre1 #12,

13Interview

Table 3: Additional ConceptsConcept Theme Pre-Test


E1 Speed of light is constant 6 Pre1 #2 N/AE2 Color, filters 5 Pre1 #11 InterviewE3 Splitting of light – refraction through

prism, diffraction grating3 Pre1 #8, 9 N/A

Because the students included in this project had one professor with specific goals among those listed in Table 1, who would choose to teach or emphasize only certain concepts from that list, it should be noted why I am claiming that the data presented in this study has a certain amount of independence from the orientation of the related lecture on spectroscopy, which took place after the lab was conducted and most of my pre- and post-testing was carried out. The students did the spectroscopy lab on March 1 and 4, before their professor’s lecture on light and spectra, which took place on March 9. (It is difficult, as several professors noted, to time the labs exactly with the material covered in lecture.)

It may seem strange to test the students’ improvement on concepts that were not covered or emphasized previously by their professor in lecture, but the lab manual and lab activities are essentially standardized regardless of the instructor of the course, and they include material touching on most of the themes identified by the professors. I will attempt to clearly state where each theme and objective made an instructional appearance: in the lab manual’s written text, in the TA’s speech prior to the lab activity, etc., with the goal of separating the various places where a topic has been covered, and therefore identifying the thing(s) that will change when the course is taught by another professor (his or her lecture) and the things that will not change (the lab manual, the TA’s lecture – which varies more from TA to TA than from class to class – etc.). (See Section 3.4 above, Assumptions and Limitations of the Study, for more explication on this subject.) While the learning gains measured are for these students in this class taught by this professor, I have attempted to design the study to make it as relevant as possible to any section of any Astronomy 1010 class using the same lab manual, as well as to any teacher of the concepts of spectroscopy.

15

4.1.2 Nature of Science ObjectivesBecause the students in Astronomy 1010 will likely enter non-science

professions, and thus may never have a need to recall the composition of Saturn’s atmosphere or even the cause of the seasons, it is important to identify what concepts non-scientists can take away from a science class that will be relevant to decisions they make in their lives and careers. A democratic society is regularly called upon to vote on issues involving science and technology, and here an ability to distinguish science from pseudoscience is imperative. Furthermore, the ability to make reasoned decisions based on evidence and experimentation may prove to be useful in any number of non-scientific settings. As Redish (1994) writes, “Society has a great need not only for a few technically trained people, but for a large group of individuals who understand science.”

The six interviewed professors, when asked about their overall goals for the lab portion of the course (as opposed to the spectroscopy lab in particular), identified as their primary goals for the students objectives such as “understand how science works,” “help encourage students to think critically; to question whether claims seen in the popular media make sense,” to be able to “distinguish between stuff they’re told and stuff they can analyze for themselves,” and to learn that “scientists don’t have all the answers, and this is how they figure them out.” They are interested in the students understanding something about scientific inquiry and the nature of science. As one professor put it, “Astronomy and planetary science are a tool – a vehicle – a motivator for doing some quantitative reasoning.” Table 4 lists some of the concepts identified by the faculty that fall under the category of nature of science objectives.

The phrase “nature of science” is defined in many different ways, but some common themes emerge (Bell and Lederman 2003): scientific knowledge is subject to change; it is empirically based, yet a certain amount of human inference is involved and so it can be subjective; it has social and cultural underpinnings; and creativity and imagination are an integral part of formulating scientific theories. Additionally, an awareness of the differences between observations and inferences, and between theories and laws, is a key component of a thorough understanding of the nature of science. The American Association for the Advancement of Science (AAAS) has published a book, Science for all Americans (1994), which identifies the following major components of scientific inquiry: “science demands evidence,” “science is a blend of logic and imagination,” “science explains and predicts,” “scientists try to identify and avoid bias,” and “science is not authoritarian.”

Studies have suggested (e.g. Lederman 1999) that students do not successfully learn much about the nature of science from learning science content. Even the act of performing scientific experiments does not necessarily lead towards an understanding of scientific inquiry among students: more explicit instruction is necessary to help the students make the link between what they are doing in lab and the nature of science. As spectroscopy has many applications to practical matters (e.g. atmospheric contaminants), it may be that the spectroscopy lab is an appropriate place to have students examine the nature of science in addition to learning about light and matter.

16

Objective Number of ProfessorsQuantitative reasoning, critical thinking, deductive logic 4Fallibility of data 3The experimental process, “scientific method” 3Link between theory and observation 2Link between observation and interpretation 1Tentative/incomplete nature of scientific knowledge 1History of scientific knowledge 1

Table 4: Nature of science and scientific inquiry goals identified by professors during faculty interviews, along with the number (out of six total) who suggested each concept.

In this study, students were not given pre- and post-lab assessments of change in their knowledge about the nature of science or scientific inquiry. This section is included to highlight the faculty objectives that do not fall under the “content” category, as their existence may be taken into account when re-designing the lab activity.

4.1.3 Affective ObjectivesIn this category, I include some of the following examples of professors’ statements

about their objectives for the students from the laboratory portion of the course: “The experience makes them feel like science may not be for them, but they

know a little more about it.” “I would hope that they come away with a sense of empowerment that they

can do some calculations, for example, the mass of the earth.” “Scientists don’t have all the answers, and this is how they figure them out –

and [the students] too can do science.” “Have the students feel like they are connected with the topic.” “Labs tend to intimidate rather than teach.” “It’s the only real experience they get – they can feel, touch, see for

themselves.”The spectroscopy lab is well-liked by some of the faculty because of how it

relates to affective goals; some feel that it is a popular and enjoyable lab. In an earlier version of the lab, students made handheld spectroscopes that they could take home. Two professors recalled this activity and noted that students could look at car lights, street lights, neon lights, etc., and that the students thought this was “cool.” However, student interest in the topic of spectroscopy may not be high to begin with; Lippert and Partridge (2004) studied students’ interests in the 13 general goals identified by the AAS study on introductory astronomy (Partridge and Greenstein 2003) and found that the topic students were the least interested in, out of all 13, was “Some knowledge of related subjects (e.g. gravity and spectra from physics) and a set of useful ‘tools’ from related subjects such as mathematics.” (When marking their interest, students may have been responding to their dislike or fear of mathematics when ranking this objective so low, so it may not actually reflect the true depth of their interest in spectroscopy and other “physics” topics.)

Changes in attitudes and beliefs about science among the students were not tested by the written pre/post assessments in this study. Surveys exist, e.g. the Science Attitude

17

Inventory II (Moore and Foy 1997), to probe such concepts. However, the professors’ feeling that the students enjoy the spectroscopy lab is backed up by some, but not all, of the interview data.

A few students talked about liking the lab for its hands-on nature and the colorful spectra. About seeing the sun’s spectrum, Aaron remarked, “I think it was pretty amazing.” Others said:

“I liked the colors, liked using the little spectroscope, I thought it was neat.” –Jennifer

“I did like it. I thought it was pretty interesting.” –Carrie “I thought it was pretty nifty.” –Nathan “I thought it was really cool…it was cool to mess around with different

elements and see what they looked like – I liked the air one, where you put it in and it looked almost like nitrogen. I thought that was pretty interesting.” –Jay

However, not all of the students agreed. “I have to say that I didn’t like this lab,” said Regina. Most of the students were frustrated by its length; it was too long for many of them to finish in the class time period, and that led to a decreased appreciation of the lab. Regina: “If we had the time, I think it would have been one of the best labs, for me.” Jennifer said, “I did like this lab; unfortunately, I was taking it very slow…and I didn’t get to finish all of it.” “I enjoyed the lab,” Richard noted, “but I think everybody was pretty overwhelmed by the amount of information.” Other students commented on how they felt unprepared for the difficulty level of the lab; some noted that it was a much more difficult lab compared to the other ones:

“I was a little frustrated because it’s such a new subject for me.” –Regina “I felt that this was the most confusing one.” –Kathryn “This was a very difficult lab for me.” –NathanSeveral of the students, during the interviews, commented on their feelings about

science. “To tell you the truth, I hate science,” Nathan confessed. None of the others were that blunt, but many made comments indicating that science was not, in fact, their favorite subject. They were quick to distance themselves from the field. “I’m not a science and math guy, as you could probably tell,” Jay said. In some cases, the remarks may have been made to cover up embarrassment during the interview about not knowing the answers to questions, but at any rate, it did indicate a certain level of dislike for the subject among some of the students.

4.2 Student DataThis section is divided into two subsections – the quantitative and qualitative

results. As the two overlap (e.g. it is important to know how many students hold a common preconception), the quantitative data section will list only frequencies of correct or incorrect answers on questions explicitly asked on the tests, as well as measured gains, whereas the qualitative section will discuss frequencies of students holding common ideas brought up during the interviews or on the write-in questions of Pre2. The prior knowledge data from Pre1 is also explained further in the qualitative section, with the aim of linking similar ideas revealed from Pre1 with the fill-in answers on Pre2.

18

4.2.1 Quantitative DataPre-Test #1 (Pre1)

The first pre-test (Appendix B) was given to 36 students in two sections. Most of the questions were multiple choice, but two of the questions asked students to draw what would happen to light rays under certain circumstances. There was no post-test to correspond with this particular pre-test, so quantitative learning gains could not be assessed on these concepts (except the ones that appeared on Pre2), but interview data provided information on common ideas expressed on the first pre-test that persisted after the lab activity.

Scores on the relevant multiple-choice questions are given in Table 5. Rather than list each question and its choices in their entirety here, please refer to Appendix B to see the questions. Note that not all the questions are discussed here; several were thrown out, because they were not useful for the purposes of this study.

Table 5: Scores for multiple-choice/true-false questions on Pre1.Question (topic) Correct Incorr. Comments1. (color vs. wavelength vs. energy)

19 17 11 students chose the exact inverse of the correct answer, while 3 each chose the statements that did not describe an inverse relationship between wavelength and energy.

2. (speed of light of different wavelengths)

19 17 This question did not capture information about how many students got the correct answer for incorrect reasons.

3. (amount of the E-M spectrum our eyes can detect)

31 5 Most students knew that most of the spectrum is outside the visible range.

4. (wavelength ranges of the E-M spectrum)

9 27 Most students left out “radio waves” and “visible light” as part of the spectrum

6. (wavelengths emitted by a star beyond the visible)

24 12 Most of the incorrect answerers (10) thought that stars only emit visible light.

7. (color and temperature of a star)

26 10 Most knew that a blue star is hotter than a red star; 7 thought the reverse; and 3 thought you cannot tell a star’s temperature from its color.

10. (filter) 14 22 See Figure 6.11. (color of a sweater) 25 11 5 students say that the red sweater

is absorbing red light, and 6 say the red sweater is emitting red light.

13. (emission of a line) 14 22 13 students answered backwards (thought energy is absorbed when

19

an electron makes a transition down in energy).

Pre-Test #2 (Pre2) and Post-Test (Post)Students’ scores on Pre2 and Post are given in Figure 3. Overall, the scores

improved after the lab activity was performed; the mean score went from 3.7 out of 11 to 5.9, for a class average of 20% improvement in grade. As discussed below, under “Learning gains,” some topics saw great improvement (up to 43%) while others had little significant gain.

To check the statistical significance of the overall score improvement, I ran a two-tailed z-test on the pre- and post-test scores (e.g. Mendenhall and Beaver 1994). The z-test is used for large samples (n>30, where the distribution can be approximated as

Pre2 and Post Test Scores

0

2

4

6

8

10

12

14

16

0 1 2 3 4 5 6 7 8 9 10 11

Score (out of 11 possible)

Num

ber o

f Stu

dent

s

Pre2 Scores (n=55)

Post Scores (n=53)

Figure 3: Student scores on Pre2 and Post (nearly identical tests). The average score on Pre2 was 3.7 (out of 11), and the average score on Post was 5.9.

Gaussian) and compares the means of two data sets to assess the probability, p, that the change in means is due to statistical fluctuation and not to actual student improvement. For the case of my data, I found the z-value, or “test statistic,” by:

z =

1 2

1

2

n1 2

2

n2

(Eq 1),

where μ1 and μ2 are the means of the pre- and post-test scores, respectively; n1=55 and n2=53 are the sample sizes; and σ1 and σ2 are the standard deviations of the populations.

20

The z-value is a point on the x-axis of the distribution, where the distribution is approximately Gaussian. To obtain the probability that the difference in the means is due to random fluctuations, you find the area under the normal curve from the z-value to infinity. This is called the p-value.

The resulting p-value for the pre- and post-test scores was 5.5 × 10-7, which means there is only a 0.000055% chance that the improvement in test scores is due to random chance.

An analysis of student quartiles is shown in Figure 4. All quartiles of the student scores improved by approximately 20%, with the first (lowest) quartile improving its mean by 2.3 points (21%), the second by 2.4 points (22%), the third by 1.9 points (17%), and the fourth by 2.5 points (23%). In other words, the highest quartile improved by 6% more than the next highest quartile; however, within the error bars, the difference

Pre2 and Post Test Scores: Quartiles

0

2

4

6

8

10

12

0 10 20 30 40 50

Student

Scor

e (o

ut o

f 11)

Pre-test ScoresPost-test Scores

Quartile Means (Pre)

Quartile Means (Post)1st Quartile

2nd Quartile (Median)3rd Quartile

Figure 4: Quartiles of student scores. Each quartile improved its mean approximately equally, within the error bars. Error bars are given by the standard error. Grey lines represent the divisions between quartiles for the pre-test group of students (n=55); quartile divisions are slightly different for the post-test group (n=53).

between improvement by quartile is not significant. Therefore, each quartile improved their scores approximately equally; that is, the initially lowest-scoring students improved their scores by as much as the initially highest-scoring students. Error bars on these means are given by the standard error, i.e., the standard deviation of each quartile divided by the square root of the number of students in the quartile.

Learning GainsStudent scores improved for every question on the post-test, but some questions

saw far greater gains than others; some gains were so small as to be statistically insignificant. Table 6 lists raw scores for each question on the pre- and post-test, as well as the percent gain on the question between the two tests. Percent gain is calculated by

21

subtracting the percent correct on the pre-test from the percent correct on the post-test. For each question, I tested the significance of the gain by using a z-test for binomial samples, where the test statistic z is given by:

z =

p1 p2

p1q1n1

p2q2n2

(Eq 2),

where p1 and p2 are the probabilities of answering the question correct on the pre- and post-test, respectively (e.g. p1 = 20/55 for question 1); q1 and q2 are the probabilities of answering incorrectly; and n1 and n2 are the sizes of the pre- and post-test populations (n1=55, n2=53).

The result is that none of the gains of 9% and under (questions 2a, 2b, 8, and 9) are statistically significant, i.e. there is a greater than 5% chance that they are due purely to random fluctuations. The learning gains found on all other questions were large enough to be statistically significant.

Next, because I seek to analyze why some objectives were learned and not others, I grouped the questions by theme and analyzed the learning gains by topic. Table 7 shows the pre- and post-test frequency of correct answers, as well as the percent gain on each topic. The last column lists where each theme was presented to the student – in the

Table 6: Question scores and Learning Gains

QuestionPre-Test 2 (n=55)

Post-Test (n=53)

Correct Incorr. Correct Incorr. Gain1. The stars are very far away; we have not yet sent a spacecraft to the nearest star (besides the sun!). How confident do you think we can be of what the distant stars are made of? Explain. 20 35 42 11 43%2a. How can light be emitted from an atom? 21 34 21 32 1%2b. Using the concept of emission lines, can you prove that atoms are quantized? 5 50 6 47 2%3. Does each element have its own set of unique spectral lines? Why or why not? 24 31 36 17 24%

4. Do you think that hydrogen by itself, H, will give a different spectrum than molecular hydrogen (H2)? Why or why not? 17 38 35 18 35%5. Are light and radio waves the same thing? What are the differences/similarities? 19 36 32 21 26%6. How can spectroscopy be used to study the planets and moons in our solar system? 25 30 36 17 22%

7. You suspect that the sun is mostly made of hydrogen. How might you test your hypothesis in a laboratory here on Earth? 26 29 45 8 38%8. If you looked at the sun using ultraviolet 17 38 21 32 9%

22

{infrared, on Post} light, do you think you could see spectrum lines?

9. What is the difference between an emission and absorption spectrum? When does one occur instead of another? 12 43 13 40 3%10. With the spectroscope you can see yellow, orange, and red emission lines when you observe a neon tube. If you just use your eyes, the tube looks reddish-orange. What is going on? That is, why does it look the color it does to your eyes? 15 40 29 24 27%

lab manual’s text, the lab activity itself, a question in the lab manual, the TA’s lecture, or the professor’s lecture.

Using, again, a z-test for binomial samples, I tested the significance of the gains by theme. In this case, for the themes under which multiple questions fell, I used for p 1

and p2 the average probability of getting the question correct between all the questions in the theme. Here, I found, unsurprisingly, that the gains of 2% and 3% (atomic structure and the light verbs) were not statistically significant (24% and 33% chance, respectively, that the gains are due to random fluctuations), while the gains on the other questions were (i.e. there is a less than 5% chance of the score improvement being due to fluctuation).

What I found from my analysis of the assessment data is that the students are making significant gains on some of the professors’ learning objectives, but no significant gain on others. My interpretations for why gains were achieved on some objectives and not others will be explored in Section 5: Discussion.

4.2.2 Qualitative DataThe open-ended questions on the assessments and the nine interviews I conducted

provided a wealth of information about common ideas held by the students. While the quantitative data presents one picture of student knowledge, the qualitative data – naïve ideas, preconceptions, misuses of jargon – allows a better window into the mental models the students have built prior to and during the lab activity.

Table 7: Learning GainsTheme Pre-

Test Freq*

Post-Test Freq*

Gain** Covered in?***

1 Atomic structure, energy levels 23% 25% 2% LM, TA2 Each element has a distinctive

signature38% 67% 29% LM, Lab, Prof

3 Tool for exploring distant objects, planetary spectroscopy

43% 72% 29% LM, Lab, TA, Prof

4 Light verbs: emit, absorb, transmit, reflect/scatter

22% 25% 3% LM, Lab, TA, Prof

5 Color, filters**** N/A N/A N/A -

23

6 E-M Spectrum, the nature of light

30% 53% 23% LM, Lab, LabQ, TA, Prof

7 Blackbody radiation and temperature****

N/A N/A N/A LM, Lab, LabQ, Prof

*Frequency of correct answer, average of all test questions that dealt with this theme**Absolute gain, Post – Pre***LM = Lab Manual text; TA = TA’s lecture; Lab = Lab activity; LabQ = Question in Lab Manual; Prof = Prof’s lecture, which was given after 66% of the post-tests and interviews were administered****Pre-test 1 and interviews only; no reliable gain measurement

Some of these “incorrect” ideas could be called “misconceptions,” but I would like to use the term “preconception” or “naïve idea” instead. Some of the interviewees were students of exceptional classroom performance, but they too revealed many of these naïve ideas and incomplete or incorrect mental models. The students’ models are internally believable and represent honest attempts at explaining the world around them, although they may run contrary to canonical scientific beliefs. For example, the students who wrote that the difference between light and radio waves is that radio waves carry sound are drawing on their experience in the real world to build their mental model of the electromagnetic spectrum. In their experience, radio waves have always been affiliated with sound. In other cases, where real-world experience provides little intuition (such as atomic structure), preconceptions may come from mis-remembered or never-fully-learned material.

Figure 5 (next 4 pages) consists of charts showing the “correct” or “canonical” idea linked with students’ common ideas revealed in the interviews and open-ended questions on the pre- and post-tests. (Note that Figure 5 shows only the common naïve ideas, and does not represent the percentage of students holding these ideas, nor does it reflect the number of students who answered questions entirely correctly. These numbers and percentages are discussed below.)

In the discussion that follows, I will first examine prior knowledge data from the two pre-tests; next, I will discuss post-test and interview data for each theme separately. For each naïve or common idea discussed, I will provide quantitative information on the number of students expressing the idea, and, when applicable, I will compare the quantitative results of the multiple choice answers from Pre1 with the open-ended answers on Pre2.

Prior KnowledgeThe first pre-assessment, Pre1 (see Appendix B), was designed to test students’

incoming knowledge about concepts more basic than the spectroscopy lab: color, light, filters, and prisms. Some of these topics were identified by professors as information they believe students should know prior to beginning the lab (Table 2). Some, but not all, of the professors interviewed intend to teach some of these concepts in a lecture prior to the students performing the spectroscopy lab. The second pre-test, Pre2, assessed incoming knowledge about specific spectroscopy lab objectives of the faculty. It was designed to be used in conjunction with a post-test to measure learning gains.

24

The results, which are perhaps not surprising but certainly illuminating, showed that students are coming into the course with a wide variety of naïve ideas about light and matter. Knowing what common preconceptions exist among the students is extremely useful for helping to steer students towards the “correct” ideas; as incoming students are anything but blank slates, it is useful to know where they are coming from, academically, in order to guide their learning towards the canonical model. The first question on Pre1 asked about the relation between color, wavelength, and energy. The correct answer out of four choices was: “Red light has a longer wavelength than blue light, and is LESS energetic.” Only half (19/36) chose this option. The second highest response (11/36), “Blue light has a longer wavelength than red light, and is LESS energetic,” is the exact opposite of the correct response, perhaps indicating a rudimentary understanding among students that wavelength and energy are inversely related. The other options were each selected by 3 students. In the same vein, when asked about the differences and similarities between visible light and radio waves (question 5 on Pre2), students revealed confusion about the wavelength/frequency/energy relation. Students thought light and radio waves had the same wavelength, but different frequencies; that light and radio waves both come in short, medium, and long wavelengths; that radio waves are “taller” and light waves are “shorter;” and that light has both higher frequency and wavelength than radio.

The first pre-test explicitly asked the students about the speed of light for different wavelengths, whereas many students brought it up on their own on the open-ended questions of Pre2. On Pre2, ten (out of 55) students said light and radio waves travel at different speeds. The majority thought visible light would be faster than radio, while one thought radio waves would be faster. The question on Pre1 read: “True or false: blue light travels faster than red light.” This question split the class nearly in half (17/36 responded “True”). The question did not take into consideration the few students who may have answered “false” for the wrong reason (i.e. they believe red light travels faster than blue light). At any rate, this reveals a common preconception that students persisted in believing, as indicated by their post-assessments, even after instruction during the TA’s lecture that the speed of light is constant. The confusion about the speed of light may be a result of students being aware of the speed of light through materials being different for different wavelength. It also might be a confusion between higher frequency (units of inverse time) and faster speed.

25

Figure 5: Common ideas expressed by the students in the written pre/post tests and interviews. Frequencies of these ideas are discussed in Section 4.2.2.

Protons orbit the nucleus

CANONICAL IDEASTUDENT IDEAS

Bohr model of an atom: electrons orbit a nucleus (consisting of protons and neutrons)

The atoms themselves move up and down energy levels

Emission is caused by an electron going to a higher energy level

Atoms only emit light during chemical reactions

Atoms only emit light through fusion

Atoms only emit light when electrons are added or subtracted

Electrons emit light when moving very fast

Atoms emit photons when an electron moves down an energy level

Emission lines of different colors correspond to different temperatures

Light emitted from atoms is made up of electrons

Emitted photons correspond in wavelength to the energy difference between levels

ATOMIC STRUCTURE

Electrons can have any energy

Difference in energy levels must correspond to a difference in wavelength range, e.g. infrared to microwave or ultraviolet to visible

Energy levels are quantized; the electron can only assume certain allowed energies

Different elements are composed of different numbers of atoms

Different elements are composed of different basic substances

Different elements have different charges

Elements are composed of protons, electrons, and neutrons; different elements are made from different combinations of these

Emission corresponds to possible energy states, absorption corresponds to impossible energy states

Emission and absorption lines both correspond to allowed energy transitions

26

Doppler shift is related to determining compositionDoppler shifted lines of are used to detect stellar motion, not composition

Spectroscopy can be used to study the composition of distant objects

We cannot determine the composition of distant objects because they are too far away

We know the composition of distant stars only because we know the composition of our own sun and solar system

We know the composition of distant stars because there are not many different types of elements in space

We know the composition of distant stars only because we know the processes that formed them

Spectroscopy can be used to study planets

Because planets aren’t emitting anything, they cannot be studied with spectroscopy

To study planets, you must send a spacecraft there

Planets are too close to use spectroscopy

USES OF SPECTROSCOPY

CANONICAL IDEA STUDENT IDEAS

To determine what elements are in the sun, perform an experiment by putting lots of hydrogen together at high temperatures

To determine what elements are in the sun, compare the sun’s spectrum with laboratory examples of elements

H2 will not have a different spectrum than H, because it is just more of the same element

Molecules have different spectra than atoms, e.g. H2 vs. H

Mars is red because it’s hot (or Neptune is blue because it’s hot)

Planets’ color is not related to temperature (unlike stars)

Planets emit infrared radiation Planets are emitting visible light that corresponds to their color

Planets are emitting nothing

PLANETARY SPECTROSCOPY


27

Spectrum is just the visible – commonly make reference to red being the longest wavelength of all

Spectrum goes from infrared to ultraviolet

Most of the E-M spectrum is outside the visible range

E-M waves travel at different speeds:Light waves (higher energy) travel faster than radio waves (lower energy)Radio waves travel fasterBoth light and radio waves can travel faster or slower

E-M waves travel at the speed of light, regardless of wavelength

Radio waves “carry” or “transmit” or are caused by sound

Radio waves travel differently than visible light waves:Radio waves are vibrations or pulses, whereas light is notLight waves and radio waves oscillate differentlyRadio waves need an atmosphere to travel inRadio waves can travel through solids, whereas light cannot

Radio waves are a range of the E-M spectrum, just like visible light

ELECTROMAGNETIC SPECTRUM

Electrons, atoms, or molecules carry light

Only visible light is a particle in addition to a wave; radio waves are not

E-M waves behave as both a wave and a particle (known as “photons”)

Light and radio waves have the same wavelength, but different frequencies

Both light and radio waves can have long, medium, or short wavelengths

Light is “higher frequency and wavelength” than radio

Frequency and wavelength are inversely related


THE LIGHT VERBS

Stars/the sun reflect lightStars emit light

An emission spectrum is light reflected off the gas you are observing

Gases that are emitting light give off an emission spectrum


28

Students had some other thoughts on the differences between visible light and radio waves. Five students mentioned that radio waves carry, transmit, or are caused by sound. Additional students did not mention the word “sound,” but described how radio waves travel differently than light waves, alluding to the concept of transverse versus pressure waves. Some of the interesting answers included:

“Light waves measure the amount of light. Radio waves measure the amount of sound.”

“They are both in the spectrum, but travel differently.” “Light waves oscillate like this (~), while radio waves are like this (↔).” “Radio waves travel via atmosphere.”

BLACKBODY RADIATION AND TEMPERATURE

Blue star has higher-energy radiation because of its hotter temperature

Blue star is hotter because of its higher energy radiation

Stars emit more high-energy light than low-energy light (or, less commonly, vice-versa)

Stars emit all wavelengths of light equally intensely

Stars emit a blackbody curve

Blue stars are hotter than red stars Red stars are hotter than blue stars


Planets emit visible light that corresponds to their color

Planets are reflecting light from the moon

Planets’ visible color comes from reflected solar radiation; different wavelengths are absorbed differentially

COLOR AND FILTERS

Sweaters, binders, and leaves (examples from the interviews) emit visible light that corresponds to their color

Room temperature objects, such as clothing, reflect and absorb different wavelengths of the ambient light differentially to produce different colors

A red filter absorbs red light

A red filter changes all the colors of light to red

A red filter absorbs all the colors of light except red


29

“Light waves are much thinner and travel very fast, in fact faster than anything can travel. Light waves transmit light and radio waves transmit sound.”

“Light waves cannot travel through solids whereas radio can. Light is faster than radio.”

“Both are waves. Radio waves are the result of vibrations, resognated [sic] at different frequencies, while light waves are dependent on light.”

“Light waves travel in a direct path. Radio waves travel in pulses.”

Two questions on Pre1 probed students’ knowledge about the wavelength ranges of the electromagnetic spectrum. No one thought that “our eyes can detect the entire spectrum of wavelengths,” but 5/36 (14%) of the students thought our eyes can detect most of the spectrum. When presented with a list (X-rays, gamma rays, visible light, lasers, cosmic rays, radio waves, infrared waves, microwaves, and ultraviolet rays), students were asked to identify which were a part of the E-M spectrum. A common trend was for students to get it all right, except they would either leave out visible light or radio waves, suggesting that these two ranges stand out as “unique” in students’ minds as being different from the rest of the spectrum.

Color and filters were another set of topics tested on the first pre-test. When asked why a red sweater appears red in a room with a white light, most (25/36) correctly answered that it is because the sweater absorbs all the other colors except red. Five

Red filter

0

2

4

6

8

10

12

14

16

a: red gets through b: blue gets through c: colors turn red

Figure 6: When asked what happens to a spectrum of colors viewed through a red filter, more than half the students (22/36) answered incorrectly.

students answered that the sweater is absorbing the red color, and six (17%) answered that the sweater is emitting red light. (Compare this to the interviews, where quite a few of the students – five out of the eight who were asked, or 62% – thought that Mars is red because it is emitting red light. When asked about my sweater, however, two of those knew the sweater wasn’t emitting light; they put planets and sweaters in different categories. The other three, or 37%, said that sweaters emit visible light that corresponds to its color. See the discussion below under “color and filters” for more post-test/interview results.)

30

To test the students’ understanding of what filters do, one question asked what would happen to a rainbow spectrum of colors when viewed with a red filter. They had three choices: the red gets through; the blue gets through; and all the colors turn red. Well over half the class answered this question incorrectly (Figure 6), with nearly equal numbers choosing each of the three options.

The light verbs suffered from much misuse at all phases of the lab: the pre-tests, the post-tests, the students’ final lab write-ups, and the interviews. A common preconception was that light is emitted from an atom by photons reflecting off the substance that is being observed. Another frequent misuse (both on the prior knowledge tests and the post assessments) was the tendency to say that light reflects off of stars, for example, “The study of spectroscopy uses colors to determine the light that the stars reflect.” One student thought that only visible light can be absorbed, whereas emission could be “produced by anything in the spectrum.”

Regarding Theme 3, the use of spectroscopy to explore distant objects, the most common preconception was that spectroscopy is not how we know about the composition of other stars. The question asked (#1 on Pre2): “The stars are very far away; we have not yet sent a spacecraft to the nearest star (besides the sun!). How confident do you think we can be of what the distant stars are made of? Explain.” Students offered the following suggestions, which can be grouped into two categories. First, we know what distant stars are made of because we know how our own star and solar system was formed:

“Very, because the solar nebulas are all the same stuff and act in a way much the same as used in creating our solar system.”

“Fairly confident because we know about how stars are formed and how our solar system is formed.”

“We can be very confident of what distant stars are made of because we know that planets USED to be stars. Therefore, we know what is inside actual stars even if we cannot get near them.”

“Very confident. We have studied our own solar system enough to find out what our own sun and planets are made of and, based on the Doppler shift of the spectroscopy charts for other stars, can figure those out too.”

Second is the notion that we know what the distant stars are made of because we know what elements make up the universe, and the universe is uniform:

“There are only so many different types of elements in space and they should interact similarly due to the basic rules.”

“Almost completely certain by the principles of uniformitarianism.” “Extremely confident considering we know what materials the majority of

everything else is made up of.” “Pretty confident, there are only a certain amount of elements in the universe

that could behave the way a star does.” “Because everything is made from the explosion at the beginning, so we can

figure they have the same properties.”These preconceptions do not allow space for the idea that the stars actually have

differences from each other in composition, and that we can tell something about these differences from spectroscopy. This idea, the use of spectroscopy to study objects remotely, is a primary objective for several professors, and the learning gains on this

31

topic are discussed in the post-test results below. In all, on the pre-test, 13 students (24%) felt that we know about the composition of distant objects only because of our knowledge about elements in the universe or in our solar system, while 20 students (36%) knew that we can use spectroscopy.

Four students suggested that we know what the distant stars are made of because of the Doppler effect (e.g. the example above, and also “Because of the Doppler effect, we are pretty sure of what distant stars are made of”). Four more students, later, invoked the Doppler effect for understanding the composition of planets.

Another objective, which also falls under Theme 3, was for students to learn that spectroscopy is performed by comparing an astronomical object’s spectrum to the spectrum of various elements in a laboratory here on Earth. When asked about how they would test the composition of the sun, however, nine students (16%) replied, in all apparent seriousness, that we should build a model sun by putting “lots of hydrogen together at high temperatures” and “perform fusion to see if the result is similar to what we see from the sun.” “Study the characteristics of igniting hydrogen on earth,” another wrote. Overall, 53% of the students answered this question incorrectly.

On the topic of atomic structure (Theme 1), students exhibited confusion about the basic building blocks of atoms, elements, and molecules, and also mistook electrons for photons. Students had a few different notions about what makes the elements different from each other: they are made up of different numbers of atoms, or they are composed of different basic materials. Each element has a different spectral signature, one student wrote, because each has a different charge.

When asked about how light is emitted from an atom, a fair number (21/55, or 38%) already knew that an electron changing its energy level would cause photons to be absorbed or emitted. Other common ideas included that light is emitted only during fusion (three students); chemical reactions (two students); that electrons emit light when moving very fast (one student); and the atoms themselves are moving up and down energy levels (two students). A few students had the basic concept correct but mixed up the direction of energy change (the electron goes from a lower to a higher level and releases light).

Post-Test/Interview DataThe themes I probed in the interviews included concepts that were not asked

about on the pre and/or post tests, such as blackbody radiation, temperature, color, and filters. I also asked about some of the topics covered by the written assessments (atomic structure, planetary spectroscopy, and emission/absorption) for the purpose of comparison and for the clarification of what mental models the students have built. The interactive nature of interviews allows a much more in-depth analysis of students’ commonly held ideas than the written tests. As an example of one of the nine interviews, Interview #4 (Carrie) is attached as Appendix C.

Some of the concepts the students are trying to explain are “basic,” while some are higher-level and much more challenging. Students struggle with being put “on the spot” and with suddenly having to develop a coherent theory. They occasionally contradict themselves, sometimes in the space of just a few sentences. Also included are some very common misuses of scientific jargon. In some cases, the misuse of language may not be a problem, since sometimes the students’ sloppiness in phraseology did not

32

correspond to confusion about the concepts. They simply misspoke or did not remember the new jargon presented in the lab, sticking instead to their own chosen words and phrases for various concepts. Nathan called electrons “elements” and repeatedly called energy levels “power levels,” while Aaron called them “energy fields.” Regina liked to call emission lines “light bars,” a very descriptive tag. However, in the case of the “light verbs,” which were a faculty objective, the misuse may reflect a deeper uncertainty within the student about which verb applies to which process.

The data collected painted an interesting picture of the students’ outgoing knowledge about the various topics and themes in the spectroscopy lab. Students’ post-lab ideas about concepts within the seven main themes are discussed individually below.

Theme 1: Atomic structureIn the post-tests, students revealed the same basic confusions about electrons,

protons, atoms, and molecules that they had during the pre-tests. Kathryn, in her interview, was asked to draw an atom; she drew a filled-in circle (the nucleus) and then hesitantly drew another circle outside it. When asked what the outer circle was, she said, “The pro.....does it start with a p?” While it is true that Kathryn’s mistake of confusing a proton and an electron may have just been one of vocabulary, her hesitancy in even wanting to draw the outer circle suggests that her mental model of the atom may be incomplete.

The spectroscopy lab, with its immediate launch into energy levels and spectral lines, may have its starting point at a level beyond what some students are prepared for. That Kathryn’s inability to draw the atom came after the spectroscopy lab was completed also suggests that, while she was presented with a diagram of an atom in the lab manual (Figure 7), the lab activity in general did not serve to fill in the gaps of more rudimentary missing knowledge. Kathryn’s responses when asked about spectroscopy indicate that she did not grasp some of the more advanced material either. With an incomplete mental model of atomic structure, a lesson on spectroscopy was very confusing. “I felt like I was really lost on this lab,” Kathryn said. “I just didn’t really understand lots of it.”

As on the pre-test, students on the post-test confused the building blocks of atoms (electrons, protons, and neutrons) with elements, compounds, and molecules; for example, a student wrote that “each element consists of a different combination of atoms.” Students seemed to go out of their way to avoid using specific terms: they would say the “materials” or “makeup” or “substances” or “compounds” are different between different elements. Others said, on the post-test as well as on the pre-test, that each element has its own signature spectrum because, for example, “they have different amounts of atoms (ingredients).” One student, on the post-test (compared to 2 students on the pre-test) wrote that different elements have different charges.

Of the four students who were asked about (or, in one case, brought up the topic herself) the quantization of atoms during the interview, two answered correctly that electrons could only occupy certain energy levels. “There weren’t an infinite amount of colors or levels that we saw, there were only certain ones…I would say that we can say they’re quantized,” Carrie said. Regina’s answer was that “it can only emit from those circular orbits…like certain photons can only be emitted say from different frequencies. I can’t give you an example because I don’t know, but like a hydrogen atom would emit like from different frequencies and a different carbon atom would emit different

33

frequencies.” The students who answered incorrectly thought instead that when we draw concentric circles around the atom, it is just “for the purpose of drawing,” as Nathan put it, but the electron can actually be anywhere. Maybe they are recalling their high school chemistry classes, where they probably learned about electron orbitals that are not spherical.

Figure 7: Diagram from the UCB Astronomy 1010 manual showing the Bohr model of an atom and the cause of emission lines of different colors.

A hazy recollection of the Bohr model leads to confusion about energy levels: “The atoms are moving within the atomic structure, moving up or down energy levels,” Richard says, indicating a belief that the atoms, rather than the electrons, are moving up and down the rungs of the energy ladder, just as a few students indicated on Pre2. These students have built an internally believable mental model that happens to disagree with the model we want the students to remember, which is that it is the electrons changing energy levels. Further, although we would like the students to build themselves the Bohr model, that too is a scientific simplification, an incomplete and nonphysical model.

To probe the topic of how a line is emitted, I asked the nine students to draw an emission spectrum on the board, and despite the bulk of their lab consisting of drawing such spectra, only five of the nine drew one correctly. The other four drew continuum or absorption spectra, with Richard drawing the latter and including the emission spectra above and below that mimicked the solar spectrum with comparison lines from the observatory heliostat. Carrie said, as she drew an absorption spectrum, “I think, so…emission is just where there are…let me think about this…is that where there are just sections missing, and everything else would be colored?”

When asked about what causes emission and absorption lines, many of the students demonstrated that they understood at least some of the critical components of the model. At first, I had three students (Jay, Aaron, and Kathryn) who looked blank and could not answer the question, but when asked to draw an atom on the board, they quickly recalled that the electron would move up or down an energy level: “OH! I know what’s causing the lines! Electrons jumping back and forth from certain levels to another,” Jay said excitedly. “That totally sparked it.” Other answers to that question included:

34

“I guess the visual that I have is when we were tossing those atoms back and forth and he was moving up and down…levels on the ladder, so when it’s emitting it’s going to be pushing you back down a level, when it’s absorbing it’s going to be moving up a level…the atoms are moving within the atomic structure, moving up or down energy levels.” –Richard

“In order for an atom to emit energy or light, they have to go down to another…a lower energy level…that’s when they emit light, and the light emitted is equal to the difference between the two energy levels and it comes in photon form.” –Jennifer

“So here’s your atom, and then you have the electron cloud outside. Now when an electron jumps from an inner energy field to a more outer energy field that requires energy. And so that is expressed by absorption.” –Aaron

“Um, when it moves down it’s…releasing energy, I think, and that’s….I keep getting them mixed up…one of them is when it goes down, it releases energy, and when it moves up, it’s absorbing energy, and one of them emits light.” –Kathryn

“I guess I don’t really understand…like I get that each one [element] has its own property but I don’t understand why…what the lines mean.” –Hal

On the written post-tests, two students answered that the way light is emitted from atoms is by fusion, and another wrote that it comes from “being charged.” One thought that protons were involved in the process. Two said that emission is caused by electrons being “added or subtracted” (or, conversely, another wrote, “Absorption is when an electron is gained.”) While it is true that light is emitted during the process of fusion, and that light can be emitted during ionization, these answers avoid the concept that transitions between energy levels are responsible for the emission lines the students saw in the glowing gas tubes.

Like Jay, Aaron, and Kathryn, Carrie was reminded of the answer as well, when asked about what causes absorption lines:

Carrie: “What caused the gaps…ummm…I’m not so sure about that.”Amy: “Say you had an atom in between your light source and you. What’s happening in the atom that would cause those?”Carrie: “Ahhh, when an electron…would move up or down a level depending on its…emission and absorption.”

Others also recalled that emission and absorption involved an electron switching energy levels, but could not remember which was which. This was common on the written post-tests as well; several got it backwards, and many just avoided the issue by being nonspecific, e.g. electrons “jump from one level to the next and each time they jump, they emit a certain amount of energy.” Nathan had it backwards, as line emission occurs when an electron moves to a lower energy level:

Nathan: “There’s a bunch of different power levels they [electrons] can jump from and if you bombard an atom with lots of electrons, it changes.”Amy: “Can you draw for me what would happen to that electron in order for something to be emitted?”

35

Nathan: “Uh, there’d be more stuff {draws an arrow towards the atom} that would need to bombard it to raise it to a different power level…there need to be more electrons…you need to bombard the atom, then this electron I guess would jump off to a different energy level {waves hand away from the nucleus} and emit some sort of energy.”

Except for Nathan’s notion that electrons are bombarding the atom (instead of photons) and that the direction of energy change is incorrect, his model is reasonable, as it includes energy levels (which he calls power levels) and he knows that a change in energy causes emission. Jay also got it backwards: “Emission would have to be, I would say, when it’s moving up an energy level and then absorption would be when it’s coming in, jumping from say out here to in here.”

Despite answering correctly what caused emission and absorption lines, some were at a loss when asked to explain why we see lines at different wavelengths, failing to make the connection between energy change and the wavelength of the emitted (or absorbed) photon. Of the students thus queried, only Jay and Carrie answered correctly: “I guess the colors represent different energy levels,” Jay said. Kathryn said she didn’t know what caused the differently colored lines, despite answering correctly what causes emission lines. Richard and Aaron thought the colored lines they saw in the gases’ emission spectra were caused by different temperatures:

Amy: “Why does the atom create those different lines? Why red? Why blue?”Aaron: “Because it’s at a different temperature.”

Theme 2: Each element has a distinctive signatureThis was one of the themes where students made clear learning gains, and it was a

primary faculty objective for three faculty members. Several mentioned the metaphor of spectral lines as “fingerprints of elements,” with one also comparing spectra to bar codes: you don’t need to know the exact numbers on the bar codes; “what you want to know is the price of the soap.” Some of the students picked up on the phrase “fingerprint” in their interviews: “If it’s like a hydrogen atom or helium atom or carbon, it gives off like a fingerprint of spectral light…so if you’re looking at like a sodium gas light it gives off the specific spectrum so you can say, ‘Oh, that’s what that is,’” Regina said. Another student said it was “like DNA fingerprinting,” bringing in a more modern metaphor. Even Hal, who appeared to struggle with this particular lab, brought up “the fact that every element has its own signature.”

During the lab activity, the students were required to draw the emission spectra of molecular nitrogen (N2), and the lab manual explained why molecules produce wide bands instead of thin lines. On the written assessments, students made a large (35%) gain on the question that asked whether molecular hydrogen would have a different spectrum than H alone. Those who still didn’t think so, on the post-test, tended to make comments such as “No, because it is the same element, just different amounts of it” (which was the same reasoning given on the pre-test). Some said it would be the same spectrum, only brighter. Conversely, some of the students who said yes, H2 will have a different spectrum than H, gave the incorrect reason that there is a greater amount of hydrogen.

36

Some of the students are aware that changing the amount of an element will not change the appearance of the spectra, while others are not.

In his interview, Richard explained, “An atom is going to interact differently than a molecule because there’s going to be different energy levels. The emission and absorption’s going to interact differently if it’s a molecule versus if it’s just an atom by itself just because of the – I guess the degree to which it’s built up.” At the introductory class level, this answer is, I think, I good one.

Overall, I feel that this objective was well-absorbed by the students. Theme 3: Spectroscopy is a tool for exploring distant objects, including planets

Theme 3 deals with the idea of how scientists support their claims with evidence. As Brickhouse et al. (2002) note, “Science educators have argued that it is insufficient to be able to recite the theories of science and not know how knowledge claims are justified, what counts as evidence, or how theory and evidence interact.” The questions on the pre- and post-tests involving this theme attempted to get the students to talk about the use of spectroscopy as a scientific tool and the link between comparing an astronomical spectrum with an element’s spectrum in a laboratory.

On the written post-assessments, students made a vast improvement on the question about whether we can use spectroscopy to study distant objects (question 1, Pre2 and Post). On the pre-test, only 36% said we could be confident of what the distant stars are made of from spectroscopy, compared to 79% on the post-test, a gain of 43%. On the pre-test, where 13 students said that we are confident about the composition of distant stars from our knowledge of the elements in the universe or in our own solar system, only 6 students invoked such claims on the post-test. Some combined it with the new knowledge that spectroscopy can be used, e.g. “Fairly confident. We are able to take spectra of distant stars. … Also, we can say that the stars have similar composition to our sun.”

Six of the nine interviewed students asked about studying distant objects using spectroscopy correctly responded that we can learn a great deal about composition. Nathan’s comment was, “It’s a great way to study stuff that you can’t physically go up to and observe in person.” Kathryn: “Just looking at the emission lines we can tell what they’re [the distant stars are] made out of and what atoms and things they consist of…Or like, what kind of gases are in them. Although…there’s things that can interfere, like if there’s an atom of something, or something else is blocking, or won’t let us see the emission lines or colors, so it could be a little distorted.” Carrie said, “I would say, fairly confident – it seems, anyway, from using spectroscopy. The only disadvantage I guess would be when the elements are actually in our atmosphere and not in the star.” Aaron’s reply was, “We can pretty much infer what element a certain substance is without knowing or without seeing the visible substance, but simply by viewing the spectrum and the light that’s given off…by noticing the pattern of the specific spectrum we can deduce what element it is.”

The others, however, remained unconvinced. “I don’t think, just using the light to detect and see what the composition of the planet or star is, I don’t think we can do that,” Jennifer said. Other students, especially on the pre- and post-tests, wrote that we can be fairly confident about what other stars are made of for the reason that we know what the entire universe, our own star, and our solar system are made of. This answer is too

37

general, since there is a whole range of stellar compositions, and distant solar systems may have very different abundances of elements. Jay said, “I think we can be pretty confident, considering we know what basically every element in the Earth and in the sun and all the other planets are, and we know what elements were in the solar system before it even became a solar system…what particles were floating around and what moved where, so I’m pretty sure we can be extremely confident in what the stars are made up of.”

The use of spectroscopy to study planets was a primary objective for some. Some of the students, however, didn’t think we could study the planets in our own solar system using spectroscopy or felt it was of less importance. Jennifer thought you would need to sample a planet directly. Other interview responses included:

“I guess really the only way to do it would be like a flyover or whatever cause they don’t really give – they emit light, but…they’re kind of far away.” –Jay

“That I wasn’t so sure about, just because it [a planet] doesn’t emit its own light; I don’t know if you could use it [spectroscopy].” –Carrie

“I would think not as much, because they [planets]...it’s not just like, light being emitted, there’s, um, I wouldn’t think so.” –Kathryn

“Planets and stuff, or like stars, we can…tell what they’re made of from the light they give off.” –Hal

“Planets, the sun, stars. I think stars, more importantly.” –RichardOn the written tests, similar statements were abundant (35% of the class did not

think planets could be studied using spectroscopy). Most wrote that we could not do it because planets aren’t emitting light (e.g. “no, spectroscopy is used to evaluate light sources and no light is emitted from the planets, as they are illuminated by the sun”), indicating that the idea of reflectance spectroscopy was not well-understood by many. One student wrote, “Besides shooting seismagraphs [sic] at planets, we can’t detect what they are made of by looking at the light spectrum from planets.” Another wrote, “Only if we obtain elements from those planets, and spectroscopy won’t help then.” This student seems to be suggesting that the elements found on planets can’t be found on Earth, and so to get samples to compare with the planet’s spectrum, we would need to travel to the planet anyway, at which point we wouldn’t need to do spectroscopy remotely.

Others knew that spectroscopy could be used to study planets. Richard went on to recall, correctly, that spectra of solids are different from spectra of gases. “I think it’d be different,” he said, “because objects…the light they emit’s related to temperature, versus their composition…So I think the spectrum would be a little bit different if you’re looking at an object versus when you’re looking at the elements.” Nathan said, “[You’d] probably see…depending on the planet…probably see different spectra from the light that’s being emitted from it.” (Under Theme 5, color and filters, there is discussion of the students’ beliefs about what kind of light planets are emitting.)

Another faculty objective within this theme was the concept that we learn composition information from looking at the astronomical object and making a direct comparison to spectra seen in the laboratory, just as the students do with the sun’s spectra when they check for neon and hydrogen. When asked about it on the pre- and post-tests, however, few students made the leap to suggesting a direct comparison; their answers were mainly along the lines of, “use spectroscopy to figure out the composition.” Therefore, it is unclear from the written test data whether the link was clearly established

38

in most students’ minds, but it is clear that improvement was made in the numbers of students making the link. On the pre-test, about half as many students mentioned comparing astronomical spectra with lab spectra as compared to the post-test.

In the interviews, most of the students made the connection:

Amy: “If you looked at a star with a spectroscope and it split the light into the different wavelengths, and you saw a bunch of lines…how would you find out what those lines are caused by?”Carrie: “By looking at…how different elements have different absorption lines…comparing them to other elements that you can view the same way…I think?”

“They could just get a tube of them [the gases] and then the sun, and see if there’s any similarities in them,” Jennifer said. Richard thought “that’s probably the most interesting aspect, just making that connection between what you can do on Earth and being able to identify what’s out there.” Hal, on the other hand, wasn’t sure how you would identify elements in distant stars: “Ummm…I don’t know,” was his answer to the question above.

Theme 4: The light verbsThe students make rampant misuse of the “light verbs” (emit, absorb, transmit,

reflect/scatter), which were a faculty objective for the students to learn. Jennifer thinks that things that are far away aren’t emitting anything (“There’s no light emitting, like, from it, cause it’s so far away.”) Aaron and Carrie say that stars are “reflecting light” towards us, as does the student who wrote on the post-test, “the light reflected off of distant stars…” Aaron, when trying to explain what caused the differently colored lines in an emission spectrum, misused “reflection” again: “So the reason that we see blue is because blue is the thing that’s being reflected.” I asked, “Reflected off of what?” He replied, “Off of the substance that you’re trying to observe.” In other words, his belief is that an emission spectrum – which we observed coming from tubes of hot gas – involves the reflection of light off that glowing substance. On the written post-test results, other students also expressed a belief that emission lines come from the reflection of light off the gas or substance, e.g., “Emission is the color that is bouncing off the element,” and “Emission is the light that is being reflected off.” While the most common idea was to confuse emission with reflection, one student wrote, “Absorption is light reflected.”

When asked to explain when we see an emission spectrum and when we see absorption, few answered correctly. Nathan said, “Emission line…isn’t that when the light’s behind the gas and…um…kind of lost here. Isn’t that when there’s a light source behind the gases and it’s the light the gas reflects?” Regina, on the other hand, is closer to the correct idea: “An emission spectrum would be like looking at a tube of gas, or the sun, and an absorption spectrum would be if there was like a cloud between the sun, the radiating object.” Regina identified the sun as an emission spectrum because “it’s emitting light,” but the solar spectrum we see from Earth is an absorption spectrum because of particles in the sun’s atmosphere.

Other ideas about emission and absorption lines were given by the students as well. One student wrote on the post-test that “emission shows all possible energy states,

39

absorption shows all non possible.” Another common idea was that the black part of an emission line spectra corresponded to absorption by the element.

As mentioned above, Richard believed that when you see an absorption spectrum, you also see the emission spectra above and below the absorption spectrum. This interesting idea was borne from the setup we have in the lab: the heliostat spectrum projected with comparison lines above and below. He said:

Richard: “The absorption spectrum will occur within the spectrum so these will be black and then the emission spectrum is going to occur outside of it…{draws lines above and below absorption spectrum, some matching the absorption lines, some not}Amy: “So if I take a spectroscope and point it at any object, that’s what I will see?”Richard: “Yeah, exactly. You’ll see the clear delineated lines and then you’ll have colors but you also have the areas where nothing’s happening.”Amy: “What would you call the one on top and what would you call the one on bottom?”Richard: “Here and here?” {Points to top and middle spectra}Amy: “Yeah.”Richard: “The absorption and the emission.”

Richard’s idea that we will see emission lines with absorption lines may also come from a part of the TA’s lecture. When I introduced the idea that atoms reradiate a photon in any direction after an absorption, a student in the class asked if that means we’d see emission lines along with the absorption spectrum. I answered yes, that there would be very faint light from emission inside the dark absorption lines. Later, when Richard saw the solar spectrum with apparent emission lines above and below, he may have mixed the two together to build his model of what an absorption spectrum looks like.

Theme 5: Color and filtersOne of the most startling areas where the students displayed naïve ideas was with

the color of solid objects, like clothing or planets. On the first pre-test, six out of thirty-six students answered that they thought a red sweater was emitting red light. The interviews revealed a different story, however: a much higher percentage of the students harbored confusion over what causes the color of objects.

A common mental model that the students appear to be building as a result of this lab is one that incorrectly extends what they’ve just learned about the relation between the temperature of a star and the color of light that’s being emitted (Theme 7) to planets. During interviews, many students explained that planets are emitting visible light that corresponds to their color. (This can also be classified as confusion regarding Theme 4, the light verbs.) For example, Jennifer said, “Mars, I think, is red. Whatever that means. Kinda hot, maybe…I thought that had to do with its temperature.”

Similarly, I asked Carrie: “Mars looks red. A planet like Neptune looks blue. Does that tell us anything about the temperatures of the objects?” She replied, “I would think…it seems like it does for stars, I don’t know why it wouldn’t for planets. I’m

40

trying to think of the temperature of Neptune compared to that of Mars…Mars is cold? I would guess Neptune is hot…if that is true, then I suppose it would tell us something.”

When asked immediately thereafter if the same rule applies to everyday objects, like my sweater and binder, most felt that it did not. I asked Carrie, “My sweater is red, and my binder is blue. Does that mean my binder is hotter?” and Carrie quickly and emphatically replied, “No!” On the other hand, with much less certainty, Jennifer said, “[Your red sweater] doesn’t seem hotter [than your blue binder] but it might be hotter, you know. My guess…we can’t – with planets, yeah, but like with binders and sweaters, no.” (Jennifer, displaying a mistaken recollection of Wien’s Law, thinks that red stars are hotter than blue; see Theme 7 for more on that topic.) Jennifer follows this statement with a correct response about why the sweater looks red: “Because the red is being reflected. All the other colors are being absorbed.” The students who were in this camp were relying on their knowledge that sweaters and binders, no matter what color, are both at room temperature. Planets, on the other hand, are not objects whose temperatures they are positive of, as in the statement by Carrie above.

Regina, on the other hand, felt the same rules that apply to planets also apply to clothing (on this day, I was wearing an aqua blue sweater):

Amy: “What about Mars? Where would Mars peak [in its thermal radiation]?”Regina: “Peak out? In its thermal radiation? In the reddish area.”Amy: “OK. How about my sweater? Where would my sweater peak?”Regina: “Aqua. Um…so the emissions would peak at the bluish green area.”

Richard and Aaron also think that sweaters (and, in Aaron’s case, a green leaf) are emitting a spectrum of light that corresponds to that color (instead of reflecting it), although they don’t mention whether this is thermal radiation. Richard said, “In the case of like, Mars…you probably would just have a real specific spectrum of light that was emitted because…what we’re seeing with the naked eye is what is being emitted…or with our shirt, you know, the green, blue. Yeah, I think it would be a real specific spectrum of like a certain color or a certain range of colors.”

Hal’s theory about why sweaters and stars are not analogous is simple and incomplete: “This [your sweater] isn’t light, though. Light acts different than, like, dye and stuff like that.” He doesn’t make the distinction that the star is emitting light while the sweater is reflecting it. When pressed as to the reason my sweater appeared red, he replied, “I don’t know! It’s dyed that way!” Jay also appealed to the difference between light and a solid, dyed object. “Because for one thing,” he said, “the temperature of this [points to the light] is a lot warmer than the temperature of you, and the sweater is a piece of fabric, whereas this [points to the light again] is different elements.” The first part of his statement suggests he understands that humans and lights are not emitting the same wavelengths of light, based on their temperatures, and this is why the light is glowing but my sweater is not. The second part of his sentence is similar to Hal’s model in that he knows there’s a difference between light and fabric, but has difficulty articulating how this relates to their apparent colors.

The students who did not think the planets are emitting visible light, like Kathryn, tended to believe, instead, that the planets are emitting nothing. Jay also exhibits some confusion on the subject:

41

Amy: “Is Mars emitting light and if so, what kind of light?”Jay: “Mars? No, it’s not emitting light…but it looks like a star from down here.”Amy: “If it’s not emitting light, why do we see it in the night sky?”Jay: “I guess it would have to be emitting light if we see it in the night sky…or it’s a reflection, not a reflection, but the moon shining onto Mars….but it’s too far away for that. So it’d have to be emitting light. I wouldn’t be able to tell you what kind of light, though, ’cause I’ve never looked at Mars through a spectroscope.”

Jay forgets that light from the sun could be what is reflecting off of Mars, bringing up the moon instead. Perhaps this stems from an association of the moon with the night sky and the sun with the day sky. He cleverly avoids having to say what kind of light Mars is emitting by saying he’s never looked at it through a spectroscope, which indicates that he may realize it is emitting light outside the visible portion of the spectrum, or it may not.

Theme 6: Electromagnetic spectrum and the nature of lightIn Astronomy 1010, the students were first presented with electromagnetic (E-M)

radiation during the TA’s lecture. The E-M spectrum ranges from radio waves (the longest wavelengths, lowest frequencies, lowest energies) to gamma rays (shortest wavelengths, highest frequencies, highest energies). In between, in order of decreasing wavelength, are microwaves, infrared radiation, visible light, ultraviolet light, and X-rays. All photons of light, regardless of energy, travel at the same constant speed in a vacuum.

Some students had clearly learned about the E-M spectrum beforehand, as evidenced by a question on the first pre-test (discussed in Section 4.2 above) that asked them to identify wavelength ranges, a question which 7 out of 36 students answered correctly. Among the wrong answers, the most common problem was students leaving out “visible light” or “radio waves” as a part of the E-M spectrum. On the post-test, students gave some incorrect responses about the makeup of the E-M spectrum, such as “radio waves include microwaves, X-rays, UV” and “radio waves include light.”

Another question asked whether the speed of light is the same for different wavelengths; almost exactly half the class answered the question wrong. On Pre2, 10 students wrote that light waves and radio waves travel at different speeds (note, however, that nearly half the class, on Pre1, believes that blue light travels faster than red light, so it is an even more common idea than these numbers indicate, as Pre2 did not explicitly ask about the speed of waves, just for the difference between light and radio waves). On the post-test, 7 students (13%) still offered that idea. Interestingly, one of the highest-scoring students on the post-test – the student correctly answered even the hardest question, which only 6 out of 53 got correct – was one of the seven with the post-lab notion that radio waves travel slower than light. Zeilik and Bisard (2000) find a frequency of 27% of their students (introductory astronomy students at the University of New Mexico and Central Michigan University) believing radio waves to be slower than visible light after the course is over, compared to our 13%, which is an extreme lower bound (for the reason mentioned above).

42

Fewer students wrote on the post-test that radio waves carry sound (only 2 on the post-test, compared to four on the pre-test, although more on the pre-test indicated confusion about the way radio waves propagate), but the idea persisted for some: one student wrote, “Light waves travel faster [than radio] and emit light. Radio waves are vibrations that create sound.” Another answer, extremely similar to an answer given on the pre-test, was “light waves measure light and radio waves measure sound.”

During the interviews, a few students made similar errors about the speed of light versus radio, although none of the interviewees mentioned the association of radio with sound. Aaron, when asked about detecting radio waves, said, “No, we can’t, because those waves – radio waves – go way too fast. Their wavelengths are way too long.” (The more common idea among students was that higher energy photons, the shorter wavelengths, travel faster.) Other students did not express mastery of the relationship between wavelength, frequency, and energy; Carrie, for example:

Amy: “What’s the difference between light waves and radio waves?”Carrie: “I think it’s that they…they’re similar in their wavelengths but I think light waves have a higher frequency than radio waves…”Amy: “And what about their speed?”Carrie: “Light waves are faster…than radio waves are.”

Some students could recall the relationship, if not its exact details. Nathan: “See, I can never remember this, but I can kind of understand. The short wavelengths are higher energy, like the real jagged one, like gamma rays; gamma rays are real high energy and then the radio is real long and it’s low energy…or is it vice versa? Yeah.”

Students had the tendency, when asked about the spectrum, to first answer with only the visible part of the spectrum; when prompted, they recalled that there is much more that the eye cannot perceive. “Do you remember what the longest wavelength part of the spectrum is?” I asked Hal. “The longest is red,” he answered. “It is at the far end, to the right.” Aaron, similarly, offered that “it [the E-M spectrum] goes from ultraviolet to infrared light, which are both extremes of light.”

Students commonly expressed confusion about photons, mixing them up with electrons and even molecules. Jay and Nathan thought that incoming electrons were absorbed by the atom to create absorption lines. Regina said, of a molecular cloud, “the cloud would absorb different elements, so it would show an absorption spectrum,” indicating as well an uncertainty about what photons are. Aaron, on the other hand, recalled the concept of photons: “Well, it’s funny, because light can be defined as both a wave – it’s hard to distinguish whether it’s a wave or a particle because there are individual particles called ‘photons’ and that’s how they act. Each of these photons follows a wavelike sequence.”

At the beginning of class, I led a demonstration where students represent the sun, a hydrogen atom, and the earth. The “sun” throws colored balls toward the earth that represent photons of different wavelengths; the “atom” in between the two catches the balls of only the correct wavelength to excite its electron. In the interview, two students recalled the demonstration, but both had a confused recollection of it. Richard once called the balls “molecules” (“like that example that we had where we had the people throwing, you know, they were throwing the molecules”) and once called them “atoms”

43

instead of photons. Aaron, too, makes reference to atoms being “transmitted through the substance,” also confusing atoms with photons.

Several students like to say “higher” and “lower” wavelength rather than “shorter” and “longer;” for example, Richard says, “light is much higher frequency and wavelength [than radio],” where “higher” here would have to mean “shorter” to be correct (frequency and wavelength have an inverse relationship). Students also used “longer frequency” instead of “lower frequency,” where perhaps “longer” refers to the period of the wave.

The students also refer to wavelengths as being on the “right” or “left” end of the spectrum, referring to the way I drew the electromagnetic spectrum on the board and the way their drawings of spectra were oriented. “The lower one [wavelength], or on the left, is the ultraviolet rays, and then to the right is the infrared,” said Kathryn, using both “lower” and “left” to describe UV rays but not referring to anything physical, like shorter wavelength, higher frequency, or higher energy. By referring to the spatial positioning of the wavelength axis as they encountered it during the lab, they reveal a tendency to draw upon visual memories when trying to remember the order of the ranges of the electromagnetic spectrum.

Jennifer expressed an interesting confusion about what the wavelength ranges (i.e. infrared, microwave, radio) represent: she thought that emission or absorption of a photon can only take place from one range to another. When asked if there could be emission in the radio part of the spectrum, she answered, “No, because radio wave’s the longest one, so we couldn’t really go any lower than that, right? I mean, that’s a tricky question, if you said like microwaves then I’d be like yeah, ’cause it’s still low but it’s not the lowest, ’cause from what I remember the atom has to go to a lower level, and I think radio waves are the lowest, right?”

The students saw, during the lab activity, that holding a piece of bleached paper up to the solar spectrum just beyond purple revealed an extra part of the spectrum, some of the near ultraviolet, that they had heretofore been unable to see. Calcium H and K lines were clearly visible. On the Pre2, I had asked if students thought they would be able to see spectral lines from the sun in the ultraviolet; 31% said yes. Because the students had just seen the lines in the UV, I felt it would be too “easy” if I asked them on the post-test if they could see such lines, so I asked if they thought we could see lines in the infrared. (Off in the deep red part of the spectrum, almost invisible, is an oxygen line that is close to the near infrared, so I’d hoped the students would extrapolate that there are lines off to the other side of the spectrum that we cannot see with the naked eye.) However, changing the question in this manner had unforeseen consequences. Students now responded based on their thoughts about the infrared: “No, because infrared is only seen in the dark,” one student wrote, referring perhaps to the lab question that asks why we can’t see other people “glowing in the dark.” Others who answered “no” made comments about heat, suggesting that the infrared, along with radio, is another part of the spectrum that has strong associations that make it “unique” from the rest of the spectrum. The question, therefore, did not provide a good pre/post lab measurement of students’ understanding of what goes on outside the visible light range, although it did provide some insight into students’ thoughts about infrared light. This was the only question that was modified between Pre2 and Post.

44

Theme 7: Blackbody radiation and temperatureTheme 7 was only listed as an objective by one professor, so it was not tested on

the matching Pre2/Post assessments, and therefore a quantitative gain could not be determined. However, considering that the lab manual devotes an activity, a calculation, and a few questions to the topic, I included it in the interview questioning to see what students understood about the topic. A question on Pre1 also asked about the temperature of red stars versus blue stars.

Although students did well on the simple multiple-choice question about Wien’s Law (Pre1, question 7), an almost universal confusion existed among the interviewed students when it came to the blackbody spectrum (Figure 8). I asked all of the interviewees to recall the solar spectrum, and the measurement they took with a photometer to find the intensity peak of the sun’s radiation. All were able to recall this experiment, and most remembered that the sun’s spectrum had its peak in the green part of the visible spectrum. When asked about the relative intensity of radiation from the sun in the form radio waves, many said (correctly) that it would be a lower intensity than the visible. However, when next asked about X-rays, some answered that there would be a greater intensity of X-ray photons than visible photons. As Aaron put it, “I think there’s a correlation. Low intensity for long wavelengths, high intensity for short wavelengths.” Jay justified his response by saying, “Well, if we could see UV I assume we would be able to see gamma rays.” Nathan summed up the sun’s emission by saying, “I guess it emits everything, but like you guys explained, you can’t see…we can’t see anything,”

Figure 8: Blackbody curves, which show the relative intensity of emitted light versus wavelength for objects of different temperatures. The peak of the blackbody curve occurs at shorter wavelengths for hotter objects, a relation described by Wien’s Law. Figure is from the UCB Astronomy 1010 lab manual.

indicating a belief that the sun is emitting all wavelengths of light more or less equally. Richard began with the same idea as Aaron and Jay above:

Amy: “If you could count the photons in the X-ray coming from the sun, do you think you’d see as many as there were in the visible? Or less? Or more?”Richard: “You’d probably see more…”

45

Then he changed his mind:

Richard: “X-ray is on the high end, like the extreme high end of the wavelength spectrum, so it has a shorter wavelength…So you probably would see less photons in the X-ray, because they’re condensed into a smaller area, like, they have a shorter wavelength....”Amy: “So you think you’d see more radio [than X-ray?]”Richard: “Yeah.”

Richard’s impression of waves packing themselves into space, and then intensity being related to the “size” of this package, is an interesting one. Richard also has a model, which he draws upon three times in the interview, whereby emission on “larger scales” is thermal (blackbody), as in the case of planets and stars, and emission on “smaller scales” is different somehow: it’s related to composition instead of temperature. The essence of this model is essentially correct, but the missing piece is that it is the interaction of atoms and molecules in a solid or a dense gas that leads to blackbody radiation. This level of depth, that blackbody radiation is a continuum because atoms in a small space affect the energy levels of the atoms around them, is only touched lightly with the following sentence in the lab manual: “A solid glowing object, such as the tungsten filament of an incandescent lamp, will not show a characteristic atomic spectrum, because the atoms are not free to act independently of one another.” The student’s model is a good, simplified picture of the situation. However, when asked to explain further what he meant by “changes as it goes to the smaller scale,” he tries to explain the difference between atomic and molecular emission lines, instead of the difference between a blackbody spectrum and an emission line spectrum from an element.

Regina was asked, “Can you tell me when we see an emission spectrum and when we see an absorption spectrum?” She correctly answered that an absorption spectrum might come from a cloud of gas between the sun and the observer, but when I asked if that cloud must also be emitting something, versus “absorbing infinitely,” she replied that she thought it was indeed only absorbing radiation and not emitting anything. During my lecture, I had taken it for granted that the students understood about energy balance, and flew through my explanation that an atom would re-radiate photons of the same energy it had just absorbed. Even if an object, a particle or a planet, is not reradiating the exact energy photons it is receiving – it converts some of the energy to heat, say – there is still a problem with a mental model of an infinitely absorbing cloud.

When it comes to temperature, however, many students knew (or learned) that blue stars are hotter than red stars. The question on Pre1 asked about blue stars and red stars; 72% of students correctly answered that a blue star is hotter than a red star. Of the interviewees, six (out of eight asked) also gave the correct relation. One did not know the answer at all. Two (Jay and Jennifer) answered initially that red stars would be hotter, but Jay corrected himself: “I’m assuming red’s hotter…actually, no, isn’t it blue’s hotter and red’s colder?” It seems to be a knee-jerk reaction based, perhaps, on a longstanding association of “red” with “hot,” as seen in the color-coding of bathroom water faucets (as Jay said, “’cause red’s a hotter color”).

46

5. DiscussionOne of the research questions this study seeks to address is why some objectives

were learned and others were not. While there is no definitive answer, the data suggest that students performed extremely well on some themes that were directly incorporated into a physical lab activity. Table 8 shows each objective, along with how the topic was presented to the students: by the TA’s lecture, the lab manual text, the lab activity, a lab question, or the professor’s lecture.) As Redish (1994) notes, the best way to help students build an appropriate mental model is by having them doing activities rather than watching.

For example, the students made large learning gains (29%) on questions that dealt with the ability of scientists to learn things about distant objects using spectroscopy, and this was a concept that the students themselves put into practice when observing the sun’s spectrum. They also made large gains (also 29%) on the concept that every element has a distinct spectral signature (which is also distinct from that of molecules and solids): this too was a concept they saw for themselves after looking at, and recording, the spectra of several elements, molecules, and glowing solids. Unfortunately, the blackbody curve, which was directly observed by the students in the form of measuring the peak of the solar spectrum, was not measured on the post assessment, so that concept cannot be used to corroborate this inference. However, post data in the form of interviews suggested that students still possessed many naïve ideas and misunderstandings about blackbody spectra and its relation to temperature after the lab.

With some exceptions, the smallest learning gains that were recorded in the post assessments were for themes that were not directly observed or performed during the lab (Table 8). These were atomic structure and the light verbs (which were only indirectly addressed by the lab activities) (2% and 3% gain, respectively). This suggests that, if the objective is of strong importance to the professor of the course, a change should be made to the lab activity to increase student learning gains on the objective.

The two topics that deviate from this trend are the high learning gains measured for “light and radio waves are the same thing” and “link to planetary spectroscopy” (26% and 22%, respectively), neither of which was covered in a lab activity.

Some of the faculty objectives, such as “atomic structure and energy levels,” do not lend themselves easily to hands-on, participatory experiences, while others, such as “planetary spectroscopy,” require different equipment than the department currently possesses. (Students made a 22% gain on the question dealing with our ability to study planets using spectroscopy, but compare this to the nearly double 43% gain on the question about using spectroscopy to study the distant stars: the lab emphasizes the ability to look at stars using spectroscopy, and many students on the post-test wrote that

Table 8Content Objectives TA LM LQ Lab Prof GainAtomic structure, energy levels, nature of atom, matter, and light a a a 1%Understand how a line is excited a a 1%Spectra prove that atoms are quantized a 2%

47

Each element has a distinctive signature – a “fingerprint” a a 24%Spectra of atoms vs. molecules vs. solids a a a 35%Spectroscopy is a tool for exploring different objects – can determine composition without going there a a a a 43%Look at spectrum in lab, make direct comparison to astronomical object a a 38%Link to planetary spectroscopy a a a 22%“Light verbs:” Emit, absorb, transmit, reflect/scatter a a a 3%Composition information can come from filters N/ALight is made up of different wavelengths, can be broken up into these wavelengths a a a a 9%Light and radio waves are the same thing a 26%Blackbody radiation a a a a a N/ATemperature/Wien’s Law a a a a N/A

Table 8: Method of instruction for each concept, along with measured learning gains. TA = TA’s lecture, LM = Lab manual text, LQ = Lab manual question, Lab = Lab activity, Prof = Professor’s lecture.

we cannot study planets using spectroscopy because they are not emitting light.) Recommendations on how to improve the labs, in light of the faculty objectives and data on student learning gains, are given in Section 6.

The discrepancy among students’ learning gains between the different concepts and themes may also be due more to the difficulty of the concepts themselves than to how they were presented in the laboratory. For example, “spectroscopy can be used to study distant objects” may be, to most, a simpler, easier concept than atomic energy levels. This was also a hypothesis posed by Zeilik and Bisard (2000), who studied a wider range of topics covered in an introductory course. These authors divided misconceptions into “structural” and “factual,” with the assumption that the former would be more resistant to change. Therefore, higher-intensity instruction was given on the structural topics, which the authors believed to be “imbedded in a well-developed, alternative cognitive structure.” What they found, however, was that no correlation existed between the two types of misconceptions as they identified them and the ultimate learning gain. Evidence for this also appears in this study. Consider the large difference (21%) between learning gains on the stellar and planetary spectroscopy questions, which both just asked if spectroscopy could be used to study these objects. This suggests that a sharp difference in learning exists between two concepts of equal difficulty if one of the questions dealt with a topic not directly experimented with during the lab activity.

The “harder” concepts can still be taught and learning gains can still be made on them – but it might be that more time and care is required in the teaching of them. Because of the limitations of this study, as outlined in Section 3.4 above, it is not fully

48

clear whether students failed to make learning gains on certain objectives because of the poor quality of their instruction on those topics by their particular TA (myself) or whether there is a deeper problem inherent in the traditional style of presenting the lab material. Nonetheless, I have attempted to argue that inferences can be made about the lab activity itself, separate from the effectiveness or ineffectiveness of the instruction given by the TA. The TA’s lecture is an introduction to concepts in a non-experiential manner, while the lab activity is supposed to help the students develop an understanding of these concepts by doing rather than hearing about.

6. RecommendationsI chose to study the spectroscopy lab in particular, out of all the activities in the

Astronomy 1010 manual and concepts taught in introductory astronomy, because I recalled my own experience learning about spectroscopy and thought it might be a confusing or difficult concept for others to learn as well. It is a very fundamental scientific procedure used in many different fields (chemistry, geology, physics, environmental science, and so on) to determine composition information. A spectroscopy lab may also be a good place to teach non-science majors some elementary theory of the nature of science and scientific inquiry.

The spectroscopy lab activity, as outlined in the lab manual, attempts to be a thorough introduction to spectroscopy and some related concepts. In many ways, it is a good lab, thoughtfully designed and comprehensive. In light of the faculty objectives, however, it falls short on the subjects where students failed to make any learning gains between the pre- and post-assessments. For each lab objective that did not exhibit learning gains, the teacher of the course should examine whether the objective should be scrapped, or how the lab can be modified or taught differently to improve student learning. Here are a few observations, supported by the qualitative and quantitative data collected in this study:

The lab is too long to complete in the allotted time period, especially since the TA needs to spend a longer-than-normal amount of time explaining background material. Students were frustrated by the fast pace and the lack of time to complete the activity. Five of the nine interviewed students, when asked about their experiences with the lab, criticized its length.

The spectroscopy lab, compared to other labs the students had done previously, felt to many of them like the hardest one conceptually (four out of the nine interviewed students expressed this feeling, but others may have agreed as well). Its difficulty, combined with its fast pace, may be responsible for the poor learning gain increases on certain concepts.

The students are entering the lab with certain preconceptions about light, and many of them are leaving with the same common ideas; the spectroscopy lab activity does not really touch on more basic concepts about light, atoms, color, and filters.

The content objectives that are not performed by the students as part of the lab activity appear to be the ones that they do not show large learning gains on.

In light of the point above about students’ prior knowledge, the text in the lab manual may be written at a level that is too high for the students. Evidence for this also includes the statements by students who commented that they felt

49

somewhat to completely lost on this lab. At any rate, the students did not seem to absorb much of the concepts they learned only from the lab text; they did much better on concepts they directly tested in experiments. This may not mean the text should be removed or changed: perhaps, instead, each objective that the professors really want the students to understand should be emphasized through other teaching methods.

The lab is more “cookbook” style than “inquiry-based” (see Section 2 above for an explanation of the distinction).

The spectroscopy lab is eight pages long (attached, Appendix A). The entire last two pages (questions IV.4 through IV.7) involve an activity that is difficult for the students and does not address any of the stated faculty objectives. The topic of this section is the analysis of spectral features, where the students are supposed to infer why a line appears a certain way (e.g. dark, broad). Here, the students are supposed to learn, from a conceptually confusing graph, that the absorption probability of photons by a given atom is related to the temperature of the star. Several paragraphs of text are given in the manual, but it left the students confused and unable to answer the questions without significant help.

The spectroscopy lab does not include any information or activities involving planetary spectroscopy.

The lab provides no information about non-astronomical applications of spectroscopy (e.g. environmental science uses, geologic uses), which might be of interest and may provide a greater sense of motivation for the students.

Four of the nine interviewed students complained about the difficulty of viewing the spectra through the handheld spectroscopes. However, this may be to a lack of patience and experimentation on the students’ part, since finding the correct angle to hold the spectroscope results in a clearly visible spectrum.

Based on the results presented here and input from the faculty, teaching assistants, learning assistants, and other interested parties, I would like to make the following recommendations.

I see a great benefit to splitting the lab to take up two full laboratory class periods. The first lab would be an introduction to light, and could include many of the activities performed by the students during a “mini-lab” they did the week after the spectroscopy lab: looking at the solar spectrum through filters, including the unusual hot pink and brown ones; looking at drawings through filters; observing light going through a prism; “re-assembling” a beam of white light using mirrors; mixing colors of light using the red, blue, and green spotlights; and going into the darkroom with a yellow sodium lamp to see what happens to colors. Because these activities are in some ways very basic, students should be asked to make predictions before each activity. For example, what will the hot pink filter do to the solar spectrum? What color will a red (yellow, white, blue) shirt appear in the room with the monochromatic sodium lamp? What color is made when red and green light are mixed? By asking the students to make predictions, and then afterwards asking if their predictions came true, they are forced to actively examine their own mental model of how light behaves.

The remainder of the class period after finishing these activities could be devoted to learning some of the basics of spectroscopy, including an introduction to the

50

electromagnetic spectrum and basic atomic structure. Here, the TA can take some time having the students perform the demonstration where differently colored balls represent photons of different wavelengths and students represent the sun, earth, a hydrogen molecule, and the rest of space. Covering these topics in the class before the spectroscopy lab would allow much more time for the students to understand and absorb the spectroscopy activities themselves. Slowing the lab down and spreading it over two sessions would also increase the amount of time for discussion, either class discussion or small-group discussions. A rushed ending to the lab period is not conducive to student learning.

Alternatively, the time remaining in the first session could be used for the students to build their own handheld spectroscopes. There are designs available for making these out of very inexpensive parts (e.g. Wakabayashi et al. 1998). By building their own, the students will gain a better appreciation of how the spectroscope works: otherwise, it is simply a black box (one of the students, Nathan, even referred to it in his interview as “that black thing”). Furthermore, the students can be asked to take their spectroscopes home and use them to look at car lights, street lights, neon lights, and so on, as part of their lab assignment. The students can then make the connection between spectroscopy and things they see in everyday life. This also provides the students the opportunity to inject some creativity into the lab instead of following step-by-step instructions; students can try turning their spectroscopes on whatever light source strikes their fancy, and then will draw their results.

The second session could then consist of many of the same activities already present in the lab manual: looking at the glowing tubes of gas and drawing their spectra; identifying a “mystery gas;” and observing the solar spectrum through the spectacular heliostat spectroscope. If the students have done the activity above (observing spectra in the outside world with their handheld scopes), they can then use what they have learned from looking at elements in the lab to try and explain the spectra they drew at home.

I would recommend removing the entire last two pages of the lab (see Appendix A), which cover a difficult topic (the reason for the appearances of the solar absorption lines) that the faculty do not expect their students to learn (it does not touch a single professor’s objectives for the spectroscopy lab). In its place, an activity to emphasize planetary spectroscopy could be performed (more on this topic below), or the additional time can be used to conduct discussions.

A well-executed discussion, whether in small groups or as a class, can help the students step back from the minutiae of the lab activity and bring them around to the big picture: the motivation behind the lab, the content and nature of science goals for the lab; and it also provides a time for feedback to the TA of the students’ learning improvements and experience with the lab. A study of students learning atomic structure and the nature of scientific experimentation in an introductory chemistry class (Niaz et al. 2002) showed that sections that participated in small-group discussions (which included selecting an answer to a supplied discussion topic, supplying reasons to justify the choice, evaluating each other’s reasoning, offering counterarguments, and defending in writing their final response) improved their understanding of scientific inquiry relative to a control group (which was presented the material as traditionally found in lectures and the textbook).

Similarly, if we hope to improve our students’ understanding of both content and nature of science topics, discussions can be an invaluable teaching tool. UCB’s STEM

51

(Science, Technology, Engineering, and Mathematics) program, designed to prepare undergraduates in science majors who are interested in K-12 teaching in these disciplines, provides teachers of undergraduate courses (such as Astronomy 1010) a useful resource in the form of “Learning Assistants,” undergraduates who bring to the classroom a background in education theory and can help design and guide student discussions.

A lab on planetary spectroscopy could potentially occupy its own lab class period, especially for professors who are more interested in emphasizing the planetary applications. Reflectance spectrometers for classroom use are on the market (e.g. the ALTA Reflectance Spectrometer,3 $160). Better connections between the spectroscopy lab and other planetary labs in the Astronomy 1010 canon (Planetary Albedos and Temperatures, The Greenhouse Effect) could be established.

A discussion of the idea of looking for life on distant planets using spectroscopy may be interesting to the students, and would inform them of cutting-edge research involving extrasolar planets. A possible lab activity could have the students discussing in small groups what spectral features might be present on a planet that harbored life, and then they could design a mission to look for those features. Because the students’ knowledge of astronomical observation techniques will probably not be very sophisticated, it would also be a good opportunity to teach them some basics, e.g. ground-based versus space-based telescopes (which ties back into spectroscopy with the concept of atmospheric wavelength windows). A group activity such as this may be helpful for drawing in students whose strengths are creativity and imagination rather than analysis, and for the students of high classroom performance who are easily bored with the “cookbook” style of labs.

The spectroscopy lab could also be moved away from the “cookbook” style of having each step explained explicitly, and students could be required to formulate hypotheses and design simple experiments within the context of the lab activities. For example, Adams and Slater (2000) have developed an inquiry-based lab called “Stellar Bar Codes” that has the students working collaboratively to understand the idea of stellar classification through spectra, and the relation to temperature (however, the former concept – spectral types – was not suggested by any of the six interviewed faculty as an objective for the spectroscopy lab). Within the context of the existing Astronomy 1010 spectroscopy lab, the activity of testing a “mystery gas” could be reformulated so that students design the experiment. Similarly, the activity where students identify spectral lines in the sun (currently, the students simply copy the information from a chart of the Fraunhofer spectrum) could be re-designed to have the students decide what information they need to determine the source of the lines in the sun.

Because there are several themes that faculty identified as objectives that cannot be directly observed by the students in a lab setting (e.g. atomic structure), instructional computer applets could be used to allow students to visually observe and experiment with what cannot be seen with the eye. The effectiveness of applet-based learning has been analyzed elsewhere (e.g. Finkelstein et al. 2004), and an understanding of its usefulness in the teaching of physics and astronomy concepts is of great educational importance. If an objective is of strong importance to a teacher of introductory astronomy that cannot be

3 https://www.lpi.usra.edu/store/products.cfm?prod=25&cat=5

52

directly observed in a laboratory, a thorough consideration of computer-based labs is worthwhile.4

Several professors made recommendations for improved or different equipment, including: a 10 micron video camera, which would allow students to see each other in the infrared; reflectance spectrometers to study surfaces; better handheld spectroscopes, to avoid the common frustration that students expressed at being unable to see the spectrum; and a commercial neon light (“Bar” or “Open,” for example) for students to observe instead of the more academic neon discharge tube.

The participatory demonstration involving colored balls and a painted stepladder, which represent photons of different wavelengths and a hydrogen atom, is currently used by the TAs to explain absorption and emission, but it is not written into the lab manual and should perhaps be standardized as a part of the lab to ensure its inclusion in the TA’s lecture. Although the demonstration was only partially correctly remembered by students in the interviews, the fact that these students recalled the demonstration without prompting suggests that it is a memorable activity and useful for developing the students’ mental models of absorption and emission. With more time to fully explore the demonstration, which could be opened up by shortening the lab or splitting it into two sessions, it could be a very effective learning tool.

It is important to note that ongoing assessment is necessary for the development of an optimal, research-based spectroscopy laboratory activity for students in introductory astronomy. This study examined the traditional method of teaching the Astronomy 1010 spectroscopy lab; future work should study modified versions of the lab to assess whether increasing numbers of students are meeting faculty learning objectives. An iterative approach to developing curricula (teach, assess, amend, repeat) will benefit the students greatly, as it looks to them for information about how well material is being learned, and it leads to data-driven improvements in activities, rather than changes based solely on teacher intuition or educational theory.

References

Adams, J., and T. Slater, 2000, “Stellar Bar Codes,” The Physics Teacher, 38, 35.\

Adams, J., and T. Slater, 2002, “Learning through sharing: Supplementing the astronomy lecture with collaborative-learning group activities,” Journal of College Science Teaching, 31(6), 384.

Bell, R.L., and N.G. Lederman, 2003, “Understandings of the Nature of Science and Decision Making on Science and Technology Based Issues,” Science Education, 87(3), 352.

Brickhouse, N.W., Dagher, Z.R., Shipman, H.L., and W.J. Letts IV, 2002, “Evidence and warrants for belief in a college astronomy course,” Science & Education, 11, 573.

4 Currently, several of the UCB Astronomy 1010 labs are computer-based, as they cover topics that would otherwise be difficult to simulate in a laboratory setting (Kepler’s Laws, planetary temperature, and the greenhouse effect, for example).

53

Diakidoy, I.N., and P. Kendeou, 2001, “Facilitating conceptual change in astronomy: a comparison of the effectiveness of two instructional approaches,” Learning and Instruction, 11, 1.

Finkelstein, N.D., K.K. Perkins, W. Adams, P. Kohl, and N. Podolefsky, 2004, “Can Computer Simulations Replace Real Equipment in Undergraduate Laboratories?”, PERC Proceedings 2004, AIP Publishing: MD (to appear, anticipated 2005).

Fraknoi, A. 2001, “Enrollments in Astronomy 101 Courses,” Astronomy Education Review, 1(1), 121.

Handelsman, J., Ebert-May, D., Beichner, R., Bruns, P., Chang, A., DeHaan, R., Gentile, J., Lauffer, S., Stewart, J., Tilghman, S.M., and W.B. Wood, 2004, “Scientific Teaching,” Science 304, 521.

Lawson, A.E., and J.W. Renner, 1975, “Piagetian Theory and Biology Teaching,” The American Biology Teacher, 37(6), 336.

Lederman, N.G., 1999, “Teachers’ understandings of the nature of science and classroom practice: Factors that facilitate or impede the relationship.” Journal of Research in Science Teaching, 29, 331.

Lippert, N., and B. Partridge, 2004. “To hear ourselves as other hear us,” Astronomy Education Review, 3(1), 31.

McDermott, L.C., 1991, “Millikan Lecture 1990: What we teach and what is learned – Closing the gap,” Am. J. Phys, 59(4), 301.

Mendenhall, W., and R.J. Beaver, 1994, Introduction to Probability and Statistics, Belmot, CA: Duxbury Press.

Moore, R.W., and R.L.H. Foy, 1997, “Science attitude inventory: A revision (SAI II),” Journal of Research in Science Teaching, 34, 327.

Niaz, M., Aguilera, D., Maza, A., and G. Liendo, 2002, “Arguments, Contradictions, Resistances, and Conceptual Change in Students’ Understanding of Atomic Structure,” Science Education, 86(4), 505.

Partridge, B., and G. Greenstein, 2003, “Goals for ‘Astro 101’: Report on Workshops for Department Leaders,” Astronomy Education Review, 2(2).

Redish, E.F. 1994, “Implications of cognitive studies for teaching physics,” Am. J. Phys, 62(9), 796.

Rutherford, F. J., and A. Ahlgren, 1994, Science for all Americans, American Association for the Advancement of Science, New York: Oxford University Press.

54

Wakabayahsi, F., Hamada, K., and K. Sone, 1998. “CD-ROM spectroscope: A simple and inexpensive tool for classroom demonstrations on Chemical Spectroscopy,” Journal of Chemical Education, 75(12), 1569.

Zeilik, M., and W. Bisard, 2000, “Conceptual change in introductory-level astronomy courses,” Journal of College Science Teaching, 29(4), 229.

Zeilik, M, and V.J. Morris-Dueer, 2005, “What are essential concepts in ‘Astronomy 101?’ A new approach to find consensus from two different samples of instructors,” Astronomy Education Review, 3(2), 61.

55

Documents

Master’s Thesis – First Draftdduncan/teachingseminar/Jor… · Web view1. Introduction. Astronomy 1010 is an introductory astronomy class designed for non-science majors at