Teaching and learning astronomy in the 21st century

You’re lecturing to your introductory college astronomy class about Newton’s law of gravitation. You’ve carefully explained that the gravitational force depends on the product of the two masses involved and on the inverse square of the distance between them. You’ve shown a few examples or perhaps videos and animations to help your students connect the abstraction of an equation to the real physical world. You may assign thoughtful homework problems, and you encourage the students to ask questions if they don’t understand, either in class or during your office hours. You’re known as a good lecturer, and your students always rate you highly at the end of the term. Yet when you give your exam, you’re dismayed to see how many of them can’t answer straightforward questions of the type you covered in class and assigned as homework. So why does the same thing happen to instructors all over the country?

Astronomy-education researchers have been working to solve that problem and many others facing instructors of astronomy survey courses for nonscience majors. Such courses are commonly called Astro 101. During a series of investigations conducted at the University of Arizona, education researchers have developed conceptual questions used to assess students’ understanding of core topics in such courses. Two of the questions are “At what location between the Earth and Moon does the net gravitational force on a spaceship become zero as it travels between the two bodies?” and “Would a waxing gibbous Moon ever be above the horizon during daytime?”

After traditional lecture-based instruction, one student (Jennifer) stated in response to the gravity question, “halfway, because exactly halfway causes the Moon’s and Earth’s gravitational pulls to cancel out.” In response to the lunar-phase question, another student (George) answered, “No, because this phase only occurs when the Sun illuminates it during our nighttime.” Those responses indicate that after instruction Jennifer and George still had conceptual and reasoning difficulties common among their peers prior to instruction. ¹

By the second time Jennifer and George answered those questions, they had both participated in an interactive learning activity designed to help Astro 101 students confront common misconceptions. After completing the activity on gravity, Jennifer correctly answered, “Closer to the Moon than to Earth, because Earth has a greater force on the spaceship than the Moon does. But when the spaceship is closer to the Moon, Earth loses some force while the Moon gains some, until their strengths become equal.” And George was now able to correctly reason that “this phase is highest in the sky at 9 PM, therefore rising 6 hours earlier at 3 PM and setting at 3 AM. So yes, it would be visible for some short time between 3 PM and 6 PM in the daytime.”

Improving scientific literacy

Research on the teaching and learning of Astro 101 has an important role to play in improving our nation’s understanding of the scientific process and of the role science plays in society. ² Last year, NSF reported that according to its Science and Engineering Indicators, only about 25% of the country’s adults were scientifically literate. Astro 101 courses reach nearly 250 000 college students each year. An astonishing 10% of all US college students take a survey astronomy course, which makes Astro 101 one of the most popular introductory science courses. ³ For many of those students, Astro 101 will be the final science course they take for the rest of their lives. The quality of their astronomy education may therefore have a lasting impact on their scientific literacy and their attitudes toward science.

The overwhelming majority of students taking Astro 101 are nonscience majors. They represent our society’s future business leaders, lawyers, journalists, politicians, historians, and—most critically—schoolteachers. As many as 40% of students taking introductory science courses say that they intend to become licensed teachers. ⁴ Schoolteachers play a critical role in inspiring and training the next generation of students to join the STEM disciplines: science, technology, engineering, and mathematics. Improving the scientific knowledge, attitude toward science, and teaching skills of prospective teachers must be critical goals for Astro 101 courses. Unfortunately, middle- and high-school teachers often emerge from college unprepared to teach their students about astronomy and space science. With so much at stake, it is clearly in our nation’s best interests to improve the teaching and learning of Astro 101.

Over the past 10 years, astronomy-education researchers have made significant gains in their understanding of how students learn the subject. Much of that work has intentionally followed the successful path blazed over the previous two decades by physics-education researchers. Physics-education research (PER) has shown that interactive learning strategies significantly improve student understanding. Astronomy-education research (AER) has begun to show that carefully adapted versions of those research-validated learning strategies can achieve large gains in the Astro 101 classroom. To determine the effectiveness that new and innovative teaching strategies are having on Astro 101 students, we have conducted a national study involving nearly 4000 students at 31 colleges and universities. Before discussing the key results of our study, we share some highlights from PER that have influenced our work.

Physics education leads the way …

Over the past several decades, a number of highly effective research and curriculum-development models have emerged from the PER community. ⁵ (See also the PHYSICS TODAY articles by Edward Redish and Richard Steinberg, January 1999, page 24 ), and by Carl Wieman and Katherine Perkins, November 2005, page 36 ). Physics-education researchers have made much progress toward determining what naive misconceptions and reasoning difficulties students have in introductory physics. The results of that research have been used to develop curricula that specifically target those difficulties. The most successful instructional strategies have focused on getting students to become actively engaged in their own learning, as opposed to passively listening to lectures.

A necessary step in the progress of PER was the creation of research-validated assessment instruments that let instructors measure the effectiveness of their instruction. Among the first such assessment instruments was the widely adopted Force Concept Inventory. ⁶ The FCI is a collection of 30 multiple-choice questions on the basic concepts of Newton’s laws. They are designed to force students to choose between Newtonian concepts and “common-sense” alternatives. The FCI was widely adopted in the physics community because it focused on a topic central to all first-term introductory courses, and also because its simple design enabled instructors to easily measure how much students gained in their understanding.

That wide use allowed Richard Hake in 1998 to report a meta-study of FCI results from 6000 students enrolled in classrooms all over the country. ⁷ As a measure of student learning in a particular course, Hake calculated the normalized learning gain

$$g=\left(〈post\%〉-〈pre\%〉\right)/\left(100-〈pre\%〉\right),$$

where 〈pre%〉 and 〈post%〉 are class-averaged scores in answering the FCI questions before and after instruction. The normalization denominator minimizes the dependence of g on the different levels of student understanding at the time of the pre-course test.

As shown in figure 1, Hake’s study yielded strong evidence that students in classes that used interactive learning strategies outperform those in traditional lecture-based classrooms. Through many such validation studies, the PER community provided evidence that helped generate acceptance of new instructional modes in the physics community.

… and astronomy follows

The highly successful PER work offered a well-marked path for the AER community to follow in developing effective instructional strategies and assessment instruments for the Astro 101 classroom. ⁸ Our long-term goal was to perform the necessary research that would culminate in a national study, similar to Hake’s, on the effectiveness of teaching Astro 101. The list of research and development tasks below provides an outline of the essential steps undertaken along the path to this national study:

Although the results from PER were very helpful to the astronomy-education researchers, there are fundamental differences between the two fields. First, the courses and student populations studied in PER and AER are very different. Introductory college physics is aimed at science and engineering majors, while Astro 101 is designed primarily for non-science majors. Second, on a practical level, the development of curricular materials for Astro 101 is constrained by the lack of recitation sessions, labs, and teaching assistants. The lecture portion of the Astro 101 class is commonly the only time instructors meet with their students.

So instructional strategies must resolve conceptual and reasoning difficulties without significant help from the instructor, and they must be designed for use in large lecture halls with fixed seats. Furthermore, because new strategies require instructors to give up precious class time normally spent lecturing, teaching innovations must be relatively brief.

Within the Astro 101 teaching community, three active-engagement strategies have been widely adopted and have been shown to improve students’ understanding: Think-Pair-Share (called peer instruction in the PER community), Lecture-Tutorials, and Ranking Tasks (see the box on page 43). ^{1 , 9 , 10} Those strategies are designed to have small groups or pairs of students work together during the standard classroom period, typically following a short 10- to 20-minute lecture.

Each strategy represents a different type of interactive learning activity. They target known conceptual difficulties and promote active intellectual engagement. By discussing challenging questions, students get to explore the reasoning behind their answers. In doing so, they improve their reasoning skills and their understanding of core topics. Systematic studies have shown that the strategies can improve students’ understanding by two full letter grades beyond what traditional lectures accomplish. ^{1 , 10}

The existence of such research results has encouraged many Astro 101 instructors to start implementing those interactive strategies in their classrooms. To further increase instructor awareness of those strategies and to help ensure their proper implementation, researchers at the University of Arizona’s Center for Astronomy Education (CAE) have developed a teaching-excellence workshop series with funding from NASA and NSF. Over the past five years, the workshops have been offered at colleges and universities and at national meetings of organizations such as the American Astronomical Society and the American Association of Physics Teachers. The workshops have been attended by more than 1000 college instructors from all types of institutions.

To conduct its own national study, the AER community needed a reliable assessment instrument like the physicists’ FCI, but one that covered a central topic of the Astro 101 curriculum. To that end, Erin Bardar and coworkers created the Light and Spectroscopy Concept Inventory (LSCI)—a collection of 26 multiple-choice questions. ¹¹ Light is the central carrier of information in astronomy, and a survey of Astro 101 instructors has shown that they consider the nature of light and the electromagnetic spectrum to be the most important topics in their courses. ¹² The topic domains of the LSCI are

The LSCI questions, like those of the FCI, were phrased, as much as possible, in ordinary language, and they include “distracters” drawn from common misconceptions. The questions are not easy, and they often require multiple reasoning steps to answer correctly. Three such questions are shown in figure 2. The entire LSCI can be seen at http://aer.noao.edu/auth/LSCIspring2006.pdf .

But does it work?

To bring about a shift in how Astro 101 is taught similar to that motivated by Hake’s study of introductory-physics teaching, we conducted a national study in collaboration with instructors who agreed to use the LSCI, to determine the effectiveness of various teaching strategies in Astro 101 classes. ¹³ Our 4000-student study covered 69 class sections at 31 colleges that represented all types of institutions, with a wide range of class sizes and instructional styles—traditional and interactive.

Figure 3 shows a plot of learning gain g, measured with the LSCI astronomy questions, versus class-averaged pre-instruction score for Astro 101 courses in our study, sorted by type of institution. For comparison, the shaded region indicates the range of results for Hake’s introductory-physics study. The boundaries at g = 0.3 and 0.7 separate the ranges of learning gain that Hake characterized as low, medium, and high.

One striking result is evident. The range of LSCI pre-instruction scores is surprisingly narrow, clustered around 25%, regardless of institution type. That’s very different from Hake’s study, in which pre-instruction scores ranged from 30% to 70%. That discrepancy illustrates a fundamental difference between the student population taking Astro 101 and that taking introductory college-level physics. Many physics students come to introductory college physics having already taken physics in high school. But Astro 101 students are mostly nonscience majors with little prior knowledge of the basic concepts of light and spectroscopy.

The class learning-gain scores in figure 3 vary widely, from almost 0 to 0.5, illustrating that the LSCI is capable of measuring changes in student understanding and, by extension, the effectiveness of teaching about light and spectroscopy in Astro 101. Because the gains appear to be independent of institution type—and also, as we find, of class size ¹³ —we conclude that neither of those two variables can explain the variation in g. This result suggests that type and effectiveness of instruction are crucial variables. Characterizing introductory-physics classes by whether instructors used any of a variety of interactive learning strategies, Hake had demonstrated that—as measured by g—the interactive classes outperformed the traditional lecture-only classrooms, on average, by about a factor of two.

In our study, we knew that a significant fraction of the instructors were using interactive learning strategies because they were members of the greater national CAE community. We developed a questionnaire for instructors that let us quantify the amount of interactive instruction occurring in each classroom. From each instructor’s responses, we calculated a nominal percentage of time, called the Interactive Assessment Score (IAS), spent on interactive learning strategies during the term. The scores ranged from 0 to 49%, suggesting that our questionnaire was successful at distinguishing different amounts of interactive instruction, and that instructors were not inflating estimates of their classes’ interactivity. If they had been, we would surely have seen many estimates of over 49% and none near 0%. Nonetheless, the IAS is only a first-order indicator of allotted time. It provides no details as to the quality of the implementation or engagement in the classroom.

In figure 4, we plot g versus IAS for the 52 Astro 101 classes in our study with at least 25 students. We excluded smaller classes because we believe that the teaching and learning in classes with a very small number of students can be a special case, bordering on personalized instruction. Although the plot shows no simple relationship between learning gain and the level of interactivity, it is notable that no class with an IAS below 25% achieved a gain above 0.30.

By contrast, classes with an IAS above 25% had gains ranging from about 0.05 to 0.5. The average learning gain for those classes was 0.29, more than twice the average gain of 0.13 found for classes with an IAS below 25%. This result is almost identical with that found by Hake for introductory physics. To determine if this dependence on IAS is real, we conducted a statistical-significance test (a t-test) and concluded that there is less than a 10^-5 chance that the recorded difference in learning gain between the two groups is just a statistical fluke. If this were a medical study of two treatment strategies for a disease, the study would be stopped at this point for ethical reasons, so that every patient could be given the more effective treatment immediately!

To further probe the relationship between interactivity and learning gain, we conducted a multivariate regression analysis to determine how individual differences (for example, personal and family characteristics, academic achievement, and student major) might be correlated with learning gain. ¹⁴ The results show that the use of interactive learning strategies is the single most important variable in accounting for the variation in student learning gains, even after controlling for individual characteristics. Furthermore, we find that the positive effects of the interactive strategies are equal for strong and weak students, men and women, regardless of ethnicity or primary language. Our results strongly suggest that all students benefit from interactive learning strategies—because those strategies are based on how humans learn. ¹⁵

Although our data suggest that spending at least 25% of class time on interactive learning strategies can have a large impact on learning, the broad spread in g for the higher-interactivity classes suggests that the use of such strategies is not enough. The combination of IAS and individual student characteristics used in our multivariate analysis accounts for only 25% of the spread. ¹⁴ So, what could account for the rest? The answer may come from research findings on the professional development of instructors which suggest that the quality of implementation of instructional strategies has a significant influence on student learning. ¹⁶

Implications

Because of its great popularity among nonscience students and its importance for future schoolteachers, Astro 101 has a central role to play in improving the scientific literacy of our nation. Inspired by the success of PER, the AER community has conducted valuable research on teaching and learning in astronomy. Our work demonstrates that every Astro 101 instructor, regardless of the type of institution or class size, can see benefits in student learning by implementing interactive learning strategies.

Yet a clear message from our research is that the mere use of such learning strategies is not enough. The quality of implementation is crucial, which points to the importance of professional development. College-level instructors typically receive no significant pedagogical training prior to teaching for the first time. Furthermore, many Astro 101 instructors have had no formal training in astronomy. Of the 250 000 students who take Astro 101 each year, 40% do so in a pure physics department, and another 40% take the course in departments that don’t offer degrees in either physics or astronomy. ³ Thus a large number of students taking Astro 101 are taking it from an instructor who has little or no formal training in astronomy.

To address such challenges, CAE has created professional-development workshops designed for Astro 101 instructors with all levels of prior preparation. Those teaching-excellence workshops focus specifically on developing instructors’ pedagogical knowledge. They are based on professional-development best practices, ¹⁶ and they use research-validated techniques that require instructors to practice teaching strategies in a peer-review environment, in which participants offer suggestions and critiques of one another’s implementation of interactive learning strategies. ¹⁷ Beyond those workshop experiences, CAE also provides online professionaldevelopment resources through our website at http://astronomy101.jpl.nasa.gov .

Lack of training is not the only barrier to the effective use of interactive learning strategies. Fundamentally changing how we instructors teach requires work on our part. In addition, there is little requirement that we document our students’ learning gains as part of our hiring, promotion, and tenure procedures. Given the amount of work and lack of reward, there can be a natural resistance to change. Thus it is critical for deans, department chairs, other senior faculty, and national organizations to encourage instructors to make the effort to change the way they teach—and reward them for doing so. For example, resources and opportunities should be provided that allow instructors, teaching assistants, and even postdocs to engage in professional-development workshops and that encourage them to implement proven interactive learning strategies in their classrooms.

The ideas of PER and AER are steadily gaining acceptance in physics and astronomy departments nationwide. It is particularly encouraging that many of those who are embracing interactive instructional strategies are early in their careers, which bodes well for the future of Astro 101 instruction. The central role of AER in improving the teaching and learning of astronomy was strengthened by the founding (in 2002) and subsequent growth of the online journal Astronomy Education Review (http://aer.aip.org ), published by the American Astronomical Society.

In addition, members of CAE recently received an NSF grant to create the Collaboration of Astronomy Teaching Scholars. CATS is a large and growing international community working to increase the number of Astro 101 instructors conducting research in astronomy education. The collaboration also aims to spur the development of research-validated curricula and assessment instruments. With a willingness to challenge ourselves—astronomers and physicists alike—to teach Astro 101 using proven instructional strategies, we can improve the way we teach this critical course and thereby improve the scientific literacy of some 250 000 Americans each year.