ThinkSpace Advisory Board Meeting with Reasearch Team. September 16, 2016
NSF has awarded WWTA funding to support a three-year research project: NSF DRK-12 for Collaborative Research titled "Thinking Spatially about the Universe- A Physical and Virtual Laboratory for Middle School Science". (DRL-1503395)
Critical breakthroughs in science (e.g., Einstein's Theory of General Relativity, and Watson & Crick's discovery of the structure of DNA), originated with those scientists' ability to think spatially, and research has shown that spatial ability correlates strongly with likelihood of entering a career in STEM. This project will develop and study three week-long middle school lab units designed to teach spatial abilities using a blend of physical and virtual (computer-based) models. "ThinkSpace" labs will help students explore 3-dimensional astronomical phenomena (moon phases and eclipses; planetary systems around stars other than the Sun; and celestial motions within the broader universe) in ways that will support both understanding of these topics and a more general spatial ability. Students will learn both through direct work with the lab unit interface and through succeeding discussions with their peers. The research program will determine which elements in the labs best promote both spatial skills and understanding of core ideas in astronomy; and how then to optimize interactive dynamic visualizations toward these ends. Virtual models of the sky and universe will be created using WorldWide Telescope, a free visualization tool that runs on desktop computers, tablets, and mobile devices. The ThinkSpace lab materials will be available at no cost on popular curriculum-sharing sites, including PBS Learning Media and BetterLesson.
The ThinkSpace team will address two main research questions: 1) How can spatial tasks that blend physical and virtual models be embedded into a STEM curriculum in ways that lead to significant improvements in spatial thinking? and 2) How can practitioners optimize design of interactive, dynamic visualizations for teaching spatially complex concepts? The first year of the study will examine two of the lab units with four teachers and about 320 students. The second year of the study will be similar. The third year of the study will test all three lab units in 10 classrooms. Over this study, each week-long ThinkSpace Lab will be formatively tested, using pre/post written assessments of astronomy content and spatial thinking; pre/post interviews with students; and in-class video of students using the lab activities. Scaffolded learning designs will support students in making connections between different spatial views of the phenomena, and will guide them to construct explanations and argue from evidence about how various phenomena (e.g. moon phases) arise in the real Universe, as Next Generation Science Standards demand. The impact of the ThinkSpace labs will be felt far beyond astronomy because the learning models being tested can transfer to other fields where spatial models are critical, and findings on optimization of dynamic visualizations can help to inform instructional design in the age of online learning. The Discovery Research K-12 program (DRK-12) seeks to significantly enhance the learning and teaching of science, technology, engineering and mathematics (STEM) by preK-12 students and teachers, through research and development of innovative resources, models and tools (RMTs). Projects in the DRK-12 program build on fundamental research in STEM education and prior research and development efforts that provide theoretical and empirical justification for proposed projects.
Research Activity School Year 2015-16
Development of ThinkSpace Moon Phases Lab: In the fall of 2015, the team adapted an older WWT-based Moon phases lab (that was developed as part of an NSF EAGER grant. IIS-1254535), to include explicit instruction around using spatial thinking to investigate and explore the phenomenon. We focused on “perspective taking,” which requires students to consider how a scene would look from a variety of viewpoints. During instruction with the physical model and the WWT visualization, students had several chances to practice imagining how the Moon would look, both from the perspective of a viewer on Earth, and from the perspective of a viewer above the Sun/Earth/Moon system, and how to mentally connect those different perspectives.
Development of Assessment Instruments: In fall 2015, the team developed a pre/post assessment instrument for the Moon Phases ThinkSpace lab that measures student understanding of the cause of Moon Phases and Eclipses, and we chose a spatial thinking assessment developed by Advisory Board member Lynn Liben that measures students’ “perspective taking” ability.
The Moon phases content questions were drawn from the MOSART (Misconceptions-Oriented Standards-Based Assessment Resources for Teachers) database (Sadler et al 2009). The team identified 20 MOSART questions relevant to the Moon lab. We then administered the questions to 260 middle school students who had recently completed an existing curriculum on Moon phases with their teachers (not the ThinkSpace lab). We used the data to calculate discrimination and difficulty statistics for each question, which then informed the final selection of items we used on our pre/post instrument.
The assessment also included a question designed to measure students’ Knowledge Integration (KI, Linn 2000) about Moon phases that we had used in the previous EAGER study.
The Penn State team developed an interview assessment protocol to gain a more in depth understanding of how students use models to explain Moon phases, and how they use (or don’t use) perspective taking in their thinking process. They pilot tested the interview protocol with local middle school students in Pennsylvania and videotaped the interviews to train the Boston-based team in how to conduct the interviews.
Advisory Board Meeting: During a daylong meeting held at the Harvard-Smithsonian Center for Astrophysics in Cambridge in October 2015, advisors were briefed on the goals of this project and data from the team’s pilot project was shared. The research team and advisors then discussed the research plan including the development of survey instruments and advisors suggested potential modifications.
Moon Lab Field Testing and Data Collection: During the 2015-2016 school year, we recruited 3 teachers in 2 different school districts to pilot test the ThinkSpace Moon Phases lab with their students. The team worked with the Office of Data and Accountability to secure appropriate permissions to gather research data in the Boston Public Schools. Almost 300 middle school students used the lab this year. All participating students took the pre/post assessment on Moon phases and on Perspective Taking. A total of 32 students were interviewed using Penn State’s assessment protocol before and after using the lab. Roughly equal numbers of students with High/Middle/Low perspective taking pre-test scores were chosen for interviews, and equal numbers of boys and girls were chosen. The interviews were videotaped, and those students were videotaped in class participating in the lab activities.
At the October 2015 Advisory Board meeting, the team was advised to gather additional data around the question of whether it is more beneficial for students to use the physical model first, then the WWT visualization, or vice versa. Half the participating classes were given each model order (WWT -> Foam vs. Foam -> WWT).
Coding and Analysis: The team is in the process of analyzing data collected during 2015-2016. The Penn State team is developing a scheme to code the interview videos and in class video, and the Boston-based team is analyzing results from the written assessments, including both the MOSART multiple choice questions, the perspective taking questions, and the open-response Knowledge Integration question.
Overall: For all cohorts that completed the ThinkSpace Moon lab in 2015-2016, students show significant gains for both Moon phases content and perspective taking.
- For 11 MOSART Moon content MC questions, Cohen’s d effect size = 1.48±0.07.
- For 15 Perspective Taking (PT) questions, Cohen’s d effect size = 0.30±0.08 (even with 38% of students scoring ≥90% on the pre-assessment).
A subset of 6 MOSART questions were given after students used a single model (foam vs. WWT) as a mid-assessment to learn whether each of the two different models help students understand particular topics better than the other.
- There were no statistically significant difference in overall gains between WWT vs foam model on mid-assessment (after half of students used only WWT and other half used only foam model).
- Students who used the Foam model first had significantly higher pre-mid assessment gains on a question regarding the Earth-Moon scale than students who used WWT first, indicating that a physical model is important for developing sense of scale.
- Students who used WWT first had significantly higher pre-mid assessment gains on a question that asks them to predict what the Moon would look like when in a particular position relative to the Sun and Earth, perhaps suggesting that WWT helped students better connect the space-based and Earth-based views.
A video version of WWTA's Phases of the Moon Unit can be viewed here.
|Introduction to the Moon Phases Tour||19.09 MB|