By: Stephanie Pfirman, Julie Ann Wrigley Global Institute of Sustainability and Innovation, Arizona State University; Lawrence Hamilton, Carsey School of Public Policy, University of New Hampshire; Margie Turrin, Lamont-Doherty Earth Observatory, Columbia University; Craig Narveson, Director of Strategic Partnerships at Kahoot!, Oslo, Norway; Carrie A. Lloyd, Arizona State University

The question, "What do US Students Know about the Polar Regions?" is an important one for us to figure out so that we can develop educational priorities to prepare the future workforce. Newly released findings that the Arctic near-surface air temperature is warming at rates of three times that of the global average (AMAP, 2021), further elevate the importance of educating our next generation of decision makers. While we can survey adults, it is harder to develop research protocols to gather a nationwide sample of students. With support from the National Science Foundation and Arizona State University, we developed the novel approach of analyzing student responses to online quizzes posted on the popular Kahoot! platform (Pfirman et al., 2021). Partners in the project include Barnard College, Lamont-Doherty Earth Observatory of Columbia University, the University of New Hampshire, Arizona State University, and Kahoot!.

Through the Polar Learning and Responding Climate Change Education Partnership, we had created a polar quiz as part of a series of "Climate Challenges" for education and engagement. While the questions were not ideally designed for evaluation, the quiz attracted a large number of players (134,000 as of June 2021), across a wide geographic area. We filtered the data for responses from US teacher-hosted games, and analyzed the aggregate data. Over the nine-month period analyzed (September 2018 through June 2019), there were more than 1,000 unique US teacher hosts (based on the number of unique IP addresses) of the polar Kahoot!. At least three teachers hosted the quiz in each of the 50 US states and Washington, DC. This shows that US teachers are interested in including polar content in their classrooms. Furthermore, more than 25,000 of the respondents to the first question were US middle and high school students.

Because the first three quiz questions were adapted from questions asked of adults in a 2016 survey (Hamilton 2016, 2018; Hamilton et al. 2017), we could assess relative student knowledge. The average score of US students on these questions was 30% (SD = 06), much lower the 49% average accuracy found in the US adult survey (SD = .18). Each of the questions had four multiple choice answers, so random choice would give a score of 25%, indicating that many of the US student Kahoot! players were guessing in their responses to the polar questions.

The Kahoot! game-based player response system records which state each response comes from, so we can get a glimpse of regional variations in student polar knowledge. We say a glimpse, because only thirteen states had enough responses to analyze. Rhode Island, Oregon, Arizona, and Washington demonstrated the highest levels of student polar knowledge, with average accuracies ranging from 53-50% (Rhode Island to Washington), SD 0.13-0.09. New York, Florida, Arkansas, Michigan, Texas, and Indiana had intermediate accuracies ranging from 45-41%, SD 0.12-0.05. California had an average accuracy of 35%, but the pattern was highly variable with the largest standard deviation of the 13 states analyzed at 0.16. The average accuracies of student responses in Ohio and Illinois were the lowest at 27% and 28%, SD 0.01 and 0.02, respectively, reflecting a high degree of guessing. Because this was an exploratory analysis, we do not yet know the reasons for these regional differences.

Now that we have this suite of questions along with benchmark scores, they can be used to understand the impact of polar educational interventions by offering them pre- and/or post-activity. The Kahoot! platform is free for educators to use and develop their own Kahoot!s to support both live learning sessions and asynchronous challenges with students. It would be terrific if others would create additional Kahoot! quizzes tag them within the Kahoot! description area with #polar to build out a searchable community inventory of polar questions.

Figure 1. Screenshot of the first four questions in the polar Kahoot! of the Climate Challenge (Turrin et al., 2019). Each question has an accompanying image. Sometimes the image provides information relevant to answering the question. Answers are hidden in this image.
Figure 1. Screenshot of the first four questions in the polar Kahoot! of the Climate Challenge (Turrin et al., 2019). Each question has an accompanying image. Sometimes the image provides information relevant to answering the question. Answers are hidden in this image.

Editor's note: Screen shot above is from the Kahoot! website.

Figure 2. Results from cluster analysis of the 13 US states with enough responses to analyze. Grouping is according to similarities in patterns of accuracy across analyzed questions. The vertical axis represents the average dissimilarity distinguishing states or groups of states. For example, 0 would be identical patterns and 100 would indicate maximum dissimilarity. Figure courtesy of Pfirman et al., 2021.
Figure 2. Results from cluster analysis of the 13 US states with enough responses to analyze. Grouping is according to similarities in patterns of accuracy across analyzed questions. The vertical axis represents the average dissimilarity distinguishing states or groups of states. For example, 0 would be identical patterns and 100 would indicate maximum dissimilarity. Figure courtesy of Pfirman et al., 2021.

References

Arctic Monitoring & Assessment Program. 2021. Arctic Climate Change Update 2021, https://www.amap.no/documents/download/6730/inline

Hamilton, L.C. 2016. Where is the North Pole? An election-year survey on global change. Durham, NH: Carsey School of Public Policy. http://scholars.unh.edu/carsey/285/

Hamilton, L.C. 2018. Self-assessed understanding of climate change. Climatic Change 151(2):349–362. https://doi.org/10.1007/s10584-018-2305-0

Hamilton, L.C., J. Wirsing, J. Brunacini and S. Pfirman. 2017. Arctic knowledge of the U.S. public. Witness the Arctic (1 May 2017). https://www.arcus.org/witness-the-arctic/2017/5/highlight/2

Pfirman, S., L. Hamilton, M. Turrin, C. Narveson and C. Lloyd. 2021. Polar knowledge of US students as indicated by an online Kahoot! quiz game. Journal of Geoscience Education 69(2):150-165. https://doi.org/10.1080/10899995.2021.1877526 https://www.tandfonline.com/doi/full/10.1080/10899995.2021.1877526

About the Authors 

Stephanie PfirmanStephanie Pfirman is Foundation Professor, School of Sustainability and Senior Sustainability Scientist, Julie Ann Wrigley Global Institute of Sustainability and Innovation, Arizona State University. Professor Pfirman's research focuses on understanding and responding to the changing Arctic, developing innovative approaches to formal and informal education, and exploring the intersection between diversity and interdisciplinarity. spfirman [at] asu.edu


Lawrence HamiltonLawrence Hamilton is professor of sociology and senior fellow at the Carsey School of Public Policy, University of New Hampshire. His research involves integrated social/natural science studies of human-environment interactions, as well as large-scale surveys of U.S. public knowledge and perceptions about science and the Earth's polar regions. lawrence.hamilton [at] unh.edu.


Margie TurrinMargie Turrin is Director of Educational Field Programs at Lamont-Doherty Earth Observatory, Columbia University, Palisades. New York. Her work involves working with 'students' of all ages, using field science as a tool to develop a sense of place in participants while engaging in problem-based learning. Her work has taken her to the field in both polar regions. mkt [at] ldeo.columbia.edu


Craig NarvesonCraig Narveson is Director of Strategic Partnerships at Kahoot! AS, Oslo, Norway, craig [at] kahoot.com


Carrie A. LloydCarrie A. Lloyd is the senior research analyst at Global Futures Laboratory, Arizona State University, Tempe, Arizona. carrie.lloyd [at] asu.edu