Learn more about the basics of evaluation, as well as how to find an evaluator for your program.
View Afterschool Alliance resources, including a glossary of terms used in the database, evaluation-related blogs, webinars and more.
Our list of evaluation resources from other organizations, including how to collect and work with data.
Want to find what we know about afterschool programs more broadly, not just individual programs? Head to our Afterschool Research page!
Year Published: 2023
This case study examines the outcomes of an out-of-school-time program’s partnership with a citizen science hub to support STEM learning. The partnership facilitated science learning activities and citizen science projects for over 200 Girl Scout troops across various regions of the U.S. All of the program’s possible student learning outcomes were present by the end of the program, including the development of science literacy, science identity, STEM confidence, and taking specific actions to address scientific and environmental challenges. A key feature of the program was its unstructured nature, allowing for youth self-direction and unintended learning outcomes.
Program Name: Girl Scouts of the USA
Program Description:
The Think Like a Citizen Scientist Journey is a partnership between Girl Scouts of the USA and SciStarter. The program was piloted in 2017 to encourage Girl Scouts’ engagement in citizen science to support STEM learning. The projects were to be outdoors, not require specialized equipment, and be appropriate for a variety of age groups. Girls chose projects by going to a SciStarter landing page where they completed a series of age-appropriate STEM learning activities, participated in a citizen science project, and then carried out a Take Action Project in their community.
Scope of the Evaluation: National
Program Type: Summer, Afterschool
Grade level: Elementary School
Program Website: https://www.girlscouts.org/en/activities-for-girls/cadettes/cadette-think-like-a-citizen-scientist-journey-badge-activity
Evaluator: Smith, H. E., Cooper, C. B., Busch, K. C., Harper, S., Muslim, A., McKenna, K., & Cavalier, D
Evaluation Methods:
Manual qualitative analyses through deductive and inductive coding of open-ended survey responses were performed, guided by the following questions:
Data merging and cleaning were performed in RStudio using R version 4.1.3 (R Core Team Citation 2022). A total of 245 completed journeys were analyzed, with participants from 231 Girl Scouts troops from 43 states and Washington, DC.
Evaluation Type: Non-experimental
Summary of Outcomes:
The study showed that the Think Like a Citizen Scientist Journey demonstrates great potential to support varied forms of learning. A wide array of cognitive, affective, and behavioral outcomes were identified among participants, including science identity, STEM confidence, and taking actions to address scientific and environmental challenges. The program’s scaffolded Girl Scout Journeys facilitated the girls’ progress towards achieving the learning outcomes, such as Engaging in Scientific Reasoning, Engaging in Scientific Practices, and Identifying and Solving Problems in the Community. Moreover, by facilitating knowledge sharing and providing outlets for citizen science engagement, the authors conclude that this program is an excellent model for how citizen science participation can foster community science literacy.
The analysis of learning outcomes revealed that all possible learning outcomes were represented throughout the Journeys and reflected girls’ ability to address problems in their communities, understand scientific content and knowledge, and develop STEM confidence. The most common outcome was the ability to identify and solve problems in the community, followed by understanding scientific content and knowledge. Participating girls could choose to solve any problem in the community for their Take Action Projects (TAPs), but 81% took an action related to science or the environment. Among that group, 66% of girls’ TAPs involved educating and inspiring others, 19% involved improving the environment, 9% involved citizen science, 3% promoted science literacy, and 2% involved taking civic action. The participants’ primary audiences were peers, adults, the general public, and civic leaders.
The authors also conclude that the facilitated and scaffolded nature of the learning experience, which moved girls from exploration to execution of a TAP, was a key attribute of facilitated citizen science that can foster the various learning outcomes. The TAP itself provided unique opportunities for learning and development of skills beyond the typical focus on scientific skills and content and process knowledge. These necessary, but often unmeasured, experiences and skills include raising awareness, advocating, organizing others, designing solutions, and purposefully participating in efforts to address science-related public matters. Moreover, the TAP component had broad guidelines, which the authors suggest permitted many positive and unintended learning outcomes from being youth-led.
Another valuable outcome of this program was the participants sharing their citizen science projects and information with the wider community. As they communicated scientific information and demonstrated expertise, their own scientific understanding, degree of agency, and confidence in STEM were strengthened, along with their confidence in their ability to take action. In turn, the participants’ STEM confidence and identity were improved. There were also broader societal implications as the information and knowledge were spread among family and peers.