Evaluator: Shooter, W., Franklin, J. D., Eddings, S., Groth, C., & Nguyen, H. Utah Education Policy Center

Evaluation Methods:

Evaluators administered staff and student surveys and collected program participation records from program sites and student education data from the Utah State Board of Education. Staff surveys were administered at the start and the end of the 2016-2017 school year, gathering information on professional development needs and feedback, program implementation, and overall experience in the program. Students were given a retrospective survey, which functioned as the pre-and posttest. They were asked questions about their interest in STEM and other STEM-related outcomes. Evaluators matched student attendance and participation data with education data from the Department of Education, including Student Assessment of Growth and Excellence (SAGE) scores, demographics, and school-day attendance. Paired sample t-tests were used to compare students’ retrospective and posttest surveys, while participation and education data were matched to report out on academic outcomes and chronic absenteeism. 

In the third year of the STEMLink program (2016-2017), 10 out of 13 grantees provided participation data for 1,987 students. 

Evaluation Type: Experimental

Summary of Outcomes:

Evaluators found increased interest and participation in STEM, greater awareness of STEM fields, and increased perseverance and critical thinking skills among students participating in the STEMLink program. The study also found that STEMLink programs performed well in areas aligned with quality programming, such as providing a positive program climate and staff with the support and tools to perform their job effectively.

Students reported greater interest in science, engineering, and technology after participating in the program, with more than 70 percent of students reporting that they were more interested in technology and engineering because of their involvement. Researchers also found that students sought out opportunities to learn more about STEM than they did prior to the program, including reading STEM-related magazines or articles, and that students’ awareness of STEM careers increased, as did their interest in STEM-related postsecondary fields or careers.

Moreover, the study found that students in the STEMLink program saw gains in their perseverance and critical thinking scores comparing retrospective and posttest results, as well as significant, positive increases in work habits, social competence, academic behavior, empathy, and prosocial behaviors.

Students overwhelmingly reported a positive program experience. More than 9 in 10 students said that they liked the activities they did in their program (97 percent), had fun (96 percent), and liked attending the program (93 percent). And more than 8 in 10 students said that they trusted the adults in their program (88 percent), looked up to the adults in their program (88 percent), and had adults in the program who they could go to with their problems (85 percent).

The evaluation also found positive results regarding aligning program activities with students’ needs and supporting staff to deliver programming. For example, 80 percent of staff reported that they were collaborating with school-day personnel, and 76 percent said that their STEMLink programs used data to make decisions around activities. Additionally, most staff reported that they participated in useful training or professional development (81 percent) and more than 9 in 10 staff reported that they implemented practices learned through their afterschool program’s professional development (98 percent), received support from their supervisors (95 percent), and had the resources they needed to do their job effectively (93 percent).