Afterschool Impacts Database
Evaluation 101
Learn more about the basics of evaluation, as well as how to find an evaluator for your program.
Afterschool Evaluation
View Afterschool Alliance resources, including a glossary of terms used in the database, evaluation-related blogs, webinars and more.
Additional Resources
Our list of evaluation resources from other organizations, including how to collect and work with data.
Afterschool Research
Want to find what we know about afterschool programs more broadly, not just individual programs? Head to our Afterschool Research page!
Aiming Higher: Assessing Higher Achievement’s Out-of-School Expansion Efforts
Year Published: 2020
The study found that students who participated in the Higher Achievement afterschool and summer program over the course of two years had statistically significantly higher grades in math, English, and science, as well as higher overall GPAs, compared to a matched control group. The program appeared to be most effective for students who joined the program on grade level. The subgroup analysis also found that male students in the program also saw greater gains in math compared to girls in year one and two of the program.
Program Name: Higher Achievement
Program Description:
Higher Achievement is an intensive summer and afterschool program that provides more than 500 additional hours of academic enrichment a year to middle school youth in low-income, under-resourced neighborhoods. The program aims to help its “scholars” develop skills, behaviors, and attitudes that will ultimately help improve their academic performance and increase their acceptance into competitive high schools. Scholars enter the program before their 5th or 6th grade year, and are expected to continue through 8th grade.
Scope of the Evaluation: Multi-city
Program Type: Summer
Location: Washington, D.C.; Baltimore, MD; Richmond, VA; Pittsburgh, PA
Grade level: Middle School
Program Demographics:
Among students in the expansion sites (Baltimore, Pittsburgh, and Richmond) in year two, 80.5 percent identified as Black, 14.5 percent identified as Hispanic, and 3 percent identified as White. Eighty-nine percent of students qualified for the free/reduced price lunch program, and families on average had an annual household income of $27,552.
Program Website: https://higherachievement.org/
Evaluator: Garcia, I., Grossman, J.B., Herrera, C., Strassberger, M., Dixon, M., & Linden, L. MDRC.
Evaluation Methods:
This study used random assignment to assess the impact of the Higher Achievement program on academic performance, specifically test scores and grades. The study included 1,817 rising 5th and 6th grade students who applied to the program in 2015 or 2016 across all 19 centers and four sites—Baltimore, Pittsburgh, Richmond, and the D.C. metro area (Washington, D.C. and Alexandria, Virginia). Three years of school record data was collected for each participant—the year prior to enrollment, the first year in the program, and the second year in the program—to assess changes in math, science, social studies, and English grades, as well as math and English language arts (ELA) standardized test scores. In addition, a randomly selected group of program and control parents participated in a phone survey to observe how their children’s academic enrichment experiences and supports received during their out-of-school time experiences differed.
Evaluation Type: Experimental
Summary of Outcomes:
The study found positive academic benefits of participation in the Higher Achievement program. After two years in the program, the program participant group of students earned higher grades in English, math, and science compared to the control group of students who did not participate in Higher Achievement. These differences were statistically significant. Further, students in the program had slightly higher test scores by the second year of program participation, however, the differences were not large enough to be statistically significant.
Additionally, the study looked at subgroups to determine whether the program was more or less beneficial for different types of students. It was found that Higher Achievement had a greater impact on students who started the program on grade level, the students that the program is typically designed for. By year two in the program, students who entered on grade level, rather than below grade level, had statistically significant greater impacts on grades in all four course areas—math, English, social studies, and science. Students who entered the program on grade level also had statistically significant greater impacts on both math and ELA test scores compared to those who entered below grade level.
Finally, the subgroup analysis revealed that program participation may be more beneficial to males than females. In both year 1 and year 2, the impacts on boys’ math grades were greater than the impacts on girls’ math grades. The study concluded that typically boys’ math performance falls much more than girls’ over time, however, with Higher Achievement, this drop in performance was reduced.
Associated Evaluation:
https://www.mdrc.org/publication/aiming-higher
Date Added: August 23, 2021
