Afterschool Impacts Database
Evaluation 101
Learn more about the basics of evaluation, as well as how to find an evaluator for your program.
Afterschool Evaluation
View Afterschool Alliance resources, including a glossary of terms used in the database, evaluation-related blogs, webinars and more.
Additional Resources
Our list of evaluation resources from other organizations, including how to collect and work with data.
Afterschool Research
Want to find what we know about afterschool programs more broadly, not just individual programs? Head to our Afterschool Research page!
Ohio 21st Century Community Learning Centers Year 5 Evaluation Report
Year Published: 2020
A 2020 evaluation of Ohio's 21st Century Community Learning Centers programs found that there was a statistically significant positive impact on academic and behavioral outcomes among students regularly attending the program. Regular program participants scored higher on math and English language arts assessments compared with similar non-participating youth, as well as saw reductions in both unexcused absences and disciplinary incidents. Youth surveys revealed that students agree that their program is beneficial, including helping them make new friends and prepare for their future.
Program Description:
Ohio’s 21st Century Community Learning Centers (21st CCLC) program, which receives federal funding through the 21st CCLC initiative, provides afterschool and summer academic enrichment opportunities for children at high-poverty and low-performing schools throughout the state. During the 2017-18 school year, 206 grant organizations operated 242 centers, serving 18,904 students.
Scope of the Evaluation: Statewide
Program Type: Summer, Afterschool
Grade level: Elementary School, Middle School, High School
Program Demographics:
During the 2017-18 program year, 78.7 percent of participants were economically disadvantaged. Regarding race and ethnicity, 55.5 percent identified as White, 28.9 percent identified as Black/African American, 6.9 percent identified as Hispanic/Latino, about 6 percent identified as multiracial, and 2 percent identified as Asian.
Evaluator: Vinson, M., Liu, F., & Lin, S., American Institutes for Research
Evaluation Methods:
This evaluation uses data collected in the 2017-18 school year. Multiple sources of data were used, including 21st CCLC attendance data, youth and teacher surveys, and Ohio Department of Education EMIS data (youth demographics, school attendance, disciplinary data, state assessments, and school level data). Basic descriptive analytic techniques, Rash analytic techniques, and propensity score matching were used to analyze data. Results in the study focus on students participating in one of two grantee path structures: Path B, which is “out-of-school-time programming focused on K-4 literacy” and Path C, “out-of-school-time programming focused on literacy and college and career readiness or dropout prevention strategies for middle and high school students.”
Evaluation Type: Quasi-experimental
Summary of Outcomes:
A 2020 evaluation of Ohio's 21st CCLC programs found a statistically significant positive impact of participation on academic and behavioral outcomes. Regular program participants (30 days or more) who were in need of improvement scored higher on both math and English language arts (ELA) assessments compared to similar nonparticipating youth attending the same schools. For math assessments, a small but statistically significant effect was found for youth in Path B attending at least 30 days of programming. For ELA assessments, a more sizeable significant effect was found for those attending at least 60 days with 60 hours in literacy activities in Path C. It was also found that higher attending program participants (45 days or more) in both Path B and C scored higher on ELA assessments compared with low-attending participants (less than 15 days).
Regarding behavioral improvements, the evaluation found statistically significant reductions in unexcused school day absences among 21st CCLC participants, as well as reductions in disciplinary incidents. Path B and C regular program participants showed statistically significant reductions school day absences, with higher levels of participation (60 days or more) associated with an even greater reduction in unexcused absences. Additionally, regularly attending students in Path B and C had statistically significant fewer disciplinary incidents compared to non-attendees. Teacher survey results also revealed behavioral improvements among program participants. Six constructs were measured (classroom behavior, initiative, engagement, problem solving, communication skills, and relationships with adults), and of those six, program participants showed statistically significant improvement pre- to post-survey in the areas of engagement, problem solving, communication skills, and relationships with adults.
Based on youth surveys, more than 9 in 10 students agree that their program has helped them learn things that will be important for their future (94 percent) and in school (94 percent), find out what is important to them (93 percent), make new friends (92 percent), and improve their confidence (91 percent).
Date Added: June 9, 2022
