Afterschool Impacts Database
Evaluation 101
Learn more about the basics of evaluation, as well as how to find an evaluator for your program.
Afterschool Evaluation
View Afterschool Alliance resources, including a glossary of terms used in the database, evaluation-related blogs, webinars and more.
Additional Resources
Our list of evaluation resources from other organizations, including how to collect and work with data.
Afterschool Research
Want to find what we know about afterschool programs more broadly, not just individual programs? Head to our Afterschool Research page!
21st Century Community Learning Centers Program Fiscal Year 2018 Year End Report (Massachusetts)
Year Published: 2019
A statewide evaluation of Massachusetts’s 21st Century Community Learning Centers (21st CCLC) program during the 2017-18 program year found that, based on teacher ratings, program participants made gains in math and reading/English language arts achievement. Teachers and program staff also noted increases in outcomes including students’ critical thinking skills, perseverance, and communication. English language learners, students receiving special education services, and economically disadvantaged students made particularly strong gains for certain outcomes.
Program Name: Massachusetts 21st Century Community Learning Centers
Program Description:
Massachusetts’s 21st Century Community Learning Center program, which receives federal funding through the 21st CCLC Initiative, provides academic enrichment opportunities for children at high-poverty and low-performing schools. During the 2017-18 program year, 42 grantees operated 143 sites, serving nearly 22,400 youth.
Scope of the Evaluation: Statewide
Program Type: Summer, Afterschool
Grade level: Elementary School, Middle School, High School
Program Demographics:
A majority of students served by Massachusetts’s 21st CCLC were elementary (65 percent), followed by middle school (26 percent), and then high school (9 percent). During FY2018, 59 percent were economically disadvantaged, 19 percent were identified as students with disabilities, and 29 percent were identified as English language learners. In regards to race and ethnicity, 45 percent identified as Hispanic, 31 percent as White, 14 percent as African American, 5 percent as Asian, and 5 percent as “other.”
Evaluator: Massachusetts Department of Elementary and Secondary Education
Evaluation Methods:
This evaluation uses the Survey of Academic and Youth Outcomes (SAYO) teacher, staff, and youth surveys to evaluate pre- and post-survey data on outcomes including academics, social and emotional skills and competencies, youth experience, and program quality. Pre- and post-surveys were taken at the beginning and at the end of the school year.
Evaluation Type: Non-experimental
Summary of Outcomes:
A statewide evaluation of Massachusetts’s 21st CCLC program found that students made behavioral and academic gains during the school year. Based on the teacher SAYO survey, the average percentage change from pre- to post-survey was 11 percent for English language arts, 10 percent for math, and 7 percent for homework completion.
Both teachers and staff members reported behavioral improvements as well, although the highest changes were reported by staff members. The average percentage change from pre- to post-survey reported by staff members were: 16 percent for communication, 16 percent for perseverance, 15 percent for critical thinking skills, 14 percent for both self-regulation and leadership skills, 13 percent for relationships with peers, and 11 percent for both relationships with adults and engagement. In addition, it was found that economically disadvantaged students made significantly greater gains on eight of the 19 outcomes compared to non-economically disadvantaged students, and those receiving special education services (SPED) made significantly greater gains on 10 of the 19 outcomes compared to non-SPED students. English language learner (ELL) students made significantly greater gains in ELA compared to non-ELL students.
Students also completed the SAYO youth survey to measure their experiences in the program. In the spring survey, with questions using a 1-4 scale, students were most likely to indicate that the program helped them to “make new friends (2.94), “find out what I’m good at doing (2.92), and “get along better with others” (2.91).
The evaluation also looked at Massachusetts’s 21st CCLC summer programs. Similar to the school year, staff completed the SAYO survey to measure improvements pre- to post- survey. The average percentage change was highest for improvements in critical thinking (18 percent), engagement (17 percent), relationships with adults (17 percent), perseverance (15 percent), and communication (15 percent).
Date Added: July 28, 2021
