Afterschool Impacts Database
Evaluation 101
Learn more about the basics of evaluation, as well as how to find an evaluator for your program.
Afterschool Evaluation
View Afterschool Alliance resources, including a glossary of terms used in the database, evaluation-related blogs, webinars and more.
Additional Resources
Our list of evaluation resources from other organizations, including how to collect and work with data.
Afterschool Research
Want to find what we know about afterschool programs more broadly, not just individual programs? Head to our Afterschool Research page!
Hawaii Statewide Evaluation of the 21st Century Community Learning Centers Program: School Year 2017-18 Evaluation Report
Year Published: 2019
A statewide evaluation of Hawaii’s 21st Century Community Learning Center (21st CCLC) programs during the 2017-18 school year examined outcomes related to participants’ educational, social, and behavioral developments. Based on teacher reports, among students who participated in the program 30 days or more, 82 percent improved turning in the homework on time and classroom participation and 79 percent improved their behavior in class. Of the sub-grantees reporting on academic improvement, 69 percent of students regularly attending the program showed improvements in English and 72 percent saw gains in math based on teacher surveys.
Program Name: Hawaii 21st Century Community Learning Centers
Program Description:
Hawaii’s 21st Century Community Learning Center program—which receives federal support through the 21st CCLC initiative—serves high-needs communities across the state, providing local afterschool and summer programming through 19 sub-grantees operating 74 centers serving 10,765 students during the 2017-18 school year.
Scope of the Evaluation: Statewide
Program Type: Summer, Afterschool
Location: Hawaii
Grade level: Elementary School, Middle School, High School
Program Demographics:
All schools served by Hawaii’s 21st CCLC programs are Title I eligible, meaning more than 40 percent of students qualify for the Federal Free and Reduced Price Lunch program. In the 2017-18 program year, 51.5 percent of were eligible for Free or Reduced Price Lunch, 7.4 percent of students had limited English proficiency, and 7.3 percent of students had special needs or disabilities. Regarding race and ethnicity, 36.9 percent of students identified as Native Hawaiian/Pacific Islander, 23.1 percent as Asian, 7.5 percent as Hispanic/Latino, 5.9 percent as White, 1 percent as Black/African American, less than 1 percent (.6 percent) as American Indian/Alaska Native, 14.5 percent as two or more races, and 10.5 percent did not provide this data.
Program Website: http://www.hawaiipublicschools.org/ParentsAndStudents/SupportForStudents/21stCCLC/Pages/default.aspx
Evaluator: Barker, L.T., Magill, K., & McLelland, C. IMPAQ International, LLC.
Evaluation Methods:
Data collected included quantitative data from the Annual Performance Reporting (APR) system, which includes teacher reported data and end-of-year surveys, and a review of the 2017-18 sub-grantee evaluation reports submitted to the Hawaii Department of Education.
Evaluation Type: Non-experimental
Summary of Outcomes:
During the 2017-2018 school year, students who regularly attended Hawaii's 21st Century Community Learning Centers program (at least 30 days) showed improvement in homework submission and classroom participation, as well as in their classroom behavior. Based on teacher reports, among all students who participated in the program 30 days or more, 82 percent improved turning in their homework on time and classroom participation and 79 percent improved their behavior in class. Examining differences by grade level and attendance levels, the vast majority of Pre-K through 5th grade students (80 percent) attending 30-59 days improved their homework and classroom participation, and classroom behavior. Among those who attended 60-89 days of programming, 88 percent improved in homework and class participation, and 86 percent improved their classroom behavior. The vast majority of 6th through 8th grade students who attended the program 30-59 days also showed improvements in homework and class participation (80 percent) and classroom behavior (78 percent). Among those who attended 90 days or more of programming, 87 percent improved their homework and class participation, and 81 percent improved their classroom behavior.
The evaluation reported out on academic data pulled from sub-grantee evaluation reports. Of the ten sub-grantees reporting on academic improvement based on teacher surveys, 69 percent of regularly attending students showed improvements in English and 72 percent in math. It is important to note that the evaluators indicated discrepancies in data, with some sub-grantees reporting on improvement of all participating students and others reporting only on students who were in need of improvement in the respective areas.
Sub-grantees additionally reported out on program-specific goals and objectives that included student and parent satisfaction. For instance, based on student and parent feedback from Kanu O Ka Aina Learning Ohana centers, 91 percent of parents agreed that the program helped their child complete homework on time, 95 percent of students indicated they “almost always” felt safe in the afterschool program, and 65 percent of students reported enjoying the program.
Date Added: August 28, 2019
