Afterschool Impacts Database
Evaluation 101
Learn more about the basics of evaluation, as well as how to find an evaluator for your program.
Afterschool Evaluation
View Afterschool Alliance resources, including a glossary of terms used in the database, evaluation-related blogs, webinars and more.
Additional Resources
Our list of evaluation resources from other organizations, including how to collect and work with data.
Afterschool Research
Want to find what we know about afterschool programs more broadly, not just individual programs? Head to our Afterschool Research page!
New Jersey 21st Century Community Learning Centers Year 4 Evaluation Report Impact Data for 2015-16
Year Published: 2017
A statewide evaluation of New Jersey’s 21st Century Community Learning Center (21st CCLC) programs during the 2015-16 school year found statistically significant positive impacts in math achievement and reduced school day truancy incidents and grade retention among students who regularly participated in the programs. Program staff also reported positive perceptions of the ability of 21st CCLC programming to connect with the school day, as well as promote youth engagement and decision-making.
Program Name: New Jersey 21st Century Community Learning Centers
Program Description:
New Jersey’s 21st Century Community Learning Center program, which receives federal funding through the 21st CCLC Initiative, provides academic enrichment opportunities for children at high-poverty and low-performing schools. During the 2015-16 school year, 52 grantees offered afterschool and summer programming through 130 centers, serving 15,449 students.
Scope of the Evaluation: Statewide
Program Type: Afterschool
Location: New Jersey
Grade level: Elementary School, Middle School, High School
Program Demographics:
All schools served by New Jersey’s 21st CCLC programs are Title I eligible, meaning more than 40 percent of students qualify for Free and Reduced Price Lunch. In the 2015-16 program year, 75.2 percent of students were eligible for Free or Reduced Price Lunch. With regard to race and ethnicity, 44.9 percent of students identified as Hispanic or Latino, 34.9 percent identified as Black, 11.1 percent identified as White, 2.2 percent identified as Asian, 0.2 percent identified as Native American, 0.1 percent identified as Pacific Islander, and 6.6 percent were identified as “unknown”.
Program Website: https://www.state.nj.us/education/students/safety/afterschool/
Evaluator: Vinson, M. & Swanlund, A. American Institutes for Research
Evaluation Methods:
This evaluation used New Jersey Department of Education’s data collection system to gather program characteristics, student demographics, and attendance and outcomes data; American Institutes for Research’s Evaluation Tracking and Reporting System to record staff surveys; and New Jersey Standards Measurement and Resource for Teaching Data Warehouse to access state assessment scores, truancy data, and grade promotion data. Propensity score matching was used to compare the academic performance of students in 21st CCLC programs versus non-attendees.
Evaluation Type: Quasi-experimental;Non-experimental
Summary of Outcomes:
The evaluation of New Jersey’s 21st CCLC programs during the 2015-16 academic year found statistically significant positive impacts in math achievement and reduced school day truancy incidents and grade retention among students who regularly participated in the programs. Math achievement for both students who participated in at least 30 hours of math-focused programming and attending 21st CCLC programs for at least 30 days and students attending 21st CCLC programs at least 60 days was higher than that of non-participants. Although effect sizes were small in math, both were significant positive results. There was no significant effect found for reading achievement among regularly attending 21st CCLC students.
There was a statistically significant negative effect on truancy found among students attending 21st CCLC programming for 60 or more days, with slightly lower truancy rates for program participants compared with non-participants. Students attending 21st CCLC programming for 60 or more days were also statistically significantly less likely to be retained a grade level.
Program staff perceived that 21st CCLC programming helped promote youth engagement, autonomy, and choice. Based program staff surveys, almost all program staff agreed or strongly agreed that program staff working with youth “actively and continuously consult and involve youth” (96.2 percent); “provide youth with meaningful choices when delivering activities” (94 percent); “provide opportunities for youth to lead activities” (90.5 percent); and, “ask for and listen to student opinions about the way things should work in the program” (87.2 percent). Program staff also agreed that they help build student ownership of their afterschool program. The vast majority of program staff agreed or strongly agreed that “youth have the opportunity to set goals for what they want to accomplish in the program” (85.1 percent); “youth are afforded opportunities to take responsibility for their own program” (82.4 percent); and, “youth help create rules and guidelines for the program” (66.6 percent).
Program staff also reported that they are able to connect what they do in afterschool with the school day. The majority of program staff agreed or strongly agreed that the activities they provided are “tied to specific learning goals that are related to the school-day curriculum” (73 percent); that they “use student assessment data to provide different types of instruction to students…based on their ability level” (56.1 percent); and, that they “coordinate the content of afterschool activities with students’ school-day homework” (52 percent).
