Afterschool Impacts Database
Evaluation 101
Learn more about the basics of evaluation, as well as how to find an evaluator for your program.
Afterschool Evaluation
View Afterschool Alliance resources, including a glossary of terms used in the database, evaluation-related blogs, webinars and more.
Additional Resources
Our list of evaluation resources from other organizations, including how to collect and work with data.
Afterschool Research
Want to find what we know about afterschool programs more broadly, not just individual programs? Head to our Afterschool Research page!
The Impact of the ACE Program on Academic Achievement 2016-2017
Year Published: 2018
A quasi-experimental study examining high school students participating in EduCare Foundation’s ACE program found higher academic achievement—as measured by the California Assessment of Student Performance and Progress (CAASPP) in English language arts (ELA) and math—among ACE participants compared to matched peers attending general afterschool programs. For example, the mean CAASPP ELA score of high school students participating in the ACE program was 35.7 points higher than the matched control group; a statistically significant difference with a moderate to large effect size. When looking at CAASPP mean math scores, ACE participants scored 30.9 points higher than their matched peers, a statistically significant difference with a moderate effect size.
Program Name: EduCare Foundation’s ACE (Achievement and Commitment to Excellence) Program
Program Description:
EduCare Foundation’s ACE (Achievement and Commitment to Excellence) Program is a comprehensive afterschool program that takes a social and emotional learning approach while also offering academic enrichment; homework help and tutoring; physical activity; arts; science, technology, engineering and math (STEM); and college access. During the 2016-17 school year, the program was offered at 13 high schools in the Los Angeles Unified School District (LAUSD). They are funded through the 21st Century Community Learning Centers initiative, which provides support to high-needs communities across the country.
Scope of the Evaluation: Local
Program Type: Afterschool
Location: Los Angeles, California
Community Type: Urban
Grade level: High School
Program Demographics:
This evaluation comprised data from 13 of the LAUSD high schools offering EduCare Foundation’s ACE program during the 2016-17 school year. Of ACE students included in the study that represented more than 1 percent when looking at race and ethnicity, 91.2 percent were Hispanic, 4.8 percent were Asian, and 3.1 percent identified as “other ethnicity”. Regarding free and reduced price lunch status, 62.7 percent of students in the study qualified for the program. Additionally, 7 percent of students were identified as special education and 4.8 percent of students were English language learners.
Program Website: http://www.educarefoundation.com/home
Evaluator: Educational Resource Consultants.
Evaluation Methods:
The evaluation was a quasi-experimental, matched-pair design comparing outcomes—specifically performance on the California Assessment of Student Performance and Progress (CAASPP) in English language arts and math—for high school students participating in the ACE program (treatment group) to matched students participating in the general afterschool program (control group) using grade level, gender, race/ethnicity, English learner status, Gifted and Talented Education status, special education status, and school day and afterschool program attendance. Comparisons of the two groups were made through paired samples t-tests, using an alpha level of .05 to determine the statistical significance.
Evaluation Type: Quasi-experimental
Summary of Outcomes:
Using scale scores from the California Assessment of Student Performance and Progress (CAASPP) in English language arts (ELA) and math, the evaluation found that students participating in the ACE program scored higher in both areas than their peers who participated in a general afterschool program. The mean CAASPP ELA score of high school students participating in the ACE program was 35.7 points higher than the matched control group; a statistically significant difference with a moderate to large effect size. Additionally, the percentage of ACE participants meeting or exceeding standards in ELA on the CAASPP was 12 percentage points higher than their matched peers (71.6 percent compared to 59.6 percent).
When looking at CAASPP mean math scores, ACE participants scored 30.9 points higher than their matched peers, a statistically significant difference with a moderate effect size. A higher percentage of ACE participants also met or exceeded standards on the math CAASPP compared to the matched control group (72.1 percent compared to 61.9 percent).
Date Added: November 27, 2018
