Learn more about the basics of evaluation, as well as how to find an evaluator for your program.
View Afterschool Alliance resources, including a glossary of terms used in the database, evaluation-related blogs, webinars and more.
Our list of evaluation resources from other organizations, including how to collect and work with data.
Want to find what we know about afterschool programs more broadly, not just individual programs? Head to our Afterschool Research page!
Year Published: 2014
Academic results from year two of a five-year randomized controlled study that follows 5,000 low-income, predominantly African-American and Hispanic students from third to fifth grade in five urban school districts located in Boston, MA; Dallas, TX; Duval County, FL; Pittsburgh, PA, and Rochester, NY. Evaluators compared student success on math tests in the fall following their summer learning programs with the success of students who did not participate in summer programming. Key findings indicate that students who participated in summer programming performed significantly better on their math tests in the fall following the programs.
Scope of the Evaluation: Multi-city
Program Type: Summer
Location: Urban school districts in Boston, MA; Dallas, TX; Duval County, FL; Pittsburgh, PA and Rochester, NY.
Community Type: Urban
Grade level: Elementary School
Program Demographics: Eighty-six percent of participating youth in the study were low-income, qualifying for free or reduced-price lunch. Just under half of participants were African-American (47 percent), 40 percent were Hispanic, 7 percent were white, and 3 percent were Asian. Close to one third of students (31 percent) were English language learners and 10 percent had special education needs.
Evaluator: McCombs, J. S., Pane, J. F., Augustine, C. H., Schwartz, H. L., Martorell, P., & Zakaras, L. RAND Corporation.
Evaluation Methods: At the start of the longitudinal study during the spring of 2013, more than 5,000 third-grade students were randomly selected to participate in the study, 3,194 of whom were assigned to the treatment group (participating in a summer learning program) and 2,445 of whom were assigned to the control group (students who did not participate in summer programming). In order to select students for the study, school districts advertised and recruited students to apply for a spot in the summer learning programming, and through a lottery program, students were randomly selected to participate in two summers of programming, with the remaining students serving as the control group. The study included direct program observation, teacher surveys, district data, as well as academic and social-emotional assessments. Math and reading achievement was measured based on student success on the Group Mathematics Assessment and Diagnostic Evaluation (GMADE) mathematics assessment and the Group Reading Assessment and Diagnostic Evaluation (GRADE) reading assessment in the fall following their summer program participation. Social and emotional competencies were measured using a version of the Devereux Student Strengths Assessment.
Evaluation Type: Experimental
Summary of Outcomes: Students participating in the summer learning programs performed better on math tests in the fall following their summer program than their non-participating peers. The study reported that a significant positive effect was found for a range of math skills, including “concepts and communication, operations and computation, and process and application.” However, researchers did not find a statistically significant difference in reading skills or social-emotional competencies between participants and non-participants.
Additionally, the study found a positive association between students’ math scores, the days students attended the summer program, and the amount of time spent in the program each day.
Associated Evaluation: http://www.rand.org/pubs/research_reports/RR815.html
Date Added: December 19, 2016