Evaluator: McCombs, J. S., Augustine, C. H., Pane, J. F., & Schweig, J. RAND Corporation.

Evaluation Methods:

This study is the third phase of a research project commissioned by the Wallace Foundation to examine implementation and outcomes associated with the National Summer Learning Project (NSLP). Summer program applicants were randomly assigned into two groups: a treatment group that participated in two consecutive summers of voluntary summer programming (2013 and 2014) and a control group that did not participate. At the start of the study during the spring 2013, 5,639 third grade students were randomly selected to participate, with more than 3,000 of whom assigned to the treatment group. Student outcomes were measured through spring 2015 for phase II, and the current follow up (phase III) looks at student outcomes in the spring 2017, three school years after the second summer of programming. Data—such as language arts and math state assessment results, course grades, social and emotional assessments, and suspension and attendance—were collected through classroom observations, teacher surveys, teacher and administrator interviews, and administrative attendance records. 

Evaluation Type: Experimental

Summary of Outcomes:

A longitudinal study of summer learning programs that followed students from 3rd through 7th grade found positive academic gains among the students randomly assigned to the summer programs compared to the control group comprised of nonparticipants. After the first summer of programming, program participants outperformed control group students on their fall math assessments. Among the students who were “high attenders” in the summer program (attending for 20 days or more), the gains students experienced accounted for approximately 25 percent of an average annual gain for fall assessments, and 13 percent of the annual gain in the spring. However, program participation was not found to have a significant effect on language arts assessments, or social-emotional skills and school-year attendance after the first year.

After the second year of summer programming, high attenders outperformed the control group in math, language arts, and social-emotional skills. These estimated gains were between 14 percent to 21 percent of typical annual gains in math, and 17 percent to 25 percent of the typical annual gains in language arts. High attenders also scored higher on the teacher administered Devereux Student Strengths Assessment-RAND Research Edition (DESSA-RRE), demonstrating stronger social-emotional skills than the control group when they returned to school in fall 2014.

Consecutive attendance was also found to have a positive impact on students’ academic performance, with those who participated at least 20 days both summers having greater gains in math and language arts assessments compared to the control group of students. The study also found that greater amount of time on task (defined as students who received a minimum of 25 hours of summer math instruction and 34 hours of language arts instruction) during both summers were positively correlated with better academic outcomes in math state assessments in the fall of 2014 and spring of 2015, and on language arts assessments in the spring of 2015.

In the follow up study three years after the students participated in the two consecutive summers of programming, it was found that the academic benefits were no longer statistically significant, however, still large enough to be important when benchmarked against typical grade level achievement gains. High attenders who participated in both summers of programming saw 19 percent of typical annual growth in language arts and 23 percent in math. 

Associated Evaluation:

https://www.wallacefoundation.org/knowledge-center/pages/every-summer-counts-a-longitudinal-analysis-of-outcomes-from-the-national-summer-learning-project.aspx

Date Added: August 23, 2021