Evaluator: Augustine, C. H., McCombs, J. S., Pane, J.F., Schwartz, H. L., Schweig, J., McEachin, A., & Siler-Evans, K. The RAND Corporation.

Evaluation Methods: At the start of this longitudinal randomized controlled trial during the spring of 2013, more than 5,000 third grade students were randomly selected to participate in the study, 3,194 of whom were assigned to the treatment group (participating in a summer learning program) and 2,445 of whom were assigned to the control group (students who did not participate in summer programming). In order to select students for the study, school districts advertised and recruited students to apply for a spot in the voluntary summer learning programming, and through a lottery program, students were randomly selected to participate in two summers of programming, with the remaining students serving as the control group. The study included direct program observation, teacher surveys, district data, as well as academic and social emotional assessments. This report continued to track students during the summer of 2014 and spring of 2015. Student academic performance, social and emotional skills and program attendance were tracked through seventh grade. Outcomes for the two groups were analyzed and compared as well as outcomes for high-attenders in comparison with all other students.

Evaluation Type: Experimental

Summary of Outcomes: The study found that students with high attendance in the summer programming saw significant near term benefits (gains in the fall after the summer program) and long-term benefits (gains seen through the following spring after the summer program) in math after summer programming in 2013 and 2014, near and long-term benefits in language arts after summer 2014, and positive benefits to their social and emotional skills after summer 2014.

When programs focused specifically on math or language arts, students saw lasting positive gains in these subjects. Students who received at least 25 hours of mathematics instruction or 34 hours of language arts outperformed students with less instruction time on subject-specific fall assessments. The estimated gains were between 23 percent and 31 percent of typical annual gains in mathematics, and 14 percent and 23 percent of typical annual gains in language arts.

The study also found that consecutive attendance in summer programs has a positive impact on students’ academic performance. Students who had a high level of attendance (20 days or more) during both the 2013 and 2014 summers saw greater gains in the math and language arts assessments compared to their non-participating peers.

The only significant results from the causal analysis between the control and treatment groups were short term impacts on math performance, and there was no clear evidence that offering two summers of programming had greater impacts on math performance than one summer. However, this is not surprising, given the fact that only 50 percent of treatment students participated in programming during summer of 2014. This decrease in participation is partially because 11 percent of students left the district before the second summer.

Associated Evaluation: http://www.rand.org/pubs/research_reports/RR1557.html

Date Added: December 19, 2016