Putting Research Into Practice – for an academic afterschool program, post COVID

This guest blog is by Lynsey Wood Jeffries, CEO of Higher Achievement, an afterschool and summer program serving middle school scholars in Washington, D.C., Baltimore, MD, and Richmond, VA. Findings from the evaluation on Higher Achievement was recently included in an Afterschool Alliance brief, “The Latest Research on the Impact of Afterschool and Summer Programs.”

Schools today are daunting. Students are 2.5 times more likely to attend a school with chronic absenteeism. Teachers are missing far more school than pre-COVID, and there are far too few school counselors. With less time and consistency in classrooms, learning is slower to rebound in marginalized communities. The systemic barriers to equitable education persist, as do the resulting achievement gaps across race and income, and the U.S. is 28th in math scores internationally.

Coupled with chronic absenteeism and achievement gaps, there is a growing public mental health crisis with 1 in 5 children facing serious mental health challenges — a 65% growth in feelings of “persistent sadness and hopelessness” since before COVID.

This is a complex knot to untangle. Amid this struggle, there is hope.

High quality after-school and mentoring programs have been proven by empirical research to address these challenges. Entities from the National League of Cities to the Biden Administration have elevated the importance of these programs to address the “long COVID” of education. The evidence demonstrating afterschool’s effects on in-school attendance is vast, as documented in a brief on 21^st CCLC programs.

At our after-school program, Higher Achievement, participants were 52% less likely to be chronically absent than students who attended the host schools last year. When student school attendance improves, academic achievement soon follows. In fact, in two randomized controlled trials by MDRC, Higher Achievement has demonstrated statistically significant results on test scores and report card grades. 

The second study, published in 2020, was funded by our landmark i3 grant from the U.S. Department of Education in 2015 and Venture Philanthropy Partners. MDRC compared grades and test scores between students admitted to Higher Achievement through a random-assignment lottery with those in the comparison (control) group, to remove any selection bias. After two years in the program, admitted students’ GPAs across all four core subjects (reading, math, social studies, science) were statistically significantly higher than those in the comparison group. Moreover, math grades for boys admitted to Higher Achievement were significantly higher after the first year in the program. Test scores were higher for the admitted group, but not statistically significant.

While many organizations struggle to maintain outcomes during expansion, this study demonstrates that Higher Achievement is an exception. We were able to replicate significant academic effects in Baltimore and Richmond, beyond the flagship site of Washington, DC.

The researchers also interviewed staff and volunteers and conducted site visits to produce a robust implementation study. We struggle with many of the same implementation struggles as other out-of-school time programs, and this study led us to make changes to address these challenges. For example:

• Front Line Staff Turnover: Our scholar outcomes are measurably stronger when the director of their Center has served in that role for at least two years. However, we were struggling to retain talent due to burnout and uncompetitive salaries. In 2021, we made a number of changes to increase retention. These changes included: increasing front-line salaries by 20% to match the 50% industry comparables, extending our Afterschool Academy by seven weeks and phasing out our Summer Academy to enable time for planning, mentor recruitment and training, and rest. As a result, our full-time staff retention increased from 71% to 92%. We now refer scholars to programs that specialize in summer learning.
   
• Coordination with Host School: Higher Achievement operates our program in school buildings, and like many of our peers, we sometimes struggle with logistics and communication with our school partners. To address this challenge, our local Directors of Program Operations and front-line Learning Directors share academic and social-emotional data with our host school leadership at least twice per year, and our Executive Directors meet with host school principals every summer, at a minimum, to align our service delivery with the schools’ priorities. As a result, 11 of our 12 host schools have been stable for the past three years.

Our hope is that the findings and actions that we’ve taken as a result of this study are helpful to other afterschool programs, eager to learn and grow. The stakes are high, and we are stronger in partnership.