Evidence-based practices in education

Photo by Andrei Firtich

The reauthorized national education law, the Every Student Succeeds Act (ESSA) puts an increased emphasis on states and school districts using evidence-based practices in many areas. Under ESSA's Title I, schools designated by their state as “in need of improvement” must create a school improvement plan with at least one activity or program that has a related study showing it meets one of the identified tiers of evidence: strong, moderate or promising (described below).

In addition to this requirement, seven different competitive grants in ESSA will give priority to applicants who meet the top three evidence-based tiers. Although 21st Century Community Learning Centers are formula funded and do not require stringent adherence to evidence based practices, eligible entities are still expected to use best practices to improve student outcomes. Fortunately, there is a substantial and growing evidence base on the positive effects afterschool has on youth development outcomes.

This March, president Obama also signed the Evidence-Based Policy Making Commission Act of 2016. The commission established by the act has designated appointees and is beginning its work. The government’s focus on evidence seems here to stay.

Below is an overview of the evidence tiers specific to ESSA, concluding with resources to find evidence-based programs and develop new studies to add to the field of research.

Here are the four tiers of evidence-based practices in ESSA

• STRONG. Strong studies show positive and meaningful (“statically significant”) results with randomized control trials (RCT). RCTs are viewed as the gold standard of evaluation because they are the best way to determine the effectiveness of a program or policy. RCTs take a large group of people and randomly assign them to the intervention being evaluated (the “treatment” group, in this case, is an afterschool program) or assign them to have no intervention (also known as the “control group”). However, the level of resources (time, money, expertise, etc.) necessary for RCT studies makes them incredibly difficult to implement and limits their availability. This is why it’s important that the law also includes the following tiers of evidence.
• MODERATE. A moderate study will demonstrate a meaningful positive result on student outcomes based on a quasi-experimental study—a study that, like RCTs, has a “control” group and a “treatment” group, but unlike RCTs, it does not include the random assignment to a group.
• PROMISING. A promising study—or correlational study—is one that shows a relationship between an activity or program and student improvements, but it does not prove that the specific activity or program under study was the cause of the change. For example, a correlational study may find that there is a relationship between gains in students’ communication skills and their participation in an afterschool program, but it would not be able to say for certain that participating in the afterschool program caused students to improve their communication skills.
• UNDER EVALUATION. In this final, fourth tier of evidence, the law recognizes that the evidence base is itself a work in progress. The “under evaluation” designation exists for activities and programs that, while yet untested, are rationally derived from research and will be tracked to see what effects they have.

To learn more, the organization Results for America has information on its webpage and their summary on ESSA’s evidence based provisions, and Education Week also provides a recap of the new provisions. As always, let us know what questions, comments and thoughts you still have and what you are doing out in the field.

Where to find evidence-based practices

• The Afterschool Alliance Evaluations Backgrounder: This document can be a useful tool for parents, programs, teachers and superintendents to make the case for afterschool. It includes studies that fall within the three tiers of evidence and connects afterschool programs with positive results in academics, school attendance, behavior and more. Check out the Afterschool Alliance research page for more materials as well.
• What Works Clearinghouse:  (look for a refreshed, user-friendly site coming mid September 2016) A great place to find a few strongly supported interventions, the What Works Clearinghouse (WWC) is located within the Institute for Education Sciences (IES), a federally funded independent education research and statistics agency. WWC uses independent evaluators from contracted research firms who look at the whole body of research on programs, products, practices and policies to determine the level of evidence behind the effectiveness of the interventions. You can look up your math textbook, the program you are using to improve third grade literacy or your curriculum to affect student behavior.  However, it will be hard to find a broad base of interventions here. ESSA only requires that a single research study within any tier show positive evidence, permitting many more interventions to meet the evidence-based standard, whereas WWC bases its tier classification on all studies conducted on an intervention. The difference is explained here.
• Journals: Education-related journals that are focused on out-of-school time, like NOIST Afterschool Matters and the Journal of Expanded Learning Opportunities, are a good place to identify new research in successful program interventions.

How to start a study or get a program evaluated

• Regional educational laboratories: The federal government funds 10 regional laboratories to be the intermediaries between research and practice. They are designed to be a direct resource for you! You can call your REL to help answer a research question, be connected with data, help your school or program organize a study or data, receive technical assistance, and be connected with other partners in your region working on similar issues.
• Researcher-practitioner partnerships in education: State and local school districts in partnership with research institutions can apply for federal grants to investigate a high priority problem for the education agency.
• RCT Yes: This one is for the want-to-be researchers out there. It’s a federal government resource of downloadable software to direct organizations to conduct their own studies and track their own data. A related resource can help train you to use already existing government data sets.
• Universities: Besides partnering to offer high quality afterschool program, universities can help afterschool providers implement and test interventions. Reach out to your local university to begin a conversation about opportunities.

Deep dive into evaluations research

• Poverty Action Lab: Introduction to Evaluations: A good resource to learn what an evaluation study entails:  what an evaluation is, what randomization is, how to implement a study, collect data and consider results.