Archives:

Select:

Evaluating afterschool: The evaluation basics, part I

READ FULL STORY

Evaluating afterschool: The evaluation basics, part I

The Afterschool Alliance is pleased to present the eighth installment of our "Evaluating afterschool" blog series, which answers some of the common questions asked about program evaluation and highlights program evaluation best practices. Be sure to take a look at earlier posts of the series, including blogs on how to collect data and how to build your own evaluation advisory board.

Regino Chávez has worked for LA’s BEST since 2007, designing and implementing formative research for program improvement and internal evaluations, as well as monitoring external evaluations of the program. LA’s BEST serves approximately 25,000 children ages 5 to 12 in close to 200 Los Angeles Unified School District elementary schools. A recent evaluation of LA’s BEST by the Center for Research and Evaluation of Student Standards and Testing (CRESST) at University of California, Los Angeles, found that LA’s BEST students with high levels of attendance in the program were 5 percent less likely to drop out of school and 6 percent more likely to graduate from high school on time compared to their peers who did not participate in the program.

During the first year implementation of a financial literacy program in LA’s BEST, we carried out a formative evaluation. Using surveys and focus groups with staff and students, the evaluation sought to identify how to make its implementation more effective. Findings from the assessment led us to revamp the training and scale down the number of activities staff needed to implement to match our program structure with no corresponding loss in the learning of crucial concepts among students and staff.

Why evaluate?

Jennie Chow, a native speaker of Mandarin, was a young girl new to Tom Sawyer Elementary. Jennie spoke little English and, as a newcomer had few friends in the school. Her mom enrolled her in LA’s BEST, as she needed to be at work until 5:30.

By the second semester, no one had ever heard Jennie speak during day school. Afterschool staff would talk with Jennie and invite her to join structured activities so she could engage with other students. In one particular activity, students drew themselves as superheroes and identified how their powers helped the community. Jennie proudly held up her drawing and loudly said, “Superhero” while smiling at those around her. This was the first time that anyone had heard her speak in English.

Most of us who work in afterschool see successes like Jennie's daily – the perceived disruptive child who is now a leader in sports; or the students in nutrition classes who now ask parents to buy healthier foods. However, one needs more than anecdotal evidence to demonstrate the difference we make in students’ lives. Evaluations will provide helpful information to identify what might need to change.

Are we ready to be evaluated?

In prepping for an evaluation, the first thing to ask is “Is my program ready to be evaluated?” If this is its first year, do not contemplate an impact evaluation as your program and its elements may not yet be fully developed, clearly understood or fully implemented.

A program model consists of activities that staff members implement to meet the goal(s) along with identified resources and assumptions made about what needs to be in place to support goal achievement. The goals define how the program is intentional in what it seeks to do. For example, the goals may focus on developing 21st century learners (a child who asks questions, gathers information, analyzes it, and uses it to problem-solve along with others); enhancing performance in literacy and math; or developing responsible decision-makers. The goals, activities and assumptions one makes in goal achievement are elements of the logic model – the “what” and the “how” one will meet the goals and corresponding outcomes.

Logic models are simply a road map that describe the relationship among inputs, outputs, outcomes and impact. Inputs are the resources you will put in to get your outputs or products of the effort; outcomes are desired results while impact refers to long-term change. Below is the beginning of a logic model for a nutrition and physical activity program designed to support youth in adopting healthy lifestyles.

 Goal: Kindergarten and 1st grade youth will develop healthy eating and physical activity habits from exposure to My Plate, the 5 basic food groups, how to put together a balanced meal, and physical activity

Inputs –Resources

Activities

Outputs - Products

Outcomes

Desired Results

Impact – Long term change

My Plate

 

Happy Healthy Me curriculum

 

Books tailored to each food group

 

BEST Fit Fitness Activity Packet

6-week lessons

 

Each lesson has activities to meet a specific objective such as developing familiarity with My Plate

 

Each lesson will have a cooking activity, a reading activity, and an indoor/outdoor physical activity

4,000 Kindergarten and 1st grade students in the LA’s BEST program are exposed to the My Plate program 

 

4,000 youth taste healthy dishes they make, learn at least 3 breathing exercises, engage in at least 3 indoor/outdoor games, and read 1 book per activity.

Students demonstrate knowledge of what constitutes a healthy meal

 

Students gain knowledge of how to prepare a healthy meal from items at home.

Students adopt healthier lifestyles

 

Students become agents for healthy eating and being physically active

Training of Staff

 

4 Hour training for 200 LA’s BEST staff to support them in implementing the My Plate program

A 4-hour training implemented

 

200 staff training in the program

Staff gain knowledge of food groups

 

Staff gain knowledge of how to prepare healthy snacks tailored to youth

 

Staff demonstrate knowledge of how to prepare heathy meals familiar to students

Staff gain confidence to eat healthier

 

Staff adopt healthier lifestyles

 

Logic models can also drafted for organization-wide initiatives. For an example of a logic model more applicable to an organization-wide SEL effort in LA’s BEST, see Figure 2.

Stay tuned for Part II and Part III of this blog. Part II will focus on the two basic types of evaluations programs can perform, how to determine which type is the best fit for a program, and how to determine which research questions to prioritize. Part III delves into the decision-making process of selecting who can conduct the program evaluation and how to find the resources to conduct an evaluation.

Help us tell the story of afterschool AND win a chance at a cash prize

We need your help to keep the drumbeat going on the ways in which COVID-19 is impacting afterschool and summer programs and the young people and families who rely on them. By completing this short survey, which should take no more than 10-15 minutes of your time, you are helping to tell your...

BY: Nikki Yamashiro      08/20/20

Survey: 70% of programs are still serving kids during COVID

For the past four months, I have heard stories from the field about the hardships afterschool program staff are seeing in their community and the creative ways they have adapted their programming to support the children and families they serve. From adjusting operations to serve children of...

BY: Nikki Yamashiro      08/04/20

New issue brief: How afterschool is supporting learning and recovery during COVID-19

During a media briefing last week, “The School Reopening Dilemma: New Survey Sheds Light on the Role Out-of-School-Time Programs Can Play in Educating Students and Supporting Families This Fall,” it was revealed that 70 percent of afterschool program providers reported serving students...

BY: Nikki Yamashiro      07/31/20

5 regional differences when looking at the impact of COVID-19 on afterschool

The significant challenges programs, families, and communities face as a result of the coronavirus is evident. As the situation continues to be in constant flux, the Afterschool Alliance is conducting ongoing surveys to track and monitor the evolving needs of afterschool and summer programs as the...

BY: Nikki Yamashiro      07/30/20

4 big questions from the Advancing Equity through Summer and School Year Partnerships webinar

Earlier this month, the Forum for Youth Investment hosted a webinar as part of their series on the new research from the National Academies of Sciences, “Shaping Summertime Experiences: Opportunities to Promote Healthy Development and Well-Being for Children and Youth”, made possible by...

BY: Bella DiMarco      07/10/20

Afterschool will be critical to recovery—tell us what your community needs!

COVID-19 has impacted afterschool and summer programs across the country, with many programs having to switch to virtual programming or closing down entirely. Many young people and their families rely on these programs every day for a variety of supports, including meals, social and emotional...

BY: Nikki Yamashiro      06/22/20

Recognizing the role of afterschool and summer programs and systems in reopening and rebuilding

By Deb Moroney and Jessy Newman, American Institutes for Research (AIR). As we look ahead to another month of the COVID-19 pandemic in the United States, education, afterschool, summer, and other community-based organizations are working to address the challenges created by unexpected...

BY: Guest Blogger      05/26/20

Recommendations from BellXcell to combat negative effects of prolonged school closures

BellXcell and the Sperling Center for Research and Innovation (SCRI) recently released a brief that discusses the impact of COVID-19 on learning gaps for students. Due to these unprecedented school closures, millions of students could be left without schooling for upwards of five to six months,...

BY: Bella DiMarco      05/20/20

Program spotlight: ourBRIDGE for KIDS working with immigrant and refugee families amidst COVID-19

Located in Charlotte, North Carolina, ourBRIDGE for KIDS (ourBRIDGE) is an afterschool program that works with immigrant and refugee students new to the United States. The program—whose three core pillars are academic support, trauma-informed care, and cultural pride—serves...

BY: Nikki Yamashiro      05/15/20

While collaboration between youth-serving organizations continues, gaps in service still exist

In a Forum for Youth Investment webinar, Shaping Summertime Experiences: Aligning Practice to Recommendations Recap and Resources, panelists Dr. Deborah Moroney, managing director at American Institutes for Research, and Chris Smith, president and executive director of Boston After School &...

BY: Bella DiMarco      05/14/20

Council for a Strong America report depicts state-level campaign to combat childhood obesity

This report was released by Mission: Readiness, one of the five organizations that comprise Council for a Strong America, which aims to strengthen national security by ensuring kids stay in school, stay fit, and stay out of trouble. A recent Council for a Strong America (CSA) Mission:...

BY: Bella DiMarco      08/06/20

Survey: 70% of programs are still serving kids during COVID

For the past four months, I have heard stories from the field about the hardships afterschool program staff are seeing in their community and the creative ways they have adapted their programming to support the children and families they serve. From adjusting operations to serve children of...

BY: Nikki Yamashiro      08/04/20

New issue brief: How afterschool is supporting learning and recovery during COVID-19

During a media briefing last week, “The School Reopening Dilemma: New Survey Sheds Light on the Role Out-of-School-Time Programs Can Play in Educating Students and Supporting Families This Fall,” it was revealed that 70 percent of afterschool program providers reported serving students...

BY: Nikki Yamashiro      07/31/20

More children are getting afterschool meals that boost their health and learning

By Clarissa Hayes. Originally published on October 25,2019 as a blog on FRAC Chat, the blog of the Food Research & Action Center.  National participation in the Afterschool Nutrition Programs — which are key to closing the hunger gap that exists afterschool for many children...

BY: Guest Blogger      11/01/19

New issue brief: Afterschool fostering protective factors

In Colorado, the Montrose Recreation District’s Summer Enrichment Program creates a welcoming environment to help students feel safe to share their feelings. At Jóvenes de Puerto Rico en Riesgo, Inc. (JPRR), also known as Puerto Rico Youth at Risk (PRYR), one-on-one...

BY: Nikki Yamashiro      09/24/19

New report: Community programs important in preventing childhood obesity

A recent article, published in the current issue of Pediatric Obesity, “The prevalence of community programmes and policies to prevent childhood obesity in a diverse sample of US communities: the Healthy Communities Study,” examines the influence of community programs and policies...

BY: Marisa Paipongna      11/26/18

Afterschool suppers and afterschool programs key to student success

By Clarissa Hayes, child nutrition policy analyst at the Food Research and Action Center (FRAC). Research shows that afterschool programs provide measurable benefits to students academically, socially, and behaviorally, and serve as an important resource for families and communities. In addition...

BY: Guest Blogger      10/12/18

New issue brief: A big-picture approach to wellness

From empowering students to take charge of their health to training staff to model healthy behaviors, afterschool and summer learning programs across the country are offering safe and supportive environments that promote young people’s healthy eating and physical activity, as well as build...

BY: Nikki Yamashiro      09/07/18

Closing the Childhood Equity Gap: New report provides a data-driven look into the prioritization of children at the state and local levels

A new report released by Save the Children provides the first-ever ranking of counties in the United States where children are most and least prioritized. The report, aptly entitled, “The Land of Inopportunity: Closing the Childhood Equity Gap for America’s Kids,” shows that where...

BY: Erin Hegarty      06/26/20

New issue brief: Afterschool and the juvenile justice system

COVID-19 has created a number of challenges for millions of families and exacerbated existing inequities, leaving certain groups more vulnerable than others to the harmful effects of the pandemic. This week, we are turning our attention toward celebrating the ways that afterschool programs across...

BY: Erin Hegarty      05/26/20

Recommendations from BellXcell to combat negative effects of prolonged school closures

BellXcell and the Sperling Center for Research and Innovation (SCRI) recently released a brief that discusses the impact of COVID-19 on learning gaps for students. Due to these unprecedented school closures, millions of students could be left without schooling for upwards of five to six months,...

BY: Bella DiMarco      05/20/20

What resources do staff need to support youth mental health and wellness?

By Selena Levy, director of Training at California School-Age Consortium (CalSAC).  May is Mental Health Awareness Month, and we’re highlighting the ways afterschool programs support children and young people as they face challenges including trauma, ACEs, substance use, and beyond....

BY: Guest Blogger      05/23/19

5 steps to effectively implement SEL initiatives in your community

Awareness of the importance of social and emotional learning is beginning to be reflected in policy measures. Practitioners and policymakers alike are continually recognizing that a whole-child approach is necessary for children to be well-positioned to succeed in their academic, personal, and...

BY: Erin Hegarty      02/08/19

Afterschool, community partners featured in Aspen Institute’s "Nation At Hope" report

Today’s youth must navigate a complex, economically competitive, and globally connected world. In efforts to help young people thrive, educators, parents, and leaders have historically focused on academic improvement as the key target for future success. But that’s not the whole...

BY: Dan Gilbert      01/15/19

New issue brief: A big-picture approach to wellness

From empowering students to take charge of their health to training staff to model healthy behaviors, afterschool and summer learning programs across the country are offering safe and supportive environments that promote young people’s healthy eating and physical activity, as well as build...

BY: Nikki Yamashiro      09/07/18

Data on child well-being: How does the nation measure up?

Understanding how our youth are faring is an important aspect of not only reinforcing, but also strengthening the supports that afterschool provides. Two separate reports released this summer delved into the state of child well-being across the country, measuring our nation’s progress over...

BY: Rina Moss      09/03/19

To elevate equity, diversity, and inclusion, go deeper into your quality standards

By Femi Vance, researcher at American Institutes for Research, Board President for the California School-Age Consortium, Field Consultant for the National Summer Learning Association, and editor of Changemakers! Equity and Access in Out-of-School Time Programs. Implementing high-quality...

BY: Guest Blogger      08/23/19

Evaluating afterschool: The evaluation basics, part III

The Afterschool Alliance is pleased to present the tenth installment of our "Evaluating afterschool" blog series, which answers some of the common questions asked about program evaluation and highlights program evaluation best practices. Be sure to take a look at earlier posts of the...

BY: Guest Blogger      07/19/19

Evaluating afterschool: The evaluation basics, part I

The Afterschool Alliance is pleased to present the eighth installment of our "Evaluating afterschool" blog series, which answers some of the common questions asked about program evaluation and highlights program evaluation best practices. Be sure to take a look at earlier posts of...

BY: Guest Blogger      07/01/19

New report on building data-driven afterschool systems within cities

A data-centric approach has potential to generate actionable information at the systematic and programmatic levels and ultimately to improve the quality and access to afterschool. The Wallace Foundation commissioned a study in partnership with Chapin Hall at the University of Chicago and the Gamse...

BY: Theresa Carr      06/03/19

Data sharing with afterschool leads to student success

With more than 10 million children participating in afterschool programming, the hours between 3 and 6 p.m. hold crucial potential for academic, social, and emotional development that extends well beyond the school day. So how can we ensure that more students have access to high quality afterschool...

BY: Rina Moss      11/13/18

Closing the Childhood Equity Gap: New report provides a data-driven look into the prioritization of children at the state and local levels

A new report released by Save the Children provides the first-ever ranking of counties in the United States where children are most and least prioritized. The report, aptly entitled, “The Land of Inopportunity: Closing the Childhood Equity Gap for America’s Kids,” shows that where...

BY: Erin Hegarty      06/26/20

New issue brief: Afterschool and the juvenile justice system

COVID-19 has created a number of challenges for millions of families and exacerbated existing inequities, leaving certain groups more vulnerable than others to the harmful effects of the pandemic. This week, we are turning our attention toward celebrating the ways that afterschool programs across...

BY: Erin Hegarty      05/26/20

Engaging the Arts Across the Juvenile Justice System: New report outlines opportunities for engagement

Arts education programs can play a significant role in preventing youth from getting involved with the juvenile justice system and supporting the mental wellbeing and successful transition out of the system for those already involved, according to a new report released by the Education Commission...

BY: Erin Hegarty      05/22/20

The Power of Mentorship: Adult Friends for Youth makes a difference in redirecting youths’ lives

This blog post was adapted from a program highlight featured in Fight Crime, Invest in Kids’ report, “From Risk to Opportunity: Afterschool Programs Keep Kids Safe.” Read the full report here. This is the first in a series of blog posts that will highlight the different ways that...

BY: Erin Hegarty      12/19/19

Afterschool turns the peak time for crime into a time of opportunity for young people

For close to 20 years, we at the Afterschool Alliance have been dedicated to the cause of expanding afterschool opportunities because we know the life changing impact programs can have on children, on families, and on communities.  Today, Fight Crime: Invest in Kids released a new report that...

BY: Jodi Grant      10/16/19

New issue brief: Afterschool fostering protective factors

In Colorado, the Montrose Recreation District’s Summer Enrichment Program creates a welcoming environment to help students feel safe to share their feelings. At Jóvenes de Puerto Rico en Riesgo, Inc. (JPRR), also known as Puerto Rico Youth at Risk (PRYR), one-on-one...

BY: Nikki Yamashiro      09/24/19

Closing the Childhood Equity Gap: New report provides a data-driven look into the prioritization of children at the state and local levels

A new report released by Save the Children provides the first-ever ranking of counties in the United States where children are most and least prioritized. The report, aptly entitled, “The Land of Inopportunity: Closing the Childhood Equity Gap for America’s Kids,” shows that where...

BY: Erin Hegarty      06/26/20

New issue brief: Afterschool and the juvenile justice system

COVID-19 has created a number of challenges for millions of families and exacerbated existing inequities, leaving certain groups more vulnerable than others to the harmful effects of the pandemic. This week, we are turning our attention toward celebrating the ways that afterschool programs across...

BY: Erin Hegarty      05/26/20

Recommendations from BellXcell to combat negative effects of prolonged school closures

BellXcell and the Sperling Center for Research and Innovation (SCRI) recently released a brief that discusses the impact of COVID-19 on learning gaps for students. Due to these unprecedented school closures, millions of students could be left without schooling for upwards of five to six months,...

BY: Bella DiMarco      05/20/20

New issue brief: Afterschool fostering protective factors

In Colorado, the Montrose Recreation District’s Summer Enrichment Program creates a welcoming environment to help students feel safe to share their feelings. At Jóvenes de Puerto Rico en Riesgo, Inc. (JPRR), also known as Puerto Rico Youth at Risk (PRYR), one-on-one...

BY: Nikki Yamashiro      09/24/19

What resources do staff need to support youth mental health and wellness?

By Selena Levy, director of Training at California School-Age Consortium (CalSAC).  May is Mental Health Awareness Month, and we’re highlighting the ways afterschool programs support children and young people as they face challenges including trauma, ACEs, substance use, and beyond....

BY: Guest Blogger      05/23/19

4 big questions from the Advancing Equity through Summer and School Year Partnerships webinar

Earlier this month, the Forum for Youth Investment hosted a webinar as part of their series on the new research from the National Academies of Sciences, “Shaping Summertime Experiences: Opportunities to Promote Healthy Development and Well-Being for Children and Youth”, made possible by...

BY: Bella DiMarco      07/10/20

Recognizing the role of afterschool and summer programs and systems in reopening and rebuilding

By Deb Moroney and Jessy Newman, American Institutes for Research (AIR). As we look ahead to another month of the COVID-19 pandemic in the United States, education, afterschool, summer, and other community-based organizations are working to address the challenges created by unexpected...

BY: Guest Blogger      05/26/20

Making the case for summer learning: A new report provides evidence and recommendations

A new report by the National Academies of Sciences, Engineering, and Medicine, sponsored by the Robert Wood Johnson Foundation and the Wallace Foundation, takes a deep dive into the summer months, the opportunities they can provide for youth, and the best ways to connect with those youth who have...

BY: Erin Hegarty      12/02/19

Two new resources to kick-off your summer learning plans!

While the New Year has just begun and we are firmly in winter’s grasp, now is actually a great time to start planning for your summer programs. To assist in this process, the Wallace Foundation has released two brand new resources to help organize your thoughts and kick-off your first steps...

BY: Erin Hegarty      01/14/19