LIGHTS ON AFTERSCHOOL
Lights On Afterschool
The Afterschool Alliance is pleased to present the eighth installment of our "Evaluating afterschool" blog series, which answers some of the common questions asked about program evaluation and highlights program evaluation best practices. Be sure to take a look at earlier posts of the series, including blogs on how to collect data and how to build your own evaluation advisory board.
Regino Chávez has worked for LA’s BEST since 2007, designing and implementing formative research for program improvement and internal evaluations, as well as monitoring external evaluations of the program. LA’s BEST serves approximately 25,000 children ages 5 to 12 in close to 200 Los Angeles Unified School District elementary schools. A recent evaluation of LA’s BEST by the Center for Research and Evaluation of Student Standards and Testing (CRESST) at University of California, Los Angeles, found that LA’s BEST students with high levels of attendance in the program were 5 percent less likely to drop out of school and 6 percent more likely to graduate from high school on time compared to their peers who did not participate in the program.
During the first year implementation of a financial literacy program in LA’s BEST, we carried out a formative evaluation. Using surveys and focus groups with staff and students, the evaluation sought to identify how to make its implementation more effective. Findings from the assessment led us to revamp the training and scale down the number of activities staff needed to implement to match our program structure with no corresponding loss in the learning of crucial concepts among students and staff.
Jennie Chow, a native speaker of Mandarin, was a young girl new to Tom Sawyer Elementary. Jennie spoke little English and, as a newcomer had few friends in the school. Her mom enrolled her in LA’s BEST, as she needed to be at work until 5:30.
By the second semester, no one had ever heard Jennie speak during day school. Afterschool staff would talk with Jennie and invite her to join structured activities so she could engage with other students. In one particular activity, students drew themselves as superheroes and identified how their powers helped the community. Jennie proudly held up her drawing and loudly said, “Superhero” while smiling at those around her. This was the first time that anyone had heard her speak in English.
Most of us who work in afterschool see successes like Jennie's daily – the perceived disruptive child who is now a leader in sports; or the students in nutrition classes who now ask parents to buy healthier foods. However, one needs more than anecdotal evidence to demonstrate the difference we make in students’ lives. Evaluations will provide helpful information to identify what might need to change.
Are we ready to be evaluated?
In prepping for an evaluation, the first thing to ask is “Is my program ready to be evaluated?” If this is its first year, do not contemplate an impact evaluation as your program and its elements may not yet be fully developed, clearly understood or fully implemented.
A program model consists of activities that staff members implement to meet the goal(s) along with identified resources and assumptions made about what needs to be in place to support goal achievement. The goals define how the program is intentional in what it seeks to do. For example, the goals may focus on developing 21st century learners (a child who asks questions, gathers information, analyzes it, and uses it to problem-solve along with others); enhancing performance in literacy and math; or developing responsible decision-makers. The goals, activities and assumptions one makes in goal achievement are elements of the logic model – the “what” and the “how” one will meet the goals and corresponding outcomes.
Logic models are simply a road map that describe the relationship among inputs, outputs, outcomes and impact. Inputs are the resources you will put in to get your outputs or products of the effort; outcomes are desired results while impact refers to long-term change. Below is the beginning of a logic model for a nutrition and physical activity program designed to support youth in adopting healthy lifestyles.
Goal: Kindergarten and 1st grade youth will develop healthy eating and physical activity habits from exposure to My Plate, the 5 basic food groups, how to put together a balanced meal, and physical activity
Outputs - Products
Impact – Long term change
Happy Healthy Me curriculum
Books tailored to each food group
BEST Fit Fitness Activity Packet
Each lesson has activities to meet a specific objective such as developing familiarity with My Plate
Each lesson will have a cooking activity, a reading activity, and an indoor/outdoor physical activity
4,000 Kindergarten and 1st grade students in the LA’s BEST program are exposed to the My Plate program
4,000 youth taste healthy dishes they make, learn at least 3 breathing exercises, engage in at least 3 indoor/outdoor games, and read 1 book per activity.
Students demonstrate knowledge of what constitutes a healthy meal
Students gain knowledge of how to prepare a healthy meal from items at home.
Students adopt healthier lifestyles
Students become agents for healthy eating and being physically active
Training of Staff
4 Hour training for 200 LA’s BEST staff to support them in implementing the My Plate program
A 4-hour training implemented
200 staff training in the program
Staff gain knowledge of food groups
Staff gain knowledge of how to prepare healthy snacks tailored to youth
Staff demonstrate knowledge of how to prepare heathy meals familiar to students
Staff gain confidence to eat healthier
Staff adopt healthier lifestyles
Logic models can also drafted for organization-wide initiatives. For an example of a logic model more applicable to an organization-wide SEL effort in LA’s BEST, see Figure 2.
Stay tuned for Part II and Part III of this blog. Part II will focus on the two basic types of evaluations programs can perform, how to determine which type is the best fit for a program, and how to determine which research questions to prioritize. Part III delves into the decision-making process of selecting who can conduct the program evaluation and how to find the resources to conduct an evaluation.
A recent survey of program providers found that 3 out of 4 providers report being physically open in at least some capacity, providing critical supports including a safe and supervised space during...
Throughout the entirety of the pandemic, the Afterschool Alliance has been conducting ongoing surveys to track programs’ current situations, concerns, and needs. While programs across the...
We all know afterschool programs are exceedingly important in supporting youth development, especially now during COVID-19, as they provide youth with a safe space to participate in academic...
Many school leaders across the nation are looking to summer as a time to help kids recover, rebound, and accelerate learning. Use our new guide, A Summer for Learning & Recovery, to educate...
Few people would be surprised to hear that children and educators alike have been negatively impacted throughout the pandemic. Children are experiencing everything from learning loss and social and...
By Jenna Doleh, communications assistant at the Wallace Foundation. This blog post was originally published on The Wallace Blog on March 3, 2021.
For the past few years, participation in...
This blog post is adapted from a program spotlight in "How Afterschool is Supporting Learning and Recovery During COVID-19," a June 2020 issue brief from the Afterschool Alliance.
Started in the mid-1990s as a response to more households having both parents working outside the...
BY: Charlotte Steinecke 01/21/21
The Delta region of Arkansas and Mississippi is an area rich in history, natural beauty, and promise, but one that also faces great challenges, with high poverty and limited economic growth. When we look at afterschool in the Delta, we see a similar story, with great potential for afterschool...
BY: Ursula Helminski 01/11/21
5 big takeaways from America After 3PM 2020
Since 2004, America After 3PM has provided an essential view into the state and circumstances of children and young people in the after-school hours. This year’s study, America After 3PM 2020, includes responses from 30,000 U.S. families and...
BY: Charlotte Steinecke 12/08/20
A few weeks ago, we concluded the third in a series of national surveys of afterschool providers in the pandemic, conducted by Edge Research and released by the Afterschool Alliance. Eight months into COVID-19, there are three key trends are shaping the landscape of afterschool programs across the...
BY: Nikki Yamashiro 11/20/20
Photo by Charles DeLoye on Unsplash
A recent webinar from Grad Nation, Building a Grad Nation: Meet the Moment 2020!, marked the release of their report, A Grad Nation 2020: Progress and Challenge in Raising High School Graduation Rates. The annual report discusses recent progress...
BY: Bella DiMarco 11/09/20
Throughout the pandemic, parks and recreation agencies have been essential partners in supporting community wellbeing and reopening efforts. According to a survey conducted by the National Recreation and Park Association (NRPA), more than 190 million U.S. residents visited a park, trail, or other...
BY: Erin Hegarty 09/28/20
Last month, the Afterschool Alliance released a blog digging into regional differences from the COVID-19 program provider survey results. Here, we will explore a few more notable differences, specifically in regards to programs’ summer plans and what the future holds.
BY: Bella DiMarco 08/27/20
We need your help to keep the drumbeat going on the ways in which COVID-19 is impacting afterschool and summer programs and the young people and families who rely on them. By completing this short survey, which should take no more than 10-15 minutes of your time, you are helping to tell your...
BY: Nikki Yamashiro 08/20/20
For the past four months, I have heard stories from the field about the hardships afterschool program staff are seeing in their community and the creative ways they have adapted their programming to support the children and families they serve. From adjusting operations to serve children of...
BY: Nikki Yamashiro 08/04/20
During a media briefing last week, “The School Reopening Dilemma: New Survey Sheds Light on the Role Out-of-School-Time Programs Can Play in Educating Students and Supporting Families This Fall,” it was revealed that 70 percent of afterschool program providers reported serving students...
BY: Nikki Yamashiro 07/31/20
This article was originally published on Graphicacy's medium.com account on January 21, 2021. It is being reposted here with their permission.
The Afterschool Alliance came to Graphicacy looking to create a compelling interactive experience with freshly gathered...
BY: Guest Blogger 01/22/21
In Los Angeles, afterschool programs earn a 95 percent satisfaction rate from parents, and a whopping 90 percent of parents support more public investments in afterschool. But the unmet demand for programs in Los Angeles is soaring, as parents cite lack of programs, cost, and transportation as...
BY: Ursula Helminski 01/05/21
In Atlanta, GA, a high number of families want to enroll their children in afterschool programs – but barriers such as cost stand in the way, pointing to a lack of affordable afterschool options. For every child in an afterschool program in Atlanta, two more are waiting to get in.
BY: Ursula Helminski 12/16/20
In October 2019, The Wallace Foundation and America’s Promise Alliance brought together youth development leaders, researchers, and educators to discuss social and emotional learning (SEL), identify obstacles to teaching SEL, and begin to brainstorm ways to address those challenges. While...
BY: Erin Hegarty 08/28/20
Understanding how our youth are faring is an important aspect of not only reinforcing, but also strengthening the supports that afterschool provides. Two separate reports released this summer delved into the state of child well-being across the country, measuring our nation’s progress over...
BY: Rina Moss 09/03/19
By Femi Vance, researcher at American Institutes for Research, Board President for the California School-Age Consortium, Field Consultant for the National Summer Learning Association, and editor of Changemakers! Equity and Access in Out-of-School Time Programs.
BY: Guest Blogger 08/23/19
The 2020 edition of the Afterschool Alliance’s America After 3PM report – which includes survey response data from more than 10,000 U.S. families of middle school students – reveals that while parents of middle school students recognize the many benefits of afterschool programs,...
BY: Chandler Hall 02/03/21
The Afterschool Alliance’s fourth edition of America After 3PM—which provides a detailed accounting of the circumstances and conditions of U.S. children during the hours of 3 to 6 p.m. and compares afterschool program participation and unmet demand statistics for 2020 with results from...
BY: Charlotte Steinecke 01/13/21
While recent reports, fact sheets, and infographics detail key findings from the America After 3PM household survey of more than 31,000 parents of school-aged children, the America After 3PM dashboard was created to provide broader access to the survey’s rich data set, which includes...
BY: Nikki Yamashiro 01/12/21
In a Forum for Youth Investment webinar, Shaping Summertime Experiences: Aligning Practice to Recommendations Recap and Resources, panelists Dr. Deborah Moroney, managing director at American Institutes for Research, and Chris Smith, president and executive director of Boston After School &...
BY: Bella DiMarco 05/14/20
A new report released on October 20 by the Food Research and Action Center highlights some good news about the federal Afterschool Meals Program. According to the report, Afterschool Suppers: A Snapshot of Participation, more than 1.4 million low-income children participated in the Afterschool...
BY: Alexis Steines 11/02/20
This report was released by Mission: Readiness, one of the five organizations that comprise Council for a Strong America, which aims to strengthen national security by ensuring kids stay in school, stay fit, and stay out of trouble.
A recent Council for a Strong America (CSA) Mission:...
BY: Bella DiMarco 08/06/20
By Clarissa Hayes. Originally published on October 25,2019 as a blog on FRAC Chat, the blog of the Food Research & Action Center.
National participation in the Afterschool Nutrition Programs — which are key to closing the hunger gap that exists afterschool for many children...
BY: Guest Blogger 11/01/19
In Colorado, the Montrose Recreation District’s Summer Enrichment Program creates a welcoming environment to help students feel safe to share their feelings. At Jóvenes de Puerto Rico en Riesgo, Inc. (JPRR), also known as Puerto Rico Youth at Risk (PRYR), one-on-one...
BY: Nikki Yamashiro 09/24/19
As many in the out-of-school-time field know, the summer months have the potential to widen existing learning inequities contributing to the achievement gap between low-income students and their higher-income peers, or serve as a valuable space to combat this exacerbation and not only prevent...
BY: Erin Hegarty 12/21/20
The Urban Institute recently released a working paper entitled, “Meeting the School-Age Child Care Needs of Working Parents Facing COVID-19 Distance Learning,” that addresses major challenges that both families and afterschool and childcare providers are currently facing, and lays out a...
BY: Bella DiMarco 09/01/20
A brief recently released by the Data Quality Campaign (DQC), “Research Partnerships are Key to Improving Practice,” highlights research partnerships and their importance in improving both policy and practice in education.
As the brief states, research partnerships—diverse...
BY: Bella DiMarco 04/09/20
For close to 20 years, we at the Afterschool Alliance have been dedicated to the cause of expanding afterschool opportunities because we know the life changing impact programs can have on children, on families, and on communities. Today, Fight Crime: Invest in Kids released a new report that...
BY: Jodi Grant 10/16/19
A new report released by Save the Children provides the first-ever ranking of counties in the United States where children are most and least prioritized. The report, aptly entitled, “The Land of Inopportunity: Closing the Childhood Equity Gap for America’s Kids,” shows that where...
BY: Erin Hegarty 06/26/20
COVID-19 has created a number of challenges for millions of families and exacerbated existing inequities, leaving certain groups more vulnerable than others to the harmful effects of the pandemic. This week, we are turning our attention toward celebrating the ways that afterschool programs across...
BY: Erin Hegarty 05/26/20
Arts education programs can play a significant role in preventing youth from getting involved with the juvenile justice system and supporting the mental wellbeing and successful transition out of the system for those already involved, according to a new report released by the Education Commission...
BY: Erin Hegarty 05/22/20
This blog post was adapted from a program highlight featured in Fight Crime, Invest in Kids’ report, “From Risk to Opportunity: Afterschool Programs Keep Kids Safe.” Read the full report here. This is the first in a series of blog posts that will highlight the different ways that...
BY: Erin Hegarty 12/19/19