Afterschool Impacts Database
Evaluation 101
Learn more about the basics of evaluation, as well as how to find an evaluator for your program.
Afterschool Evaluation
View Afterschool Alliance resources, including a glossary of terms used in the database, evaluation-related blogs, webinars and more.
Additional Resources
Our list of evaluation resources from other organizations, including how to collect and work with data.
Afterschool Research
Want to find what we know about afterschool programs more broadly, not just individual programs? Head to our Afterschool Research page!
Texas 21st Century Community Learning Centers, 2014-2015 Evaluation Report
Year Published: 2016
A statewide evaluation of Texas’ 21st Century Community Learning Center (21st CCLC) programs assessing operations, participation, and student achievement, as well as the relationship between each. Data was collected using student and staff surveys, program observations, and testing results. This evaluation found that students regularly participating in Texas 21st CCLC programs saw gains in their math performance, learning behavior, and persistence, as well as reductions in their school day absences and frequency of disciplinary incidents.
Program Name: Texas’ 21st Century Community Learning Center Program or Afterschool Centers on Education (ACE)
Program Description: Texas’ 21st CCLC program, referred to as the Afterschool Centers on Education (ACE), serves high-needs communities across the state, providing afterschool and summer programming to approximately 130,000 students during the 2014-2015 school year. The program is designed to support students’ performance in a variety of areas including, but not limited to, academics, engagement in school, and behavior at school. The Texas Education Agency (TEA) was first awarded the 21st Century Community Learning Center grant from the federal government in 2002.
Scope of the Evaluation: Statewide
Program Type: Afterschool, Before school
Location: Texas
Grade level: Elementary School, Middle School, High School
Program Demographics: All schools served are Title 1 eligible, meaning more than 40 percent of students qualify for Free and Reduced Price Lunch. Sixty-two percent were Caucasian, 20 percent were Black, 2 percent were Asian, and 19 percent were Native American. Sixty-five percent were Hispanic. Twenty-two percent of participants were English language learners and half were students designated “at-risk,” or students who were identified as “being at risk of dropping out of school” by the Texas Education Agency, Texas’ Department of Education. (See Table 16 on pg. 61 of the report for participant and non-participant demographics.)
Program Website: http://tea.texas.gov/Texas_Schools/Support_for_At-Risk_Schools_and_Students/21st_Century_Community_Learning_Centers/
Evaluator: Devaney, E., Naftzger, N., Liu, F., & Sniegowski, S. American Institutes for Research.
Evaluation Methods: Data collected included program staff interviews and surveys, student pre- and post-test surveys measuring key mindsets and behaviors, observations of program activities, and student engagement surveys, and State of Texas Assessments of Academic Readiness (STAAR) results. A comparative analysis (using paired T-tests) was used to compare student academic performance and behavior before and after programming.
Evaluation Type: Quasi-experimental
Summary of Outcomes: During the 2014-2015 academic year, students who had high participation (60+ days) in Texas’ Afterschool Centers on Education (ACE) program showed improved performance in mathematics testing. Students who had a high participation rate in the program were also more likely than their non-participating peers to have on-time grade promotion, see a decrease in disciplinary incidents, as well as a decrease in school day absences. These outcomes related to on-time grade promotion and behavior were found for all grade levels, but they were most significant for students in grades 9-11. For example, among 9th through 11th graders, “participating 60 or more days in ACE programming is associated with being 55 percent more likely to be promoted to the next grade level on time.” Student performance in the State of Texas Assessments of Academic Readiness (STAAR) reading assessment was one area where a positive relationship between frequency of participation and impact was not found.
The evaluation also included findings looking at both the 2011-12 and 2014-15 evaluations and reported that students participating in the ACE program at high levels were more likely to be promoted to the next grade level compared to their non-participating peers. Again, this outcome was seen across all grade levels, but was largest for students in grades 9-11.
Data from the 2013-2014 academic year examining students’ persistence, positive mindset, and “learner behaviors,” or behaviors that support student’s ability to do well in school, such as listening in class, being interested in class, and being prepared for class, found that students participating in the ACE program saw positive gains in all areas.
Associated Evaluation: http://tea.texas.gov/Reports_and_Data/Program_Evaluations/Out-of-School_Learning_Opportunities/Program_Evaluation__Out-of-School_Learning_Opportunities/
Date Added: November 3, 2016
