Afterschool Impacts Database
Evaluation 101
Learn more about the basics of evaluation, as well as how to find an evaluator for your program.
Afterschool Evaluation
View Afterschool Alliance resources, including a glossary of terms used in the database, evaluation-related blogs, webinars and more.
Additional Resources
Our list of evaluation resources from other organizations, including how to collect and work with data.
Afterschool Research
Want to find what we know about afterschool programs more broadly, not just individual programs? Head to our Afterschool Research page!
Citizen Schools: Achieving High School Graduation: Citizen Schools’ Youth Outcomes in Boston
Year Published: 2010
This comprises the final analysis of an eight-year evaluation of the academic outcomes in high school of former 8th grade participants in the Boston-based Citizen Schools. The study found a number of positive academic outcomes associated with participation in Citizen Schools, as well as positive school day attendance findings. Former Citizen Schools participants were more likely to pass their math and English language arts (ELA) courses, as well as more likely to earn a proficient or advanced rating on their math and ELA Massachusetts Comprehensive Assessment System (MCAS) tests than their non-participating peers. Former participants were also more likely to attend school more often and have a higher rate of on-time graduation than non-participants.
Program Name: Citizen Schools
Program Description: Citizen Schools, founded in 1995, partnered with 32 middle schools in low-income communities to provide a longer learning day to 5,300 students during the 2015-2016 school year. In this extra time, students work with adults who provide hands-on learning experiences in the hopes of giving students the skills, access, and confidence they need to succeed. Areas of focus for the program include “career exposure, high school and college preparation, and academic enrichment.”
Scope of the Evaluation: Local
Program Type: Afterschool
Location: Boston, MA
Community Type: Urban
Grade level: Middle School, High School
Program Demographics: Within each grade (9-12), about three-quarters of participants were African American, between 18 and 21 percent were Hispanic, and the remaining students were either Asian, Native American, or white. Eighty-four percent of participants were low-income, about half of participants were female, and about 20 percent had special needs or disabilities.
Program Website: http://www.citizenschools.org/
Evaluator: Vile, J. D., Arcaira, E., & Reisner, E. R. Policy Studies Associates, Inc.
Evaluation Methods: In this study, participants were matched with non-participants based on five characteristics: gender, race, grade in school, free or reduced-price lunch eligibility, and Massachusetts Comprehensive Assessment System (MCAS) test scores. The evaluation measures included the quality of the high school enrolled in, persistence in high school, on-time grade promotion, school day attendance rates, school suspension rates, English language arts (ELA) and math course grades, ELA and math MCAS test scores, and on-track graduation rates.
Evaluation Type: Quasi-experimental
Summary of Outcomes:
The study found a number of positive academic outcomes associated with participation in Citizen Schools. Former Citizen Schools participants were four times more likely than non-participants to both enroll and complete all four years of high school in a top-tier school, or schools that were “considered to have a high level of educational culture, support, and achievement.” Looking at graduation rates of former Citizen Schools students overall, not only those attending top-tier schools, the study found that former Citizen Schools students had a higher rate of graduating high school on time than their non-participating peers attending Boston Public Schools (BPS) and BPS students overall. Former Citizen Schools students attending non-top-tier schools were close to two times more likely to graduate from high school on time than their non-participating peers who also attended non-top-tier schools. However, among students attending top-tier schools, the finding was not statistically significant, although former Citizen Schools students did have a higher on-time graduation rate than their non-participating peers, the finding was not statistically significant.
Additionally, 9th-11th grade former participants were significantly more likely than non-participants to pass their math courses, and 9th and 12th grade former participants were significantly more likely to pass their English language arts (ELA) courses than non-participants. Previous Citizen Schools participants were also significantly more likely to earn a proficient or advanced ratings on their math and ELA Massachusetts Comprehensive Assessment System (MCAS) tests than their non-participating peers.
Former Citizen Schools participants also had higher rates of school day attendance than non-participants across grades 9-12. The differences in school day attendance ranged from 5 more school days in 10th grade to 13 more school days in 11th grade. However, there was no difference regarding suspension rates when comparing former participants to non-participating students.
Associated Evaluation: http://www.policystudies.com/studies/?id=39
Date Added: November 3, 2016
