Evaluating afterschool: Building an evaluation advisory board

By Jason Spector, Senior Research & Evaluation Manager at After-School All-Stars

The Afterschool Alliance is pleased to present the sixth installment of our "Evaluating afterschool" blog series, which answers some of the common questions asked about program evaluation and highlights program evaluation best practices. Be sure to take a look at the first, second, third, fourth, and fifth posts of the series.

When I joined After-School All-Stars (ASAS) in 2014, I represented the sole member of our research and evaluation department. It was a great opportunity to craft a vision, and one that I greeted with excitement, but there was definitely anxiety as well. I was fresh out of grad school—learning how to operate in a national organization while also feeling siloed. To help break down the silos, our leadership encouraged me to develop a board of strategic advisors.

During the last few years, the National Evaluation Advisory Board has played a critical role in helping us grow our department, craft a vision for our work, develop a language and strategy around our program quality assessment, deepen our evidence base, and advance the intentionality of our program model. It’s a resource I highly recommend for organizations who are looking to become more strategic in their work.

If you decide to form your own evaluation advisory board, here are four key ideas to keep in mind:

Build around your needs

Determine your evaluative needs and build your board to meet those needs. Are you looking for specific area content experts? Quantitative or qualitative methodologists? People with internal or external evaluation experience?

Most likely, you have multiple needs, so diversify your board. Leverage your professional network and keep an eye out at conferences for experts who might be a good fit. At ASAS, our evaluation board is currently comprised of six members who have held responsibilities as internal evaluators, university faculty members, leaders of research institutes, and evaluation consultants. These multiple perspectives and experiences are invaluable for contributing expertise across many issue areas. 

Budget for in-person meetings

While in-person meetings can be costly and time-consuming, they are worth it. The ASAS evaluation board meets virtually three times a year, with one additional, two-day in-person meeting.

We have found that in-person meetings bring a heightened level of focus to our work as well as serve as a bonding time for the board. These in-person sessions can also be an opportunity for organizational leadership to improve alignment with the board.

Evaluation boards should supplement, not supplant, internal capacity

Be clear about the goals you expect to achieve and the needs you expect to serve by forming an evaluation advisory board. Typically, an evaluation advisory board will serve as a strategic sounding board, guide projects and decisions, and provide feedback, evaluation resources, and partnership opportunities.

It’s crucial to have clear boundaries on the board’s responsibilities, as well. The more day-to-day work belongs firmly in the hands of your internal team. Just as a governance board can help with fundraising but would not take the place of a grant writer, the same can be said of evaluation boards and internal evaluation staff.

Personality matters

People join evaluation boards not only to contribute to the organizational mission but to broaden their professional network. The experience should be rewarding for every stakeholder and each board member’s personality plays a role in accomplishing that goal. If board members enjoy spending time with each other, this will go a long way towards fostering a collaborative environment—and make requests of board members’ time less burdensome.

Interpersonal board dynamics have the potential to deepen or stagnate your efforts, so definitely pay attention to them as you build and maintain your board!