Skills or Awareness Gap? Why Program Evaluators are More Important Than Ever

When organizations look for someone to manage their books, they hire an accountant. When they look for someone to draft up a letter of incorporation, they hire a lawyer. But what about when they want to find out how to improve their programs? Or when fundraisers want to know if they’ve invested wisely? Or when policymakers want to identify high-impact solutions to address societal challenges? That’s where the need for program evaluation comes in. The challenge is, there’s still a lack of awareness about the program evaluation process.

Dr. Chi Yan Lam, who is an evaluation scholar and a credentialed evaluator, says there’s a growing demand for program evaluators across the public, not-for-profit, and private sectors. “The need for evaluation remains persistent, if not growing, in the past decade here in Canada,” he says, noting the examples above of when an organization or government might need the services of an evaluator. “Evaluators are perfectly positioned to respond to those needs.”

The lack of awareness about the profession combined with organizations’ failure to see it as integral to their operation and therefore not allocating budget towards it confounds the issue.

“The rule-of-thumb is that organizations should be spending five to 10 percent of their program budget on evaluation,” says Dr. Lam. “Many end up stunting their own growth and impact when they underspend on evaluation. In that regard, evaluation should be regarded more so as an investment, with an expected return on it, and less so of an expense.”

As mentioned in a previous article on program evaluation, the demand for program evaluators is being driven by increased accountability and transparency in terms of government and nonprofit sector’s need for accountability. On top of that, there’s also a greater emphasis on the importance of transparency in program funding and outcomes.

Raising awareness of the need for evaluation

So perhaps it’s not so much that there’s a skills gap in the market for program evaluation but greater awareness about the growing demand for skilled evaluators, as Dr. Fiona Deller, President of Fiona Deller and Associates, who helped design the Postgraduate Certificate in Program Evaluation explains.

“There’s a trend towards requiring evaluation to be a part of the funding envelope,” says Dr. Deller. “It used to be that governments, government agencies, and philanthropic organizations would fund programs and program development and they would require financial compliance audits. What you’re seeing more and more over time is the requirement to evaluate the effectiveness of programs and their continuous improvement.

“A lot of that is being built into government funding now and what happens is you need more evaluators to do that work. You also need people within government and government agencies as well as community not-for-profits to understand evaluation even if they aren’t doing the evaluation themselves. They need to understand it well enough to manage and oversee evaluators.”

While the bulk of Dr. Deller’s experience is in the public sector, she has gotten the sense that the trend toward more transparency is also evident in the private sector.

“I would say the difference really is that in the private sector, evaluation tends to be much more outcomes and KPI driven, and financial compliance. Whereas in the public sector, you’re seeing a trend towards more community-based, developmental, and participatory evaluation like working better with vulnerable populations and community-based organizations.”

In order to give organizations, whether they’re public or private, the skills they need to keep up with program evaluation, Dr. Deller says academic institutions are just catching up.

“Academia is trying to meet the demand of people wanting to be trained evaluators,” she says, adding that program evaluation skills are being built into public policy programs.

“However, a lot of the evaluators I know come from an economic background where you have a lot of research methodology but not necessarily program evaluation skills.”

Dr. Lam emphasizes that practical program evaluation skills are important for organizations.

“It allows an organization to better understand their work and results in a rigorous and defensible manner,” he says. “In evaluating programs, we gain a big picture that doesn’t rely on anecdotes, opinions, intuition, or spotty, incomplete information.”

The need for evaluators will only continue to grow as the issues we seek to address become increasingly complex and the apparent solutions have been exhausted, Dr. Lam adds.

“There’s a greater need to develop innovative and more responsive policy and program solutions,” he says. “In turn, we need better ways to figure out if our actions are leading to our intended results – evaluation helps with that.”

So how does this benefit organizations and governments? Evaluation contributes to program and policy implementation by allowing decision-makers to fine-tune their program model, design innovative solutions, and adapt to changing contexts. Dr. Lam also points out that it allows them to scale up program models to different communities and their needs, demonstrate value for money to funders, and identify what works about their programs.

There’s never been a more exciting time to consider a career in program evaluation, as it’s constantly evolving in a world where data is tied to decision-making more than ever.