Meeting the Needs of Underserved Communities with Program Evaluation

During the pandemic, historically marginalized communities experienced higher rates of COVID-19 vaccination hesitancy, often due to concerns about side effects and distrust in healthcare providers.

Five years on from the start of the COVID-19 pandemic, we’ve seen how it exposed and exacerbated existing health inequities in our society. A report by the Alliance for Healthier Communities highlights how health inequities can be remediated through proven interventions and policy changes.

To this end, program evaluation, which evaluates the effectiveness of programs and services, can help provide the framework for meeting the needs of underserved communities. Dr. Pria Nippak, Associate Professor, Academic Coordinator, Interim Director School of Health Services Management, Ted Rogers School of Management, at Toronto Metropolitan University, outlines how program evaluation can be utilized to enhance outreach and trust, particularly within vulnerable communities.

This requires a tailored outreach effort to understand where marginalized communities go and who they engage with to ensure that information and access to program services are made available at each of these contact points. It’s also important to make sure that program staff are representative of the populations they serve or that they have strong connections to these communities and/or cultural experiences to foster trust and increase the likelihood of reaching marginalized individuals.

“As a first step, programs need to identify the marginalized groups they are intending to target,” says Pria. “Once there are clear definitional criteria outlining who is being targeted, specific data points can be developed to track these targeted populations as they engage with the program.

“While programs often aim to serve specific targets in the planning stages, the reality is that when the program becomes operational, they may not always be serving those that they intended. That’s why it’s important to assess who is being reached.”

She went on to say that in order to target marginalized groups, programs should prioritize outreach by actively engaging with those who may not traditionally participate in public services due to social, economic, or historical barriers. This could include conducting outreach through trusted community leaders, local grassroots organizations, or faith-based groups.

While this is one isolated example, there are some general guidelines that program evaluators can follow when setting up programs for successful outreach.

Key Approaches to Effective Program Evaluation

First and foremost, effective program design must always consider the key target population that needs to be served, as Pria points out.

Understanding the Population You’re Serving

“It’s extremely difficult to engage underserved communities and, as such, very detailed and targeted needs assessments that aim to engage underserved communities must be undertaken,” she says. “Once there is a clear understanding through thoughtful and targeted stakeholder engagement regarding the specific needs of those hard-to-reach and underserved community members, program planning efforts can be tailored to their unique needs.”

She adds that this type of planning and engagement must always precede the implementation of the program. Engaging community members in the evaluation process ensures that their voices and lived experiences shape the assessment. Participatory evaluation methods, such as focus groups, storytelling, and community surveys, allow underserved populations to express their concerns and preferences. Co-designing the evaluation process with community members also builds trust and ensures that findings reflect their realities.

Ensuring Cultural Responsiveness

Next, it’s important to ensure that program evaluation is culturally recognized, responsive, and respected with regard to the cultural norms, values, and languages of the communities they serve. This involves using evaluation tools that are linguistically accessible, conducting assessments in culturally familiar settings, and involving community leaders as partners in the process.

To ensure the program is equitable, traditional evaluation metrics may not fully capture the impact of programs on underserved communities. Evaluators should develop indicators that measure equity, such as access to services, improvements in quality of life, and reduction of disparities. Disaggregating data by race, ethnicity, gender, disability, and other factors can reveal hidden inequities and help tailor interventions.

“Several key factors must be considered, such as economic challenges, demographic contributors, geographic location, cultural norms and practices, and systemic barriers,” says Pria. “Understanding how each of these key factors contributes will assist with designing program activities that can meet some of the key needs of underserved communities. Some general considerations toward this effort that are commonly identified focus on accessibility, affordability, and program relevance and potential impact.”

Combining Qualitative and Quantitative Methods

It’s also important to combine qualitative and quantitative methods to provide a more comprehensive understanding of a program’s impact. While quantitative data can track service utilization and outcomes, qualitative insights from interviews and case studies offer deeper perspectives on how programs affect individuals and communities.

“Whenever possible, it’s always important to engage local communities when trying to engage underserved communities,” says Pria. “Community-driven approaches, where local leaders and individuals provide input on design are crucial.”

Assessing Long-Term Impact

Many evaluations focus on short-term outcomes, but long-term tracking is crucial to understanding whether programs create sustainable change. Evaluators should implement follow-up assessments to measure ongoing effects and adjust interventions as needed.

“Continuous quality monitoring of the program’s ability to deliver services in the intended way to the target population along with measurement of whether both the immediate and long term outcomes have been achieved will provide the necessary program level information to allow for any adaptations or changes that may be required,” says Pria. “It’s essential that programs targeted at reaching underserved communities maintain their ability to be responsive and adaptive to ensure efficacy and relevance over time is maintained.”

Informing Advocacy and Policy Influence

Program evaluation should not only assess effectiveness but also inform policy decisions. By translating findings into actionable recommendations, evaluators can advocate for policy changes that better serve underserved communities. Reports and presentations should be designed to engage policymakers, funders, and community stakeholders.

“Program evaluations provide crucial evidence of what is working and what is not, which is vital for making informed decisions about funding and program expansion,” says Pria. “Program evaluation ensures that programs are accountable to the people they serve, sponsors, advocates, community leaders, and other community representatives aligned or associated with the program.”

She adds that positive outcomes may lead to continued or increased funding, while less effective strategies can be revised or redirected to the efforts aimed at improving the program. This information can be collected via participant feedback in satisfaction surveys, outcome data, and cost-effectiveness analyses, which offer transparency and help funders and stakeholders make decisions grounded in results and impact.

“Strong program evaluation and the dissemination of the program successes can assist with the building buy-in from community representatives and leaders, which can further strengthen the partnerships and promotion of the program,” says Pria.

In short, program evaluation is a powerful tool for ensuring that social programs truly meet the needs of underserved communities. By adopting community-centered, culturally responsive, and equity-focused approaches, evaluations can lead to meaningful improvements in service delivery and policy. Ultimately, effective evaluation empowers communities by amplifying their voices and ensuring that interventions lead to lasting positive change.