Why is the phase ‘Evaluate Outcomes’ important?

Evaluate outcomes

The goal of this phase of the KTA action cycle is to determine the impact of the intervention or practice change, how well the change was adopted in the setting and the impact of the intervention or practice change on outcomes (RNAO, 2012; Proctor, 2020; Strauss et al., 2013).

Evaluation can be divided into two main types of outcomes (listed below). Choosing which outcomes to evaluate will depend on the change you are leading and/or the problem you are looking to address.

  1. Intervention outcomes
  2. Implementation outcomes

Why is it important to evaluate intervention outcomes?

Evaluating intervention outcomes provides information about whether the intervention (i.e., the strategies or activities used to support knowledge use) or practice change worked in your setting. The typical goals of implementing a practice change are to improve service delivery, the care provided and/or the quality of life of persons/patients. Evaluating intervention outcomes may give you information on the effects of the practice change on the stakeholders (e.g., persons/patients, families/caregivers, and health-care providers) by assessing the progress made to achieve the knowledge use and practice change (Centres for Disease Control and Prevention, 2020).

Intervention outcomes

Evaluating intervention outcomes answers the following questions (Centres for Disease Control and Prevention, 2020):

  • Did the implementation of the intervention or practice change result in changes in knowledge, attitudes and skills among health-care providers?
  • Does the intervention or practice work in my environment?
  • Did the practice change or intervention improve the care delivered?
  • Did the practice change or intervention improve the persons’/patients’ overall health and wellbeing?
  • Do the benefits of the practice change or intervention justify a continued allocation of resources?
  • Did the practice change or intervention have any unintended (beneficial or adverse) effects on persons/patients or health-care providers?

Intervention outcomes refer to the success or effectiveness of the intervention. Typically, data are collected before and after implementation to assess any changes in outcomes, as well as the sustainability of outcomes of the implementation.

Improvements in clinical and service outcomes are the ultimate goals of care when implementing a practice change or intervention. Change in outcomes can occur at micro, meso and/or macro levels involving the person/patient, healthcare provider, organization and the health system. Examples of measures of the impact of knowledge use at these four levels are described in the table below.

Level  Description Examples of measure Strategy for data collection

Micro level - Person/patient caregiver/family

 Impact of using or applying the knowledge on patients/ caregivers/ families 
  • Changes in health status (e.g., reduced falls, increased smoking quit attempts) 
  • Health-related quality of life
  • Satisfaction and experiences with care
  • Level of involvement in care planning
  • Standardized clinical assessments
  • Health records
  • Questionnaires
  • Interviews
  • Observations
  • Chart audits
Micro level - health-care providers Impact of using or applying the knowledge on health-care providers
  • Satisfaction with practice
  • Knowledge of the practice change
  • Attitudes and behaviours toward the practice change
  • Questionnaires
  • Interviews
  • Observations
  • Chart audits
Meso level - Organization Impact of using or applying the knowledge on the organization
  • Rate of adverse events
  • Readmissions
  • Wait times
  • Access to specialist/primary/ urgent care
  • Interviews with senior leaders and staff 
  • Chart audits
  • Clinical databases
Macro level - System/Society Impact of using or applying the knowledge on the health system or society
  • Use of medical health records 
  • Accessibility and timeliness of care
  • Costs 
  • Administrative databases
  • Policies
  • Regulations

SOURCE: Adapted from Strauss et al., 2013.