Evaluate outcomes

Evaluate outcomes

What is the phase ‘Evaluate Outcomes’? 

In the previous phase, you and your change team monitored the adoption of a new practice or intervention that you introduced in a setting. You and your change team can now start to evaluate the outcomes of this intervention or practice change. You can compare outcomes to baseline data collected prior to the intervention or practice change to determine the impact and whether the desired goals have been reached.

Evaluating the impact or outcomes of knowledge identifies what differences – if any – occurred after the knowledge was applied. It considers the outcomes from the levels of the person/patient, the health-care provider, the unit or organization, and/or the broader health system. It is a complex process and should occur on a continuous basis, versus occurring at a single point in time. As such, a systematic, multidimensional and iterative approach is needed to gather data for the stakeholders who will be using the knowledge such as persons/patients, health-care providers, managers or administrators and policymakers (RNAO, 2012).

Data collection post-implementation or post-practice change can vary and oftentimes, this phase may not be applied immediately after implementing the practice change. Instead, you may want to give time for staff and other stakeholders to adjust to the change before you evaluate the outcomes of the intervention or practice change.

Evaluation measures can be categorized into three main types, according to the Donabedian framework (1988). The three types of measures (also called indicators) – structure, process and outcome – are described in the table below. The three types of measures are interrelated. For example, by evaluating structure and process measures, change teams can better understand how these measures contribute to outcomes. 

Table: Categories of measures

Category of measure/indicators Definition Examples
Structure

The attributes required of the health system, organization, or academic institution to support knowledge use and an evidence-based practice change. 

The support or structures that enables the use of knowledge to achieve the change.  
  • Updated policies that reflect the clinical guideline’s recommendations
  • Purchased equipment to support the adherence to the practice change
  • Established staffing models (e.g., roles, levels) to support the practice change  
  • Revised or new documentation forms
  • Developed clinical pathways that align with best practice
  • Educated staff on the use of a new assessment tool
Process

The health care activities provided to, for and with the persons or populations. 

Focuses on person/patient care delivery processes to support process improvement.

  • Provided health teaching or education to persons/families and/or families
  • Integrated screening tools to assess a health risk (e.g., falls risk)
Outcome The effect or impact of the knowledge or practice change on the health status of the persons or populations. Focuses on improving health status outcomes for the person/patient.
  • Change in health status (e.g., the percentage of persons who reported decreased pain according to a validated pain assessment tool)
  • Staff satisfaction with implementing the practice change  

To provide clinical context, examples of measures or indicators as per the three categories of the Donabedian model using the Person- and Family-Centred Care (PFCC) Best Practice Guideline (BPG) are included in the table below.

Table: Examples of indicators from the PFCC BPG

Structure indicators Process indicators Outcome indicators
Availability of integrated system-wide policies consistent with best practices and guideline recommendations for supporting a person- and family-centred care health system. Percentage of care plans outlining the person’s beliefs, values, culture, goals and preferences to ensure personalized health services. Demonstrated cost-effectiveness of person- and family-centred care practices such as continuity of care and availability of decision aids and educational resources.
Availability of system-wide standardized measures to monitor the person’s experience of care. Percentage of persons asked or surveyed by health-care providers about their satisfaction with, and experience of health care. Percentage of persons and/or families reporting increased satisfaction with the experience of care.
Organizations adopt a model of care that promotes coordination of care and continuity of health-care providers.

Percentage of persons who reported that their health providers explained things in a way that is understandable.

 Percentage of boards, councils and committees with persons as advisors to assist with reforming or improving the delivery of health care at the local, provincial and federal government levels.
evaluation

Using indicators to evaluate BPG implementation

For another example of BPG implementation using Donabedian indicators, watch this 2018 Best Practice Champions Network® presentation on the implementation of RNAO's BPG Assessment and Management of Pressure Injuries for the Interprofessional Team in an acute care setting. Carol Williams, RN and BPSO Lead, and Ann Klein, RN Implementation Lead, from Southlake Regional Health Centre present.