Implementation outcomes

Image

Why is it important to evaluate how the interventions were implemented and what impact this had on the outcomes?

Given that a practice change or an intervention will not be effective if it is not implemented well, it is important to measure the implementation outcomes that were taken to achieve the desired practice change or interventions (Proctor, 2020). An evaluation of the implementation outcomes can give you information about whether the intervention or practice change was implemented as intended.

Distinguishing implementation effectiveness from intervention effectiveness is critical for implementation success. When a practice or intervention does not get properly used, or not used at all, it is important to determine why the change attempt failed. This can be due to two key factors:

1. Intervention failure – i.e., when the implementation intervention was ineffective. For example, the depth and breadth of the content of an education series were insufficient to support the development of knowledge to achieve the practice change, or the sessions included only minimal time for hands-on practice to develop the skills to achieve the practice change.  
2. Implementation failure: The implementation intervention was effective but was not deployed adequately. For example, the education series effectively addressed the competencies needed to achieve the practice change however the sessions were poorly attended as they were held during vacation season. Or the session was effective but staff were not accepting of the intervention or felt it was not sufficiently applicable to their setting.

Alternatively, when an implementation intervention successfully supports knowledge use and a practice change, it can be helpful to evaluate the strategies that were used and how they were used effectively – i.e., what steps were taken to support the practice change and how they were provided or deployed.  

The evaluation of implementation outcomes answers the following questions:

• Why does or doesn’t the practice change or intervention work in my setting?
• Did we implement what we intended?
• What worked? What didn't? 

Implementation outcomes serve three functions (Khadjesari et al., 2017). They:

• indicate implementation success (e.g., a practice change has been achieved).
• serve as indicators of implementation processes.
• serve as important intermediate outcomes for service and clinical outcomes.

Identifying and measuring implementation outcomes will allow change teams to understand the implementation processes. You can use it to determine the following regarding an implementation intervention:

• whether it occurred or not,
• whether it did or did not get used to support the practice change, or
• whether it did or did not provide the intended benefit to the person/patient.

• There are eight key implementation outcomes that serve as effective indicators of the success of an implementation intervention (Proctor et al., 2011). These eight implementation outcomes are described in the table below and describe how other variables such as staff, the local context, or the setting impact outcomes.   
• Implementation outcomes can be interrelated, and occur in a logical, but not necessarily linear sequence. Outcomes earlier in the sequence can contribute to the implementation strategies that will be used later (Glasgow et al., 2019).

Types of implementation outcomes:  

SOURCE: Proctor et al., 2011.