Monitor knowledge use

Following the rollout of implementation interventions, change teams need to determine whether the practice change and all of its components have been achieved and, related, if the interventions used to support the practice change(s) have been effective. Learn more in this section focused on the KTA action cycle phase "Monitor knowledge use". 

After the new intervention or practice is implemented into your local context, uptake of this new practice or intervention should be monitored over time (RNAO, 2002; Graham et al., 2006). Monitoring involves the continuous process of collecting and analyzing data to compare how well an intervention or practice is being implemented against intended goals. It includes routine collection, review, and analysis of data. Data sources may include progress notes, clinical databases, interviews with health professionals who are involved in the practice change, questionnaires or other data indicators that will provide the relevant information.

Monitoring knowledge use answers the following questions:

• Are activities being implemented as planned?
• Is the knowledge, practice, or intervention being used?
• Are the knowledge, attitudes and beliefs of target users being changed because of this new knowledge, practice, or intervention?

Monitoring the implementation and uptake of interventions and programs

Your unit or organization may be implementing a full intervention or a program, as well as information intended to change stakeholders’ attitudes and behaviours (conceptual), clinical practice (instrumental) or policy (persuasive) (Strauss et al., 2013). 

Monitoring the uptake of interventions and programs is crucial in understanding whether the resources allocated to implement these interventions and programs are worthwhile, which ultimately allows you to reach your desired goal of changing clinical outcomes.

Monitoring implementation using process indicators

Process indicators track the progress of the intervention or the practice change that has been implemented. They help to answer the question, “Are activities being implemented as planned?” Some examples of process indicators are:

• the number of training sessions held for health professionals
• the number of outreach activities conducted by the change team
• percentage of health professionals reached with key messages about the interventions or practice

Assessing knowledge use

Knowledge use is a critical element in providers’ adherence to evidence-based practices and recommendations. It can:

• change attitudes and beliefs about a particular practice (for example, a health professional becomes knowledgeable about a best practice, develops positive attitudes toward it and develops motivation to apply it)
• be complex (health professionals may choose to follow some recommendations but not others)
• be affected by structures and processes within the clinical setting

Types of knowledge use

There are three main types of knowledge use that change agents should consider when measuring knowledge use (Strauss et al., 2013):

Data collection approaches to monitor knowledge use

Consider using the following data collection approaches to monitor health professionals’ knowledge use or uptake of the intervention or practice. 

• Audit and feedback: You can collect data on specific aspects of the new intervention or practice, summarize them, and provide the results to health providers on how they may do better (Colquhoun et al., 2017; RNAO, 2012).
• Focus group: You can facilitate the discussion among a group of individuals about their experiences, attitudes, or behaviour about the intervention or practice. Focus groups provide dynamic interactions in a group setting to your guiding questions. These group interactions can provide meaningful information on how the change in practice or the new intervention is going.
• Interviews: You can conduct in-depth, one-on-one interviews to ask individuals  about their experiences, attitudes, or behaviours about the intervention or practice change (RNAO, 2012).
• Observation: You can conduct observations during morning huddles, team meetings and clinical practice can help you understand whether the change has been implemented in your setting (RNAO, 2012).
• Surveys: You can ask individuals to complete structured questionnaires with rating scales. You can also include open-ended questions. Surveys may be administered in person, by telephone, or via the internet (Dillman, 2007).

If the level of knowledge use/practice uptake is acceptable, you could begin to consider strategies for monitoring in the sustainability phase. Stakeholder interviews can be helpful in establishing which methods might be most effective.

If the level of knowledge use/practice uptake is less than expected or desired, it may be useful to reassess barriers to knowledge use and or modify the implementation strategies accordingly.

Identifying roles, responsibilities, and timelines for monitoring

• Assign responsibility for data collection so that everyone is clear of their roles and responsibilities. This also allows new change team members to come onto the project and get a sense of who is responsible for what, what they may be responsible for and when.
  • NOTE: Collection of monitoring data may occur regularly over short intervals or longer intervals. Depending on the intervention or practice, this could be weekly, monthly, quarterly, twice a year or annually.
• Note any requirements that are needed to collect the data, such as staffing and funds. Make sure to include monitoring activities in your budget (funds, human resources, equipment).

SOURCE: Evaluation Toolbox, 2010

Accelerate your success: When monitoring the uptake of the intervention or practice, you and your team may want to pay attention to any "emerging leaders". Emerging leaders are the early adopters of the change; they can be role models in applying the practice or intervention in their work.   Check out the SMA Framework’s ‘Emerging leaders’ to learn more t how you can identify emerging leaders that support the change. You may want to find ways to support them so they become change agents for your change initiative.

The importance of assessing knowledge, attitudes and beliefs

• Providers' knowledge, attitudes and beliefs about the new practice or intervention can play a crucial role in the extent to which the practice/intervention is used (Cook et al., 2015).
• Providers' perceptions of an intervention may act as a barrier or facilitator to its implementation (Aarons, 2006; Henggeler et al., 2008; Jensen-Doss et al., 2009).
• Providers' knowledge, attitudes and beliefs may influence:
  • whether they are willing to try the new practice or intervention
  • how they present any new activities to persons
  • whether they are willing to continue with a new practice should challenges arise (Cook et al., 2015).

Why measure knowledge, attitudes and beliefs when monitoring knowledge use?

• Adopting a new practice or intervention is significantly related to health professionals' knowledge as well as attitude.
• Recognizing that health professionals may not view the new practice or intervention as compatible with their existing practices – or are otherwise resistant – can allow you to address these issues up front (Cook et al., 2015).
• Obtaining validated and reliable measures of health professionals' knowledge, attitudes, and beliefs of the new practice, or intervention can give insight into how it is being understood and perceived.

The table below displays examples of how knowledge, attitudes and behaviours are measured, and the corresponding outcomes that represent knowledge use.

SOURCES: RNAO, 2012; The Improve Group, 2010. 

Monitoring knowledge use is important because this phase determines how, and to what extent, the practice or knowledge is used by the target users, such as point-of-care staff (RNAO, 2012; Strauss et al., 2013).

Monitoring knowledge use shows the extent to which:

• the knowledge is communicated to target users who need to be aware of the knowledge or to use the knowledge;
• the selected best practice guideline recommendations are known, accepted and applied; and 
• the intervention or practice is being used correctly. 

• Determine the minimum acceptable level of knowledge use/adoption to new practice or intervention with staff. For example, what would you consider as the minimum level of knowledge that health professionals must acquire to proceed with a change in practice?
• Identify key indicators that will allow your team to determine that the minimum acceptable level is reached.
• Consider how this minimum acceptable level will impact the outcomes you have set out to achieve from introducing a new practice or intervention.
• Secure a plan for measuring knowledge use or intervention/practice uptake using at least one of the methods in this section (for example, audit and feedback, observation, or surveys).
• Determine the appropriate frequency and time periods of measurement to which you will be administering these measures.
• Use validated and reliable measures wherever possible.

Image

Applying pragmatic and valid tools can support change teams to accurately monitor knowledge use and strengthen their understanding of these factors. Looking at the results of using the tool can also help change teams prepare for the next action cycle phase, “Evaluate outcomes”.

For each tool, two documents are provided 

1. a summary page that includes the tool’s purpose, description and applicable action cycle phases
2. a companion document that details the tools’ pragmatic and psychometric properties.

NOTE: 

*All of the tools listed can also be considered for use during the action cycle phase: Assess barriers and facilitators to knowledge use. 

**All of the tools listed can be considered for use during the action cycle phases: Identify the problem/Determine the know/do gap and Assess barriers and facilitators to knowledge use.

Image

Linking this phase to elements of the Social Movement Action Framework

You and your change team’s capacity in the “Monitor knowledge use” phase may be enhanced or accelerated by the addition of some of the elements of the Social Movement Action (SMA) Framework, as the two frameworks are complementary and used together to accelerate change uptake and sustainability. In addition to the linking example described earlier in this section, there can be many other points of connection between the two frameworks. Below are two examples for you to consider:   

• Intrinsic motivation: You and your change team can consider intrinsic motivation as a factor that positively contributes to the use of evidence in their local settings. Intrinsic motivation can be assessed by the individuals’ knowledge, attitudes and beliefs about the change. The more intrinsically motivated the staff, the more likely they are to adhere to the change. 
• Emerging leadership: Emerging leadership – whether formal or informal – takes a natural progression in the team and can lead to the monitoring of knowledge use. Emerging leaders act as influential role models and knowledge brokers who can positively impact knowledge, attitudes and beliefs towards the change and the adherence to the change. 

For more discussion about the dynamic links between the elements of the SMA Framework to the KTA Framework, see the section "Two complementary frameworks".