Evaluation Specialist

JOB SUMMARY:

The primary responsibilities of the Evaluation Specialist are to design, implement and maintain performance measures to support the quality of the data in the Nursing Quality Indicators for Reporting and Evaluation (NQuIRE®) data system by developing data dictionaries and resources to support data collection. Performance measures are aligned with the Best Practice Guidelines (BPG) and/or Best Practice Spotlight Organizations® (BPSO) Program goals to measure and evaluate the impact of implementing RNAO’s BPGs.

RESPONSIBILITIES:

• Lead the development of data dictionaries associated with performance measures (indicators / metrics / guideline-based indicators), ensuring guideline implementation and monitoring activities are informed by high-quality indicator data, thereby enhancing evidence-based practice, education, and decision-making locally, nationally, and internationally.
• Support the development, refinement, and retirement of relevant indicators / performance measurements based on rigorous methodology, using the best available evidence and stakeholder input.
• Contribute to advancing the processes and methodologies developed for indicator development and evaluation of BPG implementation.
• Assess the effectiveness of different programs by developing program-based evaluation criteria.
• Design and administer surveys to gather feedback from stakeholders on indicators, analyzing survey     results, identifying and resolving information gaps within BPSO programs and recommending strategies to address them.
• Develop clear and concise reports on the interpretation of data based on performance measures appropriate to nurses and the public.
• Act as a resource person for indicator-related inquiries and troubleshooting and as a liaison on cross-pillar collaborations to support teamwork across different teams: guideline development, implementation, and evaluation and research.
• Support the integration of real-world evidence into the BPG process with interpretation and clinical judgment.
• Foster open and transparent communication across teams and ensure equity, diversity and inclusiveness in collaboration and partnership.
• Encourage BPSOs with diverse data perspectives and interpretations to create novel solutions on BPG implementation and their impacts.
• Support interpretation and clinical judgement about the data analysis results and liaison between different teams to support the real-world data projects.
• Actively participate and contribute to the monitoring, evaluation and research portfolio as well as research proposals appropriately.
• Apply statistical concepts to evaluate performance metrics/indicators, make recommendations for improvement, and contribute to various evaluation projects using statistical methods.
• Provide formal and informal professional expertise/guidance to staff, organizational affiliates, and stakeholders with respect to the development, implementation and monitoring of quality indicators / performance measures in BPGs.
• Host national/ international evaluation and research events for BPSOs and national / international collaborations, including Knowledge-sharing calls and support.
• Provide data system support service to BPSOs by answering enquiries, providing information, and undertaking other relevant activities related to indicator development, selection and interpretation.
• Develop and deliver presentations, publications, posters and other scientific activities related to RNAO BPG indicator development and evaluation projects.
• Identify emerging issues and trends from data analysis and interpretation of NQuIRE indicator data and actively monitor relevant health system changes that may reflect on the BPSO program and/or performance metrics.
• Review documents, contribute to manuscript writing in various RNAO teams, and support BPSOs in areas of expertise.
• Support interpretation and presentation of evaluative information through compilation, statistical analysis and reporting of indicator data.
• Work with team members to prepare, analyze, and interpret statistical and trend reports.  
• Provide data analysis and evaluation support to the IABPG guideline development, implementation and evaluation teams.
• Collaborate with Senior Management, Evaluation team, Project Coordinators, IT Database / Web Developers, guideline development methodologists, and implementation science specialists, using strategic and tactical approaches to support a culture of data quality.
• Other job-related duties as assigned

SKILL, KNOWLEDGE AND EXPERIENCE:

• Registered Nurse with current registration with College of Nurses of Ontario or other recognized nursing body.
• Master’s degree is required in the fields of STEM, Population Health, Epidemiology and/or Informatics.
• The successful candidate will have at least three plus (3+) years of experience in a performance measurement area within a health environment and a thorough understanding of healthcare performance measures and quality indicators.
• Experience in indicator development, quality improvement methods, and systematic scoping and/or literature reviews.
• Knowledge of GRADE and GRADE-CER Qual methodologies for systematic reviews and guideline development is an asset.
• Familiarity with quality indicators from other clinical datasets/data repositories, such as RAI-MDS, CIHI, ICES, etc.
• Sound knowledge and demonstrated skills in epidemiological and statistical methods, applied research and survey design.
• Experience using statistical software such as R, NVIVO, STATA, or SAS and familiarity with Word, Excel, and other software packages and programming languages currently used to collate and conduct analysis of health data.
• Experience in conducting program evaluations, research protocol/proposal development, and preparation of scientific reports and presentations

• Project management experience within an interprofessional team environment.
  Excellent interpersonal and communication skills (both verbal and written), enabling strong working relationships with team members, management and other stakeholders.

• Excellent critical thinking, organizational and analytical skills and self-directed work ethic to effectively manage multiple data indicator-based projects in various stages of development.
• Proficiency in French or Spanish is also considered an asset

EFFORT:

• Continuous concentration is required to read a large volume of information daily, research, prepare written reports, and consult as appropriate.
• Retain the highest degree of privacy of information received.
• The highest degree of confidentiality, sound judgement, respect for all, consistency, fairness, and professionalism.
• The workflow requires meeting established, regular deadlines that may require re-prioritizing the workload.
• Concentration is required for short and long periods to process data.
• Exceptional attention to detail and strong logical thinking ability

WORKING CONDITIONS:

• Works in a climate-controlled office with some exposure to distractions, interruptions and stress from multiple demands.
• Travel to meetings, conferences and workshop may required from time to time.
• Typical conditions are such that no risk or injury is present.