Criteria to Assess Quality & Impact of eHealth Tools: A Practical Toolbox

Technological advancements have opened the path to many providers to easily develop and introduce eHealth tools to the public. The use of these tools is increasingly recognized as a critical quality driver in healthcare. However, choosing a quality tool in the myriad of tools available for a certain health purpose doesn’t come without challenges. To name a few, users are typically faced with tools that have not been evaluated for quality and fit and that often lack relevant features and functionality, making it quite challenging for both clinicians and patients to find, evaluate, and use the right tools.

Challenges facing eHealth assessment efforts

Although there are various initiatives working on finding ways to assess the quality of eHealth tools, there is limited information and methods describing how to realistically assess and evaluate these tools in practice. Many of the existing initiatives have not been validated with the relevant stakeholders, and/or are conceptual without granular guidance on how to use and apply them in day-to-day decision making.

We want to contribute to this discussion to advance this area that is still being shaped and developed by analyzing and aggregating existing criteria through rigorous academic research and systematic review of existing frameworks, combined with a practice-oriented approach through discussing, pressure-testing, and co-creating a relevant and accessible toolbox with a diverse group of industry experts spanning all relevant stakeholder groups (clinicians, patient experts, pharma executives, insurance experts, investors, researchers, compliance experts, and med tech providers).

Jacob et al. (2023): Assessing the Quality and Impact of eHealth Tools: Systematic Literature Review and Narrative Synthesis. JMIR Hum Factors. doi: 10.2196/45143. PMID: 36843321. https://humanfactors.jmir.org/2023/1/e45143

The results of the systematic review have now been published, laying the groundwork for the validation of the aggregated assessment criteria with practice partners in a diversified eHealth expert panel in a modified Delphi process.

A practice-oriented and accessible toolbox

Having identified the conceptual nature of many existing assessment frameworks as a challenge, as practitioners oftentimes do not know how to use these frameworks in practice, we would like to adopt a more practice-oriented approach. Our goal is to create an easy-to-use and practical toolbox that equips the relevant stakeholders with the relevant checklists, definitions, and interactive dashboard that makes the task of assessing the quality and impact of eHealth tools as practical and as accessible as possible. We intend to discuss potential solutions with the industry experts, and to pressure test the potential toolbox with them to ensure its clarity and usability. We will also develop and offer on-demand webinars and training workshops to equip potential users with the knowledge they need to properly use the toolbox.

Bringing academic research rigor together with the practical experience from the practice partners and industry experts will ensure a solid approach to this research project that also balances the practical challenges that the stakeholders face in their day-to-day work and decision-making. Gaps and shortages of previous initiatives will be discussed in detail with the practice partners and industry experts in order to close these gaps and offer a validated and practical toolbox.