November 1, 2022

Establishing digital PROs in clinical routine care: Analysis and tackling of key challenges to the wide-scale implementation of ICHOM sets

At ICHOM 2022 in Boston, the world’s leading conference on value-based healthcare, new insights by the Medical Science team were presented. Establishing digital PROs in clinical routine care: Analysis and tackling of key challenges to the wide-scale implementation of ICHOM sets takes into account a decade of implementing PROMs in routine care.

by Hannah Haneke*, Sina Nickel*, Luisa Chlupka°, Valerie Kirchberger*
*Heartbeat Medical   °Charité – Universitätsmedizin Berlin


Promoting high-quality standardized outcome sets is crucial to re-organize care pathways around patients thereby leveraging the overall quality of routine clinical care. ICHOM proposes standard sets that aim to provide a holistic understanding of specific medical conditions by recommending a combination of relevant instruments, including patient- and clinician-reported outcomes. As they meet high methodological standards, interest in implementing those is huge but often faced by practical challenges.

As the leading software provider for patient-reported outcome data in German speaking countries, Heartbeat Medical (HRTBT Medical Solutions GmbH) has gathered profound experience in the digital implementation of outcome sets. Of the 41 available ICHOM sets, 16 have been implemented to some degree. While intense effort went into overcoming methodological barriers thus facilitating implementation of these outcome sets on a broader organizational level, less emphasis has been placed on practical considerations on an individual user level so far. This study presents our findings on key challenges for the wide-scale implementation of standardized outcome measures in clinical routine care and contributes to closing this gap.


Cross-team interviews of HRTBT employees served to identify the main implementation challenges. For further structured analysis of these factors, all 41 reference guides for ICHOM-outcome sets were extracted from ICHOM Connect during a time period from May to June 2022, followed by sorting recommended outcomes and classifying categories for the challenges.

All outcomes were reviewed by conducting a literature search on Pubmed and Google Scholar, as well as reviewing the websites of specific scores (e.g. for PROMIS). Next, a quantitative analysis was conducted, providing descriptive statistics on the key obstacles aggregated according to all ICHOM-sets. Additionally, one specific set implemented by HRTBT in May 2022 served as a case study for identifying potential strategies to overcome the main obstacles.


Identified key challenges comprise licensing and language issues (missing validated German questionnaires). A total of 221 different outcomes were proposed over all sets. 38 of 41 sets presented at least one of the two identified issues (93%). For 115 outcomes, no validated German translation was found (52%). Results on licensing information are presented in Table 1. Over one third were free to use but more than half required an activity (e.g. registering on a website or contacting corresponding authors for permission) or a fee. PROMIS questionnaires are a special case, since they can be free for research but licensing has to be clarified with the national PROMIS center for the digital use in clinical routine.

The ICHOM atrial fibrillation set served as an example for deeper analysis and the following challenges were identified: (i) lack of German validated translations for two key measures, (ii) a declined license for one questionnaire and (ii) a high fee for another questionnaire. Strategies that were considered to overcome these obstacles included: (i) searching for similar instruments therefore adapting the set, (ii) performing a translation and subsequent validation ourselves, and (iii) implementing a reduced set.


This study provides first insights into challenges of implementing ICHOM sets from a technology provider’s point of view. Licensing and language barriers were identified as major obstacles for standardized outcome set implementation. Besides serving as important groundwork for developing strategies to overcome these difficulties in implementation, the results can be beneficial for the future development and revision of standardized sets. ICHOM sets might develop an even bigger international impact in the establishment of outcome measurement in clinical practice if the difficulties outlined here are overcome as for instance by including commonly used PROMs with a high methodological standard while taking those obstacles into consideration.