In the European Union (EU) the delivery of health services is a national responsibility but there are concerted actions between member states to protect public health. Approval of pharmaceutical products is the responsibility of the European Medicines Agency, whereas authorizing the placing on the market of medical devices is decentralized to independent 'conformity assessment' organizations called notified bodies. The first legal basis for an EU system of evaluating medical devices and approving their market access was the medical device directives, from the 1990s. Uncertainties about clinical evidence requirements, among other reasons, led to the EU Medical Device Regulation (2017/745) that has applied since May 2021. It provides general principles for clinical investigations but few methodological details-which challenges responsible authorities to set appropriate balances between regulation and innovation, pre- and post-market studies, and clinical trials and real-world evidence. Scientific experts should advise on methods and standards for assessing and approving new high-risk devices, and safety, efficacy, and transparency of evidence should be paramount. The European Commission recently awarded a Horizon 2020 grant to a consortium led by the European Society of Cardiology and the European Federation of National Associations of Orthopaedics and Traumatology, that will review methodologies of clinical investigations, advise on study designs, and develop recommendations for aggregating clinical data from registries and other real-world sources. The CORE-MD project (Coordinating Research and Evidence for Medical Devices) will run until March 2024; here we describe how it may contribute to the development of regulatory science in Europe.

@en

Background:
Statistical models using machine learning (ML) have the potential for more accurate estimates of the probability of binary events than logistic regression. The present study used existing data sets from large musculoskeletal trauma trials to address the following study questions: (1) Do ML models produce better probability estimates than logistic regression models? (2) Are ML models influenced by different variables than logistic regression models?

Methods:
We created ML and logistic regression models that estimated the probability of a specific fracture (posterior malleolar involvement in distal spiral tibial shaft and ankle fractures, scaphoid fracture, and distal radial fracture) or adverse event (subsequent surgery [after distal biceps repair or tibial shaft fracture], surgical site infection, and postoperative delirium) using 9 data sets from published musculoskeletal trauma studies. Each data set was split into training (80%) and test (20%) subsets. Fivefold cross-validation of the training set was used to develop the ML models. The best-performing model was then assessed in the independent testing data. Performance was assessed by (1) discrimination (c-statistic), (2) calibration (slope and intercept), and (3) overall performance (Brier score).

Results:
The mean c-statistic was 0.01 higher for the logistic regression models compared with the best ML models for each data set (range, −0.01 to 0.06). There were fewer variables strongly associated with variation in the ML models, and many were dissimilar from those in the logistic regression models.

Conclusions:
The observation that ML models produce probability estimates comparable with logistic regression models for binary events in musculoskeletal trauma suggests that their benefit may be limited in this context.@en

Purpose: The purpose of this study was to assess in which proportion of patients with degenerative knee disease aged 50+ in whom a knee arthroscopy is performed, no valid surgical indication is reported in medical records, and to explore possible explanatory factors. Methods: A retrospective study was conducted using administrative data from January to December 2016 in 13 orthopedic centers in the Netherlands. Medical records were selected from a random sample of 538 patients aged 50+ with degenerative knee disease in whom arthroscopy was performed, and reviewed on reported indications for the performed knee arthroscopy. Valid surgical indications were predefined based on clinical national guidelines and expert opinion (e.g., truly locked knee). A knee arthroscopy without a reported valid indication was considered potentially low value care. Multivariate logistic regression analysis was performed to assess whether age, diagnosis (“Arthrosis” versus “Meniscal lesion”), and type of care trajectory (initial or follow-up) were associated with performing a potentially low value knee arthroscopy. Results: Of 26,991 patients with degenerative knee disease, 2556 (9.5%) underwent an arthroscopy in one of the participating orthopedic centers. Of 538 patients in whom an arthroscopy was performed, 65.1% had a valid indication reported in the medical record and 34.9% without a reported valid indication. From the patients without a valid indication, a joint patient–provider decision or patient request was reported as the main reason. Neither age [OR 1.013 (95% CI 0.984–1.043)], diagnosis [OR 0.998 (95% CI 0.886–1.124)] or type of care trajectory [OR 0.989 (95% CI 0.948–1.032)] were significantly associated with performing a potentially low value knee arthroscopy. Conclusions: In a random sample of knee arthroscopies performed in 13 orthopedic centers in 2016, 65% had valid indications reported in the medical records but 35% were performed without a reported valid indication and, therefore, potentially low value care. Patient and/or surgeons preference may play a large role in the decision to perform an arthroscopy without a valid indication. Therefore, interventions should be developed to increase adherence to clinical guidelines by surgeons that target invalid indications for a knee arthroscopy to improve care. Level of evidence: IV.@en

Purpose: To evaluate the effectiveness of a tailored intervention to reduce low value MRIs and arthroscopies among patients ≥ 50 years with degenerative knee disease in 13 Dutch orthopaedic centers (intervention group) compared with all other Dutch orthopaedic centers (control group). Methods: All patients with degenerative knee disease ≥ 50 years admitted to Dutch orthopaedic centers from January 2016 to December 2018 were included. The tailored intervention included participation of clinical champions, education on the Dutch Choosing Wisely recommendation for MRI’s and arthroscopies in degenerative knee disease, training of orthopaedic surgeons to manage patient expectations, performance feedback, and provision of a patient brochure. A difference-in-difference analysis was used to compare the time trend before (admitted January 2016–June 2017) and after introduction of the intervention (July 2017–December 2018) between intervention and control hospitals. Primary outcome was the monthly percentage of patients receiving a MRI or knee arthroscopy, weighted by type of hospital. Results: 136,446 patients were included, of whom 32,163 were treated in the intervention hospitals. The weighted percentage of patients receiving a MRI on average declined by 0.15% per month (β = − 0.15, P < 0.001) and by 0.19% per month for arthroscopy (β = − 0.19, P < 0.001). However, these changes over time did not differ between intervention and control hospitals, neither for MRI (β = − 0.74, P = 0.228) nor arthroscopy (β = 0.13, P = 0.688). Conclusions: The extent to which patients ≥ 50 years with degenerative knee disease received a MRI or arthroscopy declined significantly over time, but could not be attributed to the tailored intervention. This secular downward time trend may reflect anoverall focus of reducing low value care in The Netherlands. Level of evidence: III.@en

Background: Report cards on the health care system increasingly report provider-specific performance on indicators that measure the quality of health care delivered. A natural reaction to the publishing of hospital-specific performance on a given indicator is to create 'league tables' that rank hospitals according to their performance. However, many indicators have been shown to have low to moderate rankability, meaning that they cannot be used to accurately rank hospitals. Our objective was to define conditions for improving the ability to rank hospitals by combining several binary indicators with low to moderate rankability. Methods: Monte Carlo simulations to examine the rankability of composite ordinal indicators created by pooling three binary indicators with low to moderate rankability. We considered scenarios in which the prevalences of the three binary indicators were 0.05, 0.10, and 0.25 and the within-hospital correlation between these indicators varied between - 0.25 and 0.90. Results: Creation of an ordinal indicator with high rankability was possible when the three component binary indicators were strongly correlated with one another (the within-hospital correlation in indicators was at least 0.5). When the binary indicators were independent or weakly correlated with one another (the within-hospital correlation in indicators was less than 0.5), the rankability of the composite ordinal indicator was often less than at least one of its binary components. The rankability of the composite indicator was most affected by the rankability of the most prevalent indicator and the magnitude of the within-hospital correlation between the indicators. Conclusions: Pooling highly-correlated binary indicators can result in a composite ordinal indicator with high rankability. Otherwise, the composite ordinal indicator may have lower rankability than some of its constituent components. It is recommended that binary indicators be combined to increase rankability only if they represent the same concept of quality of care.@en

Background: Hospital mortality, readmission and length of stay (LOS) are commonly used measures for quality of care. We aimed to disentangle the correlations between these interrelated measures and propose a new way of combining them to evaluate the quality of hospital care. Methods: We analyzed administrative data from the Global Comparators Project from 26 hospitals on patients discharged between 2007 and 2012. We correlated standardized and risk-adjusted hospital outcomes on mortality, readmission and long LOS. We constructed a composite measure with 5 levels, based on literature review and expert advice, from survival without readmission and normal LOS (best) to mortality (worst outcome). This composite measure was analyzed using ordinal regression, to obtain a standardized outcome measure to compare hospitals. Results: Overall, we observed a 3.1% mortality rate, 7.8% readmission rate (in survivors) and 20.8% long LOS rate among 4,327,105 admissions. Mortality and LOS were correlated at the patient and the hospital level. A patient in the upper quartile LOS had higher odds of mortality (odds ratio = 1.45, 95% confidence interval 1.43-1.47) than those in the lowest quartile. Hospitals with a high standardized mortality had higher proportions of long LOS (r = 0.79, p < 0.01). Readmission rates did not correlate with either mortality or long LOS rates. The interquartile range of the standardized ordinal composite outcome was 74-117. The composite outcome had similar or better reliability in ranking hospitals than individual outcomes. Conclusions: Correlations between different outcome measures are complex and differ between hospital- and patient-level. The proposed composite measure combines three outcomes in an ordinal fashion for a more comprehensive and reliable view of hospital performance than its component indicators.@en

Background: Ecological fallacy refers to an erroneous inference about individuals on the basis of findings for the group to which those individuals belong. Suppose analysis of a large database shows that hospitals with a high proportion of long length of stay (LOS) patients also have higher than average in-hospital mortality. This may prompt efforts to reduce mortality among patients with long LOS. But patients with long LOS may not be the ones at higher risk of death. It may be that hospitals with higher mortality (regardless of LOS) also have more long LOS patients-either because of quality problems on both counts or because of unaccounted differences in case mix. To provide more insight how the ecological fallacy influences the evaluation of hospital performance indicators, we assessed whether hospital-level associations between in-hospital mortality, readmission and long LOS reflect patient-level associations. Methods: Patient admissions from the Dutch National Medical Registration (2007-2012) for specific diseases (stroke, colorectal carcinoma, heart failure, acute myocardial infarction and hip/knee replacements in patients with osteoarthritis) were analysed, as well as all admissions. Logistic regression analysis was used to assess patient-level associations. Pearson correlation coefficients were used to quantify hospital-level associations. Results: Overall, we observed 2.2% in-hospital mortality, 8.1% readmissions and a mean LOS of 5.9 days among 8 478 884 admissions in 95 hospitals. Of the 10 disease-specific associations tested, 2 were reversed at hospital-level, 3 were consistent and 5 were only significant at either hospital-level or patient-level. A reversed association was found for stroke: patients with long LOS had 58% lower in-hospital mortality (OR 0.42 (95% CI 0.40 to 0.44)), whereas the hospital-level association was reversed (r=0.30, p0.01). Similar negative patient-level associations were found for each hospital, but LOS varied across hospitals, thereby resulting in a positive hospital-level association. A similar effect was found for long LOS and readmission in patients with heart failure. Conclusions: Hospital-level associations did not reflect the same patient-level associations in 7 of 10 associations, and were even reversed in 2 associations. Ecological fallacy thus potentially influences interpretation of hospital performance when patient-level associations are not taken into account.@en

Background: The double-blind, placebo-controlled food challenge test (DBPCFC) is the gold standard in cashew nut allergy. This test is costly, time consuming and not without side effects. Analysis of IgE reactivity to cashew nut components may reduce the need for food challenge tests. Methods: In a prospective and multicentre study, children with suspected cashew nut allergy underwent a DBPCFC with cashew nut. Specific IgE to cashew nut and to the components Ana o 1, 2 and 3 were determined. A skin prick test (SPT) with cashew nut extract was performed. The association between the outcome of the food challenge test and specific IgE to Ana o 1, 2 and 3 was assessed with logistic regression analyses, unadjusted and adjusted for other diagnostic variables. Discriminative ability was quantified with a concordance index (c). Results: A total of 173 children (103 boys, 60%) with a median age of 9 years were included. About 79% had a positive challenge test outcome. A steep rise in the risk of a positive challenge was observed for specific IgE to each individual component Ana o 1, 2 and 3 with estimated risks up to approximately 100%. Median values of Ana o 1, 2, 3 were 1.29 kU/l (range 0-100 kU/l), 4.77 kU/l (range 0-100 kU/l) and 8.33 kU/l (range 0-100 kU/l) respectively and varied significantly (p <0.001). Specific IgE to Ana o 1, 2 and 3 was better distinguished between cashew-allergic and tolerant children (c = 0.87, 0.85 and 0.89, respectively) than specific IgE to cashew nut or SPT (c = 0.76 and 0.83, respectively). Conclusion: The major cashew nut allergens Ana o 1, 2 and 3 are each individually predictive for the outcome of food challenge tests in cashew-allergic children.

@en

Conclusions: Although readmission rates are a promising quality indicator, several methodological concerns identified in this study need to be addressed, especially when the indicator is intended for accountability or pay for performance. We recommend investing resources in accurate data registration, improved indicator description, and bundling outcome measures to provide a more complete picture of hospital care.Introduction: Hospital readmission rates are increasingly used for both quality improvement and cost control. However, the validity of readmission rates as a measure of quality of hospital care is not evident. We aimed to give an overview of the different methodological aspects in the definition and measurement of readmission rates that need to be considered when interpreting readmission rates as a reflection of quality of care.Methods: We conducted a systematic literature review, using the bibliographic databases Embase, Medline OvidSP, Web-of- Science, Cochrane central and PubMed for the period of January 2001 to May 2013.Results: The search resulted in 102 included papers. We found that definition of the context in which readmissions are used as a quality indicator is crucial. This context includes the patient group and the specific aspects of care of which the quality is aimed to be assessed. Methodological flaws like unreliable data and insufficient case-mix correction may confound the comparison of readmission rates between hospitals. Another problem occurs when the basic distinction between planned and unplanned readmissions cannot be made. Finally, the multi-faceted nature of quality of care and the correlation between readmissions and other outcomes limit the indicator's validity.@en