The Value of System Dynamics for Healthcare Resource Modelling

Abstract

It is no secret to hospital and public health managers thatresource shortages worsen pandemics. The importance of preparedness has longbeen recognized within the European Union. One of the current H2020 innovationprojects in this domain is PANDEM-2, aiming to improve pandemic preparednessfrom the side of resource management and sharing by creating cutting-edgedigital tools. As part of these tools, a system dynamics (SD) healthcareresource model is being developed, with the ultimate goal of embedding it in adashboard accessible to pandemic managers. This is done in order to supportmanagers in rapidly making evidence-based assessments and decisions, or as inthis thesis shortened, to provide situational awareness. In short, the specificproblem we were tackling was the exploration of how can pandemic preparedness canbe achieved via current healthcare resource models and how a specific resourcemodel (developed by a previous intern) can be used. First, to gain a generalunderstanding of the state-of-the-art models, we looked into the scientificliterature from two directions: We looked at how existing resource models work,are validated, and are used via literature review. For another perspective, welooked at scientific frameworks describing modelling and validation to informour methodology. Therefore, this thesis seeks to answer the question: How tosupport healthcare resource managers in acquiring situational awareness via anSD model? To gain a better understanding, we did a literature review first tounderstand how others approach the topic of healthcare resource modelling.

We first analyzed the existing scientific literature by apreliminary search, which was also used to construct a more detailed andrefined second search. In this second search, we used the PubMed database tosearch for articles containing the keywords hospital and healthcareresource, pandemic, model, validation, andsynonyms. Then the returned articles were screened for relevancy, resulting ina total of 25 healthcare resource models analyzed. Within these analyzed models(and articles), we found that the most common approach is using SD models, andthe second most common approach is using regression models. Roughly two-thirdsof the models fall into these two categories. Furthermore, we found that there isa stronger focus on hospital resources than public health resources and that nocommon approach is used for model validation. We also found that the articlesdemonstrating that the model is used to support real-life decision-making wereusually not about SD models; therefore, examining how to use SD healthcareresource models for decision support is not mainstream. We also found that themodel used in our research is novel in the sense that it encompasses resourceson a more detailed level than existing published models.

To further our understanding, we decided to answer ourresearch question by holding a workshop, where we examine how to communicatemodel outputs. While examining the relevant modelling methodologicalframeworks, we defined the tasks that need to be done in this thesis throughthe lens of the modelling cycle. We need to perform the tasks of verification,validation, and holding a workshop, which partly encompasses evaluation. Thenexamining the literature about verification and validation, we encountered theimplication of a well-known philosophical problem of scientific theories' formodelling: It cannot be demonstrated whether the model (or the theory) is atruthful description of the phenomenon.

To overcome this problem, in modelling, validation refers tobuilding confidence that the model is fit for its purpose. In this study, thepurpose of the model changed from describing the different mechanisms foundimportant to generate semi-realistic outputs to be used in the workshop;therefore, it had to be revalidated. This was addressed by performing aparticular set of relevant validation tests. The model passed verification andthen the validation for this purpose, so we continued with the workshop. Wedecided that in the workshop, we would use a presentation to communicateintervention opportunities for the pandemic based on the model outputs. Thenafter each intervention, the participants were asked to evaluate the easinessof understanding the output and to talk about what actions the presentedinformation inspire.

By holding the workshops, we found several relevant facts:First, it was found that the goal participants were searching for was to getrid of the perceived gap. This also meant they were searching for insights thatcould be used for operational planning purposes. Furthermore, the analysis doesnot need to stop at visualizing outputs. One of the participants indicated thatfurther analyzing the graphs is not as easy for them as for an analyst workingwith the model. We have also seen that participants tend to augment thepresented data with their experiences, which (unless explicitly presented)leads to assumptions about how the model works. Some participants also pointedout that the contact tracing part of the model is already outdated (in lessthan a year). We have identified some practical ways to avoid ambiguity whilecommunicating about healthcare resource models. First, we found that despitethe insights we gained by analyzing model outputs were not novel, thediscovered scenarios were still good discussion starters in the workshop. Thisis likely the mechanism of the scenarios acting as a reminder for passiveknowledge, which participants subsequently shared. Furthermore, extra careshould be taken to explain the context of how the data got generated, especiallyconcerning the model. As the presented data left some space for interpretation,participants sometimes had different assumptions than the ones coded into themodel. While these could be resolved in the workshop to some extent, this willnot be the case for the dashboard. Given some familiarity with the audience, itis possible to expect some questions and misunderstandings, which could beproactively addressed in a description or in a `frequently asked questions'. Wealso identified two presentation types that were easier to process thanpresenting key model outputs: The first option is to analyze key model outputsfurther than graphing and present the key insights (such as peak resourcedemand) in a tabular format. Alternatively, the second option is to build allvisualization on the same template and explain that template on the firstoccurrence in detail. In subsequent occurrences, it should be enough to pointout only the interesting parts and give participants time to process theinformation.

From another perspective, participants expressed a need fordata that can be used for planning purposes. However, given the uncertaintyabout the system, these, as we call consolidative models, cannot be constructedyet. While exploratory modelling is an alternative SD technique for addressingdeep uncertainty, it does not attempt to produce numerically accuratepredictions. However, from a novel perspective, the consolidative andexploratory approaches can be viewed as two ranges on the spectrum ofuncertainty about the modelled system. Viewed from this perspective, validationmeans reducing uncertainty about the system. Nevertheless, to achieve theconsolidative models, datasets about resource usage are needed, but as far aswe know, no such dataset exists. As data to create such datasets is probablyalready being collected for operational purposes, it is likely that thecollection and aggregation of these data are not happening. However, creatingsuch datasets comes with some challenges. There is a value trade-off betweenprivacy and preparedness through data collection, and the current datacollection techniques are unlikely to be unified. Overcoming these challengeswould need quite a significant upfront investment. To answer our originalquestion of how to support healthcare resource managers in acquiringsituational awareness, this thesis argues that, by far, the biggest utilitycould be achieved by strengthening data collection and aggregation, as itenables the possibility to develop surrogate models. However, as this requiresa significant upfront investment, question-driven exploratory models remain analternative way to address these uncertainties.