Artificial Intelligence (AI)-based auto-delineation technologies rapidly delineate multiple structures of interest like organs-at-risk and tumors in 3D medical images, reducing personnel load and facilitating time-critical therapies. Despite its accuracy, the AI may produce flawed delineations, requiring clinician attention. Quality assessment (QA) of these delineations is laborious and demanding. Delineation error detection systems aim to aid QA, yet questions linger about clinician adoption, challenges, and time-saving potential. In this study, we address these queries in two stages. First, we investigate the error detection workflow of a radiotherapy technologist and a radiation oncologist from Holland Proton Therapy Center, a Dutch cancer treatment center. The user study revealed which information sources clinicians prefer to use for the error prioritization task and elucidated clinicians' slice-based navigation workflows with and without system assistance. Based on the findings from the user study, we developed a simulation model of the QA process, which we used to assess different error detection workflows on a retrospective cohort of 42 head and neck cancer patients. The simulation study results indicate potential time savings through error and dose information, contingent on per-slice analysis time remaining near the current baseline. Our findings encourage the development of user-centric delineation error detection systems and provide a new way to model and evaluate these systems' potential clinical value.@en

Objective: To obtain insight in the spectrum of narcolepsy symptoms and associated burden in a large cohort of patients. Methods: We used the Narcolepsy Monitor, a mobile app, to easily rate the presence and burden of 20 narcolepsy symptoms. Baseline measures were obtained and analyzed from 746 users aged between 18 and 75 years with a reported diagnosis of narcolepsy. Results: Median age was 33.0 years (IQR 25.0–43.0), median Ullanlinna Narcolepsy Scale 19 (IQR 14.0–26.0), 78% reported using narcolepsy pharmacotherapy. Excessive daytime sleepiness (97.2%) and lack of energy were most often present (95.0%) and most often caused a high burden (79.7% and 76.1% respectively). Cognitive symptoms (concentration 93.0%, memory 91.4%) and psychiatric symptoms (mood 76.8%, anxiety/panic 76.4%) were relatively often reported to be present and burdensome. Conversely, sleep paralysis and cataplexy were least often reported as highly bothersome. Females experienced a higher burden for anxiety/panic, memory, and lack of energy. Conclusions: This study supports the notion of an elaborate narcolepsy symptom spectrum. Each symptom’s contribution to the experienced burden varied, but lesser-known symptoms did significantly add to this as well. This emphasizes the need to not only focus treatment on the classical core symptoms of narcolepsy.

@en

Post-translational modifications (PTMs) affecting a protein's residues (amino acids) can disturb its function, leading to illness. Whether or not a PTM is pathogenic depends on its type and the status of neighboring residues. In this paper, we present the ProtoFold Neighborhood Inspector (PFNI), a visualization system for analyzing residues neighborhoods. The main contribution is a visualization idiom, the Residue Constellation (RC), for identifying and comparing three-dimensional neighborhoods based on per-residue features and spatial characteristics. The RC leverages two-dimensional representations of the protein's three-dimensional structure to overcome problems like occlusion, easing the analysis of neighborhoods that often have complicated spatial arrangements. Using the PFNI, we explored proteins' structural PTM data, which allowed us to identify patterns in the distribution and quantity of per-neighborhood PTMs that might be related to their pathogenic status. In the following, we define the tasks that guided the development of the PFNI and describe the data sources we derived and used. Then, we introduce the PFNI and illustrate its usage through an example of an analysis workflow. We conclude by reflecting on preliminary findings obtained while using the tool on the provided data and future directions concerning the development of the PFNI.@en

Diffusion-Weighted Magnetic Resonance Imaging (DWI) enables the in-vivo visualization of fibrous tissues such as white matter in the brain. Diffusion-Tensor Imaging (DTI) specifically models the DWI diffusion measurements as a second order-tensor. The processing pipeline to visualize this data, from image acquisition to the final rendering, is rather complex. It involves a considerable amount of measurements, parameters and model assumptions, all of which generate uncertainties in the final result which typically are not shown to the analyst in the visualization. In recent years, there has been a considerable amount of work on the visualization of uncertainty in DWI, and specifically DTI. In this chapter, we primarily focus on DTI given its simplicity and applicability, however, several aspects presented are valid for DWI as a whole. We explore the various sources of uncertainties involved, approaches for modeling those uncertainties, and, finally, we survey different strategies to visually represent them. We also look at several related methods of uncertainty visualization that have been applied outside DTI and discuss how these techniques can be adopted to the DTI domain. We conclude our discussion with an overview of potential research directions.@en

Diffusion Tensor Imaging (DTI) is a non-invasive magnetic resonance imaging technique that, combined with fiber tracking algorithms, allows the characterization and visualization of white matter structures in the brain. The resulting fiber tracts are used, for example, in tumor surgery to evaluate the potential brain functional damage due to tumor resection. The DTI processing pipeline from image acquisition to the final visualization is rather complex generating undesirable uncertainties in the final results. Most DTI visualization techniques do not provide any information regarding the presence of uncertainty. When planning surgery, a fixed safety margin around the fiber tracts is often used; however, it cannot capture local variability and distribution of the uncertainty, thereby limiting the informed decision-making process. Stochastic techniques are a possibility to estimate uncertainty for the DTI pipeline. However, it has high computational and memory requirements that make it infeasible in a clinical setting. The delay in the visualization of the results adds hindrance to the workflow. We propose a progressive approach that relies on a combination of wild-bootstrapping and fiber tracking to be used within the progressive visual analytics paradigm. We present a local bootstrapping strategy, which reduces the computational and memory costs, and provides fiber-tracking results in a progressive manner. We have also implemented a progressive aggregation technique that computes the distances in the fiber ensemble during progressive bootstrap computations. We present experiments with different scenarios to highlight the benefits of using our progressive visual analytic pipeline in a clinical workflow along with a use case and analysis obtained by discussions with our collaborators.

@en

One common way to aid coaching and seek to improve athletes’ performance is by recording training sessions for posterior analysis. In the case of sailing, coaches record videos from another boat, but usually rely on handheld devices, which may lead to issues with the footage and missing important moments. On the other hand, by autonomously recording the entire session with a fixed camera, the analysis becomes challenging owing to the length of the video and possible stabilization issues. In this work, we aim to facilitate the analysis of such full-session videos by automatically extracting maneuvers and providing a visualization framework to readily locate interesting moments. Moreover, we address issues related to image stability. Finally, an evaluation of the framework points to the benefits of video stabilization in this scenario and an appropriate accuracy of the maneuver detection method.@en

Phase-Contrast Magnetic Resonance Imaging (PC-MRI) surpasses all other imaging methods in quality and completeness for measuring time-varying volumetric blood flows and has shown potential to improve both diagnosis and risk assessment of cardiovascular diseases. However, like any measurement of physical phenomena, the data are prone to noise, artefacts and has a limited resolution. Therefore, PC-MRI data itself do not fulfil physics fluid laws making it difficult to distinguish important flow features. For data analysis, physically plausible and high-resolution data are required. Computational fluid dynamics provides high-resolution physically plausible flows. However, the flow is inherently coupled to the underlying anatomy and boundary conditions, which are difficult or sometimes even impossible to adequately model with current techniques. We present a novel methodology using data assimilation techniques for PC-MRI noise and artefact removal, generating physically plausible flow close to the measured data. It also allows us to increase the spatial and temporal resolution. To avoid sensitivity to the anatomical model, we consider and update the full 3D velocity field. We demonstrate our approach using phantom data with various amounts of induced noise and show that we can improve the data while preserving important flow features, without the need of a highly detailed model of the anatomy.

@en

Nowadays video plays an important role in the coaching of athletes across many different sports. For sailing training sessions, the videos are recorded from the coach boat and provide ways to review and analyze the training sessions aiming at improving the sailors performance. On one hand, videos are commonly recorded with handheld devices that are rather cumbersome to acquire and are prone to missing many important moments. On the other hand, recordings of entire sessions with an integrated camera in the coach boat are difficult to analyze given their length and the lack of image stability. We present a pipeline to facilitate the visual analysis of these full session videos. We extract manoeuvres as interesting segments, and provide a visualization framework to present the video and processed data. Manoeuvres are extracted by detecting and tracking the boat and sailors. With the visualization tool, the user can locate and visually inspect those manoeuvres for coaching tasks. We evaluated the potential of the framework and from the results we conclude that the manoeuvre detection is reasonably accurate and some coaches see potential in the presented framework.@en

In recent years the t-distributed Stochastic Neighbor Embedding (t-SNE) algorithm has become one of the most used and insightful techniques for exploratory data analysis of high-dimensional data. It reveals clusters of high-dimensional data points at different scales while only requiring minimal tuning of its parameters. However, the computational complexity of the algorithm limits its application to relatively small datasets. To address this problem, several evolutions of t-SNE have been developed in recent years, mainly focusing on the scalability of the similarity computations between data points. However, these contributions are insufficient to achieve interactive rates when visualizing the evolution of the t-SNE embedding for large datasets. In this work, we present a novel approach to the minimization of the t-SNE objective function that heavily relies on graphics hardware and has linear computational complexity. Our technique decreases the computational cost of running t-SNE on datasets by orders of magnitude and retains or improves on the accuracy of past approximated techniques. We propose to approximate the repulsive forces between data points by splatting kernel textures for each data point. This approximation allows us to reformulate the t-SNE minimization problem as a series of tensor operations that can be efficiently executed on the graphics card. An efficient implementation of our technique is integrated and available for use in the widely used Google TensorFlow.js, and an open-source C++ library.@en

Recent attempts at utilizing visual analytics to interpret Recurrent Neural Networks (RNNs) mainly focus on natural language processing (NLP) tasks that take symbolic sequences as input. However, many real-world problems like environment pollution forecasting apply RNNs on sequences of multi-dimensional data where each dimension represents an individual feature with semantic meaning such as PM_2.5 and SO₂. RNN interpretation on multi-dimensional sequences is challenging as users need to analyze what features are important at different time steps to better understand model behavior and gain trust in prediction. This requires effective and scalable visualization methods to reveal the complex many-to-many relations between hidden units and features. In this work, we propose a visual analytics system to interpret RNNs on multi-dimensional time-series forecasts. Specifically, to provide an overview to reveal the model mechanism, we propose a technique to estimate the hidden unit response by measuring how different feature selections affect the hidden unit output distribution. We then cluster the hidden units and features based on the response embedding vectors. Finally, we propose a visual analytics system which allows users to visually explore the model behavior from the global and individual levels. We demonstrate the effectiveness of our approach with case studies using air pollutant forecast applications.

@en

Anatomy, or the study of the structure of the human body, is an essential component of medical education. Certain parts of human anatomy are considered to be more complex to understand than others, due to a multitude of closely related structures. Furthermore, there are many potential variations in anatomy, e.g., different topologies of vessels, and knowledge of these variations is critical for many in medical practice. Some aspects of individual anatomy, such as the autonomic nerves, are not visible in individuals through medical imaging techniques or even during surgery, placing these nerves at risk for damage. 3D models and interactive visualization techniques can be used to improve understanding of this complex anatomy, in combination with traditional medical education paradigms. We present a framework incorporating several advanced medical visualization techniques and applications for teaching and training purposes, which is the result of an interdisciplinary project. In contrast to previous approaches which focus on general anatomy visualization or direct visualization of medical imaging data, we employ model-based techniques to represent variational anatomy, as well as anatomy not visible from imaging. Our framework covers the complete spectrum including general anatomy, anatomical variations, and anatomy in individual patients. Applications within our framework were evaluated positively with medical users, and our educational tool for general anatomy is in use in a Massive Open Online Course (MOOC) on anatomy, which had over 17000 participants worldwide in the first run.@en

Hierarchical embeddings, such as HSNE, address critical visual and computational scalability issues of traditional techniques for dimensionality reduction. The improved scalability comes at the cost of the need for increased user interaction for exploration. In this paper, we provide a solution for the interactive visual Focus+Context exploration of such embeddings. We explain how to integrate embedding parts from different levels of detail, corresponding to focus and context groups, in a joint visualization. We devise an according interaction model that relates typical semantic operations on a Focus+Context visualization with the according changes in the level-of-detail-hierarchy of the embedding, including also a mode for comparative Focus+Context exploration and extend HSNE to incorporate the presented interaction model. In order to demonstrate the effectiveness of our approach, we present a use case based on the visual exploration of multi-dimensional images.

@en

Recent advances in single-cell acquisition technology have led to a shift towards single-cell analysis in many fields of biology. In immunology, detailed knowledge of the cellular composition is of interest, as it can be the cause of deregulated immune responses, which cause diseases. Similarly, vaccination is based on triggering proper immune responses; however, many vaccines are ineffective or only work properly in a subset of those who are vaccinated. Identifying differences in the cellular composition of the immune system in such cases can lead to more precise treatment. Cytosplore is an integrated, interactive visual analysis framework for the exploration of large single-cell datasets. We have developed Cytosplore in close collaboration with immunology researchers and several partners use the software in their daily workflow. Cytosplore enables efficient data analysis and has led to several discoveries alongside high-impact publications.@en

Phase-Contrast Magnetic Resonance Imaging (PC-MRI) measures volumetric and time-varying blood flow data, unsurpassed in quality and completeness. Such blood-flow data have been shown to have the potential to improve both diagnosis and risk assessment of cardiovascular diseases (CVDs) uniquely. Typically PC-MRI data is visualized using stream- or pathlines. However, time-varying aspects of the data, e.g., vortex shedding, breakdown, and formation, are not sufficiently captured by these visualization techniques. Experimental flow visualization techniques introduce a visible medium, like smoke or dye, to visualize flow aspects including time-varying aspects. We propose a framework that mimics such experimental techniques by using a high number of particles. The framework offers great flexibility which allows for various visualization approaches. These include common traditional flow visualizations, but also streak visualizations to show the temporal aspects, and uncertainty visualizations. Moreover, these patient-specific measurements suffer from noise artifacts and a coarse resolution, causing uncertainty. Traditional flow visualizations neglect uncertainty and, therefore, may give a false sense of certainty, which can mislead the user yielding incorrect decisions. Previously, the domain experts had no means to visualize the effect of the uncertainty in the data. Our framework has been adopted by domain experts to visualize the vortices present in the sinuses of the aorta root showing the potential of the framework. Furthermore, an evaluation among domain experts indicated that having the option to visualize the uncertainty contributed to their confidence on the analysis. @en

Background: The clinical benefit of immunotherapeutic approaches against cancer has been well established although complete responses are only observed in a minority of patients. Combination immunotherapy offers an attractive avenue to develop more effective cancer therapies by improving the efficacy and duration of the tumor-specific T-cell response. Here, we aimed at deciphering the mechanisms governing the response to PD-1/PD-L1 checkpoint blockade to support the rational design of combination immunotherapy. Methods: Mice bearing subcutaneous MC-38 tumors were treated with blocking PD-L1 antibodies. To establish high-dimensional immune signatures of immunotherapy-specific responses, the tumor microenvironment was analyzed by CyTOF mass cytometry using 38 cellular markers. Findings were further examined and validated by flow cytometry and by functional in vivo experiments. Immune profiling was extended to the tumor microenvironment of colorectal cancer patients. Results: PD-L1 blockade induced selectively the expansion of tumor-infiltrating CD4⁺ and CD8⁺ T-cell subsets, co-expressing both activating (ICOS) and inhibitory (LAG-3, PD-1) molecules. By therapeutically co-targeting these molecules on the T_AI cell subsets in vivo by agonistic and antagonist antibodies, we were able to enhance PD-L1 blockade therapy as evidenced by an increased number of T_AI cells within the tumor micro-environment and improved tumor protection. Moreover, T_AI cells were also found in the tumor-microenvironment of colorectal cancer patients. Conclusions: This study shows the presence of T cell subsets in the tumor micro-environment expressing both activating and inhibitory receptors. These T_AI cells can be targeted by combined immunotherapy leading to improved survival.

@en

A bipartite graph is a powerful abstraction for modeling relationships between two collections. Visualizations of bipartite graphs allow users to understand the mutual relationships between the elements in the two collections, e.g., by identifying clusters of similarly connected elements. However, commonly-used visual representations do not scale for the analysis of large bipartite graphs containing tens of millions of vertices, often resorting to an a-priori clustering of the sets. To address this issue, we present the Who's-Active-On-What-Visualization (WAOW-Vis) that allows for multiscale exploration of a bipartite social-network without imposing an a-priori clustering. To this end, we propose to treat a bipartite graph as a high-dimensional space and we create the WAOW-Vis adapting the multiscale dimensionality-reduction technique HSNE. The application of HSNE for bipartite graph requires several modifications that form the contributions of this work. Given the nature of the problem, a set-based similarity is proposed. For efficient and scalable computations, we use compressed bitmaps to represent sets and we present a novel space partitioning tree to efficiently compute similarities; the Sets Intersection Tree. Finally, we validate WAOW-Vis on several datasets connecting Twitter-users and -streams in different domains: news, computer science and politics. We show how WAOW-Vis is particularly effective in identifying hierarchies of communities among social-media users.

@en

Functional imaging techniques provide radiobiological information that can be included into tumour control probability (TCP) models to enable individualized outcome predictions in radiotherapy. However, functional imaging and the derived radiobiological information are influenced by uncertainties, translating into variations in individual TCP predictions. In this study we applied a previously developed analytical tool to quantify dose and TCP uncertainty bands when initial cell density is estimated from MRI-based apparent diffusion coefficient maps of eleven patients. TCP uncertainty bands of 16% were observed at patient level, while dose variations bands up to 8 Gy were found at voxel level for an iso-TCP approach.@en

We present the Bladder Runner, a novel tool to enable detailed visual exploration and analysis of the impact of bladder shape variation on the accuracy of dose delivery, during the course of prostate cancer radiotherapy (RT). Our tool enables the investigation of individual patients and cohorts through the entire treatment process, and it can give indications of RT‐induced complications for the patient. In prostate cancer RT treatment, despite the design of an initial plan prior to dose administration, bladder toxicity remains very common. The main reason is that the dose is delivered in multiple fractions over a period of weeks, during which, the anatomical variation of the bladder – due to differences in urinary filling – causes deviations between planned and delivered doses. Clinical researchers want to correlate bladder shape variations to dose deviations and toxicity risk through cohort studies, to understand which specific bladder shape characteristics are more prone to side effects. This is currently done with Dose‐Volume Histograms (DVHs), which provide limited, qualitative insight. The effect of bladder variation on dose delivery and the resulting toxicity cannot be currently examined with the DVHs. To address this need, we designed and implemented the Bladder Runner, which incorporates visualization strategies in a highly interactive environment with multiple linked views. Individual patients can be explored and analyzed through the entire treatment period, while inter‐patient and temporal exploration, analysis and comparison are also supported. We demonstrate the applicability of our presented tool with a usage scenario, employing a dataset of 29 patients followed through the course of the treatment, across 13 time points. We conducted an evaluation with three clinical researchers working on the investigation of RT‐induced bladder toxicity. All participants agreed that Bladder Runner provides better understanding and new opportunities for the exploration and analysis of the involved cohort data.

@en

Technological advances in mass spectrometry imaging (MSI) have contributed to growing interest in 3D MSI. However, the large size of 3D MSI data sets has made their efficient analysis and visualization and the identification of informative molecular patterns computationally challenging. Hierarchical stochastic neighbor embedding (HSNE), a nonlinear dimensionality reduction technique that aims at finding hierarchical and multiscale representations of large data sets, is a recent development that enables the analysis of millions of data points, with manageable time and memory complexities. We demonstrate that HSNE can be used to analyze large 3D MSI data sets at full mass spectral and spatial resolution. To benchmark the technique as well as demonstrate its broad applicability, we have analyzed a number of publicly available 3D MSI data sets, recorded from various biological systems and spanning different mass-spectrometry ionization techniques. We demonstrate that HSNE is able to rapidly identify regions of interest within these large high-dimensionality data sets as well as aid the identification of molecular ions that characterize these regions of interest; furthermore, through clearly separating measurement artifacts, the HSNE analysis exhibits a degree of robustness to measurement batch effects, spatially correlated noise, and mass spectral misalignment.

@en

Innate lymphoid cells (ILCs) are abundant in mucosal tissues and involved in tissue homeostasis and barrier function. Although several ILC subsets have been identified, it is unknown if additional heterogeneity exists, and their differentiation pathways remain largely unclear. We applied mass cytometry to analyze ILCs in the human fetal intestine and distinguished 34 distinct clusters through a t-SNE-based analysis. A lineage (Lin)^-CD7⁺CD127^-CD45RO⁺CD56⁺ population clustered between the CD127⁺ ILC and natural killer (NK) cell subsets, and expressed diverse levels of Eomes, T-bet, GATA3, and RORγt. By visualizing the dynamics of the t-SNE computation, we identified smooth phenotypic transitions from cells within the Lin^-CD7⁺CD127^-CD45RO⁺CD56⁺ cluster to both the NK cells and CD127⁺ ILCs, revealing potential differentiation trajectories. In functional differentiation assays, the Lin^-CD7⁺CD127^-CD45RO⁺CD56⁺CD8a^- cells could develop into CD45RA⁺ NK cells and CD127⁺RORγt⁺ ILC3-like cells. Thus, we identified a previously unknown intermediate innate subset that can differentiate into ILC3 and NK cells.

@en