In various medical and biological modalities, in particular, electron microscopy (EM), visualization of large fields of view requires acquisition of multiple overlapping frames with their subsequent reconstruction into a single panoramic image. Such reconstruction process is hampered by several factors, including different intensity scaling and imperfect localization of the acquired frames, intensity inhomogeneity within each frame, and large content variability between different frames. This poses a significant challenge not only for visualization, but also for further quantification of such panoramic images. In this work, we present a simple yet efficient data-driven algorithm that improves reconstruction of the large panoramic views using a minimal set of assumptions. More precisely, our approach fully relies on the information from the overlap regions of the neighbouring frames. Such formulation results in a linear system of equations that can be solved numerically, when supported by proper constraints. We validated our approach on a large set of highly-diverse in-house EM panoramic views and demonstrated improved performance with respect to traditional metrics as well as network training capacity.

Exploration and analysis of high-dimensional data are important tasks in many fields that produce large and complex data, like the financial sector, systems biology, or cultural heritage. Tailor-made visual analytics software is developed for each specific application, limiting their applicability in other fields. However, as diverse as these fields are, their characteristics and requirements for data analysis are conceptually similar. Many applications share abstract tasks and data types and are often constructed with similar building blocks. Developing such applications, even when based mostly on existing building blocks, requires significant engineering efforts. We developed ManiVault, a flexible and extensible open-source visual analytics framework for analyzing high-dimensional data. The primary objective of ManiVault is to facilitate rapid prototyping of visual analytics workflows for visualization software developers and practitioners alike. ManiVault is built using a plugin-based architecture that offers easy extensibility. While our architecture deliberately keeps plugins self-contained, to guarantee maximum flexibility and re-usability, we have designed and implemented a messaging API for tight integration and linking of modules to support common visual analytics design patterns. We provide several visualization and analytics plugins, and ManiVault's API makes the integration of new plugins easy for developers. ManiVault facilitates the distribution of visualization and analysis pipelines and results for practitioners through saving and reproducing complete application states. As such, ManiVault can be used as a communication tool among researchers to discuss workflows and results. A copy of this paper and all supplemental material is available at osf.io/9k6jw, and source code at github.com/ManiVaultStudio.

RNA Velocity allows the inference of cellular differentiation trajectories from single-cell RNA sequencing (scRNA-seq) data. It would be highly interesting to study these differentiation dynamics in the spatial context of tissues. Estimating spatial RNA velocities is, however, limited by the inability to spatially capture spliced and unspliced mRNA molecules in high-resolution spatial transcriptomics. We present SIRV, a method to spatially infer RNA velocities at the single-cell resolution by enriching spatial transcriptomics data with the expression of spliced and unspliced mRNA from reference scRNA-seq data. We used SIRV to infer spatial differentiation trajectories in the developing mouse brain, including the differentiation of midbrain-hindbrain boundary cells and marking the forebrain origin of the cortical hem and diencephalon cells. Our results show that SIRV reveals spatial differentiation patterns not identifiable with scRNA-seq data alone. Additionally, we applied SIRV to mouse organogenesis data and obtained robust spatial differentiation trajectories. Finally, we verified the spatial RNA velocities obtained by SIRV using 10x Visium data of the developing chicken heart and MERFISH data from human osteosarcoma cells. Altogether, SIRV allows the inference of spatial RNA velocities at the single-cell resolution to facilitate studying tissue development.

With the advent of multiplex fluorescence in situ hybridization (FISH) and in situ RNA sequencing technologies, spatial transcriptomics analysis is advancing rapidly, providing spatial location and gene expression information about cells in tissue sections at single cell resolution. Cell type classification of these spatially-resolved cells can be inferred by matching the spatial transcriptomics data to reference atlases derived from single cell RNA-sequencing (scRNA-seq) in which cell types are defined by differences in their gene expression profiles. However, robust cell type matching of the spatially-resolved cells to reference scRNA-seq atlases is challenging due to the intrinsic differences in resolution between the spatial and scRNA-seq data. In this study, we systematically evaluated six computational algorithms for cell type matching across four image-based spatial transcriptomics experimental protocols (MERFISH, smFISH, BaristaSeq, and ExSeq) conducted on the same mouse primary visual cortex (VISp) brain region. We find that many cells are assigned as the same type by multiple cell type matching algorithms and are present in spatial patterns previously reported from scRNA-seq studies in VISp. Furthermore, by combining the results of individual matching strategies into consensus cell type assignments, we see even greater alignment with biological expectations. We present two ensemble meta-analysis strategies used in this study and share the consensus cell type matching results in the Cytosplore Viewer (https://viewer.cytosplore.org) for interactive visualization and data exploration. The consensus matching can also guide spatial data analysis using SSAM, allowing segmentation-free cell type assignment.

The cognitive abilities of humans are distinctive among primates, but their molecular and cellular substrates are poorly understood. We used comparative single-nucleus transcriptomics to analyze samples of the middle temporal gyrus (MTG) from adult humans, chimpanzees, gorillas, rhesus macaques, and common marmosets to understand human-specific features of the neocortex. Human, chimpanzee, and gorilla MTG showed highly similar cell-type composition and laminar organization as well as a large shift in proportions of deep-layer intratelencephalic-projecting neurons compared with macaque and marmoset MTG. Microglia, astrocytes, and oligodendrocytes had more-divergent expression across species compared with neurons or oligodendrocyte precursor cells, and neuronal expression diverged more rapidly on the human lineage. Only a few hundred genes showed human-specific patterning, suggesting that relatively few cellular and molecular changes distinctively define adult human cortical structure.

Characterizing cellular diversity at different levels of biological organization and across AU data: Ple modalities is a prerequisite to understanding the function of cell types in the brain. Classification of neurons is also essential to manipulate cell types in controlled ways and to understand their variation and vulnerability in brain disorders. The BRAIN Initiative Cell Census Network (BICCN) is an integrated network of data-generating centers, data archives, and data standards developers, with the goal of systematic multimodal brain cell type profiling and characterization. Emphasis of the BICCN is on the whole mouse brain with demonstration of prototype feasibility for human and nonhuman primate (NHP) brains. Here, we provide a guide to the cellular and spatial approaches employed by the BICCN, and to accessing and using these data and extensive resources, including the BRAIN Cell Data Center (BCDC), which serves to manage and integrate data across the ecosystem. We illustrate the power of the BICCN data ecosystem through vignettes highlighting several BICCN analysis and visualization tools. Finally, we present emerging standards that have been developed or adopted toward Findable, Accessible, Interoperable, and Reusable (FAIR) neuroscience. The combined BICCN ecosystem provides a comprehensive resource for the exploration and analysis of cell types in the brain.

In spatial transcriptomics (ST) data, biologically relevant features such as tissue compartments or cell-state transitions are reflected by gene expression gradients. Here, we present SpaceWalker, a visual analytics tool for exploring the local gradient structure of 2D and 3D ST data. The user can be guided by the local intrinsic dimensionality of the high-dimensional data to define seed locations, from which a flood-fill algorithm identifies transcriptomically similar cells on the fly, based on the high-dimensional data topology. In several use cases, we demonstrate that the spatial projection of these flooded cells highlights tissue architectural features and that interactive retrieval of gene expression gradients in the spatial and transcriptomic domains confirms known biology. We also show that SpaceWalker generalizes to several different ST protocols and scales well to large, multi-slice, 3D whole-brain ST data while maintaining real-time interaction performance.

Chronic intestinal inflammation underlies inflammatory bowel disease (IBD). Previous studies indicated alterations in the cellular immune system; however, it has been challenging to interrogate the role of all immune cell subsets simultaneously. Therefore, we aimed to identify immune cell types associated with inflammation in IBD using high-dimensional mass cytometry. We analyzed 188 intestinal biopsies and paired blood samples of newly-diagnosed, treatment-naive patients (n=42) and controls (n=26) in two independent cohorts. We applied mass cytometry (36-antibody panel) to resolve single cells and analyzed the data with unbiased Hierarchical-SNE. In addition, imaging-mass cytometry (IMC) was performed to reveal the spatial distribution of the immune subsets in the tissue. We identified 44 distinct immune subsets. Correlation network analysis identified a network of inflammation-associated subsets, including HLA-DR⁺CD38⁺ EM CD4⁺ T cells, T regulatory-like cells, PD1⁺ EM CD8⁺ T cells, neutrophils, CD27⁺ TCRγδ cells and NK cells. All disease-associated subsets were validated in a second cohort. This network was abundant in a subset of patients, independent of IBD subtype, severity or intestinal location. Putative disease-associated CD4⁺ T cells were detectable in blood. Finally, imaging-mass cytometry revealed the spatial colocalization of neutrophils, memory CD4⁺ T cells and myeloid cells in the inflamed intestine. Our study indicates that a cellular network of both innate and adaptive immune cells colocalizes in inflamed biopsies from a subset of patients. These results contribute to dissecting disease heterogeneity and may guide the development of targeted therapeutics in IBD.

BACKGROUND: Pancreatic ductal adenocarcinoma (PDAC) is a highly lethal malignancy in need of effective (immuno)therapeutic treatment strategies. For the optimal application and development of cancer immunotherapies, a comprehensive understanding of local and systemic immune profiles in patients with PDAC is required. Here, our goal was to decipher the interplay between local and systemic immune profiles in treatment-naïve patients with PDAC. METHODS: The immune composition of PDAC, matched non-malignant pancreatic tissue, regional lymph nodes, spleen, portal vein blood, and peripheral blood samples (collected before and after surgery) from 11 patients with PDAC was assessed by measuring 41 immune cell markers by single-cell mass cytometry. Furthermore, the activation potential of tumor-infiltrating lymphocytes as determined by their ability to produce cytokines was investigated by flow cytometry. In addition, the spatial localization of tumor-infiltrating innate lymphocytes in the tumor microenvironment was confirmed by multispectral immunofluorescence. RESULTS: We found that CD103+CD8+ T cells with cytotoxic potential are infrequent in the PDAC immune microenvironment and lack the expression of activation markers and checkpoint blockade molecule programmed cell death protein-1 (PD-1). In contrast, PDAC tissues showed a remarkable increased relative frequency of B cells and regulatory T cells as compared with non-malignant pancreatic tissues. Besides, a previously unappreciated innate lymphocyte cell (ILC) population (CD127-CD103+CD39+CD45RO+ ILC1-like) was discovered in PDAC tissues. Strikingly, the increased relative frequency of B cells and regulatory T cells in pancreatic cancer samples was reflected in matched portal vein blood samples but not in peripheral blood, suggesting a regional enrichment of immune cells that infiltrate the PDAC microenvironment. After surgery, decreased frequencies of myeloid dendritic cells were found in peripheral blood. CONCLUSIONS: Our work demonstrates an immunosuppressive landscape in PDAC tissues, generally deprived of cytotoxic T cells and enriched in regulatory T cells and B cells. The antitumor potential of ILC1-like cells in PDAC may be exploited in a therapeutic setting. Importantly, immune profiles detected in blood isolated from the portal vein reflected the immune cell composition of the PDAC microenvironment, suggesting that this anatomical location could be a source of tumor-associated immune cell subsets.

In the version of this article initially published, the Acknowledgements section was incomplete and has now been amended to include the following: “NIH BRAIN Initiative awards U01 MH121282 to J.R.E and M.M.B, U19 MH114831 to J.R.E. and E.M.C., U19 MH114830 to H.Z., U01 MH114819 to G.F., 1U01MH114828 to K.Z. and J.C., RF1MH123220 to M.H. and R.H.S., and U19 MH114821. NIH awards R01DC019370 to R.H., R24MH114815 to R.H. and O.R.W., and R24 MH114788 to O.R.W. Nancy and Buster Alvord Endowment to C.D.K.” The changes have been made to the HTML and PDF versions of the article.

High-dimensional imaging is becoming increasingly relevant in many fields from astronomy and cultural heritage to systems biology. Visual exploration of such high-dimensional data is commonly facilitated by dimensionality reduction. However, common dimensionality reduction methods do not include spatial information present in images, such as local texture features, into the construction of low-dimensional embeddings. Consequently, exploration of such data is typically split into a step focusing on the attribute space followed by a step focusing on spatial information, or vice versa. In this paper, we present a method for incorporating spatial neighborhood information into distance-based dimensionality reduction methods, such as t-Distributed Stochastic Neighbor Embedding (t-SNE). We achieve this by modifying the distance measure between high-dimensional attribute vectors associated with each pixel such that it takes the pixel's spatial neighborhood into account. Based on a classification of different methods for comparing image patches, we explore a number of different approaches. We compare these approaches from a theoretical and experimental point of view. Finally, we illustrate the value of the proposed methods by qualitative and quantitative evaluation on synthetic data and two real-world use cases.

Purpose: To develop automated vestibular schwannoma measurements on contrast-enhanced T1-and T2-weighted MRI scans. Materials and Methods: MRI data from 214 patients in 37 different centers were retrospectively analyzed between 2020 and 2021. Patients with hearing loss (134 positive for vestibular schwannoma [mean age 6 SD, 54 years 6 12; 64 men] and 80 negative for vestibular schwannoma) were randomly assigned to a training and validation set and to an independent test set. A convolutional neural network (CNN) was trained using fivefold cross-validation for two models (T1 and T2). Quantitative analysis, including Dice index, Hausdorff distance, surface-to-surface distance (S2S), and relative volume error, was used to compare the computer and the human delineations. An observer study was performed in which two experienced physicians evaluated both delineations. Results: The T1-weighted model showed state-of-the-art performance, with a mean S2S distance of less than 0.6 mm for the whole tumor and the intrameatal and extrameatal tumor parts. The whole tumor Dice index and Hausdorff distance were 0.92 and 2.1 mm in the independent test set, respectively. T2-weighted images had a mean S2S distance less than 0.6 mm for the whole tumor and the intrameatal and extrameatal tumor parts. The whole tumor Dice index and Hausdorff distance were 0.87 and 1.5 mm in the independent test set. The observer study indicated that the tool was similar to human delineations in 85%–92% of cases. Conclusion: The CNN model detected and delineated vestibular schwannomas accurately on contrast-enhanced T1-and T2-weighted MRI scans and distinguished the clinically relevant difference between intrameatal and extrameatal tumor parts.

Controlled human infections provide opportunities to study the interaction between the immune system and malaria parasites, which is essential for vaccine development. Here, we compared immune signatures of malaria-naive Europeans and of Africans with lifelong malaria exposure using mass cytometry, RNA sequencing and data integration, before and 5 and 11 days after venous inoculation with Plasmodium falciparum sporozoites. We observed differences in immune cell populations, antigen-specific responses and gene expression profiles between Europeans and Africans and among Africans with differing degrees of immunity. Before inoculation, an activated/differentiated state of both innate and adaptive cells, including elevated CD161⁺CD4⁺ T cells and interferon-γ production, predicted Africans capable of controlling parasitemia. After inoculation, the rapidity of the transcriptional response and clusters of CD4⁺ T cells, plasmacytoid dendritic cells and innate T cells were among the features distinguishing Africans capable of controlling parasitemia from susceptible individuals. These findings can guide the development of a vaccine effective in malaria-endemic regions.

The primary motor cortex (M1) is essential for voluntary fine-motor control and is functionally conserved across mammals¹. Here, using high-throughput transcriptomic and epigenomic profiling of more than 450,000 single nuclei in humans, marmoset monkeys and mice, we demonstrate a broadly conserved cellular makeup of this region, with similarities that mirror evolutionary distance and are consistent between the transcriptome and epigenome. The core conserved molecular identities of neuronal and non-neuronal cell types allow us to generate a cross-species consensus classification of cell types, and to infer conserved properties of cell types across species. Despite the overall conservation, however, many species-dependent specializations are apparent, including differences in cell-type proportions, gene expression, DNA methylation and chromatin state. Few cell-type marker genes are conserved across species, revealing a short list of candidate genes and regulatory mechanisms that are responsible for conserved features of homologous cell types, such as the GABAergic chandelier cells. This consensus transcriptomic classification allows us to use patch–seq (a combination of whole-cell patch-clamp recordings, RNA sequencing and morphological characterization) to identify corticospinal Betz cells from layer 5 in non-human primates and humans, and to characterize their highly specialized physiology and anatomy. These findings highlight the robust molecular underpinnings of cell-type diversity in M1 across mammals, and point to the genes and regulatory pathways responsible for the functional identity of cell types and their species-specific adaptations.

Single-cell genomics is rapidly advancing our knowledge of the diversity of cell phenotypes, including both cell types and cell states. Driven by single-cell/-nucleus RNA sequencing (scRNA-seq), comprehensive cell atlas projects characterizing a wide range of organisms and tissues are currently underway. As a result, it is critical that the transcriptional phenotypes discovered are defined and disseminated in a consistent and concise manner. Molecular biomarkers have historically played an important role in biological research, from defining immune cell types by surface protein expression to defining diseases by their molecular drivers. Here, we describe a machine learning-based marker gene selection algorithm, NS-Forest version 2.0, which leverages the nonlinear attributes of random forest feature selection and a binary expression scoring approach to discover the minimal marker gene expression combinations that optimally capture the cell type identity represented in complete scRNA-seq transcriptional profiles. The marker genes selected provide an expression barcode that serves as both a useful tool for downstream biological investigation and the necessary and sufficient characteristics for semantic cell type definition. The use of NS-Forest to identify marker genes for human brain middle temporal gyrus cell types reveals the importance of cell signaling and noncoding RNAs in neuronal cell type identity.

Objective: To visualize the encoding capability of magnetic resonance fingerprinting (MRF) dictionaries. Materials and methods: High-dimensional MRF dictionaries were simulated and embedded into a lower-dimensional space using t-distributed stochastic neighbor embedding (t-SNE). The embeddings were visualized via colors as a surrogate for location in low-dimensional space. First, we illustrate this technique on three different MRF sequences. We then compare the resulting embeddings and the color-coded dictionary maps to these obtained with a singular value decomposition (SVD) dimensionality reduction technique. We validate the t-SNE approach with measures based on existing quantitative measures of encoding capability using the Euclidean distance. Finally, we use t-SNE to visualize MRF sequences resulting from an MRF sequence optimization algorithm. Results: t-SNE was able to show clear differences between the color-coded dictionary maps of three MRF sequences. SVD showed smaller differences between different sequences. These findings were confirmed by quantitative measures of encoding. t-SNE was also able to visualize differences in encoding capability between subsequent iterations of an MRF sequence optimization algorithm. Discussion: This visualization approach enables comparison of the encoding capability of different MRF sequences. This technique can be used as a confirmation tool in MRF sequence optimization.