A probabilistic projection of sea-level rise uses a probability distribution to represent scientific uncertainty. However, alternative probabilistic projections of sea-level rise differ markedly, revealing ambiguity, which poses a challenge to scientific assessment and decision-making. To address the challenge of ambiguity, we propose a new approach to quantify a best estimate of the scientific uncertainty associated with sea-level rise. Our proposed fusion combines the complementary strengths of the ice sheet models and expert elicitations that were used in the Sixth Assessment Report (AR6) of the Intergovernmental Panel on Climate Change (IPCC). Under a low-emissions scenario, the fusion's very likely range (5th–95th percentiles) of global mean sea-level rise is 0.3–1.0 m by 2100. Under a high-emissions scenario, the very likely range is 0.5–1.9 m. The 95th percentile projection of 1.9 m can inform a high-end storyline, supporting decision-making for activities with low uncertainty tolerance. By quantifying a best estimate of scientific uncertainty, the fusion caters to diverse users.

@en

With the exponential growth of video data, individuals, particularly scholars in the fields of history and sociology, are increasingly reliant on video materials. However, the task of locating specific frames within videos remains a laborious and time-consuming endeavor. Advanced machine learning-assisted video processing techniques have emerged, including text-based video searches, video summarization, real-time object detection, and person re-identification. However, distinct from these, the main challenge of retrieving video frames based on given visual content is how to efficiently and accurately pinpoint the instance occurrences. To expedite the process while maintaining retrieval performance, we propose a two-stage approach, combining KeyFrame Extraction (KFE) and Content-based Image Retrieval (CBIR), underpinned a DNN-empowered framework called MoReSo. Our innovations include 1) the integration of improved statistical features with dynamic clustering in the KFE stage and 2) the development of the MoReSo framework, which consists of MobileNet and ResNet backbones with SOA layer to jointly represent video frames, achieving 2.67x increase in efficiency compared to existing solutions. Our framework is evaluated on two datasets: the annotated EHM Historical Database provided by digital history researchers and the widely-used image retrieval benchmark datasets, the Oxford and Paris datasets. The experimental results showcase that the proposed framework and scheme excel among other models in the CBVIR task. We make our code available for further exploration through our GitHub repository. This repository contains the implementation of our model and CBVIR system with a GUI prototype.

@en

Deep learning-based object detectors, while offering exceptional performance, are data-dependent and can suffer from generalization issues. In this work, we investigated deep neural networks for detecting people and medical instruments for the vision-based workflow analysis system inside Catheterization Laboratories (Cath Labs). The central problem explored in this paper is the fact that the performance of the detector can degrade drastically if it is trained and tested on data from different Cath Labs. Our research aimed to investigate the underlying causes of this specific performance degradation and find solutions to mitigate this issue. We employed the YOLOv8 object detector and created datasets from clinical procedures recorded at Reinier de Graaf Hospital (RdGG) and Philips Best Campus, supplemented with publicly accessible images. Through a series of experiments complemented by data visualization, we discovered that the performance degradation primarily stems from data distribution shifts in the feature space. Notably, the object detector trained on non-sensitive online images can generalize to unseen Cath Labs, outperforming the model trained on a procedure recording from a different Cath Lab. The detector trained on the online images achieved an mAP@0.5 of 0.517 on the RdGG dataset. Furthermore, by switching to the most suitable camera for each object in the Cath Lab, the multi-camera system can further improve the detection performance significantly. An aggregated L-camera mAP@0.5 of 0.679 is achieved for single-object classes on the RdGG dataset.

@en

This workshop aims to explore the potential of large generative models to revolutionize how we interact with multimodal information. A Large Language Model (LLM) represents a sophisticated form of artificial intelligence engineered to comprehend and produce natural language text, exemplified by technologies such as GPT, LLaMA, Flan-T5, ChatGLM, Qwen, etc. These models undergo training on extensive text datasets, exhibiting commendable attributes including robust language generation, zero-shot transfer capabilities, and In-Context Learning (ICL). With the surge in multimodal content-encompassing images, videos, audio, and 3D models-over the recent period, Large MultiModal Models (LMMs) have seen significant enhancements. These improvements enable the augmentation of conventional LLMs to accommodate multimodal inputs or outputs, as seen in BLIP, Flamingo, KOSMOS, LLaVA, Gemini, GPT-4, etc. Concurrently, certain research initiatives have developed specific modalities, with Kosmos2 and MiniGPT-5 focusing on image generation, and SpeechGPT on speech production. There are also endeavors to integrate LLMs with external tools to achieve a near "any-to-any" multimodal comprehension and generation capacity, illustrated by projects like Visual-ChatGPT, ViperGPT, MMREACT, HuggingGPT, and AudioGPT. Collectively,these models, spanning not only text and image generation but also other modalities, are referred to as large generative models. This workshop will allow researchers, practitioners, and industry professionals to explore the latest trends and best practices in the multimodal applications of large generative models.@en

Nowcasting leverages real-time atmospheric conditions to forecast weather over short periods. State-of-the-art models, including PySTEPS, encounter difficulties in accurately forecasting extreme weather events because of their unpredictable distribution patterns. In this study, we design a physics-informed neural network to perform precipitation nowcasting using the precipitation and meteorological data from the Royal Netherlands Meteorological Institute (KNMI). This model draws inspiration from the novel Physics-Informed Discriminator GAN (PID-GAN) formulation, directly integrating physics-based supervision within the adversarial learning framework. The proposed model adopts a GAN structure, featuring a Vector Quantization Generative Adversarial Network (VQ-GAN) and a Transformer as the generator, with a temporal discriminator serving as the discriminator. Our findings demonstrate that the PID-GAN model outperforms numerical and SOTA deep generative models in terms of precipitation nowcasting downstream metrics.

@en

Tide–surge interaction plays a substantial role in determining the characteristics of coastal water levels over shallow regions. We study the tide–surge interaction observed at seven tide gauges along Singapore and the east coast of Peninsular Malaysia, focusing on the timing of extreme non-tidal residuals relative to tidal high water. We propose a modified statistical framework using a no-tide–surge interaction (no-TSI) null distribution that accounts for asymmetry and variation in the duration of tidal cycles. We find that our modified framework can mitigate false-positive signals of tide–surge interaction in this region. We find evidence of tide–surge interaction at all seven locations, with characteristics varying smoothly along the coastline: the highest non-tidal residuals are found to occur most frequently before tidal high water in the south, both before and after tidal high water in the central region, and after tidal high water in the north. We also propose a semi-empirical model to investigate the effects of tidal-phase alteration, which is one mechanism of tide–surge interaction. Results of our semi-empirical model reveal that tidal-phase alteration caused by storm surges is substantial enough to generate significant change in the timing of extreme non-tidal residuals. To mitigate the effect of tidal-phase alteration on return level estimation, skew surge can be used. We conclude that (1) tide–surge interaction influences coastal water levels in this region, (2) our semi-empirical model provides insight into the mechanism of tidal-phase alteration, and (3) our no-TSI distribution should be used for similar studies globally.@en

Objectives: Current techniques in brain stimulation are still largely based on a phrenologic approach that a single brain target can treat a brain disorder. Nevertheless, meta-analyses of brain implants indicate an overall success rate of 50% improvement in 50% of patients, irrespective of the brain-related disorder. Thus, there is still a large margin for improvement. The goal of this manuscript is to 1) develop a general theoretical framework of brain functioning that is amenable to surgical neuromodulation, and 2) describe the engineering requirements of the next generation of implantable brain stimulators that follow from this theoretic model. Materials and Methods: A neuroscience and engineering literature review was performed to develop a universal theoretical model of brain functioning and dysfunctioning amenable to surgical neuromodulation. Results: Even though a single target can modulate an entire network, research in network science reveals that many brain disorders are the consequence of maladaptive interactions among multiple networks rather than a single network. Consequently, targeting the main connector hubs of those multiple interacting networks involved in a brain disorder is theoretically more beneficial. We, thus, envision next-generation network implants that will rely on distributed, multisite neuromodulation targeting correlated and anticorrelated interacting brain networks, juxtaposing alternative implant configurations, and finally providing solid recommendations for the realization of such implants. In doing so, this study pinpoints the potential shortcomings of other similar efforts in the field, which somehow fall short of the requirements. Conclusion: The concept of network stimulation holds great promise as a universal approach for treating neurologic and psychiatric disorders.

@en

We propose a manager-worker framework (the implementation of our model is publically available at: https://github.com/zcaicaros/manager-worker-mtsptwr) based on deep reinforcement learning to tackle a hard yet nontrivial variant of Travelling Salesman Problem (TSP), i.e. multiple-vehicle TSP with time window and rejections (mTSPTWR), where customers who cannot be served before the deadline are subject to rejections. Particularly, in the proposed framework, a manager agent learns to divide mTSPTWR into sub-routing tasks by assigning customers to each vehicle via a Graph Isomorphism Network (GIN) based policy network. A worker agent learns to solve sub-routing tasks by minimizing the cost in terms of both tour length and rejection rate for each vehicle, the maximum of which is then fed back to the manager agent to learn better assignments. Experimental results demonstrate that the proposed framework outperforms strong baselines in terms of higher solution quality and shorter computation time. More importantly, the trained agents also achieve competitive performance for solving unseen larger instances.

@en

Even though virtual testing of Autonomous Vehicles (AVs) has been well recognized as essential for safety assessment, AV simulators are still undergoing active development. One particular challenge is the problem of including the Sensing and Perception (S&P) subsystem into the virtual simulation loop in an efficient and effective manner. In this article, we define Perception Error Models (PEM), a virtual simulation component that can enable the analysis of the impact of perception errors on AV safety, without the need to model the sensors themselves. We propose a generalized data-driven procedure towards parametric modeling and evaluate it using Apollo, an open-source driving software, and nuScenes, a public AV dataset. Additionally, we implement PEMs in SVL, an open-source vehicle simulator. Furthermore, we demonstrate the usefulness of PEM-based virtual tests, by evaluating camera, LiDAR, and camera-LiDAR setups. Our virtual tests highlight limitations in the current evaluation metrics, and the proposed approach can help study the impact of perception errors on AV safety.

@en

Global climate models are susceptible to drift, causing spurious trends in output variables. Drift is often corrected using data from a control simulation. However, internal climate variability within the control simulation introduces uncertainty to the drift correction process. To quantify this drift uncertainty, we develop a probabilistic technique: Monte Carlo drift correction (MCDC). MCDC samples the standard error associated with drift in the control time series. We apply MCDC to an ensemble of global climate models from the Coupled Model Intercomparison Project Phase 6 (CMIP6). We find that drift correction partially addresses a problem related to drift: energy leakage. Nevertheless, the energy balance of several models remains suspect. We quantify the drift uncertainty of global quantities associated with the Earth's energy balance and thermal expansion of the ocean. When correcting drift in a cumulatively integrated energy flux, we find that it is preferable to integrate the flux before correcting the drift: An alternative method would be to correct the bias before integrating the flux, but this alternative method amplifies the drift uncertainty. Assuming that drift is linear likely leads to an underestimation of drift uncertainty. Time series with weak trends may be especially susceptible to drift uncertainty: for historical thermosteric sea level rise since the 1850s, the drift uncertainty can range from 3 to 24 mm, which is of comparable magnitude to the impact of omitting volcanic forcing in control simulations. Derived coefficients-such as the ocean's expansion efficiency of heat-can also be susceptible to drift uncertainty. When evaluating and analysing global climate model data that are susceptible to drift, researchers should consider drift uncertainty.

@en

Nowcasting is an observation-based method that uses the current state of the atmosphere to forecast future weather conditions over several hours. Recent studies have shown the promising potential of using deep learning models for precipitation nowcasting. In this paper, novel deep generative models are proposed for precipitation nowcasting. These models are equipped with extreme-value losses to more reliably predict extreme precipitation events. The proposed deep generative model contains a Vector Quantization Generative Adversarial Network and a Transformer ("VQGAN + Transformer"). For enhanced modeling and forecasting of extreme events, Extreme Value Loss (EVL) is incorporated in the autore-gressive Transformer. The numerical results show that the proposed model achieves comparable performance with the state-of-the-art conventional nowcasting method PySTEPS for predicting nominal values. By incorporating an EVL, the proposed model yields more accurate nowcasting of extreme precipitation.@en

Neurologists typically identify epileptic seizures from electroencephalograms (EEGs) by visual inspection. This process is often time-consuming, especially for EEG recordings that last hours or days. To expedite the process, a reliable, automated, and patient-independent seizure detector is essential. However, developing a patient-independent seizure detector is challenging as seizures exhibit diverse characteristics across patients and recording devices. In this study, we propose a patient-independent seizure detector to automatically detect seizures in both scalp EEG and intracranial EEG (iEEG). First, we deploy a convolutional neural network with transformers and belief matching loss to detect seizures in single-channel EEG segments. Next, we extract regional features from the channel-level outputs to detect seizures in multi-channel EEG segments. At last, we apply post-processing filters to the segment-level outputs to determine seizures' start and end points in multi-channel EEGs. Finally, we introduce the minimum overlap evaluation scoring as an evaluation metric that accounts for minimum overlap between the detection and seizure, improving upon existing assessment metrics. We trained the seizure detector on the Temple University Hospital Seizure (TUH-SZ) dataset and evaluated it on five independent EEG datasets. We evaluate the systems with the following metrics: sensitivity (SEN), precision (PRE), and average and median false positive rate per hour (aFPR/h and mFPR/h). Across four adult scalp EEG and iEEG datasets, we obtained SEN of 0.617-1.00, PRE of 0.534-1.00, aFPR/h of 0.425-2.002, and mFPR/h of 0-1.003. The proposed seizure detector can detect seizures in adult EEGs and takes less than 15 s for a 30 min EEG. Hence, this system could aid clinicians in reliably identifying seizures expeditiously, allocating more time for devising proper treatment.

@en

Electronic nose (eNose) technology is an emerging diagnostic application, using artificial intelligence to classify human breath patterns. These patterns can be used to diagnose medical conditions. Sarcoidosis is an often difficult to diagnose disease, as no standard procedure or conclusive test exists. An accurate diagnostic model based on eNose data could therefore be helpful in clinical decision-making. The aim of this paper is to evaluate the performance of various dimensionality reduction methods and classifiers in order to design an accurate diagnostic model for sarcoidosis. Various methods of dimensionality reduction and multiple hyperparameter optimised classifiers were tested and cross-validated on a dataset of patients with pulmonary sarcoidosis (n= 224) and other interstitial lung disease (n= 317). Best performing methods were selected to create a model to diagnose patients with sarcoidosis. Nested cross-validation was applied to calculate the overall diagnostic performance. A classification model with feature selection and random forest (RF) classifier showed the highest accuracy. The overall diagnostic performance resulted in an accuracy of 87.1% and area-under-the-curve of 91.2%. After comparing different dimensionality reduction methods and classifiers, a highly accurate model to diagnose a patient with sarcoidosis using eNose data was created. The RF classifier and feature selection showed the best performance. The presented systematic approach could also be applied to other eNose datasets to compare methods and select the optimal diagnostic model.

@en

In the last decade, the signal processing (SP) community has witnessed a paradigm shift from model-based to data-driven methods. Machine learning (ML) - more specifically, deep learning - methodologies are nowadays widely used in all SP fields, e.g., audio, speech, image, video, multimedia, and multimodal/multisensor processing, to name a few. Many data-driven methods also incorporate domain knowledge to improve problem modeling, especially when computational burden, training data scarceness, and memory size are important constraints.

@en

Automotive perception involves understanding the external driving environment and the internal state of the vehicle cabin and occupants using sensor data. It is critical to achieving high levels of safety and autonomy in driving. This article provides an overview of different sensor modalities, such as cameras, radars, and light detection and ranging (LiDAR) used commonly for perception, along with the associated data processing techniques. Critical aspects of perception are considered, such as architectures for processing data from single or multiple sensor modalities, sensor data processing algorithms and the role of machine learning techniques, methodologies for validating the performance of perception systems, and safety. The technical challenges for each aspect are analyzed, emphasizing machine learning approaches, given their potential impact on improving perception. Finally, future research opportunities in automotive perception for their wider deployment are outlined.

@en

To enhance the reliability and robustness of language identification (LID) and language diarization (LD) systems for heterogeneous populations and scenarios, there is a need for speech processing models to be trained on datasets that feature diverse language registers and speech patterns. We present the MERLIon CCS challenge, featuring a first-of-its-kind Zoom video call dataset of parent-child shared book reading, of over 30 hours with over 300 recordings, annotated by multilingual transcribers using a high-fidelity linguistic transcription protocol. The audio corpus features spontaneous and in-the-wild English-Mandarin code-switching, child-directed speech in non-standard accents with diverse language-mixing patterns recorded in a variety of home environments. This report describes the corpus, as well as LID and LD results for our baseline and several systems submitted to the MERLIon CCS challenge using the corpus.

@en

Language development experts need tools that can automatically identify languages from fluent, conversational speech and provide reliable estimates of usage rates at the level of an individual recording. However, LID systems are typically evaluated on metrics such as equal error rate and balanced accuracy, applied at the level of an entire speech corpus. These overview metrics do not provide information about model performance at the level of individual speakers, recordings, or units of speech with different linguistic characteristics. Overview statistics may mask systematic errors in model performance for some subsets of the data, and consequently, have worse performance on data derived from some subsets of human speakers, creating a kind of algorithmic bias. Here, we investigate how well a number of LID systems perform on individual recordings and speech units with different linguistic properties in the MERLIon CCS Challenge featuring accented code-switched child-directed speech.

@en