Introduction. Ultrasound (US) and diffuse reflectance spectroscopy (DRS) hold significant potential to enhance intra-operative decision-making by providing real-time feedback on surgical resection margins. This is particularly valuable for cancers with complex presentations, such as soft tissue sarcoma (STS), where reducing incomplete (R1) resections can improve patient outcomes.
Aim. This thesis aims to develop resection margin assessment models that leverage US and DRS for automated tumor margin (TM) predictions in ex vivo STS specimens.
Method. A high-quality multi-modal dataset of STS was acquired with a hybrid US-DRS probe. Three artificial intelligence (AI) models were developed: a segmentation model to delineate STS in US images, a classification model to differentiate tissue types in STS using DRS spectra, and a multi-modal regression model combining US and DRS data for TM predictions. The potential added value of Explainable AI (XAI) was explored through a qualitative survey with oncological surgeons.
Results. The dataset included 302 measurement locations from 49 patients, encompassing 17 STS subtypes. The US segmentation model achieved a mean TM prediction accuracy of 0.72 mm and a mean Dice score of 0.97 on an unseen test set. The DRS classifier, leveraging data from six different sampling depths, achieved a tumor detection sensitivity of 0.93 and specificity of 0.86. The multi-modal model, integrating outputs from the segmentation and classification models, demonstrated a mean TM prediction accuracy of 0.57 mm. The survey revealed key requirements for a clinically valuable user interface and showed that XAI integration enhanced model transparency and user trust.
Conclusion. This thesis developed segmentation and classification models for intra-operative resection margin assessments in STS specimens, achieving strong performances compared to existing research. Additionally, a novel multi-modal approach combining US and DRS significantly enhanced the accuracy of TM predictions, highlighting its potential to reduce R1 resections and improve surgical outcomes. Collaboration with clinical end-users ensured the clinical relevance and usability of the proposed model interface, paving the way for integration into surgical practice and advancing oncological care.

Introduction. Point-of-care ultrasound (POCUS) devices are gaining popu-larity due to their portability and affordability, making ultrasound technology more accessible in various medical settings. However, these benefits of cost and portability come with a trade-off in imaging quality. Aim. This thesis aims to enhance the image quality of POCUS devices using deep learning and a novel paired dataset of POCUS and high-end ultrasound images. Method. First, an accurately paired dataset was created, comprising ex vivo and abdominal phantom images from both POCUS and high-end ultrasound systems. This was achieved by building an automated acquisition setup to en-sure consistent capture locations, along with comprehensive image alignment and registration steps. Second, a deep learning network was developed to en-hance POCUS image quality. A conditional Generative Adversarial Network (cGAN) with a U-Net generator, pretrained on simulation data, was trained and evaluated using this paired dataset. Results. A paired dataset of 1064 images was collected. The proposed cGAN achieved significant improvements in image quality over low-quality input im-ages, increasing the Structural Similarity Index Measure (SSIM) from 0.286 ± 0.062 to 0.540 ± 0.082 and Peak Signal-to-Noise Ratio (PSNR) from 19.155 ± 1.948 dB to 22.406 ± 2.189 dB. It also reduced the Natural Image Quality Eval-uator (NIQE) from 7.948 ± 1.772 to 4.436 ± 0.528 and Perception-based Image Quality Evaluator (PIQE) from 31.116 ± 5.911 to 19.991 ± 5.722, where lower scores indicate higher quality. Additionally, qualitative assessment showed that end-users preferred the enhanced images over the original low-quality images. Conclusions. This thesis presents the first accurately paired dataset of POCUS and high-end ultrasound images. Additionally, the developed cGAN success-fully enhanced the quality of POCUS images, surpassing the results reported in similar studies. This work demonstrates the potential for reliable quality enhancement methods for POCUS while preserving its cost and portability benefits, ultimately increasing its value and impact in the medical field.

Introduction. Three-dimensional (3D) ultrasound (US) offers significant potential to enhance diagnostic imaging or intraoperative guidance by providing comprehensive volumetric insights in a noninvasive and cost-effective manner. However, existing methods for 3D reconstruction often rely on external tracking devices or specialized 3D transducers, which are costly and less suited for intraoperative and point-of-care settings.
Aim. This thesis aims to advance trackerless 3D US reconstruction, leveraging a point-of-care handheld ultrasound (POCUS) probe integrated with an inertial measurement unit (IMU) and a novel deep learning architecture. Method. Method. A high-quality dataset was acquired using a custom setup that included a POCUS probe with an integrated IMU, facilitating precise positional tracking and controlled movement of complex random motion trajectories. A CNN-Transformer network was developed, leveraging 2D US images and optical flow, to predict both local and global transformation parameters utilized for 3D US reconstruction. Ablation experiments were conducted to optimize model performance.
Results. A dataset comprising 361 US sweeps from ex-vivo surgical specimens and phantom models was collected. The optimized model, integrating IMU orientation data and applying sequence augmentation, achieved a mean Final Drift Ratio (FDR) of 11.63 ± 8.63% on an unseen test set, with a median FDR of 8.11%. Quantitative evaluations demonstrated that the model accurately captured the shape of the sweeps, particularly in translations along the x-axis, y-axis, and rotation around the x-axis. The predicted reconstructions enabled correct segmentation and visualization of anatomical structures in 3D, crucial for application in clinical settings.
Conclusion. This thesis presents a novel approach for 3D US reconstruction without external tracking devices, utilizing an integrated IMU and a CNN-Transformer network. The results demonstrate
competitive performance with state-of-the-art methods, highlighting the feasibility and potential of this approach for applications in diagnostic imaging, surgical planning, and intraoperative guidance, advancing 3D US toward clinical integration and improved patient outcomes.

Gastrointestinal stromal tumours (GISTs) are rare mesenchymal neoplasms with a worldwide incidence of one or two per 100,000. The tumours affect the entire gastrointestinal tract, but most commonly the stomach and small intestine. Neoadjuvant tyrosine kinase inhibitors (TKI) are administered in a selection of GIST patients to attain size reduction of the primary tumour and improve chances of complete resection. Response monitoring in GISTs is complex due to the presence of intra- and intertumoral heterogeneity and lack of pathological criteria to define response. Nonetheless, it is of importance to evaluate the efficacy of TKI treatment at an early stage in order to optimise treatment. In particular, early cessation of ineffective treatment is of importance in these patients, preventing unnecessary side-effects and healthcare costs. Medical imaging plays an important role to non-invasively predict and monitor treatment response of GIST patients undergoing TKI-treatment. This thesis aims to improve understanding of contrast-enhanced computed tomography (CE-CT) and 2-deoxy-2-[18F]fluoro-D-glucose ([18F]FDG) PET/CT imaging parameters to allow prediction and monitoring of neoadjuvant treatment response in GIST patients.
In Chapter 1, the added value of CE-CT and [18F]FDG-PET imaging for early prediction and monitoring of treatment response in GISTs is investigated by means of a systematic literature review. Results of this study show that heterogeneous enhancement patterns on baseline CE-CT imaging were considered to be predictive for high-risk GISTs, reflecting neovascularisation and the presence of necrosis. Current CE-CT radiographic response criteria (i.e., RECIST 1.1 and Choi) are still lacking sensitivity and are prone to errors when predicting or monitoring treatment response. Metabolic changes on [18F]FDG-PET imaging seem to precede morphological changes in size in GIST lesions and were more strongly correlated with tumour response. Although CE-CT and [18F]FDG-PET can aid in the prediction and monitoring in GIST patients, further research on cost-effectiveness is recommended.
Chapter 2 evaluates the efficacy of current radiological response criteria (RECIST 1.1, Choi and tumour volumetry) in predicting response to neoadjuvant systemic therapy, by comparing radiological response criteria with the achieved surgical benefit. Results show that size-based criteria (RECIST 1.1 and volumetry) accurately reflect surgical benefit in GIST patients treated with neoadjuvant systemic therapy (accuracy of 76.3% and 86.6% respectively) and are less prone to scanner and imaging protocol variabilities, when compared to the Choi criteria (68.4%).
In addition to volumetry, quantitative radiomic models using CE-CT and [18F]FDG-PET imaging features were trained to predict response at baseline. Preliminary results of this study are described in Chapter 3. The radiomic models presented in this study generally had a poor performance and can therefore not yet be applied in a clinical setting. To improve performance and generalisability, future research should focus on a bigger patient population and harmonisation of acquisition protocols. The conclusion of this thesis supports the utilisation of tumour diameters for radiological response assessment, where RECIST 1.1 response criteria had an accuracy of 80.0% to correctly predict volumetric response after the first response follow-up CE-CT scan. When properly executed, these manual measurements could aid in early surgical decision making.

Introduction: Magnetic Resonance Imaging is a commonly used technique for the initial diagnosis of gliomas. T1, T2, T2-FLAIR, and post-contrast T1 with gadolinium-based contrast agents (GBCAs) can show tumor characteristics. However, using this contrast agent poses a risk to patients with kidney failures, has environmental impact, and increases cost. To address these issues, we aimed to evaluate the potential of deep learning in generating synthetic post-contrast T1 images without using contrast agents.

Method: The project investigates the potential of using deep learning (DL) to generate synthetic post-contrast T1 images based on T1, T2, and T2-FLAIR provided by the Erasmus Glioma Database. Exploring different model architectures, loss functions, and input sizes to discover the optimal approach.

Results: Results show that individual loss functions, input size, and model complexity slightly impact the accuracy of synthetic post-contrast T1 images. Combining loss functions, however, was the most promising approach for image generation. Models trained with ℒm could generate low detail enhancement. Resulting in 0.0478±0.0076, 0.0139±0.0036, and 0.879±0.024 for MAE, MSE, and SSIM, respectively.

Conclusion: The study's findings indicate that DL is promising for generating synthetic post-contrast T1 images without using GBCAs. However, further research is required to generate realistic synthetic post-contrast T1 images. The study, however, provides a basis for future work and highlights the importance of reducing the use of GBCAs in clinical practice.