During the preoperative planning for breast-conserving surgery, the surgeon makes use of an MRI scan of the breast cancer patient in the prone position to accurately locate the tumour. However, surgery is performed with the patient in the supine position. The surgeon needs to mentally translate the location of the tumour from the prone position towards the supine position. The usage of mixed reality systems in the form of the Microsoft HoloLens could visually aid the surgeon by projecting the tumour in the supine position onto the patient. This requires us to obtain the tumour in the supine position by dealing with the prone-to-supine breast image registration problem.
We propose to maximize the overlap between the prone and supine surfaces of the breast through the use of soft constraints to guide the deformation. An initial subspace FEM simulation pulls the prone and supine surfaces towards each other at specified landmark correspondences between the surfaces. The resulting prone surface is projected onto the supine surface and these surface vertices are restricted to tangential movement across the surface of the breast. The tissues at the interior of the breast are deformed accordingly. Our experiments with synthesized data indicate the robustness of our method under various scenarios, except for the rigid misalignment between the prone and supine meshes. This problem of misalignment also affects the experiments with real-world data, which results in distances greater than 12mmbetween the centroids of the resulting deformed tumour and the ground-truth tumour.

Surgical navigation is a tool that surgeons rely on everyday to perform accurate surgeries all over the world. However, this technology requires good hand-eye coordination and a high level of concentration. HoloNav is a project that inquires to see if using the HoloLens and augmented reality can replace the current surgical navigation methods. To do so, the HoloLens must be able to identify the patient and the location of the surgery instruments, which uses optical reflective spheres. This study focuses on using the grayscale cameras of the HoloLens and a deep learning algorithm YOLOv5 to test if it is possible to precisely detect optical reflective spheres. 3 models were trained with two different data sets, where the results show that the model trained on a data set would perform well on the validation set. However, they would perform far worse when exposed to a data set it was not trained on.

Patient and instrument tracking are fundamental parts of surgical navigation systems. Traditional surgical navigation systems rely on stationary cameras for tracking and stationary screens for presenting information. An increased mental load is exerted by surgeons as they switch focus between the surgical site and the presented data. By projecting the required data on the patients anatomy, augmented reality surgical navigation would allow surgeons to navigate seamlessly during surgical procedures. In this paper, an augmented reality tracking algorithm for the Hololens 2 will be presented and tested against state of the art optical tracking. The algorithm is based on the use of reflective markers, which are extensively used in medical applications. An accuracy of 1.04mm was found in optimal conditions using the presented method. The presented tracking method is less accurate and less consistent than state of the art optical tracking methods with a minimum median accuracy of 25mm.

The way that surgeons currently use surgical navigation technology impacts their hand-eye coordination and their ability to view images and data critically. To tackle this issue Augmented Reality goggles, built from the HoloLens 2, have been developed specifically for the purpose of aiding surgeons during surgery and it is called: The HoloNav. Optical reflective spheres are commonly used to track surgical instruments. This research aims to find out if grayscale cameras could contribute to locating these spheres more accurately with the HoloNav. This is done by finding the spheres on the infrared images, reducing the search space on the grayscale images and finding the spheres on the reduced search spaces of the grayscale images. By triangulating those retrieved coordinates and comparing them with the optically tracked coordinates a conclusion can be drawn about the accuracy. The accuracy measured in this research is between 0.66 mm and 10.64 mm (mostly between 1.8 mm and 6.4 mm) depending on the frame. This is quite accurate and similar to the results from related work. With better image quality or different input conditions the accuracy could be improved even more.