Introduction - Accurate intrahepatic anatomical localization remains challenging for laparoscopic liver resection (LLR) of patients with colorectal liver metastases (CRLMs). Image-guided surgery (IGS) systems utilizing augmented reality (AR) visualizations have been developed to assist in these procedures by projecting virtual models onto the laparoscopic view. However, AR visualization, often presented as a comprehensive model, requires improvements to achieve an effective and positive experience for surgeons. A user study was performed to develop and evaluate multiple AR and VR visualizations and identify the most promising and effective visualizations based on feedback from surgeons. Additionally, adapters with EM sensors are required to track the 30° laparoscopes and enable AR. The secondary objective was to develop and calibrate these adapters.

Methods - Multiple visualizations, categorized into three classes, were developed and assessed on a phantom and patients. The user study consisted of two phases. Phase 1 focused on the development and evaluation of these visualizations. Feedback from three surgeons was gathered using a custom-made survey that assessed experience, effectiveness, and usability. Phase 2 involved evaluating clinical usage of the visualizations and their applications by showing videos of these tests to seven surgeons from two hospitals.

Results - In Phase 1, five patients were included for AR-guided laparoscopic resection. Improvements were made compared to the gold standard whole liver model in AR experience, effectiveness, depth perception, and information balance. In Phase 2, the most promising visualizations from class 1 were the depth shader and target
structures visualization, both preferred by six surgeons. Additionally, all seven surgeons wanted the ability to augment a predefined virtual resection plan. For class 2, all seven surgeons found the US view promising and only one surgeon indicated they would use the virtual painter. Five surgeons indicated they would use the third-person and target view of class 3 during liver procedures. Furthermore, adapters with EM sensors were developed and calibrated to track the 30° laparoscopes and enable AR.

Conclusion - AR is promising in liver surgery when surgeons can easily interpret the visualization and have a positive experience. To be effective, AR systems should accurately localize and define vessels and tumor margins, provide clear depth perception, and offer well-balanced information. Several aspects of AR visualization for the entire liver model have been improved. Future research should further improve these visualization aspects, implement non-rigid registration techniques and on-demand patient-specific activation of structures. These advancements can enable AR integration into the surgical workflow, enhancing precision and effectiveness.

Liver cancer ranks seventh in global cancer prevalence and stands as the third leading cause of cancer-related deaths. Surgical resection is considered the gold standard for treatment of liver malignancies and has witnessed a shift toward minimally invasive techniques in recent decades. Surgeons rely on laparoscopic observations, preoperative imaging, and laparoscopic ultrasound for image guidance. However, these methods pose challenges in accurately representing the complex three-dimensional (3D) structure of the liver in a two-dimensional (2D) view. Surgical navigation devices aim to enhance this process by providing additional information for improved lesion localization and vascular structure visualization with respect to tracked surgical instruments. This study's objective was to establish an electromagnetically guided laparoscopic liver resection workflow at Netherlands Cancer Institute – Antoni van Leeuwenhoek (NKI-AvL) to be implemented into clinical practice.

A method was developed for tracking instruments in laparoscopic surgery, with a focus on the laparoscopic sealer/ divider. An adapter was developed with an embedded electromagnetic sensor to allow tracking. Calibration employed a dedicated device, followed by a comprehensive validation process encompassing usability, sterilizability, attachment reproducibility, and calibration accuracy. The adapter was produced from a non-ferromagnetic material, ensuring sterilizability and straightforward surgeon attachment, exhibiting positive results in reproducibility and virtual visualization. Nevertheless, the calibration accuracy, at 2.5 mm, could be enhanced, primarily through calibration method adjustments. Additionally, a fast calibration method for the laparoscopic ultrasound (LUS) probe adapter was developed, involving a dedicated calibration tool. Validation assessments to ascertain its reproducibility showed positive results. However, accuracy measurements can be further improved as ultrasound calibration method evaluation showed a root mean squared error (RMSE) of 3.6 mm. This is important as the navigation performance relies on the calibration accuracy of the tracked LUS probe. The workflow for navigated laparoscopic liver resection was tested using a liver phantom. It demonstrated the navigation system's capacity to register preoperative and intraoperative images with a mean target registration error (TRE) of 3.7 mm and showed promising augmented reality views. Additionally, this test led to workflow enhancements related to liver sensor placement and visualization techniques.

The developed navigation workflow for laparoscopic liver resection integrated Aurora EM tracking, tracked laparoscopic instruments, and software for intraoperative liver 3D model visualization. While feasible in an operating setting during phantom tests, further enhancements in visualization and accuracy are needed. A clinical study will be conducted to determine intraoperative feasibility, providing valuable experience to technical physicians and surgeons. In the near future, the presented navigation method may enhance laparoscopic resection accuracy and reduce disease recurrence. A clinical study at NKI-AvL will soon validate this navigation system for clinical use.