The applicability of deep learning to detect the progress of laparoscopic surgery using video recordings

Abstract

The operating room is one of the most complex and expensive environments in the hospital. Research has been focusing on improving the efficient use of the OR time, for instance by using intraoperative data to update the planning of the OR during the day. This thesis used a deep learning network to automatically recognize surgical tools and pre-defined surgical phases present in the recordings, to ultimately track the progress of the procedure. The aim of this thesis is to assess the performance and applicability of this deep learning method for the use of image recognition in a medical environment. To ultimately predict the remaining surgery duration and improve the efficiency of the OR planning. Two datasets of laparoscopic recordings were used, one containing laparoscopic cholecystectomies and one containing total laparoscopic hysterectomies. The surgical tools and the pre-defined phases of the procedure were annotated in every recording, after which the deep learning network was trained with this data. The performance of the network was tested in multiple experiments. The results showed that the performance of the deep learning network was promising and in line with published literature, but that the results varied between recordings. An experiment using three different sized datasets showed that a larger dataset corresponded with the best results and results that varied the least between recordings. Testing the generalizability of the network showed that a network trained on one type of surgery can also be used to recognize similar tools in a different type of surgery. Important is that the tools have the same design. It was found that the most important resources for a project like this are a dedicated hardware with image recognition software and time. This thesis showed the applicability of a deep learning network to automatically recognize the progress of a surgery and provided insight into the steps that need to be taken to use it on a larger scale.