Improving Hand Landmark Detection in Infrared Images for Leprosy Diagnosis Using Colorization and Image Transformations

Abstract

Hand landmark detection in infrared (IR) images is essential for early leprosy diagnosis in developing countries like Nepal, helping to prevent serious complications and disability. However, current hand landmark detection models, such as Google’s detection models comprised in the MediaPipe framework, often struggle with this task due to domain mismatch. While these models are trained on RGB images, the data for this research consists of greyscale IR images. This study addresses this challenge by exploring image transformation and colorization techniques to enhance MediaPipe's hand landmark recognition accuracy on IR images. Preprocessing was chosen over retraining the existing model due to limited computational resources and the lack of labeled target domain data, which makes the retraining infeasible.

Two preprocessing pipelines were developed to address different image characteristics: images with visible hand edges but varying colors of the hand, and images where hands blend in with the background, making the edges difficult to distinguish. The transformations include turning an image into its negative, colorization, contrast enhancement using Contrast Limited Adaptive Histogram Equalization (CLAHE), and masking to remove occlusion.

To evaluate the effectiveness of these techniques, accuracy has been calculated using Percentage of Correct Keypoints (PCK) metric and were compared against two baselines: a lower bound (MediaPipe performance on unchanged IR images) and an upper bound (MediaPipe performance on similar RGB images). Preliminary findings indicate that colorization significantly improves recognition for hands with sharp color transition, while contrast enhancement boosts edge definition for hands that blend into the background. By combining these approaches, the overall accuracy of hand landmark detection improved up to 25%, depending on the threshold value, particularly for the targeted open palm-up hand position.

These results demonstrate that preprocessing techniques can effectively reduce the input domain mismatch, enhancing automated leprosy diagnosis and supporting early detection efforts in low-resource settings.