Automated Skin Lesion Detection and Instance Segmentation Using Mask R-CNN: A Comprehensive Pipeline Analysis
K. Kiruthika
Journal of Multimedia Processing and Technologies, Vol. 17, No. 3, 2026.
DOI: https://doi.org/10.6025/jmpt/2026/17/3/124-151
Link: https://www.dline.info/jmpt/fulltext/v17n3/jmptv17n3_2.pdf
Abstract:
Background: Skin cancer remains one of the most prevalent and potentially fatal forms of cancer worldwide,
with early detection being critical for improving patient outcomes. Dermoscopic imaging has become an
indispensable tool for dermatological assessment; however, manual interpretation is challenging due to
substantial variability in lesion appearance, ambiguous boundaries, and common imaging artefacts.
Automated computer-aided diagnostic systems have emerged as promising solutions to assist clinicians in
the early detection and accurate assessment of skin lesions.
Objective: This study presents a comprehensive end to end computational framework for automated skin
lesion detection and instance segmentation using the Mask R-CNN architecture. The primary objectives are
to develop a robust system that simultaneously performs lesion localization, classification, and pixel level
segmentation while providing clinically interpretable outputs to support dermatological decision making.
Methods: The proposed framework integrates a ResNet-50 backbone with a Feature Pyramid Network
(FPN) for multi scale feature extraction, a Region Proposal Network (RPN) for candidate lesion detection,
ROI Align for precise spatial feature extraction, and parallel multi task heads for classification, boundingbox
regression, and mask prediction. The model was trained and evaluated on a combined dataset of 2,500
dermoscopic images sourced from the International Skin Imaging Collaboration (ISIC) Archive and the PHÂ²
dataset, with expert annotated ground truth including bounding boxes and binary segmentation masks.
Performance was assessed using Dice coefficient, Intersection over Union (IoU), pixel accuracy, mean Average
Precision (mAP), inference time, and training convergence. The framework was systematically compared
against the U-Net++ semantic segmentation baseline.
Results: The proposed Mask R-CNN framework achieved strong performance on the validation set, with a
Dice coefficient of 0.912, an IoU of 0.847, a pixel accuracy of 0.968, and an mAP@0.5 of 0.893. These results
represent significant improvements over the U-Net++ baseline, with relative gains of 4.3% in Dice coefficient
and 6.9% in IoU. The model demonstrated robust boundary adherence across lesions of varying sizes and
morphological characteristics while maintaining clinically acceptable inference speeds of approximately48 ms per image. Training converged efficiently over 15 epochs with a final loss of 0.227. Per class analysis
revealed strong performance for both benign (Dice: 0.928) and malignant (Dice: 0.885) lesions. Statistical
analysis using paired Wilcoxon signed rank tests confirmed the significance of performance improvements
(p = 0.043, effect size d ï‚» 1.12).
Conclusion: The proposed Mask R-CNN framework provides a robust, accurate, and clinically viable solution
for automated skin lesion detection and instance segmentation. By simultaneously delivering lesion classification,
precise boundary delineation, and instance level segmentation masks, the system offers
comprehensive morphological information that directly supports quantitative lesion assessment and
computer assisted clinical decision making. While challenges remain in handling severe imaging artefacts
and domain generalization, this work establishes Mask R-CNN as a powerful foundation for computer aided
dermatological diagnosis, with substantial potential for integration into routine clinical practice to facilitate
early melanoma detection and improve patient outcomes.