How Leaders in Medical AI Reached Up to 85% Accuracy with a Distilled Lightweight Model
The SSD-KD paper proposes a self-supervised diverse knowledge distillation method for lightweight skin lesion classification using dermoscopic images.
On ISIC 2019, a large-scale open-accessed benchmark of dermoscopic images, the distilled MobileNetV2 reached an accuracy as high as 85% for the classification tasks of 8 different skin diseases, with minimal parameters and computing requirements. Compared with state-of-the-art knowledge distillation techniques, the method demonstrates improved performance. To the authors’ knowledge, it is the first deep knowledge distillation application for multi-disease classification on the large-scale dermoscopy database.
- Up to 85%
- accuracy for the classification tasks of 8 different skin diseases
- MobileNetV2
- the distilled student model, with minimal parameters and computing requirements
- ResNet-50
- teacher model, per the authors’ GitHub repository
- 1st
- deep knowledge distillation application for multi-disease classification on the large-scale dermoscopy database, to the authors’ knowledge
The problem
The authors write that most studies in skin cancer detection keep pursuing high prediction accuracy without considering the limitation of computing resources on portable devices. Knowledge distillation has been proven an efficient tool to improve the adaptability of lightweight models under limited resources, while keeping a high-level representation capability.
What they did
A dual relational knowledge distillation architecture is self-supervised trained, and weighted softened outputs let the student model capture richer knowledge from the teacher model. The authors tested it on ISIC 2019, a large-scale open-accessed benchmark of dermoscopic images.
The result
Compared with state-of-the-art knowledge distillation techniques, the method demonstrates improved performance. Ablation studies confirm the effectiveness of the intra- and inter-instance relational knowledge integration strategy.
Figures come from the source below. Anything from another place is attributed in the text.
Source: Medical Image Analysis (2023): SSD-KD, PubMed 36462373 ↗