Saliency Detection in Medical Images

non-tumor, p = 0.03	non-tumor map	tumor, p = 0.99	tumor map

Figure 1: Histopathologic samples of (a) normal tissue and (c) cancerous tissue correctly classified with a convolutional neural network and the same samples overlayed GradCAM activation map (AM) for (b) normal tissue and (d) cancerous tissue.

Overview

The following is a brief summary of the project.

• Histopathologic Cancer Detection: provides a cancer detection system to detect metastatic breast cancer from small patches of pathology whole slide images (WSI) from the Patch Camelyon dataset. The solution applies transfer learning to a ResNet50 model updated with a binary classification head. Gradient-weighted Class Activtion Mapping (GradCAM) is used to identify the regions of the images that have a high probability of containing malignant cells.
• Gradient-weighted Class Activation Mapping: makes Convolutional Neural Network (CNN)-based models more transparent by visualizing the regions of input that are "salient" for predictions from these models. It uses the class-specific gradient information passed into the final convolutional layer of a CNN to produce a localization map of the important regions in the image.

Histopathologic Cancer Detection

Lymph nodes are typically the first place breast cancer spreads and is therefore detection of lymph node metastases is an important part of identifying the extent of the cancer. Manual inspection of histopathologic slides is a laborious process and therefore prone to human error and this makes it a good candidate for automation through machinbe learning.

We build a binary image classifier to identify the presence of metastases in 96 x 96 pixel patches from digital histopathology images where the metastases can exist in a small number of cells. The histopathological images are microscopic images of lymph nodes that have been stained with hematoxylin and eosin (H&E) which produces blue, red and violet colors. Hematoxylin binds to nucleic acids producing blue coloration and eosin binds to amino-acid chains producing pink coloration. This differentiation means that cell nuclei are typically stained blue but cytoplasm and extracellular material is stained pink. We use the Patch Camelyon (PCam) dataset which contains 220K training patches obtained from 400 H&E stained whole slide images collected by Radboud University Medical Center (Nijmegen, Netherlands) and University Medical Center Utrecht (Utrecht, Netherlands).

Gradient-weighted Class Activation Mapping (Grad-CAM)

Grad-CAM is a technique for producing "visual explanations" for decisions from CNN-based models, making them more transparent it uses the gradients flowing into the final convolutional layer to produce a localization map highlighting important regions in the image for predicting the class. It can be applied as a computational saliency technique in medical imaging for of abnormality detection and computer-aided diagnosis.

Preprocessing and image augmentation

augmentation #1	augmentation #2	augmentation #3	augmentation #4

Figure 2: Augmentations of the same original image as output by the ImageDataGenerator in Keras.

The PCam dataset is not large and we therefore have to find ways to avoid overfitting and these may include data augmentaion, regularization and employing less complex architectures. The following is an outline of the image augmentions that were applied at train time using the ImageDataGenerator supplied by Keras.

• Random Roation: 0 to 90 degrees
• Random Zoom: 0% to 20%
• Random Horizontal Flip
• Random Vertical Flip
• Random Translation: 0% to 10%

Model selection

Image augmentation helps to combat overfitting but we can help further by choosing a smaller model architecture and use transfer learning to update pre-trained model weights to our dataset. Models that are small and lend themselves to our task include DenseNet169, MobileNetV2 and ResNet50. We will also build a custom model that we will call CancerNet.

Model Training

The training curves shown in Figure 3 demonstrate that by using augmentation and choosing small networks we have succefully compated overfitting while still achieving an accuracy close to 95% for all models.

Figure 3: Training curves for the four selected models

Model Evaluation

The Receiver Operating Chracteristic (ROC) curve in Figure 4 which shows the True Positive Rate against the False Positive Rate as the discrimination threshold is varied. This demonstrates that a high diagnostic ability of all classifiers. The MobileNetV2 model is able to achieve 97% accuracy while maintaining a high sensitivity.

Figure 4: Receiver Operating Chracteristic (ROC) curve of the four chosen models