Deep Learning Notes: Skin Lesion Classification with HAM10000 and Google Vision AutoML

So, as mentioned in the previous post, I have been playing around with the HAM10000 data set, which is dermatoscopic images of skin lesions. You can find this and more on the ISIC skin imaging collaboration, which is open source, and has those 10k images as well as many more. The HAM10000 set is 7 different diagnoses, including things like melanoma and basal cell carcinoma, and I’ve been using DenseNet-169 and ResNet-152 and NASNetMobile among others to try to classify. I also decided to give the Google Vision AutoML a whirl and here are some of the results, which I am sort of ashamed to say, are a lot better than what I have been coming up with with the aforementioned models.

First Model — trained for 4 hours (they give you the first hour free…)

Above are the results for the first model, which I trained for 4 hours.

Now what’s nice about Google is that you can simply continue training where you left off. Here’s what happened when I added a couple more hours.

The avg precision, (AUC?) goes down but the recall gets a bit better. Here’s a look at the precision-recall tradeoff curve…there’s a steep bit of the curve that suggests that, were you willing, you could get a decent improvement in precision for less than 1:1 cost in recall…if that’s what you wanted.

Lastly, here’s a look at the confusion matrix:

confusion matrix

What you can see is that nv (melanocytic nevi)— the majority class by far — is classified fairly easily, but the one that stands out as having the worst performance is the df ( dermatofibroma) class. There aren’t a lot of examples of it, relatively speaking, which is why that is the class that I want to really work on getting synthetic, DCGAN images for…of course, if this is rare enough in the population that it’s not of the highest priority than that’s another thing. But mathematically, it’s the worst performing one, along with mel itself — which often gets confused with nv. Perhaps, since melanoma is the whole point of the ISIC archive, I should work more on getting synthetic images of mel before all else? Would love to hear any thoughts or questions about this process and using Google AutoML, etc.

Thanks for reading.