mnist_digit_classifier_to_coreml.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# From a MNIST handwritten digit (SVM) classifier to Apple's Core ML format\n",
    "\n",
    "This tutorial will NOT go in depth of machine learning topics, it is a short tutorial on how to convert scikit-learn SVM model Apple's coreml format. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3.6.6\n",
      "/Users/erkandiken/Development/deep-learning/mnist_classifier/mnist-scikit-svm-to-coreml/bin/python\n"
     ]
    }
   ],
   "source": [
    "# import mnist_loader_py2 as mnist_loader  # enable this if python 2 is used\n",
    "import mnist_loader_py3 as mnist_loader\n",
    "from sklearn import svm\n",
    "import platform\n",
    "print(platform.python_version())\n",
    "import sys\n",
    "print(sys.executable)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1.  Loading the MNIST image data\n",
    "\n",
    "First, we will load the MNIST image data. Here, we do this with the help of an external library (mnist_loader).\n",
    "\n",
    "You can find the original python file in the following github repo: https://github.com/mnielsen/neural-networks-and-deep-learning/tree/master/src"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "training_data, validation_data, test_data = mnist_loader.load_data()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will not be using the validation data, however, we kept the standard way of data partitining.\n",
    "\n",
    "Training data is returned as a tuple of two entires. The first entry contains the actual training images in a vectorized form (two dimensional 28 pixels by 28 pixels image is flattend to 1 x 784 sized vector). In total, we have 50K traning images, hence, 50K x 784 sized training data. The second entry contains the true label (in this case a digit between 0 and 9) of each training image. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'tuple'>\n",
      "First entry:  (50000, 784)\n",
      "Second entry:  (50000,)\n",
      "Indeed, each entry of training data is numpy ndarray:\n",
      "<class 'numpy.ndarray'>\n"
     ]
    }
   ],
   "source": [
    "print (type(training_data))\n",
    "print (\"First entry: \", training_data[0].shape)\n",
    "print (\"Second entry: \", training_data[1].shape)\n",
    "print (\"Indeed, each entry of training data is numpy ndarray:\")\n",
    "print (type(training_data[0]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "true label of the first image is :  5\n"
     ]
    }
   ],
   "source": [
    "print (\"true label of the first image is : \", training_data[1][0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Test data has the same structure, however, only with 10K images."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'tuple'>\n",
      "First entry:  (10000, 784)\n",
      "Second entry:  (10000,)\n",
      "<class 'numpy.ndarray'>\n"
     ]
    }
   ],
   "source": [
    "print (type(test_data))\n",
    "print (\"First entry: \", test_data[0].shape)\n",
    "print (\"Second entry: \", test_data[1].shape)\n",
    "print (type(test_data[0]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2. Train the model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before going to model training, I will rename the training and test data components."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train = training_data[0]\n",
    "y_train = training_data[1]\n",
    "X_test = test_data[0]\n",
    "y_test = test_data[1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We use the Support Vector Machines (SVM) from scikit-learn library to build our ML model. I will use only first 1000 data images for fitting in order to quickly check if the model is built successfully."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "SVC(C=1.0, cache_size=200, class_weight=None, coef0=0.0,\n",
       "  decision_function_shape='ovr', degree=3, gamma='auto', kernel='rbf',\n",
       "  max_iter=-1, probability=False, random_state=None, shrinking=True,\n",
       "  tol=0.001, verbose=False)"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train_sub = X_train[:1000]\n",
    "y_train_sub = y_train[:1000]\n",
    "svm_model = svm.SVC(gamma='auto')\n",
    "svm_model.fit(X_train_sub, y_train_sub)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "SVM model is created. We can immediately check what will be the prediction of the model for the first digit in the test data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "predicted:  7 true value:  7\n"
     ]
    }
   ],
   "source": [
    "predicted_test_0 = svm_model.predict(X_test[0].reshape(1, -1))\n",
    "print(\"predicted: \", int(predicted_test_0), \"true value: \", y_test[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You should see 7 for the predicted and true label of the digit. Even if we used 1K data out of 50K data for training, our model was good enough to predict the first digit.\n",
    "\n",
    "We can check the performance of the model on whole test data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SVM classifier (trained with subset of training data): \n",
      "8267 of 10000 values correct.\n"
     ]
    }
   ],
   "source": [
    "# testing\n",
    "predictions = [int(a) for a in svm_model.predict(X_test)]\n",
    "num_correct = sum(int(a == y) for a, y in zip(predictions, y_test))\n",
    "print(\"SVM classifier (trained with subset of training data): \")\n",
    "print(\"%s of %s values correct.\" % (num_correct, len(y_test)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You should see the performance around 82 %. That is not bad, however, we can do better if we use all training data for building the model. This can take a while to finish."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "SVC(C=1.0, cache_size=200, class_weight=None, coef0=0.0,\n",
       "  decision_function_shape='ovr', degree=3, gamma='auto', kernel='rbf',\n",
       "  max_iter=-1, probability=False, random_state=None, shrinking=True,\n",
       "  tol=0.001, verbose=False)"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "svm_model.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Performance on the test data should be increased to around 94 %. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SVM classifier (trained with subset of training data): \n",
      "9435 of 10000 values correct.\n"
     ]
    }
   ],
   "source": [
    "# testing\n",
    "predictions = [int(a) for a in svm_model.predict(X_test)]\n",
    "num_correct = sum(int(a == y) for a, y in zip(predictions, y_test))\n",
    "print(\"SVM classifier (trained with subset of training data): \")\n",
    "print(\"%s of %s values correct.\" % (num_correct, len(y_test)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3. Converting SVM model to Core ML format"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [],
   "source": [
    "import coremltools"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [],
   "source": [
    "# we will use coremltools's converter, first we can check signature of the convert function: \n",
    "coremltools.converters.sklearn.convert?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [],
   "source": [
    "coreml_svm = coremltools.converters.sklearn.convert(svm_model, 'hand_digit', 'predicted_digit')\n",
    "# save the model\n",
    "coreml_svm.save('digit_recognizer.mlmodel')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The coreml model should be visible in the directory where you have this notebook. Optionally, you can also add more information/metadata to your model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [],
   "source": [
    "# set model metadata\n",
    "coreml_svm.author = 'Erkan Diken'\n",
    "coreml_svm.license = 'BSD'\n",
    "coreml_svm.short_description = 'Scikit-learn SVM classifier based handwritten digit recognizer '\n",
    "\n",
    "# set feature descriptions manually\n",
    "coreml_svm.input_description['hand_digit'] = 'handwritten digit'\n",
    "\n",
    "# set the output descriptions\n",
    "coreml_svm.output_description['predicted_digit'] = 'predicted digit'\n",
    "\n",
    "# save the model\n",
    "coreml_svm.save('digit_recognizer.mlmodel')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 4. Load the Core ML model to make predictions \n",
    "Note: If you do not have MacOS version of 10.13. The predict method below will throw the following exception: Model prediction is only supported on macOS version 10.13.\n",
    "\n",
    "Tested on MacOS High Sierra version 10.13.6"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [],
   "source": [
    "# load the model \n",
    "model = coremltools.models.MLModel('digit_recognizer.mlmodel')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can check the the specs of your model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<bound method MLModel.get_spec of input {\n",
       "  name: \"hand_digit\"\n",
       "  shortDescription: \"handwritten digit\"\n",
       "  type {\n",
       "    multiArrayType {\n",
       "      shape: 784\n",
       "      dataType: DOUBLE\n",
       "    }\n",
       "  }\n",
       "}\n",
       "output {\n",
       "  name: \"predicted_digit\"\n",
       "  shortDescription: \"predicted digit\"\n",
       "  type {\n",
       "    int64Type {\n",
       "    }\n",
       "  }\n",
       "}\n",
       "predictedFeatureName: \"predicted_digit\"\n",
       "metadata {\n",
       "  shortDescription: \"Scikit-learn SVM classifier based handwritten digit recognizer \"\n",
       "  author: \"Erkan Diken\"\n",
       "  license: \"BSD\"\n",
       "}\n",
       ">"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.get_spec"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Documentation of the predict function points out that predict function accept inputs as dictionary object and key of the dictionary is the string that we have provided above. The same holds for the return value of the function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.predict?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [],
   "source": [
    "digit_to_predict = dict()\n",
    "digit_to_predict['hand_digit'] = X_test[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [],
   "source": [
    "prediction = model.predict(digit_to_predict)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "7\n"
     ]
    }
   ],
   "source": [
    "print (int(prediction['predicted_digit']))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "The prediction output is the same with output of the scikit-learn SVM model for the first digit in test data. We can check the performance of coreml model on the whole test data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "10000\n"
     ]
    }
   ],
   "source": [
    "print(X_test.shape[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "10000\n"
     ]
    }
   ],
   "source": [
    "predictions = list()\n",
    "for x in range(X_test.shape[0]):\n",
    "    digit_to_predict['hand_digit'] = X_test[x]\n",
    "    predictions.append(int(model.predict(digit_to_predict)['predicted_digit']))\n",
    "print(len(predictions))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Core ml classifier: \n",
      "9435 of 10000 values correct.\n"
     ]
    }
   ],
   "source": [
    "num_correct = sum(int(a == y) for a, y in zip(predictions, y_test))\n",
    "print(\"Core ml classifier: \")\n",
    "print(\"%s of %s values correct.\" % (num_correct, len(y_test)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "9435 of 10000 values correct. This is exactly the same as the SVM classifer before converted to coreml format."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "mnist-scikit-svm-to-coreml",
   "language": "python",
   "name": "mnist-scikit-svm-to-coreml"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}